MLflow

The MLflow integration automatically sets the relevant environment variables that MLflow relies on in the run execution environment.

MLflow Logger must be configured in Composer

This integration only sets up the environment, the logger itself must still be configured in Composer. See Composer’s Logging and MLFlowLogger.

Using MLflow in a run (managed through Databricks)

This section will explain how to use MLflow for a run. To use this integration, set up your credentials with the databricks secret. To use MLflow experiment tracking in a run, include the MLflow integration in your run config:

YAML

integrations:
  - integration_type: mlflow
    experiment_name: /Users/[email protected]/my_experiment

PYTHON

from mcli import RunConfig
config = RunConfig(
    ...
    integrations=[
        {
         'integration_type': 'mlflow',
         'experiment_name': '/Users/[email protected]/my_experiment',
        }
    ],
)

• experiment_name (str, required): The name to use for the experiment. Databricks MLflow users see this page for more information about experiment and workspace organization (example Databricks experiment name: /Users/<email>/<experiment_name>).

• tracking_uri (str, optional): Default is databricks , e.g. managed MLflow through Databricks

Using MLflow in a run (unmanaged)

You do not have to have a Databricks managed MLflow account to use MLflow, however your run will be responsible for uploading the MLflow artifacts before the run terminates. See the MLflow documentation for options

YAML

integrations:
  - integration_type: mlflow
    experiment_name: my_experiment
    # See MLflow docs for all tracking_uri alternatives
    tracking_uri: file:/my/local/dir

PYTHON

from mcli import RunConfig
config = RunConfig(
    ...
    integrations=[
        {
         'integration_type': 'mlflow',
         'experiment_name': 'my_experiment',
         'tracking_uri': 'file:/my/local/dir',  # See MLflow docs for all alternatives
        }
    ],
)