Databricks

To set up your Databricks credentials within the MCloud environment, use the databricks secret:

mcli create secret databricks

This secret requires providing your workspace URL and Databricks Personal Access Token (PAT) to the MosaicML platform as host and token, respectively.

host: Your Databricks workspace URL—more information on workspace URLs can be found in this documentation.

token: Generate a Databricks PAT following the instructions here.

Instead of using the interactive prompt above, all the settings can also be provided as flags in one command:

mcli create secret databricks --host my-host --token my-token

Testing your Credentials

To test your Databricks credentials, create a databricks_secret_check.yaml with the following run config:

name: databricks-secret-check
image: homebrew/brew
compute:
  gpus: 0
command: |
  brew tap databricks/tap
  brew install databricks
  echo "[DEFAULT]
  host  = $DATABRICKS_HOST
  token = $DATABRICKS_TOKEN" >> ~/.databrickscfg
  printf "\n\n\n\n\n========================================\nDatabricks current user info:\n"
  databricks current-user me

mcli run -f databricks_secret_check.yaml --follow

If all goes well, you should see something like this at the bottom of the output:

========================================
Databricks current user info:
{
  "active":true,
  "displayName":string,
  "emails": [
    {
      "primary":true,
      "type":string,
      "value":string
    }
  ],
  "groups": [
    {
      "display":string,
      "$ref":string,
      "type":string,
      "value":string
    }
  ],
  "id":string,
  "name": {
    "givenName":string
  },
  "userName":string
}