The git repo integration clones a git repo into the working directory of your run’s execution environment comes with a number of configurable options (see below) for the cloning and setup of the repo.
In order to include the git repo integration, use
integration_type: git_repo. Note that you can have any number of these
included in your YAML.
Prerequisite: Git SSH Secret Setup#
In order to clone from private repositories you will have to set up an SSH key that gives the MosiacML platform clone access to the repo.
Follow the steps for creating a
git-ssh secret on the SSH Secrets Page to set up a git SSH secret.
The only required parameter in the Git Repos integration is the
git_repo field, which corresponds to the
repo name in
integrations: - integration_type: git_repo git_repo: mosaicml/composer
Optional parameters of the Git Repos Integration configure how the repo is cloned and installed.
Optional parameters include:
git_branch (str): Clone the repo with a specific branch checked out.
Default: the repo default branch.
git_commit (str): Commit your changes in git.
path (str): Clone the repo to a specific path inside of the image. ( Note: by default git clones to the repo name within the image’s working directory, e.g.
mosaicml/composer). Specifying this value is equivalent to runing
git clone <repo url> <path>.
Default: the repo name.
ssh_clone (bool): Use SSH keys to clone the git repo. To use HTTPS,
host (str): The hostname for the git repo.
Git SSH Secret
Note that to properly clone private repos with SSH you will need an SSH Secret set.
pip_install (str): Pip install the cloned repo. The value of this field is used in
pip install <value>.
Default: no pip install.
integrations: - integration_type: git_repo # github.com/mosaicml/composer git_repo: mosaicml/composer # The git branch to checkout (optional, default = the repo default branch) git_branch: my-branch # Clone to /workspace/composer (optional, default = the repo name) path: /workspace/my_composer_clone # Use SSH Keys to clone (optional, default = True) ssh_clone: True # pip install command for the repo (optional, default = None) pip_install: -e .[all] # host for the git repo (defaults to github.com) host: github.com
The above settings are equivalent to:
> git clone [email protected]:mosaicml/composer.git -b my-branch /workspace/my_composer_clone > cd my_composer_clone > pip install -e .[all] > cd ..
Example: Multi-Repo Install#
One common use case for the Git repo integration is to clone multiple repos in the same environment.
To do this, it is as easy as just adding multiple git integrations.
integrations: - integration_type: git_repo git_repo: mosaicml/composer path: /workspace/composer branch: v0.8.1 pip_install: -e .[all] - integration_type: git_repo git_repo: facebookresearch/xformers path: /workspace/xformers pip_install: -e . - integration_type: git_repo git_repo: myuser/privaterepo path: /workspace/myrepo
In the above example, we are cloning and adding three different repos with different instructions.
Because we set the filepaths specifically with the Git Integration
path option, the resulting filestructure looks like:
workspace ├── composer ├── myrepo └── xformers
workspace is the working directory of the container being used for the run.
Checking Out Branches
Note that the mosaicml/composer repo is checked out of
v0.8.1 and installed from the
v0.8.1 branch with the pip installation options
pip install -e .[all]