get_file#

composer.utils.get_file(path, destination, object_store=None, overwrite=False, progress_bar=True)[source]#

Get a file from a local folder, URL, or object store.

Parameters

  • path (str) โ€“

    The path to the file to retrieve.

    • If object_store is specified, then the path should be the object name for the file to get. Do not include the the cloud provider or bucket name.

    • If object_store is not specified but the path begins with http:// or https://, the object at this URL will be downloaded.

    • If object_store is not specified, but the path begins with s3://, or another backend supported by composer.utils.maybe_create_object_store_from_uri() an appropriate object store will be created and used.

    • Otherwise, path is presumed to be a local filepath.

  • destination (str) โ€“

    The destination filepath.

    If path is a local filepath, then a symlink to path at destination will be created. Otherwise, path will be downloaded to a file at destination.

  • object_store (ObjectStore, optional) โ€“

    An ObjectStore, if path is located inside an object store (i.e. AWS S3 or Google Cloud Storage). (default: None)

    This ObjectStore instance will be used to retrieve the file. The path parameter should be set to the object name within the object store.

    Set this parameter to None (the default) if path is a URL or a local file.

  • overwrite (bool) โ€“ Whether to overwrite an existing file at destination. (default: False)

  • progress_bar (bool, optional) โ€“ Whether to show a progress bar. Ignored if path is a local file. (default: True)

Raises

FileNotFoundError โ€“ If the path does not exist.