class composer.utils.LibcloudObjectStore(provider, container, chunk_size=1048576, key_environ=None, secret_environ=None, provider_kwargs=None)[source]#

Utility for uploading to and downloading from object (blob) stores, such as Amazon S3.


Hereโ€™s an example for an Amazon S3 bucket named MY_CONTAINER:

>>> from composer.utils import LibcloudObjectStore
>>> object_store = LibcloudObjectStore(
...     provider="s3",
...     container="MY_CONTAINER",
...     provider_kwargs={
...         "key": "AKIA...",
...         "secret": "*********",
...     }
... )
>>> object_store
<composer.utils.object_store.libcloud_object_store.LibcloudObjectStore object at ...>
  • provider (str) โ€“

    Cloud provider to use. Valid options are:

  • container (str) โ€“ The name of the container (i.e. bucket) to use.

  • provider_kwargs (Dict[str, Any], optional) โ€“

    Keyword arguments to pass into the constructor for the specified provider. These arguments would usually include the cloud region and credentials.

    Common keys are:

    • key (str): API key or username to be used (required).

    • secret (str): Secret password to be used (required).

    • secure (bool): Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.

    • host (str): Override hostname used for connections.

    • port (int): Override port used for connections.

    • api_version (str): Optional API version. Only used by drivers which support multiple API versions.

    • region (str): Optional driver region. Only used by drivers which support multiple regions.

  • key_environ (str, optional) โ€“ Environment variable name for the API Key. Only used if โ€˜keyโ€™ is not in provider_kwargs. Default: None.

  • secret_environ (str, optional) โ€“ Envrionment varaible for the Secret password. Only used if โ€˜secretโ€™ is not in provider_kwargs. Default: None.