ObjectStore

class composer.utils.ObjectStore

Abstract class for implementing object stores, such as LibcloudObjectStore and S3ObjectStore.

close()

Close the object store.

download_object(object_name, filename, overwrite=False, callback=None)

Download an object to the specified destination path.

Parameters

• object_name (str) – The name of the object to download.

• filename (str | Path) – Full path to a file or a directory where the incoming file will be saved.

• overwrite (bool, optional) – Whether to overwrite an existing file at filename, if it exists. (default: False)

• callback ((int) -> None, optional) – If specified, the callback is periodically called with the number of bytes already downloaded and the total size of the object.

Raises

• FileExistsError – If filename already exists and overwrite is False.

• FileNotFoundError – If the file was not found in the object store.

• ObjectStoreTransientError – If there was a transient connection issue with downloading the object.

get_object_size(object_name)

Get the size of an object, in bytes.

Parameters

object_name (str) – The name of the object.

Returns

int – The object size, in bytes.

Raises

• FileNotFoundError – If the file was not found in the object store.

• ObjectStoreTransientError – If there was a transient connection issue with getting the object size.

get_uri(object_name)

Returns the URI for object_name.

Note

This function does not check that object_name is in the object store. It computes the URI statically.

Parameters

object_name (str) – The object name.

Returns

str – The URI for object_name in the object store.

list_objects(prefix)

List all objects in the object store with the given prefix.

Parameters

prefix (str) – The prefix to search for.

Returns

list[str] – A list of object names that match the prefix.

upload_object(object_name, filename, callback=None, **kwargs)

Upload an object currently located on a disk.

Parameters

• object_name (str) – Object name (where object will be stored in the container)

• filename (str | Path) – Path to the object on disk

• callback ((int, int) -> None, optional) – If specified, the callback is periodically called with the number of bytes uploaded and the total size of the object being uploaded.

• **kwargs – other arguments to the upload object function are supported and will be passed in to the underlying object store upload call. Currently only used for S3ObjectStore.

Raises

ObjectStoreTransientError – If there was a transient connection issue with uploading the object.