LoggerDestination#

class composer.loggers.LoggerDestination(*args, **kwargs)[source]#

Base class for logger destination.

As this class extends Callback, logger destinations can run on any training loop Event. For example, it may be helpful to run on Event.EPOCH_END to perform any flushing at the end of every epoch.

Example

>>> from composer.loggers import LoggerDestination
>>> from composer.trainer import Trainer
>>> class MyLogger(LoggerDestination):
...     def log_hyperparameters(self, data):
...         print(f'Batch {int(state.timestamp.batch)}: {data}')
>>> logger = MyLogger()
>>> trainer = Trainer(
...     ...,
...     loggers=[logger]
... )
Batch 0: {'composer_version': ...}
Batch 0: {'composer_commit_hash': ...}
Batch 0: {'num_nodes': ...}
Batch 0: {'rank_zero_seed': ...}

can_upload_files()[source]#

Indicates whether LoggerDestination can upload files.

Defaults to false, should return True for derived logger classes that implement upload_file().

Returns: bool – Whether the class supports uploading files.

download_file(remote_file_name, destination, overwrite=False, progress_bar=True)[source]#

Handle downloading a file named remote_file_name to destination.

Parameters

remote_file_name (str) – The name of the file.
destination (str) – The destination filepath.
overwrite (bool) – Whether to overwrite an existing file at destination. Defaults to False.
progress_bar (bool, optional) – Whether to show a progress bar. Ignored if path is a local file. (default: True)

log_hyperparameters(hyperparameters)[source]#

Log hyperparameters, configurations, and settings.

Logs any parameter/configuration/setting that doesn’t vary during the run.

Parameters: hyperparameters (Dict[str, Any]) – A dictionary mapping hyperparameter names (strings) to their values (Any).

log_images(images, name='Images', channels_last=False, step=None, masks=None, mask_class_labels=None, use_table=True)[source]#

Log images. Logs any tensors or arrays as images.

Parameters

images (np.ndarray | Tensor | Sequence[np.ndarray | Tensor]) – Dictionary mapping image(s)’ names (str) to an image of array of images.
name (str) – The name of the image(s). (Default: 'Images')
channels_last (bool) – Whether the channel dimension is first or last. (Default: False)
step (Optional[int], optional) – The current step or batch of training at the time of logging. Defaults to None. If not specified the specific LoggerDestination implementation will choose a step (usually a running counter).
masks (Dict[str, np.ndarray | Tensor | Sequence[np.ndarray | Tensor]], optional) – A dictionary mapping the mask name (e.g. predictions or ground truth) to a sequence of masks.
mask_class_labels (Dict[int, str], optional) – Dictionary mapping label id to its name. Used for labelling each color in the mask.
use_table (bool) – Whether to make a table of the images or not. (default: True). Only for use with WandB.

log_metrics(metrics, step=None)[source]#

Log metrics or parameters that vary during training.

Parameters

metrics (Dict[str, float]) – Dictionary mapping metric name (str) to metric scalar value (float)
step (Optional[int], optional) – The current step or batch of training at the time of logging. Defaults to None. If not specified the specific LoggerDestination implementation will choose a step (usually a running counter).

log_table(columns, rows, name='Table', step=None)[source]#

Log a table.

Parameters

columns (List[str]) – Names of the columns in the table.
rows (List[List[Any]]) – 2D row-oriented array of values.
name (str) – Name of table. (Default: 'Table')
step (Optional[int], optional) – The current step or batch of training at the time of logging. Defaults to None. If not specified the specific LoggerDestination implementation will choose a step (usually a running counter).

log_traces(traces)[source]#

Log traces. Logs any debug-related data like algorithm traces.

Parameters: traces (Dict[str, float]) – Dictionary mapping trace names (str) to trace (Any).

upload_file(state, remote_file_name, file_path, *, overwrite)[source]#

Handle uploading a file stored at file_path to a file named remote_file_name.

Subclasses should implement this method to store logged files (e.g. copy it to another folder or upload it to an object store). However, not all loggers need to implement this method. For example, the TQDMLogger does not implement this method, as it cannot handle file uploads.

Note

This method will block the training loop. For optimal performance, it is recommended that this method copy the file to a temporary directory, enqueue the copied file for processing, and return. Then, use a background thread(s) or process(s) to read from this queue to perform any I/O.
After this method returns, training can resume, and the contents of file_path may change (or be may deleted). Thus, if processing the file in the background (as is recommended), it is necessary to first copy the file to a temporary directory. Otherwise, the original file may no longer exist, or the logged file can be corrupted (e.g., if the logger destination is reading from file while the training loop is writing to it).