# composer.datasets.utils#

Utility and helper functions for datasets.

Functions

 add_vision_dataset_transform Add a transform to a dataset's collection of transforms. pil_image_collate Constructs a length 2 tuple of torch.Tensors from datasets that yield samples of type PIL.Image.Image.

Classes

 NormalizationFn Normalizes input data and removes the background class from target data if desired.
class composer.datasets.utils.NormalizationFn(mean, std, ignore_background=False)[source]#

Normalizes input data and removes the background class from target data if desired.

An instance of this class can be used as the device_transforms argument when constructing a DataSpec. When used here, the data will normalized after it has been loaded onto the device (i.e., GPU).

Parameters
• mean (Tuple[float, float, float]) – The mean pixel value for each channel (RGB) for the dataset.

• std (Tuple[float, float, float]) – The standard deviation pixel value for each channel (RGB) for the dataset.

• ignore_background (bool) – If True, ignore the background class in the training loss. Only used in semantic segmentation. Default: False.

Add a transform to a dataset’s collection of transforms.

Parameters
• dataset (VisionDataset) – A torchvision dataset.

• transform (Callable) – Function to be added to the dataset’s collection of transforms.

• is_tensor_transform (bool) –

Whether transform acts on data of the type Tensor. default: False.

Returns

None – The dataset is modified in-place.

composer.datasets.utils.pil_image_collate(batch, memory_format=torch.contiguous_format)[source]#

Constructs a length 2 tuple of torch.Tensors from datasets that yield samples of type PIL.Image.Image.

This function can be used as the collate_fn argument of a torch.utils.data.DataLoader.

Parameters
• batch (List[Tuple[Image.Image, Union[Image.Image, np.ndarray]]]) – List of (image, target) tuples that will be aggregated and converted into a single (Tensor, Tensor) tuple.

• memory_format (memory_format) – The memory format for the input and target tensors.

Returns

(torch.Tensor, torch.Tensor) – Tuple of (image tensor, target tensor) The image tensor will be four-dimensional (NCHW or NHWC, depending on the memory_format).