composer.functional

Functional API for applying algorithms in your own training loop.

from composer import functional as cf
from torchvision import models

model = models.resnet50()

# replace some layers with blurpool
cf.apply_blurpool(model)
# replace some layers with squeeze-excite
cf.apply_squeeze_excite(model, latent_channels=64, min_channels=128)

Functions

apply_alibi	Removes position embeddings and replaces the attention function and attention mask as per Alibi.
apply_blurpool	Add anti-aliasing filters to strided torch.nn.Conv2d and/or torch.nn.MaxPool2d modules.
apply_channels_last	Changes the memory format of the model to torch.channels_last.
apply_factorization	Replaces torch.nn.Linear and torch.nn.Conv2d modules with FactorizedLinear and FactorizedConv2d modules.
apply_fused_layernorm	Replaces all instances of torch.nn.LayerNorm with a apex.normalization.fused_layer_norm.FusedLayerNorm.
apply_gated_linear_units	Replaces the Linear layers in the feed-forward network with Gated Linear Units.
apply_ghost_batchnorm	Replace batch normalization modules with ghost batch normalization modules.
apply_gradient_clipping	Clips all gradients in model based on specified clipping_type.
apply_gyro_dropout	Replaces all instances of torch.nn.Dropout with a GyroDropout.
apply_low_precision_layernorm
apply_squeeze_excite	Adds Squeeze-and-Excitation blocks (Hu et al, 2019) after torch.nn.Conv2d layers.
apply_stochastic_depth	Applies Stochastic Depth (Huang et al, 2016) to the specified model.
apply_weight_standardization	Weight Standardization standardizes convolutional weights in a model.
augmix_image	Applies the AugMix (Hendrycks et al, 2020) data augmentation.
colout_batch	Applies ColOut augmentation to a batch of images and (optionally) targets, dropping the same random rows and columns from all images and targets in a batch.
compute_ema	Updates the weights of ema_model to be closer to the weights of model according to an exponential weighted average.
cutmix_batch	Create new samples using combinations of pairs of samples.
cutout_batch	See CutOut.
freeze_layers	Progressively freeze the layers of the network in-place during training, starting with the earlier layers.
mixup_batch	Create new samples using convex combinations of pairs of samples.
randaugment_image	Randomly applies a sequence of image data augmentations to an image or batch of images.
resize_batch	Resize inputs and optionally outputs by cropping or interpolating.
select_using_loss	Prunes minibatches as a subroutine of SelectiveBackprop.
set_batch_sequence_length	Set the sequence length of a batch.
should_selective_backprop	Decides if selective backprop should be run based on time in training.
smooth_labels	Shrink targets towards a uniform distribution as in Szegedy et al.