# composer.functional#

Functional API for applying algorithms in your own training loop.

from composer import functional as cf
from torchvision import models

model = models.resnet50()

# replace some layers with blurpool
cf.apply_blurpool(model)
# replace some layers with squeeze-excite
cf.apply_squeeze_excite(model, latent_channels=64, min_channels=128)


Functions

 apply_alibi Removes position embeddings and replaces the attention function and attention mask as per Alibi. apply_blurpool Add anti-aliasing filters to strided torch.nn.Conv2d and/or torch.nn.MaxPool2d modules. apply_channels_last Changes the memory format of the model to torch.channels_last. apply_factorization Replaces torch.nn.Linear and torch.nn.Conv2d modules with FactorizedLinear and FactorizedConv2d modules. apply_fused_layernorm Replaces all instances of torch.nn.LayerNorm with a apex.normalization.fused_layer_norm.FusedLayerNorm. apply_gated_linear_units Replaces the Linear layers in the feed-forward network with Gated Linear Units. apply_ghost_batchnorm Replace batch normalization modules with ghost batch normalization modules. apply_gradient_clipping Clips all gradients in model based on specified clipping_type. apply_squeeze_excite Adds Squeeze-and-Excitation blocks (Hu et al, 2019) after torch.nn.Conv2d layers. apply_stochastic_depth Applies Stochastic Depth (Huang et al, 2016) to the specified model. augmix_image Applies the AugMix (Hendrycks et al, 2020) data augmentation. colout_batch Applies ColOut augmentation to a batch of images and (optionally) targets, dropping the same random rows and columns from all images and targets in a batch. compute_ema Updates the weights of ema_model to be closer to the weights of model according to an exponential weighted average. cutmix_batch Create new samples using combinations of pairs of samples. cutout_batch See CutOut. freeze_layers Progressively freeze the layers of the network in-place during training, starting with the earlier layers. mixup_batch Create new samples using convex combinations of pairs of samples. randaugment_image Randomly applies a sequence of image data augmentations to an image or batch of images. resize_batch Resize inputs and optionally outputs by cropping or interpolating. select_using_loss Prunes minibatches as a subroutine of SelectiveBackprop. set_batch_sequence_length Set the sequence length of a batch. should_selective_backprop Decides if selective backprop should be run based on time in training. smooth_labels Shrink targets towards a uniform distribution as in Szegedy et al.