# composer.algorithms.algorithm_hparams_registry#

composer.algorithms.algorithm_hparams_registry

Classes

 Algorithm Base class for algorithms. Alibi ALiBi (Attention with Linear Biases; Press et al, 2021) dispenses with position embeddings and instead directly biases attention matrices such that nearby tokens attend to one another more strongly. AugMix The AugMix data augmentation technique. BlurPool BlurPool adds anti-aliasing filters to convolutional layers. ChannelsLast Changes the memory format of the model to torch.channels_last. ColOut Drops a fraction of the rows and columns of an input image and (optionally) a target image. CutMix CutMix trains the network on non-overlapping combinations of pairs of examples and interpolated targets rather than individual examples and targets. CutOut CutOut is a data augmentation technique that works by masking out one or more square regions of an input image. EMA Maintains a shadow model with weights that follow the exponential moving average of the trained model weights. Factorize Decomposes linear operators into pairs of smaller linear operators. FusedLayerNorm Replaces all instances of torch.nn.LayerNorm with a apex.normalization.fused_layer_norm.FusedLayerNorm. GatedLinearUnits Replaces all instances of Linear layers in the feed-forward subnetwork with a Gated Linear Unit. GhostBatchNorm Replaces batch normalization modules with Ghost Batch Normalization modules that simulate the effect of using a smaller batch size. GradientClipping Clips all gradients in model based on specified clipping_type. LabelSmoothing Shrink targets towards a uniform distribution as in Szegedy et al. LayerFreezing Progressively freeze the layers of the network during training, starting with the earlier layers. MixUp MixUp trains the network on convex batch combinations. NoOpModel Runs on Event.INIT and replaces the model with a dummy NoOpModelClass instance. ProgressiveResizing Resize inputs and optionally outputs by cropping or interpolating. RandAugment Randomly applies a sequence of image data augmentations to an image. SAM Adds sharpness-aware minimization (Foret et al, 2020) by wrapping an existing optimizer with a SAMOptimizer. SWA Applies Stochastic Weight Averaging (Izmailov et al, 2018). SelectiveBackprop Selectively backpropagate gradients from a subset of each batch. SeqLengthWarmup Progressively increases the sequence length during training. SqueezeExcite Adds Squeeze-and-Excitation blocks (Hu et al, 2019) after the torch.nn.Conv2d modules in a neural network. StochasticDepth Applies Stochastic Depth (Huang et al, 2016) to the specified model.

Attributes

• Dict

• Type

• Union

• algorithm_registry