composer.algorithms.hparams#
composer.algorithms.hparams
Functions
Return the fields of a dataclass instance as a new dictionary mapping field names to field values. |
|
Returns the same class as was passed in, with dunder methods added based on the fields defined in the class. |
Classes
ALiBi (Attention with Linear Biases; Press et al, 2021) dispenses with position embeddings and instead directly biases attention matrices such that nearby tokens attend to one another more strongly. |
|
AugMix (Hendrycks et al, 2020) creates |
|
BlurPool adds anti-aliasing filters to convolutional layers to increase accuracy and invariance to small shifts in the input. |
|
Changes the memory format of the model to |
|
Drops a fraction of the rows and columns of an input image. |
|
CutMix trains the network on non-overlapping combinations of pairs of examples and interpolated targets rather than individual examples and targets. |
|
Cutout is a data augmentation technique that works by masking out one or more square regions of an input image. |
|
Decomposes linear operators into pairs of smaller linear operators. |
|
Replaces batch normalization modules with Ghost Batch Normalization modules that simulate the effect of using a smaller batch size. |
|
Shrink targets towards a uniform distribution as in Szegedy et al. |
|
Progressively freeze the layers of the network during training, starting with the earlier layers. |
|
MixUp trains the network on convex combinations of pairs of examples and targets rather than individual examples and targets. |
|
|
composer.algorithms.no_op_model.no_op_model.NoOpModel |
Apply Fastai's progressive resizing data augmentation to speed up training. |
|
Randomly applies a sequence of image data augmentations (Cubuk et al, 2019) to an image. |
|
Adds sharpness-aware minimization (Foret et al, 2020) by wrapping an existing optimizer with a |
|
Apply Stochastic Weight Averaging (Izmailov et al, 2018) |
|
Deprecated - do not use. |
|
Selectively backpropagate gradients from a subset of each batch (Jiang et al, 2019). |
|
Progressively increases the sequence length during training. |
|
Adds Squeeze-and-Excitation blocks (Hu et al, 2019) after the |
|
Applies Stochastic Depth (Huang et al, 2016) to the specified model. |
Hparams
These classes are used with yahp
for YAML
-based configuration.
Hyperparameters for algorithms. |
|
See |
|
See |
|
See |
|
ChannelsLast has no hyperparameters, so this class has no member variables. |
|
See |
|
See |
|
See |
|
See |
|
See |
|
See |
|
See |
|
See |
|
composer.algorithms.hparams.NoOpModelHparams |
|
See |
|
See |
|
See |
|
See |
|
See |
|
See |
|
composer.algorithms.hparams.SeqLengthWarmupHparams |
|
See |
|
See |
Attributes
Optional
- class composer.algorithms.hparams.AlibiHparams(position_embedding_attribute, attention_module_name, attr_to_replace, alibi_attention, mask_replacement_function=None, heads_per_layer=None, max_sequence_length=8192, train_sequence_length_scaling=0.25)[source]#
Bases:
composer.algorithms.algorithm_hparams.AlgorithmHparams
See
Alibi
- class composer.algorithms.hparams.AugMixHparams(severity=3, depth=- 1, width=3, alpha=1.0, augmentation_set='all')[source]#
Bases:
composer.algorithms.algorithm_hparams.AlgorithmHparams
See
AugMix
- class composer.algorithms.hparams.BlurPoolHparams(replace_convs=True, replace_maxpools=True, blur_first=True)[source]#
Bases:
composer.algorithms.algorithm_hparams.AlgorithmHparams
See
BlurPool
- class composer.algorithms.hparams.ChannelsLastHparams[source]#
Bases:
composer.algorithms.algorithm_hparams.AlgorithmHparams
ChannelsLast has no hyperparameters, so this class has no member variables.
- class composer.algorithms.hparams.ColOutHparams(p_row=0.15, p_col=0.15, batch=True)[source]#
Bases:
composer.algorithms.algorithm_hparams.AlgorithmHparams
See
ColOut
- class composer.algorithms.hparams.CutMixHparams(num_classes, alpha=1.0, uniform_sampling=False)[source]#
Bases:
composer.algorithms.algorithm_hparams.AlgorithmHparams
See
CutMix
- class composer.algorithms.hparams.CutOutHparams(num_holes=1, length=0.5, uniform_sampling=False)[source]#
Bases:
composer.algorithms.algorithm_hparams.AlgorithmHparams
See
CutOut
- class composer.algorithms.hparams.FactorizeHparams(factorize_convs=True, factorize_linears=True, min_channels=512, latent_channels=0.25, min_features=512, latent_features=0.25)[source]#
Bases:
composer.algorithms.algorithm_hparams.AlgorithmHparams
See
Factorize
- class composer.algorithms.hparams.GhostBatchNormHparams(ghost_batch_size=32)[source]#
Bases:
composer.algorithms.algorithm_hparams.AlgorithmHparams
See
GhostBatchNorm
- class composer.algorithms.hparams.LabelSmoothingHparams(smoothing=0.1)[source]#
Bases:
composer.algorithms.algorithm_hparams.AlgorithmHparams
See
LabelSmoothing
- class composer.algorithms.hparams.LayerFreezingHparams(freeze_start=0.5, freeze_level=1.0)[source]#
Bases:
composer.algorithms.algorithm_hparams.AlgorithmHparams
See
LayerFreezing
- class composer.algorithms.hparams.MixUpHparams(alpha=0.2, interpolate_loss=False)[source]#
Bases:
composer.algorithms.algorithm_hparams.AlgorithmHparams
See
MixUp
- class composer.algorithms.hparams.NoOpModelHparams[source]#
Bases:
composer.algorithms.algorithm_hparams.AlgorithmHparams
composer.algorithms.hparams.NoOpModelHparams
- class composer.algorithms.hparams.ProgressiveResizingHparams(mode='resize', initial_scale=0.5, finetune_fraction=0.2, resize_targets=False)[source]#
Bases:
composer.algorithms.algorithm_hparams.AlgorithmHparams
See
ProgressiveResizing
- class composer.algorithms.hparams.RandAugmentHparams(severity=9, depth=2, augmentation_set='all')[source]#
Bases:
composer.algorithms.algorithm_hparams.AlgorithmHparams
See
RandAugment
- class composer.algorithms.hparams.SAMHparams(rho=0.05, epsilon=1e-12, interval=1)[source]#
Bases:
composer.algorithms.algorithm_hparams.AlgorithmHparams
See
SAM
- class composer.algorithms.hparams.SWAHparams(swa_start='0.7dur', swa_end='0.97dur', update_interval='1ep', schedule_swa_lr=False, anneal_strategy='linear', anneal_steps=10, swa_lr=None)[source]#
Bases:
composer.algorithms.algorithm_hparams.AlgorithmHparams
See
SWA
- class composer.algorithms.hparams.ScaleScheduleHparams(ratio=1.0)[source]#
Bases:
composer.algorithms.algorithm_hparams.AlgorithmHparams
See
ScaleSchedule
- class composer.algorithms.hparams.SelectiveBackpropHparams(start=0.5, end=0.9, keep=0.5, scale_factor=0.5, interrupt=2)[source]#
Bases:
composer.algorithms.algorithm_hparams.AlgorithmHparams
See
SelectiveBackprop
- class composer.algorithms.hparams.SeqLengthWarmupHparams(duration: float = 0.3, min_seq_length: int = 8, max_seq_length: int = 1024, step_size: int = 8, truncate: bool = True)[source]#
Bases:
composer.algorithms.algorithm_hparams.AlgorithmHparams
composer.algorithms.hparams.SeqLengthWarmupHparams
- class composer.algorithms.hparams.SqueezeExciteHparams(latent_channels=64, min_channels=128)[source]#
Bases:
composer.algorithms.algorithm_hparams.AlgorithmHparams
See
SqueezeExcite
- class composer.algorithms.hparams.StochasticDepthHparams(target_layer_name, stochastic_method='block', drop_rate=0.2, drop_distribution='linear', use_same_gpu_seed=True, drop_warmup='0dur')[source]#
Bases:
composer.algorithms.algorithm_hparams.AlgorithmHparams
See
StochasticDepth