composer.optim.optimizer_hparams#

Hyperparameters for optimizers.

Hparams

These classes are used with yahp for YAML-based configuration.

`AdamHparams`	Hyperparameters for the `Adam` optimizer.
`AdamWHparams`	Hyperparameters for the `AdamW` optimizer.
`DecoupledAdamWHparams`	Hyperparameters for the `DecoupledAdamW` optimizer.
`DecoupledSGDWHparams`	Hyperparameters for the `DecoupledSGDW` optimizer.
`OptimizerHparams`	Base class for optimizer hyperparameter classes.
`RAdamHparams`	Hyperparameters for the `RAdam` optimizer.
`RMSpropHparams`	Hyperparameters for the `RMSprop` optimizer.
`SGDHparams`	Hyperparameters for the `SGD` optimizer.

class composer.optim.optimizer_hparams.AdamHparams(lr=0.001, betas=<factory>, eps=1e-08, weight_decay=0.0, amsgrad=False)[source]#

Bases: composer.optim.optimizer_hparams.OptimizerHparams

Hyperparameters for the Adam optimizer.

See Adam for documentation.

Parameters

lr (float, optional) – See Adam.
betas (float, optional) – See Adam.
eps (float, optional) – See Adam.
weight_decay (float, optional) – See Adam.
amsgrad (bool, optional) – See Adam.

class composer.optim.optimizer_hparams.AdamWHparams(lr=0.001, betas=<factory>, eps=1e-08, weight_decay=0.01, amsgrad=False)[source]#

Bases: composer.optim.optimizer_hparams.OptimizerHparams

Hyperparameters for the AdamW optimizer.

See AdamW for documentation.

Parameters

lr (float, optional) – See AdamW.
betas (float, optional) – See AdamW.
eps (float, optional) – See AdamW.
weight_decay (float, optional) – See AdamW.
amsgrad (bool, optional) – See AdamW.

class composer.optim.optimizer_hparams.DecoupledAdamWHparams(lr=0.001, betas=<factory>, eps=1e-08, weight_decay=0.01, amsgrad=False)[source]#

Bases: composer.optim.optimizer_hparams.OptimizerHparams

Hyperparameters for the DecoupledAdamW optimizer.

See DecoupledAdamW for documentation.

Parameters

lr (float, optional) – See DecoupledAdamW.
betas (float, optional) – See DecoupledAdamW.
eps (float, optional) – See DecoupledAdamW.
weight_decay (float, optional) – See DecoupledAdamW.
amsgrad (bool, optional) – See DecoupledAdamW.

class composer.optim.optimizer_hparams.DecoupledSGDWHparams(lr, momentum=0.0, weight_decay=0.0, dampening=0.0, nesterov=False)[source]#

Bases: composer.optim.optimizer_hparams.OptimizerHparams

Hyperparameters for the DecoupledSGDW optimizer.

See DecoupledSGDW for documentation.

Parameters

lr (float) – See DecoupledSGDW.
momentum (float, optional) – See DecoupledSGDW.
weight_decay (float, optional) – See DecoupledSGDW.
dampening (float, optional) – See DecoupledSGDW.
nesterov (bool, optional) – See DecoupledSGDW.

class composer.optim.optimizer_hparams.OptimizerHparams[source]#

Bases: yahp.hparams.Hparams, abc.ABC

Base class for optimizer hyperparameter classes.

Optimizer parameters that are added to TrainerHparams (e.g. via YAML or the CLI) are initialized in the training loop.

initialize_object(param_group)[source]#

Initializes the optimizer.

Parameters: param_group (Iterable[Tensor] | Iterable[Dict[str, Tensor]]) – Parameters for this optimizer to optimize.

class composer.optim.optimizer_hparams.RAdamHparams(lr=0.001, betas=<factory>, eps=1e-08, weight_decay=0.0)[source]#

Bases: composer.optim.optimizer_hparams.OptimizerHparams

Hyperparameters for the RAdam optimizer.

See RAdam for documentation.

Parameters

lr (float, optional) – See RAdam.
betas (float, optional) – See RAdam.
eps (float, optional) – See RAdam.
weight_decay (float, optional) – See RAdam.

class composer.optim.optimizer_hparams.RMSpropHparams(lr, alpha=0.99, eps=1e-08, momentum=0.0, weight_decay=0.0, centered=False)[source]#

Bases: composer.optim.optimizer_hparams.OptimizerHparams

Hyperparameters for the RMSprop optimizer.

See RMSprop for documentation.

Parameters

lr (float) – See RMSprop.
alpha (float, optional) – See RMSprop.
eps (float, optional) – See RMSprop.
momentum (float, optional) – See RMSprop.
weight_decay (float, optional) – See RMSprop.
centered (bool, optional) – See RMSprop.

class composer.optim.optimizer_hparams.SGDHparams(lr, momentum=0.0, weight_decay=0.0, dampening=0.0, nesterov=False)[source]#

Bases: composer.optim.optimizer_hparams.OptimizerHparams

Hyperparameters for the SGD optimizer.

See SGD for documentation.

Parameters

lr (float) – See SGD.
momentum (float, optional) – See SGD.
weight_decay (float, optional) – See SGD.
dampening (float, optional) – See SGD.
nesterov (bool, optional) – See SGD.