LabelSmoothing

class composer.algorithms.LabelSmoothing(smoothing=0.1, target_key=1)

Shrink targets towards a uniform distribution as in Szegedy et al.

The smoothed labels are computed as (1 - smoothing) * targets + smoothing * unif where unif is a vector with elements all equal to 1 / num_classes.

Parameters

• smoothing – Strength of the label smoothing, in \([0, 1]\). smoothing=0 means no label smoothing, and smoothing=1 means maximal smoothing (targets are ignored). Default: 0.1.

• target_key (str | int | tuple[Callable, Callable] | Any, optional) – A key that indexes to the target from the batch. Can also be a pair of get and set functions, where the getter is assumed to be first in the pair. The default is 1, which corresponds to any sequence, where the second element is the target. Default: 1.

Example

from composer.algorithms import LabelSmoothing
algorithm = LabelSmoothing(smoothing=0.1)
trainer = Trainer(
    model=model,
    train_dataloader=train_dataloader,
    eval_dataloader=eval_dataloader,
    max_duration="1ep",
    algorithms=[algorithm],
    optimizers=[optimizer]
)