- class composer.algorithms.LowPrecisionGroupNorm(apply_at=Event.INIT)#
Replaces all instances of
LPGroupNorm is a thin wrapper around
torch.nn.GroupNormwhich forces the layer to run in lower precision (torch.float16 or torch.bfloat16) if autocast is enabled. This algorithm has no effect in FP32 or DeepSpeed FP16 mode, where autocast is disabled.
This algorithm is intended to be used instead of Fused GroupNorm. They have similar behavior and performance.
apply_at (Event) – Event where algorithm is applied.