apply_fused_layernorm

composer.functional.apply_fused_layernorm(model, optimizers)

Replaces all instances of torch.nn.LayerNorm with a apex.normalization.fused_layer_norm.FusedLayerNorm.

By fusing multiple kernel launches into one, this usually improves GPU utilization.