composer.distributed#
Distributed training.
Functions
A context manager for handling the |
|
Wraps the module in a |
|
Prepare a module (assumed ComposerModel) and optimizer for use with |
|
Prepare a module (assumed ComposerModel) for use with tensor parallel. |
Classes
How and when gradient synchronization should happen. |