composer.distributed.ddp_sync_context(state, is_final_microbatch, sync_strategy)[source]#

A context manager for handling the DDPSyncStrategy.

  • state (State) โ€“ The state of the Trainer.

  • is_final_microbatch (bool) โ€“ Whether or not the context is being used during the final microbatch of the gradient accumulation steps.

  • sync_strategy (str | DDPSyncStrategy) โ€“ The ddp sync strategy to use. If a string is provided, the string must be one of the values in DDPSyncStrategy.