ddp_sync_context

composer.distributed.ddp_sync_context(state, is_final_microbatch, sync_strategy)

A context manager for handling the DDPSyncStrategy.

Parameters

• state (State) – The state of the Trainer.

• is_final_microbatch (bool) – Whether or not the context is being used during the final microbatch of the gradient accumulation steps.

• sync_strategy (str | DDPSyncStrategy) – The ddp sync strategy to use. If a string is provided, the string must be one of the values in DDPSyncStrategy.