# all_gather#

composer.utils.dist.all_gather(tensor)[source]#

Collects a Tensor from each rank.

Parameters

tensor (Tensor) – Tensor from each rank to be gathered.

Returns

Sequence[Tensor] – A sequence of tensors indexed by rank.