all_gather_object#

composer.utils.dist.all_gather_object(obj, group=None)[source]#

Collect a pickleable object from each rank and return a list of these objects indexed by rank.

See also

torch.distributed.all_gather_object()

Parameters

  • obj (TObj) โ€“ Object to be gathered.

  • group (ProcessGroup, optional) โ€“ The process group to work on. If None, the default process group will be used. Default is None.

Returns

list[TObj] โ€“ A list of objects indexed by rank.