World#

class streaming.base.world.World[source]#

Information about the nodes, ranks and workers of this run.

Warning

Be careful as to whether this object was initialized in a worker (if workers are used) or in a rank (which will claim one worker per rank).

Warning

In this World object, the counts (num_nodes, num_ranks, num_workers) are global – not to be confused with DataLoader num_workers, which is per rank.

Nodes are all assumed to contain the same number of devices (via local_world_size).

Nodes:

  • node / num_nodes

  • is_multinode

Ranks:

  • rank / num_ranks

  • rank_of_node / ranks_per_node

Workers:

  • worker / num_workers

  • worker_of_node / workers_per_node

  • worker_of_rank / workers_per_rank

  • is_leader

  • is_local_leader