World#
- class streaming.base.world.World[source]#
Information about the nodes, ranks and workers of this run.
Warning
Be careful as to whether this object was initialized in a worker (if workers are used) or in a rank (which will claim one worker per rank).
Warning
In this World object, the counts (num_nodes, num_ranks, num_workers) are global – not to be confused with DataLoader num_workers, which is per rank.
Nodes are all assumed to contain the same number of devices (via local_world_size).
- Nodes:
node / num_nodes
is_multinode
- Ranks:
rank / num_ranks
rank_of_node / ranks_per_node
- Workers:
worker / num_workers
worker_of_node / workers_per_node
worker_of_rank / workers_per_rank
is_leader
is_local_leader