World#

class streaming.base.world.World(num_nodes, ranks_per_node, workers_per_rank, worker)[source]#

Information about the nodes, ranks and workers of this run.

Warning

Be careful as to whether this object was initialized in a worker (if workers are used) or in a rank (which will claim one worker per rank).

Warning

In this World object, the counts (num_nodes, num_ranks, num_workers) are global – not to be confused with DataLoader num_workers, which is per rank.

Nodes are all assumed to contain the same number of devices (via local_world_size).

Nodes:

  • node / num_nodes

  • is_multinode

Ranks:

  • rank / num_ranks

  • rank_of_node / ranks_per_node

Workers:

  • worker / num_workers

  • worker_of_node / workers_per_node

  • worker_of_rank / workers_per_rank

  • is_leader

  • is_local_leader

copy()[source]#

Get a copy of this world state.

Returns

Self – A new copy with the same state.

classmethod detect()[source]#

Detect the world state.

Returns

Self – A new World state object according to dist and get_worker_info().

detect_workers()[source]#

Get a copy of this world state with the worker information newly detected.

Returns

Self – A new workers-newly-detected version of this World state object.

replicate(replication)[source]#

Get a copy of this world state with the given replication factor.

Parameters

replication (int) – Replication factor – how many consecutive devices that should see the same samples..

Returns

Self – A new sample replication version of this World state object.

to_json()[source]#

Get a JSON version of this config.

Returns

Dict[str, Any] – JSON config.