# composer.utils.reproducibility#

Helper utilities for configuring deterministic training to ensure reproducibility.

Note

For deterministic model initialization, seed_all() and/or configure_deterministic_mode() should be invoked before creating and initializing a model, before creating the Trainer. For example:

>>> import torch.nn
>>> from composer.utils import reproducibility
>>> reproducibility.configure_deterministic_mode()
>>> reproducibility.seed_all(42)
>>> model = MyModel()
>>> def init_weights(m):
...     if isinstance(m, torch.nn.Linear):
...         torch.nn.init.xavier_uniform(m.weight)
>>> # model will now be deterministically initialized, since the seed is set.
>>> init_weights(model)
>>> trainer = Trainer(model=model, seed=42)


Note that the seed must also be passed to the Trainer, otherwise the Trainer would generate a random seed based on the timestamp (see get_random_seed()).

composer.utils.reproducibility.MAX_SEED[source]#

The maximum allowed seed, which is $$2^{32} - 1$$.

Type

int

Functions

 configure_deterministic_mode Configure PyTorch deterministic mode. get_random_seed Get a randomly created seed to use for seeding rng objects. get_rng_state The state of the RNG objects. load_rng_state Restore the RNG state. seed_all Seed all rng objects.

Attributes

composer.utils.reproducibility.configure_deterministic_mode()[source]#

Configure PyTorch deterministic mode.

Note

When using the Trainer, you can use the deterministic_mode flag instead of invoking this function directly. For example:

>>> trainer = Trainer(deterministic_mode=True)


However, to configure deterministic mode for operations before the trainer is initialized, manually invoke this function at the beginning of your training script.

Note

When training on a GPU, this function must be invoked before any CUDA operations.

Note

Deterministic mode degrades performance. Do not use outside of testing and debugging.

composer.utils.reproducibility.get_random_seed()[source]#

Get a randomly created seed to use for seeding rng objects.

Warning

This random seed is NOT cryptographically secure.

Returns

int – A random seed.

composer.utils.reproducibility.get_rng_state()[source]#

The state of the RNG objects.

Returns

List[Dict[str, Any]] – A list of RNG State Dicts, indexed by global rank.

Restore the RNG state.

Parameters

rng_state_dicts (List[Dict[str, Any]]) – The list of RNG state dicts to restore, as returned by get_rng_state().

composer.utils.reproducibility.seed_all(seed)[source]#

Seed all rng objects.

Note

When using the Trainer, you can use the seed parameter instead of invoking this function directly. For example:

>>> trainer = Trainer(seed=42)


However, to configure the random seed for operations before the trainer is initialized, manually invoke this function at the beginning of your training script.

Parameters

seed (int) – The random seed