Benchmarker

class composer.callbacks.benchmarker.Benchmarker(min_steps: int = 50, epoch_list: Sequence[int] = (0, 1), step_list: Sequence[int] = (0, 50), all_epochs: bool = False)[source]

Bases: composer.core.callback.Callback

Fast-forward the training loop to record throughput for specific epochs and/or steps.

It modifies the step and epoch to fast-forward the training loop, so that algorithms that activate at specific times will trigger and can be profiled.

It stops the training process after the last step or epoch is profiled.

It logs:

  • The throughput (averaged over the number of steps profiled in an epoch) to the throughput/step key.

  • The total elapsed training time to the wall_clock_train key.

Warning

The Benchmarker: should NOT be used in conjunction with the SpeedMonitor, since they log to the same keys.

Warning

The Benchmarker: modifies the State, which is an exception to the convention that callbacks should NOT modify state. This callback may break other algorithms and callbacks.

Parameters

  • min_steps (int, optional) – Maximum number of steps to profile per epoch, regardless of the length of regardless of the length of step_list. Defaults to 50.

  • epoch_list (Sequence[int], optional) – List of epochs at which to measure throughput. Defaults to [0, 1].

  • step_list (Sequence[int], optional) – List of steps at which to measure throughput. Defaults to [0, 50].

  • all_epochs (bool, optional) –

    Whether to override epoch_list and profile at all epochs.

    If False (the default), then it fast-forwards to the steps and epochs being profiled (specified by epoch_list and step_list, respectively).

    Otherwise, if True, then the throughput for the first min_steps batches of every epoch are recorded.

batch_end(state: State, logger: Logger)[source]

Called on the BATCH_END event.

Parameters

  • state (State) – The global state.

  • logger (Logger) – The logger.

batch_start(state: State, logger: Logger)[source]

Called on the BATCH_START event.

Parameters

  • state (State) – The global state.

  • logger (Logger) – The logger.

epoch_end(state: State, logger: Logger)[source]

Called on the EPOCH_END event.

Parameters

  • state (State) – The global state.

  • logger (Logger) – The logger.

training_start(state: State, logger: Logger)[source]

Called on the Event.TRAINING_START event.

Parameters

  • state (State) – The global state.

  • logger (Logger) – The logger.