should_selective_backprop#

composer.functional.should_selective_backprop(current_duration, batch_idx, start=0.5, end=0.9, interrupt=2)[source]#

Decides if selective backprop should be run based on time in training.

Returns true if the current_duration is between start and end. It is recommended that SB be applied during the later stages of a training run, once the model has already โ€œlearnedโ€ easy examples.

To preserve convergence, SB can be interrupted with vanilla minibatch gradient steps every interrupt steps. When interrupt=0, SB will be used at every step during the SB interval. When interrupt=2, SB will alternate with vanilla minibatch steps.

Parameters

  • current_duration (float) โ€“ The elapsed training duration. Must be within [0.0, 1.0).

  • batch_idx (int) โ€“ The current batch within the epoch.

  • start (float, optional) โ€“ The duration at which selective backprop should be enabled, as a percentage. Default: 0.5.

  • end (float, optional) โ€“ The duration at which selective backprop should be disabled. Default: 0.9.

  • interrupt (int, optional) โ€“ The number of batches between vanilla minibatch gradient updates. Default: 2.

Returns

bool โ€“ If selective backprop should be performed on this batch.