- composer.functional.should_selective_backprop(current_duration, batch_idx, start=0.5, end=0.9, interrupt=2)#
Decides if selective backprop should be run based on time in training.
Returns true if the
end. It is recommended that SB be applied during the later stages of a training run, once the model has already “learned” easy examples.
To preserve convergence, SB can be interrupted with vanilla minibatch gradient steps every
interrupt=0, SB will be used at every step during the SB interval. When
interrupt=2, SB will alternate with vanilla minibatch steps.
current_duration (float) – The elapsed training duration. Must be within
batch_idx (int) – The current batch within the epoch.
start (float, optional) – The duration at which selective backprop should be enabled, as a percentage. Default:
end (float, optional) – The duration at which selective backprop should be disabled. Default:
interrupt (int, optional) – The number of batches between vanilla minibatch gradient updates. Default:
bool – If selective backprop should be performed on this batch.