Core SelectiveBackprop class and functions.
Selective Backprop prunes minibatches according to the difficulty of the individual training examples, and only computes weight gradients over the pruned subset, reducing iteration time and speeding up training.
The algorithm runs on
On Event.INIT, it gets the loss function before the model is wrapped. On Event.AFTER_DATALOADER, it applies selective
backprop if the time is between
See the Method Card for more details.
Prunes minibatches as a subroutine of SelectiveBackprop.
Decides if selective backprop should be run based on time in training.
Selectively backpropagate gradients from a subset of each batch.