PolynomialScheduler#
- class composer.optim.PolynomialScheduler(power, t_max='1dur', alpha_f=0.0)[source]#
Sets the learning rate to be proportional to a power of the fraction of training time left.
Specifically, the learning rate multiplier \(\alpha\) can be expressed as:
\[\alpha(t) = \alpha_f + (1 - \alpha_f) \times (1 - \tau) ^ {\kappa} \]Given \(\tau\), the fraction of time elapsed (clipped to the interval \([0, 1]\)), as:
\[\tau = t / t_{max} \]Where \(\kappa\) represents the exponent to be used for the proportionality relationship, \(t_{max}\) represents the duration of this scheduler, and \(\alpha_f\) represents the learning rate multiplier to decay to.