PolynomialScheduler

class composer.optim.PolynomialScheduler(power, t_max='1dur', alpha_f=0.0)

Sets the learning rate to be proportional to a power of the fraction of training time left.

Specifially, the learning rate multiplier $\alpha$ can be expressed as:

$$\alpha(t) = \alpha_f + (1 - \alpha_f) \times (1 - \tau) ^ {\kappa}$$

Given $\tau$, the fraction of time elapsed (clipped to the interval $[0, 1]$), as:

$$\tau = t / t_{max}$$

Where $\kappa$ represents the exponent to be used for the proportionality relationship, $t_{max}$ represents the duration of this scheduler, and $\alpha_f$ represents the learning rate multiplier to decay to.

Parameters

• power (float) – The exponent to be used for the proportionality relationship.

• t_max (str | Time) – The duration of this scheduler. Default = "1dur".

• alpha_f (float) – Learning rate multiplier to decay to. Default = 0.0.