Stochastic Gradient Descent with Warm Restarts — sgdr • deepG

Compute the learning Rate for a given epoch using Stochastic Gradient Descent with Warm Restarts. Implements approach from this paper.

Usage

sgdr(lrmin = 5e-10, lrmax = 0.05, restart = 50, mult = 1, epoch = NULL)

Arguments

lrmin: Lower limit of the range for the learning rate.
lrmax: Upper limit of the range for the learning rate.
restart: Number of epochs until a restart is conducted.
mult: Factor, by which the number of epochs until a restart is increased at every restart.
epoch: Epoch, for which the learning rate shall be calculated.

Value

A numeric value.

Examples

sgdr(lrmin = 5e-10, lrmax = 5e-2, restart = 50,
mult = 1, epoch = 5)
#> [1] 0.04877641