Skip to contents

Compute the learning Rate for a given epoch using Stochastic Gradient Descent with Warm Restarts. Implements approach from this paper.

Usage

sgdr(lrmin = 5e-10, lrmax = 0.05, restart = 50, mult = 1, epoch = NULL)

Arguments

lrmin

Lower limit of the range for the learning rate.

lrmax

Upper limit of the range for the learning rate.

restart

Number of epochs until a restart is conducted.

mult

Factor, by which the number of epochs until a restart is increased at every restart.

epoch

Epoch, for which the learning rate shall be calculated.

Value

A numeric value.

Examples

sgdr(lrmin = 5e-10, lrmax = 5e-2, restart = 50,
mult = 1, epoch = 5)
#> [1] 0.04877641