Learning Rate Range Test

class src.pipelines.tensorflow_v2.callbacks.lr_range_test.LRRangeTest(init_lr=1e-08, max_lr=10.0, beta=0.98, increment=None, iterations=None, verbose=None)

static get_iterations(init_lr, max_lr, increment)

Gets the number of iterations required to achieve the maximum learning rate, provided both the initial learning rate and the increment size (which is on a log base 10 scale)

Parameters

• init_lr (float) – initial learning rate

• max_lr (float) – maximum learning rate

• increment (float) – multiplicative increment size on a log base 10 scale

Return type

float

Returns

number of iterations

Note

Integer division is used

static get_increment(init_lr, max_lr, iterations)

Gets the increment, using a log base 10 scale, required to achieve the maximum learning rate, provided both the initial learning rate and the number of iterations

Parameters

• init_lr (float) – initial learning rate

• max_lr (float) – maximum learning rate

• iterations (int) – number of iterations

Return type

float

Returns

increment on log base 10 scale

on_train_batch_end(batch, logs={})

Updates LR and momentum (if applicable), as well as calculates smoothed loss at the end of each training batch

Parameters

• batch – batch number

• logs – TF logs

Return type

None

Returns

None

Note

Batch starts at 0 and using training loss

plot_LRTest(num_points_excl, figsize=(5, 5))

Plot the results of the LR Range test

Parameters

• figsize (Tuple[int, int]) – matplotlib.pyplot figure size

• num_points_excl (int) – number of end points to exclude, required due to error with self.model.stop_training = True, present in TF 2.3.0

Return type

None

Returns

None

Note

Requires the instance batch_log_lr and smoothed_losses variables to not be empty.