One Cycle Policy

class src.pipelines.tensorflow_v2.callbacks.one_cycle.OneCycleLR(init_lr, max_lr, final_tail_lr, iterations, tail_length, init_momentum=None, min_momentum=None, cyclic_momentum=False)

get_lr(iteration)

Gets the updated learning rate, where both the size and direction of change depends on where the current iterations falls in the LR cycle

Parameters

iteration (int) – current iteration number

Return type

float

Returns

updated learning rate

get_momentum(iteration)

Gets the updated momentum, where both the size and direction of change depends on where the current iterations falls in the LR cycle

Parameters

iteration (int) – current iteration number

Return type

float

Returns

updated momentum

on_train_begin(batch, logs={})

Overwrite the optimiser LR and momentum (if applicable) to what’s specified in this callback

Parameters

• batch (int) – batch number

• logs – TF logs

Return type

None

Returns

None

on_train_batch_end(batch, logs={})

Updates LR and momentum (if applicable)

Parameters

• batch (int) – batch number

• logs – TF logs

Return type

None

Returns

None

static get_iterations(train_length, batch_size, epochs)

Gets the number of iterations required to achieve the maximum learning rate, provided both the initial learning rate and the increment size (which is on a log base 10 scale)

Parameters

• train_length (int) – sample size of training input (n)

• batch_size (int) – batch size

• epochs (int) – number of epochs

Return type

int

Returns

the total number of iterations the optimiser will run for

Note

Integer division is used

plot_ocp_lr(figsize=(5, 5), ax=None)

Plots the instance lr, plot should show the expected 1cycle lr shape

Parameters

• figsize (Tuple[int, int]) – matplotlib.pyplot figure size

• ax (Optional[axes]) – matplotlib.pyplot axis

Return type

axes

Returns

matplotlib.pyplot axis containing the plot described above

plot_ocp_momentum(figsize=(5, 5), ax=None)

Plots the instance momentum, plot should show the expected 1cycle momentum shape

Parameters

• figsize (Tuple[int, int]) – matplotlib.pyplot figure size

• ax (Optional[axes]) – matplotlib.pyplot axis

Returns

matplotlib.pyplot axis containing the plot described above

plot_ocp_lr_momentum(plot_opt, figsize=(10, 5))

Wrapper that plots both the 1cycle momentum and lr together; the plots can either be shown side-by-side or overlaid

Parameters

• plot_opt (int) – plot option, 1 indicates that plots should be overlaid while two indicates that plots should be shown side-by-side

• figsize (Tuple[int, int]) – matplotlib.pyplot figure size

Return type

None

Returns

None