On this page
CosineAnnealingLR
class torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max, eta_min=0, last_epoch=-1, verbose=False)
[source]-
Set the learning rate of each parameter group using a cosine annealing schedule, where is set to the initial lr and is the number of epochs since the last restart in SGDR:
When last_epoch=-1, sets initial lr as lr. Notice that because the schedule is defined recursively, the learning rate can be simultaneously modified outside this scheduler by other operators. If the learning rate is set solely by this scheduler, the learning rate at each step becomes:
It has been proposed in SGDR: Stochastic Gradient Descent with Warm Restarts. Note that this only implements the cosine annealing part of SGDR, and not the restarts.
- Parameters
get_last_lr()
-
Return last computed learning rate by current scheduler.
load_state_dict(state_dict)
-
Loads the schedulers state.
- Parameters
-
state_dict (dict) – scheduler state. Should be an object returned from a call to
state_dict()
.
print_lr(is_verbose, group, lr, epoch=None)
-
Display the current learning rate.
state_dict()
-
Returns the state of the scheduler as a
dict
.It contains an entry for every variable in self.__dict__ which is not the optimizer.
© 2024, PyTorch Contributors
PyTorch has a BSD-style license, as found in the LICENSE file.
https://pytorch.org/docs/2.1/generated/torch.optim.lr_scheduler.CosineAnnealingLR.html