Policy#
Policies are objects that define how some value changes through time. Good example of them is Learning Rate Schedulers.
Learning Rate Schedulers#
Contrary to default PyTorch Learning Rate schedulers, ours does not require an optimizer to be passed during initialization.
Thunder Schedulers#
| Name | Description |
|---|---|
| Multiply | Multiplies lr on each step by specified factor. |
| Schedule | Assigns lr values according to specified callable. |
| Switch | Assigns lr values according to specified dict schedule. |