grilly.optim.sgd

SGD Optimizer

Stochastic Gradient Descent optimizer.

Classes

Iterator()

Optimizer(params, defaults)

Base class for all optimizers.

SGD(params[, lr, momentum, weight_decay, ...])

Stochastic Gradient Descent optimizer.
class grilly.optim.sgd.SGD(params, lr=0.001, momentum=0.0, weight_decay=0.0, dampening=0.0, nesterov=False, use_gpu=False)[source]

Bases: Optimizer

Stochastic Gradient Descent optimizer.

Implements: param = param - lr * grad

Note: SGD is simple enough that CPU implementation is efficient. For GPU acceleration, we could use a generic update shader in the future.

Initialize SGD optimizer.

Parameters

  • params (Iterator[numpy.ndarray]) – Iterator of parameter arrays to optimize

  • lr (float) – Learning rate (default: 1e-3)

  • momentum (float) – Momentum factor (default: 0.0)

  • weight_decay (float) – Weight decay (L2 penalty) (default: 0.0)

  • dampening (float) – Dampening for momentum (default: 0.0)

  • nesterov (bool) – Enable Nesterov momentum (default: False)

  • use_gpu (bool) – Whether to attempt GPU acceleration (default: False, CPU is efficient)

__init__(params, lr=0.001, momentum=0.0, weight_decay=0.0, dampening=0.0, nesterov=False, use_gpu=False)[source]

Initialize SGD optimizer.

Parameters

  • params (Iterator[numpy.ndarray]) – Iterator of parameter arrays to optimize

  • lr (float) – Learning rate (default: 1e-3)

  • momentum (float) – Momentum factor (default: 0.0)

  • weight_decay (float) – Weight decay (L2 penalty) (default: 0.0)

  • dampening (float) – Dampening for momentum (default: 0.0)

  • nesterov (bool) – Enable Nesterov momentum (default: False)

  • use_gpu (bool) – Whether to attempt GPU acceleration (default: False, CPU is efficient)

Dependencies: None detected from callable globals.

Variables: params (collections.abc.Iterator[numpy.ndarray], required); lr (float, optional, default 0.001); momentum (float, optional, default 0.0); weight_decay (float, optional, default 0.0); dampening (float, optional, default 0.0); nesterov (bool, optional, default False); use_gpu (bool, optional, default False).

Usage Example

import numpy as np
from grilly.optim.sgd import SGD

instance = SGD(...)
result = instance.__init__(params=np.zeros(1, dtype=np.float32), lr=0.001, momentum=0.0, weight_decay=0.0, dampening=0.0, nesterov=False, use_gpu=False)
_get_backend()[source]

Get or create backend instance

Dependencies: None detected from callable globals.

Variables: This callable does not take explicit input variables.

Usage Example

from grilly.optim.sgd import SGD

instance = SGD(...)
result = instance._get_backend()
step(closure=None)[source]

Perform a single optimization step.

Parameters

closure – Optional closure that reevaluates the model and returns loss

Dependencies: numpy.

Variables: closure (Any, optional, default None).

Usage Example

from grilly.optim.sgd import SGD

instance = SGD(...)
result = instance.step(closure=None)
load_state_dict(state_dict)

Load optimizer state from state_dict.

Parameters

state_dict (dict[str, Any]) – Dictionary containing optimizer state

Dependencies: None detected from callable globals.

Variables: state_dict (dict[str, typing.Any], required).

Usage Example

from grilly.optim.base import Optimizer

instance = Optimizer(...)
result = instance.load_state_dict(state_dict='example')
state_dict()

Return the state of the optimizer as a dict.

Returns

Dictionary containing optimizer state

Return type

dict[str, Any]

Dependencies: None detected from callable globals.

Variables: This callable does not take explicit input variables.

Usage Example

from grilly.optim.base import Optimizer

instance = Optimizer(...)
result = instance.state_dict()
zero_grad()

Clear gradients for all parameters.

Note: In this implementation, gradients are expected to be stored in a separate structure (e.g., in the model’s backward pass). This method is provided for API compatibility.

Dependencies: None detected from callable globals.

Variables: This callable does not take explicit input variables.

Usage Example

from grilly.optim.base import Optimizer

instance = Optimizer(...)
result = instance.zero_grad()