grilly.optim.adam

Adam Optimizer

Uses: adam-update.glsl, affect-adam.glsl

Classes

Adam(params[, lr, betas, eps, weight_decay, ...])

Adam optimizer using GPU-accelerated shaders.

AffectAdam(params[, lr, betas, eps, ...])

Affect-aware Adam optimizer.

Iterator()

Optimizer(params, defaults)

Base class for all optimizers.
class grilly.optim.adam.Adam(params, lr=0.001, betas=(0.9, 0.999), eps=1e-08, weight_decay=0.0, use_gpu=True)[source]

Bases: Optimizer

Adam optimizer using GPU-accelerated shaders.

Uses: adam-update.glsl

Implements the Adam algorithm: - m = beta1 * m + (1 - beta1) * grad - v = beta2 * v + (1 - beta2) * grad^2 - m_hat = m / (1 - beta1^t) - v_hat = v / (1 - beta2^t) - param = param - lr * m_hat / (sqrt(v_hat) + eps)

Initialize Adam optimizer.

Parameters

  • params (Iterator[numpy.ndarray]) – Iterator of parameter arrays to optimize

  • lr (float) – Learning rate (default: 1e-3)

  • betas (tuple) – Coefficients for computing running averages (default: (0.9, 0.999))

  • eps (float) – Term added to denominator for numerical stability (default: 1e-8)

  • weight_decay (float) – Weight decay (L2 penalty) (default: 0.0)

  • use_gpu (bool) – Whether to use GPU acceleration (default: True)

__init__(params, lr=0.001, betas=(0.9, 0.999), eps=1e-08, weight_decay=0.0, use_gpu=True)[source]

Initialize Adam optimizer.

Parameters

  • params (Iterator[numpy.ndarray]) – Iterator of parameter arrays to optimize

  • lr (float) – Learning rate (default: 1e-3)

  • betas (tuple) – Coefficients for computing running averages (default: (0.9, 0.999))

  • eps (float) – Term added to denominator for numerical stability (default: 1e-8)

  • weight_decay (float) – Weight decay (L2 penalty) (default: 0.0)

  • use_gpu (bool) – Whether to use GPU acceleration (default: True)

Dependencies: None detected from callable globals.

Variables: params (collections.abc.Iterator[numpy.ndarray], required); lr (float, optional, default 0.001); betas (tuple, optional, default (0.9, 0.999)); eps (float, optional, default 1e-08); weight_decay (float, optional, default 0.0); use_gpu (bool, optional, default True).

Usage Example

import numpy as np
from grilly.optim.adam import Adam

instance = Adam(...)
result = instance.__init__(params=np.zeros(1, dtype=np.float32), lr=0.001, betas=(), eps=1e-08, weight_decay=0.0, use_gpu=True)
_get_backend()[source]

Get or create backend instance

Dependencies: None detected from callable globals.

Variables: This callable does not take explicit input variables.

Usage Example

from grilly.optim.adam import Adam

instance = Adam(...)
result = instance._get_backend()
step(closure=None, gradients=None)[source]

Perform a single optimization step.

Parameters

  • closure – Optional closure that reevaluates the model and returns loss

  • gradients – Optional dict mapping parameter IDs to gradients. If None, tries to get gradients from param.grad attribute.

Dependencies: numpy.

Variables: closure (Any, optional, default None); gradients (Any, optional, default None).

Usage Example

from grilly.optim.adam import Adam

instance = Adam(...)
result = instance.step(closure=None, gradients=None)
_adam_update_gpu(backend, param, grad, exp_avg, exp_avg_sq, lr, beta1, beta2, eps, beta1_t, beta2_t)[source]

GPU-accelerated Adam update using adam-update.glsl shader.

Dependencies: numpy.

Variables: backend (Any, required); param (Any, required); grad (Any, required); exp_avg (Any, required); exp_avg_sq (Any, required); lr (Any, required); beta1 (Any, required); beta2 (Any, required); eps (Any, required); beta1_t (Any, required); beta2_t (Any, required).

Usage Example

from grilly.optim.adam import Adam

instance = Adam(...)
result = instance._adam_update_gpu(backend=None, param=None, grad=None, exp_avg=None, exp_avg_sq=None, lr=None, beta1=None, beta2=None, eps=None, beta1_t=None, beta2_t=None)
load_state_dict(state_dict)

Load optimizer state from state_dict.

Parameters

state_dict (dict[str, Any]) – Dictionary containing optimizer state

Dependencies: None detected from callable globals.

Variables: state_dict (dict[str, typing.Any], required).

Usage Example

from grilly.optim.base import Optimizer

instance = Optimizer(...)
result = instance.load_state_dict(state_dict='example')
state_dict()

Return the state of the optimizer as a dict.

Returns

Dictionary containing optimizer state

Return type

dict[str, Any]

Dependencies: None detected from callable globals.

Variables: This callable does not take explicit input variables.

Usage Example

from grilly.optim.base import Optimizer

instance = Optimizer(...)
result = instance.state_dict()
zero_grad()

Clear gradients for all parameters.

Note: In this implementation, gradients are expected to be stored in a separate structure (e.g., in the model’s backward pass). This method is provided for API compatibility.

Dependencies: None detected from callable globals.

Variables: This callable does not take explicit input variables.

Usage Example

from grilly.optim.base import Optimizer

instance = Optimizer(...)
result = instance.zero_grad()
class grilly.optim.adam.AffectAdam(params, lr=0.001, betas=(0.9, 0.999), eps=1e-08, weight_decay=0.0, use_gpu=True)[source]

Bases: Adam

Affect-aware Adam optimizer.

Uses: affect-adam.glsl

Similar to Adam but optimized for affect/emotion processing.

Initialize AffectAdam optimizer.

Args are the same as Adam.

Parameters

  • params (Iterator[numpy.ndarray]) –

  • lr (float) –

  • betas (tuple) –

  • eps (float) –

  • weight_decay (float) –

  • use_gpu (bool) –

__init__(params, lr=0.001, betas=(0.9, 0.999), eps=1e-08, weight_decay=0.0, use_gpu=True)[source]

Initialize AffectAdam optimizer.

Args are the same as Adam.

Dependencies: None detected from callable globals.

Variables: params (collections.abc.Iterator[numpy.ndarray], required); lr (float, optional, default 0.001); betas (tuple, optional, default (0.9, 0.999)); eps (float, optional, default 1e-08); weight_decay (float, optional, default 0.0); use_gpu (bool, optional, default True).

Usage Example

import numpy as np
from grilly.optim.adam import AffectAdam

instance = AffectAdam(...)
result = instance.__init__(params=np.zeros(1, dtype=np.float32), lr=0.001, betas=(), eps=1e-08, weight_decay=0.0, use_gpu=True)
Parameters

  • params (Iterator[numpy.ndarray]) –

  • lr (float) –

  • betas (tuple) –

  • eps (float) –

  • weight_decay (float) –

  • use_gpu (bool) –

_adam_update_gpu(backend, param, grad, exp_avg, exp_avg_sq, lr, beta1, beta2, eps, beta1_t, beta2_t)[source]

GPU-accelerated AffectAdam update using affect-adam.glsl shader.

Dependencies: None detected from callable globals.

Variables: backend (Any, required); param (Any, required); grad (Any, required); exp_avg (Any, required); exp_avg_sq (Any, required); lr (Any, required); beta1 (Any, required); beta2 (Any, required); eps (Any, required); beta1_t (Any, required); beta2_t (Any, required).

Usage Example

from grilly.optim.adam import AffectAdam

instance = AffectAdam(...)
result = instance._adam_update_gpu(backend=None, param=None, grad=None, exp_avg=None, exp_avg_sq=None, lr=None, beta1=None, beta2=None, eps=None, beta1_t=None, beta2_t=None)
_get_backend()

Get or create backend instance

Dependencies: None detected from callable globals.

Variables: This callable does not take explicit input variables.

Usage Example

from grilly.optim.adam import Adam

instance = Adam(...)
result = instance._get_backend()
load_state_dict(state_dict)

Load optimizer state from state_dict.

Parameters

state_dict (dict[str, Any]) – Dictionary containing optimizer state

Dependencies: None detected from callable globals.

Variables: state_dict (dict[str, typing.Any], required).

Usage Example

from grilly.optim.base import Optimizer

instance = Optimizer(...)
result = instance.load_state_dict(state_dict='example')
state_dict()

Return the state of the optimizer as a dict.

Returns

Dictionary containing optimizer state

Return type

dict[str, Any]

Dependencies: None detected from callable globals.

Variables: This callable does not take explicit input variables.

Usage Example

from grilly.optim.base import Optimizer

instance = Optimizer(...)
result = instance.state_dict()
step(closure=None, gradients=None)

Perform a single optimization step.

Parameters

  • closure – Optional closure that reevaluates the model and returns loss

  • gradients – Optional dict mapping parameter IDs to gradients. If None, tries to get gradients from param.grad attribute.

Dependencies: numpy.

Variables: closure (Any, optional, default None); gradients (Any, optional, default None).

Usage Example

from grilly.optim.adam import Adam

instance = Adam(...)
result = instance.step(closure=None, gradients=None)
zero_grad()

Clear gradients for all parameters.

Note: In this implementation, gradients are expected to be stored in a separate structure (e.g., in the model’s backward pass). This method is provided for API compatibility.

Dependencies: None detected from callable globals.

Variables: This callable does not take explicit input variables.

Usage Example

from grilly.optim.base import Optimizer

instance = Optimizer(...)
result = instance.zero_grad()