grilly.optim.nlms

NLMS (Normalized Least Mean Squares) Optimizer

Uses: nlms-update.glsl

Reference: ref/brain/specialist.py NLMSExpertHead

Classes

Iterator()

NLMS(params[, lr, lr_decay, lr_min, eps, ...])

NLMS (Normalized Least Mean Squares) optimizer.

Optimizer(params, defaults)

Base class for all optimizers.
class grilly.optim.nlms.NLMS(params, lr=0.5, lr_decay=0.99995, lr_min=0.1, eps=1e-06, use_gpu=True)[source]

Bases: Optimizer

NLMS (Normalized Least Mean Squares) optimizer.

Uses: nlms-update.glsl

Implements adaptive filtering with normalized learning rate: - w = w + mu * error * x / (||x||^2 + eps)

Reference: ref/brain/specialist.py NLMSExpertHead

Initialize NLMS optimizer.

Parameters

  • params (Iterator[numpy.ndarray]) – Iterator of parameter arrays to optimize

  • lr (float) – Initial learning rate (mu) (default: 0.5)

  • lr_decay (float) – Learning rate decay factor (default: 0.99995)

  • lr_min (float) – Minimum learning rate (default: 0.1)

  • eps (float) – Small constant for numerical stability (default: 1e-6)

  • use_gpu (bool) – Whether to use GPU acceleration (default: True)

__init__(params, lr=0.5, lr_decay=0.99995, lr_min=0.1, eps=1e-06, use_gpu=True)[source]

Initialize NLMS optimizer.

Parameters

  • params (Iterator[numpy.ndarray]) – Iterator of parameter arrays to optimize

  • lr (float) – Initial learning rate (mu) (default: 0.5)

  • lr_decay (float) – Learning rate decay factor (default: 0.99995)

  • lr_min (float) – Minimum learning rate (default: 0.1)

  • eps (float) – Small constant for numerical stability (default: 1e-6)

  • use_gpu (bool) – Whether to use GPU acceleration (default: True)

Dependencies: None detected from callable globals.

Variables: params (collections.abc.Iterator[numpy.ndarray], required); lr (float, optional, default 0.5); lr_decay (float, optional, default 0.99995); lr_min (float, optional, default 0.1); eps (float, optional, default 1e-06); use_gpu (bool, optional, default True).

Usage Example

import numpy as np
from grilly.optim.nlms import NLMS

instance = NLMS(...)
result = instance.__init__(params=np.zeros(1, dtype=np.float32), lr=0.5, lr_decay=0.99995, lr_min=0.1, eps=1e-06, use_gpu=True)
_get_backend()[source]

Get or create backend instance

Dependencies: None detected from callable globals.

Variables: This callable does not take explicit input variables.

Usage Example

from grilly.optim.nlms import NLMS

instance = NLMS(...)
result = instance._get_backend()
step(closure=None)[source]

Perform a single optimization step.

Parameters

closure – Optional closure that reevaluates the model and returns loss

Dependencies: numpy.

Variables: closure (Any, optional, default None).

Usage Example

from grilly.optim.nlms import NLMS

instance = NLMS(...)
result = instance.step(closure=None)
load_state_dict(state_dict)

Load optimizer state from state_dict.

Parameters

state_dict (dict[str, Any]) – Dictionary containing optimizer state

Dependencies: None detected from callable globals.

Variables: state_dict (dict[str, typing.Any], required).

Usage Example

from grilly.optim.base import Optimizer

instance = Optimizer(...)
result = instance.load_state_dict(state_dict='example')
state_dict()

Return the state of the optimizer as a dict.

Returns

Dictionary containing optimizer state

Return type

dict[str, Any]

Dependencies: None detected from callable globals.

Variables: This callable does not take explicit input variables.

Usage Example

from grilly.optim.base import Optimizer

instance = Optimizer(...)
result = instance.state_dict()
zero_grad()

Clear gradients for all parameters.

Note: In this implementation, gradients are expected to be stored in a separate structure (e.g., in the model’s backward pass). This method is provided for API compatibility.

Dependencies: None detected from callable globals.

Variables: This callable does not take explicit input variables.

Usage Example

from grilly.optim.base import Optimizer

instance = Optimizer(...)
result = instance.zero_grad()