grilly documentation

Grilly is a PyTorch-like framework powered by Vulkan compute shaders. These docs are organized as a progressive learning path so you can move from first install to end-to-end workflows quickly.

Getting Started

• Getting Started
  • Installation
    • Step 1: Install GPU drivers
    • Step 2: Create a Python environment
    • Step 3: Install Grilly
    • Step 4: Verify your setup
    • Step 5: Optional environment flags
  • Quickstart
    • Step 1: Import and initialize
    • Step 2: Run a feedforward operation
    • Step 3: Run a neural module (PyTorch-style)
    • Step 4: Use a simple training step
    • Step 5: Update weights
    • Step 6 (Optional): Use autograd Variables
  • Core Concepts
    • 1. Compute backend
    • 2. Module API
    • 3. Functional API
    • 4. Backward and gradients
    • 5. CPU fallback behavior
  • For PyTorch Users
    • Concept mapping
    • Workflow mapping
    • Parameter handling
    • Tensor type expectations

Concepts

• Concepts
  • Architecture and Runtime
    • What Grilly is
    • Core runtime objects
    • Execution flow
    • CPU fallback model
    • Design choices
    • Resource lifecycle
    • How the high-level APIs fit
  • Design Choices
    • 1. Vulkan-first backend
    • 2. PyTorch-like UX with explicit internals
    • 3. CPU fallback paths
    • 4. NumPy as primary tensor interchange
    • 5. Specialized subsystems in one framework
    • 6. Determinism tools for experimental pipelines
  • Tensor Model and Shapes
    • Primary tensor type
    • Shape conventions
    • Why shape discipline matters
    • Example input guards
    • Parameter and gradient storage
  • Module System and Functional API
    • Two ways to build models
    • grilly.nn overview
    • grilly.functional overview
    • When to choose each API
    • Minimal composition example
    • Functional example
  • Training and Optimization
    • Training model in Grilly
    • Canonical loop
    • Optimizers
    • Gradient lifecycle
    • Numerical and runtime tips
    • Related advanced learning ops
  • Attention, Transformers, and Decoding
    • Attention stack
    • Why Flash Attention 2 matters
    • Transformer-facing modules
    • Basic attention example
    • Backend flash attention example
    • Decoding usage
  • Convolution, Pooling, and Normalization
    • What is covered
    • Data layout
    • Backward support
    • Simple CNN block
    • Design notes
  • Spiking Neural Networks
    • SNN support in Grilly
    • LIF neuron model
    • Design choices
    • High-level SNN pipeline
    • Example: high-level usage
    • Example: module usage
    • Practical guidance
  • Memory, Retrieval, and FAISS-style Search
    • Retrieval primitives
    • FAISS-style APIs
    • Memory APIs
    • Example search flow
    • Use cases
  • Multimodal, Capsule, and VLM Systems
    • Multimodal module families
    • Capsule-related components
    • When to use which approach
    • Design choices
    • Example: simple multimodal fusion
    • Design considerations
  • LoRA and Adapter Fine-Tuning
    • Why LoRA
    • Core classes
    • Basic LoRALinear flow
    • Managing inference overhead
    • Checkpointing adapters
  • Experimental Language and Cognition
    • Scope
    • Primary components
    • Instant language model
    • Cognitive controller
    • Minimal ingestion example
    • Operational guidance
  • Stable Hashing and Ingestion Checkpoints
    • Why stable hashing exists
    • Stable hash utilities
    • Design choices
    • Ingestion checkpoint system
    • Checkpoint flow example
    • When to use it
  • Specialized Operations
    • Signal and spectral ops
    • Neuroscience-inspired cells
    • Bridge and conversion ops
    • Domain routing and affective ops
    • Embedding and lookup operations
    • Design choices
    • Practical recommendation
  • Compatibility and Integrations
    • PyTorch-style compatibility layer
    • Device management
    • Tensor conversion helpers
    • HuggingFace bridge
    • When to use integrations
  • Performance, Debugging, and Testing
    • Performance model
    • Design choices
    • Profiling strategy
    • Debugging checklist
    • Testing workflow
    • Reproducibility tips

Tutorials

• Tutorials
  • Tutorial 01: Compute API Basics
    • Step 1: Initialize compute
    • Step 2: Run FNN ops
    • Step 3: Run attention ops
    • Step 4: Run memory search
    • Step 5: Cleanup
  • Tutorial 02: Build Your First Network
    • Step 1: Define the model
    • Step 2: Create synthetic data
    • Step 3: Forward pass
    • Step 4: Compute a basic loss
    • Step 5: Build output gradient
  • Tutorial 03: End-to-End Training Loop
    • Step 1: Setup model and optimizer
    • Step 2: Create training batch
    • Step 3: Forward
    • Step 4: Backward
    • Step 5: Parameter update
    • Step 6: Repeat over epochs
  • Tutorial 04: Attention and Memory Workflow
    • Step 1: Prepare attention inputs
    • Step 2: Compute attention output
    • Step 3: Build memory database
    • Step 4: Retrieve nearest vectors
    • Step 5: Use retrieved context
    • Step 6: Cleanup
  • Tutorial 05: Experimental Language Ingestion
    • Step 1: Create SVC entries
    • Step 2: Initialize InstantLanguage
    • Step 3: Ingest entries
    • Step 4: Query similar sentences
    • Step 5: Integrate with controller
  • Tutorial 06: Convolution Pipeline
    • Step 1: Create modules
    • Step 2: Prepare image batch
    • Step 3: Run feature extraction block
    • Step 4: Attach a classifier head
    • Step 5: Extend to training
  • Tutorial 07: Spiking Neurons and Activity
    • Step 1: High-level SNNCompute path
    • Step 2: Module-level LIF neuron
    • Step 3: Run a small time simulation
    • Step 4: Add synaptic learning layers
  • Tutorial 08: Vector Search and Routing
    • Step 1: Initialize backend and vectors
    • Step 2: Compute distances
    • Step 3: Get top-k neighbors
    • Step 4: Build retrieval context
    • Step 5: Optional symbolic routing with bipolar keys
  • Tutorial 09: LoRA Adaptation Workflow
    • Step 1: Create a LoRA config
    • Step 2: Build a LoRA layer
    • Step 3: Run forward
    • Step 4: Merge for inference
    • Step 5: Manage multi-layer adapters
  • Tutorial 10: Multimodal Fusion
    • Step 1: Prepare modality tensors
    • Step 2: Bottleneck fusion
    • Step 3: Cross-modal attention fusion
    • Step 4: Perceiver IO on variable input
    • Step 5: Expand to full VLM stack
  • Tutorial 11: Ingestion Checkpoints and Reproducibility
    • Step 1: Build a controller
    • Step 2: Ingest a tiny SVC batch
    • Step 3: Save checkpoint
    • Step 4: Inspect without full restore
    • Step 5: Load into a fresh controller
    • Step 6: Stable hash usage
  • Tutorial 12: Cognitive Controller End-to-End
    • Step 1: Initialize controller
    • Step 2: Add baseline world knowledge
    • Step 3: Ingest structured language data
    • Step 4: Understand input
    • Step 5: Generate a response
    • Step 6: Inspect reasoning trace

API Reference

• API Reference
  • grilly.datasets
    • grilly.datasets.clean_conversations
    • grilly.datasets.merge_svc
    • grilly.datasets.validate_svc
  • grilly.experimental
  • grilly.optim
    • Optimizer
    • Adam
    • AdamW
    • AffectAdam
    • SGD
    • NLMS
    • NaturalGradient
    • HypergradientAdamW
    • AutoHypergradientAdamW
    • LRScheduler
    • StepLR
    • CosineAnnealingLR
    • ReduceLROnPlateau
    • OneCycleLR
    • grilly.optim.adam
    • grilly.optim.adamw
    • grilly.optim.base
    • grilly.optim.hypergradient
    • grilly.optim.lr_scheduler
    • grilly.optim.natural_gradient
    • grilly.optim.nlms
    • grilly.optim.sgd
  • grilly.scripts
    • grilly.scripts.ingest_svc
    • grilly.scripts.install_pyvma
  • grilly.datasets
    • grilly.datasets.clean_conversations
    • grilly.datasets.merge_svc
    • grilly.datasets.validate_svc
  • grilly.experimental
  • grilly.optim
    • Optimizer
    • Adam
    • AdamW
    • AffectAdam
    • SGD
    • NLMS
    • NaturalGradient
    • HypergradientAdamW
    • AutoHypergradientAdamW
    • LRScheduler
    • StepLR
    • CosineAnnealingLR
    • ReduceLROnPlateau
    • OneCycleLR
    • grilly.optim.adam
    • grilly.optim.adamw
    • grilly.optim.base
    • grilly.optim.hypergradient
    • grilly.optim.lr_scheduler
    • grilly.optim.natural_gradient
    • grilly.optim.nlms
    • grilly.optim.sgd
  • grilly.scripts
    • grilly.scripts.ingest_svc
    • grilly.scripts.install_pyvma