#PyTorch

6 articles tagged with PyTorch

State Space Models in 2026: Mamba-2 State Space Duality vs Transformer Attention

An authoritative mathematical and systems analysis of State Space Models. We examine Mamba-2 State Space Duality (SSD), Structured State Space (S4) recurrence, linear-time $O(N)$ inference scaling, and hybrid Mamba-Transformer production architectures.

9/2/202624 min read

Machine Learning Explained: Studying With Millions of Flashcards & Gradient Descent (2026)

9/12/202615 min read

Neural Networks Explained: Layers of Voting Neurons and Activation Functions (2026)

9/12/202615 min read

Context Window Extension: RoPE Scaling, YaRN, and LongRoPE Frequency Domain Mathematics (2026)

9/12/20265 min read

DeepSeek-V3 Multi-Head Latent Attention (MLA): Low-Rank KV Compression & 93% VRAM Reduction (2026)

9/12/20265 min read

Mixture-of-Depths (MoD) & Dynamic Token Routing: 50% FLOP Reduction in Transformer Compute (2026)

9/12/20265 min read