Ludwig - ai/optimization

Deriving Muon

Added on August 12, 2025 · 14 min read

Yann LeCun "Mathematical Obstacles on the Way to Human-Level AI"

Added on July 22, 2025 · 56:23 · 117.5K views

CUDA Mode Keynote | Andrej Karpathy | Eureka Labs

Added on July 22, 2025 · 23:21 · 37.7K views

Scaling Computing Performance Beyond the End of Moore’s Law: Song Han

Added on July 22, 2025 · 31:52 · 3.3K views

Luminal - Search-Based Deep Learning Compilers

Added on July 22, 2025 · 69:29 · 1.0K views

Fine-tune LLMs - Line by line code example

Added on July 22, 2025 · 8:20 · 4.7K views

Hands-on Jupyter notebook walks line-by-line through performing LoRA fine-tuning of a large language model using HuggingFace PEFT.

George Hotz | Programming | rewriting linearizer (tinygrad) | Day In The Life Of A Software Engineer

Added on July 22, 2025 · 106:04 · 32.7K views

George Hotz refactors tinygrad’s linearizer, exposing low-level tensor compiler optimizations that map high-level ops to efficient GPU kernels.

The Most Important Algorithm in Machine Learning

Added on July 22, 2025 · 40:08 · 767.5K views

Adam with Aggressive Gradient Clipping ≈ Smoothed SignSGD/NormSGD

Added on July 21, 2025 · 6 min read

FlexAttention: The Flexibility of PyTorch with the Performance of FlashAttention

Added on July 2, 2025 · 19 min read

Muon and a Selective Survey on Steepest Descent in Riemannian and Non-Riemannian Manifolds

Added on June 27, 2025 · 41 min read

How To Scale

Added on May 13, 2025 · 53 min read

Transformers Meet Small Datasets

Added on April 10, 2024 · 1 min read

Parameter-Efficient Sparsity Crafting from Dense to Mixture-of-Experts for Instruction Tuning on General Tasks

Added on March 6, 2024 · 34 min read

Understanding The Exploding and Vanishing Gradients Problem

Added on January 7, 2024 · 5 min read

An overview of gradient descent optimization algorithms

Added on January 5, 2024 · 28 min read

An overview of gradient descent optimization algorithms∗