How warp schedulers, tensor cores, and instruction pipelines inside an H100 SM keep massive thread counts in flight.
GPU ArchitectureLarge-Scale Distributed TrainingCompute ArchitectureResearch Deep Dive
Writing
Essays, paper deep dives, and architecture notes. Use the topic filters to jump to what interests you.
A tour of caching, shared memory, and register files on modern accelerators, with tips for keeping tensor cores fed.
GPU ArchitectureLarge-Scale Distributed TrainingMemory Systems
Explains why large-scale training depends on terabyte-per-second HBM stacks and how to budget their bandwidth.
GPU ArchitectureLarge-Scale Distributed TrainingMemory Systems
Sets the stage for the GPU architecture series—compute, memory, and interconnect pillars for large-scale training.
GPU ArchitectureLarge-Scale Distributed Training
Breaks down unCLIP’s diffusion prior, decoder, and editing workflows for high-fidelity text-to-image synthesis.
Computer VisionDiffusion ModelsResearch Deep DiveGenerative Models
Covers evidence lower bound derivations and the mechanics of VAEs for generative modeling.
Generative ModelsResearch Deep Dive
Walkthrough of the BPE tokenization algorithm that powers modern language models.
Natural Language ProcessingML Concepts
Analyzes early text-to-image transformers, discrete VAEs, and zero-shot generation techniques.
Computer VisionDiffusion ModelsResearch Deep DiveGenerative Models
A deep dive into CLIP’s contrastive training on 400M image–text pairs and its zero-shot recognition capabilities.
Computer VisionResearch Deep Dive