Writing
Deep dives, notes, and opinions. Mostly machine learning so far, but not only. Also available as an RSS feed.
2026
- SFT vs SFT + DPO: A Comparison
Comparing supervised fine-tuning alone versus combining it with Direct Preference Optimization for LLM alignment.
2025
GPUs for Machine Learning · Part 6
Multi-GPU Training, Part 2: Fully Sharded Data ParallelismHow FSDP splits weights, gradients, and optimizer states across GPUs so models too big for one GPU can still be trained.
GPUs for Machine Learning · Part 5
Multi-GPU Training, Part 1: Data ParallelismHow data parallel training splits batches across GPUs, keeps gradients in sync, and why it matches training on one big GPU.
GPUs for Machine Learning · Part 4
Streaming Multiprocessors: Scheduling and ExecutionWhat's inside a streaming multiprocessor, and how it keeps thousands of threads busy at once.
GPUs for Machine Learning · Part 3
The GPU Memory Hierarchy: L2, L1, and RegistersHow the L2 cache, L1 and shared memory, and registers bring data closer to the cores, and why that matters for fast kernels.
GPUs for Machine Learning · Part 2
High Bandwidth Memory (HBM): Why GPUs Need It for Machine LearningWhy large models depend on HBM, how it works, and what its capacity and bandwidth numbers actually mean.
GPUs for Machine Learning · Part 1
How GPUs Power Modern Machine Learning: An Introduction to GPU ArchitectureA tour of what makes GPUs good at machine learning: memory, compute, and the thread model that ties them together.
- Hierarchical Text-Conditional Image Generation with CLIP Latents
Breaks down unCLIP’s diffusion prior, decoder, and editing workflows for high-fidelity text-to-image synthesis.
- Variational Autoencoders
Covers evidence lower bound derivations and the mechanics of VAEs for generative modeling.
- Byte Pair Encoding
Walkthrough of the BPE tokenization algorithm that powers modern language models.
- Zero-Shot Text-to-Image Generation
Analyzes early text-to-image transformers, discrete VAEs, and zero-shot generation techniques.
- Learning Transferable Visual Models From Natural Language Supervision
A deep dive into CLIP’s contrastive training on 400M image–text pairs and its zero-shot recognition capabilities.