Flash Attention Explained
A practical walkthrough of how Flash Attention reduces memory traffic and speeds up transformer training.
Blog
Essays and notes on AI, GPUs, and software I am building.
GPU Kernels — First Notes
Notes from learning CUDA memory hierarchy, occupancy, and writing my first custom kernels.