corpus.blog
56,966 blogs · 4,417,066 posts Claim your blog
Record · corpus.blog/posts/01a08781-08dd-7069-b750-e37b79537b43

Dissecting ThunderKittens: Anatomy of a Compact DSL for High-Performance AI Kernels

hamzaelshafie.bearblog.dev · published 21 May 2026

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at hamzaelshafie.bearblog.dev Wayback capture
0blogs citing
First cited
—
Most recent
—
Rank this month
not ranked
Links from this post
53
Words captured
25,274

Cited by

No blog we hold has cited this post yet.

Links from this post

Link
Host
https://www.youtube.com/watch?v=5qSN-R_E3w0
youtube.com
https://pytorch.org/get-started/pytorch-2-x/
pytorch.org
https://github.com/HazyResearch/ThunderKittens/tree/main
github.com
https://hazyresearch.stanford.edu
hazyresearch.stanford.edu
Worklog: Optimising GEMM on NVIDIA H100 for cuBLAS-like Performance (WIP) Read at the source ↗
hamzaelshafie.bearblog.dev
https://hazyresearch.stanford.edu/blog/2024-11-27-tk-fp8
hazyresearch.stanford.edu
https://docs.nvidia.com/cuda/parallel-thread-execution/index.html?highlight=wgmma
docs.nvidia.com
https://github.com/HamzaElshafie/ThunderKittens/commit/01cb68cd9c0693eb56e6e21fd7e0413461a1648e
github.com
https://github.com/HazyResearch/ThunderKittens/blob/6bb12f80a854054aca3e951503964951baa9cbf5/include/ops/warp/register/tile/conversions.cuh
github.com
Inside NVIDIA GPUs: Anatomy of high performance matmul kernels Read at the source ↗
aleksagordic.com
https://github.com/HazyResearch/ThunderKittens/blob/main/include/types/global/tma.cuh
github.com
https://github.com/HamzaElshafie/CUDA_Kernels/blob/main/TK_layernorm/tk_layernorm.cu
github.com
https://github.com/HamzaElshafie/CUDA_Kernels/blob/main/TK_kernels/TK_layernorm/tk_layernorm.cu
github.com
https://docs.nvidia.com/cuda/cuda-c-programming-guide/
docs.nvidia.com
https://docs.nvidia.com/cuda/parallel-thread-execution/
docs.nvidia.com
https://hazyresearch.stanford.edu/blog/2025-03-15-tk-blackwell
hazyresearch.stanford.edu
https://newsletter.semianalysis.com/p/dissecting-nvidia-blackwell-tensor
newsletter.semianalysis.com
https://docs.nvidia.com/cuda/parallel-thread-execution/index.html?spm=a2ty_o01.29997173.0.0.883dc921B6BCcR
docs.nvidia.com
https://github.com/HazyResearch/ThunderKittens/blob/41f4c2a7e4246911e4ed2b7ced8ea13bfd295e7f/include/common/util.cuh
github.com
https://docs.nvidia.com/cuda/cuda-driver-api/group__CUDA__TENSOR__MEMORY.html
docs.nvidia.com
https://modal.com/gpu-glossary/perf/arithmetic-bandwidth
modal.com
https://docs.nvidia.com/cuda/hopper-tuning-guide/index.html
docs.nvidia.com
https://www.modular.com/matrix-multiplication-on-blackwell
modular.com
https://docs.nvidia.com/cutlass/4.3.4/media/docs/cpp/blackwell_functionality.html
docs.nvidia.com
https://github.com/HazyResearch/ThunderKittens/blob/main/kernels/gemm/bf16_b200/bf16_b200_gemm.cu
github.com
https://docs.nvidia.com/cuda/parallel-thread-execution/index.html
docs.nvidia.com
CUTLASS Tutorial: Writing GEMM Kernels Using Tensor Memory For NVIDIA® Blackwell GPUs Read at the source ↗
research.colfax-intl.com
tcgen05 for dummies Read at the source ↗
gau-nernst.github.io
https://github.com/HamzaElshafie/CUDA_Kernels/blob/main/TK_kernels/TK_GEMMs/h100_bf16_gemm.cu
github.com
https://github.com/HazyResearch/ThunderKittens/blob/main/kernels/gemm/bf16_h100/bf16_h100_gemm.cu
github.com
https://docs.nvidia.com/cutlass/latest/media/docs/cpp/blackwell_cluster_launch_control.html
docs.nvidia.com
https://pytorch.org/blog/enabling-cluster-launch-control-with-tlx/
pytorch.org
https://arxiv.org/pdf/2407.08608
arxiv.org
Strangely, Matrix Multiplications on GPUs Run Faster When Given "Predictable" Data! [short] Read at the source ↗
thonking.ai
https://benjaminfspector.com
benjaminfspector.com
https://www.aaryan-singhal.com
aaryan-singhal.com
https://arorasimran.com
arorasimran.com
https://cs.stanford.edu/~chrismre/
cs.stanford.edu
https://stuartsul.com
stuartsul.com
https://nashbrown.github.io
nashbrown.github.io
https://arxiv.org/abs/2410.20399
arxiv.org
https://hazyresearch.stanford.edu/blog/2024-05-12-tk
hazyresearch.stanford.edu
https://www.youtube.com/watch?v=Cl2B_hmg4gA&t=438s
youtube.com
https://arxiv.org/abs/2402.13499
arxiv.org
https://zhuanlan.zhihu.com/p/1923380660795598706
zhuanlan.zhihu.com
https://docs.nvidia.com/cuda/parallel-thread-execution/contents.html
docs.nvidia.com
CUTLASS Tutorial: Mastering the NVIDIA® Tensor Memory Accelerator (TMA) Read at the source ↗
research.colfax-intl.com
CUTLASS Tutorial: GEMM with Thread Block Clusters on NVIDIA® Blackwell GPUs Read at the source ↗
research.colfax-intl.com
Outperforming cuBLAS on H100: a Worklog Read at the source ↗
cudaforfun.substack.com
CUTLASS Tutorial: Persistent Kernels and Stream-K Read at the source ↗
research.colfax-intl.com
Dynamic persistent tile scheduling with Cluster Launch Control (CLC) on NVIDIA Blackwell GPUs Read at the source ↗
research.colfax-intl.com
https://docs.nvidia.com/cuda/cuda-programming-guide/04-special-topics/cluster-launch-control.html
docs.nvidia.com
https://hazyresearch.stanford.edu/blog/2026-02-19-tk-2
hazyresearch.stanford.edu