corpus.blog
56,966 blogs · 4,417,066 posts Claim your blog
Record · corpus.blog/posts/01a08783-a669-702b-a454-8fcf07a91e81

Writing Fast Attention Backward Kernel for 5090 in CUDA

hoanh.space · published 23 August 2026

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at hoanh.space Wayback capture
0blogs citing
First cited
—
Most recent
—
Rank this month
not ranked
Links from this post
17
Words captured
16,586

Cited by

No blog we hold has cited this post yet.

Links from this post

Link
Host
How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance: a Worklog Read at the source ↗
siboehm.com
Outperforming cuBLAS on H100: a Worklog Read at the source ↗
cudaforfun.substack.com
Writing Speed-of-Light Flash Attention for 5090 in CUDA C++ Read at the source ↗
gau-nernst.github.io
https://images.nvidia.com/aem-dam/Solutions/geforce/blackwell/nvidia-rtx-blackwell-gpu-architecture.pdf
images.nvidia.com
https://arxiv.org/abs/2307.08691
arxiv.org
https://docs.nvidia.com/cuda/cuda-programming-guide/05-appendices/compute-capabilities.html
docs.nvidia.com
https://docs.nvidia.com/cuda/cuda-programming-guide/01-introduction/programming-model.html
docs.nvidia.com
https://nvlabs.github.io/cuda-oxide/advanced/tensor-memory-accelerator.html
nvlabs.github.io
https://docs.nvidia.com/cuda/parallel-thread-execution/index.html
docs.nvidia.com
https://docs.nvidia.com/cuda/archive/12.9.0/cuda-driver-api/group__CUDA__TENSOR__MEMORY.html
docs.nvidia.com
https://forums.developer.nvidia.com/t/maximum-tensor-core-utilization/326341
forums.developer.nvidia.com
https://github.com/hoanhle/kernels
github.com
https://www.goodreads.com/en/book/show/7659954-programming-massively-parallel-processors
goodreads.com
Inside NVIDIA GPUs: Anatomy of high performance matmul kernels Read at the source ↗
aleksagordic.com
https://github.com/gau-nernst/learn-cuda
github.com
https://github.com/Dao-AILab/flash-attention
github.com
https://docs.nvidia.com/cuda/parallel-thread-execution/
docs.nvidia.com