Record · corpus.blog/posts/01a08723-5559-72a3-b5a7-b1e4d9f3853c

Inside NVIDIA GPUs: Anatomy of high performance matmul kernels

aleksagordic.com · published 29 September 2025

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at aleksagordic.com Wayback capture

5blogs citing

First cited: 1 Nov 2025
Most recent: 28 Aug 2026
Rank this month: not ranked
Links from this post: 34
Words captured: 17,906

Cited by

Blog

In the post

Date

iaroslavelistratov.github.io

B200 Attention Kernel from Scratch to Near-SOTA in 60 Diagrams Read at the source ↗

28 Aug 2026

hoanh.space

Writing Fast Attention Backward Kernel for 5090 in CUDA Read at the source ↗

23 Aug 2026

sequentialread.com

Usability, Community, and LLMs' Von Neumann Bottleneck Read at the source ↗

16 Jul 2026

hamzaelshafie.bearblog.dev

Dissecting ThunderKittens: Anatomy of a Compact DSL for High-Performance AI Kernels Read at the source ↗

21 May 2026

hamzaelshafie.bearblog.dev

Worklog: Optimising GEMM on NVIDIA H100 for cuBLAS-like Performance (WIP) Read at the source ↗

12 Jan 2026

kapilsh.github.io

Learn CUTLASS the hard way! Read at the source ↗

1 Nov 2025

Links from this post

Link

Host

https://developer.nvidia.com/blog/nvidia-hopper-architecture-in-depth/

developer.nvidia.com

https://developer.nvidia.com/blog/nvidia-ampere-architecture-in-depth/

developer.nvidia.com

https://x.com/gordic_aleksa

x.com

https://docs.google.com/forms/d/1z1fEirrN2xtGxAsJvptpM7yV4ByT5SF25S-XiMPrXNA/edit

docs.google.com

Strangely, Matrix Multiplications on GPUs Run Faster When Given "Predictable" Data! [short] Read at the source ↗

thonking.ai

https://feldmann.nyc/blog/smem-microbenchmarks

feldmann.nyc

https://docs.nvidia.com/cuda/cuda-binary-utilities/index.html

docs.nvidia.com

https://github.com/ademeure

github.com

https://arxiv.org/abs/1804.06826

arxiv.org

https://github.com/triton-lang/triton

github.com

How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance: a Worklog Read at the source ↗

siboehm.com

https://docs.nvidia.com/cuda/cuda-c-programming-guide/

docs.nvidia.com

https://godbolt.org/

godbolt.org

https://github.com/siboehm/SGEMM_CUDA/blob/master/src/kernels/10_kernel_warptiling.cuh

github.com

https://developer.nvidia.com/blog/cutlass-linear-algebra-cuda/

developer.nvidia.com

https://github.com/NVIDIA/cutlass/blob/b0e09d7cd371eded41f7c1e057caf1593c27ba55/media/docs/efficient_gemm.md

github.com

Outperforming cuBLAS on H100: a Worklog Read at the source ↗

cudaforfun.substack.com

https://pytorch.org/blog/cutlass-ping-pong-gemm-kernel/

pytorch.org

https://github.com/pranjalssh/fast.cu/blob/main/examples/matmul/matmul_2.cuh

github.com

https://github.com/pranjalssh/fast.cu/pull/8/files

github.com

https://docs.nvidia.com/cuda/cuda-driver-api/group__CUDA__TENSOR__MEMORY.html

docs.nvidia.com

https://github.com/NVIDIA/cutlass/blob/76c96b0be35cb263debe3e3d8418b80911a544ab/include/cute/swizzle.hpp

github.com

https://veitner.bearblog.dev/understanding-cute-swizzling-the-math-behind-32b-64b-and-128b-patterns/

veitner.bearblog.dev

https://docs.nvidia.com/cuda/parallel-thread-execution/index.html

docs.nvidia.com

https://docs.nvidia.com/cuda/inline-ptx-assembly/

docs.nvidia.com

https://github.com/pranjalssh/fast.cu/pull/1

github.com

https://www.hyperstack.cloud/

hyperstack.cloud

https://x.com/marksaroufim

x.com

https://www.youtube.com/watch?v=F_BazucyCMw&ab_channel=GPUMODE

youtube.com

https://github.com/siboehm/SGEMM_CUDA/

github.com

https://www.nvidia.com/en-us/on-demand/session/gtcfall22-a41101/

nvidia.com

https://www.youtube.com/watch?v=we3i5VuoPWk

youtube.com

https://github.com/pranjalssh/fast.cu/

github.com

https://upcommons.upc.edu/server/api/core/bitstreams/b843de39-f32f-4069-8843-48f74c030213/content

upcommons.upc.edu