Record · corpus.blog/posts/01a0e6f6-5311-730a-b65e-d71f807bc468

Strangely, Matrix Multiplications on GPUs Run Faster When Given "Predictable" Data! [short]

thonking.ai · published 29 April 2024

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at thonking.ai Wayback capture

8blogs citing

First cited: 17 Jun 2024
Most recent: 21 May 2026
Rank this month: not ranked
Links from this post: 19
Words captured: 1,832

Cited by

Blog

In the post

Date

hamzaelshafie.bearblog.dev

Dissecting ThunderKittens: Anatomy of a Compact DSL for High-Performance AI Kernels Read at the source ↗

21 May 2026

bluenotebook.io

Which H100 instance to train Nanochat Read at the source ↗

4 Mar 2026

hamzaelshafie.bearblog.dev

Worklog: Optimising GEMM on NVIDIA H100 for cuBLAS-like Performance (WIP) Read at the source ↗

12 Jan 2026

aleksagordic.com

Inside NVIDIA GPUs: Anatomy of high performance matmul kernels Read at the source ↗

29 Sept 2025

main-horse.github.io

Things I learned digging into 5090 perf Read at the source ↗

11 Apr 2025

spatters.ca

Improving FP16/16 matmul accuracy with two-stage accumulation Read at the source ↗

31 Mar 2025

main-horse.github.io

Testing the 4090 48GB Read at the source ↗

24 Dec 2024

cudaforfun.substack.com

Outperforming cuBLAS on H100: a Worklog Read at the source ↗

29 Nov 2024

alexarmbr.github.io

How To Write A Fast Matrix Multiplication From Scratch With Tensor Cores Read at the source ↗

10 Aug 2024

SemiAnalysis

100,000 H100 Clusters: Power, Network Topology, Ethernet vs InfiniBand, Reliability, Failures, Checkpointing Read at the source ↗

17 Jun 2024

Links from this post

Link

Host

https://github.com/NVIDIA/cutlass

github.com

https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html

docs.nvidia.com

https://substackcdn.com/image/fetch/$s_!q_rq!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F75646311-1680-4c0c-b35e-d7736683e34f_1397x1132.bin

substackcdn.com

https://substackcdn.com/image/fetch/$s_!J_3Q!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F23857db1-9818-4b69-ab6f-0a799563dab8_1614x464.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!B0kO!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3f209576-4ea4-4de7-918f-c146e99a3065_1662x496.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!D6Kx!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8379a7fe-2edc-4030-97c1-b81c919a45ef_499x187.jpeg

substackcdn.com

https://en.wikipedia.org/wiki/Power_gating

en.wikipedia.org

https://substackcdn.com/image/fetch/$s_!sxRm!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4e6764f5-44d9-4cc9-a32b-70c56647a81b_499x187.jpeg

substackcdn.com

https://substackcdn.com/image/fetch/$s_!2cYi!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F112e6999-afc8-47c6-8cc9-c81aa145ab98_1392x844.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!CgZB!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe5effdde-4b30-4ebb-a910-537c459d8e6b_1387x947.bin

substackcdn.com

https://www.thonking.ai/p/strangely-matrix-multiplications/comment/55271443

thonking.ai

https://substackcdn.com/image/fetch/$s_!-nnn!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0441dd3d-137c-410d-bba2-f081e70300b1_1697x947.bin

substackcdn.com

https://substackcdn.com/image/fetch/$s_!8wPi!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F882c3bb6-610b-41f0-a134-97b6780f781d_1418x947.bin

substackcdn.com

https://resources.nvidia.com/en-us-tensor-core

resources.nvidia.com

https://substackcdn.com/image/fetch/$s_!m2rl!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F85158cd4-ff77-4420-a60b-6bb639d69309_1522x947.bin

substackcdn.com

https://twitter.com/dwarkesh_sp/status/1780990840179187715

twitter.com

NVIDIA Sets New Generative AI Performance and Scale Records in MLPerf Training v4.0 Read at the source ↗

developer.nvidia.com

https://manifold.markets/chilli/what-will-be-the-maximum-achievable

manifold.markets

https://substackcdn.com/image/fetch/$s_!dKL3!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6037e109-8dbf-4d8e-9570-5c8dc23eefc5_1210x438.png

substackcdn.com