Record · corpus.blog/posts/01a08793-a866-73bd-89f5-9e5ba15e8bbb

Learn CUTLASS the hard way - part 2!

kapilsh.github.io · published 31 December 2025

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at kapilsh.github.io Wayback capture

0blogs citing

First cited: —
Most recent: —
Rank this month: not ranked
Links from this post: 57
Words captured: 10,910

Cited by

No blog we hold has cited this post yet.

Links from this post

Link

Host

http://www.kapilsharma.dev/learn-cutlass-the-hard-way

kapilsharma.dev

Outperforming cuBLAS on H100: a Worklog Read at the source ↗

cudaforfun.substack.com

https://verda.com/products

verda.com

https://github.com/gpusgobrr/explore-gemm/tree/main/cuda

github.com

http://www.kapilsharma.dev/assets/explore_gemms_2_hopper_baseline_perf.png

kapilsharma.dev

http://www.kapilsharma.dev/assets/explore_gemms_2_hopper_baseline_autotuned.png

kapilsharma.dev

http://www.kapilsharma.dev/assets/explore_gemms_2_pranjals_result.png

kapilsharma.dev

https://developer-blogs.nvidia.com/wp-content/uploads/2022/03/Thread-Block-Clusters-and-Grids-with-Clusters.jpg

developer-blogs.nvidia.com

https://developer-blogs.nvidia.com/wp-content/uploads/2022/03/Thread-Block-to-Thread-Block-data-exchange-A100-vs-H100-with-Clusters-1536x331.jpg

developer-blogs.nvidia.com

https://developer.nvidia.com/blog/nvidia-hopper-architecture-in-depth/

developer.nvidia.com

https://developer-blogs.nvidia.com/wp-content/uploads/2022/03/Asynchronous-Memory-Copy-with-TMA-on-H100-vs-LDGSTS-Instruction-on-A100.jpg

developer-blogs.nvidia.com

http://www.kapilsharma.dev/assets/explore_gemms_2_tma_nvidia_presentation.png

kapilsharma.dev

https://www.nvidia.com/en-us/on-demand/session/gtcspring23-s51413/

nvidia.com

http://www.kapilsharma.dev/assets/explore_gemms_2_async_execution.jpg

kapilsharma.dev

http://www.kapilsharma.dev/assets/explore_gemms_2_async_barrier_a100vsh100.jpg

kapilsharma.dev

https://docs.nvidia.com/cuda/parallel-thread-execution/index.html

docs.nvidia.com

http://www.kapilsharma.dev/assets/explore_gemms_2_cutlass_2x_gemm.png

kapilsharma.dev

http://www.kapilsharma.dev/assets/explore_gemms_2_cutlass_3x_gemm.png

kapilsharma.dev

https://docs.nvidia.com/cutlass/latest/media/docs/cpp/gemm_api_3x.html

docs.nvidia.com

https://github.com/NVIDIA/cutlass/tree/main/examples/49_hopper_gemm_with_collective_builder

github.com

https://docs.nvidia.com/cuda/cuda-c-programming-guide/

docs.nvidia.com

https://github.com/gpusgobrr/explore-gemm/blob/main/cuda/15_kernel_cutlass_hopper.cu

github.com

http://www.kapilsharma.dev/assets/explore_gemm_2_basic_2_stage_tma_warp_specialized.png

kapilsharma.dev

http://www.kapilsharma.dev/assets/explore_gemms_2_stage_2_ncu_sol.png

kapilsharma.dev

http://www.kapilsharma.dev/assets/explore_gemms_basic_stage_auto_benchmark.png

kapilsharma.dev

http://www.kapilsharma.dev/assets/explore_gemms_2_ncu_sol_stage_auto.png

kapilsharma.dev

http://www.kapilsharma.dev/assets/explore_gemms_2_stage_auto_tma_ws_memory_chart.png

kapilsharma.dev

http://www.kapilsharma.dev/assets/explore_gemms_2_tbc_2_benchmark.png

kapilsharma.dev

https://github.com/NVIDIA/cutlass/blob/main/include/cutlass/kernel_hardware_info.hpp

github.com

https://github.com/NVIDIA/cutlass/blob/main/include/cutlass/gemm/kernel/sm90_tile_scheduler.hpp

github.com

http://www.kapilsharma.dev/assets/explore_gemms_2_persistent_cooperative_stage_5.png

kapilsharma.dev

http://www.kapilsharma.dev/assets/explore_gemms_2_pytorch_ping_pong_fp8_blog.png

kapilsharma.dev

http://www.kapilsharma.dev/assets/explore_gemms_pingpong_persistent_constant_results.png

kapilsharma.dev

CUTLASS Tutorial: Persistent Kernels and Stream-K Read at the source ↗

research.colfax-intl.com

http://www.kapilsharma.dev/assets/wave_quantization_colfax.png

kapilsharma.dev

http://www.kapilsharma.dev/assets/explore_gemms_2_streamk_constant_stages_3.png

kapilsharma.dev

https://docs.nvidia.com/cutlass/latest/media/docs/cpp/efficient_gemm.html

docs.nvidia.com

https://www.kapilsharma.dev/posts/learn-cutlass-the-hard-way/

kapilsharma.dev

https://docs.nvidia.com/cuda/parallel-thread-execution/

docs.nvidia.com

http://www.kapilsharma.dev/assets/swizzling_64_1.png

kapilsharma.dev

http://www.kapilsharma.dev/assets/swizzling_64_2.png

kapilsharma.dev

http://www.kapilsharma.dev/assets/swizzling_64_3.png

kapilsharma.dev

http://www.kapilsharma.dev/assets/swizzling_128_1.png

kapilsharma.dev

http://www.kapilsharma.dev/assets/swizzling_128_2.png

kapilsharma.dev

https://veitner.bearblog.dev/swizzles-and-their-usage-in-cutedsl-kernels/

veitner.bearblog.dev

https://leimao.github.io/blog/CuTe-Swizzle/

leimao.github.io

https://github.com/bertmaher/simplegemm

github.com

CUTLASS 3.x: Orthogonal, Reusable, and Composable Abstractions for GEMM Kernel Design Read at the source ↗

developer.nvidia.com

https://www.nvidia.com/en-us/on-demand/session/gtc25-s72685/

nvidia.com

https://docs.nvidia.com/cutlass/media/docs/cpp/efficient_gemm.html

docs.nvidia.com

https://rohany.github.io/blog/warp-specialization/

rohany.github.io

https://pytorch.org/blog/cutlass-ping-pong-gemm-kernel/

pytorch.org

https://github.com/NVIDIA/cutlass/issues/2181

github.com

https://arxiv.org/pdf/2301.03598

arxiv.org

https://docs.nvidia.com/deeplearning/performance/dl-performance-matrix-multiplication/index.html

docs.nvidia.com

https://github.com/ColfaxResearch/cfx-article-src

github.com

https://github.com/NVIDIA/cutlass/discussions/1345

github.com