Record · corpus.blog/posts/01a0cabe-f4d5-71c5-bcf7-abe7f476ee3e

MI300X vs H100 vs H200 Benchmark Part 1: Training – CUDA Moat Still Alive

SemiAnalysis · published 22 December 2024

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at semianalysis.com Wayback capture

3blogs citing

First cited: 24 Dec 2024
Most recent: 21 Apr 2025
Rank this month: not ranked
Links from this post: 28
Words captured: 10,185

Cited by

Blog

In the post

Date

severelytheoretical.wordpress.com

Continual training of Llama-3.1-8B for 809B tokens Read at the source ↗

21 Apr 2025

main-horse.github.io

Various approaches to parallelizing Muon Read at the source ↗

22 Feb 2025

osmarks.net

Scaling meme search to 230 million images Read at the source ↗

24 Jan 2025

main-horse.github.io

Testing the 4090 48GB Read at the source ↗

24 Dec 2024

Links from this post

Link

Host

https://crusoe.ai/cloud

crusoe.ai

https://tensorwave.com/

tensorwave.com

https://nebius.com/

nebius.com

https://lambdalabs.com/

lambdalabs.com

https://hotaisle.xyz/

hotaisle.xyz

https://smc.co/

smc.co

https://firmus.co/

firmus.co

https://mlcommons.org/benchmarks/training/

mlcommons.org

https://www.semianalysis.com/p/ai-cloud-tco-model

semianalysis.com

https://www.semianalysis.com/p/accelerator-model

semianalysis.com

https://semianalysis.com/accelerator-industry-model/

semianalysis.com

https://github.com/pytorch-labs/float8_experimental/blob/fe6e08c867abf56b1acd0f34473c69cde624f0a3/benchmarks/bench_matmul.py

github.com

Neural Network Quantization & Number Formats From First Principles Read at the source ↗

semianalysis.com

https://www.ray.so/

ray.so

https://ray.so/

ray.so

https://pytorch.org/docs/stable/generated/torch.nn.attention.sdpa_kernel.html

pytorch.org

https://rocm.blogs.amd.com/artificial-intelligence/flash-attention/README.html

rocm.blogs.amd.com

https://pytorch.org/tutorials/intermediate/torch_compile_tutorial.html

pytorch.org

https://github.com/ROCm/hipBLASLt/pull/1378

github.com

https://github.com/pytorch/pytorch/issues/139116

github.com

https://github.com/pytorch/pytorch/pull/139137

github.com

https://github.com/pytorch/pytorch/pull/143507

github.com

https://github.com/pytorch/pytorch/pull/140673

github.com

https://www.torch-ci.com/failure?failureCaptures=%5B%22test_linalg.py%3A%3ATestLinalgCUDA%3A%3Atest_matmul_small_brute_force_tunableop_cuda_float16%22%5D

torch-ci.com

https://github.com/mlcommons/training_results_v4.1/blob/b87b9e396f771345d4ef122ba33456304f15228d/NVIDIA/benchmarks/gpt3/implementations/eos-dfw_n1452_ngc24.04_nemo/config_common.sh

github.com

AI Neocloud Playbook and Anatomy Read at the source ↗

semianalysis.com

https://www.youtube.com/live/vJ8aEO6ggOs?si=ViPmlckQNmDYCayJ&t=3416

youtube.com

https://pytorch.org/blog/flexattention/

pytorch.org