Record · corpus.blog/posts/01a0cabf-6537-71c3-9407-3bf4c8886737

100,000 H100 Clusters: Power, Network Topology, Ethernet vs InfiniBand, Reliability, Failures, Checkpointing

SemiAnalysis · published 17 June 2024

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at semianalysis.com Wayback capture

1blogs citing

First cited: 14 Dec 2024
Most recent: 14 Dec 2024
Rank this month: not ranked
Links from this post: 18
Words captured: 4,818

Cited by

Blog

In the post

Date

main-horse.github.io

Visualizing 6D Mesh Parallelism Read at the source ↗

14 Dec 2024

Links from this post

Link

Host

OpenAI Board Forms Safety and Security Committee Read at the source ↗

openai.com

https://semianalysis.com/accelerator-model

semianalysis.com

https://semianalysis.com/ai-datacenter-energy-dilemma-race

semianalysis.com

https://semianalysis.com/nvidias-infiniband-problem-qmx-ai

semianalysis.com

https://semianalysis.com/gpt-4-architecture-infrastructure

semianalysis.com

Strangely, Matrix Multiplications on GPUs Run Faster When Given "Predictable" Data! [short] Read at the source ↗

thonking.ai

https://pytorch.org/blog/straggler-mitigation/

pytorch.org

https://asc.llnl.gov/exascale/el-capitan

asc.llnl.gov

https://semianalysis.com/datacenter-model

semianalysis.com

https://www.marvell.com/products/optical-modules.html

marvell.com

https://semianalysis.com?p=143439831/inference-parallelism-techniques-pipeline-parallelism-tensor-parallelism-expert-parallelism-and-data-parallelism

semianalysis.com

Nvidia Blackwell Perf TCO Analysis – B100 vs B200 vs GB200 NVL72 Read at the source ↗

semianalysis.com

https://magazine.sebastianraschka.com/p/accelerating-pytorch-model-training

magazine.sebastianraschka.com

https://arxiv.org/pdf/2301.09830

arxiv.org

https://semianalysis.com/nvidias-optical-boogeyman-nvl72-infiniband

semianalysis.com

https://arxiv.org/abs/2309.08125

arxiv.org

https://pytorchtoatoms.substack.com/p/nvidia-connectx-7-16tbits-cedar-fever

pytorchtoatoms.substack.com

Nvidia’s InfiniBand Problem – Spectrum-X AI Fabric, Tomahawk-5, Jericho-3AI, Quantum-2 Read at the source ↗

semianalysis.com