56,966 blogs · 4,417,066 posts Claim your blog

Record · corpus.blog/posts/01a08756-ccbf-73ff-9291-395123c030d2

Which H100 instance to train Nanochat

bluenotebook.io · published 4 March 2026

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at bluenotebook.io Wayback capture

0blogs citing

First cited: —
Most recent: —
Rank this month: not ranked
Links from this post: 18
Words captured: 3,525

Cited by

No blog we hold has cited this post yet.

Links from this post

Link

Host

https://github.com/karpathy/nanochat/tree/master

github.com

https://github.com/karpathy/nanochat/blob/c7ba25214276d165eeefca7cb2060587975db189/nanochat/optim.py

github.com

https://arxiv.org/pdf/1910.02054

arxiv.org

https://youtu.be/l1RJcDjzK8M

youtu.be

https://docs.nvidia.com/deeplearning/nccl/user-guide/docs/usage/collectives.html

docs.nvidia.com

https://www.nvidia.com/en-in/data-center/h100/

nvidia.com

https://docs.nvidia.com/enterprise-reference-architectures/white-paper.pdf

docs.nvidia.com

https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/h100/PB-11773-001_v01.pdf

nvidia.com

https://github.com/karpathy/nanochat/blob/master/dev/LEADERBOARD.md

github.com

https://github.com/Nikhil-Kasukurthi/nanochat/blob/11dab2952bb64fe6b0d1347b1f1e5ca71dcac908/scripts/profile_comms.py

github.com

https://github.com/Nikhil-Kasukurthi/nanochat/blob/master/runs/profile_comms.sh

github.com

https://docs.pytorch.org/tutorials/recipes/recipes/tuning_guide.html

docs.pytorch.org

https://github.com/pytorch/pytorch/issues/148689

github.com

https://github.com/NVIDIA/nccl

github.com

https://arxiv.org/abs/2507.04786v1

arxiv.org

https://github.com/karpathy/nanochat/discussions/481

github.com

https://www.youtube.com/watch?v=6OBtO9niT00

youtube.com

Strangely, Matrix Multiplications on GPUs Run Faster When Given "Predictable" Data! [short] Read at the source ↗