Record · corpus.blog/posts/01a08c5f-80b1-7297-86b2-c0df36951e9b

LLM pretraining on TPU v6e with a $50 budget

vorushin.github.io · published 22 February 2026

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at vorushin.github.io Wayback capture

0blogs citing

First cited: —
Most recent: —
Rank this month: not ranked
Links from this post: 21
Words captured: 2,397

Cited by

No blog we hold has cited this post yet.

Links from this post

Link

Host

https://github.com/karpathy/nanochat

github.com

https://docs.cloud.google.com/tpu/docs/v6e

docs.cloud.google.com

https://github.com/vorushin/tpuchat

github.com

https://colab.research.google.com/github/vorushin/tpuchat/blob/master/08_tpu_ablations.ipynb?flush_caches=true

colab.research.google.com

https://antigravity.google/

antigravity.google

https://github.com/vorushin/tpuchat/blob/master/05_tpu_perf.ipynb

github.com

https://colab.research.google.com/github/vorushin/tpuchat/blob/master/05_tpu_perf.ipynb?flush_caches=true

colab.research.google.com

https://developers.googleblog.com/train-gpt2-model-with-jax-on-tpu/

developers.googleblog.com

https://docs.jax.dev/en/latest/jax.lax.html

docs.jax.dev

https://arxiv.org/abs/1904.06376

arxiv.org

https://openxla.org/xla/operation_semantics

openxla.org

https://physics.allen-zhu.com/part-4-architecture-design/part-4-1

physics.allen-zhu.com

https://arxiv.org/abs/2602.04998

arxiv.org

https://arxiv.org/abs/2507.12466

arxiv.org

https://github.com/AI-Hypercomputer/maxtext

github.com

https://docs.jax.dev/en/latest/the-training-cookbook.html

docs.jax.dev

https://jax-ml.github.io/scaling-book/

jax-ml.github.io

The modded nanogpt speedrun, but in JAX and on TPUs Read at the source ↗

nor-blog.pages.dev

https://x.com/vorushin/status/2024040663214588124

x.com

https://jax-ml.github.io/scaling-book/roofline/

jax-ml.github.io

https://cloud.google.com/tpu/docs/tpu7x

cloud.google.com