Record · corpus.blog/posts/01a0b689-a373-732f-b76c-286e2dc4c94d

TPU Inference Externalization Full Steam Ahead - InferenceX

newsletter.semianalysis.com · published 7 September 2026

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at newsletter.semianalysis.com Wayback capture

0blogs citing

First cited: —
Most recent: —
Rank this month: not ranked
Links from this post: 94
Words captured: 8,142

Cited by

No blog we hold has cited this post yet.

Links from this post

Link

Host

https://substackcdn.com/image/fetch/$s_!jJRp!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F679784de-e6df-4cb4-8bf6-3347cb9ed002_1200x588.png

substackcdn.com

https://blog.google/innovation-and-ai/infrastructure-and-cloud/google-cloud/ironwood-google-tpu-things-to-know/

blog.google

https://inferencex.semianalysis.com/

inferencex.semianalysis.com

https://newsletter.semianalysis.com/p/tpuv7-google-takes-a-swing-at-the

newsletter.semianalysis.com

https://semianalysis.com/accelerator-hbm-model/

semianalysis.com

https://newsletter.semianalysis.com/subscribe?

newsletter.semianalysis.com

https://semianalysis.com/ai-cloud-tco-model/

semianalysis.com

https://substackcdn.com/image/fetch/$s_!p96b!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8047a697-0d6f-41fe-8c16-71998778ecb4_2048x1228.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!OWyw!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd306f42c-a30b-45c7-82c9-dbaff7e654b3_2048x1228.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!q6dF!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F69317046-6bae-4d80-9b8b-f9777076792d_2048x1228.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!xG7g!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fac73c91e-efa3-4f2a-a9ea-2f5eb15e4a27_2048x1222.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!XMw9!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F01cfa04c-b0fe-456d-ba3c-367c434f373c_2048x1223.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!j22S!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb160a968-2cdc-4ce8-8a74-cd5eb63e1141_2048x1223.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!P-lB!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1883d94c-1996-47de-942b-c8eef7cf3eb1_1576x808.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!XKic!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd2210f76-7a91-405e-a3ba-a1692b080618_1550x842.png

substackcdn.com

https://docs.pytorch.org/xla/master/learn/xla-overview.html

docs.pytorch.org

https://vllm.ai/blog/2025-10-16-vllm-tpu

vllm.ai

https://substackcdn.com/image/fetch/$s_!KFHW!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F678ca15d-f0c0-4c36-a63f-b1108939cbb1_2048x1620.png

substackcdn.com

https://google.github.io/torchax/user_guide/how-it-works/

google.github.io

https://github.com/vllm-project/tpu-inference/blob/a32e183a676cf428cb1cea953f58fdd616accb3e/tpu_inference/models/vllm/vllm_model_wrapper.py

github.com

https://substackcdn.com/image/fetch/$s_!l0sG!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F63ea9afd-01ea-4df3-b632-3524e2507d04_2048x1742.png

substackcdn.com

https://github.com/sgl-project/sglang-jax

github.com

https://developers.googleblog.com/torchtpu-running-pytorch-natively-on-tpus-at-google-scale/

developers.googleblog.com

https://docs.pytorch.org/tutorials/advanced/privateuseone.html

docs.pytorch.org

https://hosted-files.sched.co/pytorchconferenceeu2026/cb/TorchTPU%20-%20PyTorch%20Paris%20%2726%20-%20Google%20Slides.pdf

hosted-files.sched.co

https://substackcdn.com/image/fetch/$s_!H5Pp!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc09d8e99-682d-43dd-9292-3cf29295f79e_1074x1148.png

substackcdn.com

Helion on TPU: Towards Hardware Heterogeneous Kernel Authoring Read at the source ↗

pytorch.org

https://substackcdn.com/image/fetch/$s_!uGCV!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9d86a8b7-cc12-4730-9b6a-a77258ad60c5_1860x720.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!KDky!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8304bd1c-ed83-4091-a71b-00ec572542a4_1200x675.png

substackcdn.com

https://github.com/vllm-project/tpu-inference/pull/2661

github.com

https://github.com/vllm-project/tpu-inference/pull/2187

github.com

https://github.com/vllm-project/tpu-inference/pull/2577

github.com

https://substackcdn.com/image/fetch/$s_!VcG6!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe1f13662-fc12-400e-889e-9958d0ad45f7_2048x770.png

substackcdn.com

https://github.com/vllm-project/tpu-inference/pull/2174

github.com

https://substackcdn.com/image/fetch/$s_!rnG7!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F59c3d46f-7b88-4e65-8715-f160b83c3d6e_2048x771.png

substackcdn.com

https://github.com/vllm-project/tpu-inference/pull/2888

github.com

https://substackcdn.com/image/fetch/$s_!FDFh!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4c82923a-c144-4447-8b73-f622ba7df27e_2048x1357.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!QwIQ!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe9c2975f-0974-414f-8af6-300bc8613121_2048x1202.png

substackcdn.com

https://github.com/vllm-project/tpu-inference/blob/main/tpu_inference/kernels/collectives/hierrs_sc/kernel.py

github.com

https://newsletter.semianalysis.com/p/tpu-inferencex-full-steam?utm_source=substack&utm_medium=email&utm_content=share&action=share

newsletter.semianalysis.com

https://github.com/vllm-project/tpu-inference/pull/2777

github.com

https://github.com/vllm-project/tpu-inference/pull/1688

github.com

https://github.com/vllm-project/tpu-inference/pull/2137

github.com

https://github.com/vllm-project/tpu-inference/pull/2836

github.com

https://substackcdn.com/image/fetch/$s_!I7Pl!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc47d1e6a-dad4-48bc-8239-9194bea17544_2048x838.png

substackcdn.com

https://github.com/vllm-project/tpu-inference/pull/2634

github.com

https://github.com/vllm-project/tpu-inference/pull/2674

github.com

https://github.com/vllm-project/tpu-inference/pull/3488

github.com

https://github.com/vllm-project/tpu-inference/pull/2004

github.com

https://substackcdn.com/image/fetch/$s_!BogK!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6ff82904-c377-4bbd-97a7-ad6eb54932ec_1875x925.png

substackcdn.com

https://github.com/vllm-project/tpu-inference/pull/2004/changes

github.com

https://github.com/vllm-project/tpu-inference/pull/2498

github.com

https://substackcdn.com/image/fetch/$s_!VkIy!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1c3573d3-b885-4a29-abc4-998df6ab3399_2048x263.png

substackcdn.com

https://github.com/vllm-project/tpu-inference/pull/2741

github.com

https://github.com/vllm-project/tpu-inference/pull/2650

github.com

https://github.com/vllm-project/tpu-inference/pull/3016

github.com

https://substackcdn.com/image/fetch/$s_!5fok!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc6dad690-9354-48a2-846f-1f7107907e3d_2048x672.png

substackcdn.com

https://github.com/vllm-project/tpu-inference/pull/1961

github.com

https://github.com/vllm-project/tpu-inference/pull/2416

github.com

https://github.com/vllm-project/tpu-inference/pull/2482

github.com

https://github.com/vllm-project/tpu-inference/pull/2627

github.com

https://substackcdn.com/image/fetch/$s_!nHOE!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fab7479c2-7b29-4ede-9796-9e8126a92b40_2048x756.png

substackcdn.com

https://github.com/vllm-project/tpu-inference/pull/2653

github.com

https://github.com/vllm-project/tpu-inference/pull/2513

github.com

https://substackcdn.com/image/fetch/$s_!eA0c!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F35b683ee-7e31-401a-8a8a-31c53e253aa2_2048x819.png

substackcdn.com

https://github.com/vllm-project/tpu-inference/pull/3080

github.com

https://github.com/vllm-project/tpu-inference/pull/3116

github.com

https://github.com/vllm-project/tpu-inference/pull/3422

github.com

https://substackcdn.com/image/fetch/$s_!fz3K!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc0cd68d1-b16d-442d-87e4-ca98067ae979_2048x856.png

substackcdn.com

https://github.com/vllm-project/tpu-inference/pull/3170

github.com

https://substackcdn.com/image/fetch/$s_!6KRh!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F725b7e9e-8c46-4d02-880e-5a7de8886e87_2048x915.png

substackcdn.com

https://github.com/vllm-project/tpu-inference/pull/3102

github.com

https://substackcdn.com/image/fetch/$s_!CgA8!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F72c0863c-a506-404a-8522-70bcadb3a89a_2048x756.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!bVa5!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2690029e-8ec9-4b9e-b517-ec6d2950e8a2_1999x1130.png

substackcdn.com

https://cloud.google.com/blog/products/compute/inside-the-ironwood-tpu-codesigned-ai-stack

cloud.google.com

https://substackcdn.com/image/fetch/$s_!SQDr!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6e6e4785-cd71-4160-90b8-40bf02e1483e_1456x542.png

substackcdn.com

https://jax-ml.github.io/scaling-book/tpus/

jax-ml.github.io

https://arxiv.org/abs/2604.15464v1

arxiv.org

https://substackcdn.com/image/fetch/$s_!egiT!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7e493590-cb03-43cf-851a-816a576d2083_1876x976.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!17HU!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff07dc2e7-090b-472a-a2bf-82c6fbb1cb67_1456x1956.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!wlNi!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ffc7f1e6f-b854-4da5-b6c1-8f1bbdb49cb8_1456x1505.png

substackcdn.com

https://semianalysis.com/ai-networking-model/

semianalysis.com

https://substackcdn.com/image/fetch/$s_!3iro!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F83cff324-fa52-4b79-b901-80da6f9345a9_2000x1268.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!ObRL!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9126a39f-54f9-4df1-bd09-4d0d64deaf17_2048x559.png

substackcdn.com

https://vllm.ai/blog/2024-10-17-spec-decode

vllm.ai

https://substackcdn.com/image/fetch/$s_!FX9c!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F282af9ce-a0c4-4c7b-8053-00026f16f945_1112x548.png

substackcdn.com

https://github.com/google/tpu-sync

github.com

https://substackcdn.com/image/fetch/$s_!Tn-L!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F62a1acf1-77d0-4c16-85da-9a046d56f2f8_1974x1342.png

substackcdn.com

https://github.com/kvcache-ai/Mooncake/issues/2662

github.com

https://substackcdn.com/image/fetch/$s_!q_Mm!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fec6b5d7e-af79-484b-bcf7-ebc50d36c3e3_2048x1204.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!hqEi!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff51b3df9-ffa4-408f-b617-1eb3dde40740_2039x975.png

substackcdn.com

https://kvcache.ai/blog/scaling-kv-cache-beyond-memory/

kvcache.ai

AgentX - InferenceXv3: Does CUDA Moat Hold up in Agentic Inferencing? Read at the source ↗

newsletter.semianalysis.com

https://substackcdn.com/image/fetch/$s_!QQ6I!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9ca94316-e44c-4bec-9d0a-041ad73df31e_2048x909.png

substackcdn.com