corpus.blog
41,554 blogs · 5,181,941 posts Claim your blog
Record · corpus.blog/posts/01a0fdd8-d8ee-72e3-b9e6-280dc13d95f2

PyTorch/XLA SPMD: Scale Up Model Training and Serving with Automatic Parallelization

PyTorch Blog · published 31 August 2023

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at pytorch.org Wayback capture
1blogs citing
First cited
3 Feb 2026
Most recent
3 Feb 2026
Rank this month
not ranked
External links
13
Words captured
1,449

Cited by

Blog
In the post
Date
dogac.dev
Training language models on TPUs shouldn’t be scary Read at the source ↗
3 Feb 2026

Links from this post

Link
Host
https://arxiv.org/pdf/2105.04663.pdf
arxiv.org
https://arxiv.org/abs/2105.04663
arxiv.org
https://github.com/pytorch/xla/blob/master/docs/spmd.md
github.com
https://github.com/pytorch/xla/blob/028df4da388468fa9a41b1f98ea08bfce13b4c63/torch_xla/experimental/xla_sharding.py
github.com
https://github.com/pytorch/xla/blob/9a5fdf3920c18275cf7dba785193636f1b39ced9/torch_xla/experimental/xla_sharding.py
github.com
https://github.com/pytorch/xla/blob/03991d44a0a0297ced3ba9fc10ba451a4b6c94ab/torch_xla/experimental/xla_sharded_tensor.py
github.com
https://github.com/pytorch/xla/issues/3871
github.com
https://github.com/pytorch/xla/tree/r2.0/test/spmd
github.com
https://github.com/pytorch-tpu/transformers/tree/yeounoh_gpt2_spmd
github.com
https://arxiv.org/pdf/2204.02311.pdf
arxiv.org
https://cloud.google.com/blog/products/ai-machine-learning/train-ml-models-on-large-images-and-3d-volumes-with-spatial-partitioning-on-cloud-tpus
cloud.google.com
https://github.com/pytorch/xla/blob/master/test/spmd/test_train_spmd_imagenet.py
github.com
https://github.com/pytorch/xla
github.com
Elsewhere on pytorch.org (1)