43,228 blogs · 5,519,570 posts Claim your blog

Record · corpus.blog/posts/01a0fe65-bc79-7048-9649-e6d6dd756b64

Introducing PyTorch Fully Sharded Data Parallel (FSDP) API

PyTorch Blog · published 14 March 2022

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at pytorch.org Wayback capture

6blogs citing

First cited: 16 Jan 2023
Most recent: 31 Mar 2026
Rank this month: not ranked
External links: 6
Words captured: 1,431

Links from this post

Link

Host

https://arxiv.org/abs/1810.04805

arxiv.org

https://arxiv.org/abs/2104.04473

arxiv.org

https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/a100/pdf/nvidia-a100-datasheet-us-nvidia-1758950-r4-web.pdf

nvidia.com

https://github.com/karpathy/minGPT

github.com

https://arxiv.org/abs/1910.02054

arxiv.org

Fully Sharded Data Parallel: faster AI training with fewer GPUs Read at the source ↗

engineering.fb.com

Elsewhere on pytorch.org (4)

https://pytorch.org/docs/stable/fsdp.html

pytorch.org

https://pytorch.org/docs/stable/generated/torch.optim.SGD.html

pytorch.org

https://pytorch.org/docs/stable/checkpoint.html

pytorch.org

https://pytorch.org/tutorials/intermediate/ddp_tutorial.html

pytorch.org