Record · corpus.blog/posts/01a0c551-9c67-7334-bd2c-8479eca70c81

Supporting efficient large model training on AMD Instinct™ GPUs with DeepSpeed

opensource.microsoft.com · published 21 March 2022

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at opensource.microsoft.com Wayback capture

0blogs citing

First cited: —
Most recent: —
Rank this month: not ranked
External links: 24
Words captured: 1,612

Cited by

No blog we hold has cited this post yet.

Links from this post

Link

Host

https://www.deepspeed.ai/

deepspeed.ai

http://aka.ms/aiatscale

aka.ms

https://www.microsoft.com/research/blog/using-deepspeed-and-megatron-to-train-megatron-turing-nlg-530b-the-worlds-largest-and-most-powerful-generative-language-model/

microsoft.com

https://www.amd.com/en/graphics/instinct-server-accelerators

amd.com

https://docs.amd.com/bundle/Hardware_and_Software_Reference_Guide/page/Hardware_and_Software_Support.html

docs.amd.com

https://www.deepspeed.ai/tutorials/zero/

deepspeed.ai

https://www.deepspeed.ai/tutorials/zero-offload/

deepspeed.ai

https://www.microsoft.com/en-us/research/blog/zero-infinity-and-deepspeed-unlocking-unprecedented-model-scale-for-deep-learning-training/

microsoft.com

https://www.microsoft.com/en-us/research/blog/deepspeed-extreme-scale-model-training-for-everyone/

microsoft.com

https://docs.amd.com/bundle/AMD_HIP_Programming_Guide_v5.0/page/Introduction.html

docs.amd.com

https://docs.amd.com/bundle/RocBLAS_documentation/page/usermanual.html

docs.amd.com

https://docs.amd.com/bundle/Welcome-to-RCCL-s-documentation----RCCL-0.8-documentation/page/library.html

docs.amd.com

https://github.com/microsoft/DeepSpeed/releases/tag/v0.6.0

github.com

https://pypi.org/project/deepspeed/

pypi.org

https://hub.docker.com/r/rocm/deepspeed

hub.docker.com

https://www.deepspeed.ai/getting-started/

deepspeed.ai

https://github.com/huggingface/transformers

github.com

https://github.com/huggingface/transformers/tree/master/examples/pytorch

github.com

https://github.com/microsoft/DeepSpeed/issues

github.com

https://github.com/RadeonOpenCompute/ROCm/issues

github.com

https://github.com/microsoft/onnxruntime/tree/master/orttraining

github.com

https://cloudblogs.microsoft.com/opensource/2021/07/13/onnx-runtime-release-1-8-1-previews-support-for-accelerated-training-on-amd-gpus-with-the-amd-rocm-open-software-platform/

cloudblogs.microsoft.com

https://github.com/pytorch/ort

github.com

https://techcommunity.microsoft.com/t5/ai-machine-learning-blog/accelerate-pytorch-transformer-model-training-with-onnx-runtime/ba-p/2540471

techcommunity.microsoft.com