Record · danmackinlay.name

Gradient descent at scale – The Dan MacKinlay stable of variably-well-consider’d enterprises

Dan MacKinlay · 1 October 2026 · 4 min read

A post on danmackinlay.name, published 1 October 2026, has not been cited by any blog yet (corpus.blog, measured 11 October 2026).

A record is what was published and who pointed at it. The text of the post is not shown here: read it at the source, or at its Wayback capture.

Read at danmackinlay.name Wayback capture

0blogs citing

Citing blogs and links by period
Period	Blogs	Links
All time	0	0
Last 90 days	0	0
Last 30 days	0	0
Last 7 days	0	0

Figures for this page
Words blogs link it with	0
External links	22
Length	771

Cited by

No blog we hold has cited this post yet.

Links from this post

Link

Host

https://lilianweng.github.io/posts/2021/09/24/train-large-neural-networks.html?ref=danmackinlay.name

lilianweng.github.io

Training extremely large neural networks across thousands of GPUs. Read at jeremyjordan.me

jeremyjordan.me

https://horovod.ai/ Who cites this page

horovod.ai

https://github.com/horovod/horovod Who cites this page

github.com

https://pytorch-lightning.readthedocs.io/en/latest/advanced/advanced_gpu.html

pytorch-lightning.readthedocs.io

https://www.microsoft.com/en-us/research/blog/zero-deepspeed-new-system-optimizations-enable-training-models-with-over-100-billion-parameters/ Who cites this page

microsoft.com

Fit More and Train Faster With ZeRO via DeepSpeed and FairScale Read at huggingface.co

huggingface.co

https://github.com/huggingface/transformers/issues/8771 Who cites this page

github.com

https://efficientdl.com/an-introduction-to-zero-offloading/?ref=danmackinlay.name

efficientdl.com

https://github.com/microsoft/DeepSpeed Who cites this page

github.com

https://github.com/microsoft/mup?tab=readme-ov-file&ref=danmackinlay.name

github.com

https://www.microsoft.com/en-us/research/blog/%C2%B5transfer-a-technique-for-hyperparameter-tuning-of-enormous-neural-networks/?ref=danmackinlay.name

microsoft.com

Deriving Muon Read at jeremybernste.in

jeremybernste.in

https://kellerjordan.github.io/posts/muon/ Who cites this page

kellerjordan.github.io

https://docs.modula.systems/

docs.modula.systems

Dion: the distributed orthonormal update revolution is here Read at microsoft.com

microsoft.com

How to Train Really Large Models on Many GPUs? Read at lilianweng.github.io

lilianweng.github.io

https://colossalai.org/docs/concepts/colossalai_overview/?ref=danmackinlay.name

colossalai.org

https://doi.org/10.48550/arXiv.2502.16982

doi.org

https://arxiv.org/abs/1910.02054 Who cites this page

arxiv.org

https://doi.org/10.1145/3394486.3406703

doi.org

https://arxiv.org/abs/2101.06840 Who cites this page

arxiv.org

Elsewhere on danmackinlay.name (4)

https://danmackinlay.name/notebook/practical_cloud_ml.html

danmackinlay.name

Pytorch – The Dan MacKinlay stable of variably-well-consider’d enterprises Read at danmackinlay.name

danmackinlay.name

https://danmackinlay.name/notebook/nn_scaling_laws.html

danmackinlay.name

https://danmackinlay.name/notebook/matrix_orthonormal.html

danmackinlay.name

Gradient descent at scale – The Dan MacKinlay stable of variably-well-consider’d enterprises

Cited by

Links from this post

Similar posts