Record · corpus.blog/posts/01a0c507-b32e-73da-b4cf-e77591b4d941

Advancing Emerging Optimizers for Accelerated LLM Training with NVIDIA Megatron

NVIDIA Developer Blog · published 22 April 2026

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at developer.nvidia.com Wayback capture

0blogs citing

First cited: —
Most recent: —
Rank this month: not ranked
Links from this post: 19
Words captured: 1,718

Cited by

No blog we hold has cited this post yet.

Links from this post

Link

Host

https://arxiv.org/abs/1802.09568

arxiv.org

https://arxiv.org/abs/2502.16982

arxiv.org

https://huggingface.co/moonshotai/Kimi-K2-Base

huggingface.co

https://arxiv.org/pdf/2602.15763

arxiv.org

https://github.com/NVIDIA-NeMo/Emerging-Optimizers

github.com

https://www.nvidia.com/en-us/data-center/gb300-nvl72/

nvidia.com

https://docs.nvidia.com/nemo/megatron-bridge/latest/index.html

docs.nvidia.com

https://github.com/NVIDIA-NeMo

github.com

https://arxiv.org/html/2409.11321v1

arxiv.org

https://www.mpich.org/static/docs/v3.2/www3/MPI_Allgatherv.html

mpich.org

https://developer.nvidia.com/megatron-core

developer.nvidia.com

https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/optimizer/layer_wise_optimizer.py

github.com

https://github.com/NVIDIA/Megatron-LM/blob/b2e93909f09ce5a84d42beb844e905eb7edd7818/megatron/core/optimizer/muon.py

github.com

https://arxiv.org/abs/2002.09018

arxiv.org

https://docs.nvidia.com/cuda/cublas/index.html?highlight=syrk

docs.nvidia.com

https://github.com/NVIDIA-NeMo/Emerging-Optimizers/blob/main/emerging_optimizers/orthogonalized_optimizers/mop.py

github.com

https://github.com/NVIDIA-NeMo/Emerging-Optimizers/blob/main/emerging_optimizers/soap/rekls.py

github.com

https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/scripts/performance

github.com

https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/optimizer/emerging_optimizers.py

github.com