corpus.blog
41,554 blogs · 4,969,865 posts Claim your blog
Record · corpus.blog/posts/01a07d5d-72e7-7112-bd07-c154f4b1f1f9

Mixture of Experts (MoEs) in Transformers

Hugging Face · published 26 February 2026

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at huggingface.co Wayback capture
0blogs citing
First cited
—
Most recent
—
Rank this month
not ranked
External links
13
Words captured
1,989

Cited by

No blog we hold has cited this post yet.

Links from this post

Link
Host
https://nlp.fast.ai/classification/2018/05/15/introducing-ulmfit.html
nlp.fast.ai
https://x.com/soumithchintala/status/1671267150101721090
x.com
https://youtu.be/CDnkFbW-uEQ
youtu.be
https://github.com/huggingface/transformers/pull/41580
github.com
https://github.com/huggingface/transformers/blob/main/src/transformers/core_model_loading.py
github.com
https://github.com/huggingface/transformers/blob/b71de73468429eb02da18caa50e9b5200400a4ed/src/transformers/core_model_loading.py
github.com
https://github.com/huggingface/transformers/pull/42697
github.com
https://docs.pytorch.org/docs/stable/generated/torch.bmm.html
docs.pytorch.org
https://docs.pytorch.org/docs/stable/generated/torch.nn.functional.grouped_mm.html
docs.pytorch.org
https://github.com/huggingface/transformers/blob/b71de73468429eb02da18caa50e9b5200400a4ed/src/transformers/integrations/tensor_parallel.py
github.com
https://unsloth.ai/docs/new/faster-moe
unsloth.ai
https://github.com/ariG23498/transformers/tree/bench-v4
github.com
https://github.com/ariG23498/transformers/tree/bench-v5
github.com
Elsewhere on huggingface.co (20)
https://huggingface.co/papers/2001.08361
huggingface.co
https://huggingface.co/openai/gpt-oss-20b
huggingface.co
https://huggingface.co/collections/Qwen/qwen35
huggingface.co
https://huggingface.co/collections/MiniMaxAI/minimax-m2
huggingface.co
https://huggingface.co/collections/zai-org/glm-5
huggingface.co
https://huggingface.co/collections/moonshotai/kimi-k25
huggingface.co
https://huggingface.co/deepseek-ai/DeepSeek-R1
huggingface.co
https://huggingface.co/deepseek-ai/DeepSeek-V2
huggingface.co
https://huggingface.co/mistralai/Mixtral-8x7B-v0.1
huggingface.co
https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/moe-transformers/moe_2y_timeline.png
huggingface.co
https://huggingface.co/collections/openai/gpt-oss
huggingface.co
Mixture of Experts Explained Read at the source ↗
huggingface.co
https://huggingface.co/docs/transformers/main/en/model_doc/auto
huggingface.co
https://huggingface.co/deepseek-ai/DeepSeek-V3/raw/main/model.safetensors.index.json
huggingface.co
https://huggingface.co/kernels-community/megablocks
huggingface.co
https://huggingface.co/docs/transformers/main/en/weightconverter
huggingface.co
https://huggingface.co/docs/transformers/main/en/internal/weight_converter
huggingface.co
https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/moe-transformers/loading_benchmark.png
huggingface.co
https://huggingface.co/docs/transformers/experts_interface
huggingface.co
https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/moe-transformers/expert_backend.png
huggingface.co