Record · corpus.blog/posts/01a0879b-306a-728d-a8ef-90e53e4171fe

DeepSeek’s Multi-Head Latent Attention

the_origami_fox · published 22 February 2025

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at liorsinai.github.io Wayback capture

0blogs citing

First cited: —
Most recent: —
Rank this month: not ranked
External links: 20
Words captured: 12,088

Cited by

No blog we hold has cited this post yet.

Links from this post

Link

Host

https://github.com/LiorSinai/TransformersLite.jl/tree/feature/mla

github.com

https://www.deepseek.com/

deepseek.com

https://edition.cnn.com/2025/01/27/tech/deepseek-ai-explainer/index.html

edition.cnn.com

https://www.technologyreview.com/2025/01/31/1110740/how-deepseek-ripped-up-the-ai-playbook-and-why-everyones-going-to-follow-it/

technologyreview.com

https://hn.algolia.com/?q=deepseek

hn.algolia.com

https://github.com/deepseek-ai/DeepSeek-VL2

github.com

https://github.com/deepseek-ai/DeepSeek-V3

github.com

https://github.com/deepseek-ai/DeepSeek-R1

github.com

https://arxiv.org/abs/1706.03762

arxiv.org

https://arxiv.org/abs/2106.09685

arxiv.org

https://arxiv.org/abs/2104.09864

arxiv.org

https://karpathy.ai/zero-to-hero.html

karpathy.ai

https://medium.com/@joaolages/kv-caching-explained-276520203249

medium.com

https://github.com/LiorSinai/TransformersLite.jl

github.com

https://github.com/LiorSinai/TransformersLite.jl/blob/feature/mla/test/MultiHeadAttention.jl

github.com

https://github.com/LiorSinai/TransformersLite.jl/blob/feature/mla/src/broadcasted_batched_mul.jl

github.com

https://pytorch.org/docs/stable/generated/torch.einsum.html

pytorch.org

https://github.com/deepseek-ai/FlashMLA

github.com

https://github.com/LiorSinai/TransformersLite.jl/blob/feature/mla/src/layers/MultiHeadLatentAttentionV2.jl

github.com

https://en.wikipedia.org/wiki/Tensor

en.wikipedia.org