Record · corpus.blog/posts/01a07d5d-72e8-7140-82cd-cd9270033fff

You could have designed state of the art positional encoding

Hugging Face · published 25 November 2024

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at huggingface.co Wayback capture

2blogs citing

First cited: 19 Dec 2024
Most recent: 27 Feb 2025
Rank this month: not ranked
Links from this post: 18
Words captured: 3,224

Cited by

Blog

In the post

Date

sayak.dev

Flavors of attention in modern diffusion models Read at the source ↗

27 Feb 2025

alexisgallagher.com

Finally, a Replacement for BERT: Introducing ModernBERT Read at the source ↗

19 Dec 2024

Links from this post

Link

Host

https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices/

ai.meta.com

https://pytorch.org/docs/stable/generated/torch.nn.MultiheadAttention.html

pytorch.org

https://www.cs.toronto.edu/~rgrosse/courses/csc321_2017/readings/L15%20Exploding%20and%20Vanishing%20Gradients.pdf

cs.toronto.edu

https://arxiv.org/abs/1706.03762

arxiv.org

https://en.wikipedia.org/wiki/Rotation_matrix

en.wikipedia.org

https://en.wikipedia.org/wiki/Noam_Shazeer

en.wikipedia.org

https://arxiv.org/pdf/2104.09864

arxiv.org

https://x.com/bojone1993

x.com

https://kexue.fm/archives/8130

kexue.fm

https://kexue.fm/archives/8265

kexue.fm

https://arxiv.org/pdf/2410.06205

arxiv.org

You could have designed state of the art Positional Encoding Read at the source ↗

fleetwood.dev

https://kazemnejad.com/blog/transformer_architecture_positional_encoding/

kazemnejad.com

Rotary Embeddings: A Relative Revolution Read at the source ↗

blog.eleuther.ai

https://www.youtube.com/watch?v=T3OT8kqoqjc

youtube.com

https://arxiv.org/pdf/1706.03762

arxiv.org

https://www.youtube.com/watch?v=T3OT8kqoqjc0

youtube.com

https://kazemnejad.com/

kazemnejad.com