corpus.blog
41,554 blogs · 5,358,632 posts Claim your blog
Record · corpus.blog/posts/01a0875f-b199-723a-924f-179c5778ecd0

GPU MODE Lecture 13: Ring Attention

christianjmills.com · published 11 November 2024

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at christianjmills.com Wayback capture
0blogs citing
First cited
—
Most recent
—
Rank this month
not ranked
External links
31
Words captured
4,650

Cited by

No blog we hold has cited this post yet.

Links from this post

Link
Host
https://www.youtube.com/watch?v=ws7angQYIxI
youtube.com
https://docs.google.com/presentation/d/180lS8XbeR1_bTMaldg21LKYQkjXftHuh9VnZ3xk27qQ/edit
docs.google.com
https://github.com/gpu-mode/lectures/blob/main/lecture_013/howto_log_sum_exp.ipynb
github.com
https://github.com/gpu-mode/ring-attention
github.com
https://aleph-alpha.com/
aleph-alpha.com
https://deepmind.google/technologies/gemini/pro/
deepmind.google
https://github.com/LargeWorldModel/LWM
github.com
https://huggingface.co/NousResearch/Yarn-Mistral-7b-128k
huggingface.co
https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turbo
platform.openai.com
https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm
databricks.com
https://huggingface.co/mosaicml/mpt-7b
huggingface.co
https://arxiv.org/abs/2108.12409
arxiv.org
https://huggingface.co/xai-org/grok-1
huggingface.co
https://largeworldmodel.github.io/lwm/
largeworldmodel.github.io
https://huggingface.co/papers/2207.12598
huggingface.co
https://www.nvidia.com/en-us/data-center/h200/
nvidia.com
https://www.amd.com/en/products/accelerators/instinct/mi300.html
amd.com
https://www.nvidia.com/en-us/data-center/gb200-nvl2/
nvidia.com
https://arxiv.org/abs/2310.01889
arxiv.org
https://github.com/zhuzilin/ring-flash-attention/blob/55ff66fd35f329dfcc24ce7a448bfdd532865966/ring_flash_attn/utils.py
github.com
https://x.com/fvsmassa/status/1580229170629849089
x.com
https://github.com/zhuzilin/ring-flash-attention/tree/main
github.com
https://arxiv.org/abs/2105.13120
arxiv.org
https://crfm.stanford.edu/2023/10/12/flashdecoding.html
crfm.stanford.edu
https://github.com/facebookresearch/xformers
github.com
https://arxiv.org/abs/2205.14135
arxiv.org
https://arxiv.org/abs/2305.19370
arxiv.org
https://arxiv.org/abs/2311.09431
arxiv.org
https://arxiv.org/abs/2402.08268
arxiv.org
https://github.com/zhuzilin
github.com
https://github.com/zhuzilin/ring-flash-attention
github.com