41,554 blogs · 5,316,985 posts Claim your blog

Record · corpus.blog/posts/01a0875f-b199-723a-924f-179c5baadf0a

GPU MODE Lecture 12: Flash Attention

christianjmills.com · published 15 September 2024

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at christianjmills.com Wayback capture

0blogs citing

First cited: —
Most recent: —
Rank this month: not ranked
External links: 10
Words captured: 11,955

Cited by

No blog we hold has cited this post yet.

Links from this post

Link

Host

https://www.youtube.com/watch?v=zEuwuCTEf_0

youtube.com

https://arxiv.org/abs/1706.03762

arxiv.org

https://huggingface.co/docs/text-generation-inference/en/conceptual/flash_attention

huggingface.co

https://cdn-lfs.huggingface.co/datasets/huggingface/documentation-images/7a59cb2f5b1c0ed0a00fe065f89e3f736aea3d5b10df95d90a7747fe543f3925?response-content-disposition=inline%3B+filename*%3DUTF-8%27%27flash-attn.png%3B+filename%3D%22flash-attn.png%22%3B&response-content-type=image%2Fpng&Expires=1726702524&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTcyNjcwMjUyNH19LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2RuLWxmcy5odWdnaW5nZmFjZS5jby9kYXRhc2V0cy9odWdnaW5nZmFjZS9kb2N1bWVudGF0aW9uLWltYWdlcy83YTU5Y2IyZjViMWMwZWQwYTAwZmUwNjVmODllM2Y3MzZhZWEzZDViMTBkZjk1ZDkwYTc3NDdmZTU0M2YzOTI1P3Jlc3BvbnNlLWNvbnRlbnQtZGlzcG9zaXRpb249KiZyZXNwb25zZS1jb250ZW50LXR5cGU9KiJ9XX0_&Signature=Q4YZK%7Ep22EKRaDzKRHXvMiHLusgwlOz9tyGjY-iqvvkirvcXRFIQJnxgkCponnm%7ED--y8kqWOO4iePmt8o27tbGDn6XWq%7EiWh9eY7YKA0g1RE0S-JIzcnspFJInINeIo2mZ8FCAxUnACT1b1xGuJOO3ecrktWPd%7E4ZyqjDUZICQXcwdKXhTpYA3SbP2pvwf59wzpjTsBM94-sFIlewNKtRrJiMW5ic2LYvNq5OsRzrjHCsGDA9MCtIayQasmFStROB1boLAojLG8Y6FjVdcXZeySLBjdKZHIDQuT9W4CyE91ZUgxFzj28CUixEa53vOto7s8GeGbdR%7ERy95JwVF8OA__&Key-Pair-Id=K3ESJI6DHPFC7

cdn-lfs.huggingface.co

https://dev-discuss.pytorch.org/t/fmas-and-softmax-and-floating-point-considered-harmful/1953/1

dev-discuss.pytorch.org

https://arxiv.org/abs/1805.02867

arxiv.org

https://triton-lang.org/main/getting-started/tutorials/06-fused-attention.html

triton-lang.org

https://godbolt.org/

godbolt.org

https://nvidia.github.io/cuda-python/overview.html

nvidia.github.io

https://github.com/Lightning-AI/lightning-thunder

github.com