corpus.blog
56,966 blogs · 4,417,066 posts Claim your blog
Record · corpus.blog/posts/01a0e2c2-31b1-7380-9151-286d8b700d6d

FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision

tridao.me · published 11 July 2024

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at tridao.me Wayback capture
1blogs citing
First cited
30 Oct 2024
Most recent
30 Oct 2024
Rank this month
not ranked
Links from this post
14
Words captured
1,718

Cited by

Blog
In the post
Date
alexzhang13.github.io
A Meticulous Guide to Advances in Deep Learning Efficiency over the Years Read at the source ↗
30 Oct 2024

Links from this post

Link
Host
https://arxiv.org/abs/2407.08608
arxiv.org
https://github.com/Dao-AILab/flash-attention
github.com
https://pytorch.org/docs/stable/generated/torch.nn.functional.scaled_dot_product_attention.html
pytorch.org
https://huggingface.co/gradientai/Llama-3-8B-Instruct-Gradient-1048k
huggingface.co
https://arxiv.org/abs/2205.14135
arxiv.org
https://resources.nvidia.com/en-us-tensor-core/gtc22-whitepaper-hopper?ncid=no-ncid
resources.nvidia.com
https://github.com/NVIDIA/cutlass
github.com
https://github.com/HazyResearch/ThunderKittens
github.com
Accelerating Transformers with NVIDIA cuDNN 9 Read at the source ↗
developer.nvidia.com
https://github.com/NVIDIA/cutlass/blob/main/media/docs/efficient_gemm.md
github.com
https://horace.io/brrr_intro.html
horace.io
https://arxiv.org/abs/2208.07339
arxiv.org
https://arxiv.org/abs/2307.13304
arxiv.org
https://arxiv.org/abs/2402.04396
arxiv.org