corpus.blog
Claim your blog
Record · corpus.blog/posts/01a0fcb2-77af-7191-b196-333f7335c19e

CUDA-Free Inference for LLMs

PyTorch Blog · published 4 September 2024

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at pytorch.org Wayback capture
0blogs citing
First cited
—
Most recent
—
Rank this month
not ranked
External links
9
Words captured
1,632

Cited by

No blog we hold has cited this post yet.

Links from this post

Link
Host
https://huggingface.co/meta-llama/Meta-Llama-3-8B
huggingface.co
https://huggingface.co/ibm-granite/granite-8b-code-base
huggingface.co
https://github.com/triton-lang/triton
github.com
https://developer.nvidia.com/nsight-systems
developer.nvidia.com
https://github.com/ROCm/triton/blob/triton-mlir/python/perf-kernels/flash-attention.py
github.com
https://github.com/triton-lang/triton/blob/main/python/tutorials/06-fused-attention.py
github.com
https://github.com/Dao-AILab/flash-attention/blob/3669b25206d5938e3cc74a5f7860e31c38af8204/flash_attn/flash_attn_triton.py
github.com
https://github.com/facebookresearch/xformers/blob/fae0ceb195a41f2ab762d89449c6012fbcf2ffda/xformers/ops/fmha/triton_splitk.py
github.com
https://github.com/pytorch/pytorch/blob/e7b870c88bc3b854a95399a96a274d2f1f908172/torch/nn/attention/flex_attention.py
github.com
Elsewhere on pytorch.org (6)