corpus.blog
41,948 blogs · 5,239,565 posts Claim your blog
Record · corpus.blog/posts/01a087b0-a346-7212-a55e-fa0470a22349

Calculating GPT-2’s inference speedups

njkumar.com · published 26 November 2024

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at njkumar.com Wayback capture
0blogs citing
First cited
—
Most recent
—
Rank this month
not ranked
External links
17
Words captured
5,476

Cited by

No blog we hold has cited this post yet.

Links from this post

Link
Host
https://www.artfintel.com/p/transformer-inference-tricks
artfintel.com
https://github.com/karpathy/nanoGPT
github.com
https://huggingface.co/openai-community/gpt2-xl
huggingface.co
https://lambdalabs.com
lambdalabs.com
https://medium.com/@joaolages/kv-caching-explained-276520203249
medium.com
Accelerating Generative AI with PyTorch II: GPT, Fast Read at the source ↗
pytorch.org
https://cloud.google.com/tpu/docs/bfloat16
cloud.google.com
https://newsletter.maartengrootendorst.com/i/145531349/symmetric-quantization
newsletter.maartengrootendorst.com
A Visual Guide to Quantization
newsletter.maartengrootendorst.com
https://github.com/pytorch/ao
github.com
https://github.com/Dao-AILab/flash-attention
github.com
Flash-Decoding for long-context inference Read at the source ↗
pytorch.org
https://github.com/pytorch/pytorch/issues/120441
github.com
https://pytorch.org/docs/stable/library.html
pytorch.org
https://x.com/finbarrtimbers
x.com
https://jalammar.github.io/illustrated-transformer
jalammar.github.io
https://kipp.ly/transformer-inference-arithmetic/
kipp.ly
Elsewhere on njkumar.com (1)