corpus.blog
41554 blogs · 4,644,446 posts Claim your blog
Record · corpus.blog/posts/01a07df2-f125-7180-895f-3ae51e9bdc19

Five years of GPT progress

Finbarr Timbers · published 27 March 2023

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at finbarr.ca Wayback capture
1blogs citing
First cited
4 Mar 2026
Most recent
4 Mar 2026
Rank this month
not ranked
Links from this post
43
Words captured
3,411

Cited by

Blog
In the post
Date
blog.rootsofprogress.org
As we may vibe Read at the source ↗
4 Mar 2026

Links from this post

Link
Host
https://finbarrtimbers.substack.com/
finbarrtimbers.substack.com
https://ai.googleblog.com/2006/08/all-our-n-gram-are-belong-to-you.html
ai.googleblog.com
https://arxiv.org/abs/1810.04805
arxiv.org
https://openai.com/research/language-unsupervised
openai.com
https://huggingface.co/datasets/bookcorpus
huggingface.co
https://openai.com/research/better-language-models
openai.com
https://paperswithcode.com/dataset/webtext
paperswithcode.com
https://arxiv.org/abs/2005.14165
arxiv.org
https://paperswithcode.com/method/sparse-transformer
paperswithcode.com
https://arxiv.org/abs/2205.14135
arxiv.org
https://uploads-ssl.webflow.com/60fd4503684b466578c0d307/61138924626a6981ee09caf6_jurassic_tech_paper.pdf
uploads-ssl.webflow.com
https://en.wikipedia.org/wiki/AI21_Labs
en.wikipedia.org
https://github.com/ai21labs/lm-evaluation
github.com
https://github.com/facebookresearch/metaseq/blob/main/projects/OPT/chronicles/OPT175B_Logbook.pdf
github.com
https://gwern.net/complement
gwern.net
https://arxiv.org/abs/1811.06965
arxiv.org
https://paperswithcode.com/method/mesh-tensorflow
paperswithcode.com
https://arxiv.org/abs/2112.11446
arxiv.org
https://arxiv.org/abs/1910.07467
arxiv.org
https://arxiv.org/abs/1901.02860
arxiv.org
https://arxiv.org/abs/1808.06226
arxiv.org
https://en.wikipedia.org/wiki/Byte_pair_encoding
en.wikipedia.org
https://arxiv.org/abs/1910.02054
arxiv.org
https://www.notion.so/b2293a2b125b4b088656e039fb3b6ca8
notion.so
https://paperswithcode.com/method/gradient-checkpointing
paperswithcode.com
https://paperswithcode.com/method/adafactor
paperswithcode.com
https://nhigham.com/2020/07/07/what-is-stochastic-rounding/
nhigham.com
https://paperswithcode.com/method/chinchilla
paperswithcode.com
https://www.lesswrong.com/posts/6Fpvch8RR29qLEWNH/chinchilla-s-wild-implications
lesswrong.com
https://www.notion.so/Factual-accuracy-issues-in-LLMs-8257c59fda4040509a40d94523701fad
notion.so
https://twitter.com/finbarrtimbers/status/1635102571567407105?s=46&t=_LCsoamG7K4pQj0vxlk0XA
twitter.com
https://twitter.com/rasbt/status/1637803700944093184?s=46&t=_LCsoamG7K4pQj0vxlk0XA
twitter.com
https://arxiv.org/abs/1911.02150
arxiv.org
https://twitter.com/rasbt/status/1637803703766769669
twitter.com
https://ai.facebook.com/blog/large-language-model-llama-meta-ai/
ai.facebook.com
https://arxiv.org/abs/2104.09864
arxiv.org
https://arxiv.org/abs/2112.05682
arxiv.org
https://arxiv.org/abs/2205.05198
arxiv.org
https://cdn.openai.com/papers/gpt-4.pdf
cdn.openai.com
https://rootnodes.substack.com/p/why-didnt-deepmind-build-gpt3
rootnodes.substack.com
https://skylion007.github.io/OpenWebTextCorpus/
skylion007.github.io
https://github.com/karpathy/nanoGPT/blob/master/data/openwebtext/readme.md
github.com
https://github.com/openai/tiktoken
github.com