Record · corpus.blog/posts/01a08771-a38d-72bf-a0b0-f06e053fd591

How much LLM training data is there, in the limit?

educatingsilicon.com · published 9 May 2024

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at educatingsilicon.com Wayback capture

2blogs citing

First cited: 14 May 2024
Most recent: 27 Mar 2025
Rank this month: not ranked
External links: 78
Words captured: 5,236

Cited by

Blog

In the post

Date

interwebalchemy.com

Computer Chess: The Byte Pair Encoding Variation Read at the source ↗

27 Mar 2025

breck.lol

The law favors AI over humans Read at the source ↗

14 May 2024

Links from this post

Link

Host

https://ai.meta.com/blog/meta-llama-3/

ai.meta.com

https://epochai.org/data/epochdb/table

epochai.org

https://aclanthology.org/D07-1090.pdf

aclanthology.org

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6195063/

ncbi.nlm.nih.gov

https://commoncrawl.org/

commoncrawl.org

https://www.together.ai/blog/redpajama-data-v2

together.ai

https://huggingface.co/datasets/HuggingFaceFW/fineweb

huggingface.co

https://huggingface.co/spaces/HuggingFaceFW/blogpost-fineweb-v1

huggingface.co

https://commoncrawl.github.io/cc-crawl-statistics/plots/languages.html

commoncrawl.github.io

https://dallascard.github.io/granular-material/post/financing-common-crawl/

dallascard.github.io

https://x.com/mark_cummins/status/1790717366130655677

x.com

https://huggingface.co/datasets/bigcode/the-stack-v2

huggingface.co

https://drive.google.com/file/d/17iGn3c-sYNiLyRSY-A85QOzgzGnGiVI3/view

drive.google.com

https://www.softwareheritage.org/

softwareheritage.org

https://www.aspectprogrammer.org/blogs/adrian/2005/03/grady_boochs_ke.html

aspectprogrammer.org

https://medium.com/modern-stack/how-much-computer-code-has-been-written-c8c03100f459

medium.com

https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0093949

journals.plos.org

https://slate.com/technology/2015/01/english-is-the-language-of-science-u-s-dominance-means-other-scientists-must-learn-foreign-language.html

slate.com

https://www.researchgate.net/figure/The-number-of-papers-over-time-The-total-number-of-papers-has-surged-exponentially-over_fig1_333487946

researchgate.net

https://wordsrated.com/number-of-academic-papers-published-per-year/

wordsrated.com

https://quantifyinghealth.com/length-of-a-research-paper/

quantifyinghealth.com

https://en.m.wikipedia.org/wiki/Anna%27s_Archive

en.m.wikipedia.org

https://patents.google.com/coverage

patents.google.com

https://ipwatchdog.com/2017/12/06/changes-patent-language-ensure-eligibility-alice/id=90721/

ipwatchdog.com

https://blog.google/products/search/15-years-google-books/

blog.google

https://prowritingaid.com/average-book-length

prowritingaid.com

https://en.wikipedia.org/wiki/Library_of_Congress

en.wikipedia.org

https://www.npr.org/2010/08/12/129160859/googles-tally-of-worlds-book-titles-129-864-880

npr.org

https://medium.com/@yoemei86/how-many-books-are-published-each-year-110496925c47

medium.com

https://free.law/2016/10/10/pacer-costs-a-billion-dollars

free.law

https://www.courtlistener.com/recap/

courtlistener.com

https://archive.org/details/usfederalcourts&tab=about?tab=abouthttps://archive.org/details/usfederalcourts&tab=about?tab=about

archive.org

https://www.bailii.org/bailii/

bailii.org

https://openlegaldata.io/

openlegaldata.io

https://x.com/XYOU/status/1789918141566304538

x.com

https://blog.twitter.com/official/en_us/a/2014/the-2014-yearontwitter.html

blog.twitter.com

techcrunch.com

https://www.degruyter.com/document/doi/10.1515/9781501511721-024/pdf?licenseType=restricted

degruyter.com

https://sci-hub.se/https://doi.org/10.1515/9781501511721-024

sci-hub.se

https://about.fb.com/news/2015/10/search-fyi-find-what-the-world-is-saying-with-facebook-search/

about.fb.com

https://cocoa.ethz.ch/downloads/2013/05/1188_06118886.pdf

cocoa.ethz.ch

https://www.theverge.com/2024/4/18/24133808/meta-ai-assistant-llama-3-chatgpt-openai-rival

theverge.com

https://tubestats.org/

tubestats.org

https://journalqd.org/article/view/4066

journalqd.org

https://archive.ph/tLjvj

archive.ph

https://www.tiktok.com/transparency/en/community-guidelines-enforcement-2023-4/

tiktok.com

https://www.pewresearch.org/internet/2024/02/22/how-u-s-adults-use-tiktok/

pewresearch.org

https://www.statista.com/statistics/1372569/tiktok-video-duration-by-number-of-views/

statista.com

https://podcastindustryinsights.com/apple-podcasts-statistics/

podcastindustryinsights.com

https://www.yaguara.co/podcast-statistics/

yaguara.co

https://riverside.fm/blog/how-long-should-a-podcast-be

riverside.fm

https://www.nielsen.com/insights/2024/amid-the-fragmented-tv-landscape-time-spent-with-content-is-the-best-planning-data-there-is/

nielsen.com

https://en.wikipedia.org/wiki/Television_in_the_United_Kingdom

en.wikipedia.org

https://www.aljazeera.com/news/2016/2/13/in-the-internet-age-radio-still-rules-the-world

aljazeera.com

https://www.nationsencyclopedia.com/WorldStats/CIA-Radio-broadcast-stations.html

nationsencyclopedia.com

https://en.wikipedia.org/wiki/BBC_Archives

en.wikipedia.org

https://en.wikipedia.org/wiki/British_Library_Sound_Archive

en.wikipedia.org

https://archive.org/details/radiostationarchives

archive.org

https://www.siliconrepublic.com/business/facebook-in-app-messaging-test-meta-messenger

siliconrepublic.com

https://gitnux.org/wechat-statistics

gitnux.org

https://www.usesignhouse.com/blog/imessage-stats

usesignhouse.com

https://www.researchgate.net/publication/299487660_WhatsApp_Usage_Patterns_and_Prediction_Models

researchgate.net

https://www.sellcell.com/blog/how-many-phone-calls-are-made-a-day-2023-statistics

sellcell.com

https://en.wikipedia.org/wiki/MYSTIC

en.wikipedia.org

https://www.researchgate.net/publication/6223260_Are_Women_Really_More_Talkative_Than_Men

researchgate.net

https://docs.google.com/spreadsheets/d/1j0gQwEMMCxu9YnNcogoJdkqQaMvnhSIE06IN25lut7c/edit

docs.google.com

https://www.prb.org/articles/how-many-people-have-ever-lived-on-earth/

prb.org

https://openaccess.thecvf.com/content_iccv_2013/papers/Bissacco_PhotoOCR_Reading_Text_2013_ICCV_paper.pdf

openaccess.thecvf.com

https://community.openai.com/t/rules-of-thumb-for-number-of-source-code-characters-to-tokens/622947/3

community.openai.com

https://twitter.com/c_stroebele/status/1724532920465907948

twitter.com

https://arxiv.org/pdf/2112.11446

arxiv.org

https://arxiv.org/pdf/2112.04426

arxiv.org

https://dcecf.psc.uscourts.gov/cgi-bin/login.pl

dcecf.psc.uscourts.gov

https://www.reuters.com/legal/government/judge-approves-125-million-pacer-fees-settlement-with-us-judiciary-2024-03-20/

reuters.com

https://www.radicati.com/wp/wp-content/uploads/2022/11/Email-Statistics-Report-2022-2026-Executive-Summary.pdf

radicati.com

http://rohitapte.com/2019/04/04/stylometric-analysis/

rohitapte.com

https://en.wikipedia.org/wiki/Enron_Corpus

en.wikipedia.org