41,554 blogs · 4,969,865 posts Claim your blog

Record · corpus.blog/posts/01a0877d-0b30-706b-95b7-1bd03b0bdc70

Why do OpenAI's GPT-2 weights beat mine? Part three: testing overtraining

Giles' blog · published 31 July 2026

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at gilesthomas.com Wayback capture

0blogs citing

First cited: —
Most recent: —
Rank this month: not ranked
External links: 10
Words captured: 5,851

Cited by

No blog we hold has cited this post yet.

Links from this post

Link

Host

https://arxiv.org/abs/2203.15556

arxiv.org

https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf

cdn.openai.com

https://wandb.ai/bkkaggle/lm-finetuning/reports/Pretraining-a-124-M-Parameter-GPT-2-Language-Model--VmlldzoyMjg4NzA

wandb.ai

https://docs.nvidia.com/dl-cuda-graph/troubleshooting/capture-failures.html

docs.nvidia.com

https://huggingface.co/gpjt/jax-with-mha-bias-no-dropout-extended

huggingface.co

https://github.com/gpjt/jax-gpt2-from-scratch/commit/3cfb1dee50c0a36750c3c5132811c7aea09518d8

github.com

https://huggingface.co/gpjt/jax-with-mha-bias-no-dropout-2-epoch

huggingface.co

https://github.com/gpjt/ddp-base-model-from-scratch/blob/e71f33fb6aa421ec646b71f5cb69f3e3368c0d84/ift_generate_test_responses.py

github.com

https://github.com/gpjt/ddp-base-model-from-scratch/blob/e71f33fb6aa421ec646b71f5cb69f3e3368c0d84/ift_judge.py

github.com

https://arxiv.org/abs/2305.16264

arxiv.org