Record · gilesthomas.com
Fun with low-rank vocab matrices (and a bonus test loss reduction?)
Giles' blog · 9 October 2026 · 45 min read
A post on gilesthomas.com, published 9 October 2026, has not been cited by any blog yet (corpus.blog, measured 12 October 2026).
A record is what was published and who pointed at it. The text of the post is not shown here: read it at the source, or at its Wayback capture.
0blogs citing
| Period | Blogs | Links |
|---|---|---|
| All time | 0 | 0 |
| Last 90 days | 0 | 0 |
| Last 30 days | 0 | 0 |
| Last 7 days | 0 | 0 |
| Words blogs link it with | 0 |
|---|---|
| External links | 49 |
| Length | 10,267 |
Cited by
No blog we hold has cited this post yet.
Links from this post
Link
Host
huggingface.co
https://huggingface.co/gpjt/jax-with-mha-bias-fw-fwedu-5050-DEPRECATED/discussions/1 Who cites this page
huggingface.co
github.com
huggingface.co
https://github.com/gpjt/ddp-base-model-from-scratch/blob/d2231213d6749921cf86f71e210d3f48da87f9da/runs/8xa100m40-lore-1-baseline-disabled/model.json Who cites this page
github.com
https://github.com/gpjt/ddp-base-model-from-scratch/blob/d2231213d6749921cf86f71e210d3f48da87f9da/runs/8xa100m40-lore-1-baseline-disabled/train.json Who cites this page
github.com
https://github.com/gpjt/ddp-base-model-from-scratch/blob/d2231213d6749921cf86f71e210d3f48da87f9da/runs/8xa100m40-lore-2-both-ordinary-init/model.json Who cites this page
github.com
https://github.com/gpjt/ddp-base-model-from-scratch/blob/d2231213d6749921cf86f71e210d3f48da87f9da/runs/8xa100m40-lore-2-both-ordinary-init/train.json Who cites this page
github.com
https://github.com/gpjt/ddp-base-model-from-scratch/blob/d2231213d6749921cf86f71e210d3f48da87f9da/runs/8xa100m40-lore-3-both-smart-init/model.json Who cites this page
github.com
https://github.com/gpjt/ddp-base-model-from-scratch/blob/d2231213d6749921cf86f71e210d3f48da87f9da/runs/8xa100m40-lore-3-both-smart-init/train.json Who cites this page
github.com
https://github.com/gpjt/ddp-base-model-from-scratch/blob/d2231213d6749921cf86f71e210d3f48da87f9da/runs/8xa100m40-lore-4-input-only/model.json Who cites this page
github.com
https://github.com/gpjt/ddp-base-model-from-scratch/blob/d2231213d6749921cf86f71e210d3f48da87f9da/runs/8xa100m40-lore-4-input-only/train.json Who cites this page
github.com
https://github.com/gpjt/ddp-base-model-from-scratch/blob/d2231213d6749921cf86f71e210d3f48da87f9da/runs/8xa100m40-lore-5-output-only/model.json Who cites this page
github.com
https://github.com/gpjt/ddp-base-model-from-scratch/blob/d2231213d6749921cf86f71e210d3f48da87f9da/runs/8xa100m40-lore-5-output-only/train.json Who cites this page
github.com
lambda.ai
https://github.com/gpjt/ddp-base-model-from-scratch/tree/6ec817f0bbf525f65444574ab7b6a572a06d8461/runs Who cites this page
github.com
pythonanywhere.com
https://github.com/gpjt/ddp-base-model-from-scratch/blob/d2231213d6749921cf86f71e210d3f48da87f9da/ddp_train.py Who cites this page
github.com
https://github.com/gpjt/ddp-base-model-from-scratch/blob/804cb3f60aba5157ce3ded2c3295de653bca6526/runs/8xa100m40-lore-6-new-seed-baseline-disabled/model.json Who cites this page
github.com
https://github.com/gpjt/ddp-base-model-from-scratch/blob/804cb3f60aba5157ce3ded2c3295de653bca6526/runs/8xa100m40-lore-6-new-seed-baseline-disabled/train.json Who cites this page
github.com
https://github.com/gpjt/ddp-base-model-from-scratch/blob/804cb3f60aba5157ce3ded2c3295de653bca6526/runs/8xa100m40-lore-7-new-seed-input-only/model.json Who cites this page
github.com
https://github.com/gpjt/ddp-base-model-from-scratch/blob/804cb3f60aba5157ce3ded2c3295de653bca6526/runs/8xa100m40-lore-7-new-seed-input-only/train.json Who cites this page
github.com
https://github.com/gpjt/ddp-base-model-from-scratch/blob/804cb3f60aba5157ce3ded2c3295de653bca6526/runs/8xa100m40-lore-8-new-seed-both-smart-init/model.json Who cites this page
github.com
https://github.com/gpjt/ddp-base-model-from-scratch/blob/804cb3f60aba5157ce3ded2c3295de653bca6526/runs/8xa100m40-lore-8-new-seed-both-smart-init/train.json Who cites this page
github.com
https://github.com/gpjt/ddp-base-model-from-scratch/blob/804cb3f60aba5157ce3ded2c3295de653bca6526/runs/8xa100m40-lore-9-new-seed-output-only/model.json Who cites this page
github.com
https://github.com/gpjt/ddp-base-model-from-scratch/blob/804cb3f60aba5157ce3ded2c3295de653bca6526/runs/8xa100m40-lore-9-new-seed-output-only/train.json Who cites this page
github.com
huggingface.co
huggingface.co
huggingface.co
docs.pytorch.org
https://en.wikipedia.org/wiki/Distribution_of_the_product_of_two_random_variables Who cites this page
en.wikipedia.org
docs.pytorch.org
mathworld.wolfram.com
https://github.com/gpjt/ddp-base-model-from-scratch/commit/6ec817f0bbf525f65444574ab7b6a572a06d8461 Who cites this page
github.com
https://github.com/gpjt/ddp-base-model-from-scratch/blob/6ec817f0bbf525f65444574ab7b6a572a06d8461/runs/8xa100m40-lore-10-both-corrected-smart-init/model.json Who cites this page
github.com
https://github.com/gpjt/ddp-base-model-from-scratch/blob/6ec817f0bbf525f65444574ab7b6a572a06d8461/runs/8xa100m40-lore-10-both-corrected-smart-init/train.json Who cites this page
github.com
https://github.com/gpjt/ddp-base-model-from-scratch/blob/6ec817f0bbf525f65444574ab7b6a572a06d8461/runs/8xa100m40-lore-11-output-only-corrected-smart-init/model.json Who cites this page
github.com
https://github.com/gpjt/ddp-base-model-from-scratch/blob/6ec817f0bbf525f65444574ab7b6a572a06d8461/runs/8xa100m40-lore-11-output-only-corrected-smart-init/train.json Who cites this page
github.com
huggingface.co
huggingface.co
Elsewhere on gilesthomas.com (9)
Similar posts
Vocabulary size: 128,000
wes.today
14 May 2026
In Which I Lose My Mind Over Embeddings (HPLM Chapter 2)
maayanroth.com
31 Jul 2026
NVIDIA AI Releases Nemotron 3 Embed: An Open Embedding Collection Whose 8B Checkpoint Ranks #1 on RTEB
marktechpost.com
17 Jul 2026
6 Oct 2026
Honey, I shrunk the embeddings: Matryoshka vs. PCA
dylancastillo.co
1 Aug 2026
Vocab Break
accidentalfactors.com
24 Aug 2026
Vocab Break
ianbarber.blog
25 Aug 2026
1 Sept 2026
14 Jul 2026
14 Aug 2026
Something wrong here? Report a problem