Record · corpus.blog/posts/01a0df50-42a2-7315-a56c-0dc7c25922e4

Continual Learning with RL for LLMs

cameronrwolfe.substack.com · published 26 January 2026

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at cameronrwolfe.substack.com Wayback capture

1blogs citing

First cited: 7 Feb 2026
Most recent: 7 Feb 2026
Rank this month: not ranked
Links from this post: 114
Words captured: 9,441

Cited by

Blog

In the post

Date

dbonattoj.github.io

Formal Verification and the Path to Machine Discovery Read at the source ↗

7 Feb 2026

Links from this post

Link

Host

https://substackcdn.com/image/fetch/$s_!SF1W!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9bb30a35-5800-4256-b07b-21dce1b0af7e_2489x1391.png

substackcdn.com

https://en.wikipedia.org/wiki/Artificial_general_intelligence

en.wikipedia.org

https://www.dwarkesh.com/p/timelines-june-2025

dwarkesh.com

https://substackcdn.com/image/fetch/$s_!_YYv!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0872d8f5-5798-4764-aac3-d6558dd69dad_2326x950.png

substackcdn.com

Online versus Offline RL for LLMs Read at the source ↗

cameronrwolfe.substack.com

https://substackcdn.com/image/fetch/$s_!OMBQ!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa8f8919a-4168-4d1f-bb04-0d2e5897d029_1024x468.png

substackcdn.com

https://arxiv.org/abs/1706.08840

arxiv.org

https://arxiv.org/abs/1611.06194

arxiv.org

https://arxiv.org/abs/1611.07725

arxiv.org

https://arxiv.org/abs/2211.04624

arxiv.org

https://substackcdn.com/image/fetch/$s_!gfq6!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcf00ae2c-e93c-467a-af25-712826836cc9_950x433.png

substackcdn.com

https://en.wikipedia.org/wiki/Independent_and_identically_distributed_random_variables

en.wikipedia.org

https://substackcdn.com/image/fetch/$s_!IPQK!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F11f29f73-d852-4686-a7e5-ba8ea232a3ac_1536x856.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!GCGn!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6962d534-c2fa-47d8-98bd-880538464115_2274x744.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!UxQS!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6490dc56-1f92-4e73-bcf0-53b1cf36cfa1_1524x760.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!gjA9!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbc3c794e-bc45-4ceb-a20f-0f0d33dea592_1948x616.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!aYEu!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ffae3483f-8a62-42b1-8c32-de4bb951731e_1986x882.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!pUiZ!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8e7616ec-350c-4bbf-adad-3a7eae044233_1708x1160.png

substackcdn.com

https://cameronrwolfe.substack.com/i/167254905/kullback-leibler-kl-divergence

cameronrwolfe.substack.com

https://cameronrwolfe.substack.com/i/175107358/proximal-policy-optimization-algorithms-1

cameronrwolfe.substack.com

https://cameronrwolfe.substack.com/p/model-merging

cameronrwolfe.substack.com

https://substackcdn.com/image/fetch/$s_!5k0p!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F407af371-d216-4215-aff0-99f5726f37ac_2258x778.png

substackcdn.com

https://cameronrwolfe.substack.com/p/nano-moe

cameronrwolfe.substack.com

https://substackcdn.com/image/fetch/$s_!Ymhf!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F63f85ed4-884b-4f96-a2c4-275586fbd7d3_2000x914.png

substackcdn.com

https://cameronrwolfe.substack.com/p/a-broad-and-practical-exposition-of-online-learning-techniques-a4cbc300dcd4

cameronrwolfe.substack.com

https://cameronrwolfe.substack.com/p/how-to-train-deep-neural-networks-over-data-streams-fdab15704e66

cameronrwolfe.substack.com

https://arxiv.org/abs/2506.13045

arxiv.org

Group Relative Policy Optimization (GRPO) Read at the source ↗

cameronrwolfe.substack.com

https://substackcdn.com/image/fetch/$s_!QiLP!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F18dc223b-89b0-4288-a5b9-fad18ada6adb_2446x588.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!q06U!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9266f619-9c28-4e83-98f5-961c6c7a2cf2_2283x637.png

substackcdn.com

https://huggingface.co/blog/NormalUhr/kl-divergence-estimator-rl-llm

huggingface.co

https://agustinus.kristia.de/blog/forward-reverse-kl/

agustinus.kristia.de

https://substackcdn.com/image/fetch/$s_!4HUl!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6f64f424-3ca8-4257-8b1e-7b805eab05a7_1212x188.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!jTKg!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F90a3f7d0-0acc-42df-bc4c-d5cd40464fa0_1340x380.png

substackcdn.com

https://cameronrwolfe.substack.com/i/167254905/deriving-the-dpo-loss

cameronrwolfe.substack.com

https://substackcdn.com/image/fetch/$s_!QEd4!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fefb26675-91a0-44a5-8919-c732b4acaabb_980x366.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!yymm!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0e226855-7c37-43ef-95f0-ddc0cf23f769_1494x444.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!pPRu!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F332da0cf-bfea-427b-933e-0ddcc9bbcae6_1235x853.png

substackcdn.com

https://arxiv.org/abs/2507.05386

arxiv.org

https://scienceqa.github.io/

scienceqa.github.io

https://textvqa.org/

textvqa.org

https://vizwiz.org/tasks-and-datasets/vqa/

vizwiz.org

https://huggingface.co/datasets/hiyouga/geometry3k

huggingface.co

https://cs.stanford.edu/people/dorarad/gqa/about.html

cs.stanford.edu

https://arxiv.org/abs/2003.10286

arxiv.org

https://lizw14.github.io/project/2023_SuperCLEVR/

lizw14.github.io

https://substackcdn.com/image/fetch/$s_!CJfe!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Feabdbf11-60ef-4629-8ba2-6af7b9d73a9a_1466x573.png

substackcdn.com

https://mmmu-benchmark.github.io/

mmmu-benchmark.github.io

https://arxiv.org/abs/2406.01574

arxiv.org

https://arxiv.org/abs/2305.10355

arxiv.org

https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct

huggingface.co

https://cameronrwolfe.substack.com/i/173306894/reinforce-leave-one-out-rloo-2

cameronrwolfe.substack.com

https://arxiv.org/abs/2310.10505

arxiv.org

https://substackcdn.com/image/fetch/$s_!U58y!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fae0c71f6-dca9-43cf-87aa-bbbe365f115a_1392x748.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!-y6i!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf8c4337-9448-4614-8e2c-8c924f63098f_1836x1046.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!HuxB!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5ef7cf1a-38ca-42a7-89d6-7e5b063c1aee_1828x466.png

substackcdn.com

https://arxiv.org/abs/2510.18874

arxiv.org

https://arxiv.org/abs/2311.07911

arxiv.org

https://arxiv.org/abs/2009.03300

arxiv.org

https://github.com/Jiayi-Pan/TinyZero

github.com

https://github.com/hendrycks/math

github.com

https://arxiv.org/abs/2406.18510

arxiv.org

https://arxiv.org/abs/2406.18495

arxiv.org

https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct

huggingface.co

https://rlhfbook.com/c/10-rejection-sampling

rlhfbook.com

https://substackcdn.com/image/fetch/$s_!KLXE!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5d87d634-88de-455e-ac16-fac9c0a820b2_1174x1180.png

substackcdn.com

https://en.wikipedia.org/wiki/Pareto_front

en.wikipedia.org

https://substackcdn.com/image/fetch/$s_!FdWJ!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8dbac20b-abe7-444e-bf79-ba1d0a318a06_1204x582.png

substackcdn.com

REINFORCE: Easy Online RL for LLMs Read at the source ↗

cameronrwolfe.substack.com

https://substackcdn.com/image/fetch/$s_!A0Eh!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6d6891fa-b030-4bd6-8aec-6659dc4c5a25_1598x1138.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!Ouym!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F93215854-0559-4640-a298-a1894b18e851_1594x1032.png

substackcdn.com

https://arxiv.org/abs/2509.04259

arxiv.org

https://substackcdn.com/image/fetch/$s_!ip1n!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2f357d29-a717-4073-a60b-cbc53a89d74f_1738x802.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!I2iJ!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46334012-3aea-46b7-8292-e097e406bb00_2110x344.png

substackcdn.com

https://huggingface.co/Qwen/Qwen2.5-3B-Instruct

huggingface.co

https://arxiv.org/abs/2503.24290

arxiv.org

https://arxiv.org/abs/2306.05301

arxiv.org

https://arxiv.org/abs/2406.09098

arxiv.org

https://arxiv.org/abs/1905.07830

arxiv.org

https://arxiv.org/abs/2109.07958

arxiv.org

https://arxiv.org/abs/1907.10641

arxiv.org

https://arxiv.org/abs/2107.03374

arxiv.org

https://substackcdn.com/image/fetch/$s_!qbOa!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2278d6ce-a052-48a4-b671-2da4ff50c95a_2246x816.png

substackcdn.com

https://en.wikipedia.org/wiki/MNIST_database

en.wikipedia.org

https://arxiv.org/abs/1708.07747

arxiv.org

https://cameronrwolfe.substack.com/p/easily-train-a-specialized-llm-peft

cameronrwolfe.substack.com

https://substackcdn.com/image/fetch/$s_!vTCx!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F83d6ac79-a6f8-46c4-a002-4a328a8e62bc_2212x1003.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!QcLQ!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F58ffa6a9-a3c3-4d20-b406-bdda17891c86_1840x882.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!S8ov!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F550e03d6-70e2-4636-acdc-1c10fac5e82b_1820x292.png

substackcdn.com

https://arxiv.org/abs/2601.02151

arxiv.org

https://substackcdn.com/image/fetch/$s_!52UY!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1981c7af-d1d0-4667-9632-6835f55d9c43_2608x1094.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!y4LD!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7f527807-a084-4e13-be45-9206ea761a41_1340x1106.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!nydC!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F02b36936-f4e2-4647-97d4-08ce0a0138e9_1916x913.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!ksfZ!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4f087448-711c-4f84-8a7e-4816673f3220_1132x1166.png

substackcdn.com

http://faculty.bicmr.pku.edu.cn/~dongbin/Publications/numina_dataset.pdf

faculty.bicmr.pku.edu.cn

https://arxiv.org/abs/2502.17387

arxiv.org

https://arxiv.org/abs/2504.13941

arxiv.org

https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507

huggingface.co

https://substackcdn.com/image/fetch/$s_!LQc2!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7477fc70-c4dd-408d-8dbe-84db2fc1d817_2263x1217.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!ha_n!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3f287cdb-5305-4abd-a2d1-7c081df8cf82_1722x1450.png

substackcdn.com

https://huggingface.co/meta-llama/Llama-3.2-11B-Vision

huggingface.co

https://substackcdn.com/image/fetch/$s_!PgSy!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4edcbf38-9810-4319-8dcd-70a2b9ae6e23_1374x1416.png

substackcdn.com

https://substackcdn.com/image/fetch/$s_!1qsp!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc13f164c-7142-4dac-a7a7-f93b9668fd37_1624x1196.png

substackcdn.com

https://huggingface.co/Qwen/Qwen2.5-1.5B

huggingface.co

https://substackcdn.com/image/fetch/$s_!gp-L!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0f7b1c9b-d8aa-44fe-9f42-2dcb07372352_2700x1374.png

substackcdn.com

https://openreview.net/forum?id=I6GzDCne7U

openreview.net

https://arxiv.org/abs/2503.18892

arxiv.org

https://substackcdn.com/image/fetch/$s_!qxVz!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F371e172e-4696-496f-ad69-df1be408d98b_1876x808.png

substackcdn.com

https://cameronrwolfe.substack.com/i/169926007/preference-fine-tuning-of-llms-should-leverage-suboptimal-on-policy-data-7

cameronrwolfe.substack.com

https://substackcdn.com/image/fetch/$s_!crTD!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F308a6aad-0ce2-4777-a880-c32a7d220fcc_1864x1516.png

substackcdn.com

https://arxiv.org/abs/2505.22617

arxiv.org

https://en.wikipedia.org/wiki/Occam%27s_razor

en.wikipedia.org

https://en.wikipedia.org/wiki/Principle_of_maximum_entropy

en.wikipedia.org

On-Policy Distillation Read at the source ↗

thinkingmachines.ai