corpus.blog
56,966 blogs · 4,417,066 posts Claim your blog
Record · corpus.blog/posts/01a0df50-42a2-7315-a56c-0dc7be96fcbc

Reinforcement Learning for LLMs: The Complete Guide

cameronrwolfe.substack.com · published 24 August 2026

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at cameronrwolfe.substack.com Wayback capture
1blogs citing
First cited
26 Aug 2026
Most recent
26 Aug 2026
Rank this month
not ranked
Links from this post
151
Words captured
19,794

Cited by

Blog
In the post
Date
AI for Software Engineers | Logan Thorneloe | Substack
The Future of Agent Impact on Media and Research | AI for Software Engineers 85 Read at the source ↗
26 Aug 2026

Links from this post

Link
Host
https://substackcdn.com/image/fetch/$s_!Hl0G!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc1e6c2d9-5c68-4614-9b21-3d1d4f943318_2276x1270.png
substackcdn.com
https://cameronrwolfe.substack.com/p/rlaif-reinforcement-learning-from
cameronrwolfe.substack.com
https://rlhfbook.com/
rlhfbook.com
https://web.stanford.edu/class/psych209/Readings/SuttonBartoIPRLBook2ndEd.pdf
web.stanford.edu
https://spinningup.openai.com/en/latest/
spinningup.openai.com
https://sebastianraschka.com/llms-from-scratch/
sebastianraschka.com
https://sebastianraschka.com/reasoning-from-scratch/
sebastianraschka.com
Approximating KL Divergence Read at the source ↗
joschu.net
https://arxiv.org/abs/1707.06347
arxiv.org
Policy Gradient Algorithms Read at the source ↗
lilianweng.github.io
https://yugeten.github.io/posts/2025/01/ppogrpo/
yugeten.github.io
https://lancelqf.github.io/note/llm_post_training/
lancelqf.github.io
https://yumoxu.notion.site/async-grpo-in-the-wild
yumoxu.notion.site
https://huggingface.co/docs/trl/en/index
huggingface.co
https://github.com/allenai/open-instruct
github.com
https://substackcdn.com/image/fetch/$s_!g3OR!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbce235e8-06fd-48fb-9f85-507aae7ad9ff_2198x856.png
substackcdn.com
Agentic RL: Frameworks and Best Practices Read at the source ↗
cameronrwolfe.substack.com
https://substackcdn.com/image/fetch/$s_!JG-8!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7bd0964e-9fae-4259-8e82-4874a05eb883_2182x246.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!PeFU!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1b18b80b-c923-4d8b-9cb5-ffea927f8496_1880x578.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!Pw9Q!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdaf22462-b340-462a-92f7-a8cc47fde7fa_1392x780.png
substackcdn.com
https://cameronrwolfe.substack.com/i/167254905/kullback-leibler-kl-divergence
cameronrwolfe.substack.com
https://cameronrwolfe.substack.com/i/175107358/generalized-advantage-estimation-gae-3
cameronrwolfe.substack.com
https://substackcdn.com/image/fetch/$s_!dAbX!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc999288b-0060-4b6e-9a88-237e4e9cec85_1530x564.png
substackcdn.com
https://en.wikipedia.org/wiki/Expected_value
en.wikipedia.org
https://substackcdn.com/image/fetch/$s_!9Q_M!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe60c0356-fc85-4a6c-be00-edaab849356b_2180x378.png
substackcdn.com
https://en.wikipedia.org/wiki/Monte_Carlo_method
en.wikipedia.org
https://substackcdn.com/image/fetch/$s_!7b9l!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa6ac44b0-d9b8-44d2-9545-4fe9376ac4fc_1244x486.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!Yopx!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff45af077-4abe-4e7f-bde8-3d3487e47a7f_1649x218.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!pA_5!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F21119ab5-6885-462c-a73b-bd845e220e42_2324x668.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!dYii!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Feafa9126-416f-4b6e-a5ce-6d38e59c2f4f_2104x217.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!gXY5!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2ffe5ebb-fa92-4e19-841c-ca12ede5c827_1357x218.png
substackcdn.com
https://spinningup.openai.com/en/latest/spinningup/rl_intro.html
spinningup.openai.com
Reward Models Read at the source ↗
cameronrwolfe.substack.com
https://cameronrwolfe.substack.com/p/decoder-only-transformers-the-workhorse
cameronrwolfe.substack.com
https://substackcdn.com/image/fetch/$s_!fXOv!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ffb8133ba-f772-44f5-bfbc-19e800a842cc_1732x570.png
substackcdn.com
https://en.wikipedia.org/wiki/Gradient_descent
en.wikipedia.org
https://substackcdn.com/image/fetch/$s_!slrY!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff3072897-d905-42be-b385-6186c24ae059_2390x302.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!j0Id!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F00d609c2-0276-4102-914e-7de5d6a5326e_1404x1012.png
substackcdn.com
http://natebrake.com/blog/end-of-sequence-explained
natebrake.com
https://substackcdn.com/image/fetch/$s_!fToU!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc29b229e-c2cf-4317-825b-71eeb79d47f9_1920x834.png
substackcdn.com
https://en.wikipedia.org/wiki/Markov_decision_process
en.wikipedia.org
https://substackcdn.com/image/fetch/$s_!5jkF!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fef8963ed-8a48-47d1-9b80-72337857e882_1626x1004.png
substackcdn.com
https://www.baeldung.com/cs/rl-deterministic-vs-stochastic-policies
baeldung.com
https://substackcdn.com/image/fetch/$s_!ooQL!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F117ac81a-c631-4209-8ea0-09051c63180d_2386x462.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!sC0E!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0e46ae90-bc33-4cd1-9be5-31d9bb0bd6a4_2274x266.png
substackcdn.com
https://en.wikipedia.org/wiki/Multi-armed_bandit
en.wikipedia.org
Reward Models Read at the source ↗
cameronrwolfe.substack.com
https://substackcdn.com/image/fetch/$s_!CJn6!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc0fd3791-df29-4a92-b185-21f6be4f2ddc_2176x642.png
substackcdn.com
https://github.com/huggingface/Math-Verify
github.com
https://substackcdn.com/image/fetch/$s_!5por!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff4cb441a-d31b-4411-83df-219e9d4371a9_1812x658.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!Ai7-!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf48e511-7418-4110-bd58-3e362067efef_1972x368.png
substackcdn.com
https://rlhfbook.com/c/15-regularization
rlhfbook.com
https://en.wikipedia.org/wiki/Information_theory
en.wikipedia.org
https://cameronrwolfe.substack.com/i/183759600/more-on-sft-and-rl
cameronrwolfe.substack.com
https://substackcdn.com/image/fetch/$s_!MIIg!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F298d0b86-f69b-42c3-87fd-ac8b88f6ba74_1456x510.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!Dq9_!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd34fe4d1-4f05-4eee-8591-52a0f58e08f7_2118x548.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!LMDY!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7b9c421d-d2b1-41e1-874c-d67000f9a958_1252x116.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!vxPz!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F20bfb738-79d4-4799-8059-21aacaea0073_1318x548.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!gZ9p!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0f87e9c8-8448-433f-9688-558f3400b5d3_1390x944.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!iEKF!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb69437bf-88b3-4485-b263-f2828f40db17_2288x762.png
substackcdn.com
https://ionides.github.io/pubs/ionides08-jcgs.pdf
ionides.github.io
https://fengyao.notion.site/off-policy-rl
fengyao.notion.site
https://substackcdn.com/image/fetch/$s_!Jir3!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4f879238-ff1c-4bfd-8b28-ca70e4f262b1_1608x468.png
substackcdn.com
https://rlhfbook.com/c/06-policy-gradients
rlhfbook.com
https://substackcdn.com/image/fetch/$s_!ghIM!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb3c62518-ebaf-41f1-9d1f-9b883be40f7f_1546x666.png
substackcdn.com
https://andrewcharlesjones.github.io/journal/log-derivative.html
andrewcharlesjones.github.io
https://substackcdn.com/image/fetch/$s_!WK3g!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F35a6e9f1-5b7e-4d1b-8de2-17464095e0d2_1864x748.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!Rkmm!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb0f526be-55f2-4eae-abd8-fa4382d8335a_1564x432.png
substackcdn.com
https://spinningup.openai.com/en/latest/spinningup/rl_intro3.html
spinningup.openai.com
https://en.wikipedia.org/wiki/Automatic_differentiation
en.wikipedia.org
https://substackcdn.com/image/fetch/$s_!TwP0!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa4bb2d85-fdea-4cfc-a46b-e6c5f78ff4f4_1613x593.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!Vl-C!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F58aa8bae-6778-4ec0-ac53-3f8b8550390f_2137x836.png
substackcdn.com
https://cameronrwolfe.substack.com/i/137421286/variants-of-the-basic-policy-gradient
cameronrwolfe.substack.com
https://substackcdn.com/image/fetch/$s_!CEl_!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe2a85cb1-2c52-462d-ad79-fb68378a06b6_1708x564.png
substackcdn.com
https://rlhfbook.com/c/11-policy-gradients.html
rlhfbook.com
https://substackcdn.com/image/fetch/$s_!rHTX!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdc872176-fcb3-4646-86a1-999df93df4bf_1502x386.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!yDWw!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa8725dec-4bf1-4790-89cd-4287d7fbbf33_1978x448.png
substackcdn.com
https://en.wikipedia.org/wiki/Moving_average
en.wikipedia.org
https://cameronrwolfe.substack.com/i/173306894/back-to-basics-revisiting-reinforce-style-optimization-for-learning-from-human-feedback-in-llms-3
cameronrwolfe.substack.com
https://substackcdn.com/image/fetch/$s_!PhLf!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F34e3a747-c449-4e12-8d65-f1b86762e3b6_1282x634.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!88ua!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8d99e810-14dc-49dd-9a71-e441ba5a94fd_2278x384.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!VIDL!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8bf32dae-2f08-46f4-ac67-0f9fbafd0a53_1730x916.png
substackcdn.com
https://courses.csail.mit.edu/6.803/pdf/steps.pdf
courses.csail.mit.edu
https://rlhfbook.com/c/11-policy-gradients
rlhfbook.com
https://substackcdn.com/image/fetch/$s_!8KwS!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F193a7b06-64ee-4be1-b8b2-02849548b5bf_2186x566.png
substackcdn.com
https://cameronrwolfe.substack.com/i/173306894/reinforce-leave-one-out-rloo-2
cameronrwolfe.substack.com
https://substackcdn.com/image/fetch/$s_!bXO6!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3ce1014d-89a2-4527-af5d-dd79364ac2ba_1372x814.png
substackcdn.com
https://arxiv.org/abs/2503.20783
arxiv.org
https://arxiv.org/abs/2501.03262
arxiv.org
PPO for LLMs: A Guide for Normal People Read at the source ↗
cameronrwolfe.substack.com
https://substackcdn.com/image/fetch/$s_!_teA!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9fe9f660-df4a-4a30-96db-e9cf5f464ed7_2192x576.png
substackcdn.com
https://en.wikipedia.org/wiki/Mean_squared_error
en.wikipedia.org
https://substackcdn.com/image/fetch/$s_!x5A5!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1a9c1514-c3dd-4692-bb7a-d63644987d5e_1784x940.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!IXsZ!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4a7d1530-a2cc-48c6-9e95-8571b781ba35_1994x792.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!XHd7!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9affa264-f085-4496-a0c5-bd9a247716df_1870x1240.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!fqSm!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F80447ac5-6fd2-4cbb-b33c-a4e385e7fc2c_1390x478.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!oHJG!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7f6be9f2-f165-4e48-be0c-e63074454d2a_2003x338.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!S1nc!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc38f9ea3-d07f-4240-898e-de3c75e66878_2264x786.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!6rr4!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1339ce1e-54ed-4d82-ac48-72fa857517e9_4155x3893.png
substackcdn.com
https://cameronrwolfe.substack.com/i/175107358/generalized-advantage-estimation-gae
cameronrwolfe.substack.com
https://substackcdn.com/image/fetch/$s_!A4K-!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4c1e98c7-da70-4da6-a365-3b2fe9cd2230_1723x896.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!_U8s!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F18ae75ed-997b-4654-b383-dda56a8d9b2e_2298x716.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!v3wn!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff11ed641-c3be-442a-ad17-b41072a721a8_2015x843.png
substackcdn.com
https://arxiv.org/abs/2402.03300
arxiv.org
https://substackcdn.com/image/fetch/$s_!dzfC!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7d12056e-a139-4bd9-bb4b-00fee858ad9c_2718x1308.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!nguf!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F97afd2cb-5a22-4990-a470-4f5bdebb8a53_2124x871.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!6kXE!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9ce0fb53-a64d-4225-9f84-acf312c16c06_2475x763.png
substackcdn.com
https://huggingface.co/Qwen/Qwen3-32B
huggingface.co
https://substackcdn.com/image/fetch/$s_!Evsv!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F93d1c6b3-e657-4d5a-839d-61c02456a157_1910x1042.png
substackcdn.com
https://modal.com/blog/how-much-vram-need-fine-tuning
modal.com
https://arxiv.org/abs/2305.17333
arxiv.org
https://cameronrwolfe.substack.com/p/moe-llms
cameronrwolfe.substack.com
https://substackcdn.com/image/fetch/$s_!mUnt!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9b72d974-8494-49fc-a763-5836c05b3150_1320x766.png
substackcdn.com
https://en.wikipedia.org/wiki/Geometric_mean
en.wikipedia.org
https://substackcdn.com/image/fetch/$s_!qwpX!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1b2b2caa-32a8-4bd3-ab98-eee49cad4ad7_592x218.png
substackcdn.com
https://huggingface.co/Qwen/Qwen3-235B-A22B
huggingface.co
https://arxiv.org/abs/2505.09388
arxiv.org
https://substackcdn.com/image/fetch/$s_!SNyV!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb039aeac-b8b3-442f-8efe-7a37c6a1679d_3130x1337.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!201L!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa4484a60-e6ab-477f-89ab-9a68a851954f_1766x756.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!hW2Z!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fed791d47-2965-48f7-8fef-eca918cb7d93_2122x1339.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!gilS!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa6afc090-8b41-415a-b1eb-5e2436e10662_1702x526.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!wrhA!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcdee6e2d-9cfe-4d20-b84c-56831fe0dc43_2007x909.png
substackcdn.com
https://docs.vllm.ai/en/latest/
docs.vllm.ai
https://docs.sglang.io/
docs.sglang.io
Fully Sharded Data Parallel: faster AI training with fewer GPUs Read at the source ↗
engineering.fb.com
https://www.deepspeed.ai/training/
deepspeed.ai
https://substackcdn.com/image/fetch/$s_!YoVu!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc1f305b8-7ab7-4ea5-ac11-feb3c5a477e9_1226x836.png
substackcdn.com
https://cameronrwolfe.substack.com/i/181791956/your-efficient-rl-framework-secretly-brings-you-off-policy-rl-training-4
cameronrwolfe.substack.com
REINFORCE: Easy Online RL for LLMs Read at the source ↗
cameronrwolfe.substack.com
https://substackcdn.com/image/fetch/$s_!HUpB!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F47a9bbe0-ad07-45de-82d2-6444addf8d96_2446x694.png
substackcdn.com
https://verl.readthedocs.io/en/latest/algo/rollout_corr_math.html
verl.readthedocs.io
https://github.com/allenai/open-instruct/blob/main/open_instruct/grpo_fast.py
github.com
https://substackcdn.com/image/fetch/$s_!s35R!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F00672cb3-fa64-4514-9c6e-728659e36bc4_1908x1308.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!9naH!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2ddd67e1-c51f-4318-8bf5-47cd5f0c2498_3124x340.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!vAiI!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4ae11b48-c050-4afa-9426-81997ece3529_2482x646.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!Bw8T!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F93342927-dc01-4c14-9975-d231f256d047_2447x475.png
substackcdn.com
https://substackcdn.com/image/fetch/$s_!eN8U!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2983668c-bf0b-49f5-b375-d99d6f40291e_1278x658.png
substackcdn.com
https://arxiv.org/abs/2512.13607
arxiv.org
https://arxiv.org/abs/2509.04259
arxiv.org
https://z.ai/blog/glm-5.2
z.ai
https://cameronrwolfe.medium.com/
cameronrwolfe.medium.com
https://en.wikipedia.org/wiki/Discounting
en.wikipedia.org
https://en.wikipedia.org/wiki/Order_of_approximation
en.wikipedia.org
https://en.wikipedia.org/wiki/Conjugate_gradient_method
en.wikipedia.org
https://en.wikipedia.org/wiki/Line_search
en.wikipedia.org
https://arxiv.org/abs/1711.05101
arxiv.org
https://kaitchup.substack.com/p/fine-tuning-llms-with-32-bit-8-bit
kaitchup.substack.com
https://huggingface.co/docs/bitsandbytes/main/en/optimizers
huggingface.co
https://arxiv.org/abs/1910.02054
arxiv.org
https://arxiv.org/abs/2304.11277
arxiv.org
https://docs.pytorch.org/docs/stable/distributed.pipelining.html
docs.pytorch.org