corpus.blog
41,554 blogs · 4,756,248 posts Claim your blog
Record · corpus.blog/posts/01a08264-ee22-7280-b835-1a12d5ab45ae

Open Source RL Libraries for LLMs

Anyscale · published 1 July 2025

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at anyscale.com Wayback capture
1blogs citing
First cited
10 Mar 2026
Most recent
10 Mar 2026
Rank this month
not ranked
External links
69
Words captured
3,833

Cited by

Blog
In the post
Date
Hugging Face
Keep the Tokens Flowing: Lessons from 16 Open-Source RL Libraries Read at the source ↗
10 Mar 2026

Links from this post

Link
Host
https://github.com/THUDM/slime
github.com
https://openai.com/o1/
openai.com
Introducing OpenAI o3 and o4-mini Read at the source ↗
openai.com
https://arxiv.org/abs/2501.12948
arxiv.org
https://github.com/NovaSky-AI/SkyRL
github.com
http://z.ai
z.ai
https://github.com/All-Hands-AI/OpenHands
github.com
https://github.com/OpenInterpreter/open-interpreter
github.com
https://github.com/openreasoner/openr
github.com
https://github.com/openai/gym
github.com
https://github.com/huggingface/trl
github.com
https://github.com/huggingface/trl/blob/b773a4c1915c6a5a9356507bb1f443e9eeeda51b/trl/trainer/ppo_trainer.py
github.com
https://github.com/huggingface/trl/blob/15ff54790b42297d2cf569fba6d7dd44c1c269e3/trl/trainer/utils.py
github.com
https://github.com/huggingface/trl/blob/8a235a9b71f4c0b77e295afb972fdd7c19a71335/trl/trainer/grpo_trainer.py
github.com
https://github.com/volcengine/verl
github.com
https://github.com/volcengine/verl/blob/9ec260be2338e5ed08f64cf481c25a943d3e486f/verl/trainer/ppo/ray_trainer.py
github.com
https://github.com/volcengine/verl/blob/59379539a0305e4948b742262bf1ba2468213674/verl/workers/rollout/vllm_rollout/vllm_rollout.py
github.com
https://github.com/volcengine/verl/blob/83ebd007e01de29bbe353de112d04245b4820b47/verl/protocol.py
github.com
https://github.com/volcengine/verl/issues/1172
github.com
https://github.com/volcengine/verl/blob/main/verl/tools/gsm8k_tool.py
github.com
https://github.com/OpenRLHF/OpenRLHF
github.com
https://github.com/OpenRLHF/OpenRLHF/blob/d86779bb15acb3187f75f1b5ee71aaa9f975ca9d/openrlhf/trainer/ppo_trainer.py
github.com
https://github.com/OpenRLHF/OpenRLHF/blob/5974dfe8bded77e90dcdb92f08f3bdba31972ff2/openrlhf/trainer/ppo_utils/experience_maker.py
github.com
https://github.com/RAGEN-AI/RAGEN
github.com
https://github.com/RAGEN-AI/RAGEN/blob/main/ragen/trainer/agent_trainer.py
github.com
https://github.com/RAGEN-AI/RAGEN/blob/9fb8527776a8490f240417bd964e6eac75777fb0/ragen/llm_agent/agent_proxy.py
github.com
https://github.com/RAGEN-AI/RAGEN/blob/main/ragen/env/base.py
github.com
https://github.com/RAGEN-AI/RAGEN/tree/9fb8527776a8490f240417bd964e6eac75777fb0?tab=readme-ov-file
github.com
https://github.com/inclusionAI/AReaL
github.com
https://github.com/inclusionAI/AReaL/blob/f2f4b67bcd375ded638f6e1a02163b80fa60a99e/realhf/impl/model/interface/ppo_interface.py
github.com
https://github.com/inclusionAI/AReaL/blob/f2f4b67bcd375ded638f6e1a02163b80fa60a99e/realhf/impl/agent/math_multi_turn_agent.py
github.com
https://github.com/inclusionAI/AReaL/blob/b63eea9d0761868148b5f6f22b28c6e50d364719/realhf/api/core/data_api.py
github.com
https://inclusionai.github.io/AReaL/customization/agent.html
inclusionai.github.io
https://github.com/willccbb/verifiers
github.com
https://github.com/willccbb/verifiers/blob/a3b7ffcb1d247ab5a557218c8290fdafd82a138d/verifiers/trainers/grpo_trainer.py
github.com
https://github.com/willccbb/verifiers/blob/a3b7ffcb1d247ab5a557218c8290fdafd82a138d/verifiers/envs/multiturn_env.py
github.com
https://github.com/willccbb/verifiers/blob/main/verifiers/trainers/grpo_trainer.py
github.com
https://github.com/willccbb/verifiers/tree/main/verifiers/envs
github.com
https://github.com/alibaba/ROLL
github.com
https://github.com/alibaba/ROLL/blob/b756338442d9d36c1fe4c951bc98ebc913bd95ae/roll/pipeline/agentic/agentic_pipeline.py
github.com
https://github.com/alibaba/ROLL/blob/b756338442d9d36c1fe4c951bc98ebc913bd95ae/roll/agentic/rollout/rollout_scheduler.py
github.com
https://github.com/alibaba/ROLL/blob/b756338442d9d36c1fe4c951bc98ebc913bd95ae/roll/agentic/env/base.py
github.com
https://github.com/alibaba/ROLL/blob/94637ff15d6ada376f796bb693b5f7abad00cc21/roll/distributed/scheduler/protocol.py
github.com
https://alibaba.github.io/ROLL/docs/English/StepByStep/agent_pipeline_start/
alibaba.github.io
https://github.com/NVIDIA/NeMo-RL
github.com
https://github.com/NVIDIA-NeMo/RL/blob/0fee015f03fa959b04a236fb656e4e5f8548e277/nemo_rl/algorithms/grpo.py
github.com
https://github.com/NVIDIA/NeMo-RL/blob/99ba9a130e72cbf87a3e20acf43bc01a47adc8ee/docs/design-docs/generation.md
github.com
https://github.com/NVIDIA/NeMo-RL/blob/main/nemo_rl/environments/interfaces.py
github.com
https://github.com/NVIDIA-NeMo/RL/blob/ebd35a342a509f6a3ba832e699d440ad08a59ec4/nemo_rl/algorithms/grpo.py
github.com
https://github.com/NVIDIA/NeMo-RL/blob/main/nemo_rl/environments/math_environment.py
github.com
https://github.com/NovaSky-AI/SkyRL/blob/b3280d8440f4973fd96229e3e5d3ed2372534812/skyrl-train/skyrl_train/trainer.py
github.com
https://github.com/NovaSky-AI/SkyRL/blob/b3280d8440f4973fd96229e3e5d3ed2372534812/skyrl-train/skyrl_train/generators/base.py
github.com
https://github.com/NovaSky-AI/SkyRL/blob/b3280d8440f4973fd96229e3e5d3ed2372534812/skygym/skygym/core.py
github.com
https://github.com/NovaSky-AI/SkyRL/blob/b3280d8440f4973fd96229e3e5d3ed2372534812/skyrl-train/skyrl_train/training_batch.py
github.com
https://skyrl.readthedocs.io/en/latest/tutorials/new_env.html
skyrl.readthedocs.io
https://github.com/THUDM/slime/blob/cbe57433fd0f932b28544a55fd13019f085c98d3/slime/backends/megatron_utils/actor.py
github.com
https://github.com/THUDM/slime/blob/0b96b53cc412895503b876a01bf6e8f3fe4bab7e/slime/rollout/sglang_rollout.py
github.com
https://github.com/THUDM/slime/blob/0b96b53cc412895503b876a01bf6e8f3fe4bab7e/slime/utils/types.py
github.com
https://github.com/THUDM/slime/tree/main/examples/search-r1
github.com
https://github.com/TIGER-AI-Lab/verl-tool
github.com
https://github.com/agentica-project/verl-pipeline
github.com
https://github.com/volcengine/verl?tab=readme-ov-file
github.com
https://openrlhf.readthedocs.io/en/latest/async_rl.html
openrlhf.readthedocs.io
https://github.com/Open-Reasoner-Zero/Open-Reasoner-Zero
github.com
https://github.com/TsinghuaC3I/MARTI
github.com
https://github.com/alibaba/ROLL/blob/b756338442d9d36c1fe4c951bc98ebc913bd95ae/assets/ROLL_map_high.png
github.com
https://github.com/NovaSky-AI/SkyRL/tree/main/skyrl-train/examples
github.com
https://tyler-griggs.github.io/
tyler-griggs.github.io
https://www.linkedin.com/in/philipp-moritz-61419682/
linkedin.com
Elsewhere on anyscale.com (1)