INTELLECT-3 technical report

Context. Open-weight models trained with RL on verifiable rewards are now strong at math and code, but the machinery that trains them, the asynchronous trainer, the environmentspackages that pair a dataset of tasks with a loop that runs the model and its tools on a task to a stopping condition, and a rubric of weighted reward functions; built here with the verifiers library and their reward checks, the code sandboxes and the evaluations, has mostly stayed inside the groups that built it or in monolithic frameworks. The INTELLECT-3 report documents one such pipeline end to end and releases it: a 106B-parameter mixture-of-expertsa model whose layers send each token to a few of many expert subnetworks, so only part of the parameters are used per token; here 12B of 106B model post-trained from GLM-4.5-Air-Basethe released pretrained 106B mixture-of-experts model of the GLM-4.5 family, before any post-training by supervised fine-tuning and then RL on math, code, science, logic, web-search and software-engineering environments, on up to 512 H200an NVIDIA data-center GPU with 141 GB of memory GPUs, together with the RL framework prime-rl, the environment library verifiers with its public Environments Hub, and a sandbox service. Most of the engineering goes into keeping generation, tool execution and training busy at the same time while controlling two gaps between the policy that sampled a token and the policy being trained: staleness, since a rolloutone sampled attempt at a task: the model's tokens and any tool calls and results, up to a stopping condition may be generated by up to 8 policy versions, and numerical differences between the inference engine and the trainer. One mechanism handles both, a per-token importance ratio taken against the inference engine's own probabilities, with tokens outside a fixed range dropped; the numerical gap is the one that led the study of RL scaling curves (Khatri et al., 2026) to compute the output layer in 32-bit precision.

The recipe has two stages. Supervised fine-tuning on reasoning traces generated by DeepSeek-R1-0528, then on agentic tool-use and software-engineering traces, brings the model to about 85% on AIME 2024 and 2025 before RL (read off the paper's training curves). RL then runs about 600 steps (the length of those curves; the text gives no step count) of 256 prompts with 16 rollouts each at a context of 65,536 tokens and adds a few points on the reasoning benchmarks tracked, by the same curves about 3 to 5 on AIME and 7 on LiveCodeBench, still rising when training stopped. The final model scores 90.8 on AIME 2024, 88.0 on AIME 2025, 69.3 on LiveCodeBench v6 and 74.4 on GPQA Diamond, above GLM-4.5-Air, the same base model with its makers' own post-training, on all six benchmarks reported. The two system choices the report singles out are in-flight weight updatesthe inference servers pause to load new weights as soon as the trainer has them and continue every unfinished rollout with them, which more than halve the time of a step, and the masked importance ratio, which the authors found necessary to keep runs from crashing days in.

Setting. Training is split across three services. An orchestratora CPU process that sends rollout requests to the inference servers, scores the results through the environments, packs them into training batches, and passes new weights from the trainer to inference sits between a trainer, built on FSDP2PyTorch's fully sharded data parallelism: each GPU holds a slice of the parameters, gradients and optimizer state and gathers whole layers when it needs them, on 16 nodes of 8 H200s, and a pool of vLLMan open-source inference engine for language models, built around paged KV-cache memory servers on 44 nodes. For a prompt $x$, the inference pool samples $G$the number of rollouts per prompt, 16 rollouts $y_1,\dots,y_G$ with rewards $r_i$, and every token of $y_i$ gets the advantage $\hat A_i$$r_i-\bar r$: the rollout's reward minus the mean reward $\bar r$ of the prompt's group, with no division by the standard deviation. Write $\pi_{\rm infer}$the token probabilities the inference engine used when it sampled the rollout, returned by vLLM with each token and $\pi_{\rm train}(\cdot\,;\theta)$the trainer's token probabilities at its current weights $\theta$ for the inference engine's and the trainer's token probabilities respectively, and $\rho_{i,t}$$\pi_{\rm train}(y_{i,t}\mid x,y_{i,\lt t};\theta)/\pi_{\rm infer}(y_{i,t}\mid x,y_{i,\lt t})$ for token $t$ of rollout $i$ for their ratio, $\rho_{i,t}=\pi_{\rm train}(y_{i,t}\mid x,y_{i,\lt t};\theta)/\pi_{\rm infer}(y_{i,t}\mid x,y_{i,\lt t})$, at token $t$ of rollout $i$. The objective keeps the ratio only inside a window: $$\begin{aligned}J(\theta)&=\mathbb E\Big[\tfrac{1}{\sum_i|y_i|}\\&\qquad\times\textstyle\sum_{i,t}M(\rho_{i,t})\,\hat A_i\Big],\\M(\rho)&=\begin{cases}\rho,&\rho\in[\alpha,\beta],\\0,&\text{otherwise},\end{cases}\end{aligned}$$ where the sums run over the rollouts of a batch and their tokens, $|y_i|$ is the length of $y_i$ in tokens, and $\alpha=0.5,\ \beta=5$why: a ratio far from 1 means the trainer and the inference engine disagree on the token, from staleness or from numerical mismatch; those tokens give noisy updates, so they are dropped. On a token inside the window the gradient is $\rho_{i,t}\hat A_i\nabla\log\pi_{\rm train}(y_{i,t}\mid x,y_{i,\lt t};\theta)$, an importance-weighted policy gradient, and outside it the gradient is zero. A whole rollout is dropped if any of its tokens has a ratio below $10^{-5}$. This is IcePopmasked token-level importance sampling (Zhao et al., 2025): weight each token's update by its trainer-to-inference ratio and exclude tokens whose ratio falls outside a window, close to CISPOclipped importance sampling policy optimization: policy gradient with each token's importance weight capped at a fixed value and treated as a constant but with masking where CISPO caps the weight. The orchestrator also discards rollouts generated by more than 8 policy versionswhy: in-flight updates let one rollout span several weight versions; past 8 the data are too far off-policy and are discarded to prevent policy drift. The optimizer is Muonan optimizer that replaces each weight matrix's momentum by the nearest orthogonal matrix, computed with a few Newton-Schulz iterations, at learning rate $10^{-6}$ in RL.

Main findings (all empirical). On AIME 2024 and 2025 (mean accuracy over 32 samples per problem), LiveCodeBench v6 (2 samples), GPQA Diamond (4 samples), the text-only part of Humanity's Last Exam and MMLU-Pro, INTELLECT-3 scores 90.8, 88.0, 69.3, 74.4, 14.6 and 81.9, against 84.6, 82.0, 61.5, 73.3, 13.3 and 73.9 for GLM-4.5-Air. GLM-4.5, about 3 times larger, is behind on the AIME sets and LiveCodeBench (85.8, 83.3, 64.5) and ahead on GPQA, Humanity's Last Exam and MMLU-Pro (77.0, 14.8, 83.5); GLM-4.6, also about 3 times larger, is ahead on AIME 2024, AIME 2025, LiveCodeBench, GPQA and MMLU-Pro (92.0, 90.3, 73.0, 78.8, 83.1); DeepSeek-R1-0528 is behind on both AIME sets (83.2, 73.4). In the online evaluations during RL the benchmarks generally trend upward to the last step.

What failed along the way. vLLM's own multi-node data parallelism stopped gaining throughput as inference nodes were added (Step 1). Runs crashed several days in from the gap between trainer and inference probabilities until tokens with extreme ratios were masked, and GSPOgroup sequence policy optimization: a clipped objective with one importance ratio per rollout, the length-normalized ratio of its sequence probabilities, in place of token ratios collapsed under staleness (Step 2). Rule-based answer checking marked a noticeable share of correct math answers wrong (Step 3). Commands sent to sandboxes through the API of Kubernetesthe standard system for running containers across a cluster; its API server records every operation in a central key-value store took up to 2.5 s each at thousands of concurrent sandboxes (Step 4). Context parallelismsplitting the attention computation of one long sequence across several GPUs reached 256K tokens but halved data parallelism and lost accuracy, asynchronous activation offloadingmoving the activations kept for the backward pass from GPU to CPU memory and back leaked memory, round-robin gathering for distributed Muon congested the network, and expert parallelismplacing different experts of a mixture-of-experts layer on different GPUs and routing tokens between them lowered throughput (Step 5).

Consequences and limits. The practice the report recommends for agentic RL: package each environment with its data, rollout loop and reward so that it can be tested apart from the trainer; run trainer and inference on separate GPUs with continuous batching and in-flight weight updates; weight each token's update by the trainer's probability over the one the inference engine recorded, dropping tokens with ratios far from 1; keep code execution off the cluster's control plane; and filter prompts by difficulty online. The evidence for most of this is the run itself. There is one training run and no ablation of the environment mix, the SFT data or the RL objective beyond the GSPO test and the step-time comparison. No score is reported for the SFT checkpoint apart from the starting points of the RL curves, which place most of the final score in the SFT stage, trained on DeepSeek-R1-0528's traces. The comparisons are the authors' own evaluations apart from two copied entries (Step 7), and the AIME numbers use rule-based checking only, which they note is stricter than leaderboards that use an LLM judge. Web-search and software-engineering environments were in the RL mix, but no agentic benchmark is reported.