?
Click "Start Demo" to see how GRPO optimizes policy through within-group sampling and comparison
❓ Input Question x
Math problem: Compute the exact value of ∫₀¹ x²·eˣ dx
Correct answer: e - 2 ≈ 0.7183
🎲 Sample G=8 responses {y₁, y₂, ..., y₈} ~ π_θ(·|x)
📐 Within-Group Normalization: Â_i = (r_i - μ) / σ
μ = (1/G) Σ r_i = ?
σ = √(Var(r)) = ?
 > 0 → Reinforce ↑
 < 0 → Suppress ↓
Group mean as baseline, replacing PPO's Critic model — zero parameters, zero training error
📊 Policy Probability Update π_θ(y|x)
⚖️ GRPO vs PPO Comparison
| Dimension | PPO | GRPO | Improvement |
|---|---|---|---|
| Model Count | Policy + Critic + Ref | Policy + Ref | 1 fewer Critic |
| Memory Usage | ≈ 3× model size | ≈ 1.5× model size | Saves ~50% |
| Baseline Source | Critic network (estimation error) | Group mean (non-parametric statistic) | More stable |
| Hyperparameters | GAE λ, Critic LR, ... | clip ε, KL β, G | Fewer & easier to tune |
| Cost | 1 response per question | G responses per question | Sampling cost ↑ |