Actor ↔ terminal

Commands & tools

Final verification

Checks P · S

FDR or SPR

Terminal reward R

GAE / TB-GAE

Reward R + values V

Critic

Prefix → V

Actor

PPO with A

Critic

Fit Ĝ = A + V
B · RPR: one trajectory supplies two training targets
Each generated prefix
V
R
A
Ĝ
The verifier computes the score; the Critic learns expected returns from each history.