{
  "slug": "self-distillation-data",
  "category": "Data production",
  "order": 88,
  "icon": 4,
  "fitExpanded": true,
  "title": [
    "自蒸馏时代，数据生产什么",
    "What Data Does Self-Distillation Need?",
    "自己蒸留の時代に、どんなデータを作るか"
  ],
  "summary": [
    "从专家答案到训练反馈，比较三条工业路线与六种蒸馏方法。",
    "From expert answers to training feedback: three industrial recipes and six distillation methods.",
    "専門家の回答から学習のフィードバックへ。3つの実用例と6つの蒸留手法。"
  ],
  "lead": [
    "模型可以从自己的回答中学习，但它仍需要好问题、有用的反馈，以及独立的检验。自蒸馏改变了这些数据进入训练的位置，也改变了数据生产的重点。",
    "A model can learn from its own answers. It still needs good questions, useful feedback, and independent tests. Self-distillation changes where these data enter training—and what is worth producing.",
    "モデルは自分の回答から学べます。それでも、良い問い、有用なフィードバック、独立した評価が必要です。自己蒸留は、データを学習に渡す場所と、作るべきデータの重点を変えます。"
  ],
  "sections": [
    {
      "title": [
        "从压缩模型，到积累能力",
        "From compression to capability building",
        "モデル圧縮から能力の蓄積へ"
      ],
      "paragraphs": [
        [
          "蒸馏最直观的用途，是让一个小模型学习大模型。现在，它也被用来汇合多个专家的能力、找回上一阶段学会的技能，或把提示词里的知识留在参数中。教师不一定更大：它可能是领域专家、旧检查点，也可能只是多看了一份解答的同源模型。",
          "Distillation can teach a small model to imitate a larger one. It can also combine specialists, recover skills from an earlier training stage, or internalize knowledge supplied in a prompt. A teacher need not be larger: it may be a specialist, a previous checkpoint, or a related model given a worked solution.",
          "蒸留は小さなモデルに大きなモデルを模倣させるだけでなく、専門能力の統合、過去に学んだ技能の回復、プロンプト内の知識の取り込みにも使われます。教師は大きいモデルとは限らず、専門家、過去のチェックポイント、解答を追加で与えた同系モデルでも構成できます。"
        ],
        [
          "理解一种方法，先问两个问题：训练轨迹是谁生成的？教师比学生多知道什么？前者决定反馈落在哪些状态上，后者决定模型能学到什么新东西。",
          "Two questions clarify a method: who generates the training trajectory, and what does the teacher know that the student does not? The first determines where supervision applies; the second determines what can be learned.",
          "手法を理解するには、学習軌跡を誰が生成するか、教師だけが何を知っているかを確認します。前者は指導が行われる状態を、後者は新たに学べる内容を決めます。"
        ]
      ]
    },
    {
      "title": [
        "DeepSeek：先积累专家答案",
        "DeepSeek: collect specialist answers",
        "DeepSeek：専門家の回答を集める"
      ],
      "paragraphs": [
        [
          "DeepSeek-V3.2 从共同的基础模型出发，分别训练数学、代码、Agent 等领域专家。专家生成各领域数据，用于蒸馏统一模型；随后再用混合 RL 同时优化推理、Agent 和人类偏好。这里先积累的是专家写出的训练答案，混合 RL 是后续独立的一步。",
          "DeepSeek-V3.2 trains specialists from a common base model. Their domain-specific outputs supply data for a unified model. A subsequent mixed-RL stage jointly optimizes reasoning, agent tasks, and human alignment. Specialist-generated answers and the later RL stage play distinct roles.",
          "DeepSeek-V3.2は共通の基盤モデルから分野別の専門家を学習し、その出力を統合モデルの学習データにします。その後、推論、Agent、人間の選好を混合RLで最適化します。専門家が生成する回答と、その後のRLは別の段階です。"
        ]
      ],
      "figure": "specialists",
      "refs": [
        0
      ]
    },
    {
      "title": [
        "MiMo：沿学生的回答给反馈",
        "MiMo: teach along the student’s path",
        "MiMo：学生がたどる経路を教える"
      ],
      "paragraphs": [
        [
          "MiMo-V2-Flash 同样先训练领域专家，MOPD 阶段则由学生自己生成轨迹。专家读取学生已经生成的前缀，给出逐 token 的分布监督；结果奖励模型 ORM 还能提供整条回答的奖励。训练需要保存学生实际走过的轨迹，并让教师在这些位置上评分。",
          "MiMo-V2-Flash also trains domain specialists, but MOPD uses student-generated trajectories. Teachers score token choices along the student’s prefixes; an outcome reward model can add a sequence-level reward. The useful training record includes the path the student actually took and teacher scores at those positions.",
          "MiMo-V2-Flashも分野別の専門家を訓練しますが、MOPDでは学生自身が軌跡を生成します。教師はその接頭辞に沿ってtokenの分布を評価し、結果報酬モデルは回答全体の報酬を加えられます。学生が実際にたどった軌跡と、その位置での教師評価が学習データになります。"
        ]
      ],
      "figure": "multi-teacher",
      "refs": [
        1
      ]
    },
    {
      "title": [
        "GLM：把前面学会的能力带回来",
        "GLM: recover earlier capabilities",
        "GLM：前段階の能力を取り戻す"
      ],
      "paragraphs": [
        [
          "GLM-5 依次进行 SFT、推理 RL、Agent RL 和通用 RL，再做跨阶段 on-policy 蒸馏。前面阶段的检查点提供教师分布，最后阶段的权重提供学生起点。它要缓解的是后续训练造成的能力回退，因此旧检查点和对应的训练问题都值得保留。",
          "GLM-5 proceeds through SFT, reasoning RL, agentic RL, and general RL, then performs cross-stage on-policy distillation. Earlier checkpoints supply teacher distributions; the final-stage weights initialize the student. Recovering earlier capabilities makes both those checkpoints and their associated training prompts valuable.",
          "GLM-5はSFT、推論RL、Agent RL、汎用RLを経て、段階間のon-policy蒸留を行います。過去のチェックポイントが教師分布を提供し、最終段階の重みが学生の初期値になります。能力の回復には、過去のモデルと対応する学習問題の両方が役立ちます。"
        ]
      ],
      "figure": "cross-stage",
      "refs": [
        2
      ]
    },
    {
      "title": [
        "同一个模型，多看一份信息",
        "One model, an information advantage",
        "同じモデルに、追加の情報を与える"
      ],
      "paragraphs": [
        [
          "更直接的自蒸馏，利用模型已有的上下文学习能力。学生只看问题，教师额外看到示范答案、经验或环境反馈。学生先生成；教师沿着同一个前缀给出更有信息的下一步分布，再把这种差异用于更新学生。",
          "A direct form of self-distillation uses the model’s in-context learning ability. The student sees the task; the teacher also sees a demonstration, experience, or environmental feedback. The student generates first. Along the same prefixes, the better-informed teacher supplies next-token targets that guide the update.",
          "自己蒸留の一つの形は、モデルの文脈内学習能力を利用します。学生には問題だけを、教師には実演、経験、環境からのフィードバックも与えます。学生が先に生成し、その同じ接頭辞に対して教師が次のtokenの分布を示して、更新を導きます。"
        ],
        [
          "下面以 SDFT 为例。P 是学生分布，Q 是带示范的教师分布，目标是缩小 KL(P ∥ Q)。教师目标在这次学生更新中停止梯度；论文采用学生权重的指数移动平均（EMA）来稳定教师。部署时，学生不再依赖那份额外示范。",
          "SDFT makes this concrete. P is the student distribution and Q is the demonstration-conditioned teacher distribution; training reduces KL(P ∥ Q). The teacher target is stop-gradient for the student update. An exponential moving average of student weights stabilizes the teacher. At deployment, the student no longer receives the demonstration.",
          "SDFTでは、学生分布Pと実演を条件にした教師分布QのKL(P ∥ Q)を小さくします。学生の更新では教師側の勾配を止め、学生の重みの指数移動平均（EMA）で教師を安定させます。運用時の学生には、この追加の実演を与えません。"
        ]
      ],
      "figure": "self-teacher",
      "refs": [
        3
      ]
    },
    {
      "title": [
        "六种方法，数据各放在哪里",
        "Six methods, different uses of data",
        "6つの手法で、データをどこに使うか"
      ],
      "paragraphs": [
        [
          "这些方法放在一起看，最有用的区别是额外信息的来源、采样方和监督方式。它们并非都采用同一种自蒸馏目标。",
          "The useful comparison is the source of extra information, the sampling policy, and the supervision it produces. These methods do not all share the same self-distillation objective.",
          "追加情報の出所、軌跡の生成元、そこから得られる指導を比べると、違いが明確になります。すべてが同じ自己蒸留の目的関数を使うわけではありません。"
        ]
      ],
      "table": {
        "headers": [
          [
            "方法",
            "Method",
            "手法"
          ],
          [
            "教师多知道什么",
            "Teacher advantage",
            "教師が持つ追加情報"
          ],
          [
            "怎样形成训练信号",
            "Training signal",
            "学習信号の作り方"
          ]
        ],
        "rows": [
          [
            {
              "text": [
                "SDFT",
                "SDFT",
                "SDFT"
              ],
              "href": "https://arxiv.org/abs/2601.19897"
            },
            [
              "示范解答",
              "Worked demonstrations",
              "実演・解答例"
            ],
            [
              "学生采样；向带示范的 EMA 教师学习。",
              "Student rollouts; demonstration-conditioned EMA teacher.",
              "学生の軌跡を、実演付きのEMA教師から学習。"
            ]
          ],
          [
            {
              "text": [
                "SDPO",
                "SDPO",
                "SDPO"
              ],
              "href": "https://arxiv.org/abs/2601.20802"
            },
            [
              "环境反馈，或同组成功尝试",
              "Environment feedback or successful attempts",
              "環境のフィードバック、成功例"
            ],
            [
              "把失败原因或成功经验转成逐 token 监督。",
              "Turn rich feedback into token-level supervision.",
              "失敗の理由や成功例をtoken単位の指導に変換。"
            ]
          ],
          [
            {
              "text": [
                "OPSD",
                "OPSD",
                "OPSD"
              ],
              "href": "https://arxiv.org/abs/2601.18734"
            },
            [
              "已知答案或参考推理",
              "Known answers or reference reasoning",
              "正解または参考推論"
            ],
            [
              "学生采样；同源教师在相同前缀上提供分布。",
              "Student rollouts; a related teacher scores the same prefixes.",
              "学生が生成し、同系教師が同じ接頭辞を評価。"
            ]
          ],
          [
            {
              "text": [
                "OPCD",
                "OPCD",
                "OPCD"
              ],
              "href": "https://arxiv.org/abs/2602.12275"
            },
            [
              "经验知识或优化后的提示词",
              "Experience or optimized prompts",
              "経験知識、最適化したプロンプト"
            ],
            [
              "在学生轨迹上蒸馏额外上下文，推理时可移除它。",
              "Distill extra context along student rollouts; remove it at inference.",
              "学生の軌跡上で追加文脈を蒸留し、推論時には除去。"
            ]
          ],
          [
            {
              "text": [
                "π-Distill",
                "π-Distill",
                "π-Distill"
              ],
              "href": "https://arxiv.org/abs/2602.04942"
            },
            [
              "成功动作、规划提示等特权信息",
              "Successful actions, planning hints, and other privileged information",
              "成功した行動、計画のヒントなどの特権情報"
            ],
            [
              "教师采样；共享参数的教师与学生联合训练。",
              "Teacher rollouts; joint training of parameter-sharing teacher and student.",
              "教師が軌跡を生成し、パラメータを共有する教師と学生を共同学習。"
            ]
          ],
          [
            {
              "text": [
                "G-OPD / ExOPD",
                "G-OPD / ExOPD",
                "G-OPD / ExOPD"
              ],
              "href": "https://arxiv.org/abs/2602.12125"
            },
            [
              "教师训练后相对参考模型的能力增益",
              "Teacher improvement over a reference policy",
              "参照方策に対する教師の能力向上"
            ],
            [
              "学生采样；调整隐式奖励与正则的比例，探索超越教师。",
              "Student rollouts; rebalance implicit reward and regularization to go beyond imitation.",
              "学生が生成し、暗黙の報酬と正則化の比率を調整して模倣を超える。"
            ]
          ]
        ]
      }
    },
    {
      "title": [
        "数据生产多了四份工作",
        "Four jobs for the data pipeline",
        "データ作成で重要になる4つの仕事"
      ],
      "paragraphs": [
        [
          "第一，补齐问题分布。持续收集新场景、难例和长尾需求，避免模型只在自己熟悉的问题上循环。问题库可以提前建设；on-policy 要求轨迹跟随学生策略更新，不要求每轮都由人实时编写新题。",
          "First, maintain coverage: new situations, difficult cases, and long-tail needs keep learning from circling familiar tasks. A prompt bank can be prepared in advance. On-policy training requires trajectories tied to the evolving student, not freshly written human questions at every update.",
          "第一に、問題の網羅性です。新しい状況、難例、ロングテールの要求を加え、既知の問題だけを繰り返す状態を避けます。問題集は事前に作れます。On-policyで更新すべきなのは学生の方策に対応する軌跡であり、毎回人間が新しい問題を書く必要はありません。"
        ],
        [
          "第二，把反馈做具体。除了标准答案，还要有能定位错误的测试、环境返回和评审规则。例如“测试失败”不如“重复提交导致第二次扣款”有用；后者给教师提供了可解释的改进方向。",
          "Second, make feedback specific. Alongside answers, produce tests, environment results, and review criteria that locate errors. “A retry charged the customer twice” gives a teacher more actionable information than “test failed.”",
          "第二に、具体的なフィードバックです。正解だけでなく、誤りを特定できるテスト、環境の結果、評価基準を整えます。「テスト失敗」より「再試行で二重に課金した」の方が、教師に改善の手掛かりを与えます。"
        ],
        [
          "第三，记录数据怎么来的。将问题、学生轨迹、教师额外信息、评分目标与模型版本关联起来。这样才能区分答案不好、反馈有误、教师判断失准，还是训练用了过旧的轨迹。",
          "Third, retain provenance. Link each task, student trace, teacher context, supervision target, and model version. This separates poor answers from faulty feedback, weak teacher judgments, and stale trajectories.",
          "第三に、来歴を残します。問題、学生の軌跡、教師の追加文脈、学習目標、モデル版を結び付けます。回答の質、フィードバックの誤り、教師の判断、古い軌跡の混入を区別できます。"
        ],
        [
          "第四，独立检验学到了什么。用部署时的输入，在隔离测试集上比较任务成功率、旧能力回退和调用成本。教师与学生互相接近，并不能证明业务任务完成得更好。",
          "Fourth, test independently. Use deployment-time inputs on a held-out set to measure task success, regression on prior skills, and inference cost. Agreement between teacher and student does not by itself establish better task performance.",
          "第四に、独立して検証します。隔離したテスト集合と運用時の入力で、成功率、旧能力の低下、推論コストを測ります。教師と学生の一致だけでは、仕事がうまくできるようになったとは言えません。"
        ]
      ]
    },
    {
      "title": [
        "让自迭代有新的信息可学",
        "Give the loop something new to learn",
        "自己改善に、新しく学べる情報を渡す"
      ],
      "paragraphs": [
        [
          "如果问题、示范和评审都由同一个模型反复产生，错误也可能被反复确认。真实任务、外部知识、可靠的执行结果和人工判断，仍然是更新学习信号的重要来源。自蒸馏降低了某些完整答案的生产负担，却提高了对问题覆盖、反馈质量和独立评测的要求。",
          "When one model repeatedly supplies the questions, demonstrations, and judgments, it can reinforce its own mistakes. Real tasks, external knowledge, reliable execution results, and human judgment remain important sources of fresh information. Self-distillation can reduce some answer-writing work while raising the value of coverage, feedback quality, and independent evaluation.",
          "一つのモデルが問題、実演、判定を繰り返し作ると、誤りまで強化しかねません。実際の仕事、外部の知識、信頼できる実行結果、人間の判断は、今も新しい学習情報の重要な源です。自己蒸留は回答作成の一部を軽くする一方、問題の網羅性、フィードバックの質、独立評価の価値を高めます。"
        ]
      ]
    }
  ],
  "figures": [
    {
      "id": "data-loop",
      "title": [
        "数据如何进入自蒸馏",
        "Where data enters self-distillation",
        "自己蒸留にデータをどう渡すか"
      ],
      "caption": [
        "以学生采样的路线为例：问题驱动生成，额外信息帮助教师给出监督；独立测试检查学生真正留下的能力。",
        "A student-sampled loop: tasks drive generation, extra information guides teacher supervision, and held-out tests check what the student retains.",
        "学生が生成する場合の例。問題から軌跡を作り、追加情報で教師の指導を強め、独立評価で学生に残る能力を確かめます。"
      ],
      "graph": {
        "width": 1060,
        "height": 580,
        "nodes": [
          {
            "id": "tasks",
            "text": [
              "问题与场景",
              "Tasks & situations",
              "問題・場面"
            ],
            "x": 50,
            "y": 70,
            "width": 240,
            "height": 110,
            "sub": [
              "覆盖真实需求",
              "Cover real needs",
              "実際の要求を網羅"
            ],
            "kind": ""
          },
          {
            "id": "rollout",
            "text": [
              "学生生成",
              "Student rollout",
              "学生が生成"
            ],
            "x": 410,
            "y": 70,
            "width": 240,
            "height": 110,
            "sub": [
              "当前策略的轨迹",
              "Current-policy traces",
              "現在の方策の軌跡"
            ],
            "kind": ""
          },
          {
            "id": "targets",
            "text": [
              "教师监督",
              "Teacher targets",
              "教師の指導"
            ],
            "x": 770,
            "y": 70,
            "width": 240,
            "height": 110,
            "sub": [
              "同一前缀上的分布",
              "Same-prefix distributions",
              "同じ接頭辞の分布"
            ],
            "kind": "accent"
          },
          {
            "id": "context",
            "text": [
              "额外信息",
              "Extra information",
              "追加情報"
            ],
            "x": 770,
            "y": 330,
            "width": 240,
            "height": 110,
            "sub": [
              "示范 · 经验 · 环境反馈",
              "Examples · experience · feedback",
              "実演・経験・環境の応答"
            ],
            "kind": "context"
          },
          {
            "id": "update",
            "text": [
              "更新学生",
              "Update student",
              "学生を更新"
            ],
            "x": 410,
            "y": 330,
            "width": 240,
            "height": 110,
            "kind": ""
          },
          {
            "id": "evaluate",
            "text": [
              "独立测试",
              "Held-out evaluation",
              "独立したテスト"
            ],
            "x": 50,
            "y": 330,
            "width": 240,
            "height": 110,
            "sub": [
              "移除教师专属信息",
              "No teacher-only context",
              "教師専用の情報を除去"
            ],
            "kind": "output"
          }
        ],
        "edges": [
          {
            "from": "tasks",
            "to": "rollout",
            "path": "M293 125 H405",
            "kind": ""
          },
          {
            "from": "rollout",
            "to": "targets",
            "path": "M653 125 H765",
            "kind": ""
          },
          {
            "from": "context",
            "to": "targets",
            "path": "M890 327 V185",
            "kind": ""
          },
          {
            "from": "targets",
            "to": "update",
            "path": "M770 150 H715 V385 H655",
            "kind": ""
          },
          {
            "from": "update",
            "to": "rollout",
            "path": "M530 327 V185",
            "kind": "feedback"
          },
          {
            "from": "update",
            "to": "evaluate",
            "path": "M407 385 H295",
            "kind": ""
          }
        ],
        "labels": [
          {
            "text": [
              "学生的下一轮",
              "Next student rollout",
              "次の学生の生成"
            ],
            "x": 415,
            "y": 242,
            "width": 230
          }
        ]
      }
    },
    {
      "id": "specialists",
      "title": [
        "DeepSeek-V3.2：专家数据与混合 RL",
        "DeepSeek-V3.2: specialist data and mixed RL",
        "DeepSeek-V3.2：専門家データと混合RL"
      ],
      "caption": [
        "专家先生成领域数据，统一模型先蒸馏、再做混合 RL。",
        "Specialists generate domain data; the unified model is distilled before mixed RL.",
        "専門家が分野別データを生成し、統合モデルは蒸留の後に混合RLを行います。"
      ],
      "graph": {
        "width": 1060,
        "height": 510,
        "nodes": [
          {
            "id": "base",
            "text": [
              "共同基础模型",
              "Shared base model",
              "共通の基盤モデル"
            ],
            "x": 50,
            "y": 65,
            "width": 240,
            "height": 112,
            "kind": ""
          },
          {
            "id": "experts",
            "text": [
              "领域专家训练",
              "Specialist RL",
              "専門分野のRL"
            ],
            "x": 410,
            "y": 65,
            "width": 240,
            "height": 112,
            "sub": [
              "数学 · 代码 · Agent",
              "Math · code · agents",
              "数学・コード・Agent"
            ],
            "kind": ""
          },
          {
            "id": "data",
            "text": [
              "专家生成数据",
              "Specialist outputs",
              "専門家の生成データ"
            ],
            "x": 770,
            "y": 65,
            "width": 240,
            "height": 112,
            "sub": [
              "各领域的高质量答案",
              "Domain-specific answers",
              "分野ごとの回答"
            ],
            "kind": "context"
          },
          {
            "id": "distill",
            "text": [
              "蒸馏统一模型",
              "Distill one model",
              "統合モデルへ蒸留"
            ],
            "x": 770,
            "y": 320,
            "width": 240,
            "height": 112,
            "kind": ""
          },
          {
            "id": "mixed",
            "text": [
              "混合 RL",
              "Mixed RL",
              "混合RL"
            ],
            "x": 410,
            "y": 320,
            "width": 240,
            "height": 112,
            "sub": [
              "推理 · Agent · 人类偏好",
              "Reasoning · agents · alignment",
              "推論・Agent・人間の選好"
            ],
            "kind": "accent"
          },
          {
            "id": "final",
            "text": [
              "DeepSeek-V3.2",
              "DeepSeek-V3.2",
              "DeepSeek-V3.2"
            ],
            "x": 50,
            "y": 320,
            "width": 240,
            "height": 112,
            "kind": "output"
          }
        ],
        "edges": [
          {
            "from": "base",
            "to": "experts",
            "path": "M293 121 H405",
            "kind": ""
          },
          {
            "from": "experts",
            "to": "data",
            "path": "M653 121 H765",
            "kind": ""
          },
          {
            "from": "data",
            "to": "distill",
            "path": "M890 180 V315",
            "kind": ""
          },
          {
            "from": "distill",
            "to": "mixed",
            "path": "M767 376 H655",
            "kind": ""
          },
          {
            "from": "mixed",
            "to": "final",
            "path": "M407 376 H295",
            "kind": ""
          }
        ]
      }
    },
    {
      "id": "multi-teacher",
      "title": [
        "MiMo-V2-Flash：多教师在线蒸馏",
        "MiMo-V2-Flash: multi-teacher on-policy distillation",
        "MiMo-V2-Flash：複数教師によるon-policy蒸留"
      ],
      "caption": [
        "保留三阶段与两路反馈：教师给逐 token 信号，ORM 给整条回答的奖励。两者共同作用于学生更新。",
        "Three stages, two feedback paths: teachers provide token-level signals; the ORM provides sequence-level rewards. Both guide the student update.",
        "3段階と2つのフィードバックを保持。教師はtoken単位の信号、ORMは回答全体の報酬を提供し、学生の更新に使います。"
      ],
      "graph": {
        "width": 1060,
        "height": 840,
        "nodes": [
          {
            "id": "sft",
            "text": [
              "SFT 模型",
              "SFT model",
              "SFTモデル"
            ],
            "x": 50,
            "y": 65,
            "width": 240,
            "height": 110,
            "sub": [
              "1 · 指令跟随",
              "1 · Instruction following",
              "1 · 指示への追従"
            ],
            "kind": ""
          },
          {
            "id": "specialize",
            "text": [
              "领域专项训练",
              "Domain training",
              "分野別の学習"
            ],
            "x": 410,
            "y": 65,
            "width": 240,
            "height": 110,
            "sub": [
              "2 · 数学 / 代码 / 搜索…",
              "2 · Math / code / search…",
              "2 · 数学・コード・検索…"
            ],
            "kind": ""
          },
          {
            "id": "teachers",
            "text": [
              "领域教师",
              "Domain teachers",
              "分野別の教師"
            ],
            "x": 770,
            "y": 65,
            "width": 240,
            "height": 110,
            "kind": "context"
          },
          {
            "id": "student",
            "text": [
              "学生模型",
              "Student model",
              "学生モデル"
            ],
            "x": 50,
            "y": 325,
            "width": 240,
            "height": 110,
            "sub": [
              "3 · MOPD",
              "3 · MOPD",
              "3 · MOPD"
            ],
            "kind": ""
          },
          {
            "id": "trace",
            "text": [
              "学生轨迹",
              "Student rollout",
              "学生の軌跡"
            ],
            "x": 410,
            "y": 325,
            "width": 240,
            "height": 110,
            "sub": [
              "token 1 → token 2 → …",
              "token 1 → token 2 → …",
              "token 1 → token 2 → …"
            ],
            "kind": ""
          },
          {
            "id": "score",
            "text": [
              "教师重评分",
              "Teacher rescoring",
              "教師による再評価"
            ],
            "x": 770,
            "y": 325,
            "width": 240,
            "height": 110,
            "sub": [
              "Prefill · Reverse KL",
              "Prefill · Reverse KL",
              "Prefill · Reverse KL"
            ],
            "kind": "accent"
          },
          {
            "id": "orm",
            "text": [
              "结果奖励模型",
              "Outcome reward model",
              "結果報酬モデル"
            ],
            "x": 410,
            "y": 585,
            "width": 240,
            "height": 110,
            "sub": [
              "ORM",
              "ORM",
              "ORM"
            ],
            "kind": ""
          },
          {
            "id": "update",
            "text": [
              "更新学生",
              "Update student",
              "学生を更新"
            ],
            "x": 770,
            "y": 585,
            "width": 240,
            "height": 110,
            "kind": "accent"
          }
        ],
        "edges": [
          {
            "from": "sft",
            "to": "specialize",
            "path": "M293 120 H405",
            "kind": ""
          },
          {
            "from": "specialize",
            "to": "teachers",
            "path": "M653 120 H765",
            "kind": ""
          },
          {
            "from": "sft",
            "to": "student",
            "path": "M170 178 V320",
            "kind": ""
          },
          {
            "from": "student",
            "to": "trace",
            "path": "M293 380 H405",
            "kind": ""
          },
          {
            "from": "trace",
            "to": "score",
            "path": "M653 380 H765",
            "kind": ""
          },
          {
            "from": "teachers",
            "to": "score",
            "path": "M890 178 V320",
            "kind": ""
          },
          {
            "from": "trace",
            "to": "orm",
            "path": "M530 438 V580",
            "kind": ""
          },
          {
            "from": "score",
            "to": "update",
            "path": "M890 438 V580",
            "kind": ""
          },
          {
            "from": "orm",
            "to": "update",
            "path": "M653 640 H765",
            "kind": ""
          },
          {
            "from": "update",
            "to": "student",
            "path": "M890 698 V765 H170 V440",
            "kind": "feedback"
          }
        ],
        "labels": [
          {
            "text": [
              "逐 token 信号",
              "Token-level signal",
              "token単位の信号"
            ],
            "x": 785,
            "y": 491,
            "width": 210
          },
          {
            "text": [
              "整条奖励",
              "Sequence reward",
              "全体の報酬"
            ],
            "x": 655,
            "y": 586,
            "width": 110
          },
          {
            "text": [
              "重新采样",
              "Sample again",
              "再サンプリング"
            ],
            "x": 235,
            "y": 752,
            "width": 180
          }
        ]
      }
    },
    {
      "id": "cross-stage",
      "title": [
        "GLM-5：阶段训练与能力回收",
        "GLM-5: staged training and capability recovery",
        "GLM-5：段階的な学習と能力の回復"
      ],
      "caption": [
        "实线连接训练阶段和权重初始化；虚线传递教师分布。SFT、推理 RL 和通用 RL 的检查点参与最终蒸馏。",
        "Solid arrows show training progression and weight initialization; dashed arrows carry teacher distributions. SFT, reasoning-RL, and general-RL checkpoints support final distillation.",
        "実線は学習の進行と重みの初期化、破線は教師分布を示します。SFT・推論RL・汎用RLのチェックポイントを最終蒸留に使います。"
      ],
      "graph": {
        "width": 1120,
        "height": 880,
        "nodes": [
          {
            "id": "pretrain",
            "text": [
              "预训练",
              "Pre-training",
              "事前学習"
            ],
            "x": 40,
            "y": 80,
            "width": 230,
            "height": 116,
            "sub": [
              "通用语料 → 代码与推理",
              "General → code & reasoning",
              "一般言語 → コード・推論"
            ],
            "kind": ""
          },
          {
            "id": "mid",
            "text": [
              "中期训练",
              "Mid-training",
              "中間学習"
            ],
            "x": 40,
            "y": 320,
            "width": 230,
            "height": 116,
            "sub": [
              "长代码 → 长上下文与 Agent",
              "Long code → long context & agents",
              "長いコード → 長文脈・Agent"
            ],
            "kind": ""
          },
          {
            "id": "sparse",
            "text": [
              "稀疏注意力适配",
              "Sparse-attention adaptation",
              "疎な注意への適応"
            ],
            "x": 40,
            "y": 560,
            "width": 230,
            "height": 116,
            "sub": [
              "形成基础模型",
              "Base model",
              "基盤モデル"
            ],
            "kind": ""
          },
          {
            "id": "sft",
            "text": [
              "SFT",
              "SFT",
              "SFT"
            ],
            "x": 400,
            "y": 80,
            "width": 230,
            "height": 116,
            "kind": ""
          },
          {
            "id": "reason",
            "text": [
              "推理 RL",
              "Reasoning RL",
              "推論RL"
            ],
            "x": 400,
            "y": 280,
            "width": 230,
            "height": 116,
            "kind": ""
          },
          {
            "id": "agent",
            "text": [
              "Agent RL",
              "Agentic RL",
              "Agent RL"
            ],
            "x": 400,
            "y": 480,
            "width": 230,
            "height": 116,
            "kind": ""
          },
          {
            "id": "general",
            "text": [
              "通用 RL",
              "General RL",
              "汎用RL"
            ],
            "x": 400,
            "y": 680,
            "width": 230,
            "height": 116,
            "kind": ""
          },
          {
            "id": "csod",
            "text": [
              "跨阶段蒸馏",
              "Cross-stage distillation",
              "段階間の蒸留"
            ],
            "x": 840,
            "y": 440,
            "width": 240,
            "height": 156,
            "sub": [
              "学生采样 · 教师评分",
              "Student samples · teachers score",
              "学生が生成・教師が評価"
            ],
            "kind": "accent"
          },
          {
            "id": "final",
            "text": [
              "GLM-5",
              "GLM-5",
              "GLM-5"
            ],
            "x": 840,
            "y": 730,
            "width": 240,
            "height": 100,
            "kind": "output"
          }
        ],
        "edges": [
          {
            "from": "pretrain",
            "to": "mid",
            "path": "M155 199 V315",
            "kind": ""
          },
          {
            "from": "mid",
            "to": "sparse",
            "path": "M155 439 V555",
            "kind": ""
          },
          {
            "from": "sparse",
            "to": "sft",
            "path": "M273 618 H330 V138 H395",
            "kind": ""
          },
          {
            "from": "sft",
            "to": "reason",
            "path": "M515 199 V275",
            "kind": ""
          },
          {
            "from": "reason",
            "to": "agent",
            "path": "M515 399 V475",
            "kind": ""
          },
          {
            "from": "agent",
            "to": "general",
            "path": "M515 599 V675",
            "kind": ""
          },
          {
            "from": "sft",
            "to": "csod",
            "path": "M633 138 H735 V470 H835",
            "kind": "signal"
          },
          {
            "from": "reason",
            "to": "csod",
            "path": "M633 338 H700 V515 H835",
            "kind": "signal"
          },
          {
            "from": "general",
            "to": "csod",
            "path": "M633 710 H735 V560 H835",
            "kind": "signal"
          },
          {
            "from": "general",
            "to": "csod",
            "path": "M633 770 H785 V590 H835",
            "kind": ""
          },
          {
            "from": "csod",
            "to": "final",
            "path": "M960 599 V725",
            "kind": ""
          }
        ],
        "labels": [
          {
            "text": [
              "基础模型训练",
              "Base-model training",
              "基盤モデルの学習"
            ],
            "x": 35,
            "y": 20,
            "width": 240
          },
          {
            "text": [
              "后训练",
              "Post-training",
              "事後学習"
            ],
            "x": 400,
            "y": 20,
            "width": 230
          },
          {
            "text": [
              "教师分布",
              "Teacher distributions",
              "教師分布"
            ],
            "x": 805,
            "y": 340,
            "width": 290
          },
          {
            "text": [
              "学生权重初始化",
              "Initialize student weights",
              "学生の重みを初期化"
            ],
            "x": 640,
            "y": 800,
            "width": 185
          }
        ]
      }
    },
    {
      "id": "self-teacher",
      "title": [
        "SDFT：相同问题，两个上下文",
        "SDFT: one task, two contexts",
        "SDFT：同じ問題、2つの文脈"
      ],
      "caption": [
        "学生生成的前缀同时交给两侧；示范只进入教师。更新沿学生分支传播，教师通过 EMA 跟随学生。",
        "Both branches see the student-generated prefix; only the teacher receives the demonstration. Gradients update the student, and the teacher follows through EMA.",
        "学生が生成した接頭辞を両側に渡し、実演は教師だけに与えます。学生を勾配で更新し、教師はEMAで追従します。"
      ],
      "graph": {
        "width": 1120,
        "height": 740,
        "groups": [
          {
            "x": 360,
            "y": 65,
            "width": 330,
            "height": 490,
            "text": [
              "同源模型",
              "Related model weights",
              "同系のモデル重み"
            ]
          }
        ],
        "nodes": [
          {
            "id": "query",
            "text": [
              "问题 + 学生前缀",
              "Task + student prefix",
              "問題＋学生の接頭辞"
            ],
            "x": 40,
            "y": 130,
            "width": 250,
            "height": 125,
            "kind": ""
          },
          {
            "id": "student",
            "text": [
              "学生",
              "Student",
              "学生"
            ],
            "x": 400,
            "y": 130,
            "width": 250,
            "height": 125,
            "sub": [
              "只看问题与前缀",
              "Task and prefix only",
              "問題と接頭辞のみ"
            ],
            "kind": ""
          },
          {
            "id": "teacher",
            "text": [
              "教师",
              "Teacher",
              "教師"
            ],
            "x": 400,
            "y": 390,
            "width": 250,
            "height": 125,
            "sub": [
              "额外看到示范",
              "Also sees demonstration",
              "実演も参照"
            ],
            "kind": "context"
          },
          {
            "id": "demo",
            "text": [
              "示范解答",
              "Demonstration",
              "実演・解答例"
            ],
            "x": 40,
            "y": 390,
            "width": 250,
            "height": 125,
            "kind": ""
          },
          {
            "id": "loss",
            "text": [
              "分布对齐",
              "Match distributions",
              "分布を近づける"
            ],
            "x": 820,
            "y": 130,
            "width": 260,
            "height": 125,
            "sub": [
              "KL(P ∥ Q)",
              "KL(P ∥ Q)",
              "KL(P ∥ Q)"
            ],
            "kind": "accent"
          },
          {
            "id": "update",
            "text": [
              "更新学生参数",
              "Update student weights",
              "学生の重みを更新"
            ],
            "x": 820,
            "y": 390,
            "width": 260,
            "height": 125,
            "kind": "accent"
          },
          {
            "id": "ema",
            "text": [
              "EMA 教师",
              "EMA teacher",
              "EMA教師"
            ],
            "x": 400,
            "y": 620,
            "width": 250,
            "height": 90,
            "sub": [
              "平滑跟随学生权重",
              "Smoothed student weights",
              "学生の重みを平滑化"
            ],
            "kind": ""
          }
        ],
        "edges": [
          {
            "from": "query",
            "to": "student",
            "path": "M293 192 H395",
            "kind": ""
          },
          {
            "from": "query",
            "to": "teacher",
            "path": "M325 192 V430 H395",
            "kind": ""
          },
          {
            "from": "demo",
            "to": "teacher",
            "path": "M293 452 H395",
            "kind": ""
          },
          {
            "from": "student",
            "to": "loss",
            "path": "M653 192 H815",
            "kind": ""
          },
          {
            "from": "teacher",
            "to": "loss",
            "path": "M653 452 H740 V230 H815",
            "kind": "signal"
          },
          {
            "from": "loss",
            "to": "update",
            "path": "M950 258 V385",
            "kind": ""
          },
          {
            "from": "update",
            "to": "student",
            "path": "M950 518 V565 H715 V300 H525 V260",
            "kind": "feedback"
          },
          {
            "from": "update",
            "to": "ema",
            "path": "M950 518 V665 H655",
            "kind": "signal"
          },
          {
            "from": "ema",
            "to": "teacher",
            "path": "M525 617 V520",
            "kind": "signal"
          },
          {
            "from": "student",
            "to": "query",
            "path": "M525 127 V32 H165 V125",
            "kind": ""
          }
        ],
        "labels": [
          {
            "text": [
              "继续采样下一 token",
              "Sample the next token",
              "次のtokenを生成"
            ],
            "x": 60,
            "y": 12,
            "width": 300
          },
          {
            "text": [
              "学生分布 P",
              "Student P",
              "学生分布 P"
            ],
            "x": 653,
            "y": 146,
            "width": 160
          },
          {
            "text": [
              "教师 Q\n停止梯度",
              "Teacher Q\nStop-gradient",
              "教師 Q\n勾配停止"
            ],
            "x": 665,
            "y": 340,
            "width": 145
          },
          {
            "text": [
              "梯度更新",
              "Gradient update",
              "勾配による更新"
            ],
            "x": 748,
            "y": 546,
            "width": 280
          }
        ]
      }
    }
  ],
  "references": [
    [
      "DeepSeek-V3.2 · §3",
      "https://arxiv.org/html/2512.02556v1#S3"
    ],
    [
      "MiMo-V2-Flash · §4.1",
      "https://arxiv.org/html/2601.02780v2#S4.SS1"
    ],
    [
      "GLM-5 · §3.5",
      "https://arxiv.org/html/2602.15763v2#S3.SS5"
    ],
    [
      "SDFT · Self-Distillation Enables Continual Learning",
      "https://arxiv.org/abs/2601.19897"
    ],
    [
      "SDPO · Reinforcement Learning via Self-Distillation",
      "https://arxiv.org/abs/2601.20802"
    ],
    [
      "OPSD · Self-Distilled Reasoner",
      "https://arxiv.org/abs/2601.18734"
    ],
    [
      "OPCD · On-Policy Context Distillation",
      "https://arxiv.org/abs/2602.12275"
    ],
    [
      "π-Distill · Privileged Information Distillation",
      "https://arxiv.org/abs/2602.04942"
    ],
    [
      "G-OPD / ExOPD · Learning beyond Teacher",
      "https://arxiv.org/abs/2602.12125"
    ]
  ],
  "related": [
    "on-policy-distillation",
    "grpo-design-choices",
    "foundation-model-data"
  ]
}
