Learning & methods · November 2026学习与方法 · 2026 年 11 月
Where Do Agent Tasks Come From?Agent 的任务从哪里来?
OS-Genesis starts with exploration; AgentTrek starts with tutorials. Both must turn a plausible goal into an execution trace that can be checked.从 OS-Genesis 的探索后命题,到 AgentTrek 的教程引导重放:把有意义的目标变成可验证的执行轨迹。
1 A task description is not a training trajectory1 一句话需求,还不是一条训练轨迹
训练一个会操作界面的 Agent,首先遇到的问题往往不是模型大小,而是让它练习什么。可以请人写任务,也可以让另一个模型批量编需求;但“把报表设置成每周发送”听起来合理,不代表当前应用确实支持,也不代表执行所需的权限、账户和数据已经齐备。
这里要分清三个对象:任务说明想达到什么结果;计划提出怎样做;轨迹记录某个初始状态下,实际做了什么、环境返回了什么。把计划改写成对话格式,并不会自动补上环境反馈。一条有用的训练样本,必须能回答:模型当时看到了什么,为什么有条件执行这个动作,动作后究竟发生了什么。
OS-Genesis 和 AgentTrek 给出两种不同的任务来源。前者从环境里的真实交互反推目标,后者从人类已经写下的教程提取目标与步骤。它们改变的是数据生产的起点,而共同的关键环节是:把目标放回环境,重新执行并检查。
When training an agent to operate interfaces, an early bottleneck is deciding what it should practise. People can write tasks, and another model can invent them at scale. Yet a plausible request such as “schedule this report every week” says little about whether the current application supports it, or whether the required account, permissions and data exist.
Three objects need separate names. A task states the desired outcome. A plan proposes a route. A trajectory records what actually happened from a particular initial state, including the environment's responses. Reformatting a plan as a conversation does not supply missing observations. A useful training example must establish what the agent could see, what action it could take, and what changed afterwards.
OS-Genesis and AgentTrek provide different sources of tasks. The former derives goals from interactions with an environment; the latter extracts goals and procedures from human-written tutorials. Their starting points differ, but both require the crucial step of executing a goal in an environment and checking the result.
Static diagram静态图
2 OS-Genesis: discover an affordance, then propose a goal2 OS-Genesis:先发现功能,再提出目标
OS-Genesis 先在 Android 或网页环境中探索可交互元素,记录动作前后的截图与动作,得到 。主体探索采用规则,输入框内容等环节调用模型。随后,标注模型把一次状态变化描述成低层操作,再将其扩展成更高层的用户目标。方法见原论文 第 3 节。
例如,点击下拉框后出现了筛选选项,可以支持“展开筛选菜单”这个低层描述,也能启发“查找本周未完成订单”的任务。但第二句话多出了日期、状态和结果正确性要求,尚未被这次点击证明。逆向命题提供了功能依据,却没有证明整个目标已被完成。
因此,论文不是给探索片段贴上高层标签就结束,而是让执行模型以新合成的高层任务为目标,再收集完整轨迹。这个顺序非常重要:探索发现“这里可能能做什么”,目标执行回答“能否按要求做完”。上面的订单场景是说明这一差别的示例,并非论文报告的实验案例。
OS-Genesis explores interactive elements in Android or web environments and records screenshots before and after an action, forming . Exploration is largely rule-based, with a model supplying content for input fields. An annotation model describes the local operation and then proposes a higher-level user goal. The pipeline is specified in Section 3.
For example, opening a dropdown may justify the local description “expand the filter menu” and suggest a task such as “find this week's unfinished orders.” The latter introduces additional requirements concerning dates, status and correctness. A click has not established those conditions. Reverse task synthesis grounds a proposed goal in an observed affordance; it does not certify completion of that goal.
The paper therefore executes the newly synthesized high-level instruction to collect a complete trajectory. Exploration discovers a possible capability; goal-directed execution tests whether it can be used to finish the task. The order example here is an explanatory illustration, not a reported experiment.
3 AgentTrek: tutorials guide; observations determine actions3 AgentTrek:教程给方向,环境决定实际动作
AgentTrek 从互联网教程出发,筛选和整理出目标网站、任务、前置条件、步骤及预期结果,再让 VLM 在浏览器中引导重放。这里的“重放”不是机械复制坐标,而是结合教程与当前截图、可访问性树,决定具体操作;原文 第 2.2 节给出了这一过程。
教程像一位熟悉产品的人提供的路线:先到哪个页面,再展开哪个菜单。真实执行仍要处理界面版本、弹窗、缺失数据和登录状态。教程说“点击保存”,当前页面却可能先要求补全必填字段。训练数据应该记录模型如何应对这个实际反馈,而不是把教程原句当作已经执行成功的事实。
还有一个容易混淆的层次:记录系统保存的内容,可以多于策略当时能看到的内容。AgentTrek 的原生浏览器 trace 保存 DOM 等调试信息,而执行时主要观察截图与可访问性树;最终视觉、文本训练形式又有各自的观察与动作映射。做数据加工时,要保留这三个层次的区别,避免把事后可见的信息塞进当时的模型输入。
AgentTrek begins with online tutorials, extracting a target website, task, prerequisites, steps and expected outcome. A VLM then performs guided replay in a browser. Replay here does not mean copying fixed coordinates: the agent combines the tutorial with the current screenshot and accessibility tree to choose an action. Section 2.2 describes this process.
A tutorial supplies a route, much as a knowledgeable user would. Execution still encounters changed interfaces, popups, missing data and login requirements. An instruction to save may meet a form that first requires another field. Training data should record how the agent responds to that feedback, instead of treating the tutorial's instruction as proof that the action succeeded.
One further distinction matters: a recorder may retain more information than the policy was allowed to observe. AgentTrek's native browser traces include debugging information such as DOM snapshots; the execution policy primarily uses screenshots and accessibility trees. Its visual and textual training formats have their own observation and action mappings. Data processing must preserve these distinctions so that information discovered later does not leak into an earlier model input.
4 Quality scores can control sampling without making training RL4 质量分数控制取样,不等于已经在做 RL
两篇论文还在如何使用不完美轨迹上作了不同选择。AgentTrek 用评估模型检查任务遵循和关键步骤完成情况。OS-Genesis 则用轨迹奖励模型衡量完成度与连贯性,对轨迹给出 1–5 分,按分数加权抽样,保留部分未完全完成但仍有信息的轨迹。
这个式子出现在 OS-Genesis 的 Algorithm 1。它表示高分轨迹更常进入训练;论文第 4.2 节使用的训练目标仍是监督学习,让模型预测低层指令和动作。存在 reward model,并不足以把整个流程叫作强化学习。应当继续看参数究竟由什么损失更新。
评估模型也不是环境真值的替代品。“已经保存”可能只是模型自己的解释,真正的状态却没有改变。对于“设置筛选并保存”这样的任务,除了观察界面,还可以复查保存结果是否存在、重新打开后条件是否保留。这样的结构化检查是本文提出的数据验收做法;不能把它写成两篇论文都已经实现的统一验证器。
The papers make different choices about imperfect trajectories. AgentTrek evaluates adherence to the task and completion of its key components. OS-Genesis scores completion and coherence from 1 to 5 with a trajectory reward model, then samples trajectories proportionally to those scores. This retains some incomplete but potentially informative experience.
This is the sampling rule in OS-Genesis Algorithm 1. Higher-scoring trajectories appear more frequently in training. The objectives in Section 4.2 remain supervised objectives for predicting instructions and actions. The presence of a reward model alone does not make a pipeline reinforcement learning. The parameter-update objective determines that distinction.
A model evaluator also cannot replace ground truth about the environment. “Saved successfully” may be the agent's own assertion even when no persistent state changed. For a filtering-and-saving task, a separate check could inspect the saved result and reopen it to verify the conditions. These structured checks are suggested acceptance procedures, not a claim that both papers implemented the same verifier.
5 Preserve the causal contract of a trajectory5 一条可用轨迹,需要保留哪些因果关系
把两条路线放到同一个数据工程视角,最应保留的不是漂亮的对话文本,而是四个对应关系:目标对应初始环境;动作对应此前可见的观察;新观察确实来自动作执行;验收结果对应最终产物。任何一处错接,都可能制造“看似成功、无法重现”的样本。
Across both pipelines, the critical artifact is a set of causal correspondences: the goal matches the initial environment; an action follows an observation available at that time; the next observation comes from executing that action; and the acceptance result refers to the resulting artifact. Breaking any of these links can create a convincing but irreproducible sample.
Static diagram静态图
假设轨迹是“创建筛选 → 保存 → 再次打开”。若保存时网络超时,之后重试成功,整理者不能删去超时观察,却把后面的修复动作保留下来;否则模型将学到没有触发条件的重试。相反,若只保留最终成功片段,也必须把它真实依赖的已创建状态作为起点,而不能伪装成空白环境。
验收应先解决这类状态与时序问题,再讨论轨迹越长是否越好。较短的有效操作链、必要的恢复片段与完整任务成功,分别服务不同训练目的。具体如何切片和配置 loss mask,可接着读轨迹如何变成训练数据。
Consider “create a filter, save it, reopen it.” If saving times out and a retry succeeds, removing the timeout while retaining the recovery action teaches a retry with no apparent cause. If only the successful suffix is retained, its starting state must include the objects already created. It cannot be presented as execution from an empty environment.
Acceptance should address state and timing before asking whether longer trajectories are better. A short valid action sequence, a recovery episode and a complete successful task support different training objectives. From trajectories to training data develops the subsequent questions of slicing and loss masks.
6 Choosing a task source also chooses a data bias6 选择任务来源,也是在选择数据偏向
有稳定教程、步骤复杂但目标清楚的产品,适合用教程降低执行探索成本;功能多、缺少任务说明的可交互环境,适合先发现功能再命题。前者可能继承教程的过时路径和热门功能偏向,后者可能偏向容易发现、容易点击的功能。这是两类来源机制带来的取样倾向,不是两个模型谁更聪明的排名。
实际选型可以先做小规模对照:在相同环境与执行预算下,统计候选任务中能满足前置条件的比例、执行有效率,以及最终验收通过率;再按功能类别观察覆盖。不同论文的模型、训练量和评测协议并不相同,不宜直接拿各自最佳分数相减,当作两种任务来源的净收益。
两种来源也可以在同一系统中共存:教程提供已有目标,探索补充未覆盖功能,统一执行器与验收协议再把它们变成数据。这是可验证的组合方案,而不是本文声称已经完成的实验。真正值得保留的共同经验是:任务生成可以从想法出发,训练轨迹必须回到发生过的执行。
Tutorial guidance is attractive when a product has stable documentation and complex procedures with clear goals. Exploration-first synthesis is attractive when an interactive environment has many affordances but few task descriptions. The former may inherit outdated instructions and emphasis on popular workflows; the latter may favour features that are easy to discover or click. These are sampling tendencies, not a ranking of model intelligence.
A useful pilot comparison would hold the environment and execution budget fixed, then measure prerequisite availability, valid execution and final acceptance, while examining coverage by function. The two papers use different models, training volumes and evaluation protocols. Subtracting their best published scores would not isolate the value of their task sources.
Both sources could coexist: tutorials supply established goals, exploration fills coverage gaps, and a common executor and acceptance protocol produce the data. That is a testable design proposal, not an experiment reported here. The shared lesson is that a task may begin as an idea, but a training trajectory must return to execution that actually occurred.