Engineering notes · November 2026工程笔记 · 2026 年 11 月

Where a GPU Job Actually Runs算力任务究竟在哪里运行

Following code, identity, and data from a terminal to a running task.沿着代码、身份与数据,理解从终端到 GPU 任务的整条路径。

A command leaves a terminal, a dashboard reports a running job, and a model returns an answer. The sequence feels continuous, although each step may happen in a different environment.

To understand that sequence, ask four questions: where is the process running, which version does it use, whose identity does it carry, and what can it reach? These questions turn an unfamiliar compute platform into a workflow that can be followed and checked.

终端发出命令,控制台显示任务运行,模型返回答案。操作看起来连贯,背后却可能经过了几个彼此独立的环境。

理解这条链路,可以始终追问四件事:进程在哪里运行,用的是哪个版本,以谁的身份访问什么?这四个问题,足以把陌生的算力平台变成一条可以追踪和核对的工作路径。

1. The terminal is an entry point1. 终端只是入口

An editor connected over SSH may open a development container, a persistent workspace, or a login environment. It provides a place to edit code, inspect files, and submit work. A working shell does not establish that this environment has a GPU, shares the target cluster’s storage, or can reach every model service.

A submitted job runs in the environment allocated by the scheduler. The development session can remain open while the actual computation starts elsewhere. Keeping these roles separate explains a familiar surprise: a script succeeds in the terminal but fails in a job, because its files, dependencies, permissions, or network access have changed.

编辑器通过 SSH 连上的,可能是开发容器、持久工作区或登录环境。它提供修改代码、查看文件和提交任务的地方。能够打开终端,不代表这里有 GPU,也不代表它共享目标集群的存储,或能访问所有模型服务。

任务提交后,真正的计算发生在调度平台分配的执行环境里。开发窗口可以一直开着,计算却已经转移到别处。于是,同一段脚本在终端成功、放进任务后失败,并不奇怪:它看到的文件、依赖、权限和网络可能都变了。

Static diagram静态图Development, versioned inputs, scheduling, and the scoped execution environment of a GPU job
Development, scheduling, and execution have distinct roles. A job brings a code revision, an image, and a configuration into an execution environment; its access to storage and model services follows the boundaries of that environment.开发、调度与执行各有分工。任务把代码版本、运行镜像和配置带入执行环境;它能读取哪些文件、调用哪些模型服务,取决于该环境的存储与网络边界。 A conceptual workflow with generic roles, rather than a map of a particular production network.图中使用通用角色表达工作关系,不对应具体生产网络的部署拓扑。

2. A job begins with three inputs2. 一次任务从三个输入开始

In the workflow illustrated here, a job combines a Git commit, a container image, and a configuration. The commit selects the code; the image supplies its runtime and dependencies; the configuration chooses the command, inputs, mounts, and resources. Together, they describe what the platform is being asked to execute.

When the platform checks out a specified remote commit, changes in the developer’s working directory are not included until they have been committed, pushed, and selected by the job. A successful submission can therefore run an older program perfectly. Recording the actual commit and image identity alongside the configuration makes that mismatch visible; an image digest is useful when a tag might change.

These three inputs are a starting point for reproducibility. The data snapshot, model weights, and relevant runtime settings also need identifiable versions when comparing results.

图中的任务由 Git commit、容器镜像和配置共同定义。commit 选择代码,镜像提供运行环境与依赖,配置指定启动命令、输入、挂载和资源。把三者放在一起,才能说清平台接到的究竟是什么任务。

如果平台按指定的远端 commit 拉取代码,开发目录里的修改就必须先提交、推送,再被任务选中。否则,任务完全可能成功运行一份旧程序。把实际使用的 commit、镜像标识和配置保存在一起,就能发现这种错配;镜像标签可能变化时,可用摘要固定镜像内容。

这三个输入是复现的起点。比较实验结果时,数据快照、模型权重和相关运行参数也需要有可辨认的版本。

3. Placement, quota, and identity3. 位置、额度与身份

A cluster describes the execution environment. A quota defines an allocation or limit within the platform’s resource policy. An account determines who submits the work and which resources that identity may use. These concepts meet during scheduling, but they answer different questions.

Copying a colleague’s job configuration does not copy their storage permissions or resource entitlement. Likewise, a container process running as root does not establish the identity used by the scheduling platform. Output locations and access checks should follow the submitting identity recorded by the platform, rather than a username copied into a script.

This is why the job’s resolved metadata matters: it tells you where the work actually landed, which allocation it used, and under whose identity it was submitted.

cluster 描述任务进入哪个执行环境,quota 描述平台资源规则下的分配或限制,account 则决定谁提交任务,以及这个身份可以使用哪些资源。它们在调度时相遇,却回答不同的问题。

复制同事的任务配置,不会同时获得对方的存储权限和资源额度。容器内的进程显示为 root,也不等于调度平台认定的提交者。输出目录和权限检查,应依据平台记录的提交身份,而不是脚本里沿用的某个用户名。

因此,任务创建后的实际元数据很重要:它说明任务最终落在哪里,使用哪份资源分配,由哪个身份提交。

4. Files and services have a scope4. 文件与服务都有边界

A path names a location within a filesystem view. The same path in two environments can refer to different volumes, or to an empty directory where an expected volume was never mounted. Diagnose a missing file by checking the job’s mounts, storage identity, and permissions together.

A service address has a similar context. In the isolated layout shown here, a model service is reachable by permitted jobs in its execution cluster; a development terminal elsewhere does not automatically have that route. Other platforms can expose services across clusters, so the relevant fact is the configured connectivity and access policy, not the cluster name alone.

Service discovery supplies the current endpoint, but knowing that endpoint does not grant a route or permission. Read the service’s published endpoint information and verify reachability from the actual client job. A stale address in a local script can otherwise obscure a healthy service.

路径是在某个文件系统视图里定位文件的名字。两个环境中的同名路径,可能对应不同存储,也可能只是一个尚未挂载预期卷的空目录。遇到文件不存在,应一起核对任务的挂载、存储来源和访问权限。

服务地址也有上下文。图中的隔离架构允许同一执行集群内的获准任务访问模型服务,其他位置的开发终端不会自动获得这条路由。有的平台支持跨集群暴露服务,因此真正需要确认的是配置好的连通性与访问策略,而不只是集群名称。

服务发现解决的是当前地址在哪里,知道地址并不会带来路由或权限。应读取服务发布的端点信息,并从实际发起请求的任务里检查可达性。本地脚本里一条过期的地址,就可能让正常服务看起来像失效了。

5. Two paths to a model5. 到达模型的两条路径

A managed gateway provides an established entry point and model aliases. A self-hosted service gives the operator control over a particular deployment, including its weights and serving settings. Choose between them by the capabilities the task needs, the accessible network, and the responsibility you are prepared to take for operating the service.

Endpoint protocol and model capability should be checked separately. Messages-style and Chat-Completions-style APIs use different request and response formats. Two compatible endpoints can still expose different models or optional fields. vLLM’s OpenAI-compatible server is one concrete example of a serving interface; compatibility does not make all deployments interchangeable.

For scoring or distillation, be precise about the values returned. Logits are scores before probability normalization. Logprobs are logarithms of probabilities; a response may provide only the selected token and a limited set of alternatives. They are not the raw logits or necessarily the full vocabulary distribution. Check the engine, endpoint, and configuration for the exact fields required. Self-hosting alone does not guarantee that they are exposed.

托管网关提供稳定的入口和模型别名;自部署服务则让使用者控制具体的权重和服务参数。选择哪一条路径,要看任务需要什么能力、客户端能访问哪个网络,以及愿意承担多少服务运维工作。

接口协议与模型能力需要分开核对。Messages 风格和 Chat Completions 风格的接口,请求与响应结构不同;即使协议兼容,背后的模型和可选字段也未必一致。vLLM 的兼容服务是一个具体例子,协议兼容并不意味着各个部署可以无差别替换。

评估或蒸馏尤其要分清返回值:logits 是概率归一化前的分数,logprobs 是概率的对数。接口可能只返回选中 token 和少量候选的 logprobs,它们不等于原始 logits,也未必覆盖整个词表。需要哪些字段,应逐项核对引擎、接口与配置;自部署本身并不保证这些字段已经开放。

6. Follow the job across its boundaries6. 沿着任务经过的边界排查

A useful diagnosis begins with the job record. Confirm placement and submitting identity, then compare the resolved commit, image, and configuration with the intended ones. From inside the execution environment, check the expected data and service access. Only then is it useful to investigate the model request itself.

The symptoms point to different boundaries. Old behavior suggests a code or image mismatch. Missing data suggests a mount, scope, or permission issue. A timeout suggests connectivity or service readiness; an invalid request points toward the API contract. A missing probability field calls for a capability check. None of these symptoms, by itself, establishes a model problem.

A running job still needs application-level confirmation that the service is ready and producing the expected output. Keep a small record of the execution context, versions, logs, and resulting artifacts. That record gives the next debugging session a reliable place to begin.

排查可以从任务记录开始:确认执行位置和提交身份,再把实际 commit、镜像、配置与预期版本对上。随后进入执行环境,检查数据是否可见、服务是否可达,最后才看具体的模型请求。

不同现象指向不同边界。行为没有更新,先查代码和镜像版本;找不到数据,查挂载、存储范围与权限;请求超时,查连通性与服务就绪情况;请求格式错误,查协议;缺少概率字段,查接口能力。单独一个现象,还不足以判断是模型出了问题。

任务显示运行后,仍应从应用层确认服务已经就绪,并产生预期输出。留下执行环境、版本、日志和产物的简短记录,下次排查就有了可靠的起点。

The most useful unit of understanding is the job: a specific version of a program, running in a specific environment, with a specific identity and set of connections.理解算力最有用的单位,是一次具体的任务:某个版本的程序,在某个环境里,以确定的身份,通过确定的连接完成计算。

References参考资料

  1. vLLM: OpenAI-compatible servervLLM:兼容服务接口
  2. vLLM: sampling parameters and logprobsvLLM:采样参数与 logprobs

Adapted from remote-development notes. Environment-specific names and operational identifiers have been replaced with generic roles.整理自远程开发实践笔记,环境名称与运行标识已替换为通用角色。