GPT-6 + π0.5
瓶子入桶实验报告
运行中断,结果不完整π0.5 在本地 CPU 上生成动作提案,GPT-6 通过 Codex 检查执行结果与动作意图,并按原项目规则选择执行或有限纠正。仿真与相机渲染由本地 GPU 运行。
01 / 结果与回放
本局结论:运行中断,结果不完整。原生任务要求所有 4 个瓶子入桶并且双臂归位。是否成功由环境的终止条件决定,不以模型文字判断或视频印象替代。
| complete | False |
| native_success | 未获得最终判定 |
| truncated | False |
| 结束原因 | 仿真进程发生 access violation;随后 RPC 连接中断(WinError 10054) |
| 原生得分 | 未形成有效得分 |
按仿真时间播放;模型等待没有保留在视频中,不代表实际墙钟运行速度。
初始观测
最后保存的观测
02 / 混合控制实际做了什么
每轮先由 π0.5 根据三路 RGB、关节状态与原始任务指令预测动作,再由 GPT 读取提案和机器人正向运动学结果,评估上一段执行与下一步意图。只有观察到执行问题或意图不匹配时,才按项目门控规则纠正;纠正幅度受 5 厘米 / 0.35 弧度等原始约束限制。
| 执行类型 | 实际记录 |
|---|---|
| 直接执行 π0.5(student) | 30 次决策 / 447 个执行步 |
| GPT 编辑 π0.5 轨迹(edit) | 0 次决策 / 0 个执行步 |
| GPT 末端位置纠正(eef) | 0 次决策 / 0 个执行步 |
| 提前结束(stop) | 0 次决策 / 0 个执行步 |
查看策略公开进度记录
以下是模型当时的公开观察,可能存在误判;不作为环境成功判定。
| 控制步 | 公开记录 |
|---|---|
| 0 | I’m using the robodojo-hybrid-rollout skill to run this episode, review each motion against the camera evidence, and record the outcome. |
| 0 | The scene contains four bottles and a dustbin on the left. Completion also requires returning both arms to their starting poses, so I’ll reserve motion steps for the final withdrawal and return. |
| 27 | The left gripper has engaged the pink bottle and begun moving it toward the dustbin. I’m checking that the bottle travels with the hand and clears the rim before release. |
| 57 | The pink bottle has dropped inside the dustbin, and the left hand is clear. Three bottles remain on the table; the right arm is approaching the small yellow bottle. |
| 87 | The yellow bottle lifted with the right hand. The next phase is a handover to the left hand so it can reach the dustbin. |
| 117 | The left fingers are approaching the lower half of the yellow bottle while the right hand continues holding it. I’ll inspect the receiving grasp before allowing the right hand to release. |
| 147 | The handover succeeded: the yellow bottle moved away with the left hand, and the right gripper is visibly empty. The left arm is carrying it toward the dustbin. |
| 177 | The yellow bottle has been released into the dustbin, and the left gripper is empty. The upright white bottle and the pale bottle lying on its side are still on the table. |
| 207 | The right fingers have closed around the white bottle. The next motion lifts it toward the handover position; I’ll verify the bottle rises with the hand. |
| 237 | The white bottle is held steadily at the handover position. Two bottles are already deposited, and 463 control steps remain for this handover, the final bottle, and the arm return. |
| 267 | Both hands now surround the white bottle, with the left fingers closed around its body. I’m moving through the release and separation phase to check that the left hand supports it independently. |
| 297 | The white-bottle handover succeeded, and the bottle is now over the dustbin as the left fingers open. Only the pale horizontal bottle remains on the table. |
| 327 | Three bottles are now deposited. The right hand is rotating above the final pale bottle for a grasp from above; the left hand has returned close to its starting area. |
| 357 | The right fingers are aligned around the final bottle’s body and have begun closing. The next check is whether the bottle lifts cleanly from the table. |
| 387 | The final bottle lifted successfully and remains held as the right hand turns it upright. The tabletop is now clear; this bottle still needs its handover and deposit before the arms return. |
| 417 | The left hand is approaching the final bottle while the right grasp stays steady. I’m keeping the handover staged so the receiving fingers close before the donor hand releases. |
| 447 | The right hand has opened while the left continues holding the final bottle. I’m checking the separation before carrying it across to the dustbin. |
本局没有执行 GPT 编辑或末端纠正动作;GPT 实际承担观察、审核与选择执行长度的作用。即使结果优于此前 π0.5-only 单局,也不能据此归因于 GPT 纠错:两局重新推理的频率不同,且各只有一个样本。
03 / 分开统计模型耗时
| π0.5 正式推理累计 | 713.87 秒 |
| π0.5 每次平均 | 23.03 秒 |
| GPT 请求间决策等待累计 | 1236.08 秒 |
| GPT 记录的服务请求数 | 62 |
| 中断前墙钟时间(估算) | 2282.33 秒 |
| 仿真动作时间 | 17.88 秒 |
π0.5 时间由 CPU worker 测量,覆盖模型推理与动作数组取回。GPT 时间从上一条服务回复到下一条请求,包含网络与本地分析工具等待,排除阻塞的 π0.5 服务和仿真执行时间。它不是服务端纯推理时间。
各计时范围起止不同;本地传输、FK 检查、图像处理和录制等也产生开销,因此不强行把这些数字相加解释为精确的总耗时分解。
04 / GPT Token 消耗
| 类别 | 累计 token 数 |
|---|---|
| 输入 token | 8,089,509 |
| 其中:缓存输入 | 7,895,936 |
| 输出 token | 18,138 |
| 其中:推理输出 | 945 |
| 总 token | 8,107,647 |
05 / 配置与三次本地实验对照
| 任务 / seed | put_bottles_into_dustbin / 0 |
| 教师 | gpt-6-astra / xhigh,已登录的 Windows Codex CLI |
| 学生权重 | RoboDojo-sim-arx_x5-joint-0/59999 |
| 权重版本 | 91f76c28d93dd20c5fa46ce6a5a1d96a4f384acd |
| 学生后端 | OpenPI / JAX 0.5.3 / WSL CPU |
| 仿真 | Windows Isaac Sim 5.1 / D3D12 |
| 硬件 | Xeon E5-2696 v4、约 32 GiB RAM、RTX 3060 Ti 8 GiB |
| 观测 | 3 路相机,各 640×480 |
| 本地适配 | 文件传输连接 WSL π0.5 与 Windows 原项目控制器;保留提案、归一化和动作验证规则 |
| 训练 / 量化 | 本局没有训练或量化 |
同任务与 seed 的记录
| 方法 | 本地单局结果 |
|---|---|
| GPT-only | 成功;653 步;37m 28s |
| π0.5-only | 未成功;700 步;11m 47s |
| GPT-6 + π0.5 | 运行中断,结果不完整;447 步;38m 02s |
仅为三个本地案例的描述性对照,不代表总体成功率。控制频率、设备、加载和计时边界不同,不用于宣称模型能力排名或严格加速倍数。
运行与资源补充
| π0.5 加载与预热(另计) | 113.33 秒 |
| 整张显卡峰值占用 | 5715 MiB / 8192 MiB |
显存数据为整张显卡占用,含桌面及其他进程;π0.5 在 CPU 推理。启动前修复了 Windows 权重路径分隔符兼容问题,该次尝试尚未重置任务。正式本局最初的 Codex 网络等待出现超时重试后恢复;相关等待计入 GPT 计时。
06 / 原始记录
图片、样式内嵌;回放和数据随报告文件夹保存。分享时使用完整 ZIP。未包含私有推理文本。