What happened
Pranjal Aggarwal and Sean Welleck’s position paper, “Computer-Use Agents Are Embodied Generalists,” frames embodied tasks as interactions with digital software. The researchers built an application for each of four ECCV workshop competitions. Each app rendered a scene, scan, video, or panorama and gave the agent controls through screenshots, keyboard actions, and mouse clicks. Across the competitions, the team kept the same agent implementation, system prompt, and action space; only the app and task description changed. They used Gemini 3.5 Flash and Gemini 3.6 Flash as released, without fine-tuning or competition-specific agent tuning.
The four entries covered distinct tasks: language-guided navigation in VLNVerse, dialogue navigation in DialNav, 3D affordance segmentation in OpenSUN3D, and video object segmentation in MeViSv2. The agent placed sixth in VLNVerse, third in DialNav, first in OpenSUN3D, and second in MeViSv2. The reported competition metrics vary by task. OpenSUN3D’s AP25 was 36.52, compared with 26.50 for the best other entry; the researchers describe this as about a ten-point lead. In VLNVerse, the entry reached an average SR@3 of 87.30. These are competition results in different settings, not a shared measure of general ability.
The authors also compared GUI and API access in a subset of each competition’s validation set, using the same model and underlying operations. The paired scores were 83.9 and 85.2 for VLNVerse, 67.4 and 63.4 for DialNav, 26.7 and 28.1 for OpenSUN3D, and 95.8 and 94.6 for MeViSv2, respectively. The two interfaces stayed within four points in each comparison, and the GUI scored higher in two competitions. The paper treats them as alternative interfaces to the same software, while noting that their efficiency may differ.
A separate preliminary experiment tested whether robot actions need a specialized action head. The team trained Qwen3.5 9B, a standard vision-language model, and π 0.5 on the same 1,000 demonstrations of two tasks in a two-arm simulation. Qwen3.5 9B wrote actions as plain text without an action head and succeeded on 56% of 1,000 unseen instances; π 0.5 succeeded on 59%. In that setup, the 9B model received a camera image every second and spent about 10% of total time on prefill, 20% decoding, and 70% idle. The paper proposes simulator-generated grounding data, reinforcement-learning environments with verifiers, and faster models and harnesses as a path for embodied AI.
What the archive adds
New Runtime’s earlier coverage described Holo4 as a model series for computer-use and tool-driven workflows (October 24, 2026 note). Another note examined how page-loading delays can cause stale clicks in computer-use agents (October 31, 2026 note). This is a new research claim about carrying computer-use methods into embodied tasks, rather than a new result in either earlier story.
Trend signal
This supports, but does not establish, the view that AI-native work shifts toward review and orchestration: the paper’s simulated tasks use explicit verifiers, and its apps leave decisions and checking to the agent. The broader assumption is that success depends on accepted outcomes and review, not output volume (the pattern). The results here concern research tasks, not enterprise adoption.
Why now
The authors draw a parallel with computer-use agents: in their account, visual grounding, reinforcement-learning environments, and speed optimization were successive phases of progress. They argue embodied AI could follow a similar sequence, using simulators that expose object positions and task outcomes as software. This is the paper’s proposed research program, not evidence that the sequence will necessarily transfer to real robots.
Connections to other events
New Runtime’s earlier coverage has also examined the operational reliability of computer-use agents: a study found that page-loading delays, rather than planning latency, drove stale clicks in tested workflows (October 31, 2026 note). The present paper moves the question into embodied tasks. Its authors argue that simulators can provide ground-truth state to verifiers even when an agent sees only camera images, while real-time physical control remains a separate challenge.
Why it matters
For teams building agents, the most actionable proposal is to treat a simulator as a software surface: render observations, expose controls, and make task success checkable. That structure could let a general-purpose agent use familiar visual grounding and exploration skills across tasks, while leaving task-specific operations in the surrounding application. It also makes evaluation more concrete than asking whether a model appears capable: each task can have a defined metric or verifier.
The evidence is promising but bounded. The four competition results depend on the apps and tools the researchers built, and the separate robot comparison is a two-task simulation experiment with 1,000 demonstrations. A near-match between two models there does not show that plain-text actions are sufficient for dexterous real-world control. Teams should read the work as a reason to test reusable interfaces and verifiers, not to remove specialized robotics systems from production.
Limits and uncertainty
The paper’s four competition placements do not establish performance on arbitrary embodied tasks or in physical environments. Its API/GUI comparison covers only a subset of validation data, and the metrics differ across competitions. The Qwen3.5 9B versus π 0.5 result is preliminary, limited to two simulated tasks and two robot arms; the reported success rates are close, but do not resolve whether specialized action heads help on more demanding tasks. The paper does not establish how reliably these methods transfer from simulation to real robots, or how much teleoperation and egocentric video data future systems will need. The authors themselves say the data requirement remains unclear.
Skill to build now
If your team owns a simulator, try exposing one task as a small agent-operable app this week: provide a rendered observation, a limited set of actions, and a deterministic success check based on simulator state. Compare the same model through GUI controls and an API where possible, keeping the underlying operations fixed. Record task success and completion time separately; the paper’s results suggest that interface choice and speed are distinct questions worth measuring.
Reader takeaway
The paper’s strongest contribution is a testable way to connect computer-use agents and embodied AI: expose simulated tasks as software, then reuse an agent’s visual grounding and action loop. Competition placements and a small simulation experiment make the proposal credible enough to investigate, but not enough to conclude that general computer-use agents are already general-purpose robots.