Four Compute Bets for Computer-Use Agents

“Give the agent more compute” sounds like one engineering decision. It is at least four.

You can give a computer-use agent more history. You can let it take more steps. You can split planning from execution. You can generate several candidate plans and select one. Those interventions spend inference compute differently, change different parts of the system, and can introduce different failures.

For local computer-use agents, compute is a budget, not a guarantee. Contextual, temporal, structural, and parallel scaling should each earn their cost through measured task completion and an acceptable failure profile. Turning every knob up at once makes a run more expensive. It does not tell you which change made the agent more reliable.

What the preprint found—and what it did not prove

A July 2026 preprint from Woongkyu Lee and Jungwook Choi examines those four forms of inference-time scaling in local computer-use agents.[^1] Their experiments use 361 OSWorld tasks in an Ubuntu environment after excluding eight Google Drive tasks that could not be run under their constraints. The agents receive screenshots rather than accessibility trees, and inference runs on one A100-80GB GPU.[^2]

That is useful evidence, but it is not a universal law for agents. It is a preprint, not peer-reviewed consensus. The study covers a specific benchmark, local vision-language models, a screenshot-only setup, and particular agent architectures. It does not report confidence intervals or wall-clock latency as a primary metric (Lee and Choi, Sections 3.1–3.4).

Lee and Choi test:

• Contextual scaling: retaining more screenshot history.
• Temporal scaling: increasing the maximum number of agent steps.
• Structural scaling: comparing a single agent with a two-stage planning and grounding architecture.
• Parallel scaling: generating more candidate high-level plans inside that two-stage architecture before a judging process selects one.[^2]

These are not interchangeable. A history window can improve the next decision without increasing the maximum trajectory length. A larger step budget can reduce timeouts while preserving a bad trajectory. A planner can add coordination overhead before the grounder takes an action. More candidate plans can rescue a malformed plan while multiplying token use.

Context can buy signal—and a different failure profile

In the tested configurations, some history helped sharply before returns flattened. The authors report a substantial improvement when agents move from no retained screenshots to a small history, with additional benefit at four screenshots and no continued linear gain at eight. Their Figure 3 makes the mechanism concrete: one trajectory without history repeats a menu-click loop, while a trajectory with four retained screenshots keeps enough state to complete the multi-step task (Lee and Choi, Section 4.1 and Figures 2–3).

That is a case for buying useful context, not for buying all available context.

The failure distribution matters too. In the Qwen3-VL-30B-A3B cohort shown with a 15-step limit, longer history reduced max-step stalls while premature false-success classifications increased.[^3] In the authors’ usage, that means the agent terminates even though the task is incomplete. The paper does not fully document the annotation procedure for that failure taxonomy, so I would not turn it into a broad claim about agent psychology or “confidence” (Lee and Choi, Section 4.1, Finding 2).

The operator question is concrete: did more history improve verified completion, or did it mainly move runs from an obvious timeout into a cleaner-looking wrong answer?

I use the same principle when treating agent context as a lifecycle problem: retain information because it changes a later decision, not because retaining it is technically possible.

Longer trajectories can preserve the wrong trajectory

A step budget controls how long an agent may keep acting. It does not control whether those actions remain useful.

Lee and Choi test maximum trajectories of 15, 50, and 100 steps. Across most tested models, larger budgets reduced max-step failures without materially raising the task-success ceiling. Qwen3 models did improve somewhat between 15 and 50 steps, with marginal gains beyond that range, so “longer horizons never help” would overstate the result.[^4]

The distinction operators need is between room to recover and permission to continue.

If successful runs often need 22 steps, a 15-step cap is an artificial failure. Raising it may unlock real completion. If an agent has already misread the interface and is repeating irrelevant actions, another 85 steps fund drift. In both cases the trace gets longer. Only one case improves the outcome.

Do not use fewer max-step stalls as the success metric. Define verified completion first, then inspect how the added budget changes completion, token use, latency, and failure severity.

Decomposition has coordination overhead even inside one agent

Breaking work into planning and execution can make a difficult task easier to reason about. It also creates an interface between the planner’s output and the component that must act on it.

In Lee and Choi’s tested two-stage setup, Qwen models generate and judge high-level plans before GTA-1-7B grounds the selected plan into an action. That architecture underperforms the corresponding single-agent baselines across the tested history lengths while consuming more prompt tokens.[^5] The authors report incomplete or visually misaligned plans as well as planner outputs that violate the required format. Figure 6 shows one planner omitting a required observation field and placing an action outside the expected code block, so the downstream component cannot parse it (Lee and Choi, Section 4.2 and Figures 5–6).

That is not proof that decomposition makes agents worse. My operating conclusion is that decomposition has to repay its own interface cost.

The right comparison charges planning tokens, formatting failures, handoff ambiguity, and additional model calls against any reduction in task complexity. A decomposition strategy that produces a smarter plan but a less executable contract has not improved the system.

Parallel plans buy diversity at a price

The paper’s parallel experiment is narrower than “run several agents and vote.” It generates multiple candidate high-level plans inside the two-stage agent, then uses a judging process to select one before grounding.[^6] The exact judging protocol is not described well enough to infer a general selection algorithm.

As the number of candidate plans rises from one to four, the authors report that the two-stage system partially recovers performance and reduces format-related failures. A valid candidate can survive when another candidate is malformed. But the setup does not close the gap with the single-agent baseline, and prompt-token use rises sharply (Lee and Choi, Section 4.2, Finding 5).

Parallelism can therefore be a targeted recovery tactic. It is not free reliability.

Use it where attempt diversity addresses a known failure mode and the task value justifies multiplied compute. For routine tasks with a strong baseline, generating four plans may be an expensive way to confirm what one plan already gets right.

Use an inference compute allocation scorecard

The paper measures the interventions; the allocation framework below is my operating synthesis.

I use a deliberately small decision unit: one compute-scaling tactic applied to one defined task class, tested against a fixed baseline under otherwise matched evaluation conditions.

“Task class” matters. I would not assign a browser form submission, spreadsheet cleanup, and open-ended research task one global history, trajectory, and recovery budget. That setting would hide the decision I am trying to measure.

Score each candidate tactic on six factors:

1. Marginal verified task success. How many percentage points did completion improve over the fixed baseline?
2. Failure-mode shift and severity. Which failures disappeared, which appeared, and did the change make failures harder or riskier to detect?
3. Incremental token or compute cost. How much additional inference did the outcome require?
4. Incremental latency. Did the added wall-clock delay remain acceptable for this workflow?
5. Run-to-run variance. Is the gain repeatable, or does it depend on occasional lucky trajectories?
6. Predeclared stop rule. Which threshold ends the experiment when gains flatten, costs rise, or failure behavior worsens?

For comparison within the same task class and matched setup, one useful derived metric is:

Percentage-point success change per 1,000 incremental tokens = (candidate success rate − baseline success rate) ÷ (average incremental tokens per run ÷ 1,000)

I do not compare that number across unrelated tasks, models, benchmarks, or success definitions. I report the baseline and candidate sample counts, uncertainty where it can be estimated, and the distribution of incremental token use. I also show the latency change and whether new failures are harder or riskier to detect. The metric is a decision aid, not an objective function.

The preprint does not provide enough extractable per-configuration token totals, repetitions, and uncertainty to calculate this operator metric responsibly from its prose. I am not presenting it as one of the paper’s results.

Allocate compute by task and failure mode

A useful allocation loop has five steps:

1. Baseline. Define verified completion and measure a representative set for one task class.
2. Add one tactic. Change contextual, temporal, structural, or parallel scaling while holding the model, decoding, tools, environment, and success criteria constant.
3. Measure the outcome. Compare verified completion with the baseline. Do not substitute longer traces, fewer stalls, or better-looking plans for completion.
4. Inspect the failures. Look for premature false-success classifications, preserved bad trajectories, planning-interface failures, and costly attempt multiplication.
5. Allocate or stop. Keep the tactic only when it clears the thresholds defined in the scorecard. Otherwise revert it or test a different tactic.

The loop starts with a declared stop condition. I stop when completion gains flatten, token cost or latency crosses the task’s budget, variance remains unacceptable, or the failure profile becomes more dangerous.

I use that last condition to avoid optimizing only for visible failure counts. If downstream systems accept an incomplete early termination as success, I would rather preserve an obvious stall than fund a quieter failure.

The same budget logic applies to recovery. As I argued in Coding-Agent Recovery Is a Budget Decision, persistence should be conditional on evidence that another attempt can change the result. Local computer-use agents need the same discipline at inference time.

The question is not whether more compute works. The useful question is which additional unit of compute changes this task’s verified outcome enough to justify its cost, latency, variance, and altered failure profile.

Fund that intervention. Stop funding the rest.

Sources

[^1]: Woongkyu Lee and Jungwook Choi, “Rethinking Inference-Time Scaling in Local Computer-Use Agents: Failure Modes and Compute Tradeoffs,” arXiv:2607.28573v1, July 2026. Abstract and Section 1. [^2]: Lee and Choi, Sections 3.1–3.4, PDF pp. 3–5. Experimental setup, scaling definitions, benchmark, models, and hardware. [^3]: Lee and Choi, Section 4.1, Findings 1–2; Figures 2–3, PDF pp. 5–6. Contextual scaling and the configuration-specific failure cohort. [^4]: Lee and Choi, Section 4.1, Finding 3; Figure 4, PDF pp. 6–7. Temporal scaling. [^5]: Lee and Choi, Section 4.2, Finding 4; Figures 5–6, PDF pp. 7–8. Structural decomposition and planner-format overhead. [^6]: Lee and Choi, Sections 3.2.3 and 4.2, Finding 5; Figure 5, PDF pp. 4 and 7–8. Parallel candidate-plan scaling.