The first direction (and why I chose it)

If an agent can operate any interface, what does the human actually need to see?

The instinct was to design for the browser the way a person would see it. An overlay showing where the cursor is about to click, the speed of a cursor moving through screens. The idea was that if you could watch the agent work, you could trust it. This was the pattern Runner H, Operator, and Claude computer use had all landed on.  It felt like an obviously correct answer to ‘how do I supervise an agent’.

A person watching another person use a browser understands what's happening in theory seems like an approach worth validating. ‍

What broke?

Two problems showed up almost immediately, and they compounded, but one showed promise.
‍
1. Agents in live environments are slow. Not one-click-slow but tediously slow. Real agents in real browsers take 20 to 60 seconds per meaningful step. A four-step task is several minutes of watching a page load, a cursor move, a selector resolve. So a person watching that is not supervising, they’re merely waiting and becoming increasingly impatient.
‍
2. Watching was uninformative. Even when the agent was working, the live view told the supervisor almost nothing about whether the task would succeed but feed back information about actions and tool-use. Seeing the agent click the right search result looked the same as seeing it click the wrong one. The visual fidelity was doing emotional work (look, it's working) but not supervisory work (here is the specific thing you need to decide about). The pattern optimised for the feeling of oversight, not the reality of it.
‍
3. Surface decision points. Some actions are low-stakes and should proceed automatically. Others (form submissions, purchases, account changes, anything irreversible) require explicit confirmation and often personal information. The interface needed to make that distinction consistent without turning every click into a dialog.

The interface I'd built was tracking these actions, but through testing it minimally useful as a tool but had some potential at completing background tasks like booking. A user would sit through two minutes of slow animation and still have no idea whether to trust the result.

The reframe

The move that saved the project was stopping trying to make one agent legible and starting to treat the agent as a worker you shouldn't have to watch at all. If one agent is slow, run several in parallel and chain them together. If watching a single browser is boring and uninformative, don't watch any of them. Show their work as a running feed through chat or through the browser, or through status's and results. The interface looks less like a browser and more like a chat with multiple running queries. The actions are obscured and removed but logs of action and steering still needs to occur.

This felt like a different supervision model. In the first version, the supervisor is a pilot with hands on the controls and in the second, the supervisor is a manager reading status updates from a team.

Concretely, the reframe changed three things:
→ Live browser became optional. It survived as a drill-down for when a specific run looked wrong, not as the primary surface.
→ Parallel runs became the unit of interaction. One task input could fan out to several agents hitting different sources or trying different strategies. The user sees a feed of concurrent work, not a single animated movie.
→ Text became the main channel of delivery, oversight and stereing. Plans, step summaries, and results are compressed into short written updates a person can scan in two seconds.

I built a small prototype of this version in the remaining time. It was visually less impressive than the browser-mirror version. It was also the version that worked.

What this unlocked

The parallel-agents-as-text-feed prototype was the seed for two subsequent products: Relay (supervision interfaces for long-running agents) and Flint (natural-language app builder). Both sit on the insight that arrived too late to ship inside Webglide's four weeks: when agents are slow and individually opaque, the interface has to scale horizontally, not pay for visual fidelity you can't cash.

That's the main thing Webglide changed in how I think about agent interfaces. The 2025 consensus (Runner H, Operator, the split-pane live-browser pattern) was an older pattern which didn't take off in usage and development. It doesn' work for the actual supervision problem, which is about managing many runs, most of which you will never look at closely, and knowing which ones need your attention.

What I'd do differently

Remove the live-browser instinct a week earlier, although the idea of seeing an agent navigate through screens felt novel and interesting, I spent the first two weeks building something that looked like Runner H because the category had legitimised that shape.

A user test on day four would have told me that nobody would want to observe an agent take tasks on your behalf and the stylistic approach could have been developed different. But it was an important one, as the pattern that emerged seemed to validate this. The pattern was later changed by the inclusion of the Tool-use and computer use in the chat, embedding this in chat. The lesson I took forward into Relay was to test the supervision model against the actual failure modes, before committing to a visual direction.

What this opened up

The broader principle that came out of Webglide is that as agents become more capable the interface becomes more important, not less. Capability without observability makes products hard to scale and that element of liability, breakdown of trust and oversight and observability layer makes scaling to users difficult and poor.

Four weeks was enough to establish the spine of the idea, as well as take approaches and techniques for super-apps and wider principles into conversational and chat interfaces for agents.

The problem of managing and steering multiple agents and swarms, all using multiple tools, actions and projects was a really interesting design prospect and challenge. This was a fast validation loop to continue to design for this product and scale is safely and steerably.

About CUGA

CUGA (ConfigUrable Generalist Agent) is an open-source generalist agent framework from IBM Research, purpose-built for enterprise automation. Designed for developers, CUGA combines and improves the best of foundational agentic patterns such as ReAct, CodeAct, and Planner-Executor — into a modular architecture enabling trustworthy, policy-aware, and composable automation across web interfaces, APIs, and custom enterprise systems.
CUGA achieves state-of-the-art performance on leading benchmarks:
→🥇 #1 on AppWorld — a benchmark with 750 real-world tasks across 457 APIs
→🥈 #2 on WebArena — a complex benchmark for autonomous web agents across application domains.

Key features relevant to the supervision question:
‍→ Complex task execution: State of the art results across Web and APIs.
→ Flexible tool integrations: CUGA works across REST APIs via OpenAPI specs, MCP servers, and custom connectors.
→ Composable agent architecture: CUGA itself can be exposed as a tool to other agents, enabling nested reasoning and multi-agent collaboration.
→ Configurable reasoning modes: Choose between fast heuristics or deep planning depending on your task’s complexity and latency needs.
→ Policy-aware instructions (Experimental): CUGA components can be configured with policy-aware instructions to improve alignment of the agent behavior.
→ Save & Reuse (Experimental): CUGA captures and reuses successful execution paths, enabling consistent and faster behavior across repeated tasks.

ReAct Agents

ReAct (Reasoning and Acting) agents are a powerful framework for AI agents that enhance a Large Language Model’s (LLM) ability to handle complex, multi-step tasks by integrating logical reasoning (Thought) with external action (Action) in a continuous, interleaved loop. The core idea is to prompt the LLM to alternate between articulating its chain-of-thought to plan and track its progress, and executing actions like using a search engine, calling an API, or interacting with a database.

More Here: https://research.ibm.com/blog/cuga-agent-framework