How the agent talks to its computer

How to read this page

Claims carry one of these tags. Solid tags are things a source says. Dashed tags and dashed boxes are things we inferred or couldn't find. Untagged notes and takeaways are our own explanation of the tagged material.

Quick answers

The questions people ask first. One row each, with the evidence and a link to the section with the details.

System map

Three zones: your apps, the server side (where the loop, the conversation and memory live), and the per-user VM (where tools run). Dashed boxes are components we believe exist but that no source names.

Bots, accounts and the one computer

In Grok Bot, a "Bot" is a named agent you create, with its own role and instructions. One account can have several. They each get their own chat, but they all run on the same computer. The colours below follow each Bot from the account to the server and onto the computer. The Bot names are made-up examples.

Team Bots: one Bot, many conversations

A Team Bot is a Bot its owner publishes for the whole team. It keeps one setup and one team memory, but its conversations run on different computers and with different accounts, depending on who is talking and where. The docs describe this behaviour; how it's wired underneath is our reading.

Every conversation uses the same Team Bot setup. Which computer and whose accounts it uses depend on the conversation. xAI docs Group chats in the app are only listed in Cursor's Teams docs. Whether the Bot's own computer is one per Bot or one per channel isn't stated. Unknown

What the computer is actually for

The computer does much more than browse the web. Anything that needs a real operating system happens there: running programs, keeping files, holding logged-in sessions, appearing on the network. The thinking, the memory and the connections to your apps all stay on the server.

Where everything lives

Every piece of data we could find, where it's kept, when it's written, how it gets used, and whether it belongs to one Bot or to the whole account.

What one turn reads and writes

The same storage, seen through one turn: what gets loaded before the first model call, what's written while it works, and what happens afterwards while you're away.

The loop as a Temporal workflow

A plausible event history for one task, using the pieces the sources describe. The workflow code only makes decisions; every call to the outside world is an activity, and Temporal records each result.

Routing a tool call

The model asks for a tool. Before anything touches a VM, the server has to decide where the tool runs and whether the computer may be woken for it.

A tool call on a sleeping computer, step by step

The most involved path through the system. The dot on each message shows its evidence. Most of the middle is inferred, because no source describes the wire protocol.

How work starts and how results go out

Every turn starts with a trigger and ends with a reply, a draft, an approval request, or nothing at all. All of it is handled on the server.

Who connects to whom

Two candidate designs for reaching the VM. The evidence suggests both are used, for different jobs.

Sleep and wake

The computer sleeps a few minutes after the last activity, never in the middle of a turn, and wakes on the next tool call that's allowed to wake it.

Inside the VM

Three nested layers, and a clear split between what survives and what doesn't.

Three kinds of state, kept apart on purpose

Cursor describes the agent loop, the machine and the conversation as three separate components. Each one can fail or be replaced without the other two noticing much.

What happens when things fail

Each failure mode, what absorbs it, and where the evidence comes from. The amber row is the one we'd watch most closely if we built this.

Security boundaries

Where the walls are, mapped onto the architecture.

How we'd build it

A minimal version of the inferred design, in TypeScript with the Temporal SDK. It's here to make the moving parts concrete. We wrote it; it isn't based on anyone's real code.

workflow.ts · runs on server workers// One workflow per task. No I/O here: only decisions.
export async function agentTask(convId: string) {
  let inbox: Msg[] = [];
  setHandler(newMessage, (m) => { inbox.push(m); });

  const state = await loadState(convId);
  for (;;) {
    const step = await inference(state.prompt(inbox));
    inbox = [];
    if (step.kind === 'send_to_user') {
      await sendToUser(convId, step.message);
      // wait briefly for follow-ups, then exit
      if (!(await condition(() => inbox.length > 0, '2 minutes'))) return;
      continue;
    }
    const result = await toolCall({ convId, callId: step.callId,
                                    tool: step.tool, args: step.args });
    await saveStep(convId, step, result);  // append + signal apps
    state.add(step, result);
  }
}

activities.ts · the tool routerexport async function toolCall(c: ToolCall) {
  if (SERVER_TOOLS.has(c.tool)) return runOnServer(c);

  const vm = await controlPlane.state(c.userId);
  if (vm === 'asleep') {
    if (!WAKING_TOOLS.has(c.tool))
      return error('The computer is asleep. Use Shell ' +
                   'or Read to wake it, then try again.');
    await controlPlane.restore(c.userId);
  }
  // the worker reconnects after a restore
  const worker = await gateway.waitFor(c.userId, '60s');
  return worker.call(c.callId, c.tool, c.args);
}

worker.ts · runs inside the VM, started by the supervisor// Dials out, so the VM needs no inbound ports and survives restores.
for (;;) {
  const conn = await dialOut(GATEWAY_URL, { workerId: WORKER_ID });   // same ID after every restore
  for await (const call of conn.calls()) {
    // de-duplicate by call ID: Temporal may retry a call that already ran
    const result = (await doneLog.get(call.id)) ?? (await doneLog.put(call.id, await execute(call)));
    conn.reply(call.id, result);
  }
  // connection dropped (snapshot restored, network blip): loop and reconnect
}

Open questions

Things we looked for and didn't find. If anyone on the team can find out more, these are worth asking.

• The worker protocolWebSocket, gRPC streaming or long-poll? How do the worker and the server authenticate each other?
• Hosted vs. self-hostedDo hosted VMs run the same dial-out worker as self-hosted machines, or something else?
• vsockDoes anything use Firecracker's host-to-guest socket instead of the network?
• What "2 idle minutes" endsThe workflow? The worker's cache? A separate session process?
• Call de-duplicationHow are retried tool calls kept from running twice inside the VM?
• The SAND_ runtimeWhat is the "computer runtime" that reserves these environment variables?
• Where policy is enforcedIs the network allowlist applied at the host or inside the VM?
• File sync before sleepWhich files are synced to cloud storage before the computer sleeps, and is that the durable disk or a separate copy?
• Where the review model runsOn the server next to the loop, or somewhere else? The docs don't say.
• Attachments and secretsWhere are personal-Bot chat attachments and the values from secure secret requests stored? (Team Bot uploads land on the chat's computer.)
• Web search and image generationServer-side tools, or something that runs on the computer?
• Team Bot routingHow does a conversation get its computer and accounts? Is the Bot's own computer one per Bot or one per channel, and does it persist? Why do command-line MCP servers that need secrets only work in the owner's chat?
• Where helpers runCursor says a computer-use helper shares the VM's Chrome and VNC, and that some helpers run on a different kind of pod. Which Grok Bot helpers run where isn't documented.

Sources

Primary sources first. The talk itself isn't online as a transcript; the slides and our transcript are covered on the talk notes page.

1. Grok Bot overview (xAI docs)
2. Grok Bot: the computer and apps (xAI docs)
3. Grok Bot security (Cursor docs)
4. Grok Bot for Teams and Enterprise (Cursor docs)
5. What we've learned building cloud agents (Cursor blog)
6. Run cloud agents on machines you manage (Cursor blog)
7. Self-hosted agents (Cursor docs)
8. Bring your own machine (Cursor docs)
9. Self-hosted machines help (Cursor help)
10. Cursor agents can now control their own computers (Cursor blog)
11. Bots (xAI docs)
12. Team Bots (xAI docs)
13. Settings and notifications (xAI docs)
14. Teams and enterprises (xAI docs)
15. Skills, routines and automations (xAI docs)
16. Chat and collaboration (xAI docs)
17. Approvals, security and privacy (xAI docs)
18. Files and results (xAI docs)
19. Grok Bot 101 (xAI guide)
20. Grok Bot security FAQ (Cursor docs)
21. Routines help (Cursor help)
22. Grok Bot architecture explained (third party)
23. A deep dive into Grok Bot (third party)
24. Building always-on agents (third party summary of the talk)