Guiding, Not Clicking: A Browser–LLM Guidance Protocol

When an AI assistant lives inside a web page and helps a visitor use that page, it needs two abilities that pull in opposite directions: it must see the current state of the page well enough to reason about it, and it must point at real interface elements precisely enough to guide a human hand. This analysis describes the communication protocol we built between the visitor’s browser and the language model to do both - and the constraint that shapes every part of it: in text guidance, the assistant does not act. It guides, and the visitor clicks.

To our knowledge there is no public equivalent of a guidance-only browser–LLM protocol; browser agents that use a similar page representation exist, but they use it to act on the visitor’s behalf, which is precisely what this protocol refuses to do.

Why not send the DOM?

The obvious design is to send the model the page’s DOM (cleaned up, perhaps) so it can reason about structure and produce CSS selectors. We measured this on real sites before writing code, in the browser, on the rendered DOM. In our previous analysis we measured the same problem in characters; here we measured it in tokens, with the runtime model’s tokenizer.

Three things follow from that table:

1. Demo sites lie. Our demo page costs 6,202 tokens as simplified DOM; the production storefront costs 335,106 - 54× more. Any protocol validated on a demo is validated on the easy case.
2. Stripping attributes helps enormously and still fails. Removing class, id and data-* cuts the storefront from 335,106 to 47,216 tokens - an 86% reduction that is still an order of magnitude away from a sustainable per-question budget.
3. Rendered text plus an indexed element list costs 2,866 tokens on the same page - a 99.1% reduction against the simplified DOM, and cheap enough to send fresh on every question.

We also evaluated an existing DOM-compression library (D2Snap) on the same storefront page. Its adaptive mode could not compress below 43,508 tokens - almost exactly where our own hand-written attribute stripping landed. That convergence is the interesting result: the hard floor is not an implementation detail, it is the markup itself. Compression solves “how do I send the page in fewer tokens.” The real question is “why am I sending the page’s markup at all.” (One idea from that library is worth keeping: scoring attributes by semantic value instead of stripping by name. We credit it, and skipped the dependency.)

The snapshot: what the browser sends

On every question, the SDK captures a live snapshot of the page as two segments:

• Rendered visible text - what the visitor actually sees, for answering questions about content.
• An indexed list of interactive elements (capped at 200), each entry carrying only an index, tag name, visible label, and - where relevant - its link target and state:

[0] A "Skip to content"
[1] A "Atlantic Solar"
[2] BUTTON "Solutions"
[3] A "Residential Solar"

The crucial design decision is what does not travel. The mapping from index to actual DOM node lives in a JavaScript Map in the browser’s memory and is never sent to the server. The model sees [2] BUTTON "Solutions"; only the browser knows which node that is. The server holds no DOM, no selectors derived from the live page, and no HTML - it cannot, even in principle, reconstruct the page structure from what it receives.

The response: a closed vocabulary of guidance actions

The model’s reply is a validated structure: a message for the visitor plus at most ten actions from a closed, typed union - highlight an element, show a tooltip, scroll to a target, wait for an element to appear (bounded at 15 seconds), ask the visitor a clarifying question, or mark the task complete. Every targeting action addresses its element through a four-tier reference: the snapshot index, a stable element id recorded when the site was first scanned, a raw selector, or a visible-text fallback.

The union is closed on purpose: an action type the schema does not know is rejected at validation, before anything touches the page. The model cannot invent a new capability - the text guidance vocabulary has no “click,” no “fill,” no “navigate,” so no prompt injection and no model error can produce one. Optional features that a site owner turns on separately are outside this protocol.

Anchoring: resolving a reference to a real element

When an action arrives, the SDK resolves its target through the tiers in order. The index tier is the interesting one: because the snapshot kept a direct reference to the DOM node, resolution is a map lookup - no selector query, so there is nothing to mismatch. If the node has been removed from the document since the snapshot (single-page apps re-render constantly), the SDK falls back to finding the element by the snapshot’s own cached label - which costs the model zero extra output tokens, because the fallback data was captured client-side.

If every tier fails, the SDK fails safe: it records a target_missing outcome for the site owner’s telemetry and tells the visitor, honestly, that it cannot find the element on this page. A guidance system that guesses is worse than one that admits it lost the thread - a spotlight on the wrong button teaches the visitor to distrust every future spotlight.

Crossing page boundaries

Guidance rarely ends on the page where it started: the visitor clicks the highlighted link and the page reloads, or an SPA swaps its view without reloading. In both cases the browser sends an automatic continuation request under the same session, carrying a fresh snapshot of the new state, so the model can lay out the next step. Continuations are capped at two per original question, and they are not billed as answers - the visitor asked one question; the multi-page walk to the answer is our cost, not the customer’s.

Note what carries the conversation across the boundary: the visitor’s own click. Not a programmatic navigation. The click the visitor performs on the real interface is simultaneously the act of using the website and the signal that guidance should continue.

The constraint that shapes it all: guiding, not clicking

Early in the design, the action vocabulary contained programmatic interaction. We removed it deliberately, and the removal is now a design rule for text guidance: the assistant draws a spotlight - a visual ring around the target element, rendered with pointer-events: none, no page dimming, the host site fully interactive throughout - and the visitor performs every action themselves.

The security and privacy consequences fall out of the protocol rather than being bolted on: the server never sees the DOM; the index-to-node mapping never leaves the browser; the text guidance vocabulary cannot express an action on the page; and what a visitor types into the site’s forms is never part of any snapshot. The assistant is an overlay that points, in the most literal sense.

Summary

Send the model what the visitor sees, not what the browser parses. Address elements by index into a browser-held map, never by shipping structure to the server. Validate the model’s output against a closed vocabulary that cannot express actions. Fail safe, continue across navigations on the visitor’s own click, and keep the human’s hands on the wheel the entire time. On production pages this costs 2,866 tokens where the DOM-shipping design costs 335,106 - the architecture that respects the visitor’s autonomy is also two orders of magnitude cheaper.

This protocol is deployed in production in AskMyWebsite AI’s concierge. The methodology - in-browser measurement on rendered DOM, with the runtime model’s tokenizer - is reproducible on any public website.