UI-IR

Overview

The User Interface Intermediate Representation (UI‑IR) is the structured layer between agent-emitted intents and the final rendered components. Rather than letting a model emit raw markup or component trees, agents produce a declarative document — the UI‑IR — that the system can validate, lint, diff, and render deterministically.

The IR sits at the boundary of trust: everything before it (agent output) is untrusted; everything after it (rendered UI) is trusted, constrained, and auditable.

Why an intermediate representation?

Agents could emit component names and props directly, but this creates several problems:

• No validation boundary. Invalid or dangerous component combinations reach the renderer unchecked.
• Tight coupling. Agents must know the component API surface, which changes as the design system evolves.
• No diffing or linting. Free-form output cannot be meaningfully compared across renders or checked for consistency.
• No audit trail. There is no inspectable artifact between "what the agent wanted" and "what the user saw."

The UI‑IR solves each of these by providing a stable, typed contract that decouples agent output from component implementation.

Structure

A UI‑IR document is a JSON tree. Each node declares its intent, data bindings, state, and children. The shape is intentionally small — complexity lives in the intent library and component schemas, not in the IR itself.

{
  "intent": "collect",
  "params": {
    "fields": [
      { "key": "email", "type": "email", "required": true },
      { "key": "name", "type": "text" }
    ],
    "validation": { "email": "format:email" }
  },
  "children": [
    {
      "intent": "review-and-confirm",
      "params": {
        "summary": "$parent.values",
        "actions": ["submit", "back"]
      }
    }
  ]
}

Key properties

Pipeline

The UI‑IR passes through a deterministic pipeline before anything is rendered.

1. Parse

Raw agent output is parsed into a UI‑IR document. Malformed output is rejected immediately.

2. Schema validation

Each node is checked against the intent's declared schema — required params, allowed types, forbidden combinations (e.g. a destructive action without a confirmation gate). Trust requirements are verified: nodes that reference trust-gated intents must have matching trust tokens.

3. Constraint solving

If validation fails, a constraint-solver pass attempts auto-repair — for example, injecting a missing ConfirmDestructiveAction before a Delete. Repairs are logged so agents can learn ("I added X because Y was missing"). If the document cannot be repaired, it is rejected entirely.

4. Lint

The linter enforces determinism and UX quality:

• Ordering — the same intent set always produces the same node order.
• Layout budgets — attention, interaction, and information-density checks per viewport.
• Cognitive-load limits — flags multiple primary actions, too many interactive controls, or scanline violations (content that doesn't read sensibly top-to-bottom).
• Accessibility invariants — every interactive node has a label, focus order is coherent, contrast is met via tokens.

5. Render

The renderer maps each validated node to concrete components, resolving tokens and injecting props. This is the only stage that produces DOM or native views.

6. Hydrate

Components are progressively hydrated with guaranteed stable heights to avoid layout shift. Skeleton states preserve layout and can indicate what will appear ("Results list", "Price breakdown") without showing incorrect content.

Capabilities and trust gating

Every node in the IR carries an implicit capability set derived from its declared intent. The validator cross-references these capabilities against the agent's permissions:

Nodes that claim capabilities their agent doesn't hold are stripped or rejected, preventing agents from constructing phishing-like flows or hiding destructive actions behind benign styling.

Diffing and audit

Because the UI‑IR is a stable JSON tree:

• Diffs between two versions of a UI are structurally meaningful — a reviewer (human or automated) can see exactly which intents changed, which params shifted, and which nodes were added or removed.
• Audit trails can preserve the IR alongside every rendered view, creating a high-integrity record of what the user was shown and why. Events emitted by components carry the originating IR node's context, supporting provenance and traceability.

Stream safety

Agents may produce the UI‑IR incrementally (e.g. during token streaming). The pipeline handles this with a few rules:

1. Append-only within a render. Agents can add sections but cannot reorder sections a user has already interacted with.
2. Stable heights. Skeleton placeholders commit to layout before content arrives, avoiding jank.
3. Progressive validation. Partial trees are validated as they grow; constraint-solver passes run incrementally.

Relationship to other concepts

• Intents define what the agent wants. The UI‑IR is the document that expresses it.
• Intent series map to parent–child or sibling relationships in the IR tree.
• Tokens are resolved at render time; trust tokens are checked at validation time.
• Components are the concrete output — the IR never names a component directly.
• UI-IR is is being explored as part of our Generative UI research.