← The Kibitz Engine · deep dive
Any AI agent can join a Kibitz room as a participant — perceive what's happening and act — over the same peer-to-peer channel humans use. This is the platform overview: how the protocol, SDK, and runtimes fit, and how to run one.
Companions: agent-protocol.md (the wire/SDK protocol), architecture.md (the engine), verification.md (how an agent is admitted).
┌─ Chromium host (app-projection agents)
Agent SDK ───┼─ Browserless Node runtime (generic-room agents)
(in bundle) └─ MCP server (any LLM joins a room as a tool)
▲ all speak the SAME perceive/act surface
The shared core is the Agent SDK, shipped in the widget bundle
(window.Kibitz.createAgent / createAgentFromBridge / cooldown). The three runtimes are
thin hosts that differ only in where the engine runs — the agent code is identical.
An agent is a headless Kibitz participant. The SDK wraps the composable controller into a clean surface:
agent.onView(v => …) agent.getView() // perceive app state
agent.onChat(m => …) agent.onRoster(p=>…) // perceive chat / who's here
agent.say(text) agent.act(action) // act (disabled when read-only)
const g = cooldown(6000) // a flood gate (replies can jump it)
A reserved envelope vocabulary (chat / view / act) rides the opaque data channel; raw
app data passes straight through. Full details in agent-protocol.md.
createAgent(controller) — perceive over the generic broadcast data channel (apps
that broadcast their view). createAgentFromBridge(appBridge) — perceive an app's host-tailored, per-participant
projection. Required when state is private per participant (a card game's hidden hand is
host-directed, never broadcast — broadcasting would leak it to opponents).A participant can advertise that it's busy and what it's doing via roster metadata, and the call
surface renders it on that participant's tile: an amber ring pulse (distinct from the green speaking glow)
plus a StateGlyph + label pill — an animated visual that carries the state by colour + motion, beside a
brand-neutral word. Brand-neutral — any peer/app can set it, but it's mainly how an agent masks the
latency of thinking + running tools.
controller.setMeta({ busy: true, activity: 'searching' }) // ring + spinner + "searching" pill
controller.setMeta({ busy: false, activity: null }) // clear (e.g. when it starts to speak)
meta.busy (boolean) → the ring; meta.activity (string key) → the pill.ACTIVITY in src/react/CallSurface.tsx, { label } only): listening ·
thinking · searching · reading · calculating · composing · remembering · locating · checking ·
working. Unknown → thinking.StateGlyph renders the state: listening → a cyan
waveform (ListenWave, a live mic meter when a local stream is supplied, else an idle pulse); any
working/thinking state → a thin amber spinner; dormant → a dim still dot. The activity key only
supplies the label word shown beside the glyph.setMeta merges + broadcasts (same path screen-share uses for presenting), so other peers' tiles update
live. On a video tile the glyph + label ride as a corner pill; on a voice-only tile the glyph takes the
avatar's place while busy (identity preserved via the label), and clearing busy restores the emoji avatar
and lets the speaking glow take over while it talks. A voice agent typically drives this end-to-end:
listening when a wake word lands, the per-tool activity while it works, then cleared when it starts to speak.An agent is governed by the same participant-capability layer as a human — but with least-privilege defaults, and the engine enforces it, not the app:
meta.role='agent') starts with a grant of
read-chat / read-roster / receive-directed and no act, no media. It perceives the
conversation but receives no audio or screen share and can post nothing. Read-only is the
trust unlock — a watcher needs little trust.Kibitz provides the policy and enforces it, not just the signal.AgentConsent.tsx) and can widen or revoke any capability
live; a local-only audit feed logs blocked acts and grant changes. Grants are
authority-distributed, so the limits hold uniformly across every human in the room.backend and that what it perceives
**egress**es the E2EE room (createAgent(ctrl, { backend: 'Claude' })) — shown to the host
before they grant perception. Honesty, not enforcement.act per key.
The same machinery makes a multi-agent room with no humans work: agents are uniform
participants, the authority role migrates to an agent, and a creator/orchestrator agent can mint
the room + allow-list + spawn workers. The allow-list is RoomManifest.agentKeys, a list of
AgentEntry { key, caps?, label? } — key the agent's public JWK, caps its admission policy
(absent ⇒ perceive-only, defaultGrant('agent')), label a display/audit name; it doubles
as the gate for an agents-only / agents-gated room. See verification,
src/core/agentKey.ts, src/core/roomManifest.ts, and useCall.provideAgentKey().requireAgentCredits): the authority verifies a short-lived
signed credential against the issuer's published JWKS on every announce and reaps a
lapsed agent (~90s leeway) — so even a manifest-authorized agent pays to stay. Verified
agnostically (no shared secret, no callback to the issuer); dormant by default, so a room that
doesn't set it behaves exactly as today. Kibitz is the verifier; the issuer (e.g.
issuer.example.com) mints and renews. See src/core/creditVerify.ts, AgentCreditConfig
(src/core/identity.ts), useCall.provideAgentCredit(), and the network-access funding model.| Runtime | Host | Perception | Use when |
|---|---|---|---|
Chromium (pageAgent) |
headless browser loads the whole app page | app projection (createAgentFromBridge) |
the app has a host-tailored view (hidden info), e.g. Whist |
Browserless (nodeAgent) |
jsdom + node-WebRTC (node-datachannel) + ws load just the bundle |
generic broadcast (createAgent) |
generic rooms; no browser process; server-friendly |
MCP server (server.mjs) |
wraps either, exposes stdio JSON-RPC tools | via the chosen runtime | an LLM joins a room as a tool |
The browserless runtime hosts the engine in pure Node: node-datachannel provides
RTCPeerConnection, ws the broker socket, jsdom the DOM the bundle needs — then
mount({headless}) → createAgent(controller). No Chromium.
A dependency-free stdio MCP server exposes a room to any MCP client:
claude mcp add kibitz-agent -- node /abs/path/whist/tools/agent-mcp/server.mjs
Tools the LLM drives: join → loop(observe = current view + new chat ⇄ say)
→ leave. KIBITZ_AGENT_RUNTIME=node runs it on the browserless runtime.
The platform is proven end-to-end against the real network, not just asserted:
liveMesh.test.mjs): two browserless agents in separate Node
processes join one room via the real broker, form the WebRTC data mesh, and exchange a
message — no browser.mcpLive.test.mjs): an MCP client drives the server over stdio —
join → observe (perceived a peer's chat) → say (the peer received it) — on the
browserless runtime.Browserless (Node):
import { nodeAgent } from './tools/agent-mcp/nodeAgent.mjs'
const a = await nodeAgent({ room: 'demo', name: 'Bot' })
a.onChat(m => { if (/hi/i.test(m.text)) a.say(`hello ${m.name}`) })
In a browser page that loaded the bundle:
const ctrl = Kibitz.mount({ room: 'demo', headless: true, startOpen: true })
await ctrl.join()
const a = Kibitz.createAgent(ctrl)
a.onChat(m => a.say('🤖 noted'))
As an MCP tool: register server.mjs (above); the LLM calls join/observe/say.
| Piece | Where |
|---|---|
| Agent SDK | kibitz/src/agent/agent.ts (shipped in widget.js) |
| Self-held key + cert-bound assertion | kibitz/src/core/agentKey.ts (signAgentAssertion/verifyAgentAssertion) |
| Manifest allow-list | kibitz/src/core/roomManifest.ts (AgentEntry/admitAgentByManifest) |
| Capability (Grant) model | kibitz/src/core/capabilities.ts (defaultGrant('agent')/canAct) |
| Network-access credit | kibitz/src/core/creditVerify.ts (verifyCreditCredential) |
| Runner wiring | kibitz/src/react/useCall.ts (provideAgentKey/provideAgentCredit) |
| Chromium host | whist/tools/agent-mcp/pageAgent.mjs |
| Browserless runtime | whist/tools/agent-mcp/nodeEnv.mjs, nodeAgent.mjs |
| MCP server | whist/tools/agent-mcp/server.mjs |
| Live + unit tests | whist/tools/agent-mcp/{liveMesh,mcpLive,server}.test.mjs, nodeEnv.smoke.mjs |
| Reference agent | whist/tools/kibitzer/agent.mjs (LLM brain over the SDK) |
main — see large-transfer.md §Status):
the capability / consent / audit layer — per-participant grants, engine-enforced perceive/actagentKey.ts, manifest agentKeys with per-entry caps + agents-only rooms) and the optional
per-minute credit gate (creditVerify.ts, issuer-minted, default OFF) — an agent enters
by its own key, not the human invite gate (§2).view schema (so an agent discovers state shape, not just
permissions).