MCP · AI agents

Drive your TVs from an AI agent

Point Claude — or any client that speaks the Model Context Protocol — at your sets, and it can read their state, drive the TV — press keys, switch power, replay a recorded session — and run your tests: write a scenario, run it, read the verdict. The agent acts with the rights of the person who connected it, never more, and every action is audited as that person. Ask it "which of my TVs are online?" or "open the Smart Hub on the office TV." — or "run the smoke suite on the office TV and tell me what failed."

AI CLIENT MCP SERVER CLOUD RELAY SITE · TV Claude / MCP client holds your token kac-mcp stateless façade Control relay token → you · ACL · audit Site agent → TV LAN OAuth 2.1 AS sign-in · PKCE · scopes 1 2 3 0
0you sign in once — OAuth issues the token 1the client sends your token 2the server holds none of its own — it just forwards 3the relay resolves it to you and acts with your rights

The numbered steps carry the sequence. The thing to notice is what the middle of it does not hold.

The MCP server has no credential of its own. It cannot reach a single television on its own behalf, there is nothing in it worth stealing, and it keeps no state between requests — it passes your token along and forgets it. That is deliberate: a façade that could act by itself would be a second place where access is decided, and there is exactly one.

That one is the relay, and it resolves your token to your account with the role you hold right now — not the one you had when the token was issued. Demote someone and their agent's reach shrinks on the very next request; remove them from a team and the sets go with it. Every command the agent sends is checked against the same rules as the web app and written into the same audit log under your name, so an agent can never quietly become a way to do what the person driving it could not do by hand.

What the agent can do

Read tools

list_sets · get_health · list_recordings · get_stats — plain reads, ACL-filtered to exactly the sets you can see. "Which TVs are online, and what did they record today?"

send_keys

Press a sequence of remote keys, in order — "HOME RIGHT OK". The agent takes control of a free TV automatically; it can't wrest it from someone who's using it.

send_text

Type straight into the TV's focused input — a search box, a login form — instead of arrow-keying an on-screen keyboard. Replaces the field's content; an empty send clears it.

launch_app + list_apps

Ask the TV what's installed (list_apps — titles, ids, versions), then start any of it by id — get the app under test on screen in one step and drive it with send_keys and send_text. Pass a payload and the app opens as a deep link: it reads the string from its app control (PAYLOAD) and can land on the screen you name.

install_package + list_packages

Hand over a .wgt build — uploaded with create_upload and curl, fetched from a public link, or picked from the package library — and it is signed and installed on the TV, by default with the debug client the tools below need. A failure names the stage and quotes the TV. delete_package clears the builds you no longer need; the app on the TV stays.

create_upload

A one-time link for a .wgt on your machine: upload it with curl -T, then install_package { uploadId }. A build on a public https link needs no upload — pass it as url.

get_console + eval_js + capture_screen

Read the running app’s console from its first line, evaluate JavaScript inside it, and see its own render — the DevTools tab, for an agent. Needs the set held, like the control tools.

list_projects + list_scenarios + get_scenario + list_runs

Browse the test library by project — every project has an address like team/project — read a scenario's full document, and the run history with each verdict and why it failed. When you can see more than one project, a tool that needs one refuses rather than guesses: a run on the wrong client's TV is not something to take back.

put_scenario

Create or edit a scenario as a JSON document — press, wait, text, launch, screenshot, check, repeat, if. Pass the etag from get_scenario back as base, and a save over somebody else's change is refused instead of overwriting it. Over OAuth it needs the write scenarios grant.

run_scenario

Run a scenario on a TV and wait for the verdict: PASS, or FAIL with what each check expected and what the screen actually said. The run lands in the same history as a run from the web, and the camera records it.

run_suite + get_suite_run

Run every scenario that wears a label — smoke, regression or your own — on one TV and get one answer: "11 of 12 passed". A member that could not run makes the suite skipped, never green. A long suite answers with a batch id to ask about later.

power

Switch the TV's mains via its smart plug — on, off, or cycle (the last-resort remote restart). Waits for the plug's confirmation.

play_scenario

Replay a recording's exact key-track onto a TV — reproduce a saved session on demand, on the same set or a different one.

start_recording + stop_recording

Record the camera while the agent works by hand. The recording ends on stop, after a few minutes without control, or after two hours at the latest. A run_scenario run is recorded by itself, however short. A TV with recording switched off is never recorded.

Install your own app from Claude Code

Claude Code runs on your machine, next to your build, and has a terminal. So the package never goes through the conversation: the agent asks Screenwhere for a one-time upload address, sends the file there with curl, and installs what arrived.

Connect once. In a terminal:

claude mcp add --transport http screenwhere https://app.screenwhere.com/mcp

then type /mcp in Claude Code, pick screenwhere → Authenticate, sign in with your Screenwhere account and grant Read and control (installing is a change, so a read-only connection cannot do it). Prefer a token? Create one in Settings → Access tokens and add --header "Authorization: Bearer <token>" to the command above.

Then just ask — "install ./build/MyApp.wgt on the office TV and show me its console". Behind that sentence the agent does three steps:

  1. create_upload with the file name and the file's SHA-256 (shasum -a 256). It answers an uploadId and a ready command. The address works once, for 15 minutes, only for your account and project, only for the bytes with that hash, and needs no token of its own — so the address showing up in a chat transcript gives nobody else a way in.
  2. It runs that command with the real path:
    curl --fail-with-body -T "./build/MyApp.wgt" "https://app.screenwhere.com/app/upload/upt_…"
    The server checks the file on arrival — a ZIP with config.xml naming the app, at most 25 MB, the declared SHA-256 and the size if the agent gave one — and answers OK: …, or REFUSED (reason): … with what to do next. The build lands in the package library like a web upload. (On Windows the command is curl.exe.)
  3. install_package with the setId and that uploadId. The box signs it with the set's certificates and installs it with the debug client, so get_console, eval_js and capture_screen work straight away.

Iterating? Every upload is a new build in the library, and they count against your storage. Ask the agent to tidy up as it goes — delete_package removes a build it no longer needs. The app already installed on the TV is not touched, and you may delete only what the set's Install tab would let you delete.

No terminal? An agent in the cloud can give install_package a url instead — a public https:// link to the .wgt, such as a CI artefact or a release asset. Screenwhere downloads it (redirects followed, 25 MB, 60 s, optionally checked against a SHA-256). Addresses inside a private network — localhost, 10.x, 192.168.x and the like — are refused, so that door can never be turned towards our network or yours.

Two ways to connect — same identity

Whichever you choose, the agent ends up holding a token that maps to your local account with your live role. There is no separate identity and no third-party login — Screenwhere is its own authority.

Personal access token

In Settings → Access tokens, create a token (shown once), name it, and paste it into your MCP client. Long-lived, revoke-only. Simplest to set up.

OAuth 2.1 click-to-connect

Add /mcp as a connector; the client opens our sign-in, you log in with your existing account (password + 2FA) and pick what it may do. Short-lived token, auto-refreshed. Nothing to copy-paste.

You choose what it may do

On the OAuth consent screen you pick one of three: Read only, Read and control, or Read, control and write scenarios. A read-only connection can look — sets, health, recordings, stats, scenarios and runs — but can't change anything or drive a TV. Read and control can drive a TV, install and run tests, but cannot change a scenario; only the third grant may overwrite, delete or move your scenarios. The relay enforces all of it server-side, so a token is powerless beyond its grant even if a client tried. A personal access token acts with your full rights. You can see every connected app, and its scope, in Settings → Connected apps, and disconnect any of them in one click — "sign out everywhere" drops them too.

Why it's safe

  • No new identity axis. A token resolves to a user with the role read live from the account — it can't out-rank or outlive you, and a role change or disable takes effect at once.
  • The server holds nothing. The MCP server has no standing credential and no state; it only forwards your token. The relay decides everything.
  • OAuth done right. PKCE is mandatory, redirect URIs are exact-matched, authorization codes are single-use and short-lived, and the sign-in is the same 2FA-gated, throttled check as the web login.
  • Least privilege + revocation. Read-only is a real, enforced scope, and writing scenarios is a separate grant; refresh tokens rotate on every use; and a token can never mint or revoke tokens — only a signed-in web session can.
  • Everything is audited as the real user — every key press, power switch, replay, run and scenario save.

The full design, the token model and the tests are in docs/MCP-PLAN.md and mcp/README.md. See also the architecture and what shipped.