termdesk

A computer inside your terminal.

termdesk draws a real desktop in your kitty, Ghostty or WezTerm pane. You drive it with your own mouse and keyboard. Your coding agent can drive it too, and you see every click.

$ curl -fsSL https://tdesk-sh.vercel.app/install | bash

macOS and Linux · about 35 MB · Python 3.9+ · Docker only for sandboxes

Watch an agent launch it

An agent opens termdesk in a split and uses the desktop on its own. It opens a terminal, starts a browser and reads this page.

Unedited run of scripts/record-demo · agent on the left, computer on the right

What it does

$ termdesk mac.local:5900

Shows your Mac or Linux box in the pane you're already in.

$ termdesk action state

Hands your agent the screen as numbered buttons and fields. It clicks by number.

$ termdesk sandbox up

Starts a throwaway desktop in Docker. Wreck it, then delete it.

Try it in two minutes

$ termdesk sandbox up --name work     # prints localhost:<port>
$ termdesk work                       # in a kitty pane: the desktop appears
$ termdesk action state               # another shell: UI as indexed elements
$ termdesk action set-value 11 https://example.org
$ termdesk action key Return

How it works

Frames: The desktop streams over VNC. termdesk redraws only the pixels that change. A full 1280x800 frame is about 60 KB, a blinking cursor about 80 bytes.
Input: Your keys and mouse go back as VNC events. Modifiers, key repeat and releases all work.
Semantics: In a sandbox, your agent reads each button's role, name and position from the accessibility tree.
Control: An AGENT ACTING badge shows while your agent drives. Grab the mouse and you're back in charge.

Prior art, credited

desktui drew a VNC desktop with kitty graphics first. sshdesk added an agent CLI over ssh. terminal-browser did it for a browser and got this started.