Repository avatar
Version Control
v1.0.0
active

tui-use

io.github.pavanmanishd/tui-use

Drive interactive terminal programs (vim, git rebase -i, ssh, REPLs, htop) from an agent.

Documentation

tui-use

Let any AI agent drive interactive terminal programs: vim, git rebase -i, npm init, ssh, REPLs, debuggers, htop, installers and database shells.

Coding agents run shell commands one at a time, without a terminal. A command that waits for input hangs until the tool times out, a full-screen program prints escape codes, and a REPL forgets everything between turns. tui-use gives the agent a real terminal that it can read and type into across as many turns as it needs.

claude mcp add tui-use -- npx -y @pavanmanishd/tui-use@1.0.0 mcp

Claude Code squashing three commits with git rebase -i through tui-use

Claude Code (Opus) squashing three commits with git rebase -i through tui-use. It rewrites the todo list with :2,3s/^pick/squash/, replaces the combined commit message and saves. This is the terminal the agent drove, as tui-use recorded it.

What the agent sees

Every call that changes something returns the screen as numbered text. The cursor is shown as [x], and highlighted text, such as the selected item in a menu, is wrapped in «…».

session s1 · git rebase -i HEAD~3 · 120x40 · cursor 2:1 · alt · running · idle 310ms
───
 1│pick 3f2a1c0 Add parser
 2│[s]quash 9b4e7d2 Fix parser edge case
 3│squash 1d8c5aa Rename parser module
 4│
 5│# Rebase 7c0e2b1..1d8c5aa onto 7c0e2b1 (3 commands)
 …
40│"~/repo/.git/rebase-merge/git-rebase-todo" 27L, 1189C

After the first screen, tui_send returns only the rows that changed:

session s1 · python3 · 120x40 · cursor 6:5 · normal · running · idle 312ms
───
 4│>>> sum(x) * 2
 5│42
 6│>>> [ ]
(3 of 40 rows changed)

Install

tui-use needs Node.js 22 or later on macOS or Linux. On Windows, use WSL. Prebuilt terminal binaries ship for x64 and arm64, so nothing is compiled during install.

Claude Code

claude mcp add tui-use -- npx -y @pavanmanishd/tui-use@1.0.0 mcp

The plugin adds the same server plus a skill that tells Claude when to use tui-use instead of the shell:

/plugin install tui-use --marketplace pavanmanishd/tui-use

Codex

~/.codex/config.toml:

[mcp_servers.tui-use]
command = "npx"
args = ["-y", "@pavanmanishd/tui-use@1.0.0", "mcp"]

Cursor

~/.cursor/mcp.json:

{
  "mcpServers": {
    "tui-use": { "command": "npx", "args": ["-y", "@pavanmanishd/tui-use@1.0.0", "mcp"] }
  }
}

Gemini CLI

~/.gemini/settings.json:

{
  "mcpServers": {
    "tui-use": { "command": "npx", "args": ["-y", "@pavanmanishd/tui-use@1.0.0", "mcp"] }
  }
}

OpenCode

opencode.json:

{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "tui-use": {
      "type": "local",
      "command": ["npx", "-y", "@pavanmanishd/tui-use@1.0.0", "mcp"],
      "enabled": true
    }
  }
}

Claude Desktop

claude_desktop_config.json:

{
  "mcpServers": {
    "tui-use": { "command": "npx", "args": ["-y", "@pavanmanishd/tui-use@1.0.0", "mcp"] }
  }
}

Any agent with a shell

npm install -g @pavanmanishd/tui-use@1.0.0
tui-use start "npm init" --name init
tui-use send init --text "my-app" --key enter

MCP tools

ToolWhat it does
tui_startStarts a program and returns its first screen. Takes command, cwd, env, cols, rows, name and wait.
tui_sendTypes text, a secret_env value and keys (or an ordered input list), waits, and returns the new screen as a diff by default. paste: true sends the text as a bracketed paste.
tui_snapshotReturns the screen without sending input. styles: "none" drops the «…» marks.
tui_waitWaits without sending input, for builds and installs.
tui_scrollbackReads output that scrolled off the screen, by line range (such as [-50, -1]) or by search.
tui_manageLists sessions, kills one (SIGTERM, then SIGKILL after 2 s), resizes one, or returns the path of its recording.

Saving and quitting vim in one call:

{ "session": "s1", "input": [{ "key": "esc" }, { "text": ":wq" }, { "key": "enter" }] }

Key names are enter, tab, esc, backspace, delete, up, down, left, right, home, end, pageup, pagedown, space, f1 to f12 and any single character, optionally prefixed with ctrl+, alt+ or shift+ (for example ctrl+c or shift+tab). An unknown key name is an error, never typed as text.

Waiting

By default, a call returns once the program has printed something and then stayed quiet for 300 ms with an unchanged screen, or after 5 seconds. If the program is still drawing, such as a spinner or a slow install, the response ends with a note saying so. For slow programs, pass a wait object:

FieldReturns when
textthis substring is on screen
regexthis pattern matches the screen
gonethis substring has left the screen
idle_msthere has been no output for this long
exitthe program exits
timeout_msthe time limit is reached (default 5000, max 600000)

Conditions combine with OR, and the response names the one that fired.

CLI

The CLI offers the same operations to agents that only have a shell. Sessions live in a background daemon, so they survive between shell calls.

tui-use start "npm init" --name init
tui-use send init --text "my-app" --key enter
tui-use send init --input esc ":wq" enter
tui-use snapshot init --diff
tui-use wait init --text "Is this OK?" --timeout 10000
tui-use scrollback init --search error
tui-use list
tui-use kill init
tui-use attach init
tui-use replay ~/.local/share/tui-use/recordings/20261121-101500-s1.cast

--json prints the raw response. Exit codes are 0 for success, 1 for a request error, 2 for a usage error and 3 when a wait times out. Quote commands that have their own flags, as in tui-use start "git rebase -i HEAD~3".

Watching or taking over

tui-use attach <session> mirrors a session in your terminal while the agent works. Anything you type goes to the program as well, and the agent's screen header shows human attached (rw) while you are attached. Press ctrl+] to detach; the session keeps running.

Configuration

~/.config/tui-use/config.json, with every key optional:

{
  "maxSessions": 8,
  "idleTimeoutMinutes": 30,
  "scrollbackLines": 10000,
  "recordings": true,
  "recordingRetentionDays": 7,
  "deny": ["^rm -rf /", "^sudo "]
}

deny is a list of regular expressions. A command that matches one is refused before anything runs. An unknown key stops the daemon from starting, so a typo can't be silently ignored.

Security

  • tui-use runs commands as you, with the environment of the agent that asked. It adds no privilege that the agent's shell tool didn't already have, and it is not a sandbox.
  • One daemon per user listens on a unix socket in a directory only you can read. Clients refuse to connect if that directory is a symlink, belongs to another user, or is readable by other users.
  • secret_env names an environment variable of the tui-use process, for example one set in your MCP config. Its value is typed into the program and replaced with *** wherever it would appear in a screen, scrollback, search result or recording. List such variables in TUI_USE_SECRETS (for example TUI_USE_SECRETS=SSH_PASS,DB_PASS) so they are not copied into the environment of the programs tui-use starts.
  • Sessions an MCP client starts are killed when that client disconnects. Other MCP clients can read them but cannot type into them.
  • Every session is recorded as an asciinema v2 file under ~/.local/share/tui-use/recordings. Recordings never contain input and are deleted after 7 days.

Known limitations

  • Highlight marks compare each cell's background with the most common background on screen. When two buttons have different colours, as in gum confirm, both are marked and the snapshot doesn't show which one is active. Use the program's shortcut key, such as y or n, instead.
  • Settle detection can't tell a spinner from real progress. For slow programs, pass a wait with text, regex or gone.
  • Redaction matches the exact secret. A program that prints it in altered form, such as masked or split by escape codes, is not caught.
  • There is no mouse input yet, and no native Windows support (WSL works).

TUI-Bench

TUI-Bench is a set of 40 interactive tasks covering editors, git, prompts, REPLs, full-screen apps, ssh and pickers. Each task runs in a Docker container and is scored by a scripted checker. The runner drives real agent CLIs (Claude Code, Gemini CLI and OpenCode) twice per task, once with only the shell tool and once with tui-use added, and reports success rate, turns, tokens and wall time. See bench/README.md to reproduce it.

Results

Claude Code 2.1.329 on all 40 tasks, with 3 repeats per cell (480 runs):

ModelShell onlyWith tui-useMedian turns (shell / tui-use)Median tokens (shell / tui-use)Median wall time (shell / tui-use)
Claude Opus 5.599%100%4 / 884k / 151k17 s / 20 s
Claude Haiku 5.599%100%4 / 590k / 114k15 s / 14 s
  • With only a shell, Claude Code still finished almost every task. When a program needed a terminal (htop, ssh password prompts, whiptail, vimdiff), it usually wrote its own Python pty.fork() driver on the spot. Both failures were htop-kill runs without tui-use.
  • With tui-use available, the agent chose it in 129 of 240 runs (Opus in 80 of 120, Haiku in 49 of 120), mostly for editors, prompts, pickers, full-screen apps and ssh. Those runs took more turns and tokens than the shell runs, because the agent reads the screen between steps instead of scripting blind.
  • For a frontier model with Python available, tui-use doesn't decide whether an interactive task gets done. It replaces improvised pty scripts with one tool the agent can see and type into, and the agent uses it when it is there.

OpenCode with DeepSeek V4.1 Flash, run through T3 Code on six interaction-heavy tasks with one repeat, passed all 12 runs in both conditions (details). Gemini CLI and the open-weight models in bench/runner/matrix.json have not been run yet; they need GEMINI_API_KEY and OPENROUTER_API_KEY.

Two caveats apply. The Claude runs used a Claude subscription token, so Claude Code inside the containers also loaded that account's organization policy, equally in both conditions. And a single repeat for the OpenCode batch is too few to separate small differences. The raw results are in bench/results/claude-full-2026-11-22.

Compared with other tools

Based on each project's README in November 2026.

tui-useterminal-use (tu)mcp-interactive-terminalpexpect-mcptmux MCP servers
MCP serveryesno, CLI onlyyesyesyes
CLI for agents with only a shellyesyesnonono
Terminal emulatorxterm.jsalacrittyxterm.jsnone (pexpect)tmux
Input, wait and new screen in one callyesno, separate commandsyesnot documentedvaries
Diff of changed rowsyesnot documentednot documentednonot documented
Selected items marked in textyesnot documentednot documentednonot documented
Secrets kept out of outputyesnot documentednot documentednot documentednot documented
Human can watch liveyes, and typeyes, watch onlynot documentednot documentedyes (tmux attach)
Mouse inputnot yetyesnot documentednot documentednot documented
Recordingsasciinemanot documentednot documentednot documentednot documented
BenchmarkTUI-Benchnononono

Development

npm ci
npm run lint && npm run typecheck
npm test                  # unit tests
npm run test:integration  # real programs in real terminals; the ssh test needs Docker

License

Apache-2.0. See LICENSE and NOTICE.

NPM
@pavanmanishd/tui-use
Install Command
npm install @pavanmanishd/tui-use