woodstitch-mcp

An MCP server for the Woodstitch API: paid human listeners judge your text-to-speech audio in blinded studies, and your agent gets a number back with a confidence interval.

Two tasks, and only two: pairwise preference (a listener hears two clips of the same text and picks one, forced choice) and human-vs-AI detection (one clip, human or AI, confidence 1–4). Not offered: MOS, CMOS, MUSHRA or any 1–5 scale.

Install

Get an API key at https://woodstitch.com/app/api-keys, then:

Claude Code

claude mcp add woodstitch -e WOODSTITCH_API_KEY=ws_live_... -- uvx woodstitch-mcp

Claude Desktop / Cursor / any MCP client (mcpServers entry):

{
  "woodstitch": {
    "command": "uvx",
    "args": ["woodstitch-mcp"],
    "env": {"WOODSTITCH_API_KEY": "ws_live_..."}
  }
}

pipx run woodstitch-mcp or pip install woodstitch-mcp && woodstitch-mcp work too. Requires Python 3.10+.

Claude Code users: the Woodstitch plugin bundles this server with a skill that knows when and how to run a listening test: /plugin marketplace add woodstitch-llc/claude-plugin then /plugin install tts-human-eval@woodstitch.

Tools

Tool	What it does	Money
`woodstitch_templates`	What is measured and what is not (no key needed)	–
`woodstitch_pricing`	Rate card and worked examples (no key needed)	–
`woodstitch_account`	Balance and spend	–
`woodstitch_upload_files`	Upload local audio, get file ids	–
`woodstitch_create_eval`	Create and price a study; `dry_run` to price only	charges nothing
`woodstitch_start_eval`	Start the study	spends money
`woodstitch_get_eval`	Progress and provisional score	–
`woodstitch_get_results`	Final numbers	–
`woodstitch_export`	Every judgement as rows (json or csv, optionally to a file)	–
`woodstitch_cancel_eval`	Stop early, pay only for work done	–
`woodstitch_list_evals`, `woodstitch_list_files`, `woodstitch_delete_file`	housekeeping	–

Safeguards built in

woodstitch_start_eval requires user_confirmed_amount_usd, the quoted total the user agreed to, and sends it to the API as the hard ceiling max_total_usd. If the quote is higher than what the user confirmed, the API refuses (over_max) and nothing starts. An agent cannot start a study the user has not seen the price of.
Every POST carries an Idempotency-Key, so a retried call never creates or starts a study twice.
The server's instructions tell the model that interim numbers are provisional and that stopping early on a threshold is optional stopping.

Typical session

woodstitch_upload_files(["a/001.wav", "b/001.wav", ...]) or use public URLs.
woodstitch_create_eval(pairs=[{"id":"001","a":"file_…","b":"file_…","passage":"001","system_a":"base","system_b":"candidate"}, …], ratings_per_pair=5) → quote, nothing charged.
Show the user the total. On yes: woodstitch_start_eval(eval_id, user_confirmed_amount_usd=<total>).
woodstitch_get_eval(eval_id) now and then; first judgements usually land within minutes.
woodstitch_get_results(eval_id) when status is completed; woodstitch_export for the rows.

Method, listener pay and controls: https://woodstitch.com/research/methodology. Machine-readable API spec: https://woodstitch.com/openapi.json.

Development

uv venv && . .venv/bin/activate && uv pip install -e .
WOODSTITCH_API_KEY=ws_live_... woodstitch-mcp        # stdio server

tts-human-eval

Documentation