
shibbo
io.github.xvr2e7/shibbo
Check prose, pages and code for AI tells, with per-model rules and a fix for each hit.
Documentation
shibbo
Track what gives AI work away, and steer your agent toward clear work.
A registry of AI tells in four layers (prose, interface, code and behavior), measured model by model, with a harness that steers your agent away from them.
Quick start · What it checks · The atlas · Docs · Contributing
People have learned to spot AI work: a stock phrase in an essay, say, or a purple gradient on a landing page. Once they spot one, they trust the rest less.
These tells shift with each release. In 2024, "delve" gave away GPT-4; today most models use it no more than people do, and some never do. Newer models have their own habits, such as Claude Opus 5's "genuinely useful", GPT-6 Astra's "rather than treating" and Claude Opus 5.5's "this matters because", which people writing on the same topics never used. Pages drift too: Claude dropped the purple gradient, and its 2026 releases reach for a cream page and one italic serif word in the headline. A list of tells from last year misses them.
Shibbo tracks each tell over time and measures which models show it, in four layers: prose, interface, code and behavior (how an agent works and how it reports). Set it up in your agent and the tells get caught while it works, or run it on your files to see which ones show up.
A stock phrase gives text away, but a draft with no tells can still bury its point. So the rules shibbo hands an agent open with clear writing: name who does what, use the plain word, put the point first.
Shibbo gathers the tells that earlier studies, lists and checkers found into one registry, and credits each source. See Prior art.
The name comes from shibboleth: a small sign, like a word said wrong, that gives away where someone comes from.
Quick start
Set up your agent's harness:
npx shibbo init <harness>
This writes shibbo's rules into the harness's instructions file and registers its MCP tools. The rules cover clear writing, the tells of the model you name, and agent habits such as claiming that tests pass without running them. It works with Claude Code, Codex, Cursor, Copilot, Gemini CLI, Devin, Cline, Roo, Aider, Kiro, Junie, Amp, opencode, Goose, Zed, Continue and Warp. In Claude Code, Codex, Gemini CLI, Cline and Aider, the agent also hears about tells after each edit; Claude Code and Gemini CLI check its final reply too. Claude Code sessions also load rules for interface work. Cursor, Copilot, Devin and Kiro have experimental hooks; the support table has the details.
Claude Code users can install the plugin, which bundles the checker, the MCP server and the hooks:
/plugin marketplace add xvr2e7/shibbo
/plugin install shibbo@shibbo
To add only the rules, as a skill for any agent that loads skills:
npx skills add xvr2e7/shibbo
To check files yourself:
$ npx shibbo check draft.md --model gpt-6
draft.md: 20/100 [heavy] (29 words)
1:1 high sycophantic-opener "Great question"
→ Start with the answer.
1:33 med delve "delve"
→ Say what you will do ("look at", "compare", "list").
1:44 med x-matters "why the distinction matters"
→ Show what goes wrong without it. Readers will see that it matters.
3:4 high contrastive-negation "is not a tool, it is"
→ State the real claim. Keep a contrast only when someone holds the other view.
3:36 med corrective-rather-than "Rather than treating"
→ Say what to do. Skip the straw version.
Each hit comes with a fix. --model weights the score toward the phrases that model over-uses, and --threshold 80 fails a CI run when a file scores lower.
npx shibbo doctor reports the version you run, whether npm has a newer one, and which rules, MCP and hook files the project has. Each install path updates in its own way: see Update and diagnose.
What it checks
| Files | Tells |
|---|---|
| Markdown, text | Stock phrases and "not X, it's Y" frames |
| HTML, SVG | Prose tells in the visible text; interface tells in the markup |
| CSS, Sass, Less | Purple gradients, or Inter set everywhere |
| JSX, TSX, Vue, Svelte, Astro | Interface and code tells |
| JS, TS, Python, Go, Rust, … | Comments that narrate the edit, or tests skipped to go green |
Agent tells, such as "All tests pass" without running them, happen during the work, so no file shows them. Shibbo puts the top ones in the agent's rules instead. Tells that need human judgment show up as questions: check --all and the MCP server list them. Three of them are about clear writing, which no pattern can score.
The atlas
The web app (npm run web:dev) maps how model output has shifted with each release, on a timeline that cites every date back to 1966. It sorts tells into four layers, the surfaces where people meet a model's work: prose, code, interfaces and agent behavior.
The prose atlas places thousands of phrases by which models over-use them, against people writing on the same topics. For the interface atlas, every model builds the same 24 one-line briefs, from landing pages to dashboards; shibbo renders each page, records its design choices in type, colour, shape, layout and motion, and maps those choices against the models, with a trail through each maker's releases: 707 pages from 30 models by thirteen makers so far. Click a page on the map to open it live, at desktop or phone width. The code layer maps 26 models' replies to the same 24 coding requests. For the behavior layer, 23 models each read the same 12 finished work logs and write the final message to the user; the map places those 822 reports by how they are built and by whether they tell the user what the log shows. A stacked view shows the four layers at once and traces one model through them. The Tells page lists every tell by layer and by the year it was first seen, with interface tells shown as small mockups.
Tells. Every tell by layer and by the year it was first seen. An interface tell appears as a small mockup. |
Arena. Humans and models write on the same topics, and the article with fewer tells wins each round. Humans lead, as expected, since a tell is something models write more often than people. |
Prior art
Many people have catalogued AI tells before, each for one kind of work: prose, interfaces or code. Shibbo aims to be one source of truth across all of them. It brings their findings together, checks them against measured data where that exists, and keeps a citation and a date on every entry. The registry cites more than a hundred sources, including:
| Kind | Examples |
|---|---|
| Measured studies | Graphite's AI Tells, Kobak et al. on "delves" in PubMed, Why does ChatGPT "delve" so much?, Antislop |
| Community lists | Wikipedia's Signs of AI writing, AI writing tropes |
| Checkers and skills | avoid-ai-writing, Slop Guard, slop-score, deslop, avoid-ai-design, no-slop-ui, Hallmark |
| Agent reports | Bug reports from the Claude Code, Codex and Gemini CLI issue trackers, and published agent system prompts |
Sources lists every one.
Other catalogs cover parts of this ground, among them aismells for prose, ai-design-tells for interfaces, and the anti-slop skill across text, code and design. As far as we know, none of them pairs prose, code, interface and agent tells with per-model measurements and a source on every entry. If you know one that does, or a source we missed, please open an issue.
The data
| Tell | Earlier models | Newest models |
|---|---|---|
| "delves" | PubMed abstracts, 2024: 28× the rate expected from earlier years (Kobak et al.) | Web articles on the same topics: people 6 uses; GPT-5.6, GPT-6, Opus 4.6, Opus 5 and Opus 5.5 at most 2 each; Gemini 3.1 Pro 18 |
| Em dashes | GPT-4.1 2.3× the human rate, GPT-5 2.2× | GPT-6 Astra 0.12×, Claude Opus 5 0.93×, Claude Opus 5.5 and Gemini 3.1 almost none |
| "not a _, it is" | Claude Opus 5: 442 uses, people 1 | |
| "that distinction matters" | GPT-6 Astra: 561 uses, people 0 | |
| "it's" | people: 14,524 uses, Claude Opus 4.6: 7 | |
| "this matters because" | Claude Opus 5.5: 210 uses, people 0 |
Models and people still share most of their words. On the same topics, 70 to 80% of each model's word use matches people's, and 52 to 66% of its two-word phrases. The atlas draws only phrases one side uses at least 4× as often, in 150 or more articles.
The PubMed figure compares 2024 abstracts with the rate earlier years predict, with no human baseline. The rest compare models with people writing web articles on the same topics. Graphite asked for plain text, and em dash rates shift with the prompt. Graphite's second dash count puts Claude Opus 5 at 3.5×.
Shibbo measures each model against people writing on the same topics. The data comes from Graphite's AI Tells study and its Opus 5.5 update for ten 2025–26 models, and from shibbo's own runs on RAID for 2022–23 instruct and chat models. For newer releases, shibbo runs the models itself through maker APIs and OpenRouter, on about 290 of the same held-out topics: so far eleven: Claude Fable 5.2, Haiku 5.5 and Sonnet 5.5, Gemini 4 Argon and Hydrogen, GPT-6.1 Sol and Terra, DeepSeek-V4.1-Flash, MiMo-V2.6 Pro, MiniMax-M3.1 Flash and Nemotron 3.5 Super. The same runs fill 2024 with seven models that are still served: GPT-4o and GPT-4o mini, Gemma 2 27B, Llama 3.1 70B and 3.3 70B, Qwen 2.5 72B and DeepSeek-V3. The Claude and GPT runs write "practical" 69 to 231 times where people wrote it 4 times; Gemini 4 Argon writes "furthermore" 162 times to their 5. These runs and the interface pages live in measure/.
A daily job watches public model lists and changelogs for new releases. Reported but unconfirmed models sit in their own file, each with a source, a date and an expiry, outside the timeline. It opens a pull request; a person checks each date on the maker's page before it goes in.
To test the checker, shibbo scores a person's article and a model's article on the same topic, across topics it never trained on, and counts how often the model's scores worse. The curated rules were written after reading all of Graphite's data, so the results run somewhat high. To test the harness, eval/ui/ has the same model build the same briefs with and without it, for people to rate blind. The data docs have the latest results and their limits.
Contributing
Each tell is a YAML entry in data/tells.yaml or data/tells.d/, with a fix and at least one source. The guides cover adding a tell and measuring a new model. The maintainer pays for the model runs from personal API accounts, so a run with your own key is a real help; see who pays for the runs.
npm install
npm test && npm run validate
npm run web:dev
Credits
Most prose measurements come from Graphite's "AI Tells" and its Opus 5.5 update (2026, CC BY 4.0); its article texts belong to their publishers and stay out of the repo. The PubMed figures come from Kobak et al. (2025). Every other source is listed in data/sources.yaml. The code is MIT-licensed, and the data keeps each source's license.
shibbonpm install shibbo