dev-diagnosis

Every token your agent spends reading a 1.7 MB data dump is money out of your pocket. Every flaky test is CI time your agent sits waiting. Every missing command is a turn your agent burns for nothing. dev-diagnosis finds all three — plus whether your build cache actually saves anything — in one command, offline, with no dependencies.

Try it: three steps, one fix

$ dev-diagnosis all
Files: 9  Size: 1.68 MB  Score: 75/100 — Some context noise found
Noise to keep away from an AI agent:
  data/export.json — large file (1.68 MB)
Estimated tokens wasted on this noise: ≈ 439k tokens (estimate, ~4 bytes per token)
FAIL  some context noise found (score 75/100)

$ dev-diagnosis context --fix
Created .aiignore with 1 entry: data/export.json
Re-scanned after fix: Score: 94/100 — Looks clean for an AI coding agent
PASS  context is clean

$ dev-diagnosis all
Files: 9  Size: 1.16 KB  Score: 94/100 — Looks clean for an AI coding agent
PASS  context is clean

The noise cost 439k tokens a turn; the fix is one entry in .aiignore.

When you reach for it

• Just took over a repo: dev-diagnosis all
• CI goes red, then green on retry, then red again: dev-diagnosis flakes ci-runs/
• Builds feel slow even with a cache: dev-diagnosis cache -- npm run build
• command not found in the IDE but works in the terminal: dev-diagnosis path <name>

What each check does — and does not do

Check	Needs	Finds	Cannot find
context	a directory	files your agent should not read, and what they cost in tokens	whether a large file is actually needed — it flags by size, you decide
path	a command name	which shell contexts resolve it, shadowed copies, toolchain installs	why a command that resolves still crashes when it runs
cache	a build command and a cache directory	whether the cache pays, cold vs warm	whether a different cache setup would be faster
flakes	JUnit XML from two or more runs	flaky, always-failing and passed-on-retry tests, ranked	why a test is flaky — only that it is

Install and run

npx dev-diagnosis-cli

Or install it once and use it anywhere:

npm install -g dev-diagnosis-cli
dev-diagnosis all

dev-diagnosis with no arguments is the same as dev-diagnosis all on the current directory, and dev-diagnosis <path> is the same as dev-diagnosis all <path>. Each check is also available on its own:

dev-diagnosis context [path]                        # files an AI agent should not read
dev-diagnosis path <command>                        # why a CLI command is missing
dev-diagnosis cache -- <build-command...>           # cold vs warm timing for a build cache
dev-diagnosis flakes <junit-dir>                    # flaky tests from JUnit XML across runs
dev-diagnosis all [path] --command <name> --flakes <dir> -- <build-command...>

all always runs context on [path] (default: the current directory) and runs path, flakes and cache when their input is given. Checks without their input are listed as skipped, with the command that would run them, instead of silently pretending they ran. A build command on a repo with no cache directory to measure is also listed as skipped rather than failing: nothing to measure is not a problem. --json gives machine-readable output for context, path, cache, flakes and all. In all --json, a check that could not run (a missing runs directory, a failing build) reports { "error": "<reason>" } instead of null, so a script can tell why the overall result failed. Add --fix to context (or all) to write the suggestions into <path>/.aiignore instead of only printing them: entries already there are left alone, nothing else in the file is touched, and a later scan skips everything .aiignore lists — so context --fix followed by context comes back clean.

all ends with a one-line summary: how many checks passed, failed or were skipped, how much context noise was found, and what the cache saved when it ran. In --json output the same fields are in the summary object.

Exit codes: 0 every check that ran passed; 1 a check found something to fix or could not run; 2 usage error. With --fix, the exit code follows a re-scan after the fix, so a fix that leaves the repo clean exits 0 — the command reports the repo as it stands after the fix, not as it was before.

Let your agent run it

One-command install

dev-diagnosis setup detects the agents on your machine (Claude Code, Codex CLI, Cursor, VS Code) and writes the MCP config that wires dev-diagnosis into each of them. It prints a preview of exactly what will be written before writing anything: a config that already mentions dev-diagnosis is left alone, a config that exists without it is merged (other entries kept), a missing config is created. A config it cannot parse is left alone and reported.

dev-diagnosis setup            # preview, then write
dev-diagnosis setup --dry-run  # preview only, write nothing
dev-diagnosis setup --check    # report which agents are wired (exit 0 when all are)

If dev-diagnosis itself is not on PATH, setup says so — install it globally (npm install -g dev-diagnosis-cli) or the agents will not be able to run it.

MCP server

dev-diagnosis --mcp serves the same four checks as MCP tools over stdio — no extra install, no network, the same code the CLI runs. Add it to your agent once:

Claude Code (.mcp.json in the repo, or claude mcp add dev-diagnosis -- dev-diagnosis --mcp):

{
  "mcpServers": {
    "dev-diagnosis": {
      "command": "dev-diagnosis",
      "args": ["--mcp"]
    }
  }
}

Codex (codex mcp add dev-diagnosis -- dev-diagnosis --mcp, or in ~/.codex/config.toml):

[mcp_servers.dev-diagnosis]
command = "dev-diagnosis"
args = ["--mcp"]

To keep the agent inside one repo, add --sandbox <dir> (alias --root): paths outside it are rejected with a plain error. Five tools: diagnose_context, diagnose_path, diagnose_cache, diagnose_flakes, diagnose_all — each returns the same JSON the CLI's --json produces.

Agent Skills

dev-diagnosis --skill-generate writes .claude/skills/dev-diagnosis/SKILL.md in the current repo (or ~/.claude/skills/dev-diagnosis/SKILL.md with --global). The skill tells an agent to run dev-diagnosis all when it starts work in a repo, fix what FAILs, and re-run to confirm. It refuses to overwrite an existing SKILL.md, so your own edits are safe.

AI agents looking for this tool: llms.txt has a short machine-readable summary.

In CI

One step runs the diagnosis and fails the job when a check fails:

jobs:
  diagnose:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - uses: WJDdidi-1/dev-doctor@main
        with:
          path: .
          args: --command node

args is passed straight to dev-diagnosis all, so --command, --flakes, --fix and a build command after -- all work the same way they do on the command line. The action runs the published npm package (@latest), so a repo using it always gets the newest checks.

Example

The repo ships a small demo project in examples/demo-repo (a build output, a log, a lockfile and a 1.7 MB data dump) and two JUnit runs in examples/runs. npm run example runs dev-diagnosis all on it. This is its real output, copied verbatim (paths and cache timings vary by machine):

$ dev-diagnosis all examples/demo-repo --command node --flakes examples/runs -- node build.mjs
dev-diagnosis — /home/hatch/workspace/dev-doctor-build/dev-doctor/examples/demo-repo

== context ==
Repo: /home/hatch/workspace/dev-doctor-build/dev-doctor/examples/demo-repo
Files: 9  Size: 1.68 MB  Score: 75/100 — Some context noise found
  build output: 98 B (0%) — lost 0 of 40
  large files: 1.68 MB (99.9%) — lost 25 of 25
  lockfiles: 95 B (0%) — lost 0 of 20
  logs: 79 B (0%) — lost 0 of 15
Noise to keep away from an AI agent:
  data/export.json — large file (1.68 MB)
Estimated tokens wasted on this noise: ≈ 439k tokens (estimate, ~4 bytes per token)
Worth a look before sharing:
  .env.example — environment template (usually safe to share)
Fix: add these to .aiignore (or the agent's ignore list):
  data/export.json
FAIL  some context noise found (score 75/100)

== path ==
current      /usr/bin/node
login        /usr/bin/node
non-login    /usr/bin/node
daemon-like  /usr/bin/node
PASS  node resolves to /usr/bin/node

== flakes ==
runs: 2  tests: 5  flaky: 1  always-failing: 1
test                                           unstable  runs    trend  classification
com.example.LoginTest::rejectsExpiredToken          2/2     2  chronic  always-failing
com.example.CheckoutTest::appliesCoupon             1/2     2      new  passed-on-retry
FAIL  1 flaky, 1 always-failing across 2 runs

== cache ==
Cache dir: /home/hatch/workspace/dev-doctor-build/dev-doctor/examples/demo-repo/.cache
Command: node build.mjs
Cold: 484 ms  Warm: 82 ms  Saved: 402 ms (83%)
Verdict: cache pays — warm run is 402 ms (83%) faster than cold
PASS  cache pays: cold 484 ms, warm 82 ms (warm run is 402 ms (83%) faster than cold)

Overall: FAIL — see the checks above
Summary: 4 checks — 2 passed, 2 failed, 0 skipped; context noise 1.68 MB; cache saved 83%

The demo repo is intentionally noisy, so all exits 1: the context and flakes checks fail while path and cache pass. That is the point of the fixture.

What each check does

context — what an AI agent should not read

Walks the repo (skipping .git, node_modules and other hidden directories) and classifies every file: build output, large files over 1 MB, lockfiles, logs, binary payloads, secret-like files and environment templates. It prints a 0–100 score, the bytes lost per category, the noisy files, the files worth a look before sharing, and the .aiignore entries that would fix it. Build-output directories are reported by their full path from the repo root (packages/foo/dist/, not a bare dist/) and rolled up into one row per directory, so the entry --fix writes matches nested build output too. A large file is fixed by its exact path from the repo root — no glob can single one file out — so --fix followed by a re-scan comes back clean even when the noise is a single oversized file. Files the repo's .aiignore already lists are excluded from the scan entirely — they are what the agent will not be handed, so they are not noise any more. Files and directories listed in .gitignore — at the repo root or in a subdirectory — are excluded the same way: they are already invisible to most agents, so flagging them would be noise about noise, and --fix will not duplicate them into .aiignore. The common pattern forms work (*, ?, [...], **, /-anchored paths, directory-only dir/, and ! negation with last-match-wins); .git/info/exclude and the global gitignore are not read. context --fix writes the suggestions into .aiignore for you; --json includes the fix outcome in the output. Review items — secret-like files and environment templates — are listed for a manual look and are never written by --fix: they cost no score points, so the entry would fix nothing, and a basename pattern such as secret.js would hide every same-named file in the repo, including test fixtures. They appear as reviewSuggestions in --json, printed as advisory.

The score is the noise-only part of ai-context-doctor's score: 100 minus the category-weighted share of noise bytes (build 40, large 25, lockfile 20, log 15). The full ai-context-doctor score also weighs context volume, one file dominating, file sprawl and binary payloads, so its composite number reads lower on the same repo. The file classification itself is the same. Only noise in a category that actually lost points is listed as noise or suggested as a fix: a 95 B lockfile in a 1.7 MB repo loses 0 points, so it is not something to fix — a 100/100 scan lists nothing to fix.

path — why a command is missing

Runs the lookup in four shell contexts — current, login, non-login and daemon-like (a minimal PATH) — and reports one row per context, plus shadowed copies (other copies of the same command that lose to the winner) and toolchain installs (npm, cargo, bun, volta, go, pip, brew). Two PATH entries that point at the same file, for example /bin/node and /usr/bin/node where /bin is a symlink to /usr/bin, count as one copy, not a shadow. The fix is copy-pasteable: the install command for the detected toolchain, or the export PATH=… line for a command that exists but is not on PATH.

cache — does the cache pay

Runs the build command twice: cold, with the cache directory moved aside, and warm, with it present, then reports both timings and the saving. The original cache is restored to its original state afterwards, and anything the cold run produced is thrown away. A cache directory that is the project root, contains it, or lies outside it is refused rather than moved. No network is involved; the check only runs your own build command. --json reports the timings and the verdict without the build's stdout, so a chatty build does not bloat the machine-readable output — in all --json the embedded cache check is the same shape.

flakes — which tests are flaky

Reads JUnit XML across CI runs — each subdirectory of <junit-dir> is one run, and each *.xml file directly under it is a run of its own — and ranks tests by instability, using a Wilson 95% lower bound so a test that failed twice in two runs outranks one that failed twice in twenty. It distinguishes passed-on-retry (failed then passed in the same run, the classic flake signature) from always-failing and plain cross-run flakes, and labels each test new, chronic or recovered from its history. CRLF line endings are normalised before parsing, so Windows-produced XML parses the same as Unix-produced XML. A file that cannot be parsed — broken XML, or a file with no root element at all (empty, whitespace-only, a lone declaration) — is printed as a warning and fails the check, so the verdict is never based on half-read data; a genuinely empty report (<testsuites></testsuites>) still counts as zero tests. --top N limits the listing to the N highest-ranked flaky tests, in the text table and in --json output alike; the totals still describe the whole run set. A run directory that contains no JUnit XML files at all is named and skipped, and also fails the check — a CI run that produced no reports is a broken upload, not a run with zero tests, and the report must not pretend it never existed. --json lists such runs under skippedRuns.

Development

npm test            # node:test, no extra dependencies — 127 tests, run serially
npm run example     # run the CLI against the demo fixture

The suite runs serially on purpose (--test-concurrency=1): the cache check times real builds, so test files running in parallel would skew its cold-vs-warm timings.

There is nothing to install: the CLI has no runtime dependencies, so npm test works straight after cloning. Run the checkout as node bin/dev-diagnosis.js … — see Troubleshooting if you reach for npx inside the checkout. The checkout ships a .aiignore that excludes the demo fixture (examples/demo-repo/), so dev-diagnosis all on the repo itself passes its own context check — the fixture is intentionally noisy test data, and it stays noisy when scanned directly (npm run example).

Requires Node.js 18 or later. CI runs the tests and the demo on Node 18, 20, 22 and 24 across Ubuntu, macOS and Windows, and smoke-tests the packed tarball by installing it in a clean directory. Windows is covered by CI only — the Windows-only code paths (the cmd.exe retry for .cmd/.bat shims, PATHEXT lookup) have not been run on a real Windows machine.

Troubleshooting

npx dev-diagnosis-cli inside a checkout of this repo fails with dev-diagnosis: not found. npx sees the checkout's own package.json — same name, same version — and tries to run the local bin, which only exists once the bin is linked. npm install does not link it: with no dependencies to install, npm creates no node_modules at all. Run the checkout directly instead — node bin/dev-diagnosis.js … — or npm link to get a dev-diagnosis command, or run npx from any other directory.

Credits

dev-diagnosis merges four tools by the same author, each of which keeps working on its own:

• ai-context-doctor — the context classification, the category weights and the score; the .aiignore fix.
• cli-path-doctor — the four shell contexts, PATHEXT lookup, shadowed copies and the toolchain search.
• cache-verdict — cold vs warm timing and the stash-and-restore safety rules.
• flake-ledger — the JUnit parsing, passed-on-retry detection and the Wilson lower-bound ranking.

The XML parser, the command runner and the report format were rewritten for the unified CLI; the rest is ported logic, adapted to a shared output format. See CHANGELOG.md for what changed per release.

License

Apache-2.0. See LICENSE.