Repository avatar
Developer Tools
v0.1.0-alpha.2
active

spirolo

io.github.spirolo/spirolo

Whole-project analyzer for JavaScript test suites

Documentation

Spirolo, a purple millipede, holding a bug in a jar

Spirolo

English | 简体中文

CI Licence npm

Spirolo is a static analyzer for JavaScript and TypeScript test suites.

It reads tests, source code and runner configuration together to find missing setup guards, untracked skips, duplicate tests and assertions that may pass without testing anything. Analysis runs locally in a Rust binary, without running your tests or sending your code anywhere.

What Spirolo will not do

  • It does not run your tests or execute your code.
  • It does not fix or rewrite your source files.
  • It does not fail CI by default: findings are advisory until you raise a rule's severity.
  • It does not send your code anywhere: analysis needs no network access.
  • It covers JavaScript and TypeScript test suites only.

Install

Node 20 or newer, from your project's root:

npm install -D spirolo@alpha
npx spirolo check

No Spirolo config is needed. Install your project's dependencies first so its TypeScript configuration and imports can resolve.

The npm launcher selects a native platform package for Linux or macOS (x64 and arm64), without a postinstall download.

To build from source instead, with the current stable Rust toolchain, from this repository's root:

cargo build --release --locked -p spirolo
./target/release/spirolo check /path/to/your/project

One real run

Captured from the release binary built at Spirolo commit 4a8d80cd, on Kaneo at commit afe97be2473e0cd97fe4a63e526fef9ac4df85b0, with its dependencies already installed. Set KANEO to the path of that checkout, then run from Spirolo's source directory:

./target/release/spirolo check "$KANEO"

Excerpt below: [...] marks omitted lines. The timing is from this run, not a benchmark.

1754 files · 351 test files · 1797 tests (1797 vitest) · 4309 expects · 0 handlers · 35 configs · 253ms
Health C (60)
[...]
● fail-closed-setup (10 files, 19 findings) · advisory: 19/19 (UNADJUDICATED)
  apps/api/vitest.config.ts (2)
    :0 setup does not guard the console (no throwing console.error/warn spy or fail-on-console package)
    :0 setup does not restore mocks between tests (no restoreMocks or afterEach restoreAllMocks)
[...]
34 findings in 3 of 16 rules · 0 warn · 0 error · 34 info · exit 0
 hidden 1226 findings: 1211 experimental (gate-truth 11, unused-mock 3, flake-candidate 1, untested-hotspot 1196), 15 from DROPPED or FAIL rules (weak-assertion 15); spirolo check --all shows them
 note   no churn input: untested-hotspot ranks by complexity × weakness alone. Save `git log --numstat --format= --since=180.days` to a file and pass it with `--churn <file>` or name it in `inputs.churn`

This run reported missing setup guards, absence-only assertions and a possible duplicate. It did not execute Kaneo's tests or verify each finding. The command exited successfully because these findings are advisory. The hidden count includes experimental hotspot rankings without churn data; it is not a count of confirmed bugs.

What it finds

The default report shows these advisory rules:

RuleWhat to inspect
absence-anchorA test checks that something is absent without checking that the expected content rendered.
dead-skipA test is skipped without a linked issue, or focused so other tests will not run.
duplicate-testTest bodies and their modeled setup look equivalent. Check whether they cover different behavior before deleting either.
fail-closed-setupRunner setup lacks guards for mock cleanup, console errors or unhandled requests.

check --all also shows experimental rules and rules whose measurements failed or were dropped. These include merge-gate checks, static flake candidates, unused mocks and source modules ranked by churn, complexity and test strength. Use spirolo explain <rule> to read a rule's claim and measured status.

Why another test tool?

Test lint plugins check patterns in test code. Coverage reports tell you what ran. Mutation tools such as Stryker check whether tests detect changes to production code. Spirolo adds a view across tests, imports, setup and runner configuration, and can read existing coverage, mutation and run reports. Keep running your tests and using those tools: a static finding answers a different question.

The tooling comparison explains that division of work.

How far to trust it

No rule is proven yet. The measurements below record how often sampled findings were judged correct, not how many bugs Spirolo can find in your project.

  • Advisory: shown by default once a rule has at least 15 decided findings and measured precision of at least 0.80. These findings report at info.
  • Experimental: findings appear only with --all unless you opt the rule in. The evidence is too limited or the measured precision is below the advisory floor.
  • FAIL or DROPPED: findings appear only with --all and never gate, even with a severity override.
  • Proven: reserved for a rule that passes the measurement bar and a fresh held-out check. None has done so.

Today, findings do not fail a command by default. To make a rule gate, set its severity to warn or error in .spirolorc.jsonc, subject to the rule's allowed severities. For example:

{ "rules": { "dead-skip": { "severity": "warn" } } }

Rule tiers explains the thresholds, labels, exit codes and override limits. spirolo rule-status prints the measurements embedded in your binary.

Measured precision

This table is generated from the embedded measurement data, using public repositories only. A perfect result on a small sample is not proof; the confidence intervals show the uncertainty. These are measurements at the recorded commit, not a new evaluation of every later change. n/a means no findings were decided, so there is no precision estimate.

Measured at spirolo commit 939f0af519fd50318e092700b74094b71cfeca23 across 40 OSS dev entries. Advisory: at least 15 decided and shown precision at least 0.80.

RuleTierStatusTrue / decidedPrecision95% Wilson intervalMeasured entries
absence-anchoradvisoryUNADJUDICATED28 / 281.0000.879–1.00016
dead-skipadvisoryUNADJUDICATED39 / 391.0000.910–1.00021
duplicate-testadvisoryUNADJUDICATED15 / 180.8330.608–0.94219
fail-closed-setupadvisoryUNADJUDICATED19 / 191.0000.832–1.00010
flake-candidateexperimentalINSUFFICIENT3 / 60.5000.188–0.8127
gate-truthexperimentalUNADJUDICATED12 / 180.6670.437–0.8378
untested-hotspotexperimentalUNADJUDICATED5 / 160.3130.142–0.55640
unused-mockexperimentalINSUFFICIENT0 / 0n/an/a1
data-layer-mockhiddenDROPPED0 / 0n/an/a0
flake-observedhiddenFAIL39 / 520.7500.618–0.8484
generator-walkhiddenDROPPED0 / 0n/an/a0
handler-drifthiddenDROPPED0 / 0n/an/a0
orphan-testhiddenDROPPED4 / 140.2860.117–0.54612
redundant-testhiddenDROPPED0 / 120.0000.000–0.2435
stale-snapshothiddenDROPPED0 / 0n/an/a3
weak-assertionhiddenDROPPED12 / 340.3530.215–0.52117

Where it runs

The native package targets are Linux with glibc and macOS, on x64 and arm64. Windows and Alpine/musl are outside the alpha's targets. The native binary needs neither Node nor network access for analysis.

The measured scope is JavaScript and TypeScript React projects using Vitest or Jest. Spirolo also reads Playwright configuration and test patterns; this does not make every framework or configuration a measured target.

Use the CLI locally, Markdown summaries or GitHub annotations in CI, or MCP in an agent client. JSON and SARIF reports are available with --format json and --format sarif. A VS Code extension using the same binary can be packaged and installed locally.

Scope

Spirolo does not run tests, launch mutation engines, collect coverage, generate tests or host run history. It reads artifacts you already have. Missing artifacts are reported as unverified context, not as evidence that the suite works.

A static flake candidate is not an observed flaky test. A duplicate finding is a reason to inspect the tests, not permission to delete them. The provisional health score is a summary of signals, not a guarantee of correctness.

Vue, Cypress, other languages and arbitrary non-React frameworks are outside the measured scope.

Status and limits

Version 0.1.0-alpha.2: advisory alpha. It is on npm as spirolo@alpha; the VS Code extension is not published yet. Before 1.0, any alpha may change commands, output and rule behaviour; the changelog lists every change.

Static analysis can miss dynamic configuration and patterns it does not model. If a TypeScript configuration cannot load, Spirolo reports module-resolution problems and marks affected findings as unreliable. Install missing dependencies and resolve the configuration warning before relying on those rows.

The human report caps displayed files and findings. spirolo check --rule <rule> --all shows the full report for that rule, including experimental or dropped findings. Review the evidence before opting a rule into CI.

Contributions are welcome: see CONTRIBUTING.md. For vulnerability reports, see SECURITY.md.

Licence

Apache 2.0.

NPM
spirolo
Install Command
npm install spirolo