Repository avatar
AI Tools
v0.2.45
active

denpex-mcp

io.github.Denpex/denpex-mcp

Diagnose GPU and ML training failures (CUDA OOM, NCCL timeouts, Xid faults) locally or via cloud.

Documentation

Denpex MCP server

Log files up to 100 MB are read in bounded chunks and reduced to diagnostic evidence before cloud submission. Binary files and invalid UTF-8 are rejected. Keep the original log: evidence selection can omit context.

Diagnose GPU/ML training failures. CUDA OOM, NCCL timeouts, Xid faults, NaN loss, checkpoint corruption, without leaving your AI tool. Ask Claude Desktop, Cursor, or Claude Code to "diagnose the crash in ./logs/train.err" and Denpex reads the log straight off disk and hands back the root cause and the fix.

  • 🩺 Root cause + escalating fix (primary → fallback → emergency), verification command, and prevention.
  • 🧭 Find the culprit rank in a distributed/NCCL hang, and route GPU Xid faults to the right node action, offline, instantly.
  • 📚 Bundled failure encyclopedia (500+ documented classes) searchable by error code or symptom.
  • 🔒 Zero-egress mode (DENPEX_LOCAL=1): diagnose entirely on-host, nothing leaves the machine. Plus client-side PII scrubbing even in cloud mode.

Quick start (60 seconds)

1. Get an API key (optional but recommended)

Sign in at denpex.com/dashboard → API Keys and create a key (dpx_…). The key unlocks authenticated cloud diagnosis, your diagnosis history, quota, and fleet incidents. Without a key, diagnose_crash and ask_followup default to running the canonical deterministic engine entirely locally (unlimited free local runs). You can escalate a specific diagnosis to the Denpex cloud without a key by passing cloud: true, which uses the frictionless anonymous engine (requires an API key or browser check if anonymous quota is exhausted). Set DENPEX_LOCAL=1 explicitly for permanent zero-egress.

2. Add it to your client

Claude Desktop

Edit claude_desktop_config.json (macOS: ~/Library/Application Support/Claude/, Windows: %APPDATA%\Claude\):

{
  "mcpServers": {
    "denpex": {
      "command": "npx",
      "args": ["-y", "denpex-mcp"],
      "env": { "DENPEX_API_KEY": "dpx_your_key_here" }
    }
  }
}

Restart Claude Desktop.

Cursor

Create .cursor/mcp.json in your project (or ~/.cursor/mcp.json globally):

{
  "mcpServers": {
    "denpex": {
      "command": "npx",
      "args": ["-y", "denpex-mcp"],
      "env": { "DENPEX_API_KEY": "dpx_your_key_here" }
    }
  }
}
Claude Code
claude mcp add denpex -e DENPEX_API_KEY=dpx_your_key_here -- npx -y denpex-mcp

3. Use it

"My training job crashed, diagnose the log at ./logs/rank0.err." "What does Xid 79 mean?" "This 64-GPU run is hung. Which rank failed first?" (→ analyze_cascade)

Or pick the diagnose-training-crash prompt your client surfaces.


Offline / air-gapped mode

Diagnose entirely on-host, no network, no logs leaving the cluster. Ideal for VPCs, secure data centers, and privacy-sensitive labs. Add DENPEX_LOCAL=1:

{ "mcpServers": { "denpex": {
  "command": "npx", "args": ["-y", "denpex-mcp"],
  "env": { "DENPEX_LOCAL": "1" }
} } }

The package bundles the same canonical deterministic diagnosis pipeline used by the hosted wrapper: 10,700+ patterns, causal detectors and contradiction gates, Xid routing, cross-rank cascade localization, and the 500+ class encyclopedia. It returns the initiating rank/node when the evidence identifies one. Only the hosted LLM fallback and live-research enrichment are unavailable offline.


Tools

ToolWhat it doesNeeds
diagnose_crashRoot cause + escalating fix from logs / file / dir. Also accepts telemetry (dmesg / nvidia-smi / ibstat), codeContext, topologyMap, typed operatorAnswers, and (after an anonymous MCP restart) the prior refinementToken.local-first by default without a key; cloud with key or cloud: true
lookup_failureSearch the bundled encyclopedia (500+ classes) by code/keyword/symptom.works offline
analyze_cascadeFind the culprit rank in a distributed hang; route Xid faults.works offline
submit_fix_outcomeReport whether a fix worked. This is what makes the next diagnosis better.API key
check_quotaYour plan and remaining diagnoses this month.API key
recent_diagnosesRecall your recent diagnoses and their fixes.API key
ask_followupConversational follow-up on a diagnosis.local text-extraction without a key; generative chat with Team plan / trial
check_versionsFlag known-bad CUDA/PyTorch/NCCL/driver combos.Team plan / trial
compare_runs"It worked yesterday", isolate what changed between a good run and a bad one.Team plan / trial
diff_environmentsDiff torch/CUDA/NCCL/driver versions and env vars between two runs.Team plan / trial
validate_checkpointFind the newest checkpoint that is actually safe to resume from.Team plan / trial
preflight_check"Is this cluster safe to launch on?", blockers before you burn GPU-hours.Scale plan
node_healthScore every monitored node, worst first.Scale plan
predict_failuresNodes trending toward a failure that has not happened yet.Scale plan
list_incidentsIncidents Denpex's monitor caught & escalated on your fleet.Scale plan

Gated tools degrade gracefully, they tell you exactly which plan unlocks them and which tools work on your current plan. The failure encyclopedia is also exposed as resources at denpex://failures/{slug}.

Structured output. Every tool above returns structuredContent alongside its markdown, validated against a declared output schema, so an agent branches on confidenceBand === "low" or iterates rankedHypotheses instead of regexing prose. Two fields deserve attention when you write the agent loop:

  • confidence: null does not mean zero. It means the engine measured this band's historical accuracy and declined to state a number. Treat the verdict as a lead. Do not substitute the raw score.
  • unfalsified: true identifies only the explicitly enabled reduced compatibility fallback, where the canonical contradiction graph did not run. Normal cloud and DENPEX_LOCAL=1 diagnosis both use the canonical engine and set it to false.

Environment variables

VariableDefaultPurpose
DENPEX_API_KEY,Your dpx_… key. Enables cloud tools.
DENPEX_API_URLhttps://api.denpex.comOverride the API base (self-host / staging).
DENPEX_LOCAL01 → diagnose entirely on-host (zero egress).
DENPEX_PRIVACYoffmask / strict → scrub PII from logs before they leave the machine.

Develop

npm install
npm run build      # regenerates the canonical engine + corpus, then bundles dist/index.js
npm test           # 111 unit + protocol-integration tests
npm run inspector  # interactive protocol inspector (MCP Inspector)

The offline corpus (src/data/denpex_patterns.json) and canonical engine (src/offline/canonical-engine.mjs) are generated from the repo's single source of truth during npm run build, never hand-edit either artifact.


Official Denpex Ecosystem & Documentation

ResourceDescriptionLink
Denpex PlatformDeterministic GPU and ML training failure diagnosticsdenpex.com
Documentation HubQuickstart guides, architecture, and operational referencedenpex.com/docs
Python Agent & CLIOn-host monitoring, evidence capture, and zero-egress CLIdenpex.com/docs/agent
REST API ReferenceProgrammatic /api/diagnose and preflight endpointsdenpex.com/docs/api
VS Code ExtensionOffline GPU crash diagnostics in VS Codedenpex.com/vscode
Open VSX ExtensionExtension for Cursor, Windsurf, VSCodium, and Gitpoddenpex.com/openvsx
Failure Encyclopedia500+ documented CUDA, NCCL, Xid, Slurm, and vLLM failuresdenpex.com/failures
Free GPU ToolsVRAM, NCCL, DeepSpeed ZeRO, and Slurm calculatorsdenpex.com/tools
NVIDIA Xid ReferenceHardware vs software Xid fault routing tabledenpex.com/xid
Release ChangelogPlatform, agent, SDK, and extension release notesdenpex.com/changelog
NPM
denpex-mcp
Install Command
npm install denpex-mcp
Runtime: npx