
overfitting-audit-mcp
io.github.tylerscomic-lab/overfitting-audit-mcp
Probability of Backtest Overfitting (CSCV), Deflated Sharpe Ratio, and purged CV splits.
Documentation
overfitting-audit-mcp
An MCP server that answers "is this edge real, or a testing-hundreds-of-variants artifact?" — implementing the Probability of Backtest Overfitting (CSCV method), Deflated Sharpe Ratio, Minimum Backtest Length, and purged/embargoed cross-validation splits.
The problem this solves
Testing enough parameter combinations against the same historical data will eventually produce a great-looking backtest by chance alone. Standard backtest metrics (Sharpe, win rate, profit factor) don't distinguish a genuine edge from the best-looking result out of hundreds of near-identical variants. This audits for that specific failure mode directly, rather than trusting a single strong-looking curve.
Tools
probability_of_backtest_overfitting
Combinatorially Symmetric Cross-Validation (CSCV) method — estimates the probability that a strategy's in-sample performance rank won't hold out-of-sample.
deflated_sharpe_ratio
Adjusts a Sharpe ratio for the number of trials run and the non-normality of returns, so it can't be inflated just by testing more variants.
minimum_backtest_length
The minimum number of independent trials/observations needed before a given Sharpe ratio is statistically meaningful at all.
purged_cv_split
Generates purged and embargoed cross-validation splits for time-series backtests, preventing the lookahead leakage that ordinary k-fold CV introduces on financial data.
Use it
Hosted (recommended): MCPize — free tier, paid Pro tier for higher limits.
Self-host:
npm install
node server.js
Part of the AlgoForge suite
Prop-firm and quant-validation tools for algo traders: prop-rules-mcp, trade-journal-mcp, payout-calc-mcp, econ-calendar-mcp, montecarlo-validator-mcp, walkforward-validator-mcp, pinescript-audit-mcp, backtest-cost-sensitivity-mcp, pinescript-mcp.
License
MIT
No installation packages available.
Remote
Related Servers
io.github.tylerscomic-lab/agent-skill-audit-mcp
Scan AI agent skills and configs for hidden Unicode, prompt injection and exfiltration.
io.github.tylerscomic-lab/backtest-cost-sensitivity-mcp
Commission/slippage sensitivity sweep, breakeven cost finder, and lookahead-bias checklist.
io.github.tylerscomic-lab/broker-advisor-check-mcp
Vet US financial advisors and firms from live FINRA BrokerCheck records with ranked red flags.