Repository avatar
AI Tools
v1.0.0
active

webfetch

io.webfetch/webfetch

Fetch any URL as markdown, text, HTML or selector-built JSON; search Google, Bing, Yahoo and Brave.

Documentation

Webfetch SDKs

Official clients for the webfetch.io scraping and search API, in JavaScript, Python and PHP. MIT licensed.

Each one wraps GET https://api.webfetch.io/v1/ and the search endpoints (/v1/google, /v1/bing, /v1/yahoo, /v1/brave). The API is ScrapingBee-compatible, so parameter names, defaults and credit costs match.

LanguageInstallRequires
JavaScriptnpm install @tuxxin/webfetchNode 18+, no runtime dependencies
Pythonpip install webfetch-io (imports as webfetch)Python 3.9+, httpx
PHPcomposer require tuxxin/webfetchPHP 8.1+, ext-curl

The shape is the same in all three

client = Webfetch(api_key)
result = client.fetch(url, render_js=True, ...)

result.content     the page, as bytes or a string
result.credits     what this call cost, from the Wf-Cost header
result.status      the status the target returned
result.resolved_url  where it ended up after redirects

Search

results = client.bing("python asyncio tutorial", country_code="us")
results["organic_results"]   [{position, title, url, displayed_url, description, ...}]
results["top_ads"]           [{position, title, url, tracking_url, domain, ...}]

client.search("yahoo", query, search_type="ads")   the ads only, same price

The response has the shape of ScrapingBee's Google Search API. Needs a paid plan with the robots.txt opt-out turned on for the key in the key portal: every engine's robots.txt disallows its result pages. Google, Bing, Yahoo and Brave are all open. 10 credits a page, 15 with light_request false; a failed search costs nothing. Reference: https://webfetch.io/documentation#search.

Credits

RequestCredits
No JavaScript rendering1
JavaScript rendering (the default)5
Premium proxy10, or 25 with rendering
Stealth proxy75
Search, light request (the default)10
Search, light_request false15
Any failed request0

mode=auto tries those routes cheapest first and charges only the one that worked; max_cost caps it. An anti-bot challenge or CAPTCHA page is never charged.

render_js defaults to true, so a call that does not set it costs 5 credits. Pass render_js=false for a plain HTTP fetch at 1 credit.

A key is required

There is no keyless tier. A free account starts with 1,000 credits and gets 250 more each month (banking up to 1,000), and needs only a confirmed email address. Each client raises WebfetchError immediately when constructed without a key, rather than spending a round trip on a request that cannot succeed.

Errors

Every SDK raises the same three-level hierarchy, so you can catch broadly or narrowly:

WebfetchError            anything went wrong
  WebfetchApiError       the API answered with an error status; carries .status and .code
    WebfetchRateLimitError   429, or 503 when free capacity is busy; carries .retry_after

A failed call is never billed, so you do not need to reconcile credits after an error.

MCP server

Webfetch is also a remote MCP server, so an AI assistant can fetch pages and run searches as tools, with no SDK in between.

Endpointhttps://webfetch.io/mcp (Streamable HTTP, POST only)
Registryio.webfetch/webfetch in the official MCP Registry; manifest: server.json
AuthAuthorization: Bearer wfk_... on every request. A free key starts with 1,000 credits and gets 250 more each month
Docshttps://webfetch.io/documentation#mcp
ToolWhat it does
fetch_pageA URL as markdown, plain text or raw HTML. JavaScript is rendered by default (5 credits); render_js false is a plain fetch (1 credit)
extract_dataJSON built from CSS or XPath selectors, so the model gets fields instead of a whole page
search_webOne page of Google, Bing, Yahoo or Brave results as JSON: organic results, ads, related searches and questions. 10 credits a page; needs search access on the key

A failed call costs nothing. Claude Code, in .mcp.json:

{
  "mcpServers": {
    "webfetch": {
      "type": "http",
      "url": "https://webfetch.io/mcp",
      "headers": { "Authorization": "Bearer ${WEBFETCH_KEY}" }
    }
  }
}

Documentation

https://webfetch.io/documentation

No installation packages available.

Remote