What it does
Webfetch fetches web pages through one GET endpoint at https://api.webfetch.io/v1/. Each request needs a url parameter, and every other parameter has a default. The service uses headless browsers or a plain HTTP fetch. Requests leave through rotating datacenter and ISP exits. The response can be raw HTML, cleaned text, Markdown, a screenshot, or JSON that you define with the extract_rules parameter. The extract_rules parameter accepts CSS or XPath selectors. Signup needs an email address and a six-digit code. The service honors robots.txt on every request by default. Parameter names and defaults match ScrapingBee's. Moving from ScrapingBee means changing the base URL and the key.
Capabilities
- Setting mode=auto retries a blocked page on progressively more capable routes, bills only the route that worked, and stays within the max_cost ceiling.
- The js_scenario parameter clicks, fills, scrolls and waits before the page is captured.
- The return_page_markdown parameter strips navigation, header, footer, forms and consent banners and makes every link absolute.
- The Search API returns ranked Google, Bing, Yahoo and Brave results and their ads as JSON for 10 credits a search.
- The country_code parameter targets a region and session_id pins an exit IP, and a country that has no exit is refused rather than served from elsewhere.
When to use it
- You need to monitor price and stock on retail pages and get the values back as JSON.
- You need clean page content for pipelines built on large language models (LLMs) and retrieval-augmented generation (RAG).
- You need full-page screenshots, kept alongside the raw HTML, for compliance and archival.
