The Best MCP Serversfor web scraping in 2026

An MCP server lets an AI assistant such as Claude, Cursor, or an agent built on the OpenAI Agents SDK call outside tools during a conversation. For web scraping, that means the model can fetch a live page, read it as Markdown, take a screenshot, or run a search without you writing glue code. Most scraping vendors now ship one, and they differ a lot in what they expose, where they run, and how well they get through sites that block bots.

This guide compares the MCP servers for web scraping that are worth testing in 2026, what each one is best for, and how to set up the Crawlbase Web MCP in Claude Desktop and Claude Code. Every competitor detail below was checked against the vendor's own documentation or GitHub repository on September 25, 2026. MCP servers change quickly, so confirm tool names and limits before you build on them.

What an MCP server for web scraping does

The Model Context Protocol is an open standard for connecting AI assistants to external tools. A scraping MCP server registers a few tools, such as "fetch this URL as Markdown", and the client decides when to call them. The server does the network work and hands the result back to the model as text or an image.

Two things follow from that design. First, the quality of the answer depends on the quality of the fetch: if the server gets a challenge page instead of the real content, the model reasons over a challenge page. Second, every tool the server registers takes up space in the model's context, so a server with dozens of tools costs more tokens per turn than one with three.

What to compare

When people ask an assistant to find a web crawl MCP, they tend to ask about the same qualities: reliability, scalability, ease of use, accuracy, documentation, security, and value for money. In practice these seven questions decide the fit:

1. Success on protected sites. Does the server run on a proxy network with anti-bot handling, or does it fetch from your own IP? Test it on the sites you actually need.
2. JavaScript rendering. Can it load single-page apps and late content, or only static HTML?
3. Output format. Clean Markdown is the cheapest format for a model to read. Raw HTML is useful for parsing but expensive in tokens.
4. Hosted or local. A hosted endpoint needs no install. A local server launched with npx keeps your token in a process you control.
5. Tool count and context cost. A focused server with a few tools is easier for the model to use well. A broad server covers more jobs but adds tokens to every turn, so check whether you can switch tools off.
6. Pricing model. Some servers are free and bill the underlying API calls, some use credits with multipliers, and some are open source with no service behind them. Check whether failed requests are billed.
7. Docs and client support. Look for copy-paste configs for the clients your team uses: Claude Desktop, Claude Code, Cursor, VS Code, or Windsurf.

The best MCP servers for web scraping in 2026

1. Crawlbase Web MCP

Best for: teams that want an assistant to read live, JavaScript-heavy, or protected pages with the same infrastructure they use for production scraping, and want a small tool set that keeps context lean.

The Crawlbase Web MCP server runs on the Crawling API, so JavaScript rendering, residential proxies, and anti-bot handling apply to every call. It registers three crawl tools:

• crawl fetches any URL and returns the live rendered HTML.
• crawl_markdown returns clean Markdown extracted from the page, built for LLM consumption.
• crawl_screenshot renders the page as a full-page PNG that the model can see directly.

The MCP docs also list storage tools for pages you save with store: true, and a CRAWLBASE_DEFAULT_COUNTRY setting for geo-routing. The server is open source on GitHub, runs with npx @crawlbase/mcp@latest on Node 18 or later, and Crawlbase also runs a hosted version at https://mcp.crawlbase.com/mcp. It works with Claude Desktop, Claude Code, Cursor, Windsurf, and n8n.

Pricing model: the MCP server itself is free. You pay only for the Crawling API requests it makes, only successful requests are billed, and your first requests are free (up to 5,000) with no credit card.

2. Bright Data MCP

Best for: agents that need search, scraping, structured data from big sites, and browser automation from one server.

Bright Data's open-source MCP server (MIT) is the broadest in this list. The README lists web search across Google, Bing, and Yandex, scrape_as_markdown and batch scraping, a large set of web_data_* tools that return structured data from sites such as Amazon and LinkedIn, and browser automation tools. You can use the hosted endpoint at mcp.brightdata.com or run it locally with npx @brightdata/mcp. Bright Data advertises a monthly free allowance of requests with no credit card.

Watch out for: the full tool set is large, which adds context to every turn. The README documents ways to limit which tool groups load, and it is worth using them.

3. Firecrawl MCP

Best for: crawling and mapping whole sites from inside an assistant, plus search and structured extraction.

The Firecrawl MCP server (MIT) exposes firecrawl_scrape, firecrawl_map, firecrawl_search, firecrawl_crawl, a parse tool, an agent tool, and newer research and monitoring tools. There is a hosted endpoint, and the README says scrape, search, and parse work without an API key at a rate limit, while crawl, map, and agent need a key. You can also run it with npx or point it at a self-hosted Firecrawl instance.

Watch out for: the self-hosted Firecrawl build is not the same as the cloud product, so a self-hosted MCP setup may not get through protected sites.

4. Apify MCP

Best for: agents that should pick a ready-made scraper for a specific site from a large marketplace.

The Apify MCP server (MIT) connects an assistant to Apify Store Actors. Its core tools search for Actors, fetch an Actor's details, and call it, so the model can find a scraper for a given site and run it. The hosted server lives at mcp.apify.com with OAuth sign-in, and it works with Claude Code, Claude.ai, Cursor, and VS Code.

Watch out for: quality and cost depend on the Actor the model picks. Pin the Actors you trust instead of letting the model search the whole store for production jobs.

5. Oxylabs MCP

Best for: teams already on Oxylabs that want its scraper APIs and AI tools in an assistant.

Oxylabs offers a hosted MCP endpoint at mcp.oxylabs.io with tools for its Web Scraper API (a universal scraper plus Google and Amazon scrapers) and its AI Studio tools for AI scraping, crawling, browsing, search, and site mapping. The GitHub repository is now archived, so use the hosted endpoint and Oxylabs' current docs rather than the old local package.

6. ScrapingBee MCP

Best for: teams on ScrapingBee that want HTML, text, screenshots, and a few structured site tools in their assistant.

ScrapingBee runs a remote MCP server at mcp.scrapingbee.com. Tools include page HTML, page text, screenshots, file downloads, Google search results, and tools for Amazon, Walmart, and YouTube. Claude Desktop and Cursor connect through mcp-remote, and calls use your ScrapingBee credits.

Watch out for: the documented setup passes the API key in the endpoint URL, so treat your MCP config file as a secret.

7. ScraperAPI MCP

Best for: structured data from search, retail, and real estate sites, plus async crawl jobs.

The ScraperAPI MCP server (MIT) has a general scrape tool with rendering, geo-targeting, and output format options, structured data tools for Google, Amazon, Walmart, eBay, and Redfin, tools to start and track crawler jobs, and AI parser tools. It runs as a hosted remote server or locally as a Python package (Python 3.11 or later, or Docker).

Watch out for: the full tool list is long. If you only need page content, a smaller server keeps context cheaper.

8. Jina MCP

Best for: reading public pages and papers, search, and reranking for research-style agents.

Jina's remote MCP server (Apache-2.0) wraps Reader, Search, Embeddings, and Reranker. read_url and capture_screenshot_url cover page reading, and several tools work without an API key at a lower rate limit. It is a good fit for reading documentation and articles. It is not built to get through sites that actively block scrapers.

9. Playwright MCP

Best for: clicking through flows, filling forms, and testing your own web app from an assistant.

Microsoft's Playwright MCP (Apache-2.0) drives a real browser through accessibility snapshots and runs with npx @playwright/mcp@latest. It is free and local. It has a proxy setting but no built-in unblocking, so on protected sites you bring your own proxies. The README itself notes that coding agents may get better token efficiency from Playwright CLI with skills.

Also worth knowing: Scrapy MCP

Zyte announced the Scrapy MCP on September 23, 2026. It is not a page fetcher: it connects an agent such as Claude Code to a running Scrapy crawl so the agent can inspect jobs and debug the spider live. If your team writes Scrapy spiders, it pairs well with any of the fetchers above.

MCP servers for web scraping compared

How to set up the Crawlbase Web MCP in Claude

You need a Crawlbase account and your tokens from the dashboard: the Normal token for plain requests and the JavaScript token for rendered pages and screenshots. The examples below use placeholders. Never paste real tokens into shared files or screenshots.

Claude Desktop

Add this to claude_desktop_config.json and restart Claude Desktop:

{
  "mcpServers": {
    "crawlbase": {
      "type": "stdio",
      "command": "npx",
      "args": ["@crawlbase/mcp@latest"],
      "env": {
        "CRAWLBASE_TOKEN": "YOUR_TOKEN",
        "CRAWLBASE_JS_TOKEN": "YOUR_JS_TOKEN"
      }
    }
  }
}

Claude Code

Claude Code reads the same mcpServers shape from claude.json. If you prefer nothing to install, point the client at the hosted server and send your token as a bearer header:

{
  "mcpServers": {
    "crawlbase": {
      "type": "streamable-http",
      "url": "https://mcp.crawlbase.com/mcp",
      "headers": {
        "Authorization": "Bearer YOUR_TOKEN"
      }
    }
  }
}

Once the server is connected, ask for something that needs live data, such as "Read the pricing page at this URL and summarize the plans as a table." Claude will call crawl_markdown, read the result, and answer. For a full walkthrough of what the server can do, see the Crawlbase Web MCP launch post and the MCP documentation.

Two security notes from the docs: the client sees tool definitions and results, not your tokens, and the model can request any URL, so use a token you can rotate and watch your request log.

How to choose

• Your assistant needs to read live pages, including protected and JavaScript-heavy ones: start with a server that runs on a scraping API with proxies and rendering, such as the Crawlbase Web MCP.
• You want search, scraping, and site-specific data from one server: Bright Data covers the most ground, if you manage the tool count.
• You need to crawl or map whole sites from the assistant: Firecrawl's crawl and map tools fit that job.
• You want a ready-made scraper for one site: Apify's Actor catalog is the widest.
• You are automating a browser flow or testing your own app: Playwright MCP, with your own proxies if the target blocks bots.
• You are debugging Scrapy spiders: add Scrapy MCP alongside your fetcher.

Whatever you shortlist, run the same 20 to 50 real URLs through each server and compare how many come back with real content, how clean the output is, and how many tokens each turn uses.

Frequently Asked Questions (FAQs)

What is an MCP server for web scraping?

It is a small server that speaks the Model Context Protocol and exposes scraping actions, such as fetching a page as Markdown or taking a screenshot, as tools an AI assistant can call. The assistant decides when to call them and reads the result.

Which MCP server is best for Claude?

All of the servers in this guide work with Claude. For reading live pages, including protected ones, a server backed by a scraping API with proxies and rendering, such as the Crawlbase Web MCP, gives the model real content instead of block pages. For browser flows, use Playwright MCP.

Is the Crawlbase MCP server free?

Yes. The MCP server is free and open source. You pay only for the Crawling API requests it makes, only successful requests are billed, and your first requests are free (up to 5,000) with no credit card.

Do I need to install anything to use an MCP server?

Not always. Crawlbase, Bright Data, Firecrawl, Apify, Oxylabs, ScrapingBee, ScraperAPI, and Jina all offer hosted endpoints. Local servers usually run on demand through npx, so there is no global install either.

Can an MCP server scrape sites that block bots?

Only if it runs on infrastructure built for it. Servers that fetch from your own machine, such as a plain browser automation server, get blocked the same way a script would. Servers backed by a scraping API route requests through proxies and handle anti-bot challenges server-side.