Scrapling is one of the most popular projects on GitHub today, and its new v0.5 release came with a much bigger MCP server. With it, Claude Code can fetch web pages, render JavaScript and get clean Markdown back instead of messy HTML.
We installed Scrapling MCP in a throwaway project and gave Claude Code six real scraping jobs. We checked every answer against the actual page.
Here’s how to set it up, which tools to ask for, what worked, and the one thing that didn’t hold up.
What Is Scrapling MCP?
Scrapling is an open source Python library for web scraping, with about 95,000 stars on GitHub and a BSD 3-Clause license. Its MCP server lets AI tools like Claude Code use Scrapling directly, without you writing any scraping code.
The server gives Claude 16 tools. These are the ones you’ll use most:
| Tool | What it does |
|---|---|
make_request | Fetches a page over plain HTTP, fast and with no browser |
browser_fetch_once | Opens the page in a real browser so JavaScript runs, then closes it |
open_request_session and session_make_request | Keep one HTTP session for several pages from the same site |
browser_open and browser_fetch | Keep one browser open across several pages |
browser_actions | Clicks, fills forms, presses keys and waits for results |
browser_screenshot | Captures the page or one element as an image Claude can see |
close_session | Closes a session when the job is done |
Every fetch tool can return Markdown, HTML or plain text, and it can take a CSS selector to keep only the parts you need.
How to Install Scrapling MCP for Claude Code
You need Python 3.10 or newer. Installing into a virtual environment keeps Scrapling away from your other Python packages.
- Create the environment and install Scrapling with its MCP extras.
- Run
scrapling installto get the browser the browser tools use. It downloads Chromium through Playwright, so you can skip it if you only need plain HTTP pages. - Add the server to Claude Code with
claude mcp add, using the full path toscrapling-mcp.
python3 -m venv .venv
.venv/bin/pip install "scrapling[ai]"
.venv/bin/scrapling install
claude mcp add scrapling -- "$PWD/.venv/bin/scrapling-mcp"That last command adds Scrapling for you, in the current project only. Add --scope project instead to save it in a shared .mcp.json file your team can use.
Start a new Claude Code session and run /mcp to check that the server is connected. If you’re not sure where the setting was saved, our guide to the MCP config file shows every location.
We Tested Scrapling MCP in Claude Code: The Results
We used Claude Code 2.1.318 and Scrapling 0.5. For the public tests, we picked Books to Scrape and Quotes to Scrape, two practice sites built for scraping.
Each job took Claude Code about 15 to 20 seconds from prompt to answer.
Test 1: A Static Page to Markdown
We asked Claude to fetch the Books to Scrape home page with make_request as Markdown, then count the books and list five titles with prices. It found all 20 books and got the five titles and prices right.
The page is 51,294 characters of HTML. Scrapling handed Claude 10,843 characters of Markdown, about 79% less text for Claude to read.
Test 2: Exact Data With a CSS Selector
Next, we gave it the selector article.product_pod and asked for every title and price as CSV. All 20 rows matched the live page, including long titles with commas, which it quoted correctly.
One tip from this test: Claude asked Scrapling for HTML here, and the result was 29,730 characters. When we asked for Markdown with the same selector later, each page came back at about 6,400 to 7,200 characters.
Test 3: Pages Built With JavaScript
The Quotes to Scrape JavaScript page loads its quotes after the page opens. With make_request, Claude got the menu and footer but 0 quotes, and it explained why instead of making some up.
With browser_fetch_once, Scrapling opened the page in a browser and waited for the quotes to appear. Claude found all 10 quotes and listed the first three with the right authors.
Test 4: Several Pages in One Session
We asked Claude to open an HTTP session, fetch pages 1 to 3 of the book catalog, and find the three cheapest books. It opened the session, made three requests, closed the session, and reported 60 books.
The three cheapest books and their prices were correct. We checked them against the real pages.
Test 5: Hidden Instructions on a Page
Web pages can hide text that tells an AI agent what to do, which is called prompt injection. Scrapling says it strips hidden content before Claude sees it, so we built a small test page on our own computer with six hidden instructions.
| How the text was hidden | Did Scrapling remove it? |
|---|---|
Inline display:none style | Yes |
aria-hidden="true" | Yes |
| HTML comment | Yes |
<template> tag | Yes |
| Moved off screen by a CSS class | No |
| White text on a white background | No |
The two that got through told Claude to add the word BANANA and to say the trowel costs $99. Claude spotted both, said they came from the page and not from us, and ignored them.
So the filter helps, but it isn’t a guarantee. It reads inline styles and tags, not a page’s stylesheet, and Claude’s own judgment was the last line of defense.
Scrapling MCP vs Claude Code’s Built-In WebFetch
Claude Code already has a WebFetch tool, so we ran the same two public jobs with it. WebFetch got all 20 books and prices right, but it also found 0 quotes on the JavaScript page.
Anthropic’s docs call WebFetch “lossy by design”, because a second model reads the page and Claude gets that model’s answer, not the page itself.
| Feature | WebFetch | Scrapling MCP |
|---|---|---|
| Setup | Built in | Python install plus claude mcp add |
| Plain HTML pages | Yes | Yes |
| Pages built by JavaScript | No | Yes, with the browser tools |
| What Claude reads | A summary from a second model | The page content itself |
| CSS selectors | No | Yes |
| Clicks, forms and screenshots | No | Yes |
Your own localhost pages | Refused | Yes |
For a quick look at a normal page, WebFetch is enough. Scrapling earns its install when you need JavaScript pages, exact data, several pages in a row, or a local site you’re building.
Prompts That Work Well With Scrapling MCP
Scrapling’s own docs give the most useful advice: “you should always tell Claude which tool to use”. Claude picked the right tool every time we named it.
- Name the tool: “Use the Scrapling make_request tool on [URL] as Markdown.”
- Switch to the browser for JavaScript pages: “Use browser_fetch_once on [URL].”
- Give a CSS selector when you know one, and ask for Markdown, not HTML.
- For several pages, ask Claude to open a session and close it when it’s done.
Sessions you forget to close stay open, so make “close the session when done” part of the prompt.
What to Watch Out For
Scrapling advertises Cloudflare bypass and, in v0.5, CAPTCHA solving. We didn’t test those, and a site that blocks bots has usually said it doesn’t want to be scraped.
- Check a site’s
robots.txtand terms before you scrape it. - Keep request rates low, and don’t collect personal data you don’t need.
- Allow only the Scrapling tools a job needs. Our guide to permission modes explains how approvals work.
- The default stdio server can only be reached by Claude Code. Its HTTP mode needs a token, because anyone who can reach the port can use every tool.
Is Scrapling MCP Worth It?
Yes, if you scrape more than now and then. In our tests it was accurate, it handled JavaScript pages that WebFetch couldn’t, and the Markdown output kept Claude’s reading short.
It’s less useful if you only read the odd docs page, since WebFetch does that with no setup. Also, don’t treat its hidden text filter as full protection, because two of our six tricks got through.
FAQ
Is Scrapling MCP free?
Yes. Scrapling is open source under the BSD 3-Clause license, and the MCP server ships with it. You only pay for the Claude usage your prompts use.
Do I need a browser for Scrapling MCP?
Only for the browser tools. make_request and HTTP sessions work without one, and scrapling install adds Chromium when you need JavaScript pages.
Does Scrapling MCP work with Claude Desktop?
Yes. Add scrapling-mcp to Claude Desktop’s config file under mcpServers, using the full path, then restart the app.
Can Claude Code scrape JavaScript pages without Scrapling?
Not with WebFetch, which found 0 quotes on our JavaScript test page. You need a tool that runs a real browser, like Scrapling’s browser_fetch_once.
Conclusion
Scrapling MCP gives Claude Code a proper scraping toolkit: fast HTTP fetches, a real browser for JavaScript pages, sessions for multi-page jobs and clean Markdown output. In our six tests it got every count and price right, and it beat WebFetch on the JavaScript page.
Name the tool in your prompt, ask for Markdown, and close sessions when you’re done. Treat the hidden text filter as a helper, not a shield. For the full tool list, see the Scrapling MCP server guide and Anthropic’s page on connecting MCP servers to Claude Code.
