Clean web data for developers and agents
Any URL to markdown or structured data.
Scrape, crawl, search and extract from one API, one credit a page.
[01 / Endpoints]
Every endpoint. One response shape.
From one page to a whole site, the envelope never changes. Learn one and you have learned all of them.
One URL in. Clean Markdown, the page’s own structured data, links, or a summary out, each response carrying its quality score. The primitive everything else builds on.
Read the docsEvery URL on a site in one call, sitemap and discovery combined, deduplicated and orderable by relevance. See the whole site before you decide what is worth pulling.
Read the docsWalk a whole site as one asynchronous job, with depth, path and subdomain controls and a webhook when it finishes. Every page returns as clean Markdown in the same envelope.
Read the docsRun a query and get ranked results that arrive already fetched and cleaned, with time, region and domain filters. Fresh, cited context for a model in one round trip.
Read the docsPoint a JSON schema at one page or a whole site and get typed records back, deduplicated across pages, with per-field confidence and the passage each value was drawn from.
Read the docsGive it a domain and get the company ready to render: logos that work on light and dark, the brand’s real colours ranked by how the site uses them, fonts and socials. Every field carries its source and confidence.
Read the docsThe brand engine’s visible tip. One URL that drops straight into an img tag and returns the company’s mark, correct on light or dark, authenticated by a key that is safe to publish in your markup.
Read the docs**Published:** 2005-09-17
# Web scraping
**Web scraping**, **web harvesting**, or **web data extraction** is data scraping used for extracting data from websites.
Scraping a web page involves fetching it and then extracting data from it. Fetching is the downloading of a page.
Web pages are built using text-based markup languages (HTML and XHTML), and frequently contain useful data in text form.
## History
After the birth of the World Wide Web in 1989, the first web robot was created in June 1993.
In December 1993, the first crawler-based web search engine, JumpStation, was launched.
## Techniques
Data extraction techniques range from manual collection to sophisticated automation.
### Human copy-and-paste59,769 → 6,831 tokensconfidence 0.991 credit
50 linksevery one with its last change1 credit
12/12 pages0 failed1 credit a page
What is RAG? - Retrieval-Augmented Generation AI Explained - AWS
3 days ago - What is Retrieval-Augmented Generation (RAG), how and why businesses use RAG AI, and how to use RAG with AWS.
What is RAG (Retrieval Augmented Generation)? | IBM
October 31, 2024 - Retrieval augmented generation (RAG) is an architecture for optimizing the performance of an artificial intelligence (AI) model by connecting it with external knowledge bases.
ELI5 What is a is Retrieval-Augmented Generation (RAG)
What is Retrieval-Augmented Generation (RAG)? | Google Cloud
Retrieval-augmented generation (RAG) combines LLMs with external knowledge bases to improve their outputs. Learn more with Google Cloud.
What Is Retrieval-Augmented Generation aka RAG | NVIDIA Blogs
October 9, 2025 - Retrieval-augmented generation is a technique for enhancing the accuracy and reliability of generative AI models with information fetched from specific and relevant data sources.
5 resultsevery one fetched and cleaned6 credits
50 records typed off one page5 credits
4 assets6 colours ranked4 socials5 credits
served from cachelocked to your domainsno credits
[02 / Behind one call]
You call one endpoint. We handle the hard part.
No headless browsers to run, no proxies to bring, no retry logic to write. The response is clean or it costs you nothing.
JS-rendered pages
Client-side apps come back as content.
React, Vue, anything that draws in the browser. You never see an empty shell where the page should be.

what comes back
<div id="root"></div>
spinning up a browser
Blocks and challenges
Challenge walls are our problem.
Pages that greet a bot with a wall come back as content anyway. When one truly will not yield, the response says so honestly.
Verifying you are human…
Challenge cleared
what your agent sees
# software engineer jobs
Sort by: **relevance** - **date**
## Careers in DevOps engineering
### Senior DevOps Site Reliability Engineer
Reynolds and Reynolds
North Andover, MA 01845
- Referral program
- Parental leave
- 401(k) matching
- Health insurance
### DevOps Engineer
Triumph Tech, LLC
Surprise, AZ 85378
- 401(k)
- Health insurance
- Paid time off
- Vision insurance
### Senior Cloud Engineer
firstPRO
Hybrid work in Marlton, NJ 08053Failures are free
Calls run, credits are charged for what came back, and a page we could not deliver never reaches your bill.
[03 / Integration]
Call it from your stack.
An official client in 6 languages, or plain HTTP from anywhere.
curl https://api.hydrafetch.com/v1/web/scrape \
-H "X-API-Key: $HYDRAFETCH_API_KEY" \
-H "Content-Type: application/json" \
-d '{"url": "https://en.wikipedia.org/wiki/Web_scraping"}'$npm install @hydrafetch/node-sdkimport { Hydrafetch } from "@hydrafetch/node-sdk";
const hf = new Hydrafetch(process.env.HYDRAFETCH_API_KEY);
const page = await hf.scrape("https://en.wikipedia.org/wiki/Web_scraping");
console.log(page.markdown);$pip install hydrafetchfrom hydrafetch import Hydrafetch
hf = Hydrafetch()
page = hf.scrape("https://en.wikipedia.org/wiki/Web_scraping")
print(page["markdown"])$go get github.com/Hydrafetch/go-sdkhf, err := hydrafetch.New("")
if err != nil {
log.Fatal(err)
}
page, err := hf.Scrape(context.Background(), "https://en.wikipedia.org/wiki/Web_scraping", nil)
if err != nil {
log.Fatal(err)
}
fmt.Println(page.Markdown)$gem install hydrafetchrequire "hydrafetch"
hf = Hydrafetch::Client.new
page = hf.scrape("https://en.wikipedia.org/wiki/Web_scraping")
puts page["markdown"]$cargo add hydrafetchuse hydrafetch::Client;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let hf = Client::from_env()?;
let page = hf.scrape("https://en.wikipedia.org/wiki/Web_scraping", None).await?;
println!("{}", page.markdown.unwrap_or_default());
Ok(())
}$composer require hydrafetch/hydrafetchuse Hydrafetch\Client;
$hf = new Client();
$page = $hf->scrape('https://en.wikipedia.org/wiki/Web_scraping');
echo $page['markdown'];For pipelines
Short calls return. Long ones hand you an id.
Requests wait inline by default, so most calls are a single round trip. A crawl or a batch hands back an id you poll instead. Wire either into a queue, a cron, or a nightly refresh and stop thinking about it.
Read the docsFor agents
Every endpoint is a tool.
Point your client at the MCP server and an agent can reach the live web mid-conversation to ground its own answers. Same primitives, same credits.
Cursor
Codex[04 / In practice]
From raw web to product features.
6 ways teams wire it in. The envelope never changes.
**Published:** 2005-09-17
# Web scraping
**Web scraping**, **web harvesting**, or **web data extraction** is data scraping used for extracting data from websites.
Scraping a web page involves fetching it and then extracting data from it. Fetching is the downloading of a page.
Web pages are built using text-based markup languages (HTML and XHTML), and frequently contain useful data in text form.
## History
After the birth of the World Wide Web in 1989, the first web robot was created in June 1993.
In December 1993, the first crawler-based web search engine, JumpStation, was launched.
## Techniques
Data extraction techniques range from manual collection to sophisticated automation.
### Human copy-and-pasteChunk, embed, upsert. Tomorrow only the changed pages cost anything.
13 credits in totalsame response shape every call
What is RAG? - Retrieval-Augmented Generation AI Explained
aws.amazon.com · 1,512 words
What is RAG (Retrieval Augmented Generation)? | IBM
ibm.com · 2,068 words
ELI5 What is a is Retrieval-Augmented Generation (RAG)
reddit.com · 1,397 words
What is Retrieval-Augmented Generation (RAG)? | Google Cloud
cloud.google.com · 924 words
What Is Retrieval-Augmented Generation aka RAG | NVIDIA Blogs
blogs.nvidia.com · 1,630 words
One round trip. The agent answers from pages that exist today.
6 credits in totalsame response shape every call
A directory becomes rows. Fields it cannot find come back null, never invented.
5 credits in totalsame response shape every call
CatchIntent
Information · Software Publishers · NAICS 513210
#000000
#2d2616
#667eea
#fee685
#f99c00
#00bb7f
Enough to render the company in your UI without a designer touching it.
5 credits in totalsame response shape every call
Same envelope every run, so a diff is a diff and not a parser change.
20 credits in totalsame response shape every call
One credit for the whole inventory. Then crawl only what actually moved.
1 credit in totalsame response shape every call
[05 / Second opinions]
Do not take our word for it. Ask an AI to check.
Point your coding agent at your own repository, or ask an assistant to compare us against the alternatives. Both answer with things we did not write.
Ask your coding agent
It reads your repo, not our pitch.
Paste one line into Claude Code, Cursor, or anything that reads your repository. It comes back with file paths, a credit estimate, and where we are not worth it.
The skill reads our live endpoint list when it runs, so it covers anything we ship after you paste this. Read it first.
Ask an assistant
Let it compare us to the alternatives.
The question is already written, and it asks for the comparison rather than the sales pitch. It opens in a new tab with the prompt filled in.
Assistants read the live site, so they answer from what is published today rather than from anything we hand them.
Getting started
Sign up at the dashboard and your key is on the first screen, with 250 free credits attached. No card required.
Yes, because keys and credits belong to a workspace. Sign up with an email address and password, or with Google. Signing in with Google is optional and uses your name and email address only, to create and secure your Hydrafetch account. We never read, send, or store anything else from your Google account.
Yes, for Node, Python, Go, Ruby, Rust and PHP, plus two browser packages for company logos. Each one wraps the same HTTP surface, so plain HTTP still works from any language in a few lines, and agents can skip HTTP entirely and connect over MCP.
Capabilities
Yes. Pages that only exist after the browser draws them come back as clean content, automatically, for the same one credit.
No. Blocks, challenges and rate limits are our problem. If a page truly cannot be delivered, the response says so and you are not billed.
Ask for it. Every scrape can return the page’s own structured data alongside Markdown, and the extract endpoint turns any page into typed JSON matching your schema, with per-field evidence.
Yes. Crawls run as asynchronous jobs with depth, path and subdomain controls, and a webhook tells you when they finish. Every page arrives in the same envelope.
Fresh enough for the job. Recently seen pages are served from a short-lived cache and refreshed when they go stale. If your use case has a stricter bar, tell the request how fresh is fresh enough and we take care of the rest. Every response states whether it came from cache.
Billing
A successful page costs one credit, whatever it took to get it. Search costs one plus one per scraped result, and extract costs five. There are no modifiers or surcharges to stack.
Never. Errors, blocks and dead ends cost nothing. You pay for delivered pages only.
No. What a page needed to come back clean is our cost, not your line item. One page, one credit.
Clean web data is one call away.
250 free credits, no card required. Failures never billed.
[ Pages processed ]
Domains reached
Searches run
Sites crawled