Scrapfly CLI

View as markdown

The Scrapfly CLI is the official command-line interface for the Scrapfly platform. It exposes every Scrapfly product as a single binary with stable JSON output - ready for shell scripts, CI pipelines, and LLM agent frameworks.

30-Second Install

The fastest way to get started:

curl -fsSL https://scrapfly.io/scrapfly-cli/install | sh

Then store your API key:

scrapfly config set-key scp-live-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

What You Can Do

Scrape any URL

Anti-bot bypass, JS rendering, and proxy rotation built in. One flag: --unblocker.

Extract structured data

Describe what you want in plain English. Get clean JSON back.

Run AI agents

scrapfly agent "<task>" plans and executes multi-step scraping tasks autonomously.

Batch processing

Feed a file of URLs, get a stream of results.

Stable Output Envelope

Every command returns the same top-level JSON structure:

{
  "success": true,
  "product": "scrape",
  "data": "..."
}

On failure the envelope is written to stderr and stdout stays empty, so scrapfly scrape ... > out.json leaves an empty file when the run fails. The failure envelope carries no product field:

{
  "success": false,
  "error": {
    "code": "ERR::SCRAPE::UPSTREAM_WEBSITE_ERROR",
    "message": "..."
  }
}

Batch runs are the exception: scrapfly scrape batch, and scrapfly scrape with more than one URL (or --from-file), stream one envelope per line on stdout whether the item succeeded or not.

No exceptions thrown to stdout. Always parseable.

Where to Go Next

Installation - all install methods and self-update
Authentication - API key setup
Commands - full verb reference
Agent Mode - autonomous task execution
GitHub Actions - using the CLI in CI
MCP - exposing Scrapfly as an MCP server
FAQ - common questions

Source code and pre-compiled releases are on GitHub.