Dashboard - Spider

quickstart

Start building with Spider

Create an API key

Export the key

export SPIDER_API_KEY="sk-..."

SDKs, the MCP server and every command read it.

Connect Claude Code

Client configs in the docs

claude mcp add spider --transport http \
  https://mcp.spider.cloud/mcp \
  -H "Authorization: Bearer $SPIDER_API_KEY"

CLAUDE.md

For web pages use spider-agent. Read https://spider.cloud/agent-skill/spider-agent/SKILL.md first.

Or the CLI. It picks HTTP or a browser per page, stops at your budget, and returns 187 bytes for named fields where asking for everything returns 9021.

Connect Codex

Every command in the docs

Read and follow https://spider.cloud/agent-skill/SKILL.md

Paste into a Codex session. The skill covers the whole API.

AGENTS.md

For web pages use spider-agent. Read https://spider.cloud/agent-skill/spider-agent/SKILL.md first.

Or the CLI. It picks HTTP or a browser per page, stops at your budget, and returns 187 bytes for named fields where asking for everything returns 9021.

Install the CLI and run it

Every command in the docs

curl -fsSL https://spider.cloud/install/spider-agent.sh | sh

Downloads the release build, checks its SHA256, and puts it in ~/.local/bin. With a Rust toolchain, cargo install spider-agent-cli works too.

spider-agent scrape https://example.com --goal markdown --json

Picks HTTP or a browser per page, escalates on a refusal, stops at the budget you set.

Connect your MCP client

Client configs in the docs

Endpoint:  https://mcp.spider.cloud/mcp
Transport: Streamable HTTP
Auth:      Authorization: Bearer sk-...

Works in Claude Desktop, Cursor, Windsurf, or any MCP client.

Make the first request

Full example in the docs

curl -X POST https://api.spider.cloud/scrape \
  -H "Authorization: Bearer $SPIDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "return_format": "markdown"}'

Install the SDK and run it

Full example in the docs

pip install spider_client

from spider import Spider
app = Spider()  # reads SPIDER_API_KEY
page = app.scrape_url("https://example.com")

Install the SDK and run it

Full example in the docs

npm install @spider-cloud/spider-client

import { Spider } from "@spider-cloud/spider-client";
const app = new Spider(); // reads SPIDER_API_KEY
const page = await app.scrapeUrl("https://example.com");

overview

credits

requests · 24h

view logs →

unlimited

request statusloading

Loading request data

recent activity

No crawls yet.

open playground →