Pinned
You can now use Spider in @llama_index as a web reader! Crawl/scrape urls and format the HTML into LLM ready markdown!
Spider is the fastest web crawler built for AI Agents and LLMs.
h/t @WilliamEspegren for the PR
Spider Cloud 🕷️
@spider_rust
A web crawler and scraper for the community
Joined September 2023
- You can now use Spider without an API key to sample real websites before creating an account. Our /scrape endpoint supports both static and JavaScript-rendered pages, returning clean markdown with a single request. Give it a try. Some limitations apply.
- Julep AI: Agent-based chat and task automation
- congrats on the raise!Excited to share that @crewAIInc raised $18 million in funding, with our series A led by @insightpartners, with @Boldstartvc leading our inception round. We're also thrilled to welcome @BlitzVentures, Earl Grey Capital(@amitvasudev_), and top AI leaders like @AndrewYNg and
