Inspiration
🧠 Inspiration Modern teams rely on countless APIs and documentation tools Swagger, GitBook, Redoc, Notion, and more. Yet every platform structures data differently, making integration and discovery painfully slow. We built Docet to automate this: a unified connector engine that extracts, standardizes, and streams documentation and API data at high speed so developers can focus on building, not parsing.
⚙️ What it does Docet is a fast, intelligent connector framework that automatically:
Detects and connects to documentation sources (Swagger, Redoc, GitBook, etc.) Extracts structured information such as endpoints, parameters, or descriptions Transforms it into a clean, unified schema Streams the results into analytics tools, vector databases, or Fivetran pipelines In short — Docet turns messy documentation into machine-readable intelligence.
🏗️ How we built it Backend: Node.js (Express) for ultra-fast concurrent fetching and parsing Connector Logic: Modular “fetch and flatten” architecture that maps Swagger/OpenAPI and GitBook JSON to a standard Docet schema Performance: Uses asynchronous requests and incremental syncs for real-time updates Integration: Fivetran REST API connector to push structured data directly into warehouses Optional AI Layer: Transformer-based summarizer to auto-generate endpoint summaries and categorize endpoints ⚔️ Challenges we ran into Parsing inconsistencies between documentation formats (e.g., Swagger vs. Redoc) Handling large nested JSONs efficiently under rate limits Balancing speed with schema stability during incremental syncs Building a connector that works seamlessly with Fivetran’s REST framework while staying language-agnostic 🏅 Accomplishments that we're proud of Built a fully functional prototype capable of parsing and syncing live Swagger docs in under 2 seconds Created a unified schema that supports multiple documentation types Achieved near real-time syncing to Fivetran with minimal latency Designed the foundation for a “plug-and-play” connector platform new sources can be added with a few lines of config 📚 What we learned The importance of schema normalization when working across heterogeneous APIs How Fivetran connectors can be extended using REST endpoints instead of heavy SDKs Deep insights into OpenAPI structures and performance bottlenecks in parsing large documentation trees That even documentation often overlooked can become a powerful source of automation and intelligence 🚀 What’s next for Docet Add connectors for Notion, Postman Collections, and Confluence Launch a Docet Cloud Hub where users can manage, preview, and sync documentation sources visually Introduce AI enrichment (summaries, tagging, code generation)
