Code Craft

Code Craft is an AI-Powered Web3 Protocol Analysis

Comment

Code Craft: AI-Powered Web3 Protocol Analysis

Inspiration

With millions of dollars in rewards distributed every month through platforms like Code4Arena, Sherlock, Cantina, CodeHawks, and Immunefi, Web3 security research has become a high-stakes, time-sensitive field. As security researchers ourselves, we realized that ~90% of bugs are rooted in the business logic of protocols - the very part that's often buried under scattered documentation and complex smart contracts.

We needed a tool that could cut through the noise - one that reads the code, understands the docs, and explains the business logic as clearly as a seasoned auditor. That’s what inspired me to build Code Craft: An AI-powered assistant that helps researchers save time, find bugs faster, and understand Web3 protocols with precision and clarity.

Without security researchers, Web3 will never be truly safe.

Over the past few years, billions have been lost to protocol exploits - despite massive growth in infrastructure, tooling, and developer adoption:

2021: ~$5.2 billion lost to hacks

2022: ~$3.6 billion lost

2023: ~$1.8 billion lost

2024: ~$2.3 billion lost

2025 (H1): ~$2.17 billion+ already lost

What It Does

CodeCraft helps researchers understand any Web3 protocol by:

  • Accepting a GitHub repository URL and documentation link
  • Parsing Solidity contracts to build ASTs, extract functions, modifiers, inheritance
  • Using NLP to extract economic models, governance structures, and business logic from documentation
  • Generating:
    • High-level protocol summary
    • Interactive Mermaid.js architecture diagrams
    • Detailed security insights with vulnerability prioritization
  • Allowing users to export analysis as JSON
  • Visualizing contract relationships inspired by SolidityVisualizer

How I Built It

Frontend

  • Clean black-and-white minimalist UI
  • Simple input interface:
    • Two fields for GitHub + Docs links
  • Dynamic result display:
    • Collapsible sections for each analysis component
    • JSON export of full analysis
  • Mermaid.js integration for interactive diagram rendering

Backend

  • Git-based code parsing using Octokit
  • AST construction via regex-based Solidity extraction
  • NLP-powered doc processing with Cheerio and LLMAdapter
  • Mermaid.js diagram generation:
    • generateDataFlow()
    • generateInteractionDiagram()
    • generateInheritanceDiagram()
  • Security pattern detection in analyzeSecurityAnalysis()
  • Complexity scoring in calculateContractComplexity()

Visualization

CodeCraft integrates interactive and exportable visualizations to help users better understand smart contract architectures, data flows, security patterns, and protocol logic.

The visualizations are inspired by tools like SolidityVisualizer , using Mermaid.js for diagram generation.

  1. Contract Interaction diagram
  2. Inheritance diagram
  3. Data flow diagram

Database

  • Store processed data in Supabase
  • Enable future RAG-style query enhancements

Challenges I Ran Into

  1. Parsing messy Solidity codebases
    • Regex-based extraction struggles with complex syntax
    • Multi-line functions and inline comments caused issues
  2. Handling non-uniform documentation formats
    • Markdown, HTML, and JSON required different parsing strategies
    • Some docs had corrupted formatting or missing content
  3. Combining static analysis and NLP results
    • Ensuring consistency between code and doc claims
    • Avoiding hallucinations in merged output
  4. Generating understandable diagrams
    • Complex inheritance trees became too dense
    • Function call graphs needed simplification
  5. Ensuring cross-device compatibility
    • Responsive design required extra CSS work
    • Mermaid.js rendering varied across browsers

Accomplishments I'm Proud Of

  1. End-to-end pipeline from GitHub/docs → analysis
    • Fully functional AI tool that mimics top-tier auditors
  2. One-click protocol understanding
    • Instant insights into:
      • Summary
      • Architecture
      • Security flags
      • Economic model
  3. Clean, expandable UI
    • Expandable details
    • Exportable reports
    • Diagram visualization
  4. Seamless code-doc synthesis
    • Matches function behavior with documentation claims
    • Detects mismatches between code and docs

What I Learned

  1. LLMs are powerful but need guidance
    • Prompt engineering is critical for accurate results
    • LLaMA 3.3-70B excels at explaining complex Web3 logic when given structured inputs
  2. Visualization matters
    • Mermaid.js makes complex architectures digestible
    • Diagram annotations improve comprehension
  3. Robustness is essential
    • Real-world repos and docs are messy
    • Tools must handle edge cases gracefully
  4. Modular design wins
    • Clean separation of concerns:
      • DocumentProcessor
      • AnalysisEngine
      • LLMAdapter
    • Makes future improvements easier

What's next for Code Craft

  • Multi-language support: (e.g., Rust, Cairo, Vyper, Move)
  • Creating a RAG flow process to get accurate outputs - By uploaing a dataset into supabase
  • Plugin system: For custom security checks
  • Team dashboards: For collaborative auditing
  • API integration: for audit firms and bounty platforms
  • On-chain deployment risk analysis with live data sources

Built With

  • bolt
  • chatgpt
  • llama-3.3-70b
  • supabase
Share this project:

Updates