RabbitReview | Devpost

Inspiration:

Security vulnerabilities often slip into production because manual testing is slow and incomplete, and running untrusted code locally is risky. We wanted to automate security testing by combining AI-generated tests with safe, isolated execution environments. The goal: catch vulnerabilities like auth bypasses, hardcoded secrets, and unsafe redirects before they reach production.

What it does:

RabbitReview is an automated security testing pipeline that generates and runs tests in isolated sandboxes to verify vulnerabilities. When a pull request is opened, CodeRabbit generates tests covering edge cases and security vulnerabilities. Daytona sandboxes execute these tests safely, and results are posted back to the PR as comments. This enables teams to catch security issues early without manual effort or local risk.

How we built it:

Architecture: GitHub Actions Workflow: Orchestrates the pipeline, triggering on PR events CodeRabbit Integration: Configured via .coderabbit.yaml to generate pytest (Python) and vitest (TypeScript) tests Daytona SDK: Python script creates ephemeral sandboxes, clones PR branches, installs dependencies, and runs test suites Demo Application: FastAPI backend and Next.js frontend with intentional vulnerabilities for demonstration Sponsor Tools Integration: CodeRabbit: Triggered via GitHub comment (@coderabbitai generate unit tests), generates tests with security focus, configured for multi-language support Daytona: Python SDK creates isolated sandboxes with Python 3.11 and Node.js 20, executes tests safely, and automatically cleans up resources

Flow:

PR opened → GitHub Actions triggers CodeRabbit generates tests → Commits to PR branch Daytona creates sandbox → Clones repo → Runs tests Results posted → PR comment with test outcomes

Challenges we ran into:

Timing: CodeRabbit test generation is async; added a wait step and retry logic Test Discovery: Tests generated in different locations; implemented file detection and organization logic Sandbox Configuration: Ensuring both Python and Node.js environments work correctly; configured environment variables and dependency installation PR Comment Posting: GitHub API authentication and formatting; implemented proper error handling and markdown formatting

Accomplishments that we're proud of:

Seamless integration of CodeRabbit and Daytona Automated end-to-end pipeline from PR to test results Multi-language support (Python and TypeScript) Successfully detecting intentional vulnerabilities (auth bypass, hardcoded secrets, unsafe redirects) Clean sandbox lifecycle management with automatic cleanup

What we learned:

How to integrate AI code generation tools into CI/CD pipelines Sandbox execution patterns for secure test running GitHub Actions workflow orchestration and async operations Security testing best practices and vulnerability patterns The importance of isolated environments when running untrusted code

What's next for RabbitReview:

Support for more languages (Go, Rust, Java) Integration with more security scanning tools (SAST/DAST) Real-time test generation feedback in PRs Custom vulnerability detection rules Performance benchmarking and optimization Dashboard for tracking security metrics across repositories