🧬 Onco

Cursor for Oncology
An AI-powered IDE where patient biology becomes the codebase

Features • Quick Start • Architecture • How It Works • Tech Stack

Oncologists analyze 1.9 million new cancer cases each year in the US alone. Clinical decision-making spans radiology, pathology, and genomics—yet these data live in disconnected silos. Clinicians flip between PACS viewers, pathology reports, and variant tables, manually synthesizing context that should flow automatically.

Software engineers have Cursor: an IDE that understands the whole codebase.
Oncologists need the same.

Onco is an agentic development environment where the "codebase" is the patient's biological data—radiology, pathology, and genomics. It replaces static medical records with an active, context-aware workspace that helps clinicians debug cancer cases in real time.

• DICOM CT Viewer with multi-planar reconstruction (Axial, Sagittal, Coronal)
• Cmd+Click Lesion Segmentation using MedSAM2 (Medical Segment Anything Model)
• RECIST 1.1 Assessment with automated diameter measurements and response categorization
• Tumor Classification via radiomics feature extraction (malignant vs benign)
• Real-time contour visualization with measurement overlays

• Whole Slide Image Viewer with zoom/pan controls
• AI Text Explanation powered by Gemini 2.5 Flash
• Highlight any text → get expert-level clinical interpretation
• Morphology-to-Driver Mapping suggesting likely molecular alterations
• Cross-references NCCN, ASCO, CAP guidelines

• NGS Panel Display with VAF, classification, actionability
• CIViC Evidence Lookup for variant interpretation
• Therapy Recommendations based on curated clinical evidence
• Clinical Trial Matching from ClinicalTrials.gov
• 3D Protein Structure Viewer (NGL) showing mutation sites

• Shared Patient State persisting across all modalities
• Terminal Log Streaming showing AI reasoning in real-time
• Structured Output Cards with actionable insights
• Live-First with Cached Fallback ensuring demo reliability

• Node.js 18+
• Python 3.11+
• Docker (optional, for containerized deployment)

git clone https://github.com/yourusername/onco.git
cd onco

cd backend

# Create virtual environment
python -m venv .venv
source .venv/bin/activate  # or `.venv\Scripts\activate` on Windows

# Install dependencies
pip install -r requirements.txt

# Set environment variables
cp .env.example .env
# Edit .env with your API keys:
# - GOOGLE_API_KEY (for Gemini)
# - MODAL_ENDPOINT_URL (for MedSAM2 inference)

# Run the server
uvicorn app.main:app --reload --port 8002

cd frontend

# Install dependencies
npm install

# Set environment variables
echo "NEXT_PUBLIC_API_BASE_URL=http://localhost:8002" > .env.local

# Run the development server
npm run dev

Navigate to http://localhost:3000

┌─────────────────────────────────────────────────────────────────┐
│                    Frontend (Next.js)                           │
│  ┌─────────────┬─────────────────────┬─────────────────────┐   │
│  │  Left Pane  │    Center Pane      │    Right Pane       │   │
│  │  Repository │    Active Viewer    │    Agent Terminal   │   │
│  │             │                     │                     │   │
│  │  • CT Scan  │  • DICOM Viewer     │  • Log Stream       │   │
│  │  • Pathology│  • WSI Viewer       │  • Structured Out   │   │
│  │  • Genomics │  • Variant Table    │  • Chat Interface   │   │
│  └─────────────┴─────────────────────┴─────────────────────┘   │
└─────────────────────────────┬───────────────────────────────────┘
                              │ REST + SSE
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│                    Backend (FastAPI)                            │
│  ┌──────────────────────────────────────────────────────────┐  │
│  │                   Copilot Handlers                        │  │
│  │  ┌────────────┐  ┌────────────┐  ┌────────────────────┐  │  │
│  │  │ Radiology  │  │ Pathology  │  │     Genomics       │  │  │
│  │  │ • MedSAM2  │  │ • Gemini   │  │ • CIViC API        │  │  │
│  │  │ • RECIST   │  │ • Mapping  │  │ • Trial Matching   │  │  │
│  │  │ • Radiomics│  │ • Extract  │  │ • Evidence Parse   │  │  │
│  │  └────────────┘  └────────────┘  └────────────────────┘  │  │
│  └──────────────────────────────────────────────────────────┘  │
│  ┌──────────────────────────────────────────────────────────┐  │
│  │  Core Services: Session Manager • Cache • SSE Emitter    │  │
│  └──────────────────────────────────────────────────────────┘  │
└─────────────────────────────┬───────────────────────────────────┘
                              │
              ┌───────────────┼───────────────┐
              ▼               ▼               ▼
        ┌──────────┐   ┌──────────┐   ┌──────────┐
        │  Modal   │   │  Gemini  │   │  CIViC   │
        │ MedSAM2  │   │   API    │   │   API    │
        └──────────┘   └──────────┘   └──────────┘

1. User Cmd+Clicks on a lesion in the CT viewer
2. Click coordinates (x, y, z) sent to backend
3. 2D slice extracted and sent to MedSAM2 on Modal
4. Segmentation mask returned and visualized
5. Measurements computed (longest diameter)
6. RECIST 1.1 rules applied for response assessment
7. Optional: Radiomics extraction → Tumor classification

1. User highlights text in pathology report
2. Clicks "Explain with Agent"
3. Gemini 2.5 Flash receives context + selected text
4. AI streams thinking process ("Analyzing query... Verifying with guidelines...")
5. Expert-level explanation returned with clinical implications

1. User clicks variant row (e.g., EGFR L858R)
2. CIViC API queried (live-first, cached fallback)
3. Evidence parsed for actionability
4. Therapy recommendations extracted
5. Clinical trials matched and returned
6. 3D structure loaded from RCSB PDB (if available)

onco/
├── frontend/                 # Next.js application
│   ├── src/
│   │   ├── app/             # App router pages
│   │   ├── components/      # React components
│   │   │   ├── layout/      # Three-pane layout
│   │   │   └── viewers/     # Radiology, Pathology, Genomics
│   │   ├── hooks/           # Custom React hooks
│   │   ├── lib/             # API clients, utilities
│   │   └── types/           # TypeScript definitions
│   └── tailwind.config.ts   # IDE-style theme
│
├── backend/                  # FastAPI application
│   ├── app/
│   │   ├── api/routes/      # API endpoints
│   │   ├── copilots/        # AI copilot handlers
│   │   │   ├── radiology/   # MedSAM2, RECIST, radiomics
│   │   │   ├── pathology/   # Gemini chat, extraction
│   │   │   └── genomics/    # CIViC, trials, evidence
│   │   ├── core/            # Session, cache, config
│   │   ├── models/          # Pydantic schemas
│   │   └── services/        # External API clients
│   └── requirements.txt
│
├── assets/                   # Demo data
│   ├── dicom/               # CT scan series
│   ├── pathology/           # Slide images
│   └── cache/               # Precomputed fallbacks
│
├── inference/               # Modal MedSAM2 deployment
│   ├── modal_medsam2.py     # Serverless inference
│   └── model/               # Model weights
│
└── docs/                    # Documentation

Backend (.env)

# Google AI (for Gemini)
GOOGLE_API_KEY=your_gemini_api_key

# Modal (for MedSAM2)
MODAL_ENDPOINT_URL=https://your-modal-endpoint.modal.run

# Optional
DEMO_MODE=false
RADIOLOGY_INFERENCE_TIMEOUT_SECONDS=30
CIVIC_API_TIMEOUT_SECONDS=10

Frontend (.env.local)

NEXT_PUBLIC_API_BASE_URL=http://localhost:8002

For reliable demonstrations, Onco supports a live-first with cached fallback architecture:

• Live inference is attempted for all AI operations
• If live fails (timeout, error), cached results are used
• Terminal logs explicitly indicate when fallback is used
• Demo patient data included in assets/ directory

To run in demo mode:

DEMO_MODE=true uvicorn app.main:app --port 8002

Contributions are welcome! Please read our contributing guidelines before submitting PRs.

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit changes (git commit -m 'Add amazing feature')
4. Push to branch (git push origin feature/amazing-feature)
5. Open a Pull Request

This project is licensed under the MIT License - see the LICENSE file for details.

• MedSAM2 - Medical image segmentation foundation model
• CIViC - Clinical Interpretation of Variants in Cancer
• RCSB PDB - Protein Data Bank for 3D structures
• Google Gemini - Large language model for text interpretation
• Cornerstone.js - Medical imaging in the browser
• NGL Viewer - Molecular visualization

Built in 24 hours at NexHacks 2026.