Skip to content

avikcodes/benchmark

Repository files navigation

Benchmark — Brutally Honest AutoML for Real-World Data

“Most ML tools fail silently. This one refuses to.”


🧠 What This Is

Benchmark is an AutoML system designed for one purpose:

Take messy, real-world datasets → produce valid ML results without crashing.

No assumptions.
No blind trust in user input.
No silent failures.


⚠️ The Problem (Why This Exists)

Most ML tools assume:

  • Clean datasets
  • Correct target selection
  • Proper task type (classification/regression)

Reality:

Issue What Happens
Wrong target column Model crashes
Time column as target Type errors
Missing values Silent data loss
Mixed formats Garbage output
Small datasets Fake accuracy

Result → Unreliable models + confused users


🔥 Core Philosophy

Fail early. Fail clearly. Never lie.


🏗️ System Architecture

        ┌────────────────────┐
        │   User Upload CSV  │
        └─────────┬──────────┘
                  ↓
        ┌────────────────────┐
        │ Target Detection   │
        │ (Type Inference)   │
        └─────────┬──────────┘
                  ↓
        ┌────────────────────┐
        │ Guardrail System   │
        │ (Validation Layer) │
        └─────────┬──────────┘
                  ↓
        ┌────────────────────┐
        │ Data Cleaning      │
        │ (X + y Processing) │
        └─────────┬──────────┘
                  ↓
        ┌────────────────────┐
        │ Model Training     │
        │ (Sklearn Models)   │
        └─────────┬──────────┘
                  ↓
        ┌────────────────────┐
        │ Results + History  │
        └────────────────────┘

🛡️ Guardrail System (The Core Innovation)

This is what makes Benchmark different.

1. Target Detection

Type Action
Numeric Regression
Categorical (<20 unique) Classification
Time/Datetime Convert → Regression
High-cardinality text ❌ Reject

2. Auto Task Correction

User selects wrong task?

✔ System overrides it
✔ Logs warning
✔ Prevents crash


3. Data Health Check

Before training:

Metric Rule
Valid rows ≥ 10
Data retained ≥ 50%
Missing target Removed

If failed → hard stop


4. Time Handling (Critical Fix)

Time is NOT a valid ML target directly.

We convert:

Example:

Time Converted
21:30:32 77432
09:15:00 33300

5. Feature Cleaning

  • Drop empty columns
  • Drop constant columns
  • Encode categorical features
  • Extract datetime features:
Feature Example
Hour 21
Day 15
Month 4

📊 Data Flow Diagram

Raw Data ↓ Validation ↓ Cleaning ↓ Feature Engineering ↓ Model Training ↓ Evaluation


🧪 Example Failure Handling

❌ Bad Input

  • Target: Time
  • Mixed formats
  • 80% invalid rows

✅ System Response

Error: Too many invalid time values in target column. Suggestion: Select a different target column.


⚙️ Tech Stack

Frontend

  • Next.js 14
  • Tailwind CSS
  • shadcn/ui

Backend

  • FastAPI
  • WebSockets

ML

  • scikit-learn

🚀 How It Works (Execution Flow)

  1. Upload CSV
  2. Select target column
  3. System analyzes:
    • Data type
    • Distribution
    • Validity
  4. Guardrails activate
  5. Data cleaned
  6. Model trained
  7. Results stored in history

📈 Example Pipeline (Real Case)

Input:

100 rows 30 invalid target values

After Processing:

70 valid rows Model trained successfully


🧯 Failure Modes (Handled)

Scenario Outcome
Wrong target Auto-correct
Too many NaN Stop
Invalid time Convert or reject
Small dataset Stop
Text target Reject

📌 Why This Matters

Most tools optimize for:

“Looks good in demo”

Benchmark optimizes for:

“Works on real data”


🧠 Design Principles

  • No silent fixes
  • No fake success
  • No blind assumptions
  • Always explain failure

📊 Comparison

Feature Typical AutoML Benchmark
Handles dirty data
Prevents crashes
Explains errors
Auto-corrects task
Time handling

🔮 Future Improvements

  • Auto feature engineering
  • Model selection optimization
  • Dataset profiling UI
  • Explainable AI layer
  • Dataset recommendations

📂 Project Structure

benchmark/ ├── app/ ├── benchmark-api/ ├── public/ ├── dataclean/ ├── package.json └── README.md


🧑‍💻 Author

Avik Ghosh
Building real-world AI tools that don’t break under pressure.


💬 Final Thought

“Anyone can build a model.
Very few can build a system that survives bad data.”

Benchmark is built for the second category.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors