🎙️ EfficientAI

Test your Voice AI Agents before Production

Open-source evaluation platform for conversational AI.

Test quality, measure performance, & ship with confidence.

📅 Book a Demo • 💻 GitHub

• ✅ Voice AI Evaluation: Test your voice AI agents with comprehensive evaluation metrics
• ✅ Persona Creation: Design diverse voice personas with unique characteristics and behaviors
• ✅ Scenario Building: Create comprehensive conversation flows and dialogue trees
• ✅ Automated Testing: Execute tests automatically across all your voice agents
• ✅ Real-time Insights: Get real-time insights on latency, accuracy, and quality metrics
• ✅ Batch Processing: Process multiple audio files and evaluations efficiently
• ✅ Modern Web Interface: Beautiful React-based UI for managing evaluations

Quick Navigation: Quick Start • CLI Commands • Development

There are two ways to run the application:

1. Start all services

   docker compose up -d

   This will automatically:

   • Build Docker images if they don't exist
   • Build the frontend during the Docker build process
   • Start all services (database, Redis, API, worker)
   • Run database migrations automatically on startup

   Note: If you make changes to the frontend or backend code, you may need to rebuild:

   # Rebuild and restart (forces rebuild even if image exists)
   docker compose up -d --build
   
   # Or rebuild without cache for a clean build
   docker compose build --no-cache api worker
   docker compose up -d

2. Configure your settings

   Edit config.yml and config.docker.yml with your settings (S3, API keys, etc.). See the Configuration section for details.

3. Create an API key

   docker compose exec api python scripts/create_api_key.py "My API Key"

4. Access the application

   • Frontend: http://localhost:8000/
   • API Docs: http://localhost:8000/docs

1. Install the package

   pip install -e .

2. Generate configuration file

   eai init-config

3. Edit config.yml with your database and Redis connection strings:

   database:
     url: "postgresql://efficientai:password@localhost:5432/efficientai"
   
   redis:
     url: "redis://localhost:6379/0"

4. Start the application and worker

   Option A: Start both together (Recommended)

   eai start-all --config config.yml

   This single command will:

   • Start the API server
   • Start the Celery worker (for background task processing)
   • Run database migrations automatically
   • Build the frontend (if needed)

   Press Ctrl+C to stop both services together.

   Option B: Start separately (for advanced use)

   In one terminal, start the application:

   eai start --config config.yml

   In another terminal, start the Celery worker:

   eai worker --config config.yml

   Or use the Celery command directly:

   celery -A app.workers.celery_app worker --loglevel=info

   The application will automatically:

   • Run database migrations (ensures schema is up to date)
   • Build the frontend (if needed)
   • Start the API server
   • Serve both API and frontend from the same server

   Important: Migrations run automatically before startup. If migrations fail, the app won't start.

   For development with hot reload:

   # Enable auto-rebuild of frontend on file changes
   eai start-all --config config.yml --watch-frontend

   This will:

   • Automatically rebuild the frontend when source files change
   • Keep the backend hot-reload enabled (by default)
   • Perfect for active frontend development

5. Access the application

   • Frontend: http://localhost:8000/
   • API Docs: http://localhost:8000/docs

For Docker Compose:

• Docker and Docker Compose installed

For CLI:

• Python 3.11+
• Node.js 18+ and npm
• PostgreSQL running (locally or remote)
• Redis running (locally or remote)

# Start both app and worker with default config.yml
eai start-all

# Start with custom config
eai start-all --config production.yml

# Start with frontend file watching (auto-rebuild on changes)
eai start-all --watch-frontend

# Start without building frontend (if already built)
eai start-all --no-build-frontend

# Start without auto-reload (production mode)
eai start-all --no-reload --no-build-frontend

# Customize worker log level
eai start-all --worker-loglevel debug

Note: This is the recommended way to run EfficientAI. It starts both the API server and Celery worker in a single command. Press Ctrl+C to stop both services.

# Start just the API server (worker must be started separately)
eai start --config config.yml

# Start with auto-reload for development
eai start --reload

# Start with frontend file watching
eai start --watch-frontend

# Start Celery worker with default config.yml
eai worker

# Start with custom config
eai worker --config production.yml

# Start with custom log level
eai worker --loglevel debug

# Or use Celery command directly
celery -A app.workers.celery_app worker --loglevel=info

Development Mode:

# Full development setup with both backend and frontend hot reload
eai start-all --watch-frontend --reload

# Start with default config.yml
eai start

# Start with custom config
eai start --config production.yml

# Start with frontend file watching (auto-rebuild on changes)
eai start --watch-frontend

# Start without building frontend (if already built)
eai start --no-build-frontend

# Start without auto-reload (production mode)
eai start --no-reload --no-build-frontend

Development Mode:

# Full development setup with both backend and frontend hot reload
eai start --watch-frontend --reload

# Start Celery worker with default config.yml
eai worker

# Start with custom config
eai worker --config production.yml

# Start with custom log level
eai worker --loglevel debug

# Or use Celery command directly (equivalent to eai worker)
celery -A app.workers.celery_app worker --loglevel=info

Note: The worker is required for processing background tasks (transcription, evaluation, etc.). If you use eai start-all, the worker starts automatically. Only use this command if you need to run the worker separately.

# Generate default config.yml
eai init-config

# Generate custom config file
eai init-config --output my-config.yml

# Run pending migrations manually
eai migrate

# Run migrations with verbose output
eai migrate --verbose

Note: Migrations run automatically on application startup. You only need to run them manually if you want to apply migrations before starting the server.

EfficientAI uses YAML configuration files for both CLI and Docker deployments. Generate a default config with:

eai init-config

# Application Settings
app:
  name: "EfficientAI Voice AI Evaluation Platform"
  version: "0.1.0"
  debug: true
  secret_key: "your-secret-key-here-change-in-production"

# Server Settings
server:
  host: "0.0.0.0"
  port: 8000

# Database Configuration
database:
  url: "postgresql://user:password@host:port/dbname"

# Redis Configuration
redis:
  url: "redis://host:port/db"

# Celery Configuration (for background tasks)
celery:
  broker_url: "redis://host:port/db"
  result_backend: "redis://host:port/db"

# File Storage
storage:
  upload_dir: "./uploads"
  max_file_size_mb: 500
  allowed_audio_formats:
    - "wav"
    - "mp3"
    - "flac"
    - "m4a"

# S3 Configuration (for data sources integration)
s3:
  enabled: false  # Set to true to enable S3 data sources
  bucket_name: "your-bucket-name"
  region: "us-east-1"
  access_key_id: "YOUR_ACCESS_KEY_ID"
  secret_access_key: "YOUR_SECRET_ACCESS_KEY"
  endpoint_url: null  # For S3-compatible services (MinIO, etc.)
  prefix: "audio/"  # Optional prefix for all objects

# CORS Settings
cors:
  origins:
    - "http://localhost:3000"
    - "http://localhost:8000"

# API Settings
api:
  prefix: "/api/v1"
  key_header: "X-API-Key"
  rate_limit_per_minute: 60

You can use a .env file to override Docker Compose defaults:

POSTGRES_USER=efficientai
POSTGRES_PASSWORD=password
POSTGRES_DB=efficientai
SECRET_KEY=your-secret-key-here

The application includes an automatic migration system that runs database schema changes on startup.

• Automatic Execution: Migrations run automatically when the application starts
• Version Tracking: Applied migrations are tracked in the schema_migrations table
• Idempotent: Each migration only runs once, even if the application restarts
• Ordered Execution: Migrations run in alphabetical order (use numbered prefixes like 001_, 002_, etc.)

Migrations are stored in the migrations/ directory. Each migration file should:

1. Have a numeric prefix: 001_description.py, 002_another.py, etc.
2. Include a description variable
3. Have an upgrade(db) function that takes a SQLAlchemy Session

Example migration:

"""
Migration: Add New Feature
"""

description = "Add new feature support"

def upgrade(db):
    """Apply this migration."""
    from sqlalchemy import text
    
    db.execute(text("CREATE TABLE IF NOT EXISTS new_table (...)"))
    db.commit()

Automatic (Recommended - Default Behavior):

• ✅ Migrations run automatically when you start the app with eai start
• ✅ Migrations also run automatically when the application starts (via lifespan handler)
• ✅ If migrations fail, the application will NOT start - this ensures database consistency
• ✅ API requests are blocked if migrations are pending
• ✅ When cloning from main, migrations will run automatically on first startup
• ✅ Each migration only runs once (tracked in schema_migrations table)

Manual:

# Run migrations manually
eai migrate

# With verbose output
eai migrate --verbose

Skip migrations (not recommended):

# Only use this if you know what you're doing
eai start --skip-migrations

1. Create a new file in migrations/ directory with the next sequential number
2. Follow the format shown above
3. Test the migration on a development database first
4. Use IF NOT EXISTS checks for idempotent operations

See migrations/README.md for detailed documentation.

Generate a visual Entity-Relationship (ER) diagram of your database schema to visualize table structures and relationships.

Install the required system and Python packages:

# Install system graphviz package
sudo apt-get update
sudo apt-get install -y graphviz libgraphviz-dev pkg-config

# Install Python packages
pip install eralchemy graphviz

Run the script to generate a PNG ER diagram:

python scripts/generate_er_diagram_simple.py

This will create schema_er_diagram.png in the project root directory, showing:

• All database tables
• Column names and types
• Primary keys
• Foreign key relationships
• Indexes

Note: The diagram is automatically generated from your current database schema, so make sure your database is running and migrations are up to date.

1. Start PostgreSQL and Redis

   docker compose up -d db redis

2. Run the application with hot reload

   # Backend auto-reload + Frontend auto-rebuild on file changes
   eai start --config config.yml --watch-frontend

   The --watch-frontend flag automatically rebuilds the frontend whenever you modify source files (.tsx, .ts, .css, etc.), so you don't need to manually rebuild after each change.

3. Run Celery worker (in separate terminal)

   celery -A app.workers.celery_app worker --loglevel=info

Option 1: Using CLI with watch mode (Recommended)

# From project root - automatically rebuilds on changes
eai start --watch-frontend

Option 2: Using Vite dev server (for instant hot module replacement)

cd frontend
npm install
npm run dev

This runs Vite dev server on http://localhost:3000 with instant hot module replacement. Note: You'll need to run the backend separately on port 8000.

Problem: After cloning the repository, you see errors like:

psycopg2.errors.UndefinedColumn: column "organization_id" of relation "api_keys" does not exist

Cause: The database schema is out of sync with the code. This happens when:

• The database was created before migrations were added
• Migrations failed to run on startup
• The database was created using an older version of the code

Solution:

1. Check migration status:

   python scripts/check_migrations.py

   This will show which migrations have been applied and identify any schema issues.

2. Run migrations manually:

   # Using CLI (recommended)
   eai migrate --verbose
   
   # Or using Python directly
   python -c "from app.core.migrations import run_migrations; run_migrations()"

3. If you're using a fresh database (just created/nuked):

   • The migration system now handles fresh databases correctly
   • If tables don't exist, migrations will skip them and init_db() will create them with the correct schema
   • However, if you see this error on a fresh DB, try:

     # Stop the application
     # Then run migrations explicitly
     eai migrate --verbose
     # Then start the application again
     eai start

4. If migrations still fail:

   • Ensure your database connection is correct in config.yml or .env
   • Check that you have the necessary permissions on the database
   • Review the migration logs for specific errors
   • You may need to manually add missing columns (see migration files in migrations/ directory)
   • For fresh databases: Make sure migrations run BEFORE any tables are created

5. For Docker setups:

   docker compose exec api eai migrate --verbose

   Important for Docker: If you nuked the DB container and created a new one:

   • The new container starts with an empty database
   • Migrations should run automatically on startup
   • If they don't, run them manually as shown above

Prevention: Always ensure migrations run successfully before using the application. Check the startup logs for migration status messages.

• 📧 Email: tejas@efficientai.cloud
• 📅 Book a Demo: Schedule a call
• 💬 LinkedIn: Connect with us
• 🐦 X (Twitter): Follow us
• 💻 GitHub: View on GitHub

MIT License - see LICENSE file for details