Overview

Relevant source files

• apps/coordinator/tsconfig.json
• apps/docker-provider/tsconfig.json
• apps/kubernetes-provider/src/labelHelper.ts
• apps/kubernetes-provider/tsconfig.json
• apps/webapp/app/entry.client.tsx
• apps/webapp/app/entry.server.tsx
• apps/webapp/app/env.server.ts
• apps/webapp/app/root.tsx
• apps/webapp/app/utils/httpErrors.ts
• apps/webapp/package.json
• apps/webapp/server.ts
• internal-packages/database/prisma/schema.prisma
• package.json
• packages/build/CHANGELOG.md
• packages/build/package.json
• packages/cli-v3/CHANGELOG.md
• packages/cli-v3/package.json
• packages/core/CHANGELOG.md
• packages/core/package.json
• packages/react-hooks/CHANGELOG.md
• packages/react-hooks/package.json
• packages/rsc/CHANGELOG.md
• packages/rsc/package.json
• packages/trigger-sdk/CHANGELOG.md
• packages/trigger-sdk/package.json
• patches/@[email protected]
• pnpm-lock.yaml
• turbo.json

Purpose and Scope

This document provides a high-level introduction to the Trigger.dev codebase, its purpose as a background jobs and task execution platform, and the overall system architecture. This page covers the platform's core components, package organization, and architectural patterns.

For detailed information about specific subsystems, see:

• Task execution mechanics: Task Execution Engine
• Database schemas and storage: Database and Data Storage
• Deployment process and CI/CD: Deployment Pipeline
• Developer-facing SDK and CLI: Trigger.dev SDK

Sources: README context from diagrams pnpm-lock.yaml1-10 package.json1-20

What is Trigger.dev?

Trigger.dev is a background jobs platform that enables developers to define, schedule, and execute long-running tasks with built-in reliability, observability, and scaling capabilities. The platform handles task orchestration, retry logic, distributed locking, concurrency management, and state persistence, allowing developers to focus on business logic.

The system operates as a multi-tenant SaaS platform (cloud.trigger.dev) or can be self-hosted. Tasks are defined using the @trigger.dev/sdk package in TypeScript, deployed as Docker containers, and executed in isolated worker environments managed by the platform.

Core Value Proposition:

• Define tasks using task() function with TypeScript
• Automatic retries, timeouts, and error handling
• Checkpoint and resume for long-running workflows
• Real-time monitoring and debugging via web dashboard
• Queue management and concurrency controls

Sources: packages/trigger-sdk/package.json1-10 apps/webapp/package.json1-10

Platform Architecture Overview

Architecture: Trigger.dev follows a layered monorepo architecture managed by pnpm workspaces and Turborepo. The codebase separates concerns into three primary tiers:

1. Developer Environment: SDK and CLI packages that developers install and use to define/deploy tasks
2. Platform Core: The apps/webapp Remix application serving both API and web dashboard
3. Execution Layer: Coordinator, Supervisor, and Run Engine orchestrating task execution

Sources: package.json4-8 turbo.json1-20 apps/webapp/app/root.tsx1-30 apps/webapp/server.ts1-20

Core Components

Web Application (apps/webapp)

The central nerve center of the platform, built with Remix 2.1.0 and Express 4.20.0.

The webapp serves dual roles:

• API Server: REST endpoints for triggering tasks, retrieving run status, managing deployments
• Web Dashboard: Real-time monitoring UI with WebSocket connections for live updates

Key Files:

• apps/webapp/server.ts1-200 - Express server setup with Socket.IO and clustering
• apps/webapp/app/entry.server.tsx1-100 - Server-side rendering configuration
• apps/webapp/app/root.tsx1-133 - Root Remix route and app shell

Sources: apps/webapp/package.json1-50 apps/webapp/server.ts1-200 apps/webapp/app/root.tsx1-133

Task Execution Services

Coordinator (apps/coordinator): Lightweight service that routes task execution requests to appropriate supervisors. Uses Socket.IO for real-time communication with the platform core.

Supervisor (apps/supervisor): Manages the lifecycle of worker containers/pods. Supports both Kubernetes (via @kubernetes/client-node) and Docker (via dockerode) as compute providers. Pulls Docker images from ECR registries and instantiates worker environments.

Run Engine 2.0 (internal-packages/run-engine): The core task execution orchestrator introduced in v4. Features:

• Worker pool with configurable concurrency (default: 4 workers × 10 concurrent tasks = 40 total)
• Queue sharding (default: 4 shards) to reduce contention
• Distributed locking via dedicated Redis instance to prevent duplicate execution
• Sophisticated queue selection algorithm with bias factors

Configuration (from apps/webapp/app/env.server.ts560-583):

Sources: apps/coordinator/tsconfig.json1-20 apps/supervisor/tsconfig.json1-20 apps/webapp/app/env.server.ts560-600

Developer-Facing Packages

@trigger.dev/sdk (packages/trigger-sdk): The primary developer interface. Exports:

• task() - Define tasks with retry, queue, and lifecycle options
• schemaTask() - Type-safe tasks with Zod validation
• trigger(), batchTrigger() - Programmatically trigger task execution
• wait, metadata, usage - Runtime utilities

trigger.dev (packages/cli-v3): Command-line interface for:

• dev - Local development server with hot reload
• deploy - Build and deploy tasks to production
• login, logout, whoami - Authentication
• @trigger.dev/cli-v3 as internal package name, published as trigger.dev

@trigger.dev/core (packages/core): Shared code used across SDK, CLI, and platform. Contains Zod schemas, OpenTelemetry configuration, IPC protocols, and runtime utilities.

Sources: packages/trigger-sdk/package.json1-50 packages/cli-v3/package.json1-50 packages/core/package.json1-50

Storage Architecture

Trigger.dev uses a three-tier storage strategy optimized for different data access patterns:

PostgreSQL (via Prisma ORM at internal-packages/database/prisma/schema.prisma1-100):

• Primary source of truth for all entities
• Three connection types: main (writes), direct (bypass poolers), read replicas
• Connection configuration: apps/webapp/app/env.server.ts52-67

ClickHouse (via internal-packages/clickhouse):

• Stores task execution events and spans for analytics
• Optimized for time-series queries and aggregations
• Materialized views for real-time usage metrics

Redis Cluster (9 instances, configured in apps/webapp/app/env.server.ts119-472):

1. Main Redis - Default fallback
2. Rate Limit Redis - API request throttling
3. Cache Redis - Query result caching
4. Realtime Streams Redis - Server-sent events for dashboard
5. PubSub Redis - WebSocket event broadcasting
6. Alert Rate Limiter Redis - Alert throttling
7. Worker Coordination Redis - Run Engine workers
8. Run Queue Redis - Task queue with sharding
9. Run Lock Redis - Distributed locks for duplicate prevention

Sources: internal-packages/database/prisma/schema.prisma1-100 apps/webapp/app/env.server.ts52-250 internal-packages/clickhouse/package.json1-20

Package Structure and Organization

The monorepo is organized using pnpm workspaces with three workspace categories defined in package.json4-8:

Workspace Categories:

1. apps/* - Deployable applications

   • Not published to npm
   • Built and deployed as Docker containers or Node.js services

2. packages/* - Published to npm registry

   • Versioned together using Changesets
   • Consumable by external developers
   • Current version: 4.3.2 (synchronized across all packages)

3. internal-packages/* - Shared internal code

   • Not published to npm
   • Used only within the monorepo
   • Enable code reuse across apps

Build Orchestration: Turborepo (turbo.json1-50) manages build pipelines with dependency-aware caching:

Sources: package.json4-10 turbo.json1-50 pnpm-lock.yaml45-107

Deployment Model

Deployment Flow:

1. Developer runs trigger.dev deploy (packages/cli-v3/package.json34)
2. CLI builds Docker image using one of three strategies:
   • Depot (default) - Remote cloud builds, faster, no local Docker required
   • Local (--local-build) - Uses Docker Buildx on developer machine
   • Native (--native-build-server) - Backend-orchestrated builds with S2 log streaming
3. Image pushed to container registry (ECR or custom)
4. Platform indexes tasks from image and creates WorkerDeployment record
5. Supervisor spawns worker containers/pods when tasks are triggered
6. Workers execute tasks in isolated environments

Registry Configuration (apps/webapp/app/env.server.ts294-333):

• Primary registry for v3: DEPLOY_REGISTRY_HOST
• V4 registry (with fallback): V4_DEPLOY_REGISTRY_HOST
• ECR support with STS AssumeRole for cross-account access
• Lifecycle policies to expire untagged images

Sources: packages/cli-v3/package.json1-50 apps/webapp/app/env.server.ts294-350 apps/kubernetes-provider/tsconfig.json1-20 apps/docker-provider/tsconfig.json1-20

Key Architectural Decisions

Multi-Tier Storage Strategy

Decision: Use PostgreSQL for transactional data, ClickHouse for analytics, and 9 specialized Redis instances for different caching/coordination patterns.

Rationale: Each storage system is optimized for its use case:

• PostgreSQL provides ACID guarantees for critical state
• ClickHouse provides fast time-series aggregations for usage analytics
• Redis instances are isolated to prevent resource contention between different concerns (rate limiting vs. caching vs. locking)

Implementation: apps/webapp/app/env.server.ts119-472

Run Engine 2.0 Architecture

Decision: Replace previous job queue system with custom Run Engine featuring queue sharding, worker pools, and sophisticated queue selection.

Rationale:

• Reduce lock contention with queue sharding (4 shards)
• Enable predictable scaling with fixed worker pool size
• Implement intelligent queue selection using bias factors (concurrency: 0.75, capacity: 0.3, age: 0.25)
• Prevent duplicate execution with distributed locking

Implementation: apps/webapp/app/env.server.ts560-583

Monorepo with Workspace Separation

Decision: Use pnpm workspaces with clear boundaries between published packages, internal packages, and applications.

Rationale:

• Published packages can be versioned independently while sharing code
• Internal packages enable code reuse without polluting public API surface
• Turborepo provides fast, cached builds across the entire monorepo

Implementation: package.json4-8 turbo.json1-50

Remix + Express Dual-Purpose Server

Decision: Build webapp as Remix application hosted by Express server, serving both API and dashboard.

Rationale:

• Single deployment reduces operational complexity
• Remix provides excellent DX for building interactive dashboards
• Express enables custom middleware (rate limiting, Socket.IO, clustering)
• Server-side rendering improves initial page load

Implementation: apps/webapp/server.ts1-200 apps/webapp/app/root.tsx1-133

Sources: apps/webapp/package.json1-50 apps/webapp/server.ts1-200 apps/webapp/app/env.server.ts1-100

---

This overview provides the foundation for understanding Trigger.dev's architecture. For detailed information on specific subsystems, refer to the linked sections: SDK and CLI (#3), Task Execution (#4), Web Application (#5), Database (#6), and Deployment (#7).