Visitran is an open-source data transformation platform designed to bridge the gap between high-level data engineering and accessible data analysis. It allows users to build robust data pipelines using Python-based models, a no-code visual interface, or natural language (AI). At its core, Visitran translates these various inputs into optimized SQL via the Ibis library and manages the execution lifecycle across major data warehouses.
VisitranModel base class visitran/templates/visitran_model.py21-23The following diagram illustrates how the primary subsystems interact, from the user interface down to the database execution layer.
Sources: README.md163-180 backend/core/routers/1-10 visitran/visitran.py1-50
Visitran is built on a modern, decoupled stack:
| Layer | Technologies |
|---|---|
| Frontend | React, Zustand (State), Monaco Editor, Socket.IO, Tailwind CSS |
| Backend | Django, Django REST Framework, Celery (Async Tasks), Redis (Broker/Cache) |
| Transformation Engine | Python 3.10, Ibis (SQL Generation), NetworkX (DAG) |
| Infrastructure | Docker, Docker Compose, Nginx |
Sources: README.md27-30 README.md81-89 pyproject.toml10-25
visitran/)The engine is a standalone Python package responsible for the "heavy lifting" of data modeling. It defines the VisitranModel visitran/templates/visitran_model.py21-23 which supports various materialization strategies: TABLE, VIEW, INCREMENTAL, and EPHEMERAL. It handles the connection to target databases through a standardized adapter layer.
For details, see Core Transformation Engine (visitran/).
Visitran provides an abstraction over Python code. The Config Parser backend/application/config_parser/parser.py15-20 takes YAML-based visual configurations and the Interpreter backend/application/interpreter/interpreter.py12-15 generates executable Python code using Jinja2 templates. The Chat AI integration allows users to generate these configurations using natural language prompts.
For details, see No-Code Model Builder and AI Assistant (Chat AI).
The Django backend manages the application state, including project metadata, database credentials (encrypted), and user authentication. It uses VisitranBackendContext backend/application/context.py35-40 to bridge the web requests to the underlying transformation engine.
For details, see Backend Application (Django).
The React-based frontend provides a professional development environment. It includes a file explorer, a lineage visualizer, and real-time execution logs streamed from the backend via socket-service.js frontend/src/service/socket-service.js5-10
For details, see Frontend Application (React).
This diagram maps high-level concepts to specific classes and files within the repository.
Sources: visitran/templates/visitran_model.py21-50 visitran/visitran.py27-60 visitran/adapters/base/adapter.py15-30 backend/application/context.py35-80
