Skip to content

Data pipeline for ingestion, extraction, databasing of, and predictive analysis from Auckland Transport (AT) public transport data

Notifications You must be signed in to change notification settings

Al-Bee/AT-Oracle

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

27 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

AT Oracle

AT Oracle is a two-stage project involving:

  • the creation of a data pipeline to ingest, process, clean, and store historical public transport usage (via Auckland Transport) and weather data

  • implementation of machine learning models to predict future AT public transport usage based on historical data and weather factors

Features:

CURRENT:

  • Periodic checking of AT's website for new usage data
  • Automated collection of xlsx download links
  • Idempotent downloading and processing of xlsx sheets
  • Cleaning and safe formatting of data for populating SQL database

PLANNED:

  • Weather data pipeline
  • Automted addition of new information to database
  • Training and testing of ML models for prediction

Installation:

# Create virtual environment (optional but recommended)
python -m venv .venv
source .venv/bin/activate # Linux / macOS
.venv\Scripts\activate # Windows

# Clone project repository
git clone https://github.com/Al-Bee/AT-Oracle.git
cd AT-Oracle

# Install dependencies
pip install -r requirements.txt

Usage:

TO COME...

About

Data pipeline for ingestion, extraction, databasing of, and predictive analysis from Auckland Transport (AT) public transport data

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages