SmartTap

SmartTap turns plain-English questions about Oregon agricultural and weather data into charts, summaries, and inspectable evidence.

The retained product surface is one shared pipeline used by the Streamlit UI and CLI:

parse -> validate -> fetch -> visualize -> explain

SmartTap is an evidence system, not a final-answer engine. Its job is to return the right data, chart, and metadata package so the next reviewer can verify what the query actually shows.

• Time-series evidence views
• Statistical summaries
• Crop ranking and crop-distribution summaries
• Coordinated OpenET + AgriMet evidence packages
• Deterministic confirmation, clarification, validation, and explanation flows

Data sources:

• OpenET field and crop data from local parquet runtime artifacts
• AgriMet weather data from local CSVs, with optional API fallback for unsupported local variables

• core/, llm/, smarttap_service.py, smarttap_ui.py, smarttap.py: runtime code
• reference/: tracked small reference assets used by code and handoff docs
• data/: local-only heavy datasets and acquisition notes
• artifacts/qa/: tracked workbook QA bundle
• artifacts/examples/partner_queries/: tracked sample evidence runs
• docs/: current docs for onboarding, architecture, and handoff
• docs/archive/: historical planning and legacy reference docs
• scripts/: active utilities
• scripts/archive/: legacy utilities kept for reference only
• tests/: retained automated test suite

More detail is in docs/REPO_HANDOFF.md.

1. Create a virtual environment and install dependencies.

python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

2. Make sure Ollama is available for natural-language parsing.

ollama pull gemma3:latest
ollama serve

3. Download the required local data into data/.

See data/README.md.

4. Run the legacy Streamlit UI.

./run_ui.sh

The UI starts at http://localhost:8501.

SmartTap now also includes a React frontend backed by a FastAPI service.

Start the API:

./run_api.sh

The API starts at http://localhost:8000.

Start the React app in a second terminal:

./run_web.sh

The React app starts at http://localhost:5173 and proxies /api/* requests to the local FastAPI backend.

Available API routes:

• GET /api/health
• POST /api/query
• POST /api/followup
• POST /api/confirmation/edit
• POST /api/confirmation/confirm

python smarttap.py "Show temperature in Corvallis for July 2024"

Tracked reference assets now live in reference/:

• reference/agrimet_stations_full_metadata.csv
• reference/CDL_Crop_Codes_Oregon.csv
• reference/openet_variable_keywords.json
• reference/crop_name_keywords.json

Required local data lives in data/:

• data/agrimet/*.csv
• data/openet/field_index.parquet
• data/openet/annual/*.parquet
• data/openet/monthly/*/*.parquet

Optional:

• AGRIMET_USE_API=1 to prefer the AgriMet API when local CSV coverage is insufficient
• data/field_points.gpkg and data/preliminary_or_field_geopackage.gpkg as offline source files for parquet materialization
• data/openet/field_combined_long.csv and data/openet/huc_combined_long.csv only for legacy explicit non-location field/HUC fetch modes

The current statewide location-query path is parquet-backed.

1. Acquire the source GeoPackages:
   • data/field_points.gpkg
   • data/preliminary_or_field_geopackage.gpkg
2. Materialize the runtime parquet store:

python scripts/materialize_openet_parquet.py

Runtime OpenET queries then read only from data/openet/. The GeoPackages are offline source inputs for materialization and are not used by the active location-query runtime.

Run the retained suite with:

python -m pytest -q

The lightweight wrapper below runs the same suite:

python tests/run_tests.py

Frontend tests and build:

cd web
npm test
npm run build

Testing/reporting helpers:

python scripts/run_regression_suite.py --output evaluation_results/regression_suite_latest
python scripts/export_qa_bundle.py --output-dir artifacts/qa --clean
python scripts/evaluate_prompts.py --output evaluation_results/prompt_benchmark_latest --prompt-dir prompts/variants --repeats 3
python scripts/build_testing_report.py --regression-report evaluation_results/regression_suite_latest --qa-run-summary artifacts/qa/run_summary.json --workbook-eval evaluation_results/workbook_evaluation_20260415_071203.json --output evaluation_results/testing_story_latest

Notes:

• scripts/evaluate_prompts.py benchmarks prompts/interpretation.txt and prompt variants against retained, stretch, and adversarial parser fixtures.
• Prompt benchmarking needs a reachable local Ollama server at OLLAMA_HOST and the configured model, such as gemma3:latest.

• docs/REPO_HANDOFF.md
• docs/ARCHITECTURE_CONTRACTS.md
• docs/QUERY_EVIDENCE_OUTPUT_SPEC.md