Skip to content

Add optional regressor predictor models and propagate their uncertainty - #2703

Merged
tcuongd merged 9 commits into
mainfrom
copilot/add-uncertainty-to-regressors
Aug 1, 2026
Merged

Add optional regressor predictor models and propagate their uncertainty#2703
tcuongd merged 9 commits into
mainfrom
copilot/add-uncertainty-to-regressors

Conversation

Copilot AI commented Dec 23, 2025

Copy link
Copy Markdown
Contributor
  • Run baseline tests (known logistic cross-validation failure pre-existing)
  • Implement opt-in regressor predictor support and logging
  • Use regressor predictors in predict and cross_validation (including uncertainty)
  • Add focused tests for regressor predictor behaviour
  • Run targeted tests to validate changes
  • Add predict/cross_validate comparisons and uncertainty/metric assertions
  • Add serialization test for regressor predictors
  • Add point accuracy bounds and cross_validation metric checks in diagnostics
Original prompt

At the moment, for time series model with exogenous regressors, cross_validate() and predict() assume you have a central estimate for these regressors at future time steps, and does not incorporate any uncertainty for the value of these estimates. To properly model this system inside prophet, we need a secondary model to predict the regressor(s), and we need to propagate this uncertainty through the system.

Of course, this is a fairly big change to the current system, so should be a completely optional, opt-in interface. We should do the simplest things by default, which means:

  • One predictor per exogenous regressor, which is also just a Prophet model
  • User can set parameters for these models when they do add_regressor(), otherwise the regressor models take the default Prophet constructor values.
  • At model.predict() time, if the regressors have models attached, prediction should happen using the regressors. This applies to both the maximum likelihood estimator (the easy case), and the MCMC case. Log when regressor models are running and their diagnostics, for visibility.
  • A similar thing should happen when running cross_validate(). Even though we have the actual values of the regressors in the test window, they should be estimated instead.

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 2 commits December 23, 2025 03:14
Co-authored-by: tcuongd <29808430+tcuongd@users.noreply.github.com>
Co-authored-by: tcuongd <29808430+tcuongd@users.noreply.github.com>
Copilot AI changed the title [WIP] Add support for uncertainty in exogenous regressors Add optional regressor predictor models and propagate their uncertainty Dec 23, 2025
Copilot AI requested a review from tcuongd December 23, 2025 03:18
Comment thread python/prophet/tests/test_prophet.py
Comment thread python/prophet/serialize.py
Co-authored-by: tcuongd <29808430+tcuongd@users.noreply.github.com>
Copilot AI requested a review from tcuongd December 23, 2025 05:10

@tcuongd tcuongd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot Nice, you've covered RMSE degredation and uncertainty widening, but not point accuracy from what I can see. Remember, you can generate a time series from an equation for the exogenous regressors if that makes it easier to assert that the predictions fall within some range.

Also, when I asked to check the cross_validate() outputs, I meant check the metrics returned by that function. The test should live in test_diagnostics.

It's clear from your test that we don't handle serializing exogenous regressor predictors that have already been fit. Is this intended, and if so, why? What would it take to serialize fitted models and load their parameters?

Co-authored-by: tcuongd <29808430+tcuongd@users.noreply.github.com>

Copilot AI commented Dec 23, 2025

Copy link
Copy Markdown
Contributor Author

@copilot Nice, you've covered RMSE degredation and uncertainty widening, but not point accuracy from what I can see. Remember, you can generate a time series from an equation for the exogenous regress...

Point accuracy now asserted via synthetic regressor/y series (predict MAE vs actual/clean bounded) and CV metrics compared in test_diagnostics; added to commit e0a0f0d. Serialization keeps predictors unset intentionally because submodels include backend-specific fit objects; serializing fitted predictors would require persisting their params/Stan fit and backend, which is currently out of scope for lightweight model JSON.

Copilot AI requested a review from tcuongd December 23, 2025 05:44
@tcuongd
tcuongd marked this pull request as ready for review December 23, 2025 12:56
@tcuongd
tcuongd merged commit 993cabd into main Aug 1, 2026
11 checks passed
@tcuongd
tcuongd deleted the copilot/add-uncertainty-to-regressors branch August 1, 2026 05:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants