Forge time-series forecasting architectures through controlled, component-level studies

Overview • Quick Start • Modular Model Build • Dataloaders • Forking Sequences • Training • Validation Strategies • Metrics • Inference • Forecast Ensembling • Citations • License

Most forecasting codebases benchmark at the model level. TSForge benchmarks at the component level.

git clone git@github.com:PotosnakW/TSForge.git
cd TSForge
pip install -e .

configs/
  base/
    default.yaml        # shared training params (batch_size, lr, max_steps, etc.)
  dataset/
    simglucose.yaml     # experiment config (train/val/test splits, horizon_override)
    sources/
      simglucose.yaml   # dataset config (path, horizon, val_size, exog_cols, etc.)
  model/
    patchtstmica.yaml   # model architecture config
  config.yaml           # top-level defaults

cd experiments
python train_model.py dataset=simglucose model=patchtstmica

↑ Back to top

When adding a model config file in configs/model/, select encoder, decoder, and output layer. Any component can be set to none to skip it.

d_model: 256
encoder: patchtst      # or rnn, lstm, cnn, none
decoder: none          # only none is currently supported
output_layer: linear_proj  # or none

↑ Back to top

Central object that owns all dataset construction and dataloader creation.

factory      = DataLoaderFactory(mcfg, dcfg)
train_loader = factory.train_dataloader()
val_loaders  = factory.val_dataloaders()
test_loaders = factory.test_dataloaders()

Which class backs a source is driven by its multivariate flag in source/{dataset} config.

Groups datasets by (horizon, is_multivariate) so all items in a batch share the same forecast length and data type. When horizon_override is set, all datasets collapse into a single group per multivariate flag, allowing free mixing across horizons. Univariate and multivariate datasets are always batched separately and never mixed within a batch.

↑ Back to top

Forking-sequences architectures generate forecasts for all FCDs simultaneously by reusing the encoder’s computations, whereas window-sampling produces forecasts for each FCD independently. We include an implementation of forking-sequences from the methods described in [1, 2].

(a) Window-Sampling               (b) Forking-Sequences

The fcd_samples parameter is set in the configs/base/default.yaml file. It can alternatively be specified in individual model configs in configs/model/.

from tsforge.dataloaders._forking_sequences import ForkingSequences

# Training — sample FCDs per series
fs_call = ForkingSequences(context_len=512)
out = fs_call(batch, fcd_samples=4, horizon=6)
# Training — Use all FCDs per series
out = fs_call(batch, fcd_samples=-1, horizon=6)

# Val / Test — all valid windows, no sampling
out = fs_call(batch, fcd_samples=-1, horizon=6)

Called during training (fcd_samples != -1) to pick one window_start per series.

`homogeneous`
Each series gets the same window_start index.
Any timestep is valid so long as L+H falls within the series length. 
Index sampling **does not** account for left-padding and mid-series gaps. 
──────────────────────────────────────────────────────────
series 1   [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]
                        [──── L ────][── H ──]

series 2   [0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]
                        [──── L ────][── H ──]

series 3   [0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1]
                        [──── L ────][── H ──]
                          ^ ^ ^ ^ L samples padding
──────────────────────────────────────────────────────────

`heterogeneous`
Each series gets a unique window_start index.
A timestep is only valid so long as L+H falls within the series length and if **all channels** have real data there.
Index sampling **does** account for left-padding and mid-series gaps. 
──────────────────────────────────────────────────────────
series 1   [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]
               [──── L ────][── H ──]

series 2   [0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]
                         [──── L ────][── H ──]

series 3   [0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1]
                                  [──── L ────][── H ──]
──────────────────────────────────────────────────────────

1. Wen, R., Torkkola, K., Narayanaswamy, B., & Madeka, D. (2018). A Multi-Horizon Quantile Recurrent Forecaster.

2. Potosnak, W., Wolff, M., Cao, M., Ma, R., Konstantinova, T., Efimov, D., Mahoney, M. W., Oreshkin, B., & Olivares, K. G. (2025). Forking-Sequences.

↑ Back to top

from tsforge.common.train import train

train(model, mcfg, train_loader, val_loaders, device=torch.device("cuda"))

# Resume from checkpoint
train(model, mcfg, train_loader, val_loaders, resume="checkpoints/final.pt")

torchrun (recommended):

torchrun --nproc_per_node=4 train_script.py

mp.spawn (programmatic, single-machine):

from tsforge.common.train import train_distributed

train_distributed(model, mcfg, factory, use_spawn=True, world_size=4)

Sharded datasets are the recommended backend for distributed training — each rank loads a non-overlapping subset of shard files.

For datasets too large to fit in RAM, write_sharded_dataset partitions data into time-blocked parquet files with configurable shard size.

from tsforge.dataloaders.ts_sharding import write_sharded_dataset

write_sharded_dataset(
    df             = full_df,           # long-format DataFrame — must have 'available_mask'
    out_dir        = "data/sharded/simglucose",
    val_size       = 2592,              # matches dcfg entry val_size
    test_size      = 2592,              # matches dcfg entry test_size
    context_length = 512,               # L — baked into shard overlap
    shard_size     = 5_000,             # train timesteps per file
    hist_exog_cols = ["CHO", "insulin"],
)

out_dir/
    shard_000000.parquet   # t = [0,          shard_size + L)
    shard_000001.parquet   # t = [shard_size,  2·shard_size + L)   ← L-row overlap
    ...
    val.parquet            # t = [train_end - L,  val_end)
    test.parquet           # t = [val_end - L,    T_total)
    static.parquet         # per-channel static features
    metadata.json          # split boundaries, schema, shard index

The L-row right-edge overlap on each train shard means windows near shard boundaries are self-contained — no cross-file reads needed.

First write the shards with write_sharded_dataset, then point your dataset config entry at the output directory:

- name: "simglucose"
  sharded_dir: "data/sharded/simglucose"
  horizon: 6
  weight: 1.0

path is not required when sharded_dir is set. The factory uses ShardedTrainDataset, ShardedValDataset, and ShardedTestDataset directly.

↑ Back to top

Select the loss via mcfg.loss:

All point-forecast losses share the same signature:

loss_fn(
    preds:   torch.Tensor,         # [B, T, H, C]
    targets: torch.Tensor,         # [B, T, H, C]
    mask:    torch.Tensor | None,  # [B, T, H, C]  1=real, 0=padded/missing
) -> torch.Tensor                  # scalar

Masked reduction applied by every loss:

(raw_loss * mask).sum() / mask.sum().clamp(min=1)

When mask is None a plain .mean() is used — equivalent to a mask of all ones.

from tsforge.metrics.eval_losses import mae, mse, rmse, mape, smape

Same (preds, targets, mask) signature as the training losses above, but on np.ndarray and returning a Python float.

preds   = results["simglucose"]["preds"].numpy()           # [n_fcds, H, C]
targets = results["simglucose"]["targets"].numpy()         # [n_fcds, H, C]
mask    = results["simglucose"]["outsample_mask"].numpy()  # [n_fcds, H, C]

print("MAE: ", mae(preds, targets, mask))
print("RMSE:", rmse(preds, targets, mask))

Quantile models — pass only the median slice to point-forecast metrics:

Q   = len(mcfg.quantiles)
mid = mcfg.quantiles.index(0.5)
p_median = preds.reshape(*preds.shape[:-1], -1, Q)[..., mid]
print("Median MAE:", mae(p_median, targets, mask))

Stability metrics operate on the overlapping structure of forking sequences — multiple forecast windows predict the same target date from different horizons, so revisions across consecutive windows can be measured directly. Lower measurements are preferred.

from tsforge.metrics.stability_metrics import excess_volatility, forecast_percentage_change

Measures harmful forecast instability: revisions that incur a quantile-loss cost without a corresponding accuracy improvement.

ev = excess_volatility(
    targets=targets, # [B, T, H, C]
    preds=preds, # [B, T, H, C, Q]
    quantiles=mcfg.quantiles, # [Q]
    mask=mask, # [B, T, H, C]
    scaling=True,
)

Measures the relative magnitude of revisions without reference to ground truth. Can be computed on live windows where targets are unavailable.

sfpc = forecast_percentage_change(
    preds=p_median, # [B, T, H, C]
    mask=mask, # [B, T, H, C]
    symmetric=True,
)

↑ Back to top

Controlled by mcfg.val_strategy:

Test always uses exhaustive evaluation regardless of val_strategy.

↑ Back to top

results = eval_test(model, factory, device=torch.device("cuda"))

Returns a nested dict keyed by dataset name:

{
    "simglucose": {
        "channel_ids":    ["patient_001", "patient_002", ...],
        "preds":          Tensor,   # [n_fcds, H, C]
        "targets":        Tensor,   # [n_fcds, H, C]
        "outsample_mask": Tensor,   # [n_fcds, H, C]   1 = real, 0 = missing
    }
}

↑ Back to top

Forking sequences produce multiple overlapping predictions for any given future timestep; ensembling these overlaps can meaningfully reduce variance before computing final metrics.

See notebooks/tsfm_ensembling_demo.ipynb for a full worked example regarding forecast ensembling with pretrained time series foundation models.

window_size=None uses all available overlapping windows. Set an integer to limit aggregation to the N most recent overlaps.

↑ Back to top

If you find this work useful, please cite:

@misc{potosnak2026tsforge,
  title        = {TSForge},
  author       = {Potosnak, Willa and Dubrawski, Artur},
  year         = {2026},
  url={https://github.com/PotosnakW/TSForge}
}

@article{potosnak2026forkingsequences,
  title        = {Forking-Sequences},
  author       = {Potosnak, Willa and Wolff, Malcolm and Cao, Mengfei and Ma, Ruijun and
                  Konstantinova, Tatiana and Efimov, Dmitry and Mahoney, Michael W. and
                  Oreshkin, Boris and Olivares, Kin G.},
  year         = {2025},
}

Our codebase was inspired by the following open-source projects, whose foundational contributions to the community have shaped the landscape of forecasting benchmarking, development, and follow-on codebases like this one.

@misc{olivares2022library_neuralforecast,
    author={Kin G. Olivares and
            Cristian Challú and
            Azul Garza and
            Max Mergenthaler Canseco and
            Artur Dubrawski},
    title = {{NeuralForecast}: User friendly state-of-the-art neural forecasting models.},
    year={2022},
    howpublished={{PyCon} Salt Lake City, Utah, US 2022},
    url={https://github.com/Nixtla/neuralforecast}
}

@inproceedings{goswami2024moment,
  title={MOMENT: A Family of Open Time-series Foundation Models},
  author={Mononito Goswami and Konrad Szafer and Arjun Choudhry and Yifu Cai and Shuo Li and Artur Dubrawski},
  booktitle={International Conference on Machine Learning},
  year={2024}
}

↑ Back to top

MIT License

Copyright (c) 2024 Auton Lab, Carnegie Mellon University

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

See MIT LICENSE for details.

↑ Back to top