Semantics-Aware Patch Encoding and Hierarchical Dependency Modeling for Long-Term Time Series Forecasting
Official PyTorch Implementation
Overview of MoU
We introduce Mixture of Universals (MoU), a novel framework designed to prevent semantic loss during patch encoding and efficiently enhance long-term dynamics through a hybrid approach. Specifically, MoU is consist of two novel designs: Mixture of Feature Extractors (MoF) and Mixture of Architectures (MoA). MoF introduces a semantics-aware encoding mechanism to preserve diverse temporal patterns and mitigating information loss. MoA, on the other hand, hierarchically captures long-term dependency with progressively expanded receptive field, improving model performance while maintaining relatively low computational costs. The proposed approach achieves state-of-the-art performance.
The overall performance of MoU for long-term forecasting is summarized in the following Table (average performance). More detailed results can be found in our paper.
| Model | MoU (Ours) | ModernTCN | PatchTST | HDMixer | RMLP | DLinear | S-Mamba | iTransformer | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Metric | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE |
| ETTh1 | 0.397 | 0.423 | 0.404 | 0.420 | 0.413 | 0.434 | 0.408 | 0.426 | 0.442 | 0.443 | 0.423 | 0.437 | 0.450 | 0.456 | 0.465 | 0.465 |
| ETTh2 | 0.317 | 0.373 | 0.323 | 0.379 | 0.331 | 0.379 | 0.320 | 0.374 | 0.377 | 0.414 | 0.431 | 0.447 | 0.369 | 0.405 | 0.385 | 0.414 |
| ETTm1 | 0.348 | 0.382 | 0.354 | 0.382 | 0.352 | 0.382 | 0.359 | 0.385 | 0.357 | 0.379 | 0.357 | 0.379 | 0.367 | 0.396 | 0.367 | 0.395 |
| ETTm2 | 0.252 | 0.315 | 0.256 | 0.316 | 0.256 | 0.317 | 0.257 | 0.316 | 0.263 | 0.318 | 0.267 | 0.332 | 0.265 | 0.326 | 0.271 | 0.329 |
| Weather | 0.221 | 0.262 | 0.225 | 0.267 | 0.226 | 0.264 | 0.235 | 0.275 | 0.236 | 0.273 | 0.240 | 0.300 | 0.236 | 0.273 | 0.238 | 0.273 |
| illness | 1.500 | 0.784 | 1.519 | 0.799 | 1.513 | 0.825 | 2.019 | 0.891 | 1.593 | 0.843 | 2.169 | 1.041 | 1.977 | 0.890 | 2.222 | 1.012 |
| electricity | 0.157 | 0.253 | 0.157 | 0.253 | 0.159 | 0.253 | 0.160 | 0.252 | 0.172 | 0.266 | 0.177 | 0.274 | 0.166 | 0.262 | 0.170 | 0.265 |
Download code:
git clone https://github.com/lunaaa95/mou.git
cd mou
A suitable conda environment named mou can be created and activated with:
conda create -n mou python=3.8
conda activate mou
pip install -r requirement.txt
Download datasets to folder ./dataset. You can download all datasets from Google Drive provided by Wu, H.
- We provide bash scripts for all datasets. Run bash scripts in folder "./scripts" to start time series long-term forecasting. For example,
bash scripts/MoU/etth1.sh
bash scripts/MoU/etth2.sh
bash scripts/MoU/ettm1.sh
bash scripts/MoU/ettm2.sh
bash scripts/MoU/weather.sh
bash scripts/MoU/electricity.sh
bash scripts/MoU/illness.sh- We also provide other short-term encoders and long-term encoders to switch the structure of model. Change parameters
entypefor other short-term encoders andltencoderfor long-term encoders. - We also give two baseline models of
PatchTSTandDLinearas well as their runing scripts.
@inproceedings{peng2025semantics,
title={Semantics-Aware Patch Encoding and Hierarchical Dependency Modeling for Long-Term Time Series Forecasting},
author={Peng, Sijia and Xiong, Yun and Zhu, Yangyong and Shen, Zhiqiang},
booktitle={Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2},
pages={2269--2280},
year={2025}
}
