thuml/iTransformer is a Python project with 2.2k stars. Official implementation for "iTransformer: Inverted Transformers Are Effective for Time Series Forecasting" (ICLR 2024 Spotlight)
Snapshot summary built from the project's own GitHub metadata — there's no written TopGit review yet. The page will update automatically when a full review is published.
WHY NO REVIEW YET
TopGit writes full reviews for the most-starred, most-requested repositories. This page is a snapshot until then — see the READ ME tab for the original README in full.
The repo is the official implementation for the paper: iTransformer: Inverted Transformers Are Effective for Time Series Forecasting. [Slides], [Poster], [Intro (CN)].
.
Updates
:triangular_flag_on_post: News (2024.10) TimeXer, a Transformer for predicting with exogenous variables, is released. Code is available here.
:triangular_flag_on_post: News (2024.05) Many thanks for the great efforts from lucidrains. A pip package for the usage of iTransformer variants can be simply installed via pip install iTransformer
:triangular_flag_on_post: News (2024.04) iTransformer has benn included in NeuralForecast. Special thanks to the contributor @Marco!
:triangular_flag_on_post: News (2024.03) Introduction of our work in Chinese is available.
:triangular_flag_on_post: News (2024.02) iTransformer has been accepted as ICLR 2024 Spotlight.
:triangular_flag_on_post: News (2023.12) iTransformer available in GluonTS with probablistic head and support for static covariates. Notebook is available here.
:triangular_flag_on_post: News (2023.12) We received lots of valuable suggestions. A revised version (24 Pages) is now available.
:triangular_flag_on_post: News (2023.10) iTransformer has been included in [Time-Series-Library] and achieves state-of-the-art in Lookback-$96$ forecasting.
:triangular_flag_on_post: News (2023.10) All the scripts for the experiments in our paper are available.
Introduction
🌟 Considering the characteristics of multivariate time series, iTransformer breaks the conventional structure without modifying any Transformer modules. Inverted Transformer is all you need in MTSF.
🏆 iTransformer achieves the comprehensive state-of-the-art in challenging multivariate forecasting tasks and solves several pain points of Transformer on extensive time series data.
Overall Architecture
iTransformer regards independent time series as variate tokens to capture multivariate correlations by attention and utilize layernorm and feed-forward networks to learn series representations.
The pseudo-code of iTransformer is as simple as the following:
Usage
Install Pytorch and the necessary dependencies.
pip install -r requirements.txt
The datasets can be obtained from Google Drive or Baidu Cloud.
Train and evaluate the model. We provide all the above tasks under the folder ./scripts/. You can reproduce the results as the following examples:
# Multivariate forecasting with iTransformer
bash ./scripts/multivariate_forecasting/Traffic/iTransformer.sh
# Compare the performance of Transformer and iTransformer
bash ./scripts/boost_performance/Weather/iTransformer.sh
# Train the model with partial variates, and generalize to the unseen variates
bash ./scripts/variate_generalization/ECL/iTransformer.sh
# Test the performance on the enlarged lookback window
bash ./scripts/increasing_lookback/Traffic/iTransformer.sh
# Utilize FlashAttention for acceleration
bash ./scripts/efficient_attentions/iFlashTransformer.sh
Main Result of Multivariate Forecasting
We evaluate the iTransformer on challenging multivariate forecasting benchmarks (generally hundreds of variates). Comprehensive good performance (MSE/MAE $\downarrow$) is achieved.
Online Transaction Load Prediction of Alipay Trading Platform (Avg Results)
General Performance Boosting on Transformers
By introducing the proposed framework, Transformer and its variants achieve significant performance improvement, demonstrating the generality of the iTransformer approach and benefiting from efficient attention mechanisms.
Zero-Shot Generalization on Variates
Technically, iTransformer is able to forecast with arbitrary numbers of variables. We train iTransformers on partial variates and forecast unseen variates with good generalizability.
Model Analysis
Benefiting from inverted Transformer modules:
(Left) Inverted Transformers learn better time series representations (more similar CKA) favored by forecasting.
(Right) The inverted self-attention module learns interpretable multivariate correlations.
Citation
If you find this repo helpful, please cite our paper.
@article{liu2023itransformer,
title={iTransformer: Inverted Transformers Are Effective for Time Series Forecasting},
author={Liu, Yong and Hu, Tengge and Zhang, Haoran and Wu, Haixu and Wang, Shiyu and Ma, Lintao and Long, Mingsheng},
journal={arXiv preprint arXiv:2310.06625},
year={2023}
}
Acknowledgement
We appreciate the following GitHub repos a lot for their valuable code and efforts.
The most recent commit recorded on thuml/iTransformer was 1.1 years ago, based on the GitHub push timestamp. The repository has 369 forks — one of the better signals of community interest.
Is thuml/iTransformer open source?
Yes — thuml/iTransformer ships under the MIT license, which makes its source code freely readable (and, depending on license terms, forkable and reusable). Source: github.com/thuml/iTransformer.
What license does thuml/iTransformer use?
thuml/iTransformer is released under the MIT license. Always verify the LICENSE file directly on GitHub for the authoritative terms — license strings can be edited out of sync with a project's actual stance.
What topics is thuml/iTransformer associated with?
GitHub's repository topics for thuml/iTransformer: "time-series-forecasting", "transformer". TopGit's editorial category is open-source.
Where can I see thuml/iTransformer in action?
The project maintains a homepage at https://arxiv.org/abs/2310.06625. The README tab on this page also usually contains screenshots and a quickstart.
Where do I read more about thuml/iTransformer?
This TopGit page is a snapshot — the READ ME tab shows the project's own README content (links stripped, images preserved). The GitHub repository at github.com/thuml/iTransformer is the definitive source.
Read full README in the tab above.
Still deciding about iTransformer?
One click hands the question to an AI along with this page — see what it says about iTransformer.
/iTransformer
