| Dataset | SES | TBATS | Theta | ETS | (DHR-) ARIMA | PR | Cat- Boost | FFNN | DeepAR | N-BEATS | WaveNet | Trans- former | Informer* |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| australian_electricity_demand | 3.277 | 2.125 | 3.292 | 6.13 | 4.691 | - | 1.35 | 1.574 | 1.831 | 1.28 | 1.403 | 1.404 | - |
| bitcoin | 6.18 | 5.546 | 6.22 | 5.818 | 6.546 | - | 5.653 | 7.027 | 7.24 | 9.02 | 5.526 | 9.228 | - |
| car_parts | 1.563 | 1.688 | 1.602 | 1.593 | 1.611 | 1.508 | 1.57 | 1.524 | 1.524 | 3.096 | 1.523 | 1.524 | - |
| cif_2016 | 1.767 | 1.128 | 1.316 | 1.099 | 1.219 | 1.371 | 1.602 | - | - | - | - | - | - |
| covid_deaths | 9.489 | 6.99 | 9.567 | 6.551 | 7.527 | 10.542 | 10.309 | 6.893 | 8.577 | 7.178 | 9.462 | 10.839 | - |
| dominick | 0.682 | 0.839 | 0.719 | 0.696 | - | 1.107 | 0.982 | 0.719 | 0.693 | 1.13 | 0.701 | 0.686 | - |
| electricity_hourly | 5.587 | 4.636 | 5.588 | 7.514 | 5.72 | 3.752 | 3.229 | 4.258 | 3.407 | 2.828 | 2.685 | 3.245 | 5.549 |
| electricity_weekly | 1.679 | 0.916 | 1.607 | 1.669 | 1.013 | 1.051 | 0.94 | 0.906 | 1.14 | 0.921 | 1.382 | 1.901 | - |
| fred_md | 0.73 | 0.596 | 0.951 | 0.564 | 0.632 | 9.644 | 1.047 | 0.719 | 0.756 | 0.699 | 0.908 | 2.027 | 18.307 |
| hospital | 0.992 | 0.938 | 0.931 | 0.936 | 0.96 | 0.953 | 0.973 | 1.019 | 0.935 | 0.963 | 0.954 | 1.206 | - |
| kaggle_web_traffic | 0.959 | 0.895 | 0.951 | 1.045 | - | 1.306 | 2.517 | 0.974 | 1.03 | 0.952 | 0.901 | 1.173 | - |
| kdd_cup | 2.405 | 2.159 | 2.406 | 2.553 | 2.863 | 2.004 | 1.937 | 1.966 | 2.485 | 2.396 | 1.935 | 2.5 | - |
| m1_monthly | 1.688 | 1.338 | 1.306 | 1.285 | 1.389 | 1.36 | 1.449 | 1.444 | 1.424 | 1.408 | 1.454 | 2.525 | - |
| m1_quarterly | 2.227 | 1.98 | 1.967 | 1.913 | 2.087 | 2.193 | 2.363 | 2.168 | 2.122 | 2.086 | 1.984 | 3.208 | - |
| m1_yearly | 5.708 | 4.105 | 4.879 | 4.416 | 5.414 | 5.325 | 5.19 | 5.13 | 5.383 | 5.101 | 5.367 | 6.386 | - |
| m3_monthly | 1.302 | 1.036 | 1.035 | 1.036 | 1.048 | 1.198 | - | 1.203 | 1.373 | 1.132 | 1.212 | 1.693 | - |
| m3_quarterly | 1.658 | 1.464 | 1.303 | 1.37 | 1.448 | 1.453 | 1.693 | 1.561 | 1.523 | 1.384 | 1.489 | 2.773 | - |
| m3_yearly | 3.626 | 3.646 | 3.204 | 3.32 | 4.017 | 3.738 | 4.347 | 3.946 | 4.065 | 3.438 | 3.49 | 3.534 | - |
| m4_daily | 1.351 | 1.356 | 1.35 | 1.461 | 1.385 | 1.358 | 1.792 | 1.342 | 2.514 | 1.417 | 1.353 | 1.592 | - |
| m4_hourly | 14.341 | 3.34 | 14.236 | 31.151 | 14.185 | 1.949 | 2.22 | 3.475 | 2.688 | 2.812 | 2.075 | 9.379 | - |
| m4_monthly | 1.382 | 1.353 | 1.163 | 1.138 | 1.163 | 1.296 | 1.304 | 1.413 | 1.39 | 1.241 | 1.389 | 2.345 | - |
| m4_quarterly | 1.656 | 1.385 | 1.429 | 1.355 | 1.43 | 1.524 | 1.558 | 1.645 | 1.486 | 1.448 | 1.443 | 1.738 | - |
| m4_weekly | 0.71 | 0.602 | 0.658 | 0.694 | 0.659 | 0.576 | 0.713 | 0.647 | 0.7 | 0.543 | 0.701 | 0.83 | - |
| m4_yearly | 4.55 | 3.92 | 3.861 | 3.927 | 4.443 | 4.115 | 4.179 | - | - | - | - | - | - |
| nn5_daily | 1.889 | 1.202 | 1.224 | 1.21 | 1.391 | 1.672 | 1.32 | 1.337 | 1.276 | 1.491 | 1.322 | 1.364 | 1.925 |
| nn5_weekly | 1.084 | 1.076 | 1.078 | 1.088 | 1.07 | 1.06 | 1.045 | 1.037 | 1.085 | 0.986 | 1.405 | 1.356 | - |
| pedestrian_counts | 1.287 | 1.525 | 1.288 | 1.533 | 5.144 | 0.353 | 0.36 | 0.377 | 0.392 | 0.556 | 0.353 | 0.396 | - |
| rideshare | 4.76 | 4.827 | 5.634 | 4.76 | 3.359 | - | 4.622 | 4.726 | 4.739 | 4.349 | 3.727 | 4.76 | - |
| saugeen_river_flow | 2.64 | 2.824 | 2.639 | 3.343 | 2.867 | 3.164 | 2.608 | 2.695 | 3.004 | 3.244 | 2.852 | 3.258 | 2.8 |
| solar_10_minutes | 3.2 | 4.802 | 3.201 | 3.2 | 2.441 | 3.2 | 3.847 | 3.194 | 3.198 | 2.926 | - | 3.2 | - |
| solar_weekly | 1.345 | 1.057 | 1.356 | 1.27 | 0.975 | 1.174 | 1.763 | 1.224 | 0.884 | 1.321 | 2.485 | 0.687 | - |
| sunspot | 0.128 | 0.077 | 0.128 | 0.128 | 0.077 | 0.103 | 0.062 | 0.219 | 0.03 | 0.376 | 0.017 | 0.013 | 0.332 |
| temperature_rain | 1.963 | 1.815 | 1.974 | 2.021 | 1.779 | - | 1.63 | 1.684 | 1.677 | 2.221 | 1.631 | 1.588 | - |
| tourism_monthly | 4.427 | 2.273 | 2.158 | 1.954 | 2.03 | 2.165 | 2.195 | 2.055 | 1.824 | 2.079 | 1.898 | 1.986 | - |
| tourism_quarterly | 3.769 | 2.209 | 2.046 | 1.905 | 2.112 | 2.003 | - | 2.036 | 1.915 | 1.811 | 2.048 | 2.17 | - |
| tourism_yearly | 3.568 | 4.05 | 3.307 | 3.713 | 4.173 | 3.784 | 3.911 | 3.717 | 3.456 | 3.25 | 3.974 | 3.847 | - |
| traffic_hourly | 2.489 | 3.173 | 2.489 | 2.904 | 3.091 | 1.821 | 2.132 | 1.413 | 1.363 | 1.698 | 1.747 | 1.32 | 3.984 |
| traffic_weekly | 1.5 | 1.494 | 1.514 | 1.509 | 1.494 | 1.499 | 1.438 | 1.551 | 1.507 | 1.419 | 1.641 | 2.098 | - |
| us_births | 4.988 | 2.209 | 2.679 | 2.212 | 2.57 | 2.667 | 2.252 | 2.647 | 2.491 | 2.286 | 2.8 | 2.501 | 6.882 |
| vehicle_trips | 2.822 | 2.339 | 2.866 | 2.963 | 2.845 | - | 2.482 | 2.33 | 2.408 | 2.655 | 2.324 | 3.002 | - |
| weather | 0.873 | 0.88 | 0.978 | 0.899 | 0.947 | 3.342 | 0.95 | 0.873 | 0.869 | 0.959 | 0.951 | 0.901 | - |
*The results of the Informer model are only recorded for the datasets with equal-length series. For the datasets with unequal-length series, the Informer model is required to be executed per each series where the execution time is considerably high (for details, see here). Furthermore, the intermittent datasets such as Carparts, Rideshare, Web Traffic, Covid Deaths and Temperature Rain are not considered for the Informer experiments.
We encourage other researchers to contribute time series datasets or benchmarking results to our repository either by directly uploading the datasets into our repository and/or by contacting us via email. You will then be listed as a contributor, and in the acknowledgements section.
If there are any copyright issues of the datasets, please contact us via email.
We are very grateful to the Department of Data Science and Artificial Intelligence of Monash University for their sponsorship.
Gareth Davies from Neural Aspect has contributed benchmark runs of many of the deep-learning methods, namely Autoformer, DLinear, NLinear, NBEATS, N-HITS, TiDE, PatchTST, and TimesFM.
