The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. ex. Some numerals are expressed as "XNUMX".
Copyrights notice
The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. Copyrights notice
L'apprentissage des représentations est une tâche cruciale et complexe pour l'analyse de données de séries chronologiques multivariées, avec un large éventail d'applications, notamment l'analyse des tendances, la recherche de données de séries chronologiques et la prévision. En pratique, l’apprentissage non supervisé est fortement préféré en raison de la rareté des étiquetages. Cependant, la plupart des études existantes se concentrent sur la représentation de sous-séries individuelles sans considérer les relations entre les différentes sous-séries. Dans certains scénarios, cela peut entraîner des échecs de tâches en aval. Ici, un modèle d'apprentissage de représentation non supervisé est proposé pour les séries temporelles multivariées qui prend en compte la relation sémantique entre les sous-séries de séries temporelles. Plus précisément, la covariance calculée par le processus gaussien (GP) est introduite dans le mécanisme d'auto-attention, capturant les caractéristiques relationnelles de la sous-série. De plus, une nouvelle méthode non supervisée est conçue pour apprendre la représentation de séries temporelles multivariées. Pour relever les défis des longueurs variables des sous-séries d'entrée, une méthode de pooling de pyramide temporelle (TPP) est appliquée pour construire des vecteurs d'entrée de longueur égale. Les résultats expérimentaux montrent que notre modèle présente des avantages substantiels par rapport à d'autres modèles d'apprentissage des représentations. Nous avons mené des expériences sur l'algorithme proposé et les algorithmes de base dans deux tâches en aval : la classification et la récupération. Dans la tâche de classification, le modèle proposé a démontré les meilleures performances sur sept ensembles de données sur dix, atteignant une précision moyenne de 76 %. Dans la tâche de récupération, l'algorithme proposé a obtenu les meilleures performances sous différents ensembles de données et tailles cachées. Le résultat de l'étude sur l'ablation démontre également l'importance de la relation sémantique dans l'apprentissage de la représentation de séries chronologiques multivariées.
Chengyang YE
Kyoto University
Qiang MA
Kyoto University
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Copier
Chengyang YE, Qiang MA, "Semantic Relationship-Based Unsupervised Representation Learning of Multivariate Time Series" in IEICE TRANSACTIONS on Information,
vol. E107-D, no. 2, pp. 191-200, February 2024, doi: 10.1587/transinf.2023EDP7046.
Abstract: Representation learning is a crucial and complex task for multivariate time series data analysis, with a wide range of applications including trend analysis, time series data search, and forecasting. In practice, unsupervised learning is strongly preferred owing to sparse labeling. However, most existing studies focus on the representation of individual subseries without considering relationships between different subseries. In certain scenarios, this may lead to downstream task failures. Here, an unsupervised representation learning model is proposed for multivariate time series that considers the semantic relationship among subseries of time series. Specifically, the covariance calculated by the Gaussian process (GP) is introduced to the self-attention mechanism, capturing relationship features of the subseries. Additionally, a novel unsupervised method is designed to learn the representation of multivariate time series. To address the challenges of variable lengths of input subseries, a temporal pyramid pooling (TPP) method is applied to construct input vectors with equal length. The experimental results show that our model has substantial advantages compared with other representation learning models. We conducted experiments on the proposed algorithm and baseline algorithms in two downstream tasks: classification and retrieval. In classification task, the proposed model demonstrated the best performance on seven of ten datasets, achieving an average accuracy of 76%. In retrieval task, the proposed algorithm achieved the best performance under different datasets and hidden sizes. The result of ablation study also demonstrates significance of semantic relationship in multivariate time series representation learning.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2023EDP7046/_p
Copier
@ARTICLE{e107-d_2_191,
author={Chengyang YE, Qiang MA, },
journal={IEICE TRANSACTIONS on Information},
title={Semantic Relationship-Based Unsupervised Representation Learning of Multivariate Time Series},
year={2024},
volume={E107-D},
number={2},
pages={191-200},
abstract={Representation learning is a crucial and complex task for multivariate time series data analysis, with a wide range of applications including trend analysis, time series data search, and forecasting. In practice, unsupervised learning is strongly preferred owing to sparse labeling. However, most existing studies focus on the representation of individual subseries without considering relationships between different subseries. In certain scenarios, this may lead to downstream task failures. Here, an unsupervised representation learning model is proposed for multivariate time series that considers the semantic relationship among subseries of time series. Specifically, the covariance calculated by the Gaussian process (GP) is introduced to the self-attention mechanism, capturing relationship features of the subseries. Additionally, a novel unsupervised method is designed to learn the representation of multivariate time series. To address the challenges of variable lengths of input subseries, a temporal pyramid pooling (TPP) method is applied to construct input vectors with equal length. The experimental results show that our model has substantial advantages compared with other representation learning models. We conducted experiments on the proposed algorithm and baseline algorithms in two downstream tasks: classification and retrieval. In classification task, the proposed model demonstrated the best performance on seven of ten datasets, achieving an average accuracy of 76%. In retrieval task, the proposed algorithm achieved the best performance under different datasets and hidden sizes. The result of ablation study also demonstrates significance of semantic relationship in multivariate time series representation learning.},
keywords={},
doi={10.1587/transinf.2023EDP7046},
ISSN={1745-1361},
month={February},}
Copier
TY - JOUR
TI - Semantic Relationship-Based Unsupervised Representation Learning of Multivariate Time Series
T2 - IEICE TRANSACTIONS on Information
SP - 191
EP - 200
AU - Chengyang YE
AU - Qiang MA
PY - 2024
DO - 10.1587/transinf.2023EDP7046
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E107-D
IS - 2
JA - IEICE TRANSACTIONS on Information
Y1 - February 2024
AB - Representation learning is a crucial and complex task for multivariate time series data analysis, with a wide range of applications including trend analysis, time series data search, and forecasting. In practice, unsupervised learning is strongly preferred owing to sparse labeling. However, most existing studies focus on the representation of individual subseries without considering relationships between different subseries. In certain scenarios, this may lead to downstream task failures. Here, an unsupervised representation learning model is proposed for multivariate time series that considers the semantic relationship among subseries of time series. Specifically, the covariance calculated by the Gaussian process (GP) is introduced to the self-attention mechanism, capturing relationship features of the subseries. Additionally, a novel unsupervised method is designed to learn the representation of multivariate time series. To address the challenges of variable lengths of input subseries, a temporal pyramid pooling (TPP) method is applied to construct input vectors with equal length. The experimental results show that our model has substantial advantages compared with other representation learning models. We conducted experiments on the proposed algorithm and baseline algorithms in two downstream tasks: classification and retrieval. In classification task, the proposed model demonstrated the best performance on seven of ten datasets, achieving an average accuracy of 76%. In retrieval task, the proposed algorithm achieved the best performance under different datasets and hidden sizes. The result of ablation study also demonstrates significance of semantic relationship in multivariate time series representation learning.
ER -