The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. ex. Some numerals are expressed as "XNUMX".
Copyrights notice
The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. Copyrights notice
Dans les communications véhiculaires à ondes millimétriques (mmWave), la déconnexion des relais multi-sauts par blocage de la ligne de visée (LOS) est un problème critique, en particulier au début de la phase de diffusion des véhicules disponibles aux ondes mm, où tous les véhicules ne disposent pas de dispositifs de communication mmWave. Cet article propose une méthode de contrôle de position distribuée pour établir de longs trajets de relais à travers des unités routières (RSU). Ceci est réalisé par un système par lequel les véhicules autonomes changent leurs positions relatives pour communiquer entre eux via des chemins LOS. Même si les véhicules avec la méthode proposée n'utilisent pas toutes les informations de l'environnement et ne coopèrent pas entre eux, ils peuvent décider de leur action (par exemple, changement de voie et dépassement) et former de longs relais en utilisant uniquement les informations de leur environnement (par exemple, positions environnantes du véhicule). Le problème de prise de décision est formulé comme un processus de décision markovien tel que les véhicules autonomes peuvent apprendre une stratégie de mouvement pratique pour effectuer de longs relais grâce à un algorithme d'apprentissage par renforcement (RL). Cet article conçoit un algorithme d'apprentissage basé sur un algorithme sophistiqué d'apprentissage par renforcement profond, l'avantage acteur-critique asynchrone (A3C), qui permet aux véhicules d'apprendre rapidement une stratégie de mouvement complexe grâce à son architecture de réseau neuronal profond et son mécanisme d'apprentissage multi-agents. Une fois la stratégie bien maîtrisée, les véhicules peuvent se déplacer de manière indépendante pour établir de longs relais et se connecter aux RSU via les relais. Les résultats de la simulation confirment que la méthode proposée peut augmenter la longueur et la couverture du relais même si les conditions de trafic et le taux de pénétration des dispositifs de communication mmWave dans les phases d'apprentissage et d'exploitation sont différents.
Akihito TAYA
Kyoto University
Takayuki NISHIO
Kyoto University
Masahiro MORIKURA
Kyoto University
Koji YAMAMOTO
Kyoto University
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Copier
Akihito TAYA, Takayuki NISHIO, Masahiro MORIKURA, Koji YAMAMOTO, "Deep-Reinforcement-Learning-Based Distributed Vehicle Position Controls for Coverage Expansion in mmWave V2X" in IEICE TRANSACTIONS on Communications,
vol. E102-B, no. 10, pp. 2054-2065, October 2019, doi: 10.1587/transcom.2018EBP3299.
Abstract: In millimeter wave (mmWave) vehicular communications, multi-hop relay disconnection by line-of-sight (LOS) blockage is a critical problem, particularly in the early diffusion phase of mmWave-available vehicles, where not all vehicles have mmWave communication devices. This paper proposes a distributed position control method to establish long relay paths through road side units (RSUs). This is realized by a scheme via which autonomous vehicles change their relative positions to communicate with each other via LOS paths. Even though vehicles with the proposed method do not use all the information of the environment and do not cooperate with each other, they can decide their action (e.g., lane change and overtaking) and form long relays only using information of their surroundings (e.g., surrounding vehicle positions). The decision-making problem is formulated as a Markov decision process such that autonomous vehicles can learn a practical movement strategy for making long relays by a reinforcement learning (RL) algorithm. This paper designs a learning algorithm based on a sophisticated deep reinforcement learning algorithm, asynchronous advantage actor-critic (A3C), which enables vehicles to learn a complex movement strategy quickly through its deep-neural-network architecture and multi-agent-learning mechanism. Once the strategy is well trained, vehicles can move independently to establish long relays and connect to the RSUs via the relays. Simulation results confirm that the proposed method can increase the relay length and coverage even if the traffic conditions and penetration ratio of mmWave communication devices in the learning and operation phases are different.
URL: https://global.ieice.org/en_transactions/communications/10.1587/transcom.2018EBP3299/_p
Copier
@ARTICLE{e102-b_10_2054,
author={Akihito TAYA, Takayuki NISHIO, Masahiro MORIKURA, Koji YAMAMOTO, },
journal={IEICE TRANSACTIONS on Communications},
title={Deep-Reinforcement-Learning-Based Distributed Vehicle Position Controls for Coverage Expansion in mmWave V2X},
year={2019},
volume={E102-B},
number={10},
pages={2054-2065},
abstract={In millimeter wave (mmWave) vehicular communications, multi-hop relay disconnection by line-of-sight (LOS) blockage is a critical problem, particularly in the early diffusion phase of mmWave-available vehicles, where not all vehicles have mmWave communication devices. This paper proposes a distributed position control method to establish long relay paths through road side units (RSUs). This is realized by a scheme via which autonomous vehicles change their relative positions to communicate with each other via LOS paths. Even though vehicles with the proposed method do not use all the information of the environment and do not cooperate with each other, they can decide their action (e.g., lane change and overtaking) and form long relays only using information of their surroundings (e.g., surrounding vehicle positions). The decision-making problem is formulated as a Markov decision process such that autonomous vehicles can learn a practical movement strategy for making long relays by a reinforcement learning (RL) algorithm. This paper designs a learning algorithm based on a sophisticated deep reinforcement learning algorithm, asynchronous advantage actor-critic (A3C), which enables vehicles to learn a complex movement strategy quickly through its deep-neural-network architecture and multi-agent-learning mechanism. Once the strategy is well trained, vehicles can move independently to establish long relays and connect to the RSUs via the relays. Simulation results confirm that the proposed method can increase the relay length and coverage even if the traffic conditions and penetration ratio of mmWave communication devices in the learning and operation phases are different.},
keywords={},
doi={10.1587/transcom.2018EBP3299},
ISSN={1745-1345},
month={October},}
Copier
TY - JOUR
TI - Deep-Reinforcement-Learning-Based Distributed Vehicle Position Controls for Coverage Expansion in mmWave V2X
T2 - IEICE TRANSACTIONS on Communications
SP - 2054
EP - 2065
AU - Akihito TAYA
AU - Takayuki NISHIO
AU - Masahiro MORIKURA
AU - Koji YAMAMOTO
PY - 2019
DO - 10.1587/transcom.2018EBP3299
JO - IEICE TRANSACTIONS on Communications
SN - 1745-1345
VL - E102-B
IS - 10
JA - IEICE TRANSACTIONS on Communications
Y1 - October 2019
AB - In millimeter wave (mmWave) vehicular communications, multi-hop relay disconnection by line-of-sight (LOS) blockage is a critical problem, particularly in the early diffusion phase of mmWave-available vehicles, where not all vehicles have mmWave communication devices. This paper proposes a distributed position control method to establish long relay paths through road side units (RSUs). This is realized by a scheme via which autonomous vehicles change their relative positions to communicate with each other via LOS paths. Even though vehicles with the proposed method do not use all the information of the environment and do not cooperate with each other, they can decide their action (e.g., lane change and overtaking) and form long relays only using information of their surroundings (e.g., surrounding vehicle positions). The decision-making problem is formulated as a Markov decision process such that autonomous vehicles can learn a practical movement strategy for making long relays by a reinforcement learning (RL) algorithm. This paper designs a learning algorithm based on a sophisticated deep reinforcement learning algorithm, asynchronous advantage actor-critic (A3C), which enables vehicles to learn a complex movement strategy quickly through its deep-neural-network architecture and multi-agent-learning mechanism. Once the strategy is well trained, vehicles can move independently to establish long relays and connect to the RSUs via the relays. Simulation results confirm that the proposed method can increase the relay length and coverage even if the traffic conditions and penetration ratio of mmWave communication devices in the learning and operation phases are different.
ER -