Physics-Informed Parametric Bandits for Beam Alignment in mmWave Communications

In millimeter wave (mmWave) communications, beam alignment and tracking are crucial to combat significant path loss. As scanning the entire directional space is inefficient, designing an efficient and robust method to identify optimal beam directions is essential. Since traditional multi-armed bandit algorithms require a long time horizon to converge with large beam spaces, many works propose structured bandit algorithms for beam alignment by relying on unimodality or multimodality assumptions on the reward function. However, such assumptions often cannot be strictly satisfied in practice, which causes such algorithms to converge to choosing suboptimal beams.In this work, we propose two physics-informed bandit algorithms PR-ETC and PR-GREEDY that exploit the sparse multi-path property of mmWave propagation - a generic but realistic assumption - which is connected to the Phase Retrieval problem. Our algorithms treat the propagation paths as a black box and maintain estimates of them based on historical rewards. PR-ETC starts with a random exploration phase and then commits to the optimal beam under the estimated reward function. PR-GREEDY performs such estimation in an online manner and chooses the best beam under current estimates. Our algorithms can also be easily adapted to beam tracking in the mobile setting. Through experiments using both the synthetic DeepMIMO dataset and the real-world DeepSense6G dataset, we demonstrate that both algorithms outperform existing approaches in a wide range of scenarios across diverse channel environments, showing their generalizability and robustness.