MAMBA: A Multi-armed Bandit Framework for Beam Tracking in Millimeter-wave Systems

Irmak Aykin, Berk Akgun, Mingjie Feng, Marwan Krunz

Research output: Chapter in Book/Report/Conference proceedingConference contribution

42 Scopus citations


Millimeter-wave (mmW) spectrum is a major candidate to support the high data rates of 5G systems. However, due to directionality of mmW communication systems, misalignments between the transmit and receive beams occur frequently, making link maintenance particularly challenging and motivating the need for fast and efficient beam tracking. In this paper, we propose a multi-armed bandit framework, called MAMBA, for beam tracking in mmW systems. We develop a reinforcement learning algorithm, called adaptive Thompson sampling (ATS), that MAMBA embodies for the selection of appropriate beams and transmission rates along these beams. ATS uses prior beam-quality information collected through the initial access and updates it whenever an ACK/NACK feedback is obtained from the user. The beam and the rate to be used during next downlink transmission are then selected based on the updated posterior distributions. Due to its model-free nature, ATS can accurately estimate the best beam/rate pair, without making assumptions regarding the temporal channel and/or user mobility. We conduct extensive experiments over the 28 GHz band using a 4x8 phased- array antenna to validate the efficiency of ATS, and show that it improves the link throughput by up to 182%, compared to the beam management scheme proposed for 5G.

Original languageEnglish (US)
Title of host publicationINFOCOM 2020 - IEEE Conference on Computer Communications
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages10
ISBN (Electronic)9781728164120
StatePublished - Jul 2020
Event38th IEEE Conference on Computer Communications, INFOCOM 2020 - Toronto, Canada
Duration: Jul 6 2020Jul 9 2020

Publication series

NameProceedings - IEEE INFOCOM
ISSN (Print)0743-166X


Conference38th IEEE Conference on Computer Communications, INFOCOM 2020


  • Millimeter-wave
  • beam tracking
  • directional communications
  • multi-armed bandit
  • reinforcement learning

ASJC Scopus subject areas

  • General Computer Science
  • Electrical and Electronic Engineering


Dive into the research topics of 'MAMBA: A Multi-armed Bandit Framework for Beam Tracking in Millimeter-wave Systems'. Together they form a unique fingerprint.

Cite this