Vision-Based Target Detection and Localization via a Team of Cooperative UAV and UGVs

Sara Minaeian, Jian Liu, Young Jun Son

Research output: Contribution to journalArticlepeer-review

252 Scopus citations


Unmanned vehicles (UVs) play a key role in autonomous surveillance scenarios. A major task needed by these UVs in undertaking autonomous patrol missions is to detect the targets and find their locations in real-Time. In this paper, a new vision-based target detection and localization system is presented to make use of different capabilities of UVs as a cooperative team. The scenario considered in this paper is a team of an unmanned aerial vehicle (UAV) and multiple unmanned ground vehicles (UGVs) tracking and controlling crowds on a border area. A customized motion detection algorithm is applied to follow the crowd from the moving camera mounted on the UAV. Due to UAVs lower resolution and broader detection range, UGVs with higher resolution and fidelity are used as the individual human detectors, as well as moving landmarks to localize the detected crowds with unknown independently moving patterns at each time point. The UAVs localization algorithm, proposed in this paper, then converts the crowds' image locations into their real-world positions, using perspective transformation. A rule-of-Thumb localization method by a UGV is also presented, which estimates the geographic locations of the detected individuals. Moreover, an agent-based simulation model is developed for system verification, with different parameters, such as flight altitude, number of landmarks, and landmark assignment method. The performance measure considered in this paper is the average Euclidean distance between the estimated locations and simulated geographic waypoints of the crowd. Experimental results demonstrate the effectiveness of the proposed framework for autonomous surveillance by UVs.

Original languageEnglish (US)
Article number7330001
Pages (from-to)1005-1016
Number of pages12
JournalIEEE Transactions on Systems, Man, and Cybernetics: Systems
Issue number7
StatePublished - Jul 2016


  • Algorithms
  • automation cooperative systems
  • geographic information systems (GISs)
  • position measurement

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Human-Computer Interaction
  • Computer Science Applications
  • Electrical and Electronic Engineering


Dive into the research topics of 'Vision-Based Target Detection and Localization via a Team of Cooperative UAV and UGVs'. Together they form a unique fingerprint.

Cite this