Unmanned Vehicle Autonomy for Long-Duration Surveillance Missions

Long duration unmanned vehicle missions with unreliable communication require flexible onboard planning to assure controls are appropriate for an uncertain environment. While the autonomous unmanned vehicle (AUV) can command its own actions, environmental dynamics are uncontrollable and typically uncertain. This paper applies the Markov Decision Process (MDP) to an environmental sampling mission with uncertain sampling target locations. AUV movements are discretized and the reward function maximizes likelihood of collecting valuable data. By modeling environmental features as MDP state features rather than injected noise, mission planning can be robust to changes in environmental conditions. This paper efficiently integrates transitions over controllable AUV motions in the presence of uncertain, uncontrollable, spatially-varying environmental dynamics. An AUV exploration case study is investigated with MDP states including location, time, three-dimensional water current velocity, temperature, surface air pressure, and other spatially and temporally varying ocean environmental parameters generated from the Regional Ocean Modeling System (ROMS). This paper contributes a novel controllable/uncontrollable partitioning for AUV decision state and examines its use for an AUV operating in a realistic ocean environment.