Learning to soar in turbulent environments

Birds and gliders exploit warm, rising atmospheric currents (thermals) to reach heights comparable to low-lying clouds with a reduced expenditure of energy. Soaring provides a remarkable instance of complex decision-making in biology and requires a long-term strategy to effectively use the ascending thermals. Furthermore, the problem is technologically relevant to extend the flying range of autonomous gliders. The formation of thermals unavoidably generates strong turbulent fluctuations, which make deriving an efficient policy harder and thus constitute an essential element of soaring. Here, we approach soaring flight as a problem of learning to navigate highly fluctuating turbulent environments. We simulate the atmospheric boundary layer by numerical models of turbulent convective flow and combine them with model-free, experience-based, reinforcement learning algorithms to train the virtual gliders. For the learned policies in the regimes of moderate and strong turbulence levels, the virtual glider adopts an increasingly conservative policy as turbulence levels increase, quantifying the degree of risk affordable in turbulent environments. Reinforcement learning uncovers those sensorimotor cues that permit effective control over soaring in turbulent environments.