Designing adaptive passive personal mobile sensing methods using reinforcement learning framework

Abstract

Smartphone embedded sensors have created unprecedented opportunities to study human behavior in natural conditions through continuous mobile sensing. However, continuous mobile sensing poses critical energy challenge to smartphone’s daily usage. There is an urgent need to enhance energy efficiency of mobile sensing applications while capture sufficient data to accurately predict user state. In this work, we propose an adaptive passive sensing framework to control low-level sensing cycles using an off-policy reinforcement learning (RL) algorithm namely Q-learning with linear approximation and decaying exploration. We propose two different formulations to meet different energy efficiency demands with various designs in their state spaces, action spaces, and reward signals. Using real continuous mobile sensing data from 220 participants for more than 2 weeks, we show consistently better performances on energy saving for our proposed RL strategies when compared to four different baseline methods. To verify the impacts of our proposed strategies on data utility, we predict social anxiety and daily negative affect using active data collected during the same study window. Our proposed RL strategies show equivalent prediction performance when compared to the baseline strategies and continuous sensing.