What, When, How: A Sensor-based Driver Awareness System to Improve Human-Computer Interaction (CHI LBW 2016, T-SET UTC)

The goal of this project is to help drivers safely interact with ubiquitous HCI demands and benefit from proactive information services in cars. Our prior and ongoing projects primarily explore the ‘interruptive’ feature of ubiquitous HCI demands in cars. We have been rigorously addressing the issues of when to intervene by using our sensor-based assessment technologies that estimate drivers’ cognitive load in near real-time. In this project, we extend the key technologies in the projects to help users understand how an in-car system is interpreting the state of the world around them. We aim to support intelligibility of system behavior in cars, including the issues of what intervention to make and how to intervene.

Figure 5. A single Android application that collects real-time sensor data streams from a range of sensors (sensing module), estimates driver/driving states in real-time (computing module), and delivers multimodal information to drivers (feedback module).

In this project, we collect big sensor data streams from a least intrusive set of wearable or internet-of-things sensors, worn by vehicle users and/or embedded in vehicles, including daily smart devices (Figure 5 – left). During a set of human-subject experiments in naturalistic field driving situations, we are investigating how drivers interact with proactive adjustments of HCI demands initiated by system intelligence rather than user demand. We consider presentation methods and types of interaction schemes across human visual, auditory, and haptic sensory channels (Figure 5 – Right). The near goal is to create a smarter, contextually intelligent cyber-physical system that supports intelligibility of system behavior. These experiments will provide a set of sensor-based real-time models of  drivers’ cognitive load, ‚ user interruptibility, and ƒ user experience of proactive information services. Ultimately, these technologies will help drivers safely interact with proactive interventions of machine intelligence in futuristic cars (e.g., intelligent automotive physical systems such as self-driving cars).