The goal of this project is to create an in-car information system that adapts the delivery timings of HCI demands to drivers based on in-situ driving and cognitive load models for safe navigation. Our latest work examined situations in which drivers spontaneously enter high cognitive load states and then identified when a driver can be interrupted (e.g., push notifications can be delivered to the driver). We estimated driver/driving states in real-time by examining a broad range of sensor data streams (e.g., motion capture, peripheral interaction monitoring, psycho-physiological responses, etc.). The study presented a model-based driver/driving assessment to mediate interruptions safely and with minimal increase in driver workload.

In our latest work, we collected sensor data from 25 drivers during naturalistic driving (approx. 1.25 hours / driver). To obtain the data, we equipped participants’ cars with an on-board diagnostics (OBD) device. Participants wore five sensor devices: four accelerometer sensors for capturing body motion and one chest belt sensor for tracking physiological responses (See Figure 1a). We installed two smartphones in each car – one on the front windshield to record traffic in adjacent lanes and the other on the passenger seat headrest to detect drivers’ activities. We obtained information from the OBD device and body-worn sensors via Bluetooth and logged sensor data streams in real-time. In total, we extracted 152 sensor features (OBD: 72; accelerometer sensors: 40; physiological sensor: 40) and 5 manually annotated features related to traffic from videos (one car driving state and four traffic states around the vehicle, i.e. front, right, left, and oncoming traffic).

We used the instances of drivers engaging in peripheral interactions as moments of ground truth for drivers’ split attention while managing the interruption. As a result, we demonstrated that the sensor data can be used to build a machine learning classifier that can determine interruptibility every second with a 94% accuracy. The average classification accuracy of the individual models of our drivers was 94.9% (SD=2.6%, range: 90.2% from 98.2%), which was similar to the population model (94.3%). Accuracy for each driver was greater than 90% (See Figure 1b). We also identified sensor features that best explained the states in which drivers performed peripheral interactions and contributed high system performance. Based on our findings, we proposed a classifier that could be used to build systems that mediate when drivers use technology to self-interrupt and when technology interrupts drivers.

Figure 1. Detecting driver interruptibility every second through monitoring of peripheral interaction states.