Adapting Safety Interfaces using Driver Cognitive Factors Inference
Authors: Emily Sumner, Jonathan DeCastro, Deepak Gopinath, Jean Costa, Tiffany Chen, and Guy Rosman
One of Toyota’s most ambitious aspirations is a future without traffic fatalities or injuries. This is a profound goal, given that approximately 1.19 million lives are lost per year and between 20–50 million more people suffer non-fatal injuries in traffic accidents worldwide [1]. At TRI, developing new human-centric technologies to push the boundaries of performance and safety for the driver and vehicle is one of our key research areas in the Human Interactive Driving division.
Today, most modern vehicles have some form of Advanced Driver-Assistance System (ADAS) to alert the driver of lane departures or imminent collisions. These systems are often standardized, only activating in certain situations without consideration for the behavior or preferences of human drivers. This led us to consider that failing to account for different users could make the warnings less effective at reducing safety. For example, some drivers could benefit from earlier alerts if they previously exhibited risky driving behavior. Additionally, people more often become dissatisfied with and desensitized to ill-timed or unnecessary warnings, which can lead drivers to ignore them altogether.
In our , we explored the possibility of enhancing next-generation ADAS with the capability to understand and adapt to different drivers’ styles and preferences. We believe that this would allow the technology to intervene in an unprecedented way and improve safety as a result.
Intelligent vehicle technology holds promise for improving driver safety by applying the correct warnings for particular drivers. Certain cognitive factors, such as impulsivity and inhibitory control, are responsible for a large number of traffic accidents [2]. Inhibitory control is the ability to control and prevent impulsive or automatic responses. For example, most people will reach for their phones instinctively when they see or hear a text notification. This reaction is dangerous if it occurs while driving, so drivers use inhibitory control to ignore the urge to check their phones when a text notification pops up. This ability varies per individual, and the amount of risk depends on the situation.
In our study, we explored the following questions:
- Do cognitive factors relate to how people drive and respond to warning systems?
- Can we build a warning system that can both understand and leverage the human driver’s natural cognitive factors without collecting additional information other than their driving patterns?
- Does a warning system that understands cognitive factors improve safety over a similarly designed but driver-agnostic warning system?
The structure of our study is shown below.
Relationship between cognitive factors and how people respond to Human-Machine Interactions (HMIs)
To answer our first question, “Do cognitive factors relate to how people drive and respond to warning systems?” we conducted an experiment using visual cues. We created a variety of visual HMI modalities, such as virtual lines painted on the road or a yellow dot to regulate speed (similar to a traffic light), to assist the driver with stopping safely. We then designed a driving task in a high-fidelity motion simulator, capturing the “zone-of-dilemma,” which is the decision-making process drivers face when deciding to slow down or speed up for traffic lights. We controlled the presentation of the HMI as per the data collection setup shown below. The hypothesis is that individuals with high inhibitory control are more often inclined to slow down when a light turns yellow than those with low inhibitory control.
In our study of 27 participants, we discovered two findings that validated our first hypothesis. First, people with lower levels of inhibitory control, as measured by questionnaires and psychological tests, were more likely to speed and take risks when reacting to traffic lights. Second, we discovered, with significant statistical significance, that particular HMIs indeed influence humans to change their behavior in response to a traffic light.
People with different levels of cognitive factors should respond differently to HMIs
We then turned to the second question, “Can we build a warning system that can understand and leverage the human driver’s natural cognitive factors with no additional information other than their driving patterns?”
In our study, we leveraged a learned embedding to match a driver’s set of cognitive factors with data on their driving behavior and used that to personalize the presence of the safety warnings. The data required was in line with information available in most modern cars equipped with basic vision detectors and telemetry, as well as a suitable HMI or warning system.
To train this embedding, we adopted an architecture appropriate for time series data — a long-short-term memory (LSTM) network — using driving data obtained from human participants. Only a portion of that data was labeled based on the cognitive factors measured a priori per participant during the study. This is similar to many personalization approaches applied to commerce, medicine, education, and other fields, with the key difference being that the model’s cues were instead taken from driving behavior alone.
The embedding reflects, both qualitatively and quantitatively, the relevant executive functions measures without presuming a single unified taxonomy to describe the driver’s behavior and facilitates reasoning about the drivers’ individual differences.
Building an adaptable system
Proceeding to the final question: “Can a warning system with this understanding improve safety over a similarly designed but driver-agnostic warning system?” we observed the following result.
Using the embedding space that we learned earlier, we constructed a classifier to distinguish users who would benefit from HMI activation from those who would not or whose behaviors would become worse. In other words, we identified users for visual warnings that would decrease the average speed at the zone of dilemma yellow light crossing, as captured via randomized trials.
The resulting classifier enabled us to increase accuracy by 6% in determining a suitable warning HMI selection. Furthermore, basing HMI deployment decisions on this classifier reduced drivers’ average speeds at yellow light crossings by 2.26 m/s.
In summary
Our findings have several significant outcomes. First, we concluded that there exists a positive correlation between certain cognitive factors, such as inhibitory control and baseline driving behaviors, in everyday driving scenarios. Second, we confirmed that a specific visual HMI design can aid those with lower inhibitory control. Lastly, there is evidence to support that a model trained on driving data can identify those individuals with low inhibitory control, allowing the right HMI to be deployed for them.
While our study focused on a particular instance of driving and a specific cognitive factor, we believe there is promise for the broad deployment of personalized assistance technologies. By using in-vehicle optimization to control how much assistance systems interact with drivers to meet their particular safety needs, we are hopeful that this could result in safer roads.
Adapting technology to human drivers is a core focus of the Human Interactive Driving division. To learn more about our work, visit http://jeetwincasinos.com/toyotaresearch/hid/home.
Check out our Nature paper for more specific details of this study at .
References:
[1] World Health Organization, Global status report on road safety (2023).
[2] Department of Transportation, U. S. NHTSA releases 2019 crash fatality data (2019).
