Marta Kasprzak Feb 17, 2021 - Interior sensing

In the blink of an eye – introduction to visual-based DMS approaches

Authors: Marta Kasprzak, Anna Stróż, Anna Olejniczak-Serowiec. Background image source: [16].

 

Distraction and drowsiness are among main causes of road accidents, as they can severely impair driving abilities. Researchers and engineers all around the world are on their way to better understand these phenomena, as well as to develop legal or industrial solutions in order to minimize their impact on road safety.

As three of us work together in the field of driver-related safety measures, we would like to offer you a short series describing drowsiness and distraction measurements, which are most popular in research and DMS applications. Tentatively, we can split them into behavioral, physiological, and performance measures. Today we will dive into visual-based behavioral metrics, which have been thoroughly investigated – even for decades – and finally some of them may reach the market.

Let’s start from the beginning – how drowsiness and distraction can be understood in driving context? Is there any way to parameterize such concepts?

Distraction occurs when driver’s attention is directed towards non-driving activities. This may include interactions with infotainment systems, using mobile phone, talking to passengers, and many more [1]. With the growing number of elements in the environment, both inside and outside of the car, distraction is one of the main factors in road safety. If you are an active driver, you probably have encountered such issues, for example when your mobile phone started ringing or a huge billboard caught your attention for a while.

Drowsiness can be described as a state in which the driver is sleepy or fatigued, however these aren’t exact synonyms. In such state driver’s abilities, like reaction time and decision making, are impaired [2]. Even if the driver isn’t extremally sleepy, there is a risk of microsleeps’ occurrence. Microsleeps are a very short sleep episodes, dangerous especially because the driver might not be aware of experiencing them [3]. Sometimes people claim to be aware of being sleepy, but it isn’t always so, and this is the place where DMS come in handy.

Distraction measurements based on visual methods rely mostly on eyes or face behaviour observations. During driving we look not only straight ahead on the road, but also at different elements connected to the driving task, like mirrors or instrument cluster; the main concern however is to capture signs of visual attention being targeted to non-driving activities. This can be done by tracking the driver’s gaze, but there are also more complex measures. Nevertheless, detecting real distraction can be a challenge.

• Glance characteristics: Glance can be described as a single fixation of the eyes on the target [4]. Different parameters come in handy when speaking about glances, and these can be:
  1. Glance duration, which tells how long the driver is looking at a specific point. The duration counts in the time of moving the sight towards the target (transition time) and the time the gaze is focused on it (dwell time) [4]. It can be used as a single glance duration or be calculated as a total glance time, meaning how long the driver looks at a specific object or a category of objects in selected time unit [5].
  2. Glance frequency, on the other hand, considers not how long, but how often the driver is looking at something during specific time. For example, glance targets can be split into two categories: on-road and off-road glances. As research [6] has shown, drivers tend to look away from forward field of view more often before a safety-critical event occurred.
• Gaze dispersion illustrates the space relations between gaze targets. Most gazes during the drive are distributed along the horizontal axis of the driver’s field of view, with some differences between straight sections and curves [7]. With higher sleepiness the dispersion grows, and a change in gaze patterns can be observed [8].
• Total Eyes-Off-Road Time (TEORT) is the overall time during a set time period (for example a specific task) in which the driver is looking away from the road [4]. The importance of this parameter is usually based on 2 seconds threshold, limiting safe length of looking off the road. Glances above that value have much higher risk of accident occurrence [9].
• Percent Road Centre (PRC) describes what percent of a time unit the driver’s gaze was directed towards the centre of the road. Road centre is usually understood as an area in the forward field of view and is a strategic area for driving safety. PRC is therefore a good indicator of driver distraction [10].

Although these measurements can give us a lot of information about the driver’s attention, a more comprehensive approach, based on general patterns of the driver’s eye behaviour, including on-road glances, is a good solution. Its advantage lies in wider understanding of the road situation’s context, as not all off-road glances are associated with distraction [5].

Drowsiness measurements also use face and head observations but basically depend on features like blinking behaviours instead of gazes. These are one of the most legible indicators of sleepiness and fatigue, therefore are widely used in research and application.

• Blinking characteristics: The patterns in which we blink can give a lot of insight of the state we’re in. Information about how often we blink and how long the blink is are widely used in assessing the drowsiness level.
  1. Blink duration describes how long the blink is. It is a very useful method of detecting drowsiness, as when the driver gets sleepy, his blinks become longer. Electrooculography comes in handy for precise measurements, but this parameter can also be assessed during a video-based observation. Moreover, it is correlated with the subjective sleepiness measured with Karolinska Sleepiness Scale [11].
  2. Blink frequency is a measure of how often the driver blinks in a time unit. Usually, people blink more often if they are sleepier, however this may be affected by individual differences and may vary between people [12].
• PERCLOS is based on blink duration. The name abbreviates from PERcentage of eyes CLOSed, and describes the percent time value of closed eyes in a selected time unit (for example 1 minute). The “closed eye” is usually understood as being closed in more than 80 percent (PERCLOS80) [13] or more than 70 percent (PERCLOS70) [14].
• Eye movement velocity is a value describing how fast the eye moves, usually while changing glance from one point to the other. Just like many eye behaviour characteristics, eye movement velocity is also affected by drowsiness. Generally, the more drowsy the person is, the slower are his/her eye movements [15].
• Head pose/head movement When a person experiences drowsiness, his/her head movements change – keeping the head straight becomes harder, and nodding occurs in some cases [9], but as can be expected, signs of sleepiness like yawning are also taken into consideration [11].

All the above measures are complex issues and can be influenced by many factors. The overview explains them only briefly, as every one of them can be a subject of an article itself. Of course, these are the most popular ones, yet there are several other measures, sometimes being variants of the mentioned ones. As the interest in driving monitoring systems development is growing steadily year by year, such measures – when properly validated – are finally reaching the market, being the next huge step toward improvement in road safety. However, a robust DMS usually draws from numerous metrics tracked simultaneously and integrated.

We hope we gave you an insight on ways of measuring distraction and drowsiness. In the next part of our blog post we will provide you with a summary of yet another approach, based on physiological signals recording, such as electroencephalography (EEG) or galvanic-skin response (GSR).

 

Sources:

 

[1] Distracted Driving. (2021, January 12). NHTSA. https://www.nhtsa.gov/risky-driving/distracted-driving

[2] Dangers of Drowsy Driving. (2020, May 28). Centers for Disease Control and Prevention. https://www.cdc.gov/sleep/features/drowsy-driving.html

[3] Paul, A., Boyle, L. N., Tippin, J., & Rizzo, M. (2005). Variability of Driving Performance During Microsleeps. Proceedings of the Third International Driving Symposium on Human Factors in Driver Assessment, 18–24. https://doi.org/10.17077/drivingassessment.1138

[4] NHTSA. (2013). Visual-Manual NHTSA Driver Distraction Guidelines For In-Vehicle Electronic Devices.

[5] Seppelt, B. D., Seaman, S., Lee, J., Angell, L. S., Mehler, B., & Reimer, B. (2017). Glass half-full: On-road glance metrics differentiate crashes from near-crashes in the 100-Car data. Accident Analysis and Prevention, 107(May), 48–62. https://doi.org/10.1016/j.aap.2017.07.021

[6] Olson, R. L., Hanowski, R. J., Hickman, J. S., & Bocanegra, J. (2009). Driver Distraction in Commercial Vehicle Operations.

[7] Ren, Y. Y., Li, X. S., Zheng, X. L., Li, Z., & Zhao, Q. C. (2015). Analysis of drivers’ eye-movement characteristics when driving around curves. Discrete Dynamics in Nature and Society, 2015. https://doi.org/10.1155/2015/462792

[8] Shiferaw, B. A., Downey, L. A., Westlake, J., Stevens, B., Rajaratnam, S. M. W., Berlowitz, D. J., Swann, P., & Howard, M. E. (2018). Stationary gaze entropy predicts lane departure events in sleep-deprived drivers. Scientific Reports, 8(1), 2220. https://doi.org/10.1038/s41598-018-20588-7

[9] Klauer, S. G., Klauer, S. G., Dingus, T. a., Dingus, T. a., Neale, V. L., Neale, V. L., Sudweeks, J. D., Sudweeks, J. D., Ramsey, D. J., & Ramsey, D. J. (2006). The Impact of Driver Inattention On Near Crash/Crash Risk: An Analysis Using the 100-Car Naturalistic Driving Study Data.

[10] Ahlstrom, C., Kircher, K., & Kircher, A. (2009). Considerations When Calculating Percent Road Centre From Eye Movement Data in Driver Distraction Monitoring. Proceedings of the Fifth International Driving Symposium on Human Factors in Driver Assessment, 132–139. https://doi.org/10.17077/drivingassessment.1313

[11] Anund, A., Fors, C., Hallvig, D., Åkerstedt, T., & Kecklund, G. (2013). Observer Rated Sleepiness and Real Road Driving: An Explorative Study. PLoS ONE, 8(5), e64782. https://doi.org/10.1371/journal.pone.0064782

[12] Khan, M. Q., & Lee, S. (2019). Gaze and Eye Tracking: Techniques and Applications in ADAS. Sensors, 19(24), 5540. https://doi.org/10.3390/s19245540

[13] Wierwille, W. W., Ellsworth, L. A., Wreggit, S. S., Fairbanks, R. J., & Kirn, C. L. (1994). Research on vehicle-based driver status/performance monitoring; development, validation, and refinement of algorithms for detection of driver drowsiness. https://doi.org/10.1037/e526682009-001

[14] Dinges, D. F., Maislin, G., Powell, J. W., & Mallis, M. M. (1998). Evaluation of Techniques for Ocular Measurement as an Index of Fatigue and the Basis for Alertness Management.

[15] Cazzoli, D., Antoniades, C. A., Kennard, C., Nyffeler, T., Bassetti, C. L., & Müri, R. M. (2014). Eye Movements Discriminate Fatigue Due to Chronotypical Factors and Time Spent on Task – A Double Dissociation. PLoS ONE, 9(1), e87146. https://doi.org/10.1371/journal.pone.0087146

[16] Background image source: Pexels from Pixabay.