REAL-TIME ALARM MONITORING SYSTEM FOR DETECTING DRIVER FATIGUE IN WIRELESS AREAS

The purpose of this paper was to develop a real-time alarm monitoring system that can detect the fatigue driving state through wireless communication. The drivers’ electro-encephalogram (EEG) signals were recorded from occipital electrodes. Seven EEG rhythms with different frequency bands as gamma, hbeta, beta, sigma, alpha, theta and delta waves were extracted. They were simultaneously assessed using relative operating characteristic (ROC) curves and grey relational analysis to select one as the fatigue feature. The research results showed that the performance of theta wave was the best one. Therefore, theta wave was used as fatigue feature in the following alarm device. The real-time alarm monitoring system based on the result has been developed, once the threshold was settled by using the data of the first ten minutes driving period. The developed system can detect driver fatigue and give alarm to indicate the onset of fatigue automatically


INTRODUCTION
Driver fatigue is a constant occupational hazard for drivers, which is the major cause of road accidents and has implications for road safety [1][2][3].According to statistics, highway traffic accidents accounted for 11.09% of total number of accidents [4].Many reasons can cause traffic accidents such as driver fatigue, unsafe lane change manoeuvres [5,6], overloading, illegal parking, illegal overtaking, and over-speeding.Driver fatigue is the largest contributor to the highway traffic accidents.The highway has a wide and flat pavement, fewer spatial references and high travel speed traffic.Driving on a highway is free from interference of pedestrians and other low-speed motor vehicles, all vehicles following their respective lanes, moving in orderly fashion on the highway at high speed.Long duration of driving in this monotonous traffic environment requires drivers' sustained attention for long periods of time [7].This is inevitably accompanied by a decrease in alertness and results in performance decrements and a higher risk of accidents [8].
Driver, vehicle and driving environment are the three main factors involved in the driving manoeuvre, with the driver's performance being the most important factor among them.Many studies found that the situation of traffic accidents varies with the age of the driver or the driving age.Drivers younger than 30 years of age are at risk from driving fatigue, one reason being that they are more sleep-deprived than any other age group before driving over long distances on a highway [9].The other reason is the driving attitude of young drivers, who tend to underestimate the risk of crashing (sometimes they do not even realize that they present a risk) while overestimating their driving skills [10,11].Almost 50% of young drivers may continue to drive even when they are aware of their fatigue situation [12,13].Meanwhile, the novice drivers (driving age ≤ 3 years) lack experience, and they accept and process large amounts of information indiscriminately, which can consume a lot of energy and cause fatigue onset in a relatively short time.In the recent years, the fatigue detecting technology has represented a widespread hope in the prevention of fatigue-related accidents.And the major obstacle of this technology is the challenge of establishing an objective, reliable, non-intrusive and real-time fatigue detecting system, which is cited from National Transportation safety improvements.It means that a sensor should warn the driver of fatigue symptom onset, which is needed.At this time, if corrective measures could be taken, then disasters could be prevented.
Physiological features, as mentioned above, contribute significantly to fatigue recognition because a person usually has little control over them, which R. Fu, S. Wang, S. Wang: Real-time Alarm Monitoring System for Detecting Driver Fatigue in Wireless Areas means they could provide reliable and objective source of information to determine person's state of fatigue [14].Methods based on EEG features are perhaps the most accurate, valid and logical ones [15].There are several studies using EEG power spectra to design fatigue driving detection or alarm system.Lin et al. [16] developed and evaluated a real-time auditory alarm system for detection of drowsiness.Budi et al. [1] detected driver fatigue by EEG frequency components.Inspired by Lin et al. and Budi et al., we constructed a fatigue detection system based on optimal EEG feature.Comparing with their previous work, the differences can be summarized by the following points: (i) Lin et al. built an alarm system by visual and biological features.Budi et al. studied the driver fatigue using EEG and EOG features.To avoid the limitation of visual features and trouble of EOG collection, we utilized the biological features from EEG signals.These data were collected in a wireless way.This is one of the differences from previous works.(ii) The previous Lin's and Budi's studies detected driver fatigue through 30 channels of EEG signals.The frequency components as alpha-theta rhythms in Lin et al.'s work and delta-theta-alpha-beta rhythms in Budi et al.'s work were studied.This paper developed a driver fatigue detection system through only 2 channels of EEG signals and theta rhythm.The fewer electrodes and features we use during their driving, the more comfort there is for the drivers.This will enlarge the application possibilities.(iii) The feature selection of Lin's study was realized by multiple trials of EEG processing; however, our study realized feature selection by Relative Operating Characteristic (ROC) curves and Grey Relational Analysis, which is single trial EEG processing method.This makes the fatigue inferring by optimal EEG feature with fewer electrodes.

Participants
Eleven healthy students aged 22 to 31 (24±9.3)were recruited to perform a monotonous driving simulator task on voluntary basis in this study.Participants were selected in compliance with the following strict instructions to be included into the study.The subjects were required to keep regular sleeping hours for one week before the experiment, which ensured that they have more than 7-hour continuous sleeping time (went to bed earlier than midnight and woke up at about 8 a.m.).They were asked to refrain from consuming alcohol, caffeinated drinks, tea or drowsiness-causing medications as well as smoking approximately 12 hours before the study, and they reported compliance with these instructions.In addition, the drivers were generally in good physical and mental health, and they had no physical barriers to ensure safe driving and complete the driving task successfully.Prior to the experiment, the drivers got familiarized with the operation of the driving simulator.

Driving task
This study was conducted around 14:00-17:00 according to former study [17].In the experiment stage, the participants completed a 2-hour driving.While in the application stage, two driving sessions were completed by the drivers.The initial driving session was approximately 10 minutes of alert driving, which would serve as a threshold measure.Following the alert driving session was monotonous driving session, in which participants were required to drive between 40 and 80 km/h continuously.These two sessions involved the participants driving with very few road stimuli.The subjects were instructed to use automatic shift and to refrain from turning on the radio or using other in-car devices.The participants were also instructed to avoid unnecessary movements in order to reduce artefacts in the EEG recording [18].

EEG equipment and data collection
Simultaneous physiological measurements were recorded during the driving sessions.The modular and portable Biofeedback 2000 x-pert system were used in recording the physiological parameters.The 10-20 international standard of electrode placement was applied.Sensors record the signals non-invasively from the brain skin surface placed at O1 and O2.The reasons for choosing sensor locations as O1 and O2 are given as follows.Considering the need of practical application of this research and the results of previous research, EEG signals were collected from O1 and O2 electrodes, which were placed at the very back of the brain (occipital lobe), and collected and interpreted visual images [19].Meanwhile, the electrodes selected in this paper are consistent with the results of many other previous studies on neurophysiology of mental fatigue [20][21][22][23].From the ergonomic point of view, the fewer electrodes we use during their driving, the more comfort there is for the drivers.The reference electrode is placed on the mastoid bone behind the left ear.These sensor signals are filtered, amplified, digitalized and transmitted via a cordless Bluetooth connection to a computer.
The EEG potential is amplified by a difference amplifier with very high input resistance (>2G Ohm) and digitalized using a 24-bit processor with a sampling rate of 200 Hz.The EEG signals were subject to a sharp digital notch filter with stop-band centred at 50Hz.To prevent aliasing, the amplified signal is band-limited before digitization with an eighth-order filter at a cut-off frequency of 40 Hz.In order to minimize the common-mode interference, the reference channel is provided only with a driven right leg circuit.This circuit generates a floating mass that increases the common-mode rejection ratio (CMMR) and provides the necessary reference potential for the EEG recording.The raw signal is divided into the usual EEG frequency bands by means of a Fast Fourier transformation with the buffer width of 256.The FFT provides a spectral analysis of the real and imaginary parts of the recorded and digitalized EEG potential.Then the EEG can be derived as gamma (30)(31)(32)(33)(34)(35)(36)(37)(38)(39)(40), high beta (20-30 Hz), beta (15-20 Hz), sigma (12-15 Hz), alpha (8-12 Hz), theta (4-8 Hz) and delta (0.4-4 Hz).
Karolinska Sleepiness Scale (KSS) was used as the subjective basis for our estimation.In our study, the participants were all in good sleep condition before the driving experiments.However, KSS = 9 represents that a driver is extremely sleepy fighting sleep.This fatigue level is not included in this research because the participants are not sleep deprived.In our experiment, when the drivers finish the driving task, KSS value equals 5 or 7.This means that the participants fall into the drowsy state at that time.We used the participants KSS-report to roughly classify the driver state as alert, mild fatigue, and fatigue, and we obtained three levels of driver states, KSS=1-3 gave the first level; KSS = 3-5 fall into the second level; and KSS=5-7 respond to level three.

Feature extraction
The purpose of feature extraction was to choose one wave from the seven signals with different frequency bands as the fatigue feature.The main concern of this part is to find one wave that can give the best result in identification of fatigue, and the redundant features may have negative impact on the classification performance.Therefore, it is necessary to use the key feature to realize the classification.Once the underlying important feature is obtained and redundant features are removed, the classification problem can be greatly simplified and the more accurate result is given.
The key fatigue feature is worthy of investigation so that the optimal of the feature can be determined.In the current paper, the Relative Operating Characteristic (ROC) curves and Grey Relational Analysis (GRA) are used to identify the optimal indicator of driver fatigue.

Relative Operating Characteristic curves
With the help of the characters of visualization and efficiency of ROC curves, the best way to evaluate the performance of each wave and identify the fatigue feature is the construction of the experimental ROC curves.
The ROC curves are built from a learning set of experimental records where the classification of each record is known as the training data set before the ROC curves are used.These curves can give the relationship between the correct classified records and the incorrect classified ones, which are named as probability of detection and probability of false alarms, respectively.

Grey Relational Analysis
Grey Relational Analysis (GRA) is introduced in this paper, which can be used to determine the optimal feature from the alternative features of driver fatigue.The grey relational grade is calculated to show the relationship among the sequences in the grey relational analysis [24].The proximity degrees of the comparability sequences and ideal result can be also given by the grey relational grade, and the ideal result is represented by using the reference sequence in this algorithm [25].The larger the grey relational grade value of the comparability sequences, the better it is [26].Therefore, a comparability sequence with higher grey relational grade than the other ones is more important than the other comparability sequences to the reference sequence [27][28][29].

System framework
This study is directed at young drivers and chooses highway as experiment scenario.In order to choose one sub-wave as the optimal fatigue feature from EEGbased gamma, hbeta, beta, sigma, alpha, theta and delta waves, the performances of these waves are assessed in the current study.In these waves, theta wave gave an apparent decreasing trend over time.Therefore, theta wave was used as fatigue feature in the following alarm device.We have developed the real-time alarm monitoring system based on the result of theta wave decrease as fatigue onset, once the threshold is settled using the data of the first ten minutes of the driving time.The system can detect driver fatigue and give alarm to warn of the onset of fatigue automatically.Simulated driving experiment and driver fatigue detection system framework are illustrated in Figure 1.

The optimal feature of driver fatigue
As mentioned above, seven waves were obtained.The most important wave needs to be determined to evaluate the driver fatigue.So, before the construction of the online alarm system, the optimal feature of driver fatigue needs to be confirmed based on the off-lines analysis.There are two steps used in the current paper, Relative Operating Characteristic curves and Grey Relational Analysis.

Construction of ROC curves
Generally speaking, ROC curve is the entire set of possible true and false positive fractions attained by dichotomizing a continuous test result T with different thresholds [30][31][32].Thus, ROC curves of the sub-waves can be shown as in Meanwhile, the area under ROC curve of each wave can be plotted in Figure 3.They have almost the same value of AUC; they are 0.935 and 0.938, respectively.According to the evaluation criteria mentioned above, theta wave has a better place than delta wave because theta wave is closer to the upper left hand corner, but the AUC value of delta wave is slightly bigger than theta's.As to theta wave and delta wave, which one can be selected as the fatigue indicator?This contradiction needs to be solved.Grey Relational Analysis GRA uses the information provided by the grey system to compare each factor dynamically and quantificationally and establish a relationship according to the level of similarity factors together with the level of variability.Hence, the decision report can be made by grey relational degree, the sequence of the higher grey relational degree places greater influence on the reference sequence.
In order to determine the best fatigue indicator from theta wave and delta wave, the Grey Relational Analysis is used by the current paper.The steps of applying such an analysis are presented below [33].a) Data normalization: Prior to the GRA, the data were normalized in the range between 0 and 1. b) Form the grey relational matrix X by using the two candidate target sequences: theta and delta.Both of them are contained in two states, that is, alert and fatigue.
( ) ( ) ( ) X t x t x t theta delta theta alert delta alert theta fatigue delta fatigue In the current paper, the evaluation factors are theta and delta wave.They are composed of 5-min data under the alert state and 5-min data under the fatigue state.As mentioned in Section 2.3, the sampling rate is 200 Hz, so the theta and delta with the length of 60,000 are used in grey relational analysis.c) Determine the reference sequence: X=X 0 (t), each element in this sequence is 1 or 2, the fatigue state is assigned to 1 and the alert state is assigned to 2. d) Compute the grey relational coefficients (GRC): the GRC between the sth candidate target sequences, as shown in Equation 1, and reference sequences were obtained by using where ρ∈[0,1] is the resolution ratio, the smaller value of ρ is used for higher accuracy, and 0.5 is the general value and widely accepted [34].e) Calculate the grey relational grade: To obtain the grey relational grade, the average value of grey relational coefficient is computed: The results of the grey relational grades are shown in Figure 4, in which the grey relational grade of theta was larger than that of delta.Therefore, theta was selected as the optimal indicator of driver fatigue.

The driver fatigue online alarm system
Based on the former discussion, the optimal indicator of driver fatigue is settled as theta wave obtained from the occipital lobe.The initial driving session was approximately 10 min of alert driving, which would serve as a threshold measure.Following the alert driving session was the monotonous driving session, in which participants were required to drive continuously between 40 and 80 km/h.The alert and fatigue theta data with the length of 30 seconds were extracted and plotted in one figure as an example, which is shown in Figure 5.And the threshold value is settled as the median by former research and experimentation experience.
The result was obtained that the amplitude of theta wave decreased as the driver was feeling drowsy.This result was consistent with the result given by previous study (Lin et al. [16]).According to this result, we generated synthetic data by Gaussian white noise with different amplitudes.The wave profile can be seen in Figure 6.The wave profile with higher amplitude was generated by Gaussian white noise with mean value as 3.Meanwhile, the wave profile with lower amplitude was generated by Gaussian white noise with mean value as 1.  Figure 6 shows that the amplitude of the wave under the fatigue state is lower than that under the alert state, so that the wave below the threshold is the part we are concerned about, and the audio signal is given by software.However, in the alert section, there also exists a part below the threshold.So the key point of distinguishing the fatigue state is the proportion of the part below the threshold per unit of time, and this unit is settled as 1 min.To avoid the onset of the audio signal which is given by software, the hardware is configured to realize this purpose.It receives the audio signal when the amplitude of theta signal is lower than the threshold, and records the accumulated time of audio signal every minute as t i , which means the accumulated time of audio signal in i-th minute.We can obtain the result that t i in fatigue state is longer than that in alert state, so the mean value of the first five minutes in the alert state is chosen as the base time t; the following t i is compared with t.When t i -t ≥ 10 s, then the alarm is given by hardware.As we know, the randomness and individuality of the biological signals are the barrier of the application, which may be due to the nature of the induced fatigue and characteristics of individual subjects [35], so individual variability was accommodated through individualization of comparing with the base time to reduce the negative effect of biological signals on the application in the current paper.

DISCUSSION AND CONCLUSION
This work focused on developing the real-time alarm monitoring system for the most accident-prone situation that is, young driver age drivers on the monotonous highway, which can detect driver fatigue and give alarm to prevent the onset of fatigue automatically.We collected the EEG signal from the occipital lobe of the subjects according to the research result of predecessors, this means EEG signal can be obtained by using relatively fewer electrodes, because the drivers, especially the long-haul drivers, will feel more comfortable when fewer electrodes are placed on their heads.The optimal indicator of driver fatigue was interesting in this case, so seven EEG based subwaves with different frequency bands were chosen as the evaluation indicators and analyzed here by using Relative Operating Characteristic (ROC) curves and Grey Relational Analysis.In order to show the effectiveness of the proposed feature, ROC curves of other ten participants were plotted in Figure 7. From Figure 7, the curves given by different sub-waves were plotted in the same sub-figure; it can provide the intuitive comparing appreciation of different indices.From these results of different participants, the curves with the bulge shape and near to the upper left hand corner of the figure show that theta and delta have better performance in distinguishing between the alert status and the fatigue status.These show that theta and delta have better performance in distinguishing between the alert status and the fatigue status for almost all participants.The results of the grey relational analysis of all ten participants can be seen in Figure 10.Theta wave with higher grey relational grade was chosen as optimal indicator of driver fatigue.
Theta wave provided better results than other waves in detecting the fatigue state.Now, the statistical analysis of theta is given by Kolmogorov-Smirnov Test (p<0.001).Meanwhile, Figure 11 is the bar graph of theta between the alert and fatigue states.Figure 11 shows the amplitudes of theta waves in fatigue state R. Fu, S. Wang, S. Wang: Real-time Alarm Monitoring System for Detecting Driver Fatigue in Wireless Areas become shorter than when they are in alert state for all participants.These changes show consistency in all participants.However, there still exist individual differences as higher amplitudes of participant 3 and 4.There are 2 in 11 participants who have high amplitude values, which may be caused by high exciting degrees of the participants.This account is acceptable in our utilization.Viewed from an economic or ergonomic aspect with respect to product development, the current study had the following four advantages: (1) This system supplies drivers with an objective evaluation of their ability to drive; they can keep drivers from continuing to drive, and therefore they can reduce the probability of vigilance-related accidents.(2) Accommodate the individual variability by using the algorithm developed to detect driver fatigue with predetermined thresholds obtained by each subject.(3) This system realized the computation and alarm application in a portable wireless hardware.(4) Provide a hardware platform to apply the algorithm in the current paper.Meanwhile, it should be noted that although this study has produced promising results for the detection of driver fatigue automatically, there still exist some challenges that one needs to be aware of.First, the present study was designed to investigate the driving course performed at a fixed time of the day.Future studies can focus on breaking the time limit and performing the driving task at different time durations of day and night.Second, for the purpose of development of the real-time alarm system in this paper, only several simple feature parameters were analyzed.In the future, the acceleration of the speed of online operation can be obtained, and more complex feature parameters can be used in this system to detect the driver fatigue.

Figure 1 -
Figure 1 -Driver fatigue detection system framework

Figure 2 ,Figure 2 -
Figure 2 -ROC curves of the sub-waves ROC curves of different waves can be plotted in the same figure.This can provide the intuitive comparing appreciation of different indices.The curve with the bulge shape and near to the upper left hand corner of the figure means that this index can contribute more to the evaluation value.The area under ROC curve (AUC) is an important statistic, and for these areas, the larger the better.Comparing with the other six waves,

Figure 3 -
Figure 3 -Areas under ROC curves for different waves

Figure 4 -
Figure 4 -Results of the grey relational analysis

Figure 5 -
Figure 5 -Fatigue detection based on theta waves in different states

Figure 6 -
Figure 6 -Fatigue detection illustrated by synthetic data

R.Figure 7 -
Figure 7 -ROC curves combined all sub-waves of ten participants

Meanwhile, Figure 8
illustrated the areas under ROC curves of all ten participants for the different waves.For each participant, theta and delta have higher AUC values.

Figure 8 -
Figure 8 -Areas under ROC curves of all ten participants for the different wavesBoxplots of AUC values for different waves are shown in Figure9.The variability between the participants can be given by the boxplots.For each subwave group, the AUC values of all ten participants are represented in one boxplot, the mean values and median values are shown in blue stars and red circles, respectively.From the change tendency, with both the highest mean and median values, theta wave gives the best performance in discriminant normal and fatigue states.And for different curves, theta wave gives the smallest variability due to the small length value of theta boxplot.From Figure9, it seems that the boxplot of gamma group has the smallest length.However its box is obtained by ignoring two outliers which are shown as red crosses.Combining the smallest variability and the highest AUC statistical values, theta wave is the best choice of feature represented of the driver state.The results of the grey relational analysis of all ten participants can be seen in Figure10.Theta wave with higher grey relational grade was chosen as optimal indicator of driver fatigue.Theta wave provided better results than other waves in detecting the fatigue state.Now, the statistical analysis of theta is given by Kolmogorov-Smirnov Test (p<0.001).Meanwhile, Figure11is the bar graph of theta between the alert and fatigue states.Figure11shows the amplitudes of theta waves in fatigue state

Figure 9 -
Figure 9 -Boxplots of AUC values for different waves