Bio-Medical Materials and Engineering 24 (2014) 1185–1192 1185

DOI 10.3233/BME-130919
IOS Press

Emotion recognition based on the sample

entropy of EEG
Xiang Jiea,b*, Cao Ruia and Li Lia
a
College of Computer Science and Technology, Taiyuan University of Technology, Taiyuan 030024,
People’s Republic of China
b
The International WIC Institute, Beijing University of Technology, Beijing 100022, People’s Repub-
lic of China

Abstract. A sample entropy (SampEn)-based emotion recognition approach was presented. The SampEn results of notable
EEG channels screened by K-S test were fed to the support vector machine (SVM)-weight classifier for training, after which
it was applied to two emotion recognition tasks. One is to distinguish positive and negative emotion with high arousal and the
other genitive emotion with different arousal status. Results showed that channels related to emotions were mostly located on
the prefrontal region, i.e., F3, CP5, FP2, FZ, and FC2. And they were applied to form the input vectors of SVM-weight clas-
sifier. The accuracies of the present algorithm for the two tasks were 80.43% and 79.11%, respectively indicated by the
leave-one-person-out validation procedure, demonstrating that the present algorithm had a reasonable generalization capabil-
ity.

Keywords: Emotion recognition, sample entropy, SVM, brain computer interface, EEG

1. Introduction

Emotion is one common physiological reaction of human to external stimuli. With additional infor-
mation related to operators’ emotion, it is feasible to allow the user-friendly or humanism designing.
Also the change of emotion can be sign of further monitoring or intervention to expression disorder
patients. The recent focus on those issues has stimulated researches on the automatic emotion recogni-
tion.
The direct reflection of emotion should be the facial expression and the speech as well. It requires,
prior to the emotion recognition, tracing of face or separation of speech, which are, however, still open
problems. Evidence has suggested recently that the emotion transformation can lead to variation in
many physiological signals, such as EMG [1], ECG [2], EEG [3-8], skin temperature [2], respiration
[1], and blood volume pulse [1], etc. Those signals have actually been well-studied for decades to
meet various purposes. Also their recordings are easy to be acquired by both commercial and custom-
er-designed equipment.
The emotion is accepted as an activity of cerebral cortex, which maybe provides as a link between it
and EEG. Several EEG-based emotion recognition approaches have been reported in recent years [3-8].

*Corresponding author. E-mail: xiangjie@tyut.edu.cn.

0959-2989/14/$27.50 © 2014 – IOS Press and the authors. All rights reserved
1186 J. Xiang et al. / Emotion recognition based on the sample entropy of EEG

Generally, the emotion recognition, just like other pattern recognition works do, consists of two essen-
tial steps. One is the choosing of features and the other the training of classifier. Researchers have ap-
plied various time domain parameters (e.g., different statistics [5]), frequency domain parameters (e.g.,
signals in different frequency band [3]), mathematical transforms (e.g., Laplace transform and wavelet
transform [8]), or nonlinear quantities (e.g., approximate entropy (ApEn) [9]), as inputs of a couple of
well-studied classifiers including the support vector machine (SVM) [2-4, 6, 9], back-propagation neu-
ral network [5], and linear classifier [8], etc.
However, it is relatively hard to directly make any appropriate evaluation on those algorithms, since
they were usually studied in different data sets, or they were designed for different recognition purpos-
es. Also, their high accuracy was usually showed by a relatively small data set (mostly 12 subjects) [7].
In addition, it is now widely-accepted that the EEG data, similar to any other physiological signals, are
outputs of the inherent nonlinear physiological systems. The introducing of nonlinear indices should
probably increase the recognition accuracy. Researchers have made attempts by applying ApEn. How-
ever, ApEn has already been showed to lack relative consistency and robustness. It is biased estima-
tion because of self-matching. The sample entropy (SampEn) statistic is, by contrast, a more robust
and consistent index without self-matching. The SampEn has already been applied successfully to re-
searches on motor imagery classification, identification of epileptic, sleep, concentration, etc. [10-15].
Therefore, we aimed, in the present work, to apply the SampEn results of EEG in different channels
as the features to distinguish different emotional states. The SVM classifier was trained in an open
database for the consideration of reproducibility. And finally, the validity of the present algorithm will
be showed by cross-validation procedure.

2. Method

This study included three main stages, which are briefly summarized in Fig. 1. The first stage was
the preprocessing of EEG data. Next one was to extract features from the preprocessed EEG data, and
to screen notable EEG channels by K-S test. Finally, the classifier was trained and tested by two vali-
dation procedures.

Fig. Schematic representation of the different stages of this study

 J. Xiang et al. / Emotion recognition based on the sample entropy of EEG 1187

2.1. DEAP database

The database consists of numerous EEG episodes recorded from 32 subjects when watching 40 pe-
riods of music videos, which can well stimulate various emotions [16, 17]. They last for 1 min; thus all
EEG episodes are recorded for 63 s, including baseline data of 3 s. Each contains 40 channels and is
sampled at 512 Hz. Prior to data collection, these videos have been tagged by the Valence-Arousal-
Dominance (VAD) model with different emotions through a number of behavioral experiments. In
terms of values in the first 2 dimension (VA), each video can be divided into 4 types of emotions, i.e.,
high arousal high valence (HAHV), low arousal high valence (LAHV), high arousal low valence
(HALV), and low arousal low valence (LALV). In addition, all subjects have evaluated those videos
by VAD according to their own feels when watching them.

2.2. EEG signal preprocessing

EEG data are always contaminated by various noises and artifacts, i.e., frequency interference, blink
and eye movement, and artifacts from other physiological signals including respiration, EOG, ECG,
and EMG, etc. EOG appears very significant in the forebrain regions. It serves as the main interfer-
ence which has usually very high amplitude (often 10~100 times higher than that of normal EEG sig-
nals). Thus the artifacts removal is essential to subsequent analysis. In our study, the EEG signals we
used have already been preprocessed by provider.
It is usually accepted that the  band (13~30 Hz) of EEG data is related to the mental and emotional
tension [18]. Thus frequency filtering based on the wavelet package decomposition [19] was finally
applied to filter out the  band from each EEG episode for all following analysis.
To assure that each subject has settled down to certain emotions, we always removed the first 23 s
of each EEG episode. Also, EEG of the last 20 s was removed subsequently to avoid the possible fa-
tigue effects.

2.3. Feature extraction

SampEn of each ECG episode was applied in this study. It can capture the irregularity of time-series,
which means the likelihood that runs of patterns that are close remain close on next incremental com-
parisons [20]. It has been proved to be convergent for relatively short and noisy data. We summarized
the SampEn algorithm for a time-series {v ( i ) ,1 ≤ i ≤ N } of N points as follows.
(1) Form vectors X m ( i ) = ª¬v ( i ) , v ( i + 1) ,, v ( i + m − 1) º¼ where i = 1, 2, , N − m and m is the em-
bedding dimension.
(2) Define the distance between two such vectors as

( )
d ª¬ X m ( i ) , X m ( j ) º¼ = max v ( i + k ) − v ( j + k ) , (1)

where k = 0,1, , m − 1 .
(3) Define Bim ( r ) as the average number of X m ( j ) within r of X ( i ) , where

Bim ( r ) =
{ }
No. d ª¬ X m ( i ) , X m ( j ) º¼ < r
, (2)
N − m −1
1188 J. Xiang et al. / Emotion recognition based on the sample entropy of EEG

and r is the threshold value, j = 1, 2, , N − m, j ≠ i .

(4) Define the average of Bim ( r ) for i = 1, 2, , N − m as

1 N −m m
Bm ( r ) = ¦ Bi ( r ) . (3)
N − m i =1
(5) Repeat steps (1) to (4) for m + 1 and define B m +1 ( r ) according to B m ( r ) .
(6) Define SampEn as
B m +1 ( r )
SampEn ( m, r ) = − ln . (4)
Bm ( r )

Thus its calculation requires two pre-defined parameters̾ m and r . The first one determines the
embedding dimension, which is usually set at 1 or 2, whereas r defines the criterion of similarity
which is usually set at 0.1 to 0.25 times the standard deviation (SD) of the original time-series. We
chose m = 2 and r = 0.2 * SD in this study.
Notable channels were then screened after the calculation of SampEn, since not all channels can ef-
fectively reflect the emotion-related characteristics. The SampEn results of the notable channels were
finally used as the classification features. Since the SampEn results did not always follow the normal
distribution, we did not employ the paired t -test as had done in [21, 22] which could only work under
the assumption of normal distribution. We used the Kolmogorov-Smirnov (K-S) test [23] in our study
for screening channels, since evidence showed that an effective and stable nonlinear statistic was high-
ly robust against noise and also highly sensitive to nonlinear data [24].

2.4. Classifier training and validation

The SVM algorithm has already been proved to be effective for kinds of pattern recognition tasks in
various researches [25-27]. It is capable to manipulate relatively small samples with good generaliza-
tion capability. Considering the possible unbalance of samples, we applied the SVM-weight algorithm,
which is a combination of cost-sensitive learning and SVM. Its main idea is to allocate large punish-
ment factors to important classes (minority class), so as to guarantee that the optimal hyper plane is
closer to the majority class. A couple of studies have proved its effectiveness to unbalanced problems
[28, 29].

3. Results

3.1. Valid samples

VAD values evaluated by subjects may not always be in line with the pre-defined results due to var-
ious reasons. EEG episodes with these kinds of VAD values were excluded. Table 1 shows the valid
samples in each kind of emotions. They are indeed unbalanced. Samples of HAHV were significantly
higher than those of HALV. They were pre-processed and SampEn of each sample was calculated af-
terwards.
 J. Xiang et al. / Emotion recognition based on the sample entropy of EEG 1189

Table 1
Valid samples

Emotion type HAHV LAHV LALV HALV

Video numbers 152 100 81 72

3.2. Notable channels

Notable channels between HAHV and HALV, as well as those between LALV and HALV, are
showed in Table 2 and 3, respectively. It shows that channels F3, CP5, FP2, FZ, and FC2 have signifi-
cant different SampEn results between HAHV and HALV (all p<0.05). And SampEn results in FP1,
T7, and AF4 are significant between LALV and HALV (all p<0.02), accordingly.
Table 2
Notable channels between HAHV and HALV
Electrode number Channel Name p
3 F3 0.04
9 CP5 0.04
17 Fp2 0.04
19 Fz 0.03
23 FC2 0.04
Table 3
Notable channels between LALV and HALV

Electrode number Channel Name p

1 Fp1 0.02
8 T7 <0.01
18 AF4 <0.01

3.3. Classification accuracy

Fig. 2 Results of 3-fold cross-validation

SampEn results of channels F3, CP5, FP2, FZ, and FC2 were used to form the input vectors for
recognition of positive and negative emotions with high arousal (HALV-HAHV), and those of FP1,
1190 J. Xiang et al. / Emotion recognition based on the sample entropy of EEG

T7, and AF4 were applied IRU recognition of negative emotions with different degree of arousal
(LALV-HALV). Results were finally validated by both the 3-fold cross-validation and the leave-one-
person-out (LOPO) procedures, which are showed in Fig. 2 and 3, respectively. The average accura-
cies for the recognition of HALV-HAHV and LALV-HALV are 80.43% and 71.16% according to 3-
fold cross-validation. And when using LOPO, they result to 79.11% for HALV-HAHV and 64.47%
for LALV-HALV, respectively.

Fig. 3 Results of LOPO

4. Discussion and conclusion

Notable channels are mostly located in the prefrontal region as shown in Table 2 and 3, which are
consistent with previous studies in cognitive neuroscience [30, 31]. It is also similar to previous EEG-
based emotion recognition researches [4, 9, 32], which have based the recognition task on other fea-
tures.
The average accuracies shown in Fig. 2 and 3 are higher than the random level, indicating the capa-
bility of SampEn to distinguish emotions with different arousal and valence states. Besides, the accu-
racies showed by the 3-fold cross-validation are much higher than those shown by LOPO. The main
reason maybe lies in the principle of LOPO procedures̾features of training set are used to identify the
emotion of a new subject. The training and testing sets are different, whereas all samples are randomly
divided into training and testing sets in the 3-fold cross-validation procedure; thus the training and
testing samples can contain data from the same subject. In addition, the relatively high accuracies in-
dicated by LOPO suggest that our algorithm has reasonable generalization capability.
To summarize, a SampEn based emotion recognition approach was studied in this work. SampEn
results of notable channels screened by K-S test were fed to the SVM-weight classifier for training,
after which it was applied to distinguish emotions with different arousal and valence states. Reasona-
ble recognition accuracy was showed by rigorous validation procedures. Future studies will concen-
trate on the influence of EEG data length and frequency bands. Also, improving the performance of
SampEn will also be focused.
 J. Xiang et al. / Emotion recognition based on the sample entropy of EEG 1191

5. Acknowledgments

This study was supported by the National Natural Science Foundation of China (61170136,
61373101), the Natural Science Foundation of Shanxi Province (2011011015-4), and the Beijing Post-
doctoral Research Foundation (Q6002020201201).

References

[1] Picard R W, Vyzas E, Healey J. Toward machine emotional intelligence: Analysis of affective physiological state. IEEE
Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(10): 1175-1191.
[2] Kim K H, Bang S W, Kim S R. Emotion recognition system using short-term monitoring of physiological signals. Med-
ical & Biological Engineering & Computing, 2004, 42(3): 419-427.
[3] Li M, Lu B L. Emotion classification based on gamma-band EEG. // Engineering in Medicine and Biology Society,
2009. Annual International Conference of the IEEE. Minnesota: IEEE, 2009: 1223-1226.
[4] Petrantonakis P C, Hadjileontiadis L J. Emotion recognition from EEG using higher order crossings. IEEE Transactions
on Information Technology in Biomedicine, 2010, 14(2): 186-197.
[5] Yuen C T, San W S, Seong T C, et al. Classification of human emotions from EEG signals using statistical features and
neural network. International Journal of Integrated Engineering, 2011, 1(3): 71-79.
[6] Nie D, Wang X W, Shi L C, et al. EEG-based emotion recognition during watching movies. //Neural Engineering
(NER), 2011 IEEE/EMBS 5th International Conference on Neural Engineering, 2011: 667-670.
[7] Murugappan M, Rizon M, Nagarajan R, et al. Inferring of human emotion states using multichannel EEG. International
Journal of Soft Computing and Applications , 2009, 48(2): 281-299.
[8] Murugappan M, Nagarajan R, Yaacob S. Combining spatial filtering and wavelet transform for classifying human emo-
tions using EEG Signals. Journal of Medical and Biological Engineering, 2011, 31(1): 45-51.
[9] Hosseini S A, Naghibi-Sistani M B. Emotion recognition method using entropy analysis of EEG signals. International
Journal of Image, Graphics and Signal Processing, 2011, 3(5): 30-36.
[10] Bai D, Qiu T, Li X. The sample entropy and its application in EEG based epilepsy detection. Journal of Biomedical
Engineering, 2007, 24(1): 200-205.
[11] Zhou P, Ge J, Cao H, et al. Classification of motor imagery based on sample entropy. Information and Control -
Shenyang-, 2008, 37(2): 191-196.
[12] He W X, Chen X P, Shao J T. Sleep EEG staging based on sample entropy. Journal of Jiangsu University (Natural Sci-
ence Edition), 2009, 30(5): 501-504.
[13] Liang X H, He W X, Ma D L. Analyzing sleep EEG using correlation dimension and sample entropy. Microcomputer
Information, 2008, 24(9-3). 220-222.
[14] Ge J, Zhou P, Zhao X, et al. Research on sleep EEG time-series using nonlinear sample entropy. Chinese Journal of
Electron Devices, 2008, 31(3): 972-975.
[15] Nan Y, Jue W, Na W, et al. Feature exaction and classification of attention related electroencephalographic signals
based on sample entropy. Journal of Xian Jiaotong University, 2007, 41(10): 1237-1241.
[16] Koelstra S, Muhl C, Soleymani M, et al. Deap: A database for emotion analysis; using physiological signals. IEEE
Transactions on Affective Computing, 2012, 3(1): 18-31.
[17] DEAP Dataset summary. http://www.eecs.qmul.ac.uk/mmv/datasets/deap/readme.html.
[18] Güntekin B, Baar E. Event-related beta oscillations are affected by emotional eliciting stimuli. Neuroscience Letters,
2010, 483(3): 173-178.
[19] Han Q. Evaluation of human mental stress states based on wavelet package transformation and nonlinear analysis of
EEG signals. Journal of Vibration and Shock, 2013, 32(2): 182-188.
[20] Richman J S, Moorman J R. Physiological time-series analysis using approximate entropy and sample entropy. Ameri-
can Journal of Physiology-Heart and Circulatory Physiology, 2000, 278(6): H2039-H2049.
[21] Wang D, Miao D, Blohm G. Multi-class motor imagery EEG decoding for brain-computer interfaces. Frontiers in Neu-
roscience, 2012, 6: 151.
[22] Wang D, Miao D, Blohm G. A new method for EEG-based concealed information test . IEEE Transactions on Infor-
mation Forensics and Security, 2013, 8(3): 520-527.
[23] Wu S, Wang Z. Non-parametric statistical methods. Beijing: Higher Education Press, 1996: 144-150.
1192 J. Xiang et al. / Emotion recognition based on the sample entropy of EEG

[24] Hou S, Li Y, Liu G. A new method of detecting nonlinear for time series based on KS test. Journal of Electronics &
Information Technology, 2007, 29(4): 808-810.
[25] Mitchell T , Hutchinson R , Niculescu R S , et al . Learning to decode cognitive states from brain images. Machine
Learning, 2004, 57: 145-175.
[26] Pereira F, Mitchell T, Botvinick M. Machine learning classifiers and fMRI : A tutorial overview. NeuroImage, 2009,
45(1): S199-S209.
[27] Zhang J D, Qin G H, Cui Y. An EEG signal system based on fast ICA and SVM. Journal of Computer Research and
Development, 2008: S1.
[28] Tang Y, Zhang Y Q, Chawla N V, et al. SVMs modeling for highly imbalanced classification. IEEE Transactions on
Systems, Man, and Cybernetics, Part B: Cybernetics, 2009, 39(1): 281-288.
[29] Akbani R, Kwek S, Japkowicz N. Applying support vector machines to imbalanced datasets. //Machine Learning: Eu-
ropean Conference on Machine Learning Springer 2004. Springer Berlin Heidelberg, 2004: 39-50.
[30] Davidson R J. Anterior cerebral asymmetry and the nature of emotion. Brain and Cognition, 1992, 20: 125 -151.
[31] Davidson R J. Childhood temperament and cerebral asymmetry: A neurobiological substrate of behavioral inhibition. In
K H Rubin & J B Asendorpf (Eds), Social Withdrawal, Inhibition, and Shyness in Childhood, 1993: 31-48.Hillsdate,
NJ: Erl-baum.
[32] Hoseingholizade S, Golpaygani M R H, Monfared A S. Studying emotion through nonlinear processing of EEG. Proce-
dia-Social and Behavioral Sciences, 2012, 32: 163-169.
Copyright of Bio-Medical Materials & Engineering is the property of IOS Press and its
content may not be copied or emailed to multiple sites or posted to a listserv without the
copyright holder's express written permission. However, users may print, download, or email
articles for individual use.