CWERDFORON BOARD BRAIN-COMPUTER INTERFACE

ACTIVITY
BASED OoN THE RCOGNITIONOF PATTERNS OF BRAIN
THROUGH A CNN

THESIS
SUBMITTED TO THE
RAFFLES UNIVERSITY,
FORTHE DEGREE
OF
DOCTOR OF PHILOSOPHY

IN
COMPUTER SCIENCE & ENGINEERING

RAFFLES
UNIVERSITY

BY
Rajendra Singh
(Registration No 21018T001)
Under the supervision of
Dr. Shiv Kant
SCHOOL OF ENGINEERING &TECHNOLOGY
RAFFLES UNIVERSTY, NEEMRANA
ALWAR, RAJASTHAN- 301705INDIA

Year202324
«4 NEW METHODFOR ON-BOARD BRAIN-COMPUTER
INTERFACE
BASED ON THE RECOGNITION OF PATTERNS OF BRAIN ACTIVITY
THROUGH A CNN"
A

THESIS

SUBMITTED TO THE

RAFFLES UNIVERSITY,
FOR THE DEGREE

OF

DOCTOR OF PHILOSOPHY
IN

COMPUTER SCIENCE & ENGINEERING

RAFFLES
UNIVERSITY

BY

Rajendra Singh
(Registration No. 2101ETOO1)
Under the supervision of
Dr. Shiv Kant

SCHOOL OF ENGINEERING & TECHNOLOGY

RAFFLES UNIVERSITY,NEEMRANA

ALWAR, RAJASTHAN -301705

Year 2023-24
 Declaration by the Candidate

New Method for On

Ihereby declare that the work presented in this report entitled $A
Pattern of Brain
Board Brain-Computer Interface based on the Recognition of
matter
Activity through a CNN", was carried out by me. I have not submitted the

embodied in this report for the award of any other degree or diploma
of any other

University or Institute.
words, ideas, diagrams,
Ihave given due credit to the original authors/sources for all the

graphics, computer programs, experiments, results, that are my original contribution. I

credit to the original
have used quotation marks to identify verbatim sentences and given
authors/sources.

experiments and results

Iaffirm that no portion of my work is plagiarized, and the
plagiarism and
reported in the report are not manipulated. In the event of a complaint of
and
the manipulation of the experiments and results, I shall be fully responsible
answerable.

Name : Rajendra Singh

Enroll. No. :2101ETO01

(Candidate Signature)

()
 Certificate

This is to certify that the thesis entitled *A New Method for On-Board Brain
Computer Interface based on the Recognition of Pattern of Brain Activity Through
A CNN" submitted by Mr. Rajendra Singh for the award of the degree of Doctor of
Philosophy to the Raffles University, Neemrana, Rajasthan, is a record of a bonafide
research work carried out by him under my guidance. To the best of my knowledge: (i)
The candidate has not submitted the same research work to any other institution for any

degree/diploma, Associate ship, Fellowship, or other similar titles, (ii) The thesis
submitted is a record of original work done by the student during the period of study
under my supervision, and (iii) the thesis represents independent research work on the
part of the student.

Inmy opinion, this thesis is of the standard required for the award of "Doctor of
Philosophy".

Shwlant
Dr. Shiv Kant,

Associate Professor, School of Engineering and Technology, Rafles University

Place: Neemrana) Date:

(i)
 Abstract

Quadriplegia, Brain Stem Stroke, Spinal Cord Injury, Amyotrophic Lateral Sclerosis,
Muscular dystrophy, and Multiple Sclerosis may impair the neurological route through
which the brain interacts with and controls the limbs. In such circumstances, mobility
limitation may result. Wheelchairs and other mobility aids need muscular inputs from the
upper body to operate, which individuals with motor disability cannot give.
Consequently,there is aneed for amobility aid whose control is independent of physical
effort.

Brain-computer interface (BCI), a cutting-edge technology that interprets the

user's brain activity to operate an external device,may be utilized as an alternate control
for mobility assistance. Motor-impaired individuals may use motor imagery (MI)
Electroencephalography (EEG) scalp signals to move their limbs when they visualize
moving their limbs. Consequently, an effective signal analysis system for recognizing the
user's intent from the MI EEG data for theoperation of a mobility-assistive device is
investigated in this thesis.
The Common Spatial Pattern (CSP) spatial filtering technique is a commonly used pre
processing and feature extraction approach that collects discriminant MI patterns in MI
based BCI systems. The initial contribution of this thesis is to explore the use of
ERD/ERS (event-related desynchronization/event-related synchronization) information
for processing to improve the performance of CSP-based MI classification. In addition,
multiclass MIEEG data are obtained using adefined acquisition technique and analyzed
todetermine the efficacy of the proposed processing and classification system. Using a
classification accuracy measure fora publiclyaccessible dataset and an acquired dataset,
the performance of the proposed two-class (left-hand vs. right-hand) MI classifier is
compared. ERD/ERS plots are used to determine the subject-specific time segment and
frequency band for preprocessing the MI EEG data. This improves the performance of
the CSP method.

(iii)
 Acknowledgement

and assistance of numerous

This Doctor of Philosophy thesis is the result of the work
individuals, to whom I am extremely grateful.
Without you, this would not have been

possible.
Shiv
great pleasure to express mny sincere gratitude to my supervisor, Dr.
It gives me
School of Engineering and Technology, Raffles University, in
Kant, Associate Professor,
unwavering support and faith in me,
they personally and
the first place. His
His able
have always inspired me to push myself and exceed my limits.
professionally,
encouragement provided me with
an extraordinary learning
guidance and constant
insightful
ultimately led to the completion of this project. Their
experience, which aided my
different stages of my research have greatly
comments and suggestions
at
counsel will assist me in becoming a competent
development. His
experiences and
individual.
and a humble (Dean) &
Dr. Jaspreet Hira
researcher
Technology,
Schoolof Engineering and
Iam grateful to the supportive and encouraging
University, Neemrana for being so
faculties at Raffles
throughout my journey. Manisha
and gratitude to my wife Mrs.
thanks
journey, Iowe special supportive,
During my Ph.D. always being so
unforgettable friends for
members and
Gurjar and family environment."
providing a "stress buster
caring,and helpful
and for

Rajendra Singh
Table of Contents
ii
Abstract..
Listof Table.... ..xi
...xi
List of Figures..
........XVi
List of Abbreviations.
Chapter1- INTRODUCTION....
1.1.INTERFACE BETWEEN THE BRAIN AND COMPUTER (BCI).... ..2.
1.2.ANATOMICAL STUDY OF THE BRAIN eee

1.3.NEUROIMAGING FOR BCI SYSTEMS: CONCEPTUAL FRAMEWORKS.....5

.ese..6
1.3.1. The electrocardiogram (ECoG).
1.3.2. Recording of Neurons in the Cortices... 6
1.3.3. Functional Magnetic Resonance Imaging (tMRI)....
1.3.4. Functional Near Infrared Spectroscopy (fNIRS)... ......7

1.3.5. Magneto encephalography (MEG)....

1.3.6. Electroencephalography (EEG).
1.4.USE OF EEG SIGNALS FOR BCI.
1.4.1. The Steady-state Evoked Potentials (SSEP)......... ... 11
1.4.2. P300.. ...12
12
1.4.3. Slow Cortical Potentials (SCP)...
*...... J2
1.4.4. Sensorimotor Rhythms (SMR).
1.5.MI-BASED BCI SYSTEMS FOR MOBILITY ASSISTANT DEVICE CONTROL
. . . . .15

1.6.ESSENCE OF THE WORK... .15

1.7.PURPOSE OF THE WORK. .16
1.8.QUICK SUMMARY OF THE WORK... .... .16
1.9.ORGANIZATION OF THE DISSERTATION.... . .17

Chapter 2 - LITERATURE REVIEW. 18

2.1. INTRODUCTION 18

2.2, SCHOLARLY WORKS ON BRAIN-COMPUTER INTERACTION (BCI).....18

2.3. EEG-BASED STUDIES ON BRAIN ACTIVITY MEASUREMENT FOR BCI.21
2.4. EEG DATA OF MOTOR IMAGERY (MI) FOR BCI SYSTEMS.. ...24
2.5. ALGORITHMSFOR MI EEG SIGNAL PROCESSING AND
CLASSIFICATION ..27

2.5.1 Literature on Common Spatial Pattern (CSP) spatial filter variations ....28
2.5.2 CIassifier Algorithms - The Literature ...31
2.6. INFERENCE -LITERATURE REVIEW... .34
2.7. THE FORMULATION OF THE RESEARCH PROBLEM .34

(vii)
 Chapter 1

INTRODUCTION

Locomotors disability is an impairment of the bone, ligaments, or muscles that

which
restricts the movement of the limbs. Disruptions in neuromuscular channels,
regulate the brain-limb relationship, may cause this. Mobility may be limited by
spinal cord stroke, quadriplegia, ALS, ruptured disc, MS or Muscular Dystrophy.
population of
1.4% of the impaired population in India is affected by 3.75 lakhs
locomotors disability according to the data of persons with disabilities
in India (2021).
electric or
Such individuals need help with walking and mobility. These people use
mobility. However, the use of
manual wheelchairs, canes and walkers to improve their
which not all individuals with
these devices requires physical or muscular control,
necessary to research alternate
mobility disability can give. Consequently, it is
muscle-free controls for controlling mobility devices.

records brain activity to interpret

A cutting-edge brain-computer interface (BCI)
Motor-impaired individuals with a BCI may
human intent and run an external device.
mobility-assistive equipment. For a better
use it as an alternate control for their
and the
understanding of using BCI to control mobility aids, a short overview of BCI
brain activity and controls mobility aids.
following sections describe how BCI records