Braingate Technology
Braingate Technology
On
submitted to
Jawaharlal Nehru Technological University
in partial fulfillment of requirements for the award of degree of
BACHELOR OF TECHNOLOGY
in
COMPUTER SCIENCE AND ENGINEERING
Submitted by
SCHOOL OF ENGINEERING
Department of Computer Science and Engineering
2023
CERTIFICATE
Sincerely,
By
1. INTRODUCTION
1
2. PROPOSED WORK 2
2.1 Advantages
3
3.WORKING 4
3.1 How does brain control motor works?
7
3.2 Basic elements of Braingate
8
3.3 Brain Computer Interface(BCI)
11
3.4 BCI versus neuroprosthetics
12
3.5 Software Behind Braingate
13
4. ARCHITECTURE
14
5. IMPLEMENTATION
5.1 Research on Braingate in animals
15
5.2 Human Clinical Trials
16
6. DISADVANTAGES
17
7. APPLICATIONS 18
8.FUTURE SCOPE
19
9.CONCLUSION
20
9.REFERENCES
22
1.INTRODUCTION
[1]
2.PROPOSED WORK
[2]
2.1 ADVANTAGES
The BrainGate Neural Interface System is being designed to one day allow the
user to interface with a computer and or other devices at a level of speed,
accuracy and precision that is comparable to, or even faster than, what is
possible with the hands of a non-disabled person.
The BrainGate System may offer substantial improvement over existing
assistive technologies. Currently available assistive devices have significant
limitations for both the person in need and the caregiver. For example, even
simple switches must be adjusted frequently, a process that can be time
consuming.
In addition, these devices are often obtrusive and may prevent the user from
being able to simultaneously use the device and at the same time establish eye
Fontact or carry on conversations with others.
Potential advantages of the BrainGate System over other muscle driven or
brain-based computer interface approaches include: its potential to interface
with a computer without weeks or months of training; its potential to be used
in an interactive environment, where the user's ability to operate the device is
not affected by their speech, eye movements or ambient noise; and the ability
to provide significantly more usefulness and utility than other approaches by
connecting directly to the part of the brain that controls hand movement and
gestures.
Use your PC
[3]
3. WORKING
The Brain-Gate system is based on specialized technology developed to
sense, transmit, analyse and apply the language of neurons - in essence,
interpret brain signals. The BrainGate Neural Interface device consists of a
tiny chip containing 100 microscopic electrodes that is surgically implanted
in the brain's motor cortex. The whole apparatus is the size of a baby
aspirin. The chip can read signals from the motor cortex, send that
information to a computer via connected wires, and translate it to control
the movement of a computer cursor or a robotic arm. According to Dr. John
Donaghue of Cyberkinetics, there is practically no training required to use
BrainGate because the signals read by a chip implanted, for example, in the
area of the motor cortex for arm movement, are the same signals that would
be sent to the real arm. A user with an implanted chip can immediately
begin to move a cursor with thought alone. However, because movement
carries a variety of information such as velocity, direction, and
acceleration, there are many neurons involved in controlling that
movement. BrainGate is only reading signals from an extremely small
sample of those cells and, therefore, only receiving a fraction of the
instructions. Without all of the information, the initial control of a robotic
hand may not be as smooth as the natural movement of a real hand. But
with practice, the user can refine those movements using signals from only
that sample of cells.
The BrainGate pilot device consists of a Sensor of the size of a contact
lens, a cable and pedestal, which connects the chip to the computer, a cart
which consists the signal processing unit.
The BrainGate Neural Interface Device is a proprietary brain- computer
interface that consists of an internal neural signal sensor and external
processors that convert neural signals into an output signal under the users
own control. The sensor consists of a tiny chip smaller than a baby aspirin,
with one hundred electrode sensors each thinner than a hair that detect
brain cell electrical activity. The sensor chip may be planted in the area of
the brain responsible for body movements.
The chip is implanted on the surface of the brain in the motor cortex area
that controls movement. In the pilot version of the device, a cable connects
[4]
the sensor to an external signal processor in a cart that contains computers.
The computers translate brain activity and create the communication output
using custom decoding software. Importantly, the entire BrainGate system
was specifically designed for clinical use in humans and thus, its
manufacture, assembly and testing are intended to meet human safety
requirements. Five quadriplegics patients in all are enrolled in the pilot
study, which was approved by the U.S. Food and Drug Administration
(FDA).
PREPROCESSING
The raw EEG signal requires some preprocessing before the feature extraction. This
preprocessing includes removing unnecessary frequency bands, averaging the current
brain activity level, transforming the measured scalp potentials to cortex potentials and
demonizing. Frequency bands of the EEG:
BAND FREQUENCY AMPLITUDE LOCATION
Alpha(_) 8-12 10-150 Occipital/parietal
regions
u-rhythm 9-11 varies Precentral/Postcentral
regions
Beta(_) 14-30 25 Typically frontal
[5]
regions
Theta(_) 4-7 varies varies
Delta(_) <3 varies varies
A.Detection
The detection of the input from the user and them translating it into an
action could be considered as key part of any BCI system. This detection
means to try to find out these mental tasks from the EEG signal. It can be
done in time-domain, e.g. by comparing amplitudes of the EEG and in
frequency-domain. This involves usually digital signal processing for
sampling and band pass filtering the signal, then calculating these time -or
frequency domain features and then classifying them. These classification
algorithms include simple comparison of amplitudes linear and nonlinear
equations and artificial neural networks. By constant feedback from user to
the system and vice versa, both partners gradually learn more from each
other and improve the overall performance.
B.Control
The final part consists of applying the will of the user to the used
application. The user chooses an action by controlling his brain activity,
which is then detected and classified to corresponding action. Feedback is
provided to user by audio-visual means e.g. when typing with virtual
keyboard, letter appears to the message box etc.
C. Training
The training is the part where the user adapts to the BCI system. This
training begins with very simple exercises where the user is familiarized
with mental activity which is used to relay the information to the computer.
D. Bio-Feedback
The definition of the biofeedback is biological information which is
returned to the source that created it, so that source can understand it and
have control over it. This biofeedback in BCI systems is usually provided
visually, e.g. the user sees cursor moving up or down or letter being
selected from the alphabet.
[6]
[7]
3.1 How Does the brain control motor function?
The brain is "hardwired" with connections, which are made by billions of neurons that
make electricity whenever they are stimulated. The electrical patterns are called brain
waves. Neurons act like the wires and gates in a computer, gathering and transmitting
electrochemical signals over distances as far as several feet. The brain encodes
information not by relying on single neurons, but by spreading it across large
populations of neurons, and by rapidly adapting to new circumstances.
Motor neurons carry signals from the central nervous system to the muscles, skin and
glands of the body, while sensory neurons carry signals from those outer parts of the
body to the central nervous system. Receptors sense things like chemicals, light, and
sound and encode this information into electrochemical signals transmitted by the
sensory neurons. And interneurons tie everything together by connecting the various
neurons within the brain and spinal cord. The part of the brain that controls motor skills
is located at the ear of the frontal lobe.
How does this communication happen? Muscles in the body's limbs contain embedded
sensors called muscle spindles that measure the length and speed of the muscles as they
stretch and contract as you move. Other sensors in the skin respond to stretching and
pressure. Even if paralysis or disease damages the part of the brain that processes
movement, the brain still makes neural signals. They're just not being sent to the arms,
hands and legs.
A technique called neuro feedback uses connecting sensors on the scalp to translate
brain waves into information a person can learn from. The sensors register different
frequencies of the signals produced in the brain. These changes in brain wave patterns
indicate whether someone is concentrating or suppressing his impulses, or whether he is
relaxed or tense.
[8]
3.2THE BASIC ELEMENTS OF BRAIN GATE ARE:
● The Chip
● The Connector
● The Converter
● Computer
The Chip
A 4-millimeter square silicon chip studded with 100 hairthin,
microelectrodes is embedded in brain primary motor cortex. The chip,
about the size of a baby aspirin, contains 100 electrode sensors, each
thinner than a human hair. The sensors detect tiny electrical signals
generated when a user imagines. Though paralyzed, a quadriplegic still has
the ability to generate such signals -- they just don't get past the damaged
portion of the spinal cord. With Brain Gate, the signals travel through a
wire that comes out of the skull and connects to a computer. Brain Gate
uses technology similar to cochlear implants that help deaf people hear.
[9]
There will be two surgeries, one to implant the BrainGate and one to
remove it. Before surgery, there will be several precautionary measures in
order to prevent infection; patients will have daily baths with antimicrobial
soap and take antibiotics. In addition, MRI scans will be done to find the
best place on the brain for the sensor. Under sterile conditions and general
anesthesia, Doctor will drill a small hole into the skull and implant the
sensor using the same methods as in the monkey studies. Patients will
receive post-surgical care including a CT scan, some blood tests, and
wound care in the hospital for 1 to 5 days after surgery. After surgery, one
of the study doctors will see the patients at least once a week for six weeks,
then monthly and as needed. A nurse will also check the patients regularly
and will always carry a 24-hour pager. The skin around the pedestal will
need to be carefully monitored during the study. Detailed instructions will
be provided so that the patient’s daily care provider can help with skin care.
The sensor of the size of a contact lens is implanted in brain's percental
gyrus which control hand and arm movements. A tiny wire connects the
chip to a small pedestal secured in the scull. A cable connects the pedestal
to a computer. The brain's 100bn neurons fire between 20 and 200 times a
second .The sensor implanted in the brain senses these electrical signals
and passes to the pedestal through the wire. The pedestal passes this signals
to the computer through the cable. The computer translates the signals into
a communication output, allowing a person to move a cursor on a computer
screen merely by thinking about it.
The Connector
It is attached firmly to the skull of the patient and it passes the signals
received by the chip to the converter. Most handicapped people are
satisfied if they can get a rudimentary connection to the outside world.
Brain Gate enables them to achieve far more than that. By controlling the
computer cursor, patients can access Internet information, TV
entertainment, and control lights and appliances – with just their thoughts.
[10]
The Converter
The signal travels through a shoebox-sized amplifier where it's changed to
Digital data and recoiled by fiber-optic cable to a computer.
The Computer
A neuroprosthetic device known as Brain gate converts brain activity into
computer commands. A sensor is implanted on the brain, and electrodes are
hooked up to wires that travel to a pedestal on the scalp. From there, a fiber
optic cable carries the brain activity data to a nearby computer.
[11]
3.3 Brain Computer Interface (BCI)
Brain Gate learns to associate patterns of brain activity with particular
imagined movements - up, down, left, right - and to connect those
movements to a cursor. A braincomputer interface uses
electrophysiological signals to control remote devices. They consist of
electrodes applied to the scalp of an individual. These electrodes pick up
the signals and carry it into amplifier that amplify the signal approximately
ten thousand times and then pass the signal via an analog to digital
converter to a computer for processing. The computer processes the
Electroencephalography (EEG) [5] signal and uses it in order to accomplish
tasks such as communication and environmental control. BCIs are slow in
comparison with normal human actions, because of the complexity and
noisiness of the signals used, as well as the time necessary to complete
recognition and signal processing. Software behind Brain Gate System uses
algorithms and pattern-matching techniques to facilitate communication.
The algorithms are written in C, JAVA and MATLAB. . Signal processing
software algorithms analyze the electrical activity of neurons and translate
it into control signals for use in various computer-based applications.
[12]
3.4 BCI VERSUS NEUROPROSTHETICS
Neuroprosthetics is an area of neuroscience concerned with neural
prostheses — using artificial devices to replace the function of impaired
nervous systems or sensory organs. The most widely used neuroprosthetic
device is the cochlear implant, which was implanted in approximately
100,000 people worldwide as of 2006. There are also several
neuroprosthetic devices that aim to restore vision, including retinal
implants, although this article only discusses implants directly into the
brain.
The differences between BCIs and neuroprosthetics are mostly in the ways
the terms are used: neuroprosthetics typically connect the nervous system,
to a device, whereas the term “BCIs” usually connects the brain (or nervous
system) with a computer system. Practical neuroprosthetics can be linked to
any part of the nervous system, for example peripheral nerves, while the
term "BCI" usually designates a narrower class of systems which interface
with the central nervous system.
[13]
The terms are sometimes used interchangeably and for good reason.
Neuroprosthetics and BCI seek to achieve the same aims, such as restoring
sight, hearing, movement, ability to communicate, and even cognitive
function. Both use similar experimental methods and surgical techniques.
[14]
4.ARCHITECTURE
● A brain–comput1er interface (BCI), sometimes called a direct neural
interface or a brain-machine interface, is a direct communication
pathway between a human or animal brain (or brain cell culture) and
an external device.
● In one-way BCIs, computer either accept commands from the brain
or send signals to it (for example, to restore vision) but not both.
● Two-way BCIs would allow brains and external devices to exchange
information in both directions but have yet to be successfully
implanted in animals or humans.
[15]
4.1 Principle
The principle of operation of the BrainGate Neural Interface System is that with
intact brain function, neural signals are generated even though they are not sent
to the arms, hands and legs. These signals are interpreted by the systems and a
cursor is shown to the user on a computer screen that provides an alternate
"BrainGate pathway". The user can use that cursor to control the computer, just
as a mouse is used.
[16]
The technology driving this breakthrough in the Brain-Machine- Interface field
has a myriad of potential applications, including the development of human
augmentation for military and commercial purposes.
5.IMPLEMENTATION
5.1 Research on Braingate in animals
Experiment on Rats
Rats implanted with BCIs in Theodore Berger's experiments. Several laboratories have
managed to record signals from monkey and rat cerebral cortexes in order to operate
BCIs to carry out movement. Monkeys have navigated computer cursors on screen and
commanded robotic arms to perform simple tasks simply by thinking about the task and
without any motor output. Other research on rats has decoded visual signals.
[17]
In 1999, researchers led by Garrett Stanley at Harvard University decoded neuronal
firings to reproduce images seen by rats. The team used an array of electrodes
embedded in the thalamus (which integrates all of the brain's sensory input) of sharp-
eyed rats. Researchers targeted 177 brain cells in the thalamus lateral geniculate nucleus
area, which decodes signals from the retina. The rats were shown eight short movies,
and their neuron firings were recorded. Using mathematical filters, the researchers
decoded the signals to generate movies of what the rats saw and were able to reconstruct
recognizable scenes and moving objects.
Experiment on Monkeys
Later experiments by Nicolelis using rhesus monkeys, succeeded in closing the
feedback loop and reproduced monkey reaching and grasping movements in a robot
arm. With their deeply cleft and furrowed brains, rhesus monkeys are considered to be
better models for human neurophysiology than owl monkeys. The monkeys were
trained to reach and grasp objects on a computer screen by manipulating a joystick
while corresponding movements by a robot arm were hidden. The monkeys were later
shown the robot directly and learned to control it by viewing its movements. The BCI
used velocity predictions to control reaching movements and simultaneously predicted
hand gripping force.
[18]
● This took place in a specialized residence at The Boston Home in
Dorchester, Massachusetts, where Ms. Hutchinson resided.
● In humans the clinical test was first done in Mathew Nagle in the
year 2004, he was the first person to use the BCI to replace
functionality which was lost because of paralysis.
● He was paralyzed from neck to down. Gerhard Friehs, a
Neurosurgeon implanted the Brain gate chip in Nagle’s brain.
● After this, he was up to control a computer mouse cursor and with
its help he was able to operate T.V, check e-mails
6.DISADVANTAGES
Costly
High risk surgery
Not been wireless yet
Difficulty in understanding and learning
The newest technology is 20bits per minute
[19]
7.APPLICATIONS
In addition to the current patient portfolio, Brain Gate is focused on the interpretation of
neural recordings through software and neural network innovation. For example, a
potential use of this would be study of neurological patterns in a patient with epilepsy.
[20]
Various embodiments of an ambulation and movement assist system are helpful for
those whose legs are paralyzed or their muscles involved in movement are affected. For
example, an ambulation system for a patient may comprise an exoskeleton device
attached to the patient, an FES device at least partially implanted in the patient, and a
biological interface apparatus. The biological interface apparatus comprises a sensor
having a plurality of electrodes for detecting multi cellular signals, a processing unit.
configured to receive the multi cellular signals from the sensor, process the multi
cellular signals to produce a processed signal, and transmit the processed signal to a
controlled device. At least one of the exoskeleton device and the FES device is the
controlled device of the biological interface apparatus. This helps the patient in
achieving movement using these.
BIOLOGICAL INTERFACE SYSTEM WITH SURROGATE CONTROLLED
DEVICE
Various embodiments of a biological interface system and related methods are
disclosed. The system may include a sensor comprising a plurality of electrodes for
detecting multi cellular signals emanating from one or more living cells of a patient, and
a processing unit configured to receive the multi cellular signals from the sensor and
process the multi cellular signals to produce a processed signal. The processing unit
may be configured to transmit the processed signal to a controlled device. The system
further includes a first controlled device configured to send the processed signal, and a
second controlled device configured to receive the processed signal. The first controlled
device may provide feedback to the patient to improve cobtrol of the second controlled
device.
[21]
configured to receive the processed signal. The system may also include a patient
training apparatus configured to receive a patient training signal that causes the patient
training apparatus to controllably move one or more joints of the patient. The system
may be configured to perform an integrated patient training routine to produce the
patient training signal, to store a set of multi cellular signal data detected during a
movement of the one or more joints, and to correlate the set of multi cellular signal data
to a second set of data related to the movement of the one or more joints.
LIMB AND DIGIT MOVEMENT SYSTEM
The system comprises a biological interface apparatus and a joint movement device
such as an exoskeleton device or FES device. The biological interface apparatus
includes a sensor that detects the multi cellular signals and a processing unit for
producing a control signal based on the multi cellular signals. Data from the joint
movement device is transmitted to the processing unit for determining a value of a
configuration parameter of the system. Also disclosed is a joint movement device
including a flexible structure for applying force to one or more patient joints, and
controlled cables that produce the forces required .
DARPA
The Brown University group was partially funded by the Defence Advanced Research
Projects Agency (DARPA), the central research and development organisation for the
US Department of Defence (DoD). DARPA has been interested in Brain-Machine-
Interfaces (BMI) for a number of years for military applications like wiring fighter
pilots directly to their planes to allow autonomous flight from the safety of the ground.
Future developments are also envisaged in which humans could 'download' memory
implants for skill enhancement, allowing actions to be performed that have not been
learned directly.
THE MENTAL TYPEWRITER
Scientists demonstrated a brain-computer interface that translates brain signals into
computer control signals. This application demonstrates how a paralyzed patient could
[22]
communicate by using a mental typewriter alone without touching the keyboard. In the
case of serious accident or illness, a patient's limbs can be paralyzed, severely restricting
communication with the outside world. The interface is already showing how it can help
these patients to write texts and thus communicate with their environment. The person
operating the mental typewriter uses the cursor to select a letters field. The next step
reduces the choice, and after a few more steps we arrive at the individual letters, which
can be used to write words. This process enables simple sentences to be constructed
within minutes. A first prototype of the mental typewriter is currently available. In a
series of experiments, different spelling methods are tested in terms of their usability
and are adapted. It will be some years, though, before the mental typewriter can be used
in everyday applications. Further research is needed, in particular to refine the EEG
sensors.
8.FURTHER SCOPE
● Current new advances include a second-generation interface software
M*Power controller that will enable users to perform a wide variety
of daily activities without assistances of technician.
[23]
● Smaller, wireless device.
● The user will have an improved control of respiratory system, limb
with muscle stimulation or robotics.
9.CONCLUSION
The invention of Brain gate is such a revolution in medical field.
[24]
● The remarkable breakthrough offers hope that people who are
paralyzed will one day be able to independently operate artificial
limbs, computers or wheelchairs.
● The idea of moving robots or prosthetic devices not by manual
control, but by mere “thinking” has been a fascinated approach.
10.REFERENCES
[25]
● G. RAJENDRA KUMAR, SAMUEL VARA PRASADA RAJU and
D. SANTHOSH KUMAR “Brain gate technology”. Oriental Journal of
Computer Science & Technology (2008).
● Sharanbasappa Sali. International Journal of Innovative Research in
Computer and Communication Engineering · June (2017)
● Sandra VA and Devika Ramgopal. “Brain Gate Technology”.
International Journal of Engineering Research and Technology 3.28
(2015): 1-4.
● Levine SP., et al. “A direct brain interface based on event-related
potentials”. IEEE Transactions on Rehabilitation Engineering 8.2
(2000): 180-185.
● Wessberg J., et al. “Real-time prediction of hand trajectory by
ensembles of cortical neurons in primates”. Nature 408.6810 (2000):
361-365.
● Leigh R Hochberg., et al. “Neuronal ensemble control of prosthetic
devices by a human with tetraplegia”. Nature 442.7099 (2006): 164-
171.
● M Parameswari T Sasilatha and K Mahalakshmi P. “Real time Brain
computer interface system. International Journal of recent
technology and engineering”. 7.5S2 (2019): 231-233.
● Bhavani VR., et al. Braingate technology. International Journal of
engineering research and technology 3.12 (2015): 1-3.
● Sharanbasappa Sali and Swapna Ganure. “A Study on Brain Gate
Technology for Bio Medical Applications”. International Journal of
innovative Research in Computer and Communication Engineering
5.4 (2017): 399-403.
● Taylor DM., et al. “Direct cortical control of 3D neuroprosthetic
devices”. Science 296.5574 (2002): 1829-1832.
● Stanley GB., et al. “Reconstruction of natural scenes from ensemble
responses in the lateral geniculate nucleus”. Journal of Neuroscience
19.18 (1999): 8036-8042.
● Vidal JJ. “Towards direct brain-computer communication”. Annual
review of biophysics and bioengineering (1973): 157-180.
[26]