W

earing a blindfold and noise-­

canceling headphones, Igor
Spetic gropes for the bowl in front
of him, reaches into it, and picks
up a cherry by its stem. He uses
his left hand, which is his own flesh and blood.
His right hand, though, is a plastic and metal
prosthetic, the consequence of an industrial acci-
dent. Spetic is a volunteer in our research at the
Louis Stokes Cleveland Veterans Affairs Medi-
cal Center, and he has been using this
“myoelectric” device for years, control-
Restoring the ling it by flexing the muscles in his right
arm. The prosthetic, typical of those
Human Touch used by amputees, provides only crude
control. As we watch, Spetic grabs the
cherry between his prosthetic thumb
and forefinger so that he can pull off the stem.
Instead, the fruit bursts between his fingers.¶
Next, my colleagues and I turn on the haptic sys-
tem that we and our partners have been devel-
Prosthetics oping at the Functional Neural Interface Lab at
imbued with Case Western Reserve University, also in Cleve-
haptics give land. Previously, surgeons J. Robert Anderson
their wearers and Michael Keith had implanted electrodes in
fine motor Spetic’s right forearm, which now make contact
control and with three nerves at 20 locations. Stimulating
a sense of different nerve fibers produces realistic sensa-
connection tions that Spetic perceives as coming from his
missing hand: When we stimulate one spot, he
By Dustin J. Tyler feels a touch on his right palm; another spot
Photography by
Mike McGregor produces sensation in his thumb, and so on.��

28  | may 2016 | North American | SPECTRUM.IEEE.ORG

05.HapticProsthetics.NA [P].indd 28 4/13/16 11:40 AM

05.HapticProsthetics.NA [P].indd 29 4/13/16 11:40 AM
 for more than a few weeks in the missing limb. We’re now
working toward a fully implantable system, which we hope
to have ready for clinical trials within five years.
What would adding a sense of touch to prosthetics do? Right
now, people with prostheses typically can use their fake limbs
only for tasks that don’t require precision, such as bracing
and holding. The sensory feedback from our haptic system
would improve control and confidence, allowing greater
use of the prosthesis for all the many small tasks of daily life.
Beyond that, we hope to restore one of the most basic
forms of human contact. Imagine what it must be like to
lose your sense of touch—touch gives us such a profound
sense of connection to others. When we ask Spetic and other
prosthetic wearers how to improve their mechanical limbs,
universally they say they want to hold a loved one’s hand
and really feel it. Our technology should one day enable
them to achieve this very human goal.

I
’ve spent my entire career studying the
marriage between human and machine. My work at
the intersection of biomedical engineering and neu-
ral engineering has driven me to seek the answers
to some basic questions: How can electronic cir-
cuits speak to the nervous system in a way that the nervous
system will understand? How can we use that capability
to restore a sweeping range of sensations to someone who
has lost a hand? And how can that technology be used to
enhance and augment other people’s lives?
WITH THESE TWO HANDS: Igor Spetic, who lost his right hand in an The past few decades have seen remarkable advances in
industrial accident, can feel sensations in his missing hand, thanks the field, including better hardware that can be implanted
to an experimental haptic system developed by the author’s group. in the brain or body and better software that can under-
stand and mimic the natural neural code. In that code, elec-
To test whether such sensations would give Spetic bet- trical impulses in the nervous system convey information
ter control over his prosthetic hand, we put thin-film force between brain cells or along the neurons in the peripheral
sensors in the device’s index and middle fingers and thumb, nerves that stretch throughout the body. These signals
and we use the signals from those sensors to trigger the cor- drive the actuators of the body, such as the muscles, and
responding nerve stimulation. Again we watch as Spetic they provide feedback in the form of sensation, limb posi-
grasps another cherry. This time, his touch is delicate as he tion, muscle force, and so on.
pulls off the stem without damaging the fruit in the slightest. By inserting electrodes directly into muscles or wrap-
In our trials, he’s able to perform this task 93 percent of ping them around the nerves that control the contraction
the time when the haptic system is turned on, versus just of the muscles, we can send commands to those electrodes
43 percent with the haptics turned off. What’s more, Spetic that roughly replicate the signals associated with moving a
reports feeling as though he is grabbing the cherry, not just hand, standing up, or lifting a foot, for example. More recent
using a tool to grab it. As soon as we turn the stimulation on, efforts are aimed at understanding and restoring the sen-
he says, “It is my hand.” sory system, through funding from the U.S. Department of
Eventually, we hope to engineer a prosthesis that is just Veterans Affairs and the Defense Advanced Research Proj-
as capable as the hand that was lost. Our more immediate ects Agency’s Hand Proprioception and Touch Interfaces
goal is to get so close that Spetic might forget, even momen- (HAPTIX) program.
tarily, that he has lost a hand. Right now, our haptic system is Our work on haptic interfaces falls under both of these
rudimentary and can be used only in the lab: Spetic still has new programs, but the focus is instead on restoring the sen-
wires sticking out of his arm that connect to our computer sory signals from the missing limb to the brain. Engineering
during the trials, allowing us to control the stimulation pat- such an interface is difficult because it has to allow precise
terns. Nevertheless, this is the first time a person without a patterns of stimulation to the person’s peripheral nerves,
hand has been able to feel a variety of realistic sensations without damaging or otherwise altering the nerves. It also

30  | may 2016 | North American | SPECTRUM.IEEE.ORG

05.HapticProsthetics.NA [P].indd 30 4/13/16 11:40 AM

 Circular
electrode cuff

Wires to
external computer
Nerve

Sites of perceived sensation Flat electrode

cuff

Nerve
Nerves
n Radial
n Ulnar
n Median

must function reliably for years

within the harsh environment
of the body.
There are several approaches RESTORING THE SENSE OF TOUCH: To allow
to designing an implanted inter- a person with a prosthetic hand to perceive
face. The least invasive is to sensations, researchers at Case Western Reserve
University surgically implanted electrode cuffs
embed electrodes in a muscle, around the median, radial, and ulnar nerves in the
near the point where the tar- affected arm. The flattened cuff [right] is more
get nerve enters that muscle. effective than the traditional circular cuff [above
left] because electrical signals can access the
Such systems have been used nerve fibers more easily. When precise patterns
to restore function following of electrical pulses are sent to each electrode, the
spinal-­cord injury, stroke, and other forms of neurological subject feels sensations at specific sites on the
front and back of his hand [lower left], as well as
damage. The body tolerates the electrodes well, and surgi- different textures. Although this experimental
cally replacing them is relatively easy. When the electrodes system uses an external computer, the eventual
need to activate a muscle, however, it often requires a cur- goal is to implant a controller, which will wirelessly
communicate with the prosthetic hand.
rent of up to 20 milliamperes, about the same amount you
get when you shuffle across a carpet and get “shocked”;
even then, the muscle isn’t always completely activated. treat epilepsy and to help stroke patients speak and swal-
The most invasive approach involves inserting electrodes low. More complicated, multiple-channel versions have
deep into the nerve. Placing the stimulating contacts so close been used reliably for nearly a decade in clinical trials to
to the target axons—the parts of nerve cells that conduct restore upper- and lower-­extremity function following a
electrical impulses—means that less current is required and spinal-cord injury.
that very small groups of axons can be selectively activated. Since the late 1990s, my group has been working on such
But the body tends to reject foreign materials placed within encircling electrodes, also known as nerve cuffs. One early
2-D hand illustration: james provost

the protective layers of its nerves. In animal experiments, problem we tackled was how to increase access to the nerve
the normal inflammatory process often pushes these elec- without actually penetrating it. The small surface area and
trodes out of the nerve. cylindrical shape of a traditional electrode cuff weren’t well
Somewhere between these two approaches are systems suited to the task. We therefore flattened out the nerve cuff
that encircle the nerve and place electrical contacts on so that it fit around an oblong cross section of the nerve.
the surface of the nerve. Simple systems that stimulate In 2014, we unveiled the latest version of the flattened
just one site on one nerve are commercially available to cuff, which has eight contact points, each connected to a

3-D illustration by Bryan Christie Design SPECTRUM.IEEE.ORG | North American | may 2016 |  31

05.HapticProsthetics.NA [P].indd 31 4/13/16 11:40 AM

 different channel for stimulation. To date, we’ve
implanted our eight-channel cuff in a handful
of subjects. Spetic, the cherry-­plucking volun-
teer, has the flat electrode cuffs placed around
the median and ulnar nerves, two of the three
main nerves in his arm. He has a traditional cir-
cular electrode placed around the radial nerve.
This provides a total of 20 stimulation chan-
nels in his forearm: eight each on the median
and ulnar nerves and four on the radial nerve.
The first time Spetic tested our system, we
didn’t know whether any of the channels would
actually translate to different sensations or dif-
ferent locations. Anxiously, we turned it on
and activated a contact on Spetic’s median
nerve. “Wow!” he said. “That’s the tip of my
thumb. That’s the first time I’ve felt my hand
since the injury.” It was one of those moments
a researcher lives for.
Further testing revealed that our 20 stimu- COMPLETE CONTROL: time we see them in the brain is during abnormal
lation points created sensations at 19 places on With the haptics in activity, such as an epileptic seizure. We think
his prosthetic hand
Spetic’s missing hand, including spots on the turned on, Spetic can this kind of stimulation causes a group of sev-
left and right sides of his palm, the back of his perform delicate tasks eral hundred neurons to fire together, creating
hand, his wrist, his thumb, and his fingertips. like plucking a grape, an unusual signal that the brain interprets as a
grasping a flower petal,
The next generation of our cuff will have four and unscrewing a cap. generic sensation of tingling.
times as many contacts. The more channels, the “It is my hand,” he says. In our next experiments we varied the pattern
more selectively we’ll be able to access small of electric pulses that we sent up the nerves to the
groups of axons and provide a more useful range brain. We tried changing the timing of pulses and
of sensations. In addition to the tactile, we’d like to produce interspersing the sequence with pairs of pulses. Neither
sensations like temperature, joint position (known as pro- of these tests made a significant difference. And because
prioception), and even pain. Despite its negative connota- there were so many variables, it proved difficult and time-­
tion, pain is an important protective mechanism. During consuming to understand how changing the pattern of pulses
our tests, one stimulation channel did cause a painful sen- affected what Spetic felt.
sation. Eventually, we would like to include such protec- To move the experiment forward, I ended up testing many
tive mechanisms. of the patterns on myself. Using a clinically available, non-
For now, we are exploring the other channels and continu- invasive nerve-stimulation system, a team member placed
ing to work with Spetic, who has had the implanted system electrodes on my finger where they could activate a super-
since May 2012. It’s still working well. When the system is ficial nerve, and then I got my students to “buzz” me with
turned off, he says, he doesn’t even realize he has anything varying patterns. We found that changing the pulse strength
implanted in his body. in a wavelike pattern, increasing and then decreasing in
about a one-second cycle, changed the sensation from tin-

O
gling to a more natural feeling of pressure—it felt as though
f c ou r s e , t r i g g e r i ng a basic sensa- something was squeezing my finger.
tion is one thing; controlling how that sensa- We were then ready to try the pattern on Spetic. As the
tion feels is another. It’s analogous to talking: stimulation started, he looked confused for a moment, and
You need to generate sound, but to be under- then he placed the fingers of his remaining hand on his neck.
stood, that sound has to come out in distinct “It doesn’t feel like tingling anymore,” he said. “It’s a puls-
patterns that can be interpreted as language. In our first ing pressure, like I put my fingers on my neck and felt my
experiments, we excited the nerves with regular pulses at pulse.” With a little adjustment, we were able to remove
a constant strength. This regular stimulation resulted in a the pulsing, and he reported a natural touch, “like some-
tingling sensation called paresthesia—the pins-and-needles one just laid a finger on my hand.”
feeling of a foot that’s fallen asleep. So we were generating We think that the weaker pulses activate fewer of the neu-
sound but not speech. rons in the nerve, whereas stronger pulses cause more of
Such electrical impulses aren’t part of the nervous sys- them to fire. The variation in the firing rates of the differ-
tem’s repertoire when it’s operating properly: The only ent neurons is part of the neural code that the brain under-

32  | may 2016 | North American | SPECTRUM.IEEE.ORG

05.HapticProsthetics.NA [P].indd 32 4/13/16 11:40 AM

 stands. If the pattern we apply resembles a pattern that the nels that will need to be coordinated to create motion and
brain already knows, it interprets the sensation according to feeling. And all of this activity must be carried out with
its experience: In effect, the brain says, Okay, that’s touch. minimal time delays.
We are now working to understand how more complex As we refine our system, we’re trying to find the optimal
patterns can produce more nuanced perceptions of sen- number of contacts. If we use three flattened electrode cuffs
sation. So far, Spetic has reported feeling textures that he that each have 32 contacts, for example, we could hypotheti-
described as Velcro and sandpaper and also feeling objects cally provide sensation at 96 points across the hand. So how
moving, fluttering, and tapping on his skin. What’s more, many channels does a user need to have excellent function
Spetic can manipulate fine and delicate objects in a manner and sensation? And how is information across these chan-
that he was unable to do before. He no longer has to rely on nels coordinated and interpreted?
vision alone to know how his prosthesis is performing. And To make a self-contained device that doesn’t rely on an
he’s far more confident using the prosthesis when he has external computer, we’ll need miniature processors that
sensation than when he does not. can be inserted into the prosthesis to communicate with
the implant and send stimulation to the electrode cuffs. The

S
implanted electronics must be robust enough to last years
o how will all this knowledge help others? inside the human body and must be powered internally,
Working with our partners at Medtronic and Law- with no wires sticking out of the skin. We’ll also need to
rence Livermore National Laboratory, we are work out the communication protocol between the pros-
creating a fully implantable stimulation system thesis and the implanted processor.
paired with an advanced anthropomorphic hap- It’s a daunting engineering challenge, but when we suc-
tic prosthetic. The project aims to have a working device ceed, this haptic technology could benefit more than just
within three years so that it will be ready for clinical trials prosthetic users. Such an interface would allow people to
by the last year of our five-year contract. touch things in a way that was never before possible. Imagine
Building a sophisticated neural stimulation device that an obstetrician feeling a fetus’s heartbeat, rather than just
actually works outside the laboratory won’t be easy. The relying on Doppler imaging. Imagine a bomb disposal spe-
prosthesis will need to continuously monitor hundreds of cialist feeling the wires inside a bomb that is actually being
tactile and position sensors on the prosthesis and feed that handled by a remotely operated robot. Imagine a geologist
information back to the implanted stimulator, which then feeling the weight and texture of a rock that’s thousands
must translate that data into a neural code to be applied of kilometers away or a salesperson tweeting a handshake
to the nerves in the arm. At the same time, our system to a new customer.
will determine the user’s intent to move the prosthesis by Such scenarios could become reality within the next decade.
recording the activity of up to 16 muscles in the residual Sensation tells us what is and isn’t part of us. By extending
limb. This information will be decoded, wirelessly trans- sensation to our machines, we will
mitted out of the body, and converted to motor-drive com- expand humanity’s reach—even if post your comments at
http://spectrum.ieee.org/
mands, which will move the prosthesis. In total, the system that reach is as simple as holding a hapticprosthetic0516
will have 96 stimulation channels and 16 recording chan- loved one’s hand. n

SPECTRUM.IEEE.ORG | North American | may 2016 |  33

05.HapticProsthetics.NA [P].indd 33 4/13/16 11:40 AM