ISSN (Online) 2321 – 2004

IJIREEICE ISSN (Print) 2321 – 5526

INTERNATIONAL JOURNAL OF INNOVATIVE RESEARCH IN ELECTRICAL, ELECTRONICS, INSTRUMENTATION AND CONTROL ENGINEERING
Vol. 4, Issue 5, May 2016

Gesture Controlled Robot using LabVIEW

R. Gopalakrishnan1, S. Abirami2, P. Sathiya3
Assistant Professor, Instrumentation &Control Engineering, Saranathan College of Engineering, Trichy, India 1, 3
Research Scholar, Department of Instrumentation Engineering, Annamalai University, Chidambaram, India2

Abstract: In few years ROBOTS are going to decree the cosmos. Man supremacy is abridged by replacing robots. In
the existing system, human hand movements is sensed by the robot through sensors and it imitates the same, but this is
made probable by placing the sensor in the accelerometer and which is to be carried by the person(user’s hand). As the
person moves his/her hand, the accelerometer also moves accordingly which in turn displaces the sensor and this sensor
senses the constraints according to the position of hand and this sensed constraints are given to the sensors placed in
robotic arm which makes the movements of robot arm accordingly. Hand gestures and even the intact body posture
play a very vital role in human communication. The intention of this paper primarily is to generate a script in vision
assistant and real time image through the vision acquisition acquired. The acquired image is to be transformed into
digital signal using image processing in LabVIEW. Interfacing the robot with LabVIEW is processed through DAQ.
The digital indication of the robot is made to control by several hand gestures and drive the robot in different direction.

Keywords: Gesture, Lab VIEW, DAQ, Acquisition.

I. INTRODUCTION

Computer analysis of hand gestures has become a key These real time images are acquired into the simulation
research spot in the Human Computer Interface (HCI), and processed.
which is not restricted to the keyboard, mouse or some
other direct input devices. Ubiquitous Computing, Smart
Rooms, Virtual Reality and many other important
technical fields include gestures as a way of
communication; therefore, research on computer analysis
of hand gestures and body posture has significant
scientific value [1]. IMAQ vision toolbox presents a
comprehensive set of digital image processing and Figure1: Block diagram of the Model
acquisition functions that improve the efficiency of the
projects and diminish the programming effort of the users The digital signal of the images are processed and
obtaining better results in shorter time [2]. A vision based converted in to the analog signal. The analog signals are
hand gesture interface, where by hand states (open/close) taken out using the DAQ and NI-ELVIS. These signals are
and a class of hand postures in a taxonomy of grass types given to the dc gear motor to move the robot.
can be detected and recognized with a single passive
camera. Inferring the full articulations of the fingers form III. LABVIEW TOOLS
a single camera is an exigent problem due to the
complexity of the hand articulations, the occlusions of the a) Vision Assistant
fingers, and complications in segmentation of the hand The NI Vision Development Module is designed develop
from the background image. In the past, several and deploy machine vision applications. It includes
researchers have developed hand pose estimation methods hundreds of functions to acquire images from a multitude
for vision-based gesture interfaces [3]. Using the captured of cameras and to process images by enhancing them,
data, several methods have been proposed for the robot to checking for presence, locating features, identifying
classify human emotions or to follow a human teacher’s objects, and measuring parts. Vision Assistant is a tool for
instruction [4]. In this project we used the technique of prototyping and testing image processing applications. To
edge detection in the vision assistant. The edges of the prototype an image processing application, build custom
images are detected by the change in contrast of the image. algorithms with the Vision Assistant scripting feature. The
diverse hand gestures are taken for the faultless scripting feature records every step of the processing
recognition of the images. these gestures are progression algorithm. After completing the algorithm, you can test it
to acquire the analog voltage through daq from the nielvis on other images to make sure it works.
and interfacing it with the robot.
The algorithm is recorded in a Builder file, which is an
II. PROCESS DESCRIPTION ASCII text file that lists the processing functions and
relevant parameters for an algorithm prototype in Vision
The distinct hand gestures are exposed before the web Assistant. Using the LabVIEW VI Creation Wizard, you
camera. The web camera is worn to acquire the image can create a LabVIEW VI that performs the prototype
from the real time. which has been created in Vision Assistant.

Copyright to IJIREEICE DOI 10.17148/IJIREEICE.2016.4550 198

 ISSN (Online) 2321 – 2004
IJIREEICE ISSN (Print) 2321 – 5526

INTERNATIONAL JOURNAL OF INNOVATIVE RESEARCH IN ELECTRICAL, ELECTRONICS, INSTRUMENTATION AND CONTROL ENGINEERING
Vol. 4, Issue 5, May 2016

Vision Assistant offers three types of image acquisitions: f) Communication Bus

snap, grab, and sequence. A snap acquires and displays a DAQ devices are connected to a computer through a slot
single image. A grab acquires and displays a continuous or port. The bus serves as the communication interface
set of images. A sequence acquires images according to between the DAQ device and computer for passing
settings that you specify and sends the images to the instructions and measured data. DAQ devices are offered
Image Browser. Using Vision Assistant, we can acquire on the most common computer buses including USB, PCI,
images with various National Instruments digital and PCI Express, and Ethernet. More recently, DAQ devices
analog IMAQ devices, such as the PCI-1428, PXI/PCI- have become available for 802.11 Wi-Fi for wireless
1409, PXI/PCI-1411, and so on. Vision Assistant provides communication. There are many types of buses, and each
specific support for several Sony, JAI, and IEEE 1394 offers different advantages for different types of
cameras. We can also simulate an acquired image if you applications.
do not have an IMAQ device. Vision Assistant allows you
to simulate the acquisition process by displaying a IV. HARDWARE REQUIREMENTS
sequence of images. We can stop the sequence at any
frame, capture the image and send the image to the Image NI-ELVIS is a hardware which is used as DAQ
Browser for processing. accessories from which we can be able to get the voltage
or we can be able to acquire the signal from the real world.
b) Data Acquisition Another hardware used is the dc gear motor. In this
Data acquisition (DAQ) is the process of measuring an episode NI-ELVIS and the working of the motor are
electrical or physical phenomenon such as voltage, explained.
current, temperature, pressure, or sound with a computer.
A DAQ system consists of sensors, DAQ measurement a) NI-ELVIS
hardware, and a computer with programmable software. The National Instruments Educational Laboratory Virtual
Compared to traditional measurement systems, PC-based Instrumentation Suite (NI ELVIS) is a hands-on design
DAQ systems exploit the processing power, productivity, and prototyping platform that integrates the 12 most
display, and connectivity capabilities of industry-standard commonly used instruments – including oscilloscope,
computers providing a more powerful, flexible, and cost- digital multimeter, function generator, bode analyzer and
effective measurement solution. more – into a compact form factor ideal for the lab or
classroom. It connects to your PC through USB
c) Parts of a DAQ System connection, providing quick and easy acquisition and
display of measurements. Based on NI LabVIEW
graphical system design software, NI ELVIS offers the
flexibility of virtual instrumentation and the ability of
customizing your application.
NI ELVIS is also an integral part of the NI electronics
education platform, combining simulation of circuits and
measurements from NI ELVIS instruments inside the NI
Multisim capture and simulation environment. Designed
with education in mind, NI ELVIS is a comprehensive tool
for teaching everything from circuit design,
instrumentation, controls, telecommunications, and
Figure2: DAQ System
embedded/MCU theory.
d) Signal Conditioning
V. COMPUTER INTERFACE
Signals from sensors or exterior world are noisy or too
dangerous to measure directly. Signal conditioning
a) Image Acquisition in real time:
circuitry manipulates a signal into a form that is suitable
The image is acquired in the Lab VIEW via image
for input into an ADC. This circuitry includes
acquisition from
amplification, attenuation, filtering, and isolation. Some
DAQ devices include built-in signal conditioning designed Vision motion Vision express Vision
for measuring specific types of sensors. acquisition.
e) Analog-To-Digital Converter In vision acquisition, camera which has to be used was
Analog signals from sensors must be converted into digital selected and tested in select acquisition source. The type
before they are manipulated by digital equipment. An of acquisition was selected whether we want to acquire
ADC (Analog-to Digital Converter) is a chip that provides continuously, single acquisition, finite acquisition with
a digital representation of an analog signal at an instant of inline processing and finite acquisition with post
time. In practice, analog signals continuously vary over processing in select acquisition type. In configure
time and an ADC takes periodic “samples” of the signal at acquisition settings we can change the resolution of the
a predefined rate. These samples are transferred to a camera, brightness, mode, sharpness, etc. In the next step
computer over a computer bus where the original signal is controls and indicators are selected to get the outputs. To
reconstructed from the samples in software. access the camera we need the ni-imaqdx.

Copyright to IJIREEICE DOI 10.17148/IJIREEICE.2016.4550 199

 ISSN (Online) 2321 – 2004
IJIREEICE ISSN (Print) 2321 – 5526

INTERNATIONAL JOURNAL OF INNOVATIVE RESEARCH IN ELECTRICAL, ELECTRONICS, INSTRUMENTATION AND CONTROL ENGINEERING
Vol. 4, Issue 5, May 2016

d) Browse Image:
The next step is browse image. To view the images that
are taken is seen by clicking the browse image icon. In
that we can delete the unwanted images by selecting the
unwanted image and clicking the close image icon. Open
image icon is used to replace the selected image or insert
the new image from the save images in the computer.

e) Process Image:
The final step is to process the acquired images. There are
many processing functions to get clear image detection.
We are using the edge detector from the machine vision.
In the edge detector setup edge detector and the look for
boxes are changed in to simple edge tool and all edges
respectively. After getting the clear edges it is saved as a
script. This script is acquired in the LabVIEW.

Figure3: Vision acquisition express

b)Vision assistant:
Vision Assistant is a tool for prototyping and testing image
processing applications. To prototype an image processing
application, build custom algorithms with the Vision
Assistant scripting feature. The scripting feature records
every step of the processing algorithm. After completing
the algorithm, you can test it on other images to make sure
it works.

c) Acquiring image:
To create a script on vision assistant the first step
is to acquire the image from the camera. The types of Figure5: Process image
acquiring the image are
f) LabVIEW programming
 Acquires image The vision acquisition is placed in the block diagram by
 Acquire image (IEEE 1394 or GigE) click and drag from vision express. This opens the dialog
 Acquires image (smart camera) box of NI vision acquisition express. Work to be done in
 Acquire image (RT) vision acquisition is explained in previous topic. The
 Simulate Acquisition vision assistant also placed in the block diagram. It opens
the ni vision assistant dialog box. In this the script we
As we are using the web camera it comes under the type of saved is opened and clicks the select controls. The output
IEEE 1394 or GigE. On selecting the type of acquiring a to be taken out is selected and clicks the finish button.
window with the icons like acquire continuous image,
acquire single image will be opened. Click on acquire For the movement of the robot case structure is used. The
continuous image to get a view of the finger and click on output from the vision assistant is in the form of numeric it
acquire single image. Keep doing it for getting desired is given to the case structure for changing the cases. In
image. After getting the image click the store acquired each case two DAQ is placed for controlling two robots. In
image in Browser to save the images. the daq the voltage for the motor is generated by generate
signals. In the generate signals voltage and the output port
is selected. For each motor separate port is used. After
selecting the port generation mode is selected as 1sample
(on demand). Give the voltage for the motor and click the
run for testing the DAQ. Then click ok for creating the
DAQ. By repeating these, create the cases for the different
motion of the robot on giving varying voltage to the DAQ.
The output image from the vision acquisition is given as
the input for the vision assistant. The number of edges
from the vision assistant is given to the case structure.

The stopping condition in the vision acquisition is changed

in to the Boolean for the continuous acquiring of real time
Figure4: Image Acquisition images.

Copyright to IJIREEICE DOI 10.17148/IJIREEICE.2016.4550 200

 ISSN (Online) 2321 – 2004
IJIREEICE ISSN (Print) 2321 – 5526

INTERNATIONAL JOURNAL OF INNOVATIVE RESEARCH IN ELECTRICAL, ELECTRONICS, INSTRUMENTATION AND CONTROL ENGINEERING
Vol. 4, Issue 5, May 2016

Figure9: Hardware Interface

Figure6: DAQ Assistant
VII. RESULTS AND DISCUSSIONS
g) Interfacing robot with Ni elvis:
The NI elvis has the two analog output port. The positive The simulation is tartan with the alteration in the wave
terminal of the robot is given to the different port and the form. When the gesture change the wave form also
negative terminal is connected to the ground. The changes. The figure shows the different gestures and
maximum voltage given got from the ni elvis is 10v. corresponding edge detected. On showing one finger it
will detect the two edges of the finger.

Figure10: Gesture Images

The figure 11 shows how the wave from is changed

according to the image.

Figure7: Interfacing robot with NI-elvis

VI. ROBOT DESIGN

The hard ware is made of simple card board with two dc

gear motors. The motor is fixed with the card board by tag.
The front wheel is taken from a plastic bottle and the rear
wheel is taken from the toy.

Figure11: Wave form Generation

Then instead of the wave form generation the DAQ is

replaced to generate the voltage. This generated voltage is
given to dc gear motor for controlling the motor direction
in turn it will change the direction of the robot.

Figure8: Hardware

Controlling the motor direction inturn it will change the

direction of the robot. The figure 2.28 shows the hard ware
interfaced with the LabVIEW through the daq. Figure12: Hardware Interface

Copyright to IJIREEICE DOI 10.17148/IJIREEICE.2016.4550 201

 ISSN (Online) 2321 – 2004
IJIREEICE ISSN (Print) 2321 – 5526

INTERNATIONAL JOURNAL OF INNOVATIVE RESEARCH IN ELECTRICAL, ELECTRONICS, INSTRUMENTATION AND CONTROL ENGINEERING
Vol. 4, Issue 5, May 2016

The figure 12 shows the hard ware interfaced with the

LabVIEW through the DAQ.

VIII. CONCLUSION

The autonomous robot vehicle has been lucratively

interfaced with the LabVIEW through DAQ. The gesture
input was rehabilitated into digital pulse signals. These
digital pulse signals are given to the dc gear motor.
Primarily the robot was tested for forward and reverse
movement according to the hand gesture. And it was also
effectively tested for movement in all directions. This
mission is functional in the places where minimum
requirement for substantial exertion is required. It is used
as a vehicle which carries load in industries, laboratories
and factories. It can also be used physically by disabled
populace, where they will be proficient to control the
movement of the vehicle with just their hand gestures
without others assistance.

REFERENCES

[1] Raghuraman Gopalan and Behzad Dariush, Toward a vision based

hand gesture interface for robotic grasping, The 2009 IEEE/RSJ
International Conference on Intelligent Robots and Systems
October 11-15, 2009 St. Louis, USA
[2] Rubén Posada-Gómez, Oscar Osvaldo Sandoval-González, Albino
Martínez Sibaja, Otniel Portillo-Rodríguez and GinerAlor-
Hernández Instituto Tecnológico de Orizaba, Digital Image
Processing Using LabView, Departamento de Postgrado e
Investigación, México.
[3] Wu.Y,Huang.T.Vision-basedgesturerecognition;A review In: The
3rdGestureWorkshop.1999: 103-115.
[4] Y. Sakagami, R. Watanabe, C. Aoyama, S. Matsunaga, N. Higaki,
and K.Fujimura, “The intelligent ASIMO: System overview and
integra- tion,” in IEEE Int. Conf. Intell. Robots Syst., 2002, pp.
2478–2483.
[5] Ganesan A, Nishanth R, Kamban.S, Gopalakrishnan.R, design and
control of 3-dof articulated robotic arm using labview and ni-
myrio,IJIREEICE, Vol.3, Issue3, Mar 2015. DOI:
10.17148/IJIREEICE.2015.3342
[6] Gopalakrishnan.R, P.Kavitha, Umamaehswari.B, Rotor Position
Estimation of Switched Reluctance Motor and Its Validation,
National journal of Technology, psg Coimbatore, Vol. 8, Issue. 3,
Sep 2012.
[7] Po-Lei Lee, Hsiang-Chih Chang, Tsung-Yu Hsieh, Hua-Ting Deng,
and Chia-Wei Sun, A Brain-Wave-Actuated Small Robot Car Using
Ensemble Empirical Mode Decomposition-Based Approach, IEEE
TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS,
OL. 42, NO. 5, SEPTEMBER 2012
[8] Jonathan Kofman, Xianghai Wu, Timothy J. Luu, and Siddharth
Verma, Teleoperation of a Robot Manipulator Using a Vision-
Based Human–Robot Interface, IEEE TRANSACTIONS ON
INDUSTRIAL ELECTRONICS, VOL. 52, NO. 5, OCTOBER
2005

Copyright to IJIREEICE DOI 10.17148/IJIREEICE.2016.4550 202