FYP Report Template
FYP Report Template
By
Supervisor
Dr. Muhammad Shoaib
The candidate confirms that the work submitted is their own and appropriate credit has
been given where reference has been made to the work of others.
COMSATS University Islamabad, Sahiwal
Pakistan
A project presented to
COMSATS Institute of Information Technology, Islamabad
In partial fulfillment
of the requirement for the degree of
By
M Usman CIIT/SP16-BSE-024/SWL
--------------------------- ---------------------------
45
CERTIFICATE OF APPROVAL
It is to certify that the final year project of BS (CS) “Project title” was developed by
STUDENT 1 NAME (CIIT/FAXX-BCS/SE/TN-000) and STUDENT 2 NAME
(CIIT/FAXX-BCS/SE/TN-000) under the supervision of “SUPERVISOR NAME” and co
supervisor “CO-SUPERVISOR NAME” and that in (their/his/her) opinion; it is fully
adequate, in scope and quality for the degree of Bachelors of Science in Computer Sciences.
---------------------------------------
Supervisor
---------------------------------------
External Examiner
---------------------------------------
Head of Department
(Department of Computer Science)
Executive Summary
In public places, there is often a need for monitoring people and different activities going on,
which can be referred later for many reasons including security. Appointing humans for this
task involves many problems such as increased employee hiring, accuracy problem, trust, no
proof for later use, and also the fact that a human can remember things till a certain time
limit. Talking about the current security system, they use dumb still cameras with a
continuous recording facility ir-respective of the fact that any event may happen or not.
Moreover they are usually pointing at a specific user defined locations so more than one
cameras are required to cover the entire region.
To prevent all these problems from prevailing, the CSCS is developed. It is a surveillance
system, which provides solution to many of these problems. It is a stand-alone application
which doesn’t require any computer to operate. It monitors different situations using a
camera which is able to rotate intelligently based on sensor messages and captures the scene
in the form of video or photos later reference as well.
It is an embedded system consisting of Linux fox kit with embedded a running server
application also a camera, USB storage device and a sensor node base station is attached with
fox kit. LAN communication is used by user to download the videos and to operate the
system manually.
Acknowledgement
All praise is to Almighty Allah who bestowed upon us a minute portion of His boundless
knowledge by virtue of which we were able to accomplish this challenging task.
We are greatly indebted to our project supervisor “Dr. Majid Iqbal Khan” and our Co-
Supervisor “Mr. Mukhtar Azeem”. Without their personal supervision, advice and valuable
guidance, completion of this project would have been doubtful. We are deeply indebted to
them for their encouragement and continual help during this work.
And we are also thankful to our parents and family who have been a constant source of
encouragement for us and brought us the values of honesty & hard work.
--------------------------- ---------------------------
Abbreviations
SRS Software Require Specification
PC Personal Computer
Table of Contents
Table of Content
Contents
Introduction ....................................... Error! Bookmark not defined.
Design methodology and software process modelError! Bookmark not d
Module 1: PYTHON ......................... Error! Bookmark not defined.
Module 2: Database ........................ Error! Bookmark not defined.
Module 3: Text Processor ............... Error! Bookmark not defined.
Module 4: Feature extraction and image processingError! Bookmark n
System overview................................ Error! Bookmark not defined.
Architectural design ........................ Error! Bookmark not defined.
Design models ................................... Error! Bookmark not defined.
Class Diagram .................................. Error! Bookmark not defined.
Sequence Diagram .......................................................................19
State Transition Diagram..............................................................20
Data Flow Diagram .......................... Error! Bookmark not defined.
Data design ........................................ Error! Bookmark not defined.
DATABASE: ................................................................................. 19]
Algorithm & Implementation ............ Error! Bookmark not defined.
Screen images .................................... Error! Bookmark not defined.
Appendix............................................ Error! Bookmark not defined.
List of Figures
Fig 1.1 Block Diagram .......................................................................................................................................... 8
Fig 2.1 Use Case Diagram.................................................................................................................................... 9
Introduction
Deaf and dumb people are usually deprived of normal communication with other people in
the society. It has been observed that they find it very difficult at times to interact with normal
people with their gestures, as only a very few of those are recognized by most people. Since
people with hearing impairment or deaf people cannot talk like normal people so they are
depend on some sort of visual communication in most of the time. The problem arises when
dump or deaf people are sign language Grammars. This is because normal people usually
Unaware of these Grammars . As a result it has been seen that communication of dump
person are only limited within his/her family or deaf community .The importance of sign
language is emphasized by growing public approval and funds for international project . At
this age of technology the demand for a computer based system is highly demanding for
dump community . However, researchers have been attacking the problem for quite some
time now and the result are showing some promise interesting technologies are being
developed for sign recognition is actually there in the current market. The gesture based
communication are used as a means of communication between deaf and dumb is clearly out
of reach in lack of definition , and also unadapt to the use of dumb. It is unadapt for private
communications, as the language spoken to one is spoken to all present who’ll understand it.
Spoken language can be whispered, or the volume can be reduced and it can’t be listened to
other ears than those for which it is expected but the power of the sign language can’t be
changed, and when private conversations are going on , written language is commonly used.
Gesture based communication is important means of communication in the deaf and dumb
society.
Brief
To design a system that will help deaf and dumb people who are unable to communicate, or
having difficulties in communication. This project helps in communication between deaf and
dumb person and a normal one .Dumb people are usually deprived of normal communication
with other people in the society. It has been observed that they find it really difficult at times
to interact with normal people with heir gestures, as only a very few of those are recognized
by most people. Since people with hearing impairment or deaf people cannot talk like normal
people so they have to depend on some sort of visual communication in most of the time. The
gap between normal human beings and deaf and dumb is wide and ever increasing day by
day. The unavailability of a software that provide the facility of understanding the gesture
The origins of data mining are databases, statistics. Machine learning involves the study of
algorithms that can extract information automatically. Machine-learning uses data
mining techniques and another learning algorithm to build models of what is happening
behind some data so that it can predict future outcomes .
Revelance to database
MLDB is the Machine Learning Database. MLDB is an open-source database designed
for machine learning. Store data, explore it using SQL, then train machine learning models
and expose them as APIs.
Project Background
This project provides a facility to the deaf and dumb to communicate with normal ones. The
deaf are unable to get any job or they may face difficulty to be understood by speaking
persons. Our main goal is to provide a desktop application so ,that the educated mutes can use
it. Educated mutes that follow the gestures of PSL( Pakistan Sign Language) .Because
uneducated deaf and literate deaf have difference in gestures . And this Machine will be
learned by following PSL gestures .It is entirely new software and not a next growing product
line.
Literature Review
GRS — Gesture based Recognition System for Indian Sign Language
Recognition System for Deaf and Dumb People
D.Anbarasan, R. Aravind , K. Alice
Department of Computer Science and Engineering.
G.K.M. College of Engineering and Technology , Chennai , India
Anbarasan et al [?] proposed the model that have three phases for gesture recognition training, phase
A testing phase and a Storage output phase . A total of 50 sign gestures were loaded into the training
model. The trained model accuracy is measured and then the output is extracted in the form of the
mentioned language symbol. The Detection mechanism of the proposed model is compared with the
other detection methods such as hidden Markov Model (HMM). Convolutional Neural Networks
(CNN) and Support Vector Machine (SVM). The accuracy obtained % 99 Percent in comparison
with the other detection methods . Then with the help of the color intensity variation and edge
detection, gestures are recognized. The Gestures are recognized with the edge detection. Further, the
image needs preprocessing for the accurate gesture detection . The model classified 50 symbols.
Sood and Anju Mishra [3] have proposed a sign recognition system based on Harris algorithm for
extraction of feature in which after the image pre-processing part, the feature is extracted and stored in
the Nx2 matrix. This matrix is further used to match the image from the database. There are some
limitations to the system. The very light brown to somewhat dark brown background gives error as
they are considered in the range value for skin segmentation. But the results are efficient.
Module 2: Database
It will store old and new gestures of deaf and mutes. Then system will search out gestures
with existing gestures and match them. And if it doesn’t match then it will save the new
gesture.
Problem Definition
To design a system that will help he deaf and dumb people who are unable to communicate,
or having difficulties in communication. This project helps in communication between deaf
and dumb person and a normal one .Dumb people are usually deprived of normal
communication with other people in the society. It has been observed that they find it really
difficult at times to interact with normal people with heir gestures, as only a very few of those
are recognized by most people. Since people with hearing impairment or deaf people cannot
talk like normal people so they have to depend on some sort of visual communication in most
of the time. The gap between normal human beings and deaf and dumb is wide and ever
increasing day by day.
Problem Statement
The aim of this research is bridge of communication barrier, Human Computer Interaction
and Image Processing will help us to build the solution for this problem.
The main aim of this project is to create user friendly technology to communicate the
physically liable people like dumb and deaf . This project uses the images and videos to make
real time communication between disabled people. In this project image processing is used to
recognize gesture, comparing that with stored database, recognizing correct expression using
Python and displaying output in the form of text through text processor.
Before starting my project we all visit the Govt. Secondary School of Special Education
Hearing Impaired Sahiwal we meet their principal javed who appreciate our work and show
great interest in our project . He tell about the Pakistan sign language and also said that this
work is difficult because of gesture is vary from place to place also vary from city to city
because of deaf and dumb are initially teach the meaning of initial gesture from their parents .
Parents are not well knowledge about the gesture .parents understand the gesture and assign
the meaning to the gesture and teach to deaf and dumb child about the gesture therefore
gesture vary from place to place. He said if you want to develop this kind of the software are
system follow Pakistan sign language in Pakistan every Special Education school follow. He
also said if you successful in your project then it’s very helpful to the deaf and dumb people
and teachers . Know we achieve what we want me and my group member decide to initially
install our project to govt. school this is our main aim of the project to serve humanity
This system is based on the indian sign language and this project is build on limited
gesture. They use Support Vector Machine algorithm .Support Vector Machine" (SVM)
is a managed machine learning algorithm which can be utilized for both grouping or
relapse challenges. In this algorithm, we plot every data item as a point in n-
dimensional space (where n is number of highlights you have) with the estimation of
each element being the estimation of a specific coordinate. At that point, we perform
classification by finding the hyper plane that separate the two classes exceptionally well.
But our project is totally different from this project. we use conventional neural network
algorithm . This algorithm widely used for large project.
Figure 2.1: Sample picture
The following table (Table 2.1) is sample table; you are required to follow the same style of
numbering and caption for the whole report.
Table 32.1: Sample Table
Header 1 Header 2 Header 3
Text Text Text
The following list style is the sample to consistently follow in the whole report.
List items 1
List items 2
Requirement Analysis
The following parts of Software Requirements Specification (SRS) report should be include
in this chapter.
Use Cases Diagram(s)
A use case diagram is a graphic depiction of the interactions among the elements of a system.
A use case is a methodology used in system analysis to identify, clarify, and organize system
requirements.
A use case diagram contains four components
The boundary, which defines the system of interest in relation to the world around it.
The actors, usually individuals involved with the system defined according to their roles.
The use cases, which are the specific roles played by the actors within and around the
system.
The relationships between and among the actors and the use cases.
Functional Requirements
Functional Requirements
Functional requirements can be defined as the services that should be provided by the system.
Those are calculations, technical details or other specific functionality that show how a use-
case(a description of a system’s behavior as it responds to a request that originates from
outside) is to be fulfilled.
-Camera
-Python
-CNN Model
-Training using Keras library
-Segmentation
-Image processing
-Classification based on trained model
-Laptop
Non-Functional Requirements
System Architecture
In neural networks, Convolutional neural network (Conv. Nets or CNNs) is one of the main
categories to do images recognition, images classifications. Objects detections, recognition
faces etc , are some of the areas where CNNs are widely used. CNN image classifications
takes an input image, process it and classify it.
Back-propagation is the essence of neural net training. It is the practice of fine-tuning the
weights of a neural net based on the error rate (i.e. loss) obtained in the previous epoch (i.e.
iteration). Proper tuning of the weights ensures lower error rates, making the model reliable by
increasing its generalization.
DATABASE:
It will store old and new gestures of deaf and dumb . Then system will search out gestures
with existing gestures and match them . And if it doesn’t match then it will save the new
gesture.
A- Acquire images of dumb and deaf gestures by camera.
B- Problem Solving and Artificial Intelligence .
C- Feature Extraction and Image Processing.
D -Gesture Recognition and Decision Making.
.
Process Flow/Representation
Sequence Diagram
State Transition Diagram
Fig: 3 State Transition Diagram
For efficient hand gesture recognition, data design should be a accurate as possible. The key
to ensuring this kind of of accuracy is by ensuring the selection a suitable input device. The
main aim of this process is to divide the image into meaningful structures or segments to
make it easier to analyze these structures and process them. Dumb people are usually
deprived of normal communication with other people in the society. It has been observed that
they find it really difficult at times to interact with normal people with heir gestures, as only a
very few of those are recognized by most people. Since people with hearing impairment or
deaf people cannot talk like normal people so they have to depend on some sort of visual
communication in most of the time. The gap between normal human beings and deaf and
dumb is wide and ever increasing day by day. The unavailability of a software that provide
the facility of understanding the gesture based language and their grammar and syntax. Then
their discussions are limited with people.
Implementation
This chapter will discuss implementation details supported by UML diagrams (if
applicable). You will not put your source code here. Any of the following sections
may be included based on your project.
Algorithm
Convolutional neural networks. Sounds like a weird combination of biology and math with
a little CS sprinkled in, but these networks have been some of the most influential
innovations in the field of computer vision. 2012 was the first year that neural nets grew to
prominence as Alex Krizhevsky used them to win that year’s Image Net competition
(basically, the annual Olympics of computer vision), dropping the classification error record
from 26% to 15%, an astounding improvement at the time.Ever since then, a host of
companies have been using deep learning at the core of their services. Facebook uses neural
nets for their automatic tagging algorithms, Google for their photo search, Amazon for their
product recommendations, Pinterest for their home feed personalization, and Instagram for
their search infrastructure.
Mention the algorithm(s) used in your project to get the work done with regards to
major modules. Provide a pseudo code OR a natural language explanation regarding
the functioning of main features. Be sure to use the correct syntax and semantics for
algorithm representations.
External APIs
Describe the APIs used in the following table.
Name of API Description of API Purpose of usage List down the function/class
name in which it is used
User Interface
Details about user interface with descriptions.
Testing and Evaluation
This chapter may include the following sections. (Students are required to perform the testing
both manually and automatedly.
Manual Testing
After the completion of our project we apply testing on our project . First of all we test our
project on our hostel friend’s and test for first gesture we first teach our friend the gesture A
then we recognize his hand gesture . Our system correctly answer to his Gesture . We
repeatedly perform this for all our friend of hostel for different gesture ultimately we calculate
higher accuracy of 97%. This accuracy is much batter then the other project .
System testing
After the successfully development of our project we test it for the purpose of either it’s meet
our requirement or not after the testing we realize that our project is successful develop
because it’s meet our and customer requirement . We train our system for 20 gesture and all
result are according to our purpose .
We visit Govt. Secondary School of Special Education Hearing Impaired Sahiwal and test it on deaf
and dumb student . Our system accurately answer the gesture of deaf and dumb people . Ultimately the
accuracy of our project is 97%.
Unit Testing
Test case/Test
No. script Attribute and Expected result Result
Value
Expected result
is correctly
recognize the
input gesture of
allaf and its
Test for gesture 1 expected The resultant
1. allaf User name : accuracy is 99% accuracy is 97%
Ali
Expected
result is
Test for Username : 99% for
gesture Mazhar The resultant
gesture
2. 2 bay accuracy is 97%
bay
1.
2.
Functional Testing
We train our system for 20 gesture and all result are according to our Purpose . We visit Govt.
Secondary School of Special Education Hearing Impaired Sahiwal and test it on deaf and dumb student .
Our system accurately answer the gesture of deaf and dumb People . Ultimately the accuracy of our
project is 97%.
2.
Integration Testing
Tools used:
Tool Name Tool Description Applied on [list of Results
related test cases / FR /
NFR]
Conclusion and Future Work
The Gesture Recognition System is applicable to normal people and physically challenged
people for non-verbal communication between human to human and computer and human. It
helps and aids dumb and deaf people to live independently. It eliminates gap among people and
leads to achieve better society. Here the system takes the hand gestures as input and gives output
in the form of speech and text and additional. In our system the gestures have to be given in
white background and in bright light conditions, in future enhancement the gestures can be given
in any background conditions and phrasing of sentences from sequence of words can be done
which will be helpful for advance and convenient communication.
Conclusion
The gesture recognition system is developed through this model .The GRS model recognize the
input signal with a complete accuracy. The Combination of the phases induced the system with
larger symbol detection mechanisms. Comparison of recognition technique such as HMM ,
SVM the GRS technique concluded to have an higher classification and recognition of various
symbols in ISL. The model classified 500 symbols. The GRS be a boon to the challenged people
and to gain higher standards in education.
Future Work
The future work is to develop a mobile, application for the sign recognition system. In future
enhancement the gestures can be given in any background conditions and phrasing of sentences
from sequence of words can be done which will be helpful for advance and convenient
communication.
References
References to any book, journal paper or website should properly be acknowledged. Please
consistently follow the style. The following are few examples of different resources i.e. journal
article, book, and website.
1. Lyda M.S. Lau, Jayne Curson, Richard Drew, Peter Dew and Christine Leigh, (1999), Use
Of VSP Resource Rooms to Support Group Work in a Learning Environment, ACM 99,
pp-2. (Journal paper example)