Mini Project Front Page-1
Mini Project Front Page-1
A
REPORT
On
“Face Recognition-Based Authenticated Voice Assistant
System”
Submitted in Partial Fulfillment of the requirements for the Fifth semester
BACHELOR OF ENGINEERING
IN
COMPUTER SCIENCE AND ENGINEERING
Submitted By
BABU G V 1SJ22CS020
DEEPAK S N 1SJ22CS047
POORNA CHANDRA TEJASWI P 1SJ22CS118
SHASHANK S R 1SJ22CS146
Carried out at
Project Lab,
Dept of CSE,
SJCIT
CERTIFICATE
This is to certify that the project work entitled “Face Recognition-Based Authenticated
Voice Assistant System”is a Bonafide work carried out by BABU G V(1SJ22CS020),
DEEPAK.S.N(1SJ22CS047),POORNACHANDRATEJASWI.P(1SJ22CS118),
SHASHANK.S.R(1SJ22CS146) in partial fulfillment for the award of Bachelor of
Engineering in Computer Science and Engineering of Visvesvaraya Technological
University, Belagavi during the year 2024- 2025. It is certified that all
corrections/suggestions indicated for internal assessment have been incorporated in the
report. The Mini project report has been approved as it satisfies the academic
requirements with respect to Mini project work prescribed for the Bachelor of
Engineering degree in Fifth Semester.
DEEPAK.S.N
1SJ22CS032
SHASHANK.S.R
1SJ22CS061
i
ABSTRACT
The primary objective of this project is to develop a system that ensures high accuracy in face recognition
while maintaining low latency for seamless real-time use. The system addresses the growing demand for
secure access to sensitive information and resources across personal, corporate, and smart home environments.
Additionally, it emphasizes safeguarding user privacy by implementing advanced data security measures.
The methodology involves a structured approach comprising requirement analysis, system design,
implementation of facial recognition algorithms, virtual assistant integration, and rigorous testing. The project
also incorporates state-of-the-art machine learning models to improve recognition accuracy under varying
conditions such as lighting, pose, and expression.
This system not only improves security but also enhances convenience, making it a viable solution for modern
authentication challenges. The results demonstrate its effectiveness in delivering a high-performance and
reliable authentication mechanism. The proposed system has significant potential for scalability and future
enhancements, such as incorporating multimodal biometrics and extending functionality to diverse platforms.
ii
ACKNOWLEDGEMENT
With reverential pranam, we express my sincere gratitude and salutations to the feet of his holiness Paramapoojya
Jagadguru Byravaikya Padmabhushana Sri Sri Sri Dr. BalagangadharanathaMaha Swamiji, his holiness
ParamapoojyaJagadguru Sri Sri Sri Dr. Nirmalanandanatha Maha Swamiji, andSri SriMangalnath Swamiji,
Sri Adichunchanagiri Mutt for their unlimited blessings.
First and foremost, we wish to express our deep sincere feeling and gratitude to our institution, Sri
Jagadguru Chandrashekaranatha Swamiji Institute of Technology, for providing us an opportunity for
completing the Mini-Project Work successfully.
We extend deep sense of sincere gratitude to Dr. G T Raju, Principal, SJC Institute of Technology,
Chickballapur, for providing an opportunity to complete the Mini-Project Work.
We extend special in-depth, heartfelt, and sincere gratitude to HOD Dr. Manjunatha Kumar B H, Head of
the Department, Computer Science and Engineering, SJC Institute of Technology, Chickballapur, for his
constant support and valuable guidance of the Mini-Project Work.
We convey our sincere thanks to Project Guide Deepthi N, Assistant Professor, Department of
Computer Science and Engineering, SJC Institute of Technology, for his/her constant support, valuable guidance
and suggestions of the Mini-Project Work.
We also feel immense pleasure to express deep and profound gratitude to Mini-Project Coordinators
Mrs. Bhavya R A, Assistant Professors, Department of Computer Science and Engineering, SJC Institute of
Technology, for their guidance and suggestions of the Mini-Project Work.
Finally, we would like to thank all faculty members of Department of Computer Science and Engineering,
SJC Institute of Technology, Chickaballapur for their support
We also thank all those who extended their support and co-operation while bringing out this Mini-Project work.
BABU.G.V 1SJ22CS020
DEEPAK.S.N 1SJ22CS032
POORNA CHANDRA TEJASWI.P 1SJ22CS118
SHASHANK.S.R 1SJ22CS061
i
CONTENTS
Declaration i
Abstract ii
Acknowledgement iii
Contents iv
ii
Proposed System
Advantages
5 SYSTEM DESIGN 15-20
5.1Project Modules
5.2Activity Diagram
5.3UseCase Diagram
5.4Dataflow Diagram
5.5Sequence Diagram
6 IMPLEMENTATION 21-23
Algorithm/Pseudo-code module wise
TESTING 24-26
Methods of Testing
Unit Testing
7.1.2Validation Testing
7.1.3Functional Testing
7.1.4Integration Testing
7.1.5User Acceptance
Testing Test Cases
8 PERFORMANCEANALYSIS 27-28
9 CONCLUSION &FUTUREENHANCEMENT 29
BIBLIOGRAPHY 30
APPENDIX 31
Appendix A: Abbreviation
v
Chapter 1: Introduction
Overview
Problem Statement
The growing reliance on digital systems for critical applications has underscored the
limitations of conventional authentication methods, such as passwords and PINs. These
methods are susceptible to threats like phishing, brute force attacks, and poor password
practices. A need exists for a reliable, efficient, and user-friendly authentication
mechanism that ensures the security of sensitive data without compromising ease of
access.
Integrating face recognition into virtual assistant systems holds immense potential in
transforming user interactions across a variety of contexts, including:
This technology not only enhances security but also provides a seamless and efficient
user experience.
4
............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
Objectives
Methodology
5
Chapter 2: Literature Survey
Introduction
The literature survey examines existing research and technologies in face recognition,
biometric authentication, and virtual assistant systems. This analysis identifies the
strengths and limitations of current systems, providing a foundation for the development
of the proposed solution.
1. Traditional Approaches
⚫ Early face recognition methods like Principal Component Analysis (PCA) and
Linear Discriminant Analysis (LDA) were effective for small datasets but
struggled with variations in lighting, pose, and expression.
⚫ These methods were computationally efficient but lacked the robustness
required for real-world applications.
⚫ Algorithms like YOLO and MTCNN enable efficient face detection and
recognition in real-time scenarios.
⚫ These advancements are widely adopted in surveillance systems and consumer
devices.
6
Biometric Authentication Systems
⚫ Assistants like Siri, Alexa, and Google Assistant rely on voice commands for
interaction but lack robust security mechanisms.
⚫ Integrating face recognition into virtual assistants can enhance both security and
personalization.
2. Challenges in Integration
Identified Gaps
1. Integrating advanced face recognition algorithms with virtual assistants for secure and
personalized authentication.
7
2. Employing optimized models to ensure real-time performance without compromising
accuracy.
3. Implementing privacy-preserving measures to enhance user trust and data security.
Hardware Requirements
1. Processor: High-performance CPU, such as Intel Core i5 or AMD Ryzen 5
(or higher).
2. Memory (RAM): Minimum 8 GB; recommended 16 GB for smoother
operation.
3. Storage: SSD with at least 256 GB for faster read/write operations.
4. GPU: Dedicated GPU (e.g., NVIDIA GTX 1050 or higher) for running deep
learning models efficiently.
5. Camera: High-resolution webcam for real-time facial data input.
Software Requirements
1. Operating System: Windows 10 or Linux-based OS.
2. Programming Language: Python (primary) with relevant libraries like
TensorFlow, Keras, OpenCV, and Numpy.
3. Frameworks:
- TensorFlow or PyTorch for machine learning.
- Flask/Django for the application backend.
4. Database: MySQL or SQLite for managing user data and embeddings.
5. **IDE/Editor**: PyCharm, VSCode, or Jupyter Notebook.
Functional Requirements
1. Facial recognition using real-time video feed.
2. Secure authentication based on face recognition.
3. Virtual assistant interaction through voice or text.
4. User-friendly interface for easy navigation and interaction.
5. Data encryption to protect sensitive user information.
Non-Functional Requirements
8
1. Performance: High accuracy (target >95%) with low latency (target
<1s per operation).
2. Scalability: Support multiple user authentications concurrently.
3. Reliability: Uptime of >99% under typical operating conditions.
4. Privacy and Security: Implement advanced data encryption and
ensure compliance with data protection standards.
Performance Metrics
1. Accuracy: Minimum target 95% for face recognition under various lighting
and pose conditions.
2. Latency: Average response time of <1000 ms.
3. Error Rates:
- False Positive Rate: <2%.
- False Negative Rate: <5%.
9
Chapter 4: System Analysis
Existing System
Overview
1. Security Vulnerabilities:
3. Scalability Issues:
10
Proposed System
Overview
The proposed system integrates face recognition technology with virtual assistant
capabilities to create a secure, efficient, and user-friendly authentication mechanism.
This system addresses the limitations of existing methods by leveraging biometric data,
which is unique, non-transferable, and difficult to forge.
1. Enhanced Security:
3. Scalability:
11
2. High Accuracy: Advanced deep learning models ensure reliable recognition
even under varying conditions.
3. Real-Time Performance: Optimized algorithms enable rapid detection and
authentication.
4. Integration with Virtual Assistant: Combines security with personalized
functionality, enhancing usability.
5. Data Privacy: Ensures compliance with industry standards for user privacy
and data protection.
⚫ Captures real-time video input and detects faces using advanced algorithms
like MTCNN or Haar cascades.
⚫ Recognizes faces using deep learning models such as FaceNet or DeepFace.
2. Authentication Module:
⚫ Verifies user identity by comparing the detected face with the stored facial
data.
⚫ Ensures secure and efficient data handling during the authentication process.
⚫ Manages storage and retrieval of user data, including facial features and
authentication logs.
⚫ Implements encryption to ensure data security and privacy.
12
5. System Interface Module:
Activity Diagram
The activity diagram represents the overall workflow of the system from face detection
to authentication and user interaction:
13
Activity Diagram
Level 0 DFD
14
Level 1 DFD
Sequence Diagram
The sequence diagram illustrates the Flowchart of event scheduling and updating
database:
15
Chapter 6: Implementation
Sequence Diagram
16
The implementation of the "Face Recognition-Based Authentication Virtual Assistant
System" follows a systematic approach, illustrated in the sequence diagram. This
highlights interactions between various system components, including the user, the face
recognition module, the authentication system, and the virtual assistant.
Algorithm/Pseudo-code (Module-wise)
Face Detection and Recognition Module
Algorithm:
Authentication Module
Pseudo-code:
Input: Captured facial features, Database of registered users
Output: Authentication result
1. Start
2. For each captured facial feature:
a. Calculate similarity with stored embeddings using
cosine similarity.
b. If similarity > threshold:
Authenticate user.
Allow access to system.
Else:
Deny access.
3. End
Algorithm:
17
1. Receive input (voice or text) from the user.
2. Authenticate the user using the face recognition module.
3. Fetch user-specific data or preferences from the database.
4. Generate a personalized response based on the input.
5. Output the response via text or speech.
Pseudo-code:
1. Start
2. Encrypt all user data before storage.
3. Store facial embeddings with associated user IDs.
4. Provide secure retrieval methods for authentication and logs.
5. End
CHAPTER 7: TESTING
Methods of Testing
18
Unit Testing
Validation Testing
Test the system against known user data to validate accuracy and reliability of
authentication.
Functional Testing
Verify that all functionalities, including face recognition, authentication, and virtual
assistant integration, work as expected.
Integration Testing
Ensure smooth interaction between different modules, such as face recognition and the
virtual assistant.
Engage real users to evaluate the system's usability, response times, and overall
experience.
Test Case
19
Overview
Performance Metrics
Accuracy Evaluation
Testing Dataset
Results: OverallAccuracy:95.1%
Test Scenario Accuracy (%)
Latency Analysis
20
Measurement
⚫ Average time measured for face detection, recognition, and authentication combined.
Scalability Testing
Methodology
10 950 30
100 1200 50
500 2000 80
Observations
The system performs optimally for up to 100 users with minimal impact on response
time. Performance degrades gradually beyond 500 users, indicating the need for resource
scaling.
Observations
⚫ The system exhibits a low error rate, ensuring reliability in user authentication.
System Uptime
Performance Summary
The system demonstrates high accuracy and low latency under normal operating
conditions. While scalability is effective for small to medium user bases, additional
resources may be required for larger deployments. Error rates are minimal, ensuring
robust security and reliability.
22
Chapter 8: Conclusion and Future Work
Conclusion
The system’s performance metrics confirm its viability for real-world applications in
personal devices, smart homes, corporate environments, and beyond. By combining
cutting-edge technology with a focus on usability and security, the project sets a strong
foundation for modern authentication systems.
23
Future Work
While the system meets its primary objectives, there are opportunities for further
improvement and expansion. Key areas for future work include:
1. Multimodal Authentication:
2. Cloud Integration:
5. Performance Optimization:
24
Bibliography
1. Abate, A. F., Nappi, M., Riccio, D., & Sabatino, G. (2007). 2D and 3D face
recognition: A survey. Pattern Recognition Letters, 28(14), 1885-1906.
https://doi.org/10.1016/j.patrec.2007.04.018
2. Schroff, F., Kalenichenko, D., & Philbin, J. (2015). FaceNet: A unified embedding
for face recognition and clustering. Proceedings of the IEEE Conference on
Computer Vision and Pattern Recognition (CVPR), 815-823.
https://doi.org/10.1109/CVPR.2015.7298682
3. Taigman, Y., Yang, M., Ranzato, M. A., & Wolf, L. (2014). DeepFace: Closing the
gap to human-level performance in face verification. Proceedings of the IEEE
Conference on Computer Vision and Pattern Recognition (CVPR), 1701-1708.
https://doi.org/10.1109/CVPR.2014.220
4. Parkhi, O. M., Vedaldi, A., & Zisserman, A. (2015). Deep Face Recognition. British
Machine Vision Conference (BMVC), 41.1-41.12. https://doi.org/10.5244/C.29.41
5. Dalal, N., & Triggs, B. (2005). Histograms of Oriented Gradients for Human
Detection. Proceedings of the IEEE Computer Society Conference on Computer
Vision and Pattern Recognition (CVPR), 886-893.
https://doi.org/10.1109/CVPR.2005.177
6. Viola, P., & Jones, M. J. (2001). Rapid object detection using a boosted cascade of
simple features. Proceedings of the IEEE Computer Society Conference on
Computer Vision and Pattern Recognition (CVPR), I-511-I-518.
https://doi.org/10.1109/CVPR.2001.990517
7. Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press.
https://www.deeplearningbook.org/
8. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep Residual Learning for Image
Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition (CVPR), 770-778. https://doi.org/10.1109/CVPR.2016.90
25
9. Google Cloud AI. (n.d.). Using face recognition APIs for personalized applications.
Retrieved from https://cloud.google.com/ai
11. OpenCV. (n.d.). Open Source Computer Vision Library. Retrieved from
https://opencv.org
26
Appendix
Appendix A: Abbreviations
Abbreviation Full Form
AI Artificial Intelligence
CNN Convolutional Neural Network
DFD Data Flow Diagram
FNR False Negative Rate
FPR False Positive Rate
GPU Graphics Processing Unit
MTCNN Multi-Task Cascaded Convolutional Networks
PCA Principal Component Analysis
RAM Random Access Memory
SSD Solid State Drive
27