0% found this document useful (0 votes)
6 views6 pages

Paper Publication IEEE

The document presents 'Eye Sync', an innovative system that enables individuals with physical disabilities to control a computer mouse using only their eye movements, enhancing accessibility and productivity. Utilizing real-time eye-tracking technology and computer vision libraries like OpenCV and Dlib, the system translates eye movements into precise cursor navigation, allowing for a hands-free interface. The project aims to improve the quality of life for users with mobility impairments by providing an intuitive and user-friendly alternative to traditional input methods.

Uploaded by

Nanditha S
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
6 views6 pages

Paper Publication IEEE

The document presents 'Eye Sync', an innovative system that enables individuals with physical disabilities to control a computer mouse using only their eye movements, enhancing accessibility and productivity. Utilizing real-time eye-tracking technology and computer vision libraries like OpenCV and Dlib, the system translates eye movements into precise cursor navigation, allowing for a hands-free interface. The project aims to improve the quality of life for users with mobility impairments by providing an intuitive and user-friendly alternative to traditional input methods.

Uploaded by

Nanditha S
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 6

Eye Sync: Synchronizing Eye Movements with Mouse Cursor Control for Enhanced

Productivity

Eye Sync: Synchronizing Eye Movements with


Mouse Cursor Control for Enhanced Productivity

Mayur Gandharva D L, Mitali Jain, Nanditha S, Nidhishree S Awati

Department of Computer Science And Engineering, Don Bosco Institute of Technology, Bengaluru,
Visvesvaraya Technological University, Belgavi.

Abstract: For individuals with physical and recognizing objects. Meanwhile, Dlib provides
disabilities, operating a computer mouse can be a robust framework for building advanced vision-
a significant challenge, restricting their ability based algorithms. By combining these technologies,
to interact with digital devices. To overcome this
our system is designed to accurately track and
limitation, we present an innovative solution
that allows users to control the cursor solely interpret eye movements, translating them into
through eye movements, providing an intuitive smooth and precise cursor control. The goal is to
and accessible alternative to traditional input create a seamless, user-friendly experience that
methods. This system employs real-time eye- enhances accessibility and empowers users with
tracking technology, utilizing iris detection to greater independence in the digital world.
convert ocular movements into precise cursor
navigation on the screen. By simulating natural
1.1 PROBLEM STATEMENT
human behavior, it ensures a smooth and user-
friendly experience. Developed using Python Traditional input devices like keyboards and mice
and a standard webcam for tracking, this can be challenging or even inaccessible for
hands-free interface enhances accessibility individuals with physical disabilities or in scenarios
without requiring additional hardware. With a where hands-free control is essential. This project
user-centered design, the system prioritizes ease seeks to create an innovative system that enables
of use, empowering individuals with mobility users to navigate and control a computer mouse
impairments to engage more effectively with
solely through eye movements, offering a seamless,
computers and improving their overall digital
access and quality of life. intuitive, and inclusive alternative to conventional
interaction methods. By utilizing computer vision
Index Terms – Face detection, Object detection,
libraries such as OpenCV, Dlib, and Imutils, along
Eye gaze, Mouse cursor, Eye tracking, Gesture
control system, Technology Advancements. with automation tools like PyAutoGUI, the system
will track eye movements in real-time and convert
them into cursor commands. The focus will be on
I. INTRODUCTION ensuring precision, responsiveness, and ease of use,
With technology evolving at an incredible pace,
providing an accessible alternative for human-
human-computer interaction (HCI) has seen
computer interaction.
groundbreaking innovations, one of the most
exciting being eye movement-based control
Scope of the Project: The scope includes the
systems. These systems let users operate computers
design, implementation, and testing of key features
using just their eyes, providing a more natural and
like gaze direction analysis, blink detection for click
intuitive way to navigate digital spaces. For
simulation, and real-time responsiveness. The
individuals with motor impairments or injuries, this
system will also include a calibration process to
advancement could be life-changing, giving them
adapt to various users and environments, ensuring
newfound access to digital tools and opportunities
accuracy and usability across different screen sizes
that were once out of reach. This project is centered
and lighting conditions. Integration with pyautogui
around developing a real-time eye-tracking mouse
will allow cursor control on various operating
control system using the powerful capabilities of
systems, making the solution versatile and widely
OpenCV and Dlib. OpenCV, a widely recognized
applicable. Furthermore, the project emphasizes
computer vision library, offers a vast array of tools
robustness by addressing common challenges such
for processing images and videos, detecting features,
as occlusions from glasses or head movements and

1
Eye Sync: Synchronizing Eye Movements with Mouse Cursor Control for Enhanced
Productivity

improving performance through noise reduction then used to control the cursor’s movement. The
techniques. purpose of this guide is to provide an overview of
Objectives: current applications of eye-tracking technology and
• Develop a real-time system to detect and its potential role in human-computer interaction,
particularly in usability studies. [3].
track eye movements using dlib's facial
landmarks and OpenCV. K. Takemura, K. Takahashi, J. Takamatsu, and T.
• Implement algorithms to determine the Ogasawara Unlike traditional portable eye-tracking
user's gaze direction using eye landmarks. methods that determine the position of a mounted
camera using 2D image coordinates, the techniques
• Use blink detection as a mechanism to proposed here offer more detailed information about
simulate left and right mouse clicks. a person’s gaze as they move across a larger area.
• Ensure minimal delay between eye These methods also visualize scan paths when a user
with a head-mounted device makes natural head
movement detection and cursor response movements. We incorporate a Visual SLAM
for seamless interaction. (Simultaneous Localization and Mapping)
technique to estimate the head's position and extract
relevant environmental data. [4].
II. LITERATURE REVIEW
Chandra Murthy, Daggula Anitha, Narra Manidhar O. Ferhat and F. Vilarino, While infrared (IR) gaze
This project is to design a system for assisting tracking systems are known for their high accuracy
individuals with limited hand mobility by and commercial availability, visible light gaze
developing an innovative eye-controlled cursor tracking has emerged as an appealing alternative due
movement method. Based on OpenCV algorithms, to its scalability and the absence of the need for
the system is aimed at the precise detection and specialized hardware. Recent research has shown
stabilization of the pupil position for cursor that visible light-based gaze tracking can achieve
navigation. performance comparable to IR-based systems. This
First, a webcam captures live video or image study offers an in-depth review of current
acquisition, which then serves as input for eye advancements in remote gaze tracking using visible
tracking. Techniques of computer vision applied in light, exploring different approaches from several
eye detection would need face detection to more angles, such as calibration methods, resilience to
accurately determine where to track. The system head movement, and gaze estimation techniques.
also integrates a module for tracking head [5].
movements, which enhances the ability to make
adjustments for the cursor according to the III. SYSTEM ARCHITECTURE
orientation of the user's head. [1]. Webcam (Camera Feed) : The system starts by
capturing a live video feed from the webcam. This
Preethi G Vignesh V, Praveena S This project feed is processed in real-time to detect facial features
presents an innovative approach to assessing the and track eye movements. Face Detection and
advantages and limitations of various eye-tracking Landmark Detection (Using dlib and OpenCV): The
technologies, including head-mounted and screen- dlib library is utilized to detect the user's face and
based systems. It provides a comprehensive analysis identify facial landmarks, which include key points
of their key strengths and potential drawbacks. around the eyes, nose, and mouth. OpenCV is used
Exploring these different technologies can help in to capture video frames and perform preprocessing
selecting the most suitable option for specific needs, tasks (such as converting to grayscale and resizing)
ultimately leading to more effective outcomes [2]. to enhance processing speed.

Akshada Dongre, Ameya Patkar, Rodney Pinto, Dr Eye Detection and Tracking: After detecting the
Minal Lopes the proposed system outlines a face, the system focuses on the eye region using
complete process for controlling the mouse cursor previously identified landmarks. OpenCV or Imutils
on a desktop using facial gestures. It starts by is used to isolate this area for real-time tracking. The
capturing real-time input from the user, which is system then monitors the iris position within the eye

2
Eye Sync: Synchronizing Eye Movements with Mouse Cursor Control for Enhanced
Productivity

region to determine the user's gaze direction. Gaze


Estimation (Determining Gaze Direction): By
analyzing the position of the iris or pupil, the system
determines the direction of the user's gaze (left,
right, up, or down). Various techniques, such as the
Hough transform or direct pixel-based analysis, can
be applied to estimate gaze direction. This
information is then translated into corresponding
cursor movements on the screen, enabling the user
to control the pointer through eye movement.

Fig 2 Flowchart of Cursor Control

These algorithms analyze the direction of the user's


gaze and determine the intended movement. This
Fig 1 Mouse Cursor Control System Architecture information is then used to map the gaze direction to
corresponding cursor movements on the screen,
IV. FLOWCHART effectively allowing the user to control the cursor
Our flowchart represents the real-time controlling without physical input devices such as a mouse. The
of cursor using eye tracking, facilitated by OpenCV. final steps in the process include executing cursor
The process begins with capturing live video input movements and simulating click events. The system
from a webcam, which serves as the foundation for translates the detected eye movements into precise
detecting and tracking facial and eye movements. cursor displacement on the screen, ensuring smooth
The next step involves leveraging OpenCV, an open- and responsive control.
source computer vision library, to process the video
frames and detect facial features. This includes V. SEQUENCE DIAGRAM
identifying the face within the video frame, which is Sequence diagram would depict the step-by-step
crucial for isolating the eye region to proceed with interaction between components involved in
further tracking and cursor control operations.Once capturing real-time video, processing facial and eye
face detection is complete, the system moves on to movements, and controlling the cursor. The
eye detection, which focuses on pinpointing the sequence starts with the user interacting with the
exact position of the eyes within the detected face. system, followed by the webcam capturing video
input. The face detection module processes the
frames, identifying facial landmarks and locating the
eyes. The eye tracking algorithm then continuously
monitors the gaze direction, sending movement
signals to the cursor control module, which updates
the cursor's position accordingly. Finally, the click
event handler detects intentional gestures (such as
blinking or gaze fixation) to simulate mouse clicks.

3
Eye Sync: Synchronizing Eye Movements with Mouse Cursor Control for Enhanced
Productivity

TensorFlow. For PC-based systems, Linux-based


distributions such as Ubuntu or mainstream options
like Windows 10/11 are recommended.
Programming Languages: This project primarily
utilizes Python due to its ease of use and extensive
library support for visual computing, ML, and
hardware integration. OpenCV is employed for real-
time eye detection and tracking, while Dlib is Used
for detecting key facial features, facial landmark
detection is a technique that identifies and maps
specific points on a person's face to accurately
identify the eye region.
Machine Learning Frameworks: Machine
learning frameworks Improve the precision and
reliability of gaze tracking in dynamic
Fig 3 Sequence Diagram environments. TensorFlow and PyTorch are the two
leading frameworks for this project, offering tools to
train or fine-tune models for detecting gaze direction
VI. IMPLEMENTATION AND TESTING
and pupil movements.

i) Hardware Implementation
VII. Different Data Values
Eye-Tracking Camera : An eye-tracking camera is
In the mouse cursor control project using eye
the core hardware component that captures eye
movements, different data values refer to the variety
movements in real-time. A high-frame-rate webcam
of inputs and scenarios that the system needs to
(30–60 frames per second) is essential to ensure
process, analyze, and respond to accurately during
smooth and accurate tracking of quick eye
testing and operation. These data values represent
movements.
variations in user behaviour, environmental factors,
Processing Unit: The processing unit is responsible
and system configurations that can influence
for performing gaze detection and managing cursor
performance.
movement. The choice of processing unit depends
Gaze Data Values The system must track gaze
on the application's needs and performance
points across the screen, ranging from the center to
requirements. For standalone implementations, a
the extreme edges. Different users move their eyes
microprocessor such as the Raspberry Pi serves as
at varying speeds, which the system should
an effective solution.
accurately capture and translate into smooth cursor
Display Monitor: A standard computer screen is
movement. Variations in blinking rates and
used as the visual interface for the system. The
durations need to be handled, especially if blinks are
monitor serves multiple purposes, such as providing
used for click actions.
feedback on cursor movements and displaying
User-Specific Variations. Differences in eye shape,
calibration targets during the setup process.
size, color, and other physiological features between
Power Source and Storage: The system requires a individuals. Reflections or distortions caused by
stable power supply to ensure uninterrupted eyewear can alter the accuracy of tracking and must
operation. In desktop-based setups, power can be be addressed. Variability in head poses or tilts while
drawn directly from the PC for peripherals like the interacting with the system.
camera and IR LEDs.
VIII. Results and Discussions
ii) Software Implementation This section introduces a hands-free assistive
Operating System: The operating system forms the
foundation for the software implementation, technology that enables users to operate a computer
offering compatibility with the hardware and cursor solely through eye movements. The system
libraries. For a standalone system using Raspberry employs computer vision techniques, including
Pi, Raspberry Pi OS is the ideal choice as it is
OpenCV-based face and eye detection, along with
lightweight, optimized for the hardware, and
supports Python libraries like OpenCV and eye-tracking algorithms, to convert a user's gaze into

4
Eye Sync: Synchronizing Eye Movements with Mouse Cursor Control for Enhanced
Productivity

cursor navigation and intentional blinks into click gaze direction and translates it into smooth cursor
actions. navigation on the screen. By continuously tracking
eye position, it converts visual focus into directional
commands, enabling effortless cursor movement in
Figure 4 highlights an additional functionality where all directions—left, right, up, or down—based on
the system detects when a user opens their mouth, where the user is looking.

triggering the activation of eye-tracking inputs. In


X. REFERENCES
the displayed image, facial landmark detection
[1] A. S. Rama Chandra Murthy, D. Anitha, N. M.
successfully identifies the open mouth, signaling the Chowdary and S. Siddhartha Sai Kumar,
system to initiate gaze-based cursor control. "Mouse Cursor Control with Eye
Gestures," 2024 International Conference on
Inventive Computation Technologies (ICICT),
Lalitpur, Nepal, 2024, pp. 980-983, doi:
10.1109/ICICT60155.2024.10544549.

[2] S. Kanakaprabha, P. Arulprakash, S. Praveena,


G. Preethi, T. S. Hussain and V. Vignesh,
"Mouse Cursor Controlled by Eye Movement
for Individuals with Disabilities," 2023 7th
International Conference on Intelligent
Computing and Control Systems (ICICCS),
Fig 4 Activated Cursor State Madurai, India, 2023, pp. 1750-1758, DOI:
10.1109/ICICCS56967.2023.10142467.
Fig. 5 demonstrates the successful movement of the
cursor based on eye gaze direction. In the provided [3] A. Dongre, R. Pinto, A. Patkar and M. Lopes,
image, the system accurately detects the user's eye "Computer Cursor Control Using Eye and Face
Gestures," 2020 11th International Conference
position and determines that the gaze is directed
on Computing, Communication and
towards the right. As a response, the cursor moves Networking Technologies (ICCCNT),
smoothly in the rightward direction, aligning with Kharagpur, India, 2020, pp. 1-6, doi:
the detected gaze movement. 10.1109/ICCCNT49239.2020.9225311.DOI:
10.1109/ICCCNT49239.2020.9225311

[4] K. Takemura, K. Takahashi, J. Takamatsu, and T.


Ogasawara, “Estimating 3-D point-of-regard in
a real environment using a head-mounted eye-
tracking system,” IEEE Transactions on
Human-Machine Systems, vol. 44, no. 4, pp.
531–536, 2022.

[5] O. Ferhat and F. Vilarino, “Low cost eye


Fig 5 Left Movement of Cursor tracking: the current panorama,”
Computational Intelligence and Neuroscience,
vol. 2016, Article ID 8680541, pp. 1–14, 2021.

IX. CONCLUSION [6] D. Neogi, N. Das and S. Deb, "BLINK-CON: A


HANDS FREE MOUSE POINTER
Our project focuses on developing a system for
CONTROL WITH EYE GAZE
navigating and operating the mouse cursor through TRACKING," 2021 IEEE Mysore Sub Section
eye movement, along with additional features for International Conference (MysuruCon),
enhanced control and system activation. The Hassan, India, 2021, pp. 50-57, DOI:
primary goal of this system is to empower users to 10.1109/MysuruCon52639.2021.9641704.
interact with a computer interface using only their
eye movements, making digital access more
inclusive, especially for individuals with limited
mobility. Utilizing real-time eye-tracking
technology, the system precisely monitors the user's

5
Eye Sync: Synchronizing Eye Movements with Mouse Cursor Control for Enhanced
Productivity

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy