A Robust Technique for Delineation and Features

Extraction ofECG Signal from Mobile-Phone

Photography

Rupendra Nath Mitra Sucharita Mitra

Network Planning & Optimization Department ofElectronics
Nokia Siemens Networks Pvt. Ltd. Netaji Nagar Day College
Kolkata, India Kolkata, India
rupen.mitra@gmail.com susa68@hotmail.com

Sayak Pramanik Bidyut Baran Chaudhuri

Department ofElectronics & Communication Computer Vision & Pattern Recognition Unit
Institute ofEngineering and Management Indian Statistical Institute
Kolkata, India Kolkata, India
sayakpramanik@yahoo.com bbc@isical.ac.in

Abstract-This paper reports the development of a software suite through remote tele-medicine. This requires an efficient
to be accessed in future with any General Packet Radio Service machine intelligence which is capable of receiving and
(GPRS) or High Speed Packet Access (HSPA) enabled mobile analyzing the ECG images with no human interference or
phone or Personal Digital Assistant (PDA) for the extraction and minimal interference of semi-skilled resource. We consider the
analysis of disease-related features from the photograph of paper soft copy images of ECG strips to be analyzed through our
based ECG records. In India and other developing countries, the algorithm as the paper strip based ECG is prevalently used in
cheaper paper based ECG machines are prevalently used. In
these countries. We have reviewed a substantial amount of
rural areas of these countries cardiac diseases are still the major
approaches proposed since last couple of decades for ECG
silent killers due to the acute dearth of qualified cardiologists.
signal or image processing towards automated feature
One way of addressing this problem is Tele-medicine which
extraction and abnormality detection.
necessitates an intelligent cardiac parameter extraction
algorithm. In our bid to address this requirement, an algorithm is Our previous endeavors deals with several types of ECG
developed with the help of few image processing techniques. signals viz. scanned image, electrical signal of ECG wave,
Initially, the background noises i.e. the gridlines are removed by digitally sampled ECG signals etc [1 to 6]. In present
thresholding technique. Applying the Sauvola method for endeavour we exclusively focus on the digital photographs of
adaptive image binarization and subsequent morphological
paper based ECG images. Substantial amount of image
operations to get pure ECG signature on white background, this
processing analysis, transmission technique and machine vision
algorithm intelligently applies Bresenham's line drawing
algorithms have not been developed on this form of ECG
algorithm to join the disjoined ECG signature where required.
records. Jimena Rodriguez et al. proposed a smart PDA real­
Then thinning has been done for extraction of digital time-plane
data and then Discrete Wavelet Transform (DWT) and water
time ECG rhythm and beat classifier with modern wireless
reservoir based pattern recognition technique are subsequently
technologies viz. Bluetooth, infrared as aiding connectivity to
used to delineate other important time-plane features for ECG
the higher digital equipment [9]. QRS Diagnostic
interpretation. commercializes the EKG Card, which converts a laptop, or
personal digital assistant (PDA) into an electrocardiograph
Keywords- Bresenham Line Drawing Algorithm, Discrete monitor that allows the visualization and storing ofECG Data
Wavelet Transform (DWT), ECG Feature extraction, Sauvola [10]. Vitaphone markets a digital health card that transmits
Binarization technique, Water Reservoir technique. ECG data by infrared to a GPRS based mobile phone that
transmits the ECG to a centralized server where the ECG
I. INTRODUCTION analysis can be done [11]. A wide range of techniques has been
This Electrocardiogram (ECG) is a non-invasive, used, including statistical pattern recognition, Expert Systems,
transthoracic diagnostic technique which gives an inkling of the Artificial Neural Networks, Wavelet Transform, Fuzzy and
electrical activity of human heart over time. For the interior Neuro-fuzzy Systems for auto diagnosis. Hermite functions and
most rural parts of countries like India, Pakistan, Bangladesh Self- organizing Maps are used for clustering the signal
we can provide on time cardiac monitoring and suggestions complexes [12]. More recently, artificial neural network

978-1-4673-4700-6/12/$31.00 ©2012 IEEE 121

Authorized licensed use limited to: VIT University. Downloaded on October 20,2020 at 03:44:17 UTC from IEEE Xplore. Restrictions apply.
techniques have also been used for signal classification [13]. The following equation is used to calculate the threshold
Moreover, Fuzzy approach is proposed for computerized considering the values of the neighboring pixels,
electrocardiogram diagnosis [14]. All these technological
proposals are mainly based on transmission and analysis of
digital ECG signals but countries like India are still indulging
T(x, y) = m(x, y). [ 1 + k. ec:y) - 1 )] (1)
in paper recording based ECG. In developing countries like
India preliminary cardiac health monitoring is performed on the Where local mean m(x,y) and local standard deviation
basis of Electrocardiogram (ECG). The main principle of s(x,y)are as in Niblack's formula [18] also R is the dynamic
disease detection lies in various patterns of ECG waves. Our range of standard deviation, and the parameter k gets positive
previous communication reported an efficient algorithm which value. Subsequently Bresenham's line drawing algorithm was
performs pattern recognition to extract the time plane features applied to join whatever discontinuity was present in the ECG
of ECG wave (Fig. 1) and subsequently calculates other image [19]. Fig. 1 depicts the exclusive output of the above
clinically required parameters [7]. In this paper we added said modules of the algorithm. Then point to point data signal
further intelligent techniques to make the algorithm much extraction was done on the image obtained.
robust. This newly equipped algorithm is now capable of
handling noises and blurriness which becomes integral part of a
photograph taken hastily specially in case of mobile
photography. As the basic software package detects patterns,
delineates time-plane features and calculates essential clinical
measurements of ECG wave hence it can be extended further
as a cardiac disease inference engine by just configuring
different conditions to diagnose a particular disease. The
conclusion in favor of any cardiac disease can be made on the
basis of a knowledgebase regarding the abnormal patterns and
estimations of ECG signature.

QRS
voltage
Pwidth Figure 2. At the top two sample mobile photographs of ECG strips which are
used. At middle the outputs of the Sauvola [mage Binarization technique. At
bottom the binarised images of pure ECG signatures intelligently rejoined by
Bresenham Line joining algorithm

The most important parameter of ECG signal is patient's

ST intervCll heart rate. In general R peak is the high frequency component
QRS in a QRS complex and its general location was found by
Width computing the 2nd order derivative of the captured signal by
using 5-point Lagrangian interpolation formula for
QTintelvaJ
differentiation. Thus two consecutive set of R peaks' locations
were noted for performing mean heart beat calculation. The
wavelet transform provides a description of the signal in the
Figure I. Different time plane features of ECG time-scale domain which helps in the depiction of the different
characteristic features of a signal at different resolutions; hence
II. METHODOLOGY it is an appropriate tool to analyze the ECG signal, which is
Under this scheme, from the patient side, an ECG characterized by a cyclic occurrence of patterns with different
technician will generate snapshots of the paper ECG report frequency content (QRS complexes, P and T waves). Therefore
with the help of a mobile phone or digital camera and send the I-dimensional DWT is used to decompose the ECG signal into
images to the processing unit through mobile internet. This is higher as well as lower frequency contents [17]. As there is no
one of the ways of distant image acquisition techniques over well defined criterion for choosing the mother wavelet, "db2"
mobile internet. The developed software then intelligently is taken as the mother wavelet from the Daubechies family as it
filters and subsequently extracts important ECG parameters produces the optimum result. The delineated QRS complex
from the received ECG image. The subsequent processing and hence obtained needs optimization as ECG signal is erratic and
analyzing techniques are elucidated very concisely with real each person's ECG signature differs significantly. To find the
input images used and the output of different modules of the accurate QRS complex, a pattern recognition feature named
proposed algorithm under this section. water reservoir was employed for labeling the crest portion as
R and the trough portion as S or Q in the thinned image. The
As the varied ECG images can be from different sources basic is that if 2D equivalent of water is poured from a side of a
under different conditions, Sauvola's textual binarization component, the concave regions of the component where water
process was implemented to calculate the modified threshold. will be stored are considered as reservoirs ( [20]-[21]). Likewise

122 2012 International Conference on Communications, Devices and Intelligent Systems (CODIS)

Authorized licensed use limited to: VIT University. Downloaded on October 20,2020 at 03:44:17 UTC from IEEE Xplore. Restrictions apply.
if the component is rotated 180 degree then the crest regions but the patterns of different time-plane features are also
become reservoirs. In the present case we assume water is compared.
being poured on the ECG signal, the portions where water
Table II enlists the findings by the developed algorithms of
could be stored were labeled as Q or S. Figure 3 shows the
few of the samples. For the better elucidation of the evaluation
pixel filled snap of two of the processed images. The extracted
process we have given the collection of the original photos of
data is then processed and analyzed by the software, which
the four samples which were actually used as the input to the
generates an initial report about the abnormality of the
incoming ECG from a decision based algorithm, which has algorithm in Fig. 4. Table I gives a statistical viewpoint to the
accuracy level of the proposed algorithm where the mean of the
been developed from the feedback of cardiologists and by
differences (up to 2 decimal places) between the measurements
consulting some books on cardiology ([15]-[16],[8], [21]). The
of the program and the ones provided by the medical
main advantage of the system is that it will not require constant
practitioner have been calculated. The data for the tables are
monitoring or presence of any doctor at the initial stage.
not computed on the basis of only the four samples given in
Table II but for more than 100 samples. As far as the clinical
parameters are concerned the computation is done on the input
of lead II of the paper ECG report. The most frequently asked
clinical parameters which could be calculated from a standard
12 leadECG are Heart Rate, QRS voltage, R to R duration and
QRS duration. The algorithm we have developed calculates the
above said cardiac parameters and shows them all together in a
precise format. To evaluate the accuracy in calculating the
cardinal clinical parameters fromECG wave we produce mean
of the differences between the measurements of the program
and the ones observed by the medical practitioner. It was
fascinating to come up with the result given in Table I where
Figure 3. QRS detected from ECG signals
for each of the parameter the developed algorithm maintains a
precision of ±1%. Heart rate is calculated from R-R duration of
III. RESULTS & DISCUSSIONS
lead II. In Table I these clinical parameters are given as
Since an intelligent ECG features extraction algorithm is calculated by the developed algorithm and the same by medical
developed deploying some image processing and pattern expert for four samples out of a large pool. The pattern
recognition techniques, performance of this algorithm must be recognition of QRS complex of all the different leads are done
reviewed. Apart from that, the efficiency of pattern extraction by deploying water reservoir technique already elucidated. This
and accuracy of cardiac parameters calculated are also tested novel technique evolves as a very useful one to recognize the
and evaluated in this section. In this article we only considered different patterns constitute QRS complex ofECG wave. In our
the soft copy images of paper strip based ECG records. We previous communication we reported the pattern recognition of
collected a number of paper ECG records from local QRS complex only; here we are successfully reporting the
diagnostics centers and hospitals and took the snap of theECG pattern recognition of P and T wave respectively. Table II
signature of different leads. The images were taken with the describes that the developed algorithm recognized the QRS
primary camera of a standard mobile phone. To evaluate the complex, P wave and T wave pattern almost flawlessly. Here
performance of the developed algorithm we compared several we have placed four different samples out of a large pool Fig.
4. As far as U wave is concerned it is typically small and not
always observed in an electrocardiogram. The proposed
algorithm is also competent enough to detect the U wave in an
ECG by using water reservoir technique, as done in the
detection of T and P wave. U wave can be distinguished from
the P wave of next cycle by duration as the area of the
container of P wave is much greater than that of U wave.

TABLE l. STATISTICAL OBSERVATION OF ACCURACY IN DETECTION

Parameters Mean of the deviation

Heart rate (beat per minute) -I

Duration of QRS cOIl1pie-c (ms) -0.99

Figure 4. Randomly selected photos as sample of paper ECG
QRS voltage (V) -0.99
cardiac parameters computed by the algorithm to the values
R-to-R Duration -0.99
computed by experts. Not only the values of clinical parameters

2012 International Conference on Communications, Devices and Intelligent Systems (CODIS) 123

Authorized licensed use limited to: VIT University. Downloaded on October 20,2020 at 03:44:17 UTC from IEEE Xplore. Restrictions apply.
 TABLE [I. ALGORITHM PERFORMS ITS PROCESSING TECHNIQUES ON THE A PORTION OF THE DI FFERENT EXTR ACTED FEATURES CAL C UL ATED BY THE
DEVELOPED ALGORITHM

QRS Complex P wave Twave QRS Duration QRS voltage R-to-R

Description Pattern Pattern Pattern (mS) (mY) Duration Heart Beat Rate
Non-
Sample] RS Inverted inverted 75 0. 68 598 100

Sample 2 QR Inverted Inverted 95 1. 54 786 76

Non- Non-
Sample 3 RSr inverted inverted 93 1. 02 757 79
Non- Non-
Sample 4 qRS inverted inverted 92 1. 66 587 102

[6] U. Pal and P. P. Roy, "Multi-oriented and curved text lines extraction
IV. CONCLUSIONS from Indian documents," IEEE Trans. On SMC - Part B, vol. 34, pp.
The theme of our work is to come up with a robust 1676-1684,2004.

algorithm which is capable enough to recognize and delineate [7] S. Pramanik, S. Mitra, R.N.Mitra, B.B.Chaudhuri, "A novel approach
for delineation and feature extraction in QRS complex of ECG
the time plane features and extract other clinically required
signal",IEEE International Conference on Image Information
parameters from a soft copy image of an ECG strip. The soft Processing,Nov. 2011, [SBN: 978-1-61284-859-4 ( Print Version)
copy images (say .jpgl.gif format) of theECG strips are used as [8] J. R. Hampton, The ECG Made Easy, 6th ed. , Churchill Livingstone,
input to the algorithm. As of now we successfully reached at a 2003.
length from where it is possible to locate and recognize the [9] J. Rodriguez, A. Goiii, and A. IIIarramendi, "Real-Time Classification of
pattern of different time plane features and delineate the ECGs on a PDA," IEEE Transactions on Information Technology in
significant patterns with immense medical implications with Biomedicine,vol. 9,no. I,March 2005,pp. 23 - 34.
astonishing accuracy. Not only that the clinically required [10] (2011) The QRS Diagnostic website. [Online]. Available:
cardiac parameters, reported in the performance evaluation http://wwwqrsdiagnostic.com/

section of this article, are also computed with extreme precision [II] N. Daja, L Relin, and B. Reljin, "Telemonitoring in cardiology-ECG
transmission through mobile phones," J.Annals Academy Studenica,
with this robust algorithm. In our last communication of this
vol4,200 I,pp. 63-66.
work we have successfully done all these things but only for
[12] M. Lagerholm, C Peterson,G. Braccini, L Edenbrandt, and L. Si)rnmo,
the QRS complex and in this article we are successfully "Clustering ECG complexes using hermite functions and self­
reporting the advancement of the work as the successful pattern organizing maps," IEEE Trans Biomed Eng., July 2000, 47(7), pp. 838-
recognition, delineation and computation of clinical parameters 48.
for the other two important time plane features of ECG [13] G. Bortolan, C Brohet, and S. Fusaro, "Possibilities of using neural
signature, P and T wave [7]. We strengthen the algorithm to networks for ECG classification," Journal of Electrocardiology, vol. 29,
1996,pp. 10-[6.
make it work for the images with poor qualities like defocused
ECG signature, blurred gridlines or uneven distribution of [14] R. Degani, "Computerized Electrocardiogram Diagnosis: Fuzzy
Approach," Methods of Inform. Med., 31(1992), pp. 225-233.
luminance over the image. In this context it is apt to mention
[15] M. J. Goldman, Principles of Clinical Electrocardiography, Lange
that we have extended this algorithm for automatic cardiac Medical Pubns., 12th edition,1986.
disease detection and have carried through testing. Still we are
[16] A. L. Goldberger, Clinical Electrocardiography: A Simplified Approach,
working on it to come up with a complete cardiac disease 6th ed. ,Harcourt India Pvt Ltd.
inference engine with intelligent feedback capability by [17] S. Mallat, "A theory for multi resolution signal decomposition: the
deploying the presently available wireless accessibility (like wavelet representation," IEEE Pattern Anal. and Machine IntelL,vol. II,
EDGE) features and high speed packet accessibility (HSPA or no. 7,pp. 674-693,1989.
LTE). [18] J. Sauvola, M. PietikaKinen, " The journal of the pattern recognition
society", issue 33 (2000), pp. 225-236
REFERENCES [19] The Bresenham Line-Drawing Algorithm,
.. http://wwwcs.helsinki. fi/group/goa/mallinnus/lines/bresenh. html ..
[I] S. Mitra and M. Mitra, "An automated data extraction system from 12
lead ECG images," Computer Methods and Programs Biomedicine, [20] U. Pal and P. P. Roy, "Multi-oriented and curved text lines extraction
2003 May,vol. 71,pp. 33-38. from Indian documents," IEEE Trans. On SMC - Part B, vol. 34, pp.
1676-1684,2004.
[2] S. Mitra, M. Mitra, and B. B. Chaudhuri, "Pattern defined heuristic rules
and directional histogram based online ECG parameter extraction," [21] K. Roy, A. Banerjee, and U. Pal, "A system for word-wise handwritten
Measurement,vol. 42,Issue I,Jan 2009,pp. 150-156. script identification for Indian postal automation," India Annual
Conference,2004,Proceedings of the IEEE INDICON 2004,Issue 1.
[3] S. Mitra, B. B. Chaudhuri, and M. Mitra, "A Simple Online Histogram
and Pattern Recognition Based ECG Analyzer," IETE journal of [22] Jalel Chebil, Jamal AI-Nabulsi and Mohammed AI-Maitah, "A Novel
research,2008,vol. 54,Issue 3,pp. 241-248. Method for Digitizing Standard ECG Papers", Proceedings of the
[4] M. Mitra and S. Mitra, "A Software Based Approach for Detection of [nternational Conference on Computer and Communication Engineering,
Kuala Lumpur,Malaysia,pp.1308-1312,2008.
QRS Vector of ECG Signal," 3rd Kuala Lumpur International
Conference on Biomedical Engineering 2006, [FMBE Proceedings, [23] M. Sanroma·n-Junquera, LMora-Jime'nez , A,J.Caaman-o , lAlmendral
2007,vol. 15,part 10,pp. 348-351. , F.Atienza , L. Castilla, A.Garcl'a-Alberolad, J.LRojo-A· Ivarez,
[5] S. Mitra and M. Mitra, "Detection of QRS Complex of ECG Signals "Digital recovery of biomedical signals from binary images," Vol 92
Issue I,Jan 2012 ,Pages 43-53.
from Square-Derivative Curve," AMSE journal (Advances in
Modeling),France,vol. 65(2),pp. 33-38,2003.

124 2012 International Conference on Communications, Devices and Intelligent Systems (CODIS)

Authorized licensed use limited to: VIT University. Downloaded on October 20,2020 at 03:44:17 UTC from IEEE Xplore. Restrictions apply.