SECURE IMAGE FILTERING

Nan Hu, Sen-ching S. Cheung

Thinh Nguyen

University of Kentucky
Electrical & Computer Engineering
Lexington, KY, 40506
nan.hu@uky.edu, cheung@engr.uky.edu

Oregon State University

EECS Department
Corvallis, OR 97330
thinhq@eecs.oregonstate.edu

ABSTRACT
In todays heterogeneous network environment, there is a growing
demand for distrusted parties to jointly execute distributed algorithms
on private data whose secrecy needed to be safeguarded. Protocols
that support such kind of joint computation without complete sharing of information are called Secure Multiparty Computation (SMC)
protocols. Applying SMC protocols in image processing is a challenging problem. Most of the existing SMC protocols are implemented based on cryptographic primitives like Oblivious Transfer
that are too computational intensive for pixel-based operations. In
this paper, we develop two efficient SMC protocols for distributed
linear image filtering between two parties, one party with the original image and the other with the image filter. The first protocol is
based on a combination of rank reduction and random permutation.
The second one uses random perturbation with the help of a noncolluding third party. Experimental results show that both of them
execute significantly faster than oblivious-transfer based techniques.
Index Terms Communication system security, Image Processing, Distributed Algorithms, Cryptography
1. INTRODUCTION
The proliferation of imaging and storage devices and the ubiquitous
presence of computer networks make sharing of visual data easier
than ever. Such casual exchange of visual data, however, has increasingly raised questions on how sensitive visual information can be
protected. Consider the scenario in which a user of a cellular-phone
camera wants to send his/her pictures to an online photo-processing
laboratory for image enhancement such as red-eye removal. The
user would be concerned about the privacy of his/her pictures while
the online store would need to protect the proprietary enhancement
technologies against reverse-engineering. Consider another scenario
that the police wants to search for possible suspects in a surveillance video owned by private company A, using a proprietary software from yet another private company B. The three parties involved
(police, company A, company B) all have information they do not
want to share with each other (criminal biometric database from the
police, surveillance tape from company A and proprietary software
from company B). To support such kinds of applications, one needs
to establish a joint computation and communication platform at the
pixel level that can guarantee the secrecy of private data and algorithms, and at the same time achieve a well-defined objective that
benefits all parties involved.
This is, however, not a new problem. Cryptographic protocols
that support joint computation among multiple distrusted parties without complete sharing of information are called Secure Multiparty

Computation (SMC) protocols. The general problem of SMC can be

traced back to the classical paper by Yao [1]. The biggest challenge
to any SMC technique is how to satisfy the security requirement.
There are two types of security models - the ideal model and the
real model [2]. In the ideal model, the computation is performed by
a separate party trusted by all participants. This is perfectly secure
as none of the participants gain information beyond the knowledge
of their own data. However, the ideal model is not practical because
all critical data reside at a single party and can easily be compromised. In the real model, the computation is performed via a SMC
protocol agreed upon by all parties. The protocol is said to be secure
in a semi-honest environment if all parties respect the protocol and
are not able to derive more information than what can be deduced
from the final results. Most of the SMC protocols, including those
described in this paper, are developed under this model. Many SMC
problems can be solved by using the Oblivious Transfer (OT) protocol in which one party (Alice) can select a particular item from a
set owned by another party (Bob) without Bob knowing Alices selection or Alice knowing other items from Bobs set [2]. The OT
Transfer is typically implemented using different public-key encryption systems [3].
There has been little work in applying SMC to image processing
problems. The only work known to us is by S. Avidan et al. on applying classical SMC protocols for two-party face detection [4]. In a
typical classification task such as face detection, a significant portion
of an image is transformed into feature vectors, which in most cases
cannot be used to recover the original image. The manipulation of
feature vectors is thus secure by definition and no special SMC protocols are required. As a result, the complex SMC protocols do not
significantly affect the overall performance of the classification task.
On the other hand, many common image processing applications
require pixel-by-pixel processing. The sheer number of pixels in
common images render most of the classical SMC techniques useless. Thus, it is imperative to develop fast computation techniques
for these applications. Among all the image processing techniques,
linear filtering is arguably the most basic and useful one. It is used in
almost all image processing and computer vision applications such
as enhancement, denoising, and feature extraction. Even though linear filtering by itself is inherently insecure as we will demonstrate
in Section 2, we focus in this paper on secure and efficient linear
filtering algorithms that can be used in conjunction with other types
of non-linear processing. Our contributions are two novel secure linear filtering protocols that are significantly faster than those based on
OT. These protocols are capable of processing mega-pixel images in
less than one second using common desktops.
The organization of the paper is as follows: our starting point in
Section 2 is the problem definition and some notations of linear filter-

ing. The classic two-party protocol based on OT is briefly reviewed

in Section 3.1. In Section 3.2, we review the secure inner product
protocol in [5] and introduce our two-party linear filtering protocol
based on this inner product protocol. One drawback of this new protocol is that it cannot be used successively to implement multi-stage
linear filtering. To solve this problem, we need the help from a third
party. As we show in Section 3.3, the protocol guarantees that none
of the parties have full knowledge of the data as long as no two parties collude to share data externally. The efficiency of the proposed
techniques are demonstrated by the experimental results in Section
4. Finally, we conclude the paper and describe ongoing work in Section 5.
2. PROBLEM DEFINITION AND NOTATIONS
Given an image {x(m, n) : 0 m M, 0 n N } and a
filtering operation f () described by a set of parameters , we define
the output y(m, n) of applying this filter to x as follows:
y(m, n) = f (x; )

(1)

In the secure image filtering model, we have two users, Alice and
Bob, who own the image data x and the filter parameter respectively. Our goal is to establish a computation protocol between Alice
and Bob so that
1. Alice obtains f (x; ) without any knowledge of , and
2. Bob does not know anything about x.
For linear filtering, the filter parameters are specified as a filter
mask h defined as {h(m, n) : 2l m 2l and 2l n 2l }.
The linear filtering operation can then be written as
y(m, n) = x h =

l/2
X

l/2
X

h(i, j)x(m i, n j).

(2)

i=l/2 j=l/2

It is easy to see that Equation (2) is a scalar product between two

(l + 1)2 dimensional vectors.
All of our secure linear filtering protocols use the following conceptual model: Alice first forms a M N (l + 1)2 matrix Xw whose
rows are the signal data needed for the inner product operation. The
total number of rows of Xw is the total number of pixels in the output
image1 . If we denote the i-th row of Xw as Xw (i, :) and the output
image as a vector y = [y1 yM N ]T , the image filtering could be
written as
y = Xw h
(3)
A secure linear filtering protocol decomposes every yi = Xw (i, :)h
into yi = a + b such that Alice computes a without any knowledge
of h and Bob computes b without any knowledge of Xw (i, :).
If linear filtering is the end goal of the processing, Bob sends
back his portion to Alice to compute the output image y. Using
both the input x and y, Alice can estimate h using the least square
estimate
T
= (Xw
h
Xw )1 Xw y.
In other words, linear filtering is intrinsically insecure to Bob no
matter how secure the protocols are. General non-linear filtering, on
the other hand, is much harder to invert based on limited number of
input and output pairs. If linear filtering is used as part of a secure
non-linear image processing system, it is thus important that neither
Alice nor Bob has the entire output of the linear filtering protocol.
1X

can be made to have M N rows with appropriate boundary handling.

3. SECURE LINEAR FILTERING

In this section, we will develop two types of secure filtering protocol:
1) a two-party protocol based on rank reduction and random permutation and 2) a three-party protocol based on random perturbation of
the data. Before introducing the new protocols, we first review the
classic two-party protocol based on OT. A detailed explanation of
OT could be found in [2], and an application of OT in face detection
could be found in [4].
3.1. Classical Two-party Solution
Oblivious transfer allows Alice to select one element from the whole
dataset Bob holds without revealing to Bob which element Alice has
selected and without knowing any other element in the dataset rather
than the one selected. Our current notion of oblivious transfer was
originally suggested by Even, Goldreich and Lempel [6] as a generalization of Rabins oblivious transfer [7]. We will denote OT1m
to indicate the protocol that Alice choose one out of the m elements
from Bobs dataset. The state-of-the-art OT1m requires computing at
least two long integer modular exponential, typically in the range of
512 bit to 1024 bit, per selection [3].
Secure scalar product can be implemented based on oblivious
transfer as shown in [4]. Alice and Bob have x, h F n respectively, where F is a finite field (0-255 for grayscale image pixels).
Let the totalPnumber of element in the finite field F be NF . To comn
pute y =
i=1 xi hi , Bob computes for each hi a table of NF
entries. The j-th entry of the table is j hi ri where ri , 1 i n
are random numbers known only to Bob. Alice and Bob then engage
in N rounds of OT1NF protocols in which Alice selects the j-th entry ofPthe table in the i-th round if xi = j.
Finally, Alice computes
Pn
a= n
i=1 ri . It is obvious that
i=1 xi hi ri and Bob has b =
y = a + b while Bob and Alice know nothing about each others
vector. To prevent Alice from inferring Bobs data based on successive execution of OT, Bob must use unpredictable encryption keys in
each OT which also adds to the complexity of the algorithm.
3.2. Two-party Solution
Our first protocol on linear filtering is based on the secure inner protocol proposed in [5]. We describe this protocol using the pseudocode InnerProductAlice and InnerProductBob listed below. Alice has a n-dimensional vector x and Bob has a n-dimensional vector y. They both know an invertible matrix P and itsn inverse P 1 .
B
P is broken
down into top and bottom halves T Rb 2 cn and
n
1
nb n
c
2
R(nb 2 c)n , while
P
into
left
and
right
halves
L

R
and
n
R Rn(nb 2 c) . The inner product xT y can then be decomposed
as follows:
xT y = xT P 1 P y = xT LT y + xT RBy

(4)

Alice then sends xT R to Bob who computes xT RBy while Bob

sends Alice T y so that she can compute xT LT y.
The security of the protocols comes from the observation that
xT R and T y project x and y into lower-rank subspaces, and thus
the components of the original vectors inside the null spaces of the
matrices are irrecoverably lost. In [5], the authors proposed a design
of P based on decoding matrices used in error control coding so as
to spread the projections of neighboring vectors as far as possible. In
this paper, we adopt this method to generate an invertible matrix G
and compute the actual P by multiplying G with a random orthogonal matrix. This approach allows us to generate different P if the
protocols require multiple invertible matrices. Unlike the OT-based

Algorithm 1 InnerProductAlice(x, P 1 )
n

Require: x R . P
= (L R) is a n n invertible matrix
where n 2; L Rnbn/2c and R Rn(nbn/2c) .
1: x1 LT x
2: x2 RT x
3: Transmit x2 to Bob.
4: Receive y1 from Bob.
5: return x1 T y1
Algorithm 2 InnerProductBob(y, P )
Require: y Rn . P T = (T T B T ) is a n n invertible matrix
where n 2; T Rbn/2cn and B R(nbn/2c)n .
1: y1 T y
2: y2 By
3: Receive x2 from Alice.
4: Transmit y1 to Alice.
5: return x2 T y2

protocol which requires complex long-integer exponential and random key generation, this protocol requires only the highly optimized
matrix multiplications.
It may seem intuitive to implement secure linear filtering by applying the inner product algorithm on Xw row by row. However, it
is not secure as adjacent rows in Xw overlap with each other. As a
result, the redundancy in the rank-reduced data sent to Bob allows
him to form a least-square estimation of the original image. This
least square problem involves solving a least-square data matrix of
2
c M N . To achieve secrecy, Alice needs to first
size M N b (l+1)
2
randomly scramble the order of the rows in Xw before carrying out
the inner product. Without the proper row order, Bob has no means
of formulating the least-square estimation. The protocol is described
in FilterAlice and FilterBob.
Algorithm 3 FilterAlice(x, P 1 )
Require: x Rm . P 1 is a n n invertible matrix as described in
InnerProductAlice where n is the length of the linear filter.
1: Reformat x into Xw .
2: Scramble the row order of Xw .
3: for i 1 to m do
4:
rA (i) InnerProductAlice(Xw (i, :), P 1 )
5: end for
6: return rA

Algorithm 4 FilterBob(h, m, P )
Require: h Rn . m is the number of inner product operations
required. P T is a n n invertible matrix as described in
InnerProductBob.
1: for i 1 to m do
2:
rB (i) InnerProductBob(h, P )
3: end for
4: return rB
At the end of FilterAlice and FilterBob, the filtered result
y is decomposed into rA + rB where Alice has rA and Bob has rB ,
all in a scrambling order known only to Alice. As discussed earlier,
rB cannot be sent back to Alice for unscrambling as this will allow

Alice to estimate Bobs filter. This creates a problem for multiple

stages of linear filtering as Bob does not know the scrambling order
of rB . Multiple stages of linear filtering are often used in image processing such as separable filtering (horizontal and vertical filtering)
or wavelet transform (multiple stages of subband filtering). Thus, it
is highly desirable to have a secure protocol that can support multiple
stages of linear filtering. This is the subject of Section 3.3.
3.3. Three-party Solution
In this section, we show how to implement the secure linear filtering
with the help of a third party Clark, who we assume will not collude
with either Bob or Alice. This protocol does not use any scrambling
and thus allow multiple stages of linear filtering. On the other hand,
its application is comparatively limited as a non-colluding third party
may not be always present. The basic idea is that, instead of scrambling the rows of Xw , we randomly inject random noise into the rows
of Xw and h for each inner product operation. The dependency between successive rows vanishes as random noise is used each time.
The proposed protocol is shown in 3PartyInnerProductAlice,
3PartyInnerProductBob, and 3PartyInnerProductClark. The
problem notation is the same as in Section 3.2 where Alice holds x
and Bob holds y and they want to jointly compute xT y. Alice generates a random n-dimensional vector xa and computes xb = x xa .
Similarly, Bob generate a random n-dimensional vector ha and computes yb = y ya . Then the inner product can be rewritten as
xT y = xTa ya + xTa yb + xTb ya + xTb yb ,

(5)

xa or xb alone provides no information about x. Neither does

ya or yb alone about y. Unfortunately, it is impossible for Bob
and Alice to compute all the four terms in (5) by just receiving
one component of the vector from each other. For example, if Alice sends Bob xa , he can compute the first and the second terms
rb = xTa ya + xTa yb . If Bob sends Alice ya , Alice can compute
the third term but not the fourth. If Bob sends yb , Alice can compute the fourth but not the third. To solve this conundrum, we introduce a third party Clark. If Bob sends Alice ya and Alice computes
ra = xTb ya , Alice and Bob can send Clark xb and yb so that Clark
can compute the remaining term in (5), i.e. rc = xTb yb . Provided
that no two parties collude with each other, the information Alice,
Bob and Clark have are all random data which disclose no information about either x or h.
Algorithm 5 3PartyInnerProductAlice(x)
Require: x Rn .
1: Generate random vector xa .
2: xb (x xa )
3: Transmit xa to Bob.
4: Transmit xb to Clark.
5: Receive ya from Bob.
6: return ra = xb T ya
To use this new protocol for linear filtering, we can simply modify FilterAlice and FilterBob by removing the scrambling and
replacing InnerProductAlice with 3PartyInnerProductAlice
and InnerProductBob with 3PartyInnerProductBob. At the
end of the protocol, Alice, Bob and Clark will have rA , rB and rC
respectively such that the output image y = rA + rB + rC . To perform a second stage linear filtering with, say Bobs g, we can simply
apply the distribution rule for convolution y g = rA g + rB
g + rC g and compute each term using the secure linear filtering
protocol between Bob and each of Alice and Clark.

Algorithm 6 3PartyInnerProductBob(y)
Require: y Rn .
1: Generate random vector ya .
2: yb (y ya )
3: Receive xa from Alice.
4: Transmit ya to Alice.
5: Transmit yb to Clark.
T
6: return rb = xT
a ya + xa yb

rithm described in Section 3.2 and 3.3 on a dual Wintel CPU (P43.4GHz) desktop with 1GB memory. The reason we did not test
the classic protocol on real images is because it will take hours to
do a linear filtering on a single image. The oblivious transfer based
technique takes about 20 minutes to compute the inner product of
two 20-dimensional vectors while our two-party protocol uses only
30 milliseconds and our three-party protocol uses 47 milliseconds.
Despite our non-optimal implementation of the oblivious transfer
protocol, its slow performance can be attributed to the handling of
very long integers in the encryption/decryption process as well as the
large amount of information exchanged between Alice and Bob. For
linear filtering using a 7 7 Gaussian mask on the same computing
platform, our two-party solution takes on average 0.7 seconds to denoise a 128 128 image and our three-party solution takes around
0.6 seconds.

Algorithm 7 3PartyInnerProductClark()
1: Receive xb from Alice.
2: Receive yb from Bob.
3: return rc = xT
b yb

4. EXPERIMENTS

5. CONCLUSION

All the algorithms described in Section 3 are implemented and tested

with MATLAB version 7.0. To ensure the accuracy of the design,
all data exchange between Alice and Bob are conducted using actual
TCP/IP connections. To encourage future development and independent verification, we have also made all our source code available at
http://www.vis.uky.edu/nhu2/secureImage.
Figure 1(a) shows the original 128 128 Lena image. Figure
1(b) shows the result of filtering Lena using a 77 Gaussian denoising filter. This image is the result of combining Bobs and Alices
partial copies. We use P R4949 to completely hide the details of
the filter and the output of FilterBob is shown in Figure 1(c). As
shown, all the details from the original image are scrambled.

This paper presents two different solutions for the secure linear filtering problem a two-party solution based on rank reduction and
random permutation, and a three-party solution based on random
perturbation. A drawback of the two-party solution is that the scrambling prevents repeated use of the protocols. To avoid scrambling,
our third-party solution relies on a third party to participate in independent random perturbation of each inner product computation.
The comparison between our proposed protocols and the classic protocol based on OT shows that our protocols are much faster and more
appropriate for pixel-wise computation. A more detailed evaluation
of the security of these algorithms, especially under malicious attack, is among the most pressing issue we are currently addressing.
Another important area is to assess the bandwidth requirement for
these algorithms, which could quickly become a bottleneck as we
move from image to video applications..
6. REFERENCES
[1] A. C. Yao, How to generate and exchange secretes, proceeding of 27th FOCS, pp. 162167, 1986.
[2] O. Goldreich, Foundations of Cryptography: Volume II Basic
Applications, Cambridge, 2004.

(a)

(b)

[3] M. Naor and B. Pinkas, Efficient oblivious transfer protocols,

Proc. 12th Ann. Symp. Discrete Algorithms, pp. 448457, 2001.
[4] S. Avidan, Blind vision, ECCV2006, May 2006.
[5] W. Du et al., Privacy-preserving multivariate statistical analysis: Linear regression and classification, proceeding of the 4th
SIAM International Conference on Data Mining, pp. 222233,
2004.
[6] A. Lempel S. Even, O. Goldreich, A randomized protocol for
signing contracts, Communications of the ACM 28, pp. 637
647, 1985.

(c)
Fig. 1. (a) 128 128 Lena; (b) Lena smoothed by a 7 7 Gaussian
filter using secure filtering with a disguise matrix M R4949 ; (c)
is the partial result held by Bob
Our protocols are also computationally efficient. As a comparison, we have implemented a classic two-party protocol based on the
description from [4], using our own 512-bit RSA public-key cryptosystem (PKCS). We then compare its performance with the algo-

[7] M. O. Rabin, How to exchange secrets by oblivious transfer,

Tech. Memo TR-81, Aiken Computation Laboratory, 1981.