On the Complexity of Strong and Epistemic Credal Networks

Denis D. Maua Cassio P. de Campos Alessio Benavoli Alessandro Antonucci

Istituto Dalle Molle di Studi sullIntelligenza Artificiale (IDSIA),
Manno-Lugano, Switzerland,
{denis,cassio,alessio,alessandro}@idsia.ch

Abstract spite this fact, Bayesian networks have been success-

fully applied to a wide range of applications (Koller
Credal networks are graph-based statistical and Friedman, 2009).
models whose parameters take values in a An arguably more principled approach to coping with
set, instead of being sharply specified as in imprecision in parameters is by means of closed and
traditional statistical models (e.g., Bayesian convex sets of probability mass functions called credal
networks). The computational complexity of sets (Levi, 1980). Unlike the representation of knowl-
inferences on such models depends on the ir- edge by precise probability mass functions, credal
relevance/independence concept adopted. In sets allow for the distinction between randomness and
this paper, we study inferential complexity ignorance (Walley, 1991). Bayesian networks whose
under the concepts of epistemic irrelevance parameters are specified by conditional credal sets are
and strong independence. We show that in- known as credal networks (Cozman, 2000). Credal
ferences under strong independence are NP- networks have been successfully applied to knowledge-
hard even in trees with ternary variables. based expert systems, where it has been argued that
We prove that under epistemic irrelevance allowing parameters to be imprecisely specified facili-
the polynomial time complexity of inferences tates elicitation from experts (Antonucci et al., 2007,
in credal trees is not likely to extend to 2009; Piatti et al., 2010).
more general models (e.g. singly connected
networks). These results clearly distinguish A Bayesian network provides a concise representation
networks that admit efficient inferences and of the (single) multivariate probability mass function
those where inferences are most likely hard, that is consistent with the network parameters and
and settle several open questions regarding factorizes over the graph. Analogously, a credal net-
computational complexity. work provides a concise representation of the credal
set of multivariate mass functions that are consistent
with the local credal sets and satisfy (at least) the
irrelevances encoded in the graph. The precise charac-
1 INTRODUCTION terization of this joint credal set depends however on
the concept of irrelevance adopted.
Bayesian networks are multivariate statistical models
where irrelevance assessments between sets of variables The two most commonly used irrelevance concepts in
are concisely described by means of an acyclic directed the literature are strong independence and epistemic
graph whose nodes are identified with variables (Pearl, irrelevance. Two variables X and Y are strongly in-
1988). The graph encodes a set of Markov conditions: dependent if the joint credal set of X, Y can be re-
non-descendant non-parent variables are irrelevant to garded as originating from a number of precise proba-
a variable given its parents. The complete specification bility mass functions in each of which the two variables
of a Bayesian network requires the specification of a are stochastically independent. Strong independence
conditional probability distribution for every variable is thus closely related to the sensitivity analysis inter-
and every configuration of its parents. This is no easy pretation of credal sets, which regards an imprecisely
task. Whether these parameters (i.e., the conditional specified model as arising out of partial ignorance of
distributions) are estimated from data or elicited from an ideal precisely specified one (Kwisthout and van der
experts, they inevitably contain imprecisions and ar- Gaag, 2008; Antonucci and Piatti, 2009; Zaffalon and
bitrariness (Kwisthout and van der Gaag, 2008). De- Miranda, 2009). A variable X is epistemically irrel-
evant to a variable Y if the marginal credal set of Y are unbounded). Table 1 summarizes both the previ-
according to our model is the same whether we observe ously known complexity results and the contributions
the value of X or not (Walley, 1991). of this work, which appear in boldface.
Typically, credal networks are used to derive tight
bounds on the expectation of some variable conditional 2 CREDAL NETWORKS
on the value of some other variables. The complexity
of such an inferential task varies greatly according to Let X1 , . . . , Xn be variables taking values x1 , . . . , xn
the topology of the underlying graph, the cardinality in finite sets X1 , . . . , Xn , respectively. We write X :=
of the variable domains, and the irrelevance concept (X1 , . . . , Xn ) to denote the n-dimensional variable tak-
adopted. For instance, the 2U algorithm of Fagiuoli ing values x in X := X1 Xn . Given X and
and Zaffalon (1998) can solve the problem in polyno- I {1, . . . , n} := N , the notation XI denotes the vec-
mial time if the underlying graph is singly connected, tor obtained from X by discarding the coordinates not
variables are binary and strong independence is as- in I. We also write XI to denote the Cartesian prod-
sumed. When instead epistemic irrelevance is adopted, uct (in the proper order) of the sets Xi with i I, and
no analogous polynomial-time algorithm for the task xI to denote an element of XI . Note that XN = X
is known. On the other hand, de Cooman et al. (2010) and XN = X .
developed a polynomial-time algorithm for inferences Let Z be some finite set (multidimensional or not),
in credal trees under epistemic irrelevance that work and Z a variable taking values in Z. A probability
for arbitrarily large variable domains. No such algo- mass function (pmf ) p(Z) isPa non-negative real-valued
rithm is known under strong independence. Recently, function on Z such that zZ p(z) = 1. Given a
Maua et al. (2012) showed the existence of a fully pmf p on Z, we definePthe expectation operator as the
polynomial-time approximation scheme for credal net- functional Ep [f ] := zZ f (z)p(z) that maps every
works of bounded treewidth and bounded variable car- real-valued function f on Z to a real number. Given
dinality under strong independence. It is still unknown a pmf p(X) on X , and disjoint subsets I, J and K,
whether an analogous result can be obtained under we say that variables XK are stochastically irrelevant
epistemic irrelevance. to XI given XJ if p(xI |xJK ) = p(xI |xJ ) for all x,
In this paper, we show that epistemic irrelevance and where the conditional pmfs are obtained by application
strong independence induce the same upper and lower of Bayes rule on p(X). Variables XI and XK are
predictive probability in HMM-like (hidden Markov independent conditional on XJ if, given XJ , XI and
model-like) credal networks (i.e., when we query the XK are irrelevant to each other.
value of the last state node given some evidence), A credal set C(Z) is a closed and convex set of pmfs
and that they induce the same marginal expectation p(Z) on Z (Levi, 1980). The extrema of a credal set
bounds in any network where non-root nodes are pre- are the points that cannot be written as convex com-
cise and root nodes are vacuous. The former im- binations of other points in the set. The extrema of
plies that we can use the algorithm of de Cooman C(Z) are denoted by ext C(Z). We assume that every
et al. (2010) for epistemic credal trees to compute credal set has finitely many extrema, which are used
tight bounds on the predictive probability in HMM- to represent it. Thus, the credal sets we consider are
like credal networks also under strong independence. geometrically polytopes. The vacuous credal set is the
The latter implies that computing tight posterior prob- largest credal set on a given domain Z, and is denoted
ability bounds in singly connected credal networks over by V (Z). Given a set M (Z) of pmfs p(Z) on Z we
ternary variables under epistemic irrelevance is NP- write co M (Z) to denote its convex closure, that is,
hard, as we show this to be the case under strong in- the credal set obtained by all convex combinations of
dependence (de Campos and Cozman (2005) have pre- elements of M (Z). Given a credal set C(X), disjoint
viously shown the NP-hardness of inference in singly subsets I and J of N , and an assignment xJ to XJ ,
connected strong networks when variable cardinalities we define the conditional credal set C(XI |xJ ) (induced
by C(X)) as the set co{ p(XI |xJ ) : p C(X), p(xJ ) >
0 }. It can be shown that the C(XI |xJ ) remains the
Table 1: Inferential Complexity of Credal Networks same if we replace C(X) with its extrema ext C(X).
MODEL STRONG EPISTEMIC For disjoint subsets I, J and K of N , we say that a
variable XK is strongly irrelevant to XI given XJ (and
(Predictive) HMM P P
w.r.t. C(X)) if XK is stochastically irrelevant to XI
Tree NP-hard P
given XJ in every extreme p ext C(X). This implies
Singly connected NP-hard NP-hard
that C(XI |xJK ) = C(XI |xJ ) for all x. Variables XI
Multiply connected NPPP -hard NPPP -hard
and XK are strongly independent given XJ if condi-
 1 2 for every variable Xi and value xnd(i) of Xnd(i) . The
epistemic extension satisfies the Markov condition
w.r.t. epistemic irrelevance: the non-descendant non-
3
parents are irrelevant to a variable given its par-
ents. Equation 2 implies that (and is equivalent to
Figure 1: A Simple Polytree for Boolean variables Xi )

tional on XJ both XI is strongly irrelevant to XK min q(xi |xpa(i) ) p(xi |xnd(i) ) max q(xi |xpa(i) ) (3)
and XK is strongly irrelevant to XI . Since stochas-
tic irrelevance implies stochastic independence, strong for all x and p C(X) with p(xnd(i) ) > 0, where the
independence is implied by strong irrelevance. optimizations are over q Q(Xi |xpa(i) ). Note that
these inequalities can be turned into linear inequalities
We say that variables XK are epistemically irrelevant by multiplying both sides by p(xnd(i) ).
to XI given XJ if C(XI |xJK ) = C(XI |xJ ) for all
values of x. One can show that strong irrelevance Given a function f of a query variable Xq , and an
implies epistemic irrelevance (and the converse is not assignment xO to evidence variables XO , the primary
necessarily true) (Cozman, 2000; de Cooman and Trof- inference with credal networks is the application of the
faes, 2004). Variables XI and XK are epistemically generalized Bayes rule (GBR), which asks for a value
independent conditional on XJ if, given XJ , XI and of that solves the equation
XK are epistemically irrelevant to each other (Walley, X
1991, Ch. 9). min [f (xq ) ]p(x) = 0 , (4)
pC(X)
xxO
Let G = (N, A) be an acyclic directed graph. We
denote the parents of a node i by pa(i). The set of non-
where the sum is performed over the values x of X
descendants of i, written nd(i), contains the nodes not
whose coordinates indexed by O equal xO . Assuming
reachable from i by a directed path. Note that pa(i)
that minpC(XO ) p(xO ) > 0, it follows that
nd(i). We say that G is singly connected if there is at
most one undirected path connecting any two nodes in
= min Ep [f ] , (5)
the graph; it is a tree is if additionally each node has at pC(Xq |xO )
most one parent. If a graph is not singly connected, we
say it is multiply connected. Singly connected directed that is, is the lower expectation of f on the pos-
graphs are also called polytrees. terior credal set C(Xq |xO ) induced by the (strong or
A (separately specified) credal network N associates epistemic) extension of the network.
to each node i in G a variable Xi and a collection Example 2. Consider the network in Example 1,
Q(Xi |Xpa(i) ) of local credal sets Q(Xi |xpa(i) ) indexed and let q = 3, f = I(x3 = 0) and O is the empty
by the values of Xpa(i) . When every local credal set is set. Then applying the GBR is equivalent to finding
a singleton the model specifies a Bayesian network. the lower marginal probability = minpC(X3 ) p(0)
Example 1. Consider the credal network N over induced by the network extension. Assuming strong
Boolean variables X1 , X2 , X3 whose graph is shown in independence (hence strong
P extension), the inference
Figure 1. Let [p(0), p(1)] represent a pmf of a Boolean is solved by = min x1,2 q1 (x1 )q2 (x2 )q3 (0|x1,2 ) =
variable. The local credal sets are Q(X1 ) = Q(X2 ) = 1 + min{2q1 (0)q2 (0) q1 (0) q2 (0)} = 1 1/2 = 1/2 ,
co{[0.4, 0.6], [0.5, 0.5]} and Q(X3 |x1,2 ) = {[I(x1 = where the minimizations are performed over q1 (X1 )
x2 ), I(x1 6= x2 )]}, where I() is the indicator function. ext Q(X1 ), q2 (X2 ) ext Q(X2 ), and q3 (X3 |x1,2 ) =
[I(x1 = x2 ), I(x1 6= x2 )]. The epistemic extension is
The strong extension is the credal set C(X) whose the credal set of joint pmfs p(X) on X such that
extrema p(X) satisfy for all x the condition
Y min q(x1 ) p(x1 |x2 ) max q(x1 ) ,
p(x) = q(xi |xpa(i) ), (1) qQ(X1 ) qQ(X1 )
iN
min q(x2 ) p(x2 |x1 ) max q(x2 ) ,
qQ(X1 ) qQ(X1 )
where q(Xi |xpa(i) ) ext Q(Xi |xpa(i) ). The strong ex-
tension satisfies the Markov condition w.r.t. strong in- q(x3 |x1,2 ) p(x3 |x{1,2} ) q(x3 |x1,3 ) .
dependence: every variable is strongly independent of
its non-descendant non-parents given its parents. The The inference under epistemic irrelevance is the
epistemic extension is the joint credal set C(X) such value of the solution of the linear program =
that min{p(0, 0, 0) + p(1, 1, 0) : p(X) C(X)} = 5/11 <
C(Xi |xnd(i) ) = Q(Xi |xpa(i) ) (2) 1/2, where C(X) is the epistemic extension.
3 COMPLEXITY RESULTS 1 2

In this section we present new results about the com-

4 3
putational complexity of GBR inferences in credal net-
works. We make use of previously unknown equiv-
alences between strong and epistemic extensions to Figure 2: HMM Over Four Variables
derive both positive and negative complexity results.
The section is divided in subsections addressing net- 1 3 n-1 n+1

works in increasing order of topological complexity.

2 4 n
3.1 HIDDEN MARKOV MODELS

An imprecise hidden Markov model (HMM) is a credal Figure 3: HMM Over 2n + 1 Variables
network whose nodes can be partitioned into state and
manifest nodes such that the state nodes form a chain where h (x4 ) = I(x4 = 0) , q(X1 ), q(X2 |X1 ) and
(i.e., a sequence of nodes with one node linking to the q(X4 |X1 ) are defined as before, and q(X3 |x1,2,4 )
next and to no other in the sequence), and each man- Q(X3 |x2 ) for every x1,2,4 . Solving the equation above
ifest node is a leaf with a single state node as par- for we get that = 13/28.
ent. As the following example shows, there are GBR
inferences in HMMs which depend on the irrelevance GBR inferences in HMMs are polynomial-time
concept adopted, even in the case of binary variables. computable under epistemic irrelevance, but no
Example 3. Consider an HMM over four Boolean polynomial-time algorithm is known under strong in-
variables X1 , X2 , X3 , X4 , where X1 and X2 are state dependence except for the case of binary variables (in
variables and X3 and X4 are manifest variables. which case the aforementioned 2U algorithm can be
The topology of the corresponding credal network used). The following result shows that there exists a
is depicted in Figure 2. The local credal sets class of GBR inferences in HMMs which are insensitive
are given by Q(X1 ) = Q(X2 |0) = Q(X4 |0) = to the irrelevance concept adopted. This implies that
{[3/4, 1/4]}, Q(X2 |1) = Q(X4 |1) = {[1/4, 3/4]}, and the GBR is polynomial-time computable in such cases
Q(X3 |0) = co{[1/4, 3/4], [1/2, 1/2]} and Q(X3 |1) = also under strong independence.
co{[3/4, 1/4], [1/2, 1/2]}. Consider the query f (X4 ) = Theorem 1. Consider an HMM over n + 1 variables
I(x4 = 0), and evidence x3 = 0. Under strong indepen- whose state nodes are identified with odd numbers and
dence, the GBR is to solve for the equation manifest nodes are identified with even numbers (see
X X Figure 3). Assume that the query node is q = n + 1,
min q(0|x2 )g (x2 ) = min q(0|x2 )g (x2 ) = 0 ,
x2 x2
and that evidence xO is set on a subset O of the man-
ifest nodes. Then the posterior lower expectation of
where the minimizations are performed over any function f on Xq conditional on xO is the same
q(X3 |x2 ) Q(X3 |x2 ), x2 = 0, 1, and whether we assume epistemic irrelevance or strong in-
X dependence.
g (x2 ) = q(x1 )q(x2 |x1 )q(x4 |x1 )[I(x4 = 0) ] ,
x1,4
Proof. Let g (xn+1 ) be equal to f (xn+1 ) . To com-
with q(X1 ) = q(X2 |0) = q(X4 |0) = [3/4, 1/4] and pute the GBR under strong independence, we need to
q(X2 |1) = q(X4 |1) = [1/4, 3/4]. The values of q(0|x2 ) find a number such that
depend only on the signs of g (x2 ), x2 = 0, 1. Solving
n+1
for for each of the four possibilities, and taking the X Y
minimum value of , we find that = min{p(0|0) : min q(x1 ) q(xi |xpa(i) )g (xn+1 ) = 0 ,
xxO i=2
p(X4 |X3 ) C(X4 |X3 )} = 4/7.
Under epistemic irrelevance, the GBR is equal to where the minimization is performed over q(x1 )
X Q(X1 ) and q(Xi |xpa(i) ) Q(Xi |xpa(i) ) for i =
min q(x1 )q(x2 |x1 )q(x4 |x1 )q(0|x1,2,4 )h (x4 ) = 2, . . . , n + 1. The minimization above is equivalent to
x1,2,4 the constrained program
X
(1 ) q(x1 )q(x2 |x1 )q(0|x1 ) min q(0|x1,2 , x4 = 0) X n+1
Y
x1,2 minimize q(x1 ) q(xi |x1:i1 )g (xn+1 ) (6)
X
xxO i=2
q(x1 )q(x2 |x1 )q(1|x1 ) max q(0|x1,2 , x4 = 1) = 0 ,
x1,2 subject to q(xi |x1:i1 ) = q(xi |xpa(i) ), i > 2 , (7)
with variables q(x1 ) Q(X1 ), q(Xi |xpa(i) ) Corollary 1. Consider again the HMM of Theorem 1
Q(Xi |xpa(i) ), i = 2, . . . , n + 1 and q(Xi |x1:i1 ) and the same query setting, and assume that the local
Q(Xi |xpa(i) ), i = 3, . . . , n + 1. The objective function credal sets associated to manifest nodes i are singletons
in (6) can be rewritten as Q(Xi |xi1 ) = {q(xi |xi1 )} such that q(xi |xi1 ) = 1
n
whenever xi = xi1 . Then the posterior lower expec-
X
q(x1 )
Y
q(xi |x1:i1 )
X
q(xn+1 |x1:n )g (xn+1 ). tations of any function f given xO is the same whether
x1:n xO i=2 xn+1
we assume epistemic irrelevance or strong indepen-
dence.
We show the result by induction. Consider a state
node i + 1 and assume that the constrained program The proof of Corollary 1 follows directly from Theorem
(6)(7) is equivalent to 1. Observe that since q(xi |xi1 ) for a manifest node i
are 0-1 probabilities, the HMM reduces to a(n impre-
i+1
X Y cise) Markov Chain. Thus, strong and epistemic ex-
min q(x1 ) q(xj |x1:j1 )hi+1 (xi+1 ) , tensions coincide also in Markov Chains when evidence
s.t.(7)
x1:i+1 xO j=2
is before (w.r.t. the topological order) the query node.
for some function hi+1 on Xi+1 , and let In the case of evidence after the query, de Cooman
q (Xi+1 |X1:i ) := {q (Xi+1 |x1:i ) : x1:i } be the et al. (2010) have shown by a counterexample that
solutions to the linear optimizations inferences in Markov Chains are sensitive to the irrel-
X evance concept adopted.
hi (xi1 ) := min q(xi+1 |x1:i )hi+1 (xi+1 )
qQ(Xi+1 |xi1 )
xi+1 3.2 CREDAL TREES
for different values of X1:i . Then q (Xi+1 |X1:i ) satis- Imprecise HMMs are particular cases of credal trees.
fies (7) and minimizes (6) w.r.t. q(Xi+1 |X1:i ), thus De Cooman et al. (2010) showed that GBR inferences
i+1
can be computed in polynomial time in credal trees un-
min
X
q(x1 )
Y
q(xj |x1:j1 )hi+1 (xi+1 ) = der epistemic irrelevance. In this section we show that
s.t.(7)
x1:i+1 xO j=2
the same type of inference under strong independence
is an NP-hard task.
X i
Y
min q(x1 ) q(xj |x1:j1 )hi (xi1 ) . In the intermediate steps of reductions used to show
s.t.(7)
x1:i xO j=2 hardness results we make use of networks whose nu-
merical parameters are specified by (polynomial-time)
Similarly, consider a manifest node i and assume that
computable numbers, which might not be encodable
(6)(7) is equivalent to
trivially as rationals. A number r is computable if
X i
Y there exists a machine Mr that, for input b, runs in
min q(x1 ) q(xj |x1:j1 )hi (xi1 ) . (8) at most time poly(b) (the notation poly(b) denotes an
s.t.(7)
x1:i xO j=2 arbitrary polynomial function of b) and outputs a ra-
tional number t such that |r t| < 2b . Of special
Let q (Xi |X1:i1 ) be the solutions to the linear opti- relevance are numbers of the form 2t1 /(1 + 2t2 ), with
mizations |t1 |, |t2 | being rationals no greater than two, for which
X we can build a machine that outputs a rational t with
hi1 (xi1 ) := min q(xi |x1:i1 )hi (xi1 )
qQ(Xi |xi1 ) the necessary precision in time poly(b) as follows: com-
xi xO
pute the Taylor expansions of 2t1 and 2t2 around zero
for different values of X1:i1 . Then, q (Xi |X1:i1 ) sat- with sufficiently many terms (depending on the value
isfies (7) and minimizes (6), therefore (8) equals of b), and then compute the fractional expression. The
following lemma ensures that any network specified
i1
X Y with computable numbers can be approximated arbi-
min q(x1 ) q(xj |x1:j1 )hi1 (xi1 ) .
s.t.(7) trarily well by a network specified with rational num-
x1:i1 xO j=2
bers.
The basis for i = n follows trivially by setting Lemma 1. Consider a credal network N over n vari-
hn+1 (xn+1 ) = g (xn+1 ). Thus, the unconstrained ables whose numerical parameters q(xi |xpa(i) ) are spec-
minimization in (6) (without the constraints (7)) ified with computable numbers encoded by their respec-
achieves the same value of the constrained program. tive machines, and let b be the size of the encoding of
Moreover, it can be shown that the unconstrained pro- the network. Given any rational number 2poly(b) ,
gram is the epistemic extension of the network (Be- we can construct in time poly(b) a credal network
navoli et al., 2011), so that the result follows. N 0 over the same variables whose numerical param-
eters are all rational numbers, and such that there is 0
a polynomial-time computable bijection (p, p0 ) that as-
sociates any extreme p of the strong extension N with ... n
1 2 3
an extreme p0 of the strong extension of N 0 satisfy-
ing maxxI XI |p0 (xI ) p(xI )| for every subset of
variables XI . n+1 n+2 n+3 ... 2n

Proof. Take N 0 to be equal to N except that each com- Figure 4: Credal Tree Used To Prove Theorem 2
putable number r, given by its machine Mr , used in
the specification of N is replaced by a rational t such
that |t r| < 2(n+1)(v+1) , where v := maxiN |Xi | Before proving our hardness results, we state and dis-
is the maximum number of values any variable can as- cuss some facts about the partition problem, which
sume. Because 2poly(b) , we can use Mr with will be used later. The partition problem is stated
input poly(b) + (n + 1)(v + 1) to obtain t in time as follows: given positive integers z1 , . . . , zn , decide
O(poly(poly(b) + (n + 1)(v + 1))) = O(poly(b)). Ex- whether P is S N := {1, . . . , n} such that
there
actly one of the probability values in each pmf in N 0 is
P
z
iS i = / zi , where the notation i
iS / S de-
computed as one minus the sum of the other numbers notes that i N \ S. This is a well-known NP-
to ensure that the total mass of the pmf is exactly one; hard problem (Garey and Johnson,P 1979). We define
its error is at most (v 1) 2(n+1)(v+1) < 2n(v+1) . vi := zi /z, i = 1, . . . , n, where z := i zi /2, and work
w.l.o.g. with the
P partition problem using vi instead of
Let q(xi |xpa(i) ) and q 0 (xi |xpa(i) ) denote the parame-
zi . Let vS :=
P iS vi . Then, it follows for any S that
ters of N and N 0 , respectively, and consider an as-
vS = 2 iS / vi . Also, if an instance of the par-
signment x to all variables X in N (or in N 0 ). By
tition problem is a yes-instance, there is S for which
design |q 0 (xi |xpa(i) ) q(xi |xpa(i) )| 2n(v+1) . It fol-
vS = 1, whereas if it is a no-instance, then for any S,
lows from the binomial expansion of the factorization
|vS 1| 1/(2z). Consider the function
of p0 (x) that (there is a term for p(x) in the expansion
and 2n 1 terms that an be written as a product of
2n(v+1) by numbers less than or equal to one) 2(vS 1) + 2vS 1
h(vS ) = . (9)
Y 2
p0 (x) = q 0 (xi |xpa(i) )
iN Seen as a function of a continuous variable vS
Y
n(v+1) [0, 2], the function above is strictly convex, symmet-
[2 + q(xi |xpa(i) )]
ric around one, and achieves the minimum value of
iN
X Y one at vS = 1. Thus, if the partition problem is a
= q(xi |xpa(i) )[2nvn ]n|S| yes-instance, then minS h(vS ) = 1, while if it is a no-
SN iS instance, then minS h(vS ) 21/(2z)1 + 21/(2z)1
Y 4
2n 2nvn + q(xi |xpa(i) ) 2(2z) > 1 + (2z)4 /2 = 1 + 1/(32z 4 ), where the sec-
iN ond inequality is due to Lemma 24 in (Maua et al.,
= p(x) + 2nv . 2012), and the strict inequality follows from the first-
4
order Taylor expansion of 2(2z) .
Similarly, we can show that
The following result shows that inferences under strong
n
Y independence are hard even in credal trees.
p0 (x) [q(xi |xpa(i) ) 2n(v+1) ] p(x) 2nv .
i=1 Theorem 2. Computing the GBR in credal trees un-
der strong extension is NP-hard, even if all numerical
Thus, maxx |p0 (x) p(x)| 2nv . Now consider a parameters are rational numbers, and all variables are
subset of P the variables XI and a value xI XI . Since at most ternary.
p0 (xI ) = xxI p0 (x), each term p0 (x) in the sum sat-
isfies p0 (x) p(x) + 2nv , and there are less than
v n 2vn terms being summed, we have that
Proof. We use a reduction from the partition prob-
X lem as previously described. We build a credal tree
p0 (xI ) [p(x) + 2vn ] p(xI ) + .
xxI
over variables X0 , . . . , X2n with graph as in Figure 4.
The root node is associated to the ternary variable
An analogous argument can be used to show that X0 , with X0 := {1, 2, 3} and uniform pmf q(x0 ) =
p0 (xI ) p(xI ) . Thus, maxxI |p0 (xI ) p(xI )| 1/3. The remaining variables are all Boolean. For
. i = 1, . . . , n, specify the local sets Q(Xi |x0 ) as single-
tons {q(Xi |x0 )} such that which is greater than 2n /(64z 4 ). The gap in the value
of is at least
vi vi
2 /(1 + 2 ),
if x0 = 1, 1 1
v 1 1 g( 32z 4 ) g( 64z 4 )
q(xi = 1|x0 ) = 1/(1 + 2 i ), if x0 = 2, = 1 1
g(1/(64z 4 )) g(1/(32z 4 )) g( 64z 4 )g( 32z 4 )
1/2, if x0 = 3.

1 1
g( 32z 4 ) g( 64z 4 )

Finally, for i = 1 + n, . . . , 2n specify the local credal > 1 2

g( 32z 4)
sets such that q(Xi |xin ) Q(Xi |xin ) satisfies
2n /(64z 4 )
q(xi = 1|xin ) [, 1] for all xin , where  = > 1
(1 + (1 + 32z n )2
2n3 /(64z 4 ). Consider the computation of the GBR 4 )2

with observed nodes O = {1 + n, . . . , 2n}, observation 2n

> .
xO = (1, . . . , 1) XO , query node q = 0, and query 4 64z 4
f (x0 ) = I(x0 = 3). Using the results from (Antonucci n

and Zaffalon, 2006) about the conservative inference

2
So we apply Lemma 1 with = 21 464z 4 and use the

rule, one can show that f is minimized at an extrema same rational numbers q(xi = 1|xo = 2) as in the spec-
p(X) such that p(xi = 1|xin = 1) 6= p(xi = 1|xin = 0), ification of the new network instead of the irrational
that is, if p(xi = 1|xin = 1) is chosen to be equal to , values 1/(1 + 2vi ) to approximate g(), which guar-
then p(xi = 1|xin = 0) = 1, and vice-versa. Hence, antees that the gap will continue to exist. Alterna-
tively, we can use the same argument as in Theorem 3
= min Ep [f ] = max p(x0 = 3|xO ) of (de Campos, 2011) to constructively find a suitable
p p
 n encoding for the numerical parameters and g().
1+ 1/3
= max
p 2 p(xO ) 3.3 POLYTREES AND BEYOND
1
= max , In general networks, it is still unclear which type of
S g(aS )
inferences depend on the irrelevance concept used.
 n Q
where g(a) = 1 + (1 + a) 1+ 2 n vi There is however one situation where we can show
i=1 1/(1 + 2 ) is
they coincide, and this is particularly important for
defined forQany real number Qa, and aS v := bS + bN \S 1
the hardness results that we prove later on.
and bS := iS (2vi + ) iS / (1 + 2 i
) are defined
for all S N . Note that g(aS ) > 1 + (1 + aS )2n . It Lemma 2. Consider a credal network of arbitrary
follows from the Binomial Theorem that topology, where all nodes are associated to precise pmfs
apart from the root nodes, which are associated to vac-
2vS bS (2vS + 2n )(1 + )n uous credal sets. Then the result of the GBR for an ar-
bitrary function f of a variable Xq associated to a non-
(2vS + 2n )(1 + 2n)
root node q and no evidence is the same whether we
2vS + 2n+2  assume epistemic irrelevance or strong independence.
where we use the inequality (1 + r/k)k 1 + 2r valid
Proof. Let XR be the variables associated root nodes
for r [0, 1] and positive integer k (Maua et al., 2011,
(hence to vacuous local credal sets), and XI denote
Lemma 37). Thus,
the remaining variables (which are associated to sin-
h(vS ) 1 aS h(vS ) + 2n+3  1 . gleton local credal sets). The result of the GBR under
epistemic irrelevance is given by
Now if the partition problem is a yes-instance, then X
aS 1/(64z 4 ), while if it is a no-instance, we have = min Ep [f ] = min p(xI |xR )p(xR )f (xq )
p(X) p(X)
x
that aS > 1/(32z 4 ). Hence, there is a gap of at X X
least 1/(64z 4 ) in the value of aS between yes- and no- = min p(xR ) p(xI |xR )f (xq )
p(X)
instances, and we can decide the partition problem by xR xI

verifying whether 1/g(), where := 3/(128z 4 ).

X
= min p(xR )g(xR ) ,
This proof shall be completed with the guarantee that p(XR )
xR
we can approximate in polynomial time the irrational P Q
numbers used to specify the credal tree and g(a) well where g(xR ) := xI iI q(xi |xpa(i) )f (xq ), and
enough so that 1/g() falls in the gap between the q(xi |xpa(i) ) are the single pmfs in the local credal sets
values of for yes- and no-instances. First, note that of the non-root variables XI . According to the last
equality, is a convex combination of g(xR ). Hence,
     n Yn
1 1 1 2 1 XY
g g = , min g(xR ) = min q(xi |xpa(i) )f (xq ) .
32z 4 64z 4 64z 4 1+ i=1
1 + 2vi xR xR
xI iI
The rightmost minimization is exactly the value of the 1 2 3 n
GBR under strong independence, and since the strong
extension is contained in the epistemic extension, the n+1 n+2 n+3 n+4 2n+1
inequality above is tight.

The above result will be used in combination with Figure 5: Polytree Used To Prove Theorem 3
hardness results for strong credal networks to demon-
strate that computing the GBR under epistemic irrele-
Table 2: Local Pmfs Used To Prove Theorem 3
vance is also hard. First, we focus on singly connected
networks. q(xi |xi1 , xin1 ) xi = 1 xi = 2 xi = 3

For polytrees, the next theorem shows that inferences xi1 = 1, xin1 = 1 2vi 0 1 2vi
in credal networks, either under epistemic irrelevance xi1 = 2, xin1 = 1 0 1 0
or strong independence, are NP-hard, even if variables xi1 = 3, xin1 = 1 0 0 1
are at most ternary. If we allowed variables to have xi1 = 1, xin1 = 0 1 0 0
any arbitrary finite number of states, then this result xi1 = 2, xin1 = 0 0 2vi 1 2vi
would follow from the proof of NP-hardness of infer- xi1 = 3, xin1 = 0 0 0 1
ences in polytrees given by de Campos and Cozman
(2005), because the polytree presented there is similar
to the one in Figure 5 with no evidence and query in contains the pmf q(Xi |xi1 , xin1 ) as speci-
the last (topological) node. By Lemma 2, we could use fied in Table 2. Consider a joint pmf p(X)
an inference in the epistemic polytree to solve the same which is an extreme of the strong extension
inference, demonstrating that such inference is NP- of the network. One can show that p(x) =
Q2n+1 Q
hard too. In this paper we devise a stronger result, as q(xQn+1 ) i=n+2 q(x i |x i1 , xin1 ) iS I(xi =
the hardness is shown even when variables are at most 1) iS/ I(x i = 0) for all x, where S N . Thus,
Q2n+1
ternary. For this purpose, we perform a polynomial- p(x) is equal to 31 i=n+2 q(xi |xi1 , xin1 ) if
time reduction from the partition problem. A much xS = 1 and xN \S = 0, and otherwise van-
P
similar reduction has been used to show that select- ishes. It follows that p(x2n+1 = 1) = x p(x) =
ing optimal strategies in limited memory influence di- (2vS )/3 and p(x2n+1 = 2) = (2vS 2 )/3. Let
agrams is NP-hard (Maua et al., 2012). While the re- := (1 + z 4 /64)/3. By computing the GBR with
duction used here closely resembles the reduction used query f (x2n+1 ) = I(x2n+1 = 1) + I(x2n+1 = 2) and
in that work, due to a technicality we cannot directly no evidence, we can decide the partition problem,
use that result (mainly because the influence diagram as minp E[f ] = minS h(vS )/3 , if and only if the
could have multiple utility nodes, which would much partition problem is a yes-instance. According to
complicate the reduction). Instead, we directly reduce Lemma 2, this result does not change if we assume
an instance of the partition problem to a computa- epistemic irrelevance. It remains to show that we
tion of the GBR without evidence in a credal polytree can polynomially encode the numbers 2vi . This
whose non-root nodes are all associated to precise pmfs is done by applying Lemma 1 with a small enough
and root nodes are associated to vacuous credal sets. computable in time polynomial in the size of the
By using this reduction in conjunction with Lemma partition problem: = 1/(3 64z 4 ) suffices.
2, we prove the hardness of GBR computations also
under epistemic irrelevance. The hardness of inference in multiply connected credal
Theorem 3. Given a credal polytree, computing the networks under epistemic irrelevance comes from the
GBR with a function f of the query variable Xq and no fact that general credal networks under strong inde-
evidence is NP-hard, whether we assume epistemic ir- pendence can always be efficiently mapped to credal
relevance or strong independence, even if all variables networks under strong independence with all non-root
are (at most) ternary and all numbers are rational. nodes precise and vacuous root nodes. Because such
inferences in general credal networks under strong in-
Proof. We build a credal polytree with underlying dependence are NPPP -hard (Cozman et al., 2004), and
graph as in Figure 5. The variables (associated because Lemma 2 demonstrates that marginal infer-
to nodes) on the upper row are Boolean and vac- ences without evidence in these networks are equiv-
uous, namely X1 , . . . , Xn , while the remaining alent to the same inference in credal networks under
variables Xn+1 , . . . , X2n+1 are ternary and as- epistemic extension, the hardness result is obtained
sociated to singleton local credal sets such that also for epistemic networks. For completeness, we
Q(Xn+1 ) contains a uniform pmf q(xn+1 ) = 1/3, write the complete proof of such result using a re-
and, for i = n + 2, . . . , 2n + 1, Q(Xi |xi1 , xin1 ) duction from the E-MAJSAT problem, since previous
work has only provided a sketch of such proof (Cozman Xt , XR = (X1 , . . . , Xn ) are the root variables as-
et al., 2004). The proof differs only slightly from the sociated to Boolean variables in . The value of
proof of NPPP -hardness of MAP inference in Bayesian p(xt = 1|xI )p(xI |xR ) is equal to one if and only if
networks given by Park and Darwiche (2004). the assignment xR satisfies (by construction) and is
Theorem 4. Computing the GBR in credal networks zero otherwise. Thus, p(Xt = 1) = #SAT/2nk , where
under either epistemic irrelevance or strong indepen- #SAT is the number of assignments to the variables
dence is NPPP -hard even if all variables are binary. Zk+1 , . . . , Zn that satisfy with the first k Boolean
variables set to x1:k . By maximizing over the possible
choices of degenerate pmfs q(Xi ), i = 1, . . . , k, we have
Proof. The hardness result follows from a reduction
that minp p(xt = 0) < 1/2 if and only if there is an as-
from E-MAJSAT. Given a propositional formula
signment to the first k variables Z1 , . . . , Zk such that
over Boolean variables Z1 , . . . , Zn and an integer 1
more than half of the assignments to the remaining
k < n, the E-MAJSAT is the problem of deciding
n k variables Zk+1 , . . . , Zn satisfy the formula .
whether there exists an assignment to Z1 , . . . , Zk such
that the majority of the assignments to the remaining
variables Zk+1 , . . . , Zn satisfy . The reduction pro- 4 CONCLUSION
ceed as follows. Create a credal network over Boolean
variables X = (X1 , . . . , Xk , Xk+1 , . . . , Xn ) such that In this paper, we have showed new computational com-
X1 , . . . , Xk are associated to root nodes and vacuous plexity results for inference in credal networks under
credal sets, and Xk+1 , . . . , Xn are associated to non- both epistemic irrelevance and strong independence.
root nodes and have uniform pmfs. The root variables There are three main contributions. First, by exploit-
act as selectors for the Boolean variables in the propo- ing the relations between these two irrelevance con-
sitional formula. Each variable Xi is associated to the cepts in HMM-like credal networks, we have shown
Boolean variable Zi of the original formula . Build that predictive inferences under strong independence
one new binary variable Xi (using a suitable sequence can be computed in polynomial time using the same al-
of numbers i = n + 1, n + 2, . . . ) for each operator in gorithm developed for epistemic credal trees. To com-
the Boolean formula of the E-MAJSAT problem such plement such result, we have proved that inferences
that Xi has as parents its operands, that is, for logical with strong independence in general trees are NP-hard
operations (Xa Xb ) and (Xa Xb ), with a, b < i, even if all variables are (at most) ternary, which shows
Xi has as parents the two operands Xa and Xb and that it is unlikely that more general polynomial-time
is associated to a singleton credal set containing the algorithms for inferences under strong independence
pmf q(xi |xa , xb ) = I(xi = xa xb ), where denotes the will ever exist. Moreover, using the relation between
respective binary operation; for the operation (xa ), strong and epistemic irrelevance concepts in networks
with a < i, Xi has a single parent Xa and is associ- where the imprecision appears only in root nodes (de-
ated to a singleton local credal set containing the pmf fined by vacuous credal sets), we were able to prove
q(xi |xa ) = I(xi 6= xa ). There is more than one way to that inferences in polytrees under epistemic irrelevance
build such a network, depending on the order one eval- are NP-hard, even if all variables are (at most) ternary.
uates the operations in the Boolean formula, and any This result closes the gap between known polynomial-
valid evaluation order can be used. The final network time algorithms (which were known for trees and some
encodes a circuit for evaluating the formula . polytrees) and potentially any more complicated net-
work. To the best of our knowledge, these complexity
Let t be the last node in the network in topologi- results were all open, specially for the case of epistemic
cal order; variable Xt represents the satisfiability of irrelevance.
the
Q whole formula. Consider a joint pmf p(x) =
iN q(xi |xpa(i) ), for some choice of pmfs from the
extrema of local credal sets. Acknowledgements
Qk By design, there is a sin-
gle x1:k X1:k such that i=1 q(xi ) evaluates to one.
Let x1:k be such (joint) value. We have that This work has been partly supported by the Swiss NSF
grants no. 200020 137680/1 and 200020 134759/1, and
n
X Y the Hasler Foundation grant no. 10030.
p(Xt = 1) = p(xt = 1|xI )p(xI |xR ) q(xi )
xxt i=1
1 X References
= p(xt = 1|xI )p(xI |xR ) ,
2nk Antonucci, A., Bruhlmann, R., Piatti, A., and Zaf-
xx1:k
falon, M. (2009). Credal networks for military iden-
where XI are the non-root variables that represent tification problems. International Journal of Ap-
the logical operations in the formula apart from proximate Reasoning, 50(4):666679.
Antonucci, A. and Piatti, A. (2009). Modeling unre- Garey, M. R. and Johnson, D. S. (1979). Computers
liable observations in Bayesian networks by credal and Intractability: A Guide to the Theory of NP-
networks. In Proceedings of the 3rd International Completeness. W.H. Freeman.
Conference on Scalable Uncertainty Management Koller, D. and Friedman, N. (2009). Probabilistic
(SUM), volume 5785 of Lecture Notes in Computer Graphical Models. MIT press.
Science, pages 2839. Springer.
Kwisthout, J. and van der Gaag, L. C. (2008). The
Antonucci, A., Piatti, A., and Zaffalon, M. (2007). computational complexity of sensitivity analysis
Credal networks for operational risk measurement and parameter tuning. In Proceedings of the 24th
and management. In International Conference on Conference in Uncertainty in Artificial Intelligence
Knowledge-Based and Intelligent Information & En- (UAI), pages 349356. AUAI Press.
gineering Systems (KES), volume LNCS 4693, pages
604611. Levi, I. (1980). The Enterprise of Knowledge. MIT
Press, London.
Antonucci, A. and Zaffalon, M. (2006). Equivalence
between Bayesian and credal nets on an updating Maua, D. D., de Campos, C. P., and Zaffalon, M.
problem. In Proceedings of 3rd International Con- (2011). Solving limited memory influence diagrams.
ference on Soft Methods in Probability and Statistics CoRR, abs/1109.1754.
(SMPS), pages 223230. Springer. Maua, D. D., de Campos, C. P., and Zaffalon, M.
Benavoli, A., Zaffalon, M., and Miranda, E. (2011). (2012). Solving limited memory influence diagrams.
Robust filtering through coherent lower previ- Journal of Artificial Intelligence Research, 44:97
sions. IEEE Transactions on Automatic Control, 140.
56(7):15671581. Maua, D. D., de Campos, C. P., and Zaffalon, M.
Cozman, F. G. (2000). Credal networks. Artificial (2012). Updating credal networks is approximable
Intelligence, 120(2):199233. in polynomial time. International Journal of Ap-
proximate Reasoning, 53(8):11831199.
Cozman, F. G., De Campos, C. P., Ide, J. S., and
da Rocha, J. C. F. (2004). Propositional and rela- Park, J. D. and Darwiche, A. (2004). Complexity re-
tional Bayesian networks associated with imprecise sults and approximation strategies for MAP expla-
and qualitative probabilistic assessments. In Pro- nations. Journal of Artificial Intelligence Research,
ceedings of the 20th Conference on Uncertainty in 21:101133.
Artificial Intelligence (UAI), pages 104111. AUAI Pearl, J. (1988). Probabilistic Reasoning in Intelligent
Press. Systems: Networks of Plausible Inference. Morgan
de Campos, C. P. (2011). New complexity results for Kaufmann, San Mateo, California.
MAP in Bayesian networks. In International Joint Piatti, A., Antonucci, A., and Zaffalon, M. (2010).
Conference on Artificial Intelligence (IJCAI), pages Building Knowledge-Based Systems by Credal Net-
21002106. AAAI Press. works: A Tutorial. Nova Science.
de Campos, C. P. and Cozman, F. G. (2005). The Walley, P. (1991). Statistical Reasoning with Imprecise
inferential complexity of Bayesian and credal net- Probabilities. Chapman and Hall, London.
works. In Proceedings of the 19th International Joint
Conference on Artificial Intelligence (IJCAI), pages Zaffalon, M. and Miranda, E. (2009). Conserva-
13131318, San Francisco, CA, USA. Morgan Kauf- tive inference rule for uncertain reasoning under in-
mann. completeness. Journal of Artificial Intelligence Re-
search, 34:757821.
de Cooman, G., Hermans, F., Antonucci, A., and
Zaffalon, M. (2010). Epistemic irrelevance in
credal nets: the case of imprecise Markov trees.
International Journal of Approximate Reasoning,
51(9):10291052.
de Cooman, G. and Troffaes, M. (2004). Coherent
lower previsions in systems modelling : products and
aggregation rules. Reliability engineering & system
safety, 85(1-3):113134.
Fagiuoli, E. and Zaffalon, M. (1998). 2U: An exact
interval propagation algorithm for polytrees with
binary variables. Artificial Intelligence, 106(1):77
107.