Routing in A Cyclic Mobyspace
Routing in A Cyclic Mobyspace
Contact probability
50 0.8 0.16
0.1
0.6 0.14
Contact probability
Contact probability
0.4 0.12 0.08
A B 0.1
p=0.7 0.2
0.06
0 0.08
0 10 20 30 40
0.06 0.04
Time slot
0.04
(a) (b) 0.02
0.02
0 0
0 200 400 600 800 1000 1200 1400 0 200 400 600 800 1000 1200 1400
Time slots (one minute) Time slots (one minute)
Figure 1: Nodes A and B have a contact with p = 0.7
(a) Shifts 01/AM & 03/AM (b) Shifts 32/PM & 21/EVE
during time slot 0 every 50 time slots.
A&B
0.6
C 37/B 0/0
0.4 .5
23
1
0.2
7/
/1
0
0 10 20 30 40 50 B (3
7 ,0
.5 37/C
1
) 30/B 30/1 0/B
0.8
(0,0.7) 33/1
B&C
(30,0.7) C
0.6
0.4 0/0.7 0/0.7
B 0.2
4) 27/1
0 0. 30/A 30/1 0/A
A 0 10 20 30 40 50
1 0, 10/C
1
A (
20
/1
0.8
/1
10
.4
A&C
0.6
0.4
10/A
0/0
0.2
0
0 10 20 30 40 50
(a) G (b) G′
(a) (b)
Figure 4: The time-space graph G and the state-
Figure 3: A physical cyclic MobiSpace (a). The dis- space graph G′ of the cyclic MobiSpace in Fig-
cretized probabilistic contacts between each pair of ure 3(a).
nodes of a common motion cycle T (b).
0
d1,2 /p 1,2
0.
a11
23
1.
1 ax
1
7/
m
5
s1 a12
/1
1 s2
/p
a21 d2,1 /p 2,1
2 0
1
d
/p
d 1,2 1,2 30/B 30/1 0/B Goal 51.7 0.3 66.4
2,g
d 1,
d 2 /p max
d2,g/p 1
2 0.7 0.7
d1,
s s2 0/0.7 0/0.7
g
/p 1
g
/p 1
1 g
0.3
d3
30/A 30/1 0/A 63.1 59.9 0
,
Goal
2 ,g
/1
0
/p
1.
ma
10
3
x
20
/1
g
0
0.
1.
/1
10
.4
6
73.1
s3 20/A 10/A
0/0
49.9
0.4
(a) (b) (c) (d)
Figure 5: (a) A Markov decision process (MDP) model (b) Implicit action (c) The MDP of G′ in Figure 4(b)
with C as destination (d) The values and optimal actions after applying value iteration in Figure 5(c).
2.5 Remarks der the control of the decision maker. MDP is a generalized
In G, a message can be in any node at any time. However Dijkstra’s algorithm for probabilistic graphs.
in G′ , a message can only be at a state in its correspond- We reformulate a variation of MDP as a 5-tuple (S, A, T,
ing time slot. This results in an important difference in D, SG ) as explained below. At any given time, the system
G′ : links are time-independent and they are always avail- can be at only one state s in the set of all states S. Each
able when messages are in the corresponding states. For state s has a set of actions As ⊆ A (the set of all actions).
example, whenever a message is in state 30/A, it always has Only one action a is allowed to take effect at a time. The
a chance of being transited to 30/B through the discretized effect of applying action a is that the system transits from
probabilistic contact available at time slot 30. In the rest of state s to another state. The transition probability function
this paper, we use the EMD of state to refer to the EMDs Ta (s, s′ ), specifies the probability of transiting from state s
of the message in this state. to state s′ when applying action a. Note that for any s and a,
P ′ ′ 2
Our probabilistic graph models differ from the models de- s′ ∈S Ta (s, s ) = 1. The delay function D(s, s ) specifies
fined in previous papers [2, 3] in that a contact in the pre- the delay of transiting from state s to state s′ . A value
vious models is a deterministic connection during a specific function V (s), gives the expected minimum total delay for
period of time, while a discretized probabilistic contact is transiting from s to any goal state g ∈ SG (the set of all goal
conceptual and is drawn from the the cyclic contact history states). V (g) = 0. An MDP is illustrated in Figure 5(a),
or from the prior knowledge of the cyclic contact pattern where s1 , s2 , and g ∈ SG are different states. s1 has two
between a pair of nodes. The purpose of the discretization actions a11 and a21 . If a11 takes effect, the system will transit
is the creation of our graph model G′ with time-independent from s1 to state s2 with delay d1,2 and probability p11,2 .
links where EMDs can be calculated. Note that the transla- The MDP problem is to find the set of expected minimum
tion from G to G′ is lossless since G can be reproduced from total delay values V (s) for each state s to reach any g ∈ SG
G′ by combining the states of the every node. and the corresponding optimal action chosen at each state
The number of states in G′ depends on the number of in order to achieve the minimum values. Value iteration
nodes and the discretized probabilistic contacts of each node [12] solves MDPs by iteratively updating the value functions
which is bounded by the number of time slots in T . Methods (Equation 1) of every state until they converge. In each
to reduce the number of states includes increasing the size of round of the iteration, based on the resulting values in the
time slots and dropping the discretized probabilistic contacts last iteration, the value V (s) of each state s ∈ / SG is updated
whose probabilities are below a certain threshold. by choosing an action a ∈ As such that V (s) is minimized.
For an action a, the value of s is the sum of the delays
D(s, s′ ) + Vt (s′ ) (for each possible next state s′ ) are weighed
3. EXPECTED MINIMUM DELAYS by Ta (s, s′ ) (refer to Figure 5(a)).
IN A STATE-SPACE GRAPH
X
This section reviews and reformulates a variation of the Vt+1 (s) = min {Ta (s, s′ ) × [D(s, s′ ) + Vt (s′ )]}. (1)
Markov decision process (MDP). Using MDP, the values as- a∈A(s)
s′ ∈S
sociated with the states in G′ are updated iteratively and
finally converge to their EMDs. An efficient approach is also The value functions are considered to converge sufficiently
proposed to calculate EMDs by extending a recent work in when the maximum difference between the values of all states
solving MDPs. in two consecutive iterations is less than some threshold
value. When the values of the states are properly initialized
3.1 Markov Decision Process (MDP) (such as iteratively applying Equation 2, which is essentially
State-space searching is a very common problem in AI the Bellman-Ford algorithm), the value iteration is guaran-
planning (or decision-making), which is similar to routing. teed to converge with the minimum values in the states [13].
The Markov decision process (MDP) [10, 11] provides a
mathematical framework for modeling decision-making in 2
In the traditional MDP models, cost function C is used.
situations where outcomes are partly random and partly un- We change it to delay function D in the networking context.
Goal Goal
Table 1: Comparison of value iteration (VI) and our
C4 C3 extended topological value iteration (E-TVI).
C2
90/1 (a) Resulting value.
0/A 10/A 30/A 120/*
states 0/A 0/B 10/A 30/A 30/B 37/B
C1
/1
VI 59.87 66.41 49.88 63.14 51.70 44.70
83
0/B 30/B 37/B
E-TVI 59.92 66.46 49.91 63.19 51.73 44.73
(b) Computation time.
nodes 100 200 300 400 500
Figure 6: An efficient approach for the MDP in Fig-
VI 0.515 1.256 3.515 7.194 10.25
ure 5(c).
E-TVI 0.137 0.360 0.668 1.225 2.426
V0 (s) = min { min [D(s, s′ ) + V0 (s′ )]} (2) shown in Figure 5(c). Applying value iteration on this MDP,
a∈A(s) s′ :Ta (s,s′ )>0 we get the values (shown inside the states) and the optimal
actions (labeled on the links) of each state in Figure 5(d).
3.2 Deriving EMDs using MDP Note that in Figure 5(d), some of the links are removed such
Our probabilistic time-space graph model G′ differs from as the one from states 0/A to 0/B. This shows the situations
the MDP model in that the states in G′ are not associated where a message is not forwarded from a node with a lower
with actions which tell the actual state transition proba- EMD to a node with a higher EMD.
bilities. However, implicit actions can be determined by
the node’s forwarding preferences. A forwarding preference 3.3 An efficient approach for EMDs
shows the current node’s decision on which node to forward In MDPs, if a state s is a successor state of s′ (there is a
the message to (or to keep the message in the current node) transition from s′ to s), then the value V (s′ ) is dependent
when connected with multiple nodes. Inspired by a research on V (s). This causal relation is transitive. In [13], topology
work on robotics [14], we derive the optimal implicit action value iteration (TVI) is used to reduce the number of rounds
without enumerating all implicit actions. of updates performed to each state in the value iteration. In
Remember that each link in G′ represents a possible state TVI, Kosaraju’s [15] algorithm is used to find the set of
transition and it has a delay d, and a maximal transition strongly-connected components {Ci } and their topological
probability pmax (see Section 2.4). In Figure 5(b), state s order. If component C1 is a successor of C2 (i.e., for any
may transit to three other states for which the corresponding u ∈ C1 and v ∈ C2 , u is a successor of v), then the values of
delays and the maximal probabilities are given. If the pref- the states in C2 are dependent on those in C1 (but not vice
erential order of state transitions is {s1 , s2 , s3 } (which means versa). That is, components C1 and C2 can be calculated
s will transit to s1 whenever possible, and it will transit to separately in a reverse topological order. TVI significantly
s3 only when the transition to neither s1 nor s2 is possi- reduces the number of rounds in value iteration by breaking
ble), the implicit action (which indicates the probabilities the computation of the whole graph into the computation
p1 , p2 , and p3 of transiting from s to s1 , s2 , and s3 respec- of small components.
tively) is calculated as p1 = pmax 1 , p2 = (1 − p1 ) × pmax
2 , TVI cannot be directly applied to our graph model G′
and p3 = (1 − p1 − p2 ) × pmax 3 . Note that the sum of all since in G′ the non-goal states are likely to be connected
transition probabilities of an action calculated in this way as a strongly-connected component. As in Figure 5(c), (1)
is always 1 since for each state s there is at least one bidi- states of the same node are strongly-connected by direc-
rectional link and thus at least one pmax is 1. Also, some tional links, and (2) states of different nodes are connected
transition probabilities might be zero. For example, p3 = 0 by bidirectional links. However, if we group the states of the
if pmax
2 = 1. same time slots into components (not necessarily strongly-
All implicit actions for a state can be obtained from the connected components), we find that these components form
permutation of the preferential order of state transitions a partial order if, given a time slot tb , each link starting at a
from this state. Fortunately, for an optimal forwarding pro- time slot before tb and ending at time slot in or after tb is re-
tocol this order is always an increasing order of the expected moved. For example, if we change the layout of Figure 5(c),
costs of the transitions. For instance, if s can only tran- and remove the corresponding links by setting tb = 0, we
sit to states s1 and s2 , and their costs D(s, s1 ) + V (s1 ) < get the components C4 , . . . , C1 in a partial order as shown
D(s, s2 ) + V (s2 ), then state s1 is preferred over state s2 . in Figure 6. In this figure, the set of removed links are shown
This is because the value of state s is the weight sum p1 × by dashed lines which we denote as Lb .
[D(s, s1 ) + V (s1 )] + p2 × [D(s, s2 ) + V (s2 )], and to minimize Our extended TVI algorithm has two steps. In step one,
this weight sum given p1 + p2 = 1, p1 has to be maximized. the states are updated by TVI after modifying G′ with the
To calculate the EMD of a message in time slot t from following steps: (a) we let tb = 0 and remove Lb from the
node A to node C in the G′ (Figure 4(b)), we modified G′ graph, (b) we add a state s∗ with its time slot being 32 T ,
as follows. We replace all states of C with goal states Goal (c) for each state s which has an outgoing link removed
whose EMDs are 0, and we remove all outgoing links from (in set Lb ), we add a link from s to s∗ . Figure 6 shows
Goal. If state t/A does not exist in G′ , then we add state an example where the state added is 120/∗ and the links
t/A to G′ and add a link from t/A to the consecutive state added are shown by thick lines. The TVI is then applied to
of A. For example, if t = 20, the resulting MDP from G′ is update the component in their reversed topological order,
0.6 30
i.e., C1 , . . . , C4 . In step two of our extended TVI, we recover Spray&focus* Spray&focus*
Spray&focus Spray&focus
the original graph by removing the added state s∗ and the 0.5 MaxProp*
MaxProp 25
MaxProp*
MaxProp
added links, and adding back the links in Lt . Then, we
Delay (hours)
0.4
Delivery rate
continuously update the components in the same topological 0.3 20
Delay (hours)
Delay (hours)
20
Delivery rate
Delivery rate
0.6 0.6 18
18
16
0.4 0.4 16
14
Spray&wait 14
0.2 0.2 Spray&focus* Spray&focus* Spray&focus*
MaxProp* 12 MaxProp* 12 MaxProp*
RCM RCM RCM
0 0 10 10
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Attendance rate Clustering factor Attendance rate Clustering factor
Delay (hours)
Delay (hours)
19
Delivery rate
Delivery rate
0.6 0.6 18
18
17
17
0.4 0.4 16
16
Spray&wait Spray&wait 15
0.2 Spray&focus* 0.2 Spray&focus* 15 Spray&focus* Spray&focus*
MaxProp* MaxProp* 14 MaxProp* 14 MaxProp*
RCM RCM RCM RCM
0 0 13 13
100 150 200 250 300 350 400 450 500 1 1.5 2 2.5 3 3.5 4 4.5 5 100 150 200 250 300 350 400 450 500 1 1.5 2 2.5 3 3.5 4 4.5 5
# of students # of tickets # of students # of tickets
Figure 8: Comparison of delivery rate using the Figure 9: Comparison of delay using the NUS trace.
NUS trace.
where tij , tik are the historical encounter times of i with j other iff they are in the same classroom at the same time.
and k respectively. (2) Sessions start on the hour and end on the hour, which
Note that all of the protocols that we implement aim to means that hour is the unit of time for the contact duration.
compare different metrics for delivery probability, and all (3) Only the contacts that take place during class hours are
other optimizations that have orthogonal effects on the per- used. Non-class hours are removed to compress time. For
formance of the protocols are not implemented. The or- example, suppose the last class session on Monday ends at
thogonality means that these optimizations can be added to 9pm, and the first session on Tuesday starts at 8am. If Mon-
all of our implemented algorithms and they are expected to day 8pm to 9pm corresponds to the 10th hour, then Tuesday
provide an equal level of improvement in their routing per- 8am to 9am is the 11th hour. The advantages of the trace
formance. Such optimizations may include buffer manage- synthesized in this model are that they exhibit the same set
ment [8], global estimation of message delivery probability of characteristics to those observed in the real world and
[19] and social centrality of the nodes [6], geometric informa- it provides contact patterns of a large population (several
tion [20], etc. Similarly, we assume that an ideal mechanism orders of magnitude larger than any reported real traces)
serves to clear out buffers in the network of delivered data. over a long period. The schedules of the 4,885 classes and
First, we conduct simulation studies to compare the per- enrollment of 22,341 students for each of the class for each
formance of Spray&focus, MaxProp, and their extended ver- week of 77 class hours are publicly available on [22].
sions using the UMassDieselNet trace. The simulation en- We select a number of students N (100 ≤ N ≤ 500) in
vironment and settings will be the same as specified in the each of experiment due to the memory constraint in the sim-
Section 4.3. As shown in Figure 7, Spray&f ocus∗ is much ulation environment. Contacts related to the non-selected
better than Spray&f ocus in both delivery ratio and delay, students are ignored. We generate non-determinant traces
while M axP rop∗ slightly outperforms M axP rop. Note that by taking absentees into consideration. Each student at-
we use an ideal version of MaxProp where we assume each tends a class with a attendance probability Pattend . Our
node knows the cost function f of the other nodes, which data processing include the following steps. (1) We break
gives an upper performance bound. In practice, MaxProp each class session into several one hour class sessions and
has a much bigger amortized overhead in communication reassigned unique IDs to them. 159 conflicting enrollments
and computation than M axP rop∗ since the latter has a con- are resolved by removing those enrollment with a lower class
stant cost function f . session ID. (2) The selection of N students is tricky. If se-
lected randomly, the network becomes too sparse for mes-
4.2 NUS student contact trace sages to be delivered. On the other hand, when students
Accurate information of human contact patterns is avail- are selected by maximizing their similarities (the number
able in scenarios such as university campuses. As shown of common classes they are enrolled in), the network be-
by the National University of Singapore (NUS) student con- comes over connected. To prevent the above extremes and
tact trace model [21], when the class schedules and student maintain the small-world property in the size-reduced stu-
enrollment for each class on a campus are known, accurate dent networks, we use the following process. We select the
information about contact patterns between students over first student randomly. To select the kth student, we di-
large time scales can be obtained without a long-term con- vide the k − 1th selected students into two groups S1 and
th
tact data collection. Their contact model is simplified in S2 , and
P select the k student
P s as the one with the highest
several ways. (1) Two students are in contact with each score s1 ∈S1 sim(s, s1 ) − s2 ∈S2 sim(s, s1 ) among the stu-
20 0.3 1 1
Spray&wait Spray&wait
18 Epidemic Epidemic
0.25 0.8 Spray&focus* 0.8 Spray&focus*
16
Number of contacts
MaxProp* MaxProp*
14 0.2 RCM RCM
Delivery rate
Delivery rate
Percentage
0.6 0.6
12
0.15
10
0.4 0.4
8 0.1
6 0.2 0.2
0.05
4
2 0 0 0
6 8 10 12 14 16 18 20 22 24 0 5 10 15 20 25 30 0 5 10 15 20 0 5 10 15 20
Hour Contact duration (seconds) Hour Hour
(a) Contact distribution (b) Duration distribution (a) Delivery rate (1 ticket). (b) Delivery rate (3 tickets).
28 28
Epidemic Epidemic
Figure 10: Statistics in the UMassDieselNet traces. 26 Spray&focus*
MaxProp*
26 Spray&focus*
MaxProp*
24 24
RCM RCM
Delay (hours)
Delay (hours)
22 22
20 20
18 18
dents that are not yet selected, where the similarity func- 16 16
14 14
tion sim is defined as the number of common class ses- 12 12
sions enrolled by two students. We define a clustering factor 10 10
0 5 10 15 20 0 5 10 15 20
C = |S1 |/(|S1 |+|S2 |) which determines the degree of connec- Hour Hour
tivity in the network. (3) If two students enroll in the same (c) Delay (1 ticket). (d) Delay (3 tickets).
class session, they have a discretized probabilistic contact in
2 30 30
the hour of the class session with probability Pabsent . With Epidemic
Spray&focus*
Epidemic
Spray&focus*
25 25
all of the discretized probabilistic contacts, the probabilis- MaxProp*
RCM
MaxProp*
RCM
tic state-space graph can be built. (4) Finally, we generate 20 20
Hop-count
Hop-count
2
traces by generating contacts with probability Pattend for 15 15
each pair of students and each class session they enroll in. 10 10
0.8 1.8
Delay (hours)
Delivery ratio
0.6 1.6
0.4 1.4
Spray&wait
0.2 Spray&focus* 1.2 Spray&focus*
MaxProp* MaxProp*
RCM RCM
0 1
0 50 100 150 200 0 50 100 150 200
Minutes Minutes
0.8 40
Delay (hours)
Delivery ratio
0.6 30
0.4 20
Spray&wait
Figure 12: Our synthetic bus traces are generated 0.2 Spray&focus*
MaxProp*
10 Spray&focus*
MaxProp*
from metro maps: Miami (left) and Madrid (right). 0
RCM
0
RCM
0 50 100 150 200 250 300 350 0 50 100 150 200 250 300 350
Minutes Minutes