Information Theoretic Measures for Ant Colony Optimization

Abstract

We survey existing measures to analyze the search behavior of Ant Colony Optimization (ACO) algorithms and introduce a new uncertainty measure for characterizing three ACO variants. Unlike previous measures, the group uncertainty allows for quantifying the exploration of the search space with respect to the group assignment. We use the group uncertainty to analyze the search behavior of Group-Based Ant Colony Optimization.

Appendix: Theoretical Relations

True Entropy Bounded by Rating Entropy Given random variable X ∈ { 1, …, N} and assuming that without loss of generality nodes $N - k + 1,\ldots,N$ have already been visited and let r _j be the rating for choosing X = j, then it can be shown that the true entropy H(X(s _ν )) is bounded from above by the rating entropy H(X).

$$\displaystyle\begin{array}{rcl} H(X) - H(X(s_{\nu }))& =& -\sum _{j=1}^{N}\left ( \frac{r_{j}} {\sum _{l=1}^{N}r_{l}}\log \frac{r_{j}} {\sum _{l=1}^{N}r_{l}}\right )\!+\!\sum _{j=1}^{N-k}\left ( \frac{r_{j}} {\sum _{l=1}^{N-k}r_{l}}\log \frac{r_{j}} {\sum _{l=1}^{N-k}r_{l}}\right ) {}\\ & \geq & -\sum _{j=1}^{N-k}\left ( \frac{r_{j}} {\sum _{l=1}^{N}r_{l}}\log \frac{r_{j}} {\sum _{l=1}^{N}r_{l}} - \frac{r_{j}} {\sum _{l=1}^{N-k}r_{l}}\log \frac{r_{j}} {\sum _{l=1}^{N-k}r_{l}}\right ) \geq 0 {}\\ \end{array}$$

since $\frac{r_{j}} {\sum _{l=1}^{N}r_{l}} \leq \frac{r_{j}} {\sum _{l=1}^{N-k}r_{l}}$ and $x\log x$ is monotonically increasing.

Rating Entropy Smaller Than Pheromone Entropy The pheromone entropy H _τ (X) will generally be larger than the rating entropy H(X) since the heuristic incorporated in H(X) will increase the differences in probability by favoring some events. The heuristic values permitted in this case can be calculated in advance and are constant for the whole algorithm execution.

Uncertainty Relations

Assume that true uncertainty $\mathbb{U}_{w}(X_{i})$ in the sliding window of length w is calculated based on the true probabilities p _j ^(. 0)of each iteration t instead of the empirical estimations. Then the average probabilities $\bar{p}_{j}$ in the sliding window w are known. It holds that $\mathbb{U}_{1}(X) = H(X^{(1)}(s_{\nu })) = H_{1}(X(s_{\nu }))$ and that the true uncertainty can be bounded from above by the sum of the average entropy in the same sliding window and the average Kullback–Leibler divergence between the corresponding random variables.

$$\displaystyle\begin{array}{rcl} \mathbb{U}_{w}(X)& =& -\sum _{j=1}^{N}\bar{p}_{ j}\log \bar{p}_{j} = -\sum _{j=1}^{N}\left ( \frac{1} {w}\sum _{t=1}^{w}p_{ j}^{(.0)}\right )\log \left ( \frac{1} {w}\sum _{t=1}^{w}p_{ j}^{(.0)}\right ). {}\\ \end{array}$$

Applying Jensen’s inequality leads to the following:

$$\displaystyle\begin{array}{rcl} \mathbb{U}_{w}(X)& \leq & -\sum _{j=1}^{N}\left ( \frac{1} {w}\sum _{t=1}^{w}p_{ j}^{(.0)}\right )\left ( \frac{1} {w}\sum _{t=1}^{w}\log p_{ j}^{(.0)}\right ) = - \frac{1} {w^{2}}\sum _{j=1}^{N}\sum _{ k=1}^{w}\sum _{ l=1}^{w}p_{ j}^{(k)}\log p_{ j}^{(l)} {}\\ & =& \frac{1} {w^{2}}\sum _{k=1}^{w}\sum _{ l=1}^{w}\mathop{\underbrace{ H\left (X^{(k)};X^{(l)}\right )}}\limits _{\mathrm{ cross-entropy}} = \frac{1} {w^{2}}\sum _{k=1}^{w}\sum _{ l=1}^{w}H\left (X^{(k)}\right ) +\mathop{\underbrace{ D_{\mathrm{ KL}}\left (X^{(k)}\vert \vert X^{(l)}\right )}}\limits _{\mathrm{ Kullback-Leiblerdivergence}} {}\\ & =& H_{w}\left (X(s_{\nu })\right ) + \frac{1} {w^{2}}\sum _{k=1}^{w}\sum _{ l=1}^{w}D_{\mathrm{ KL}}\left (X^{(k)}\vert \vert X^{(l)}\right )\,. {}\\ \end{array}$$