Zero-determinant strategies

Because a discounting factor δ is one, the payoffs of players are the average payoffs with respect to the stationary distribution of the Markov chain. Let P^(s)(σ) denote the stationary distribution, which may depend on the initial condition when the Markov chain is not irreducible. It satisfies (3) Taking summation of both sides of Eq (3) with respect to σ_−n ≡ σ∖σ_n with an arbitrary n, we obtain (4) where we have defined (5) Regarding as representing the strategy “Repeat”, where player n repeats the previous action with probability one, one can readily see that Eq (4) is an extension of Akin’s lemma [15, 18, 27, 28], relating a player’s strategy with the stationary distribution, to the multi-player multi-action public-monitoring case. Letting (6) Eq (4) means that the average of with respect to the stationary distribution is zero for any n and σ_n. We remark that all players are assumed to know the functional form of W(τ|σ′), and that , and thus T_n(σ_n|σ′) as well, are solely under control of player n. Because of the normalization condition , the relation (7) holds.

Let , which we call the strategy vector of player n associated with action σ_n. (Another name for is the Press-Dyson vector [27].) A strategy of player n is represented as an M × M_n matrix composed of the strategy vectors for her actions σ_n ∈ {1, …, M_n}. For a matrix A, let span A be the subspace spanned by the column vectors of A. Let 0_m and 1_m denote the m-dimensional zero vector and the m-dimensional vector of all ones, respectively. From Eq (7), one has (8) for any player n, implying that the dimension of is at most (M_n − 1).

Let ρ ≡ (P^(s)(σ))_σ∈Σ be the vector representation of the stationary distribution P^(s)(σ). When player n chooses a strategy , for any vector , one has due to Eq (4). In other words, the expectation of v with respect to the stationary distribution ρ vanishes.

Let and . The following definition is an extension of the notion of the ZD strategy [6, 27] to multi-player multi-action public-monitoring games.

Definition 1. A zero-determinant (ZD) strategy is defined as a strategy for which dim V_n ≥ 1 holds.

To see that this is indeed an extended definition of the ZD strategy, note that any vector is represented as , where is the coefficient vector. Let be the vector with element e_n equal to the expected payoff e_n ≡ 〈s_n(σ)〉_s of player n in the steady state. When player n employs a ZD strategy, it amounts to enforcing linear relations on e with α satisfying .

Consistency

A question naturally arises: When more than one of the players employ ZD strategies, are they “consistent”, that is, do linear payoff relations enforced by the players always have solutions? For example, in a two-player game, when player 1 enforces by a ZD strategy and player 2 enforces by a ZD strategy, do the simultaneous equations of (e₁, e₂) have a solution? Let N′ be the set of players who employ ZD strategies. The set consists of all combinations of the expected payoffs that satisfy the enforced linear relations by the players in N′. If E is empty, then it implies that the set of ZD strategies is inconsistent in the sense that there is no valid solution of the linear relations enforced by the players.

Definition 2. ZD strategies are said to be consistent when E is not empty.

In the multi-player setting, one may regard N′ as a variant of a ZD strategy alliance [15], where the players in N′ agree to coordinate on the linear relations to be enforced on the expected payoffs. The above question then amounts to asking whether it is possible for a player to serve as a counteracting agent who participates in the ZD strategy alliance with a hidden intention to invalidate it by adopting a ZD strategy that is inconsistent with others.

The following proposition is the first main result of this paper.

Proposition 1. Any set of ZD strategies is consistent.

Proof. We first note that the following property holds for strategy vectors, whose proof is given in Methods.

Lemma 1. Let . Then .

For any set span(V_n)_n∈N′ of ZD strategies, let K be the dimension of span(V_n)_n∈N′, and let be a basis of span(V_n)_n∈N′. The expected payoff vector should be given by a non-zero solution of the linear equation in , where we define A, b, and as (9) One has (10) where .

The Rouché-Capelli theorem [29] tells us that is a necessary and sufficient condition for the linear equation in to have a solution, that is, for span(V_n)_n∈N′ to be consistent (because A is augmented matrix). An equivalent expression of this condition is that there is no vector such that and hold (which ensures that there is no elementary operations which make the rank of augmented matrix larger than that of the original matrix). Assume to the contrary that there exist such that and hold. One would then have (11) On the other hand, is a linear combination of , so that Lemma 1 states that it should be zero if it is proportional to 1_M, leading to contradiction.

Proposition 1 states that it is impossible for any player to serve as a counteracting agent to invalidate ZD strategy alliances. This statement is quite general in that it applies to any instance of repeated games covered by our formulation.

In Ref. [19], it was shown that every player can have at most one master player, who can play an equalizer strategy on the given player (that is, controlling the expected payoff of the given player), in multi-player multi-action games. Indeed, our general result on the absence of inconsistent ZD strategies (Proposition 1) immediately implies that more than one ZD players cannot simultaneously control the expected payoff of a player to different values. Therefore, our result generalizes their result on equalizer strategy to arbitrary ZD strategies.

Since the dimension of is at most (M_n − 1), depending on , it should be possible for player n with M_n ≥ 3 to adopt a ZD strategy for which dim V_n ≥ 2 holds. The dimension of V_n corresponds to the number of independent linear relations to be enforced on the expected payoffs of the players, so that it implies that one player may be able to enforce multiple independent linear relations. On the other hand, our result on the absence of inconsistent ZD strategies implies that for any set N′ of ZD players the dimension of span(V_n)_n∈N′ should be at most N, the number of players, since any set of ZD strategies should contain at most N independent linear relations if it is consistent. This in turn implies that if the dimension of span(V_n)_n∈N′ is equal to N for a subset N′ of players then players not in N′ cannot employ independent ZD strategy any more.

Independence

Another naturally-arising question would be regarding independence for a set of ZD strategies, which we define as follows:

Definition 3. A set of ZD strategies is independent if any set {v_n}_n∈N′ of non-zero vectors v_n in V_n is linearly independent. Otherwise, is said to be dependent.

If a set of ZD strategies is dependent, then there exists a ZD player whose ZD strategy adds no linear constraints other than those already imposed by other ZD players. One of the simplest example of a dependent set of ZD strategies is the case where two players enforce exactly the same linear relation to the expected payoffs. Our second main result is to show that any set of ZD strategies is independent under a general condition.

Proposition 2. Let N′ be a subset of players. Assume that does not have zero elements for any n ∈ N′ and any σ_n ∈ {1, …, M_n}. Then, any set of ZD strategies of players in N′ is independent.

See Methods for the proof.

It should be noted that when has zero elements then one might have dependent ZD strategies. A simple example can be found in a two-player two-action perfect-monitoring (iterated prisoner’s dilemma) game: Let the payoff vectors s₁ and s₂ for players 1 and 2 be and , with T ≠ S. If player 1 adopts the strategy (12) then it enforces the linear payoff relation e₁ = e₂. This strategy is a well-known tit-for-tat strategy [6]. By symmetry, player 2 can also adopt the same strategy , implying that these two strategies are indeed dependent.

Simultaneous multiple linear relations by one player

As mentioned above, when the number M_n of possible actions for player n is more than two, player n may be able to employ a ZD strategy with dim V_n ≥ 2 to simultaneously enforce more than one linear relations. (We note that this is impossible for public goods game [15, 16] because the number of action for each player is two.) Such a possibility has never been reported in the context of ZD strategies. Here, we provide a simple example of such a situation in a two-player three-action symmetric game.

We consider the 3 × 3 symmetric game (13) We remark that s₁, s₂, and 1₉ are linearly independent when r₁ ≠ r₂ and r₁ ≠ −r₂. We choose strategies of player 1 as (14) with 0 ≤ p ≤ 1, 0 ≤ q ≤ 1, 0 ≤ p′ ≤ 1, 0 ≤ q′ ≤ 1, q ≤ p, and p′ ≤ q′. Then we obtain (15) (16) Therefore, player 1 can simultaneously control average payoffs of both players, e₁ and e₂, as e₁ = e₂ = 0. Note that σ with s₁(σ) = 0 is an absorbing state regardless of the strategy of player 2 in this case.

In general, when one player simultaneously enforces two linear relations in two-player multi-action symmetric games, only e₁ = e₂ = C is allowed with some C. This is explained as follows: Assume that player 1 can simultaneously enforce e₁ = C₁ and e₂ = C₂ with C₁ ≠ C₂ by one ZD strategy. Because the game is symmetric, player 2 can also simultaneously enforce e₁ = C₂ and e₂ = C₁ independently by one ZD strategy. This contradicts the consistency of ZD strategies (Proposition 1). Therefore, the only possibility is e₁ = e₂ = C.

The above argument can be extended straightforwardly to the multi-player case. For that purpose, we introduce some notions of symmetric multi-player games. The following definition of a symmetric multi-player game is due to von Neumann and Morgenstern [30, Section 28].

Definition 4. A game is symmetric with respect to a permutation π on {1, …, N} if M_n = M_π(n) holds for any n ∈ {1, …, N} and if π preserves the payoff structure of the game, that is, (17) holds for any σ ∈ Σ and for any n ∈ {1, …, N}, where σ_π ≡ (σ_π(1), …, σ_π(N)).

The following definition is due to Ref. [31].

Definition 5. A game is weakly symmetric if for any pair of players n and there exists some permutation π on {1, …, N} satisfying such that the game is symmetric with respect to π.

Consider an N-player weakly symmetric game. Assume that one player simultaneously enforces N independent linear relations on the average payoffs {e_n}_{n ∈ {1, …, N}} of N players via adopting an N-dimensional ZD strategy. (Note that for this to be possible the number M_n of actions should satisfy M_n ≥ N + 1). Then, the average payoffs {e_n}_{n∈{1,…,N}} should be simultaneously controlled, but they should satisfy e₁ = e₂ = ⋯ = e_n due to the consistency of ZD strategies.

The difficulty of construction of a ZD strategy of one player with dimension N in weakly symmetric N-player games can be seen in the following two propositions, whose proofs are given in Methods.

Proposition 3. In a weakly symmetric N-player game, if the strategy vectors of one player contain no zero element, then a ZD strategy of the player with dimension N is impossible.

Proposition 4. In a weakly symmetric N-player game, if payoffs s_n(σ) of player n are different from each other for all σ, then a ZD strategy with dimension N is impossible.

Proof of Lemma 1

Assume to the contrary that with γ ≠ 0. Taking the inner product of v with the stationary distribution ρ, one has since is represented as a linear combination of the strategy vectors and since the inner product of a strategy vector and the stationary distribution is zero. On the other hand, holds because of the normalization of the stationary distribution. Therefore we obtain γ = 0, leading to contradiction.

Proof of Proposition 2

We first show the following lemma.

Lemma 2. Let N′ be a subset of players. Assume that does not have zero elements for any n ∈ N′ and any σ_n ∈ {1, …, M_n}. For n ∈ N′, let v_n be an arbitrary non-zero vector in . Then {v_n}_n∈N′ are linearly independent.

Proof. We assume to the contrary that {v_n}_n∈N′ are linearly dependent. Then there is a set of coefficients {a_n}_n∈N′ with which ∑_n∈N′a_nv_n = 0_M holds. Without loss of generality we assume a_n ≠ 0 for n ∈ N′.

Since , it is expressed as with a non-zero vector . Let , where ties may be broken arbitrarily, and . With Eq (8), one obtains (18) and thus (19)

We show that the inequality (20) holds for any n, any σ_n ∈ {1, …, M_n}, and any σ′ ∈ Σ satisfying . We first note that for any strategy vector with action σ_n ∈ {1, ⋯, M_n}, one has, from Eq (6), (21) Fix any σ′ ∈ Σ satisfying for a moment. Then, for one has by definition, making the left-hand side of Eq (20) equal to zero. For , on the other hand, one has by definition. Also, since , from Eq (21) one has . These imply that the inequality (20) holds for . Putting the above arguments together, we have shown that the inequality (20) holds for any n, any σ_n ∈ {1, …, M_n}, and any σ′ ∈ Σ satisfying .

Fix any σ′ ∈ Σ satisfying for all n ∈ N′. The above argument has shown that the inequality (20) holds for any n and any σ_n ∈ {1, …, M_n}. On the other hand, at the beginning of the proof we have assumed that (22) holds, implying that the summand is equal to zero for any n ∈ N′ and any σ_n ∈ {1, …, M_n}. By assumption, a_n ≠ 0 and , so that one has , and consequently, , leading to contradiction.

The proof of Proposition 2 is straightforward by taking v_n as belonging to in Lemma 2.

Proof of Proposition 3

We first show the following lemma.

Lemma 3. Consider an N-player game which is symmetric with respect to a permutation π on {1, …, N}. Assume that the column vectors of are linearly independent. For any pair of players n and satisfying , if the strategy vectors of these players contain no zero element, then it is impossible for these players to adopt ZD strategies with which player n enforces linear relation with α ≠ 0_N+1, and where player enforces , where .

Proof. We assume to the contrary that there exists α ≠ 0_N+1 satisfying the properties stated in Lemma 3. By assumption, and . There then exist c_n and satisfying and . One has (23) where the second equality is due to the assumed symmetry of the game with respect to π. Letting , , and , one has (24) implying that holds. Let .

Let and , where ties may be broken arbitrarily, and and . One then has (25) Recalling that we have assumed , let σ′ ∈ Σ be an arbitrary state satisfying and . Then, in view of Eq (21), one has (26) implying that v(σ′) = 0 holds. Since for all σ_n ∈ {1, …, M_n}, they are all equal to zero. Since is assumed non-zero, one has for all σ_n ∈ {1, …, M_n} and consequently . One similarly has . Therefore, from Eq (8) one has . Due to the assumption of linear independence of the columns of , it in turn implies that α = 0_N+1 holds, leading to contradiction.

It should be noted that Lemma 3 holds even if one takes , in which case the Lemma implies that, if the game is symmetric with respect to π, player n with π(n) ≠ n cannot enforce linear relations simultaneously. It should also be noted that Lemma 3 furthermore implies that it is impossible for that player to enforce a linear relation satisfying α_π = α ≠ 0_N+1. In other words, in a symmetric game no player to whom the game is symmetric can enforce a linear relation with the same symmetry as the game itself.

Proposition 3 is a direct consequence of Lemma 3 in weakly symmetric multi-player games.

Proof of Proposition 4

Without loss of generality, we assume that player k takes an N-dimensional ZD strategy determining the average payoffs e_n for n = 1, ⋯, N. Due to the above discussion, only e₁ = ⋯ = e_N = C is allowed. Letting for n ∈ {1, …, N}, one can take as a basis of the N-dimensional ZD strategy. Let c⁽ⁿ⁾ be defined as (27) By the assumption of weak symmetry, for any player n ≠ k, there exists a permutation π satisfying π(n) = k such that the game is symmetric with respect to π. Noting that , from Eq (23) one has (28)

For n ∈ {1, …, N}, define and , where ties may be broken arbitrarily provided that holds, and and . From Eq (7), one has (29) Then, from Eqs (28) and (21), we obtain for an arbitrary σ* satisfying and (30) (31) implying s_n(σ*) = C. On the other hand, we also obtain for an arbitrary σ** satisfying and (32) (33) implying s_n(σ**) = C. Then, because we have assumed that all elements of the payoff vector s_n are different from each other, we have arrived at a contradiction.