Identifying long-term precursors of financial market

crashes using correlation patterns

Hirdesh K. Pharasi1 , Kiran Sharma2 , Rakesh Chatterjee1,3 ,
arXiv:1809.00885v2 [q-fin.CP] 7 Sep 2018

Anirban Chakraborti2 , Francois Leyvraz1,4 , and Thomas H.

Seligman1,4
1
Instituto de Ciencias Fı́sicas, Universidad Nacional Autónoma de México,
Cuernavaca-62210, México
2
School of Computational and Integrative Sciences, Jawaharlal Nehru University,
New Delhi-110067, India
3
School of Mechanical Engineering and Sackler Center for Computational Molecular
and Materials Science, Tel Aviv University, Tel Aviv-6997801, Israel
4
Centro Internacional de Ciencias, Cuernavaca-62210, México
E-mail: hirdeshpharasi@gmail.com; anirban@jnu.ac.in

10 September 2018

Abstract. The study of the critical dynamics in complex systems is always

interesting yet challenging. Here, we choose financial market as an example of a
complex system, and do a comparative analyses of two stock markets – the S&P
500 (USA) and Nikkei 225 (JPN). Our analyses are based on the evolution of cross-
correlation structure patterns of short time-epochs for a 32-year period (1985-2016).
We identify “market states” as clusters of similar correlation structures, which occur
more frequently than by pure chance (randomness). The dynamical transitions between
the correlation structures reflect the evolution of the market states. Power mapping
method from the random matrix theory is used to suppress the noise on correlation
patterns, and an adaptation of the intra-cluster distance method is used to obtain
the “optimum” number of market states. We find that the USA is characterized by
four market states and JPN by five. We further analyze the co-occurrence of paired
market states; the probability of remaining in the same state is much higher than the
transition to a different state. The transitions to other states mainly occur among
the immediately adjacent states, with a few rare intermittent transitions to the remote
states. The state adjacent to the critical state (market crash) may serve as an indicator
or a “precursor” for the critical state and this novel method of identifying the long-term
precursors may be very helpful for constructing the early warning system in financial
markets, as well as in other complex systems.

Keywords: market crash, return cross-correlations, market state, power mapping

method, multidimensional scaling
Identifying long-term precursor 2

1. Introduction

A financial market is a highly complex and continuously evolving system [1–3].

To understand the statistical behavior of the financial market and its constituent
sectors [4–9], researchers focused their attention on the information of co-movements
and correlations among the stocks of the market. It is well known that the mean
correlation among the stocks assumes much higher values during market crashes than in
normal business periods [10]. Similarly, certain correlation structures seem to occur more
frequently than by pure chance (randomness), specially when markets approach a critical
period or crash [11,12]. However, to identify such similar (clusters) correlation patterns,
referred as “market states”, as was previously attempted by Munnix et al. [13, 14], is
rather challenging due to many factors. The first factor is that financial time series is
non-stationary; second factor is that there is always noise present in the correlations
computed over finite length time series data [15], and it is essential to suppress the
corresponding noise in correlation matrices to reveal the actual correlations. To tackle
the first factor of non-stationarity, we work with short time series so that the number
of time steps over which we compute the correlations can be considered as reasonably
stationary. However, with short time series the correlation matrices become highly
singular [16–18]. To tackle the second factor of noise-reduction, various techniques
[19, 20] are available. Here, we shall use a recent and efficient one, namely the power
map method [19, 21, 22], for noise reduction as well as breaking the degeneracy in the
eigenvalues so that the correlation matrices are no longer singular. Furthermore, the
problem of finding similar clusters (groups) of the correlation patterns is a daunting
task by itself. To go beyond the simple quantification of financial market states in
terms of the average correlation, clustering techniques seem promising as does the
study of eigenvalues of the correlation matrix of the corresponding time series [15].
In the research of clustering, the k-means method has had some success for top-to-
down clustering, but it suffers from one major drawback: the number of clusters
and thus the number of states is largely arbitrary (or ad hoc). Earlier, Munnix et
al. [13] had provided a scheme where all the correlation frames at different time-
epochs were initially regarded as a single cluster and then divided into sub-clusters
by a procedure based on the k-means algorithm. They stopped the division process
when the average distance from each cluster center to its members became smaller than
a certain threshold. Based on the top-to-down hierarchical clustering method and the
threshold at 0.1465, which represented the best ratio of the distances between clusters
and their intrinsic radii, Munnix et al. had determined the number of markets states for
USA to be eight. In the present paper, for determining the “optimal” number of clusters,
we use multidimensional scaling (MDS) technique [23] with two/three-dimensional
representations, which are comparatively easier for visualization and studying time-
evolution. So, using multidimensional scaling map, we apply k-means clustering to
divide the clusters of similar correlation patterns into k groups. We propose a new
way, based on the cluster radii, of estimating the number of clusters k, which is fairly
Identifying long-term precursor 3

robust and stable. We thus have a considerable degree of confidence in determining the
“optimal” number of market states identified by the new prescription. For our research,
we have used adjusted closure price data from Yahoo finance [24] for the S&P 500 (USA)
and Nikkei 225 (JPN) stock exchanges, for the 32-year period (1985-2016). The stock
list has been filtered such that we have stocks which were included in the market index
for the entire period of 32 years. Among others, our main finding is that there exist
four market states in USA and five in JPN. We then study the dynamical transitions
between the market states, in a probabilistic manner; we also analyze the co-occurrence
of paired market states and find that the probability of remaining in the same state
is much higher than jumping to another state. The transitions mainly occur among
adjacent states, with a few rare intermittent transitions to the remote states. The state
adjacent to the critical state may indicate a “precursor” to the critical state (market
crash) and this novel method of identifying the long-term precursors may be very helpful
for constructing the early warning system in financial markets, and in other complex
systems.
The paper is organized as follows: We present briefly the methodology and the
data description. Then we present the main part of data analyses along with the above
mentioned findings. Finally, we present summary and concluding remarks.

2. Data Description, Methodology and Results

2.1. Data description

We have used the database of Yahoo finance [24], for the time series of adjusted
closure price for two countries: United States of America (USA) S&P 500 index and
Japan (JPN) Nikkei 225 index, for the period 02-01-1985 to 30-12-2016, and for the
corresponding stocks as follows:
• USA — 02-Jan-1985 to 30-Dec-2016 (T = 8068 days); Number of stocks N = 194;
• JPN — 04-Jan-1985 to 30-Dec-2016 (T = 7998 days); Number of stocks N = 165,
where we have included the stocks which are present in the indices for the entire duration.
The sectoral abbreviations are given in Table 1.
The list of stocks (along with the sectors) for the two markets are given in the
Tables S1 and S2 in Supplementary Information.

2.2. Cross-correlation matrix and power mapping method

We present a study of time evolution of the cross-correlation structures of return
time series for N stocks, and determination of the optimal number of market states
(correlation patterns that exist more frequently then by pure chance or randomness);
also, the dynamical evolution of the market states over different time-epochs. The daily
return time series is constructed as rk (t) = ln Pk (t) − ln Pk (t − 1) , where Pk (t) is
Identifying long-term precursor 4

Table 1. Abbreviations of different sectors for S&P 500 and Nikkei 225 markets

Labels Sectors Labels Sectors

CD Consumer Discretionary ID Industrials
CS Consumer Staples IT Information Technology
CP Capital Goods MT Materials
CN Consumer Goods PR Pharmacuticles
EG Energy TC Technology
FN Financials UT Utilities
HC Health Care

the adjusted closing price of the k-th stock at time t (trading day). Then, the cross-
correlation matrix is constructed using equal-time Pearson cross-correlation coefficients,
Cij (τ ) = (hri rj i − hri ihrj i)/σi σj , where i, j = 1, . . . , N and τ indicates the end date of
the time-epoch of size M days. Here, we computed daily return cross-correlation matrix
C(τ ) computed over the short time-epoch of M = 20 days, for (a) USA with N = 194
stocks of S&P 500 for a return series of T = 8060 days, and (b) JPN with N = 165
stocks of Nikkei 225 for T = 7990 days, during the calendar period 1985-2016. We use
time-epochs of 20 days, such that there is a balance between choosing short time-epochs
for detecting changes and long ones for reducing fluctuations. In figure 1, we show the
time evolution of the return of the market index, r(τ ), along with the mean market
correlation (average of all the elements of the cross-correlation matrix), µ(τ ), and the
Gini coefficient that characterizes the inequality in the distribution of the correlation
coefficients. Evidently, whenever there is a market crash (fall in the r(τ )), the mean
market correlation µ(τ ) rises a lot, and the Gini coefficient falls drastically, indicating
that market is extremely correlated and all the stocks behave similarly (see Ref. [10]).
Since the assumption of stationarity manifestly fails for longer return time series, it is
often useful to break the long time series of length T , into shorter n time-epochs of
size M (such that T /M = n). The assumption of stationarity then improves for the
shorter time-epochs used. However, if there are N return time series such that N > M ,
then this implies that the correlation matrices are highly singular with N − M + 1 zero
eigenvalues, leading to poor statistics. As mentioned in the introduction, we thus use
the power map technique [19, 21, 22] to suppress the noise present in the correlation
structure of short time series. In this method, a non-linear distortion is given to each
cross-correlation coefficient within an epoch by: Cij → (sign Cij )|Cij |1+ , where  is
the noise-suppression parameter. This also gives rise to an “emerging spectrum” of
eigenvalues, arising from the breaking of the degeneracy of the zero eigenvalues (see
Ref. [15] for a recent review).
Identifying long-term precursor 5

(a) (b)

Figure 1. Results of market evolution for (a) USA and (b) JPN,
respectively. The top row shows the returns of the respective market indices. The
middle row shows the mean market correlation (averaged over all the cross-correlation
coefficients) of the respective markets. The bottom row shows the inequality in the
distribution of the cross-correlation coefficients, as characterized by the Gini coefficient.
Evidently, whenever there is a market crash, the mean market correlation becomes very
high and the Gini coefficient becomes very low, indicating that all the stocks behave
very similarly.

2.3. Noise-suppression in a short time cross-correlation frame

First, we study the effect of noise-suppression parameter  on the cross-correlation matrix
and its eigenvalue spectrum within a time-epoch. The cross-correlation structure can be
visualized easily through a two/three dimensional map of coordinates generated through
a multidimensional scaling algorithm. The MDS is a tool of non-linear dimensional
reduction to visualize the similarity of the data set in a D-dimensional space. Each
object is assigned to a coordinate space in D-dimensional space keeping the between-
object distance preserved, as close as possible. The choice of D = 2 or D = 3 is for
optimizing the object location to two/three-dimensional scatter plot or map. As an
input to the MDS algorithm, we provide the distance matrix [25], generated from the
correlation matrix, using the non-linear transformation:
q
dij = 2(1 − Cij ).

The effect of the variation of the parameter  on noise reduction and determining the
optimal number of market states, can thus be better captured through the MDS. The
question is what should be the ideal choice of the noise-suppression parameter ? A very
small value of , say  = 0.01, surely breaks the degeneracy of eigenvalues (giving rise
to an “emerging spectrum” with interesting properties [10]) but does not contribute
much to noise-suppression. On the other hand, a large value, say  = 0.5, suppresses
the noise in the correlation pattern and helps in clustering better way; however, the
emerging spectrum approaches towards the main Marc̆enko-Pastur distribution [26]. In
this paper, we are more interested in noise-suppression in the cross-correlation matrix
Identifying long-term precursor 6

within a single time-epoch rather than properties of the emerging spectrum; hence, we
use  = 0.6 and this choice of a high value is based on the robustness and finding distinct
clusters of stocks using MDS. The effect can be clearly seen through the supplementary
figures S2 and S3. Further, our main aim is to find the optimal number of market
states, based on correlation structures which are similar and appear more frequently.
Hence, we formulate a similarity measure between different cross-correlation matrices
at different time-epochs τ , and then find similar groups of correlation frames across
different time-epochs. We find that with  = 0.6, the noise suppressed cross-correlation
structures can be grouped well into similar clusters, as we will describe below. However,
we find that the number of market states is not very sensitive to the noise-suppression
parameter. A higher value of  lowers the mean of the cross-correlation coefficients, µ
(see supplementary figure S1) and the maximum eigenvalue λmax of the cross-correlation
matrix.
Figure 2 shows the results of the noise-suppression on the short time cross-
correlation matrix using power mapping method [10, 16, 19, 27]. Figure 2(a) shows
a correlation frame computed for the short time-epoch M = 20 days for USA with
N = 194 stocks of S&P 500 ending on 30/11/2001 (arbitrarily chosen date). The
eigenvalue spectrum and MDS map of the correlation frame is shown in figures 2(b)
and (c), respectively. As mentioned earlier, for any short time series M < N , the
highly singular correlation matrices will have N − M + 1 degenerate eigenvalues at zero.
Hence, in our case the eigenvalue spectrum consists of 175 eigenvalues at zero, followed
by 19 distinct positive eigenvalue. The non-linear power mapping method removes the
degeneracy of eigenvalues at zero, leading to an emerging spectrum [10,15]. Figure 2(d)
shows the correlation pattern for  = 0.01. The effect of the small distortion on the
corresponding eigenvalue spectrum and MDS map is shown in figures 2(e) and (f),
respectively. The effect is less visible on MDS map; λmax reduces its value by a small
amount from 44.05 to 43.67. Next, we use a high value of noise-suppression parameter
 = 0.6 to reduce considerably the noise of the correlation frame (shown in figure 2(g)).
The effect of  = 0.6 is clearly visible on the corresponding eigenvalue spectrum and
MDS map, as shown in figures 2(h) and (i), respectively. The shape of the eigenvalue
spectrum changes completely. The emerging spectrum from 175 eigenvalues at zero
is now non-degenerate in nature, and shows a spread around zero with some negative
eigenvalues. Inset of the figures 2(e) and (h) show the emerging spectra in greater details,
while for the inset of figure 2(b) the emerging spectrum is absent. Note that, for  = 0.6,
the value of highest eigenvalue λmax decreases by a large amount to 27.27; the clusters
of stocks in the MDS maps are distinct and denser as compare to low noise-suppression
( = 0.01) or without noise-suppression ( = 0).
Identifying long-term precursor 7

(a) (b) (c)

(d) (e) (f )

(g) (h) (i)

Figure 2. Noise-suppression in a short time cross-correlation frame. (a), (b)

and (c) show the correlation pattern, eigenvalue spectra and MDS map, respectively,
for a correlation matrix of short time-epoch of M = 20 days and N = 194 stocks of
USA, ending on 30/11/2001. The power mapping method [19] is used to: (i) reduce
the noise of the singular correlation matrix (M < N ) formed by the short time series,
or (ii) break the degeneracy of the zero eigenvalues. Two different noise-suppression
parameter values,  = 0.01 and  = 0.6, are used for this purpose. A small value of
 = 0.01 is used for (d), (e) and (f). The change in λmax as well as the eigenvalue
spectrum is clearly visible (the height and spread of the “emerging spectrum” shown
in the inset); the clustering does not change much at this small value. In (g), (h)
and (i), when a higher distortion of  = 0.6 is given to the correlation frame, the
shape of emerging spectrum as well as the MDS map change drastically. The emerging
spectrum for  = 0.6 is broader compared to  = 0.01. In the MDS plot, the stocks
with high correlations come nearer to each other and form more compact and distinct
clusters, as compared to  = 0 and  = 0.01.

2.4. Noise-suppression in a similarity matrix among correlation frames over different

time-epochs
The noise-suppressed cross-correlation structures of return matrices C(τ ) across
different times τ = 1, ..., n, can be compared based on their similarities. If there
 Identifying long-term precursor 8
(a) (b) (c) (d)

(e) (f ) (g) (h)

Figure 3. Noise-suppression in a similarity matrix among correlation frames

over different time-epochs. (a) and (e) show the similarity matrices (without noise-
suppression  = 0 and with noise-suppression  = 0.6) among 805 correlation frames;
(b) and (f) show the corresponding MDS maps for USA. (c) and (g) show the similarity
matrices (without noise-suppression  = 0 and with noise-suppression  = 0.6) among
798 correlation frames; (b) and (f) show the corresponding MDS maps for JPN. The
similarity matrices give insight of the stock market evolution over 32 years (1985-
2016). Red-yellow strips in the similarity matrices exhibit the crashes of the respective
markets. The effect of noise-suppression is clearly visible on both similarity matrices
as well as MDS maps.

are two correlation matrices C(τ1 ) and C(τ2 ) at different time-epochs τ1 and τ2 ,
each computed over a short time-epoch of M days, then to quantify the similarity
between the correlation structures, the similarity measure is computed as: ζ(τ1 , τ2 ) ≡
h| Cij (τ1 ) − Cij (τ2 ) |i, where | ... | denotes the absolute value and h...i denotes the
average over all matrix elements {ij} [13]. We then use the MDS map to visualize the
information contained in n × n similarity matrix, where each element is ζ(τp , τq ), where
p, q = 1, ...n.
Interestingly, the noise-suppression applied to individual correlation frames in short
time-epochs, has a dramatic effect in the similarity matrix too. Figure 3 shows the ef-
fect of noise-suppression on the similarity matrix [13] and the corresponding MDS map.
Each correlation frame is computed with N = 194 stocks of USA; hence, for the time
series of length T = 8060 days during the period 1985-2016, there are n = 805 correla-
tion frames constructed from short time-epochs of M = 20 days and shifts of ∆τ = 10
days (50% overlapping time-epochs). Similarly, we have N = 165 stocks of JPN; the
time series of length T = 7990 days in the same period yield n = 798 correlation frames.
The sharp changes in the structural patterns of the similarity matrices become evident
at higher  = 0.6. It is noteworthy that figure 3(e) shows the block structure for the
Identifying long-term precursor 9

USA market and reveals the fact that behavior of USA market was relatively calmer
till 2002 and it became more volatile afterwards; the red-yellow stripes highlighting the
crash periods. Similarly, figure 3(g) shows that the JPN market became more volatile
from 1990 onward; also, it went through more critical periods as compared to USA
market. Importantly, the MDS maps with the noise-suppression parameter  = 0.6 are
more compact and denser, which lead to better clustering and determination of optimal
number of markets states (see also supplementary figures S2 and S3).

2.5. Determining optimal number of market states

To determine the number of market states, we find the number of clusters that can group
together the noise-suppressed cross-correlation return matrices C(τ ) across different
time-epochs τ = 1, ..., n, based on their similarities [13]. We use the MDS map to
visualize the information contained in n × n similarity matrix, and then use this MDS
map with n objects for k-means clustering. The k-means clustering, which is a heuristic
algorithm, aims to partition n numbers of correlation frames into k clusters or groups
in which each object/frame belongs to the cluster with the centroid (nearest mean
correlation), serving as a prototype of the cluster. In k-means clustering, the value of k
can be optimized by different techniques [28, 29]. Here, we propose a new approach
for optimizing k. We measure the mean and the standard deviation of the intra-
cluster distances using an ensemble of fairly large number (say 500) of different initial
conditions (choices of random coordinates for the k-centroids or equivalently random
initial clustering of n objects); each set of initial conditions may result in slightly different
clustering of the n different correlation frames. If the clusters are distinct (or far apart
in coordinate space) then even for different initial conditions, the k-means clustering
yield same results, yielding a small variance of the intra-cluster distance. The problem
of allocating the frames into the different clusters becomes acute when the clusters are
very close or overlapping, as the initial conditions can influence the final clustering. So
there is a larger variance of the intra-cluster distance. Therefore, the minimum variance
or standard deviation for a particular number of clusters displays the robustness of the
clustering. For optimizing the number of clusters, we propose that one should look for
maximum k, which has the minimum variance or standard deviation in the intra-cluster
distances with different initial conditions. We suppose this is easier than determining
the “elbow point” from the intra-cluster distance versus number of clusters curve [29].
For each cluster, one computes the average/variance of the point-to-centroid
distances for all the points belonging to the cluster; the mean/variance of the intra-
cluster distances is the mean/variance of the k values obtained from each of the k
clusters. Next, we use 500 different initial conditions for the k-means clustering, each
yielding a slightly different clustering result. One then computes the average as well
as the variance (or standard deviation) of the mean intra-cluster distances among the
ensemble of 500 runs. Then, the plots of average intra-cluster distance as functions of the
Identifying long-term precursor 10

(a) (b)

Figure 4. Plots of intra-cluster distance as a function of number of clusters.

Results shown for (a) USA and (b) JPN, for the noise-suppression parameter  = 0.6.
The k-means clustering is performed on the MDS map generated from 805 noise-
suppressed correlation frames of USA and 798 noise-suppressed correlation frames of
JPN, with 500 initial conditions in k-means clustering. The errorbars are the standard
deviations of the intra-cluster distances arising from the ensemble of 500 random initial
conditions to centroids for the initial clustering. The plots show the minima of standard
deviations at k = 4 for USA and k = 5 for JPN, which indicate the “optimal” number of
clusters. Inset: Plot of intra-cluster distance vs k for all 500 random initial conditions.
Each colored line corresponds to one such initial condition.

number of clusters k for USA and JPN are shown in figures 4(a) and (b), respectively.
The standard deviations of the intra-cluster distances measured for 500 initial conditions
are shown as the error bars. The insets of figures 4(a) and (b), show the plots for 500
initial conditions. As mentioned earlier, the value of k is optimized by keeping the
standard deviation lowest and the number of clusters highest; note that for k = 1,
the standard deviations are always trivially zero. We find that for USA, the standard
deviations are low till k = 4 and then grow for higher number of clusters; thus, k = 4 is
the optimal number of clusters. For JPN, which is more complex than USA, the standard
deviation is low for k = 1, 2, 3, increases for k = 4 and then decreases drastically for
k = 5; beyond that again the standard deviation is higher. Thus, k = 5 is the optimum
number of clusters for JPN.
The final k-means clustering of the correlation frames in the similarity matrix is
therefore performed for k = 4 clusters (USA) and k = 5 clusters (JPN), as shown
in figures 5(a) and (b), respectively. We identify the points in each cluster (different
colors represent different clusters) with similar correlation patterns and nearby mean
correlation as one market state. Based on k-means clustering, figure 5(c) shows four
different market states S1, S2, S3 and S4 of USA, where S1 corresponds to the calm
state (with low mean correlation) and S4 corresponds to the crash or critical state (with
high mean correlation); figure 5(d) shows five market states S1, S2, S3, S4 and S5 of
 Identifying long-term precursor 11

(a) (b)

(c)

(d)

Figure 5. Market states. (a) Classification of the USA market into four market
states. (b) Classification of the JPN market into five market states. k-means clustering
is performed on MDS map constructed from noise suppressed ( = 0.6) similarity
matrix. The coordinates assigned in the MDS map are the corresponding correlation
frames. For USA, we have 805 correlation frames of time-epoch M = 20 days with a
shift of ∆t = 10 days; for JPN, we have 798 correlation frames for the same. (c) shows
the four different states of USA market S1, S2, S3 and S4, where S1 corresponds to
a calm state (with low mean correlation) and S4 corresponds to the crash or critical
state (with high mean correlation). (d) shows the fives different states of JPN market
S1, S2, S3, S4 and S5, where S1 corresponds to the calm state and S5 corresponds to
the critical state.
Identifying long-term precursor 12

JPN, where S1 corresponds to the calm state and S5 corresponds to the critical state,
respectively. The states are arranged in the increasing order of mean correlation. Here,
we can also see clear differences structure-wise among the correlation matrices, e.g.,
there are strong intra-sectoral correlations within the energy, finance and utility sectors,
in each of the market states of USA.
It may also be mentioned that the selection of noise-suppression parameter  = 0.6
is not totally arbitrary. We compared the plots of the average intra-cluster distance as
function of the number of clusters for both USA and JPN, using  ranging from 0.1 to
0.7 (shown in supplementary figures S2 and S3). The outcome of the comparison is that
 = 0.6 yields the best results.

2.6. Co-occurrence probabilities and dynamical transitions of market states

Once the classification of the short-time cross-correlation frames into different market
states are complete, one can follow the evolution of the market as dynamical transitions
of the different markets states. Figures 6(a) and (c) show the evolution dynamics of
market states of USA and JPN, during 1985-2016. In USA, the market oscillates among
the four states S1, S2, S3 and S4. Often S1 or S2 states (with relatively low mean
correlations) tend to remain in the same state for a long time; at other times, the market
jumps to a higher mean correlation state S3 or S4. Similarly, for JPN the dynamical
transitions among the five market states S1, S2, S3, S4 and S5. The probabilistic
plots of the market states dynamics are shown in figures 6(b) and (d), for USA and
JPN, respectively. The color length of any market state is the probability of that state
computed during 110 days (10 overlapping epochs). Evident from the probability plots:
(a) In USA, before 2002 the market was mostly in state S1; the market became more
volatile, with more frequent transitions to other states, 2002 onward, and (b) in JPN,
market became more volatile from 1990 onward. The same kind of behavior is also
observed from the temporal evolution of the mean correlation (see supplementary figure
S1).
Figure 7(a) and (b) show the bar plots of the co-occurrences of the market states
for USA and JPN, respectively; the networks representing the transition probabilities
(co-occurrences of paired market states) for USA and JPN are respectively shown in
figures 7(c) and (d), with corresponding values given in Tables 2 and 3. The probability
of the co-occurrence of paired market states (S3, S4) of USA is about 6%. If we neglect
the diagonal entries of the bar plot, which shows the high probabilities of staying in
the same states, then we can safely infer that with the significant transition probability,
the state S3 of USA acts like a “precursor” to the state S4 (market crash); similarly,
for JPN the state S4 acts like a “precursor” to the critical state S5, with significant
transition probability of about 8%. Entries just above and below the diagonals of the
3D bar plots are also quite high, which show that the transitions primarily happen
between immediately adjacent states, and only exceptions of remote transitions being
Identifying long-term precursor 13
(a)

(b)

(c)

(d)

Figure 6. Dynamical evolution of market states for USA and JPN. (a)
Temporal dynamics of the USA in four different states (S1, S2, S3 and S4) for the
period of 1985-2016. (b) Probability plot of the four market states with each color
length corresponds to the evolution probability of these four states during 110 days
(10 overlapping epochs). (c) and (d) show similar results for JPN with five market
states (S1, S2, S3, S4 and S5).

in the cases like the Black Monday crash of 1987, etc.

Finally, let us test the simple hypothesis whether the system jumps randomly from
state Si to Sj with probabilities Wij or not. Note that, if we simply look at the curves in
figures 7 (c) and (d), it is not obvious that this is indeed the case. However, if we make
this hypothesis, we can obtain expressions for the probability that the system should be
in one state over long times. This follows from the general theory of Markov chains [30],
but for the sake of keeping the paper self-contained, we briefly explain the details below.
Let Pi (n) be the probability that the system be in state i after n steps (time-
epochs). Using the definition of Wij , as well as the assumption that the transition to j
Identifying long-term precursor 14

(a) (b)

(c) (d)

Figure 7. Transition probabilities of market states and determination of

long-term precursors of critical states. (a) and (b) 3D bar plots of co-occurrence
frequencies of paired market states (MS) for USA and JPN, respectively. (c) and (d)
represent the networks of transition probabilities between different states for USA and
JPN, respectively. See Tables 2) and 3 for the transition probabilities. The probability
of the co-occurrence of paired market states (S3, S4) of USA is 6%, which indicates
that state S3 of USA acts like a precursor to the market state S4 (crashes); similarly,
for JPN, the probability of co-occurrence of paired critical market states (S4, S5) of
JPN is about 8%, which indicates that the S4 state of JPN acts like a precursor to the
critical state S5 (crashes).

depends only on the previous state via Wij , and in no way on the previous history, we
obtain
X
Pi (n + 1) = Wji Pj (n), (1)
j

where the sum is over all possible states j. After long times, it is plausible, and can
in fact be proved rigorously, that the probability distribution becomes independent of
(0)
n; in other words, the distribution reaches an equilibrium state Pi . The latter then
satisfies the equations
(0)
X (0)
Pi = Wji Pj . (2)
j

This can be solved explicitly, if Wij is known. The solution can be proved to be always
Identifying long-term precursor 15

2nd MS →
1st MS S1 S2 S3 S4
↓
S1 0.869 0.112 0.017 0.002
S2 0.221 0.623 0.152 0.004
S3 0.033 0.333 0.575 0.058
S4 0 0 0.273 0.727
Table 2. USA: Co-occurrence probability of four market states (MS) (first is followed
by second).
2nd MS →
1st MS S1 S2 S3 S4 S5
↓
S1 0.809 0.155 0.023 0.009 0.005
S2 0.150 0.634 0.179 0.033 0.004
S3 0.014 0.234 0.603 0.120 0.029
S4 0.011 0.075 0.330 0.511 0.075
S5 0.036 0 0.107 0.393 0.464
Table 3. JPN: Co-occurrence probability of five market states (MS) (first is followed
by second).

positive, and can always be normalized such that

X (0)
Pi = 1, (3)
i
(0)
so that the numbers Pi can indeed be interpreted as a set of probabilities.
In the cases where the Wij ’s are given by Table 2 (for the USA) or Table 3 (for
JPN), it is straightforward to compute the equilibrium distributions: for the USA, one
finds:
(0) (0) (0) (0)
P1 = 0.523 P2 = 0.288 P3 = 0.149 P4 = 0.040. (4)
For JPN, on the other hand:
(0) (0) (0)
P1 = 0.274 P2 = 0.308 P3 = 0.263
(0) (0)
P4 = 0.119 P5 = 0.036. (5)
The actual frequencies for the four characteristic market states S1, S2, S3, and
S4 of USA, obtained from figure 6(a), enable us to compute the probabilities:
0.523, 0.287, 0.149, and 0.041, respectively. Similarly, actual frequencies for the five
characteristic market states S1, S2, S3, S4 and S5 of JPN, obtained from figure 6(c),
enable us to compute the probabilities: 0.277, 0.308, 0.262, 0.118 and 0.035, respectively.
These probabilities are indeed very close to those in Eqs. 4 and 5, and therefore our
hypothesis is correct.
Identifying long-term precursor 16

3. Summary and concluding remarks

In summary, we have studied the identification of market states and long-term precursors
to critical states (crashes) in financial markets, based on the probabilistic occurrences of
correlation patterns, determined using noise-suppressed short-time correlation matrices.
We analyzed and compared the data of the S&P 500 (USA) and Nikkei 225 (JPN) stock
markets over a 32-year period. We used the power mapping method to reduce the
noise of the singular correlation matrices and obtained distinct and denser clusters in
the two/three dimensional MDS maps. The effects are prominent also on the similarity
matrices and the corresponding MDS maps. The evolution of the market can be followed
by the dynamics transitions between the market states. Using multidimensional scaling
maps, we applied k-means clustering to divide the clusters of similar correlation patterns
of different time-epochs into k groups or market states. We showed that based on
the cluster radii we could have a fairly robust determination of the optimal number
of clusters. In each market, the value of optimal number of clusters was chosen by
keeping the standard deviation of the intra-cluster distance ‘minimum’ and number of
clusters ‘highest’. Thus, based on the modified prescription of finding similar clusters of
correlation patterns, we characterized USA by four market states and JPN by five.
One must mention that this method yields the correlation frames that correspond
to the critical states (or crashes). We have verified that these indeed correspond to
the well-known financial market crashes; also, specifically studied the properties of the
emerging spectrum and characterization of the critical states (catastrophic instabilities)
in Refs. [10, 15]. We also analyzed the co-occurrence probabilities of the paired market
states. We observed that the probability of remaining in the same state is much higher
than the transition to a different state. It implies that market states also feel an “inertia”
– stay in the same states for a long time. Also, probable transitions are the nearest
neighbor transitions and from the co-occurrence table we showed that the probability
reduces very fast if one moved away from the diagonal. Hence, the transitions to other
states mainly occurred in immediately adjacent states with a few rare intermittent
transitions to the remote states. The state adjacent to the critical state (crash) behaved
like a long-term precursor for the critical state, and this prescription could be helpful in
constructing an early warning system for financial market crashes.

Acknowledgments

A.C. and K.S. acknowledge the support by grant number BT/BI/03/004/2003(C) of

Govt. of India, Ministry of Science and Technology, Department of Biotechnology,
Bioinformatics division, University of Potential Excellence-II grant (Project ID-47) of
JNU, New Delhi, and the DST-PURSE grant given to JNU by the Department of
Science and Technology, Government of India. K.S. acknowledges the University Grants
Commission (Ministry of Human Research Development, Govt. of India) for her senior
research fellowship. H.K.P. and R.C. are grateful for postdoctoral fellowships provided
Identifying long-term precursor 17

by UNAM-DGAPA. F.L. acknowledges support from the project UNAM-DGAPA-

PAPIIT IN103017 and CONACyT CB-254515. A.C., K.S. and T.H.S. acknowledge
the support grant by CONACyT through Project FRONTERAS 201, and also support
from the project UNAM-DGAPA-PAPIIT IG 100616.

References

[1] Vemuri V 1978 Modeling of Complex Systems: An Introduction (Academic Press, New York)
[2] Gell-Mann M 1995 Complexity 1 16–19
[3] Bar-Yam Y 2002 Encyclopedia of Life Support Systems (EOLSS), UNESCO, EOLSS Publishers,
Oxford, UK
[4] Mantegna R N and Stanley H E 2007 An introduction to econophysics: correlations and complexity
in finance (Cambridge University Press, Cambridge)
[5] Bouchaud J P and Potters M 2003 Theory of Financial Risk and Derivative Pricing: from
Statistical Physics to Risk Management (Cambridge University Press)
[6] Sinha S, Chatterjee A, Chakraborti A and Chakrabarti B K 2010 Econophysics: an introduction
(John Wiley & Sons)
[7] Chakraborti A, Muni Toke I, Patriarca M and Abergel F 2011 Quantitative Finance 11 991–1012
[8] Chakraborti A, Muni Toke I, Patriarca M and Abergel F 2011 Quantitative Finance 11 1013–1041
[9] Chakraborti A, Challet D, Chatterjee A, Marsili M, Zhang Y C and Chakrabarti B K 2015 Physics
Reports 552 1–25
[10] Chakraborti A, Sharma K, Pharasi H K, Das S, Chatterjee R and Seligman T H 2018 arXiv
preprint arXiv:1801.07213
[11] Sornette D 2004 Why Stock Markets Crash: Critical Events in Complex Financial Systems
(Princeton University Press)
[12] Buchanan M 2000 Ubiquity: Why Catastrophes Happen (Three Rivers Press, New York)
[13] Münnix M C, Shimada T, Schäfer R, Leyvraz F, Seligman T H, Guhr T and Stanley H E 2012
Scientific reports 2 644
[14] Chetalova D, Schfer R and Guhr T 2015 Journal of Statistical Mechanics: Theory and Experiment
2015 P01029
[15] Pharasi H K, Sharma K, Chakraborti A and Seligman T H 2018 Complex market dynamics
in the light of random matrix theory New Perspectives and Challenges in Econophysics and
Sociophysics ed Abergel F, Chakrabarti B, Chakraborti A, Deo N and Sharma K (Springer New
Economic Windows)
[16] Schäfer R, Seligman T H et al. 2013 Physical Review E 88 032115
[17] Laloux L, Cizeau P, Bouchaud J P and Potters M 1999 Physical review letters 83 1467
[18] Plerou V, Gopikrishnan P, Rosenow B, Amaral L A N and Stanley H E 1999 Physical review letters
83 1471
[19] Guhr T and Kälber B 2003 Journal of Physics A: Mathematical and General 36 3009
[20] Bouchaud J P and Potters M 2000 Theory of Financial Risks (Cambridge University Press,
Cambridge)
[21] Schmitt T A, Schäfer R, Wied D and Guhr T 2016 Empirical Economics 50 1091–1109
[22] Vinayak and Seligman T H 2014 AIP Conference Proceedings 1575 196
[23] Borg I and Groenen P 1997 Modern Multidimensional Scaling: Theory and Applications Springer
series in statistics (Springer)
[24] 2017 Yahoo finance database.accessed on 7th july, 2017, using the r open source programming
language and software environment for statistical computing and graphics URL https://
finance.yahoo.co.jp/
[25] Mantegna R N 1999 The European Physical Journal B - Condensed Matter and Complex Systems
11 193–197
Identifying long-term precursor 18

[26] Marčenko V A and Pastur L A 1967 Mathematics of the USSR-Sbornik 1 457

[27] Schäfer R, Nilsson N F and Guhr T 2010 Quantitative Finance 10 107–119
[28] Gonzalez T F 1985 Theoretical Computer Science 38 293 – 306
[29] Bholowalia P and Kumar A 2014 International Journal of Computer Applications 105 17–24
[30] Ross S M 1996 Stochastic processes (Wiley, New York)

Supplementary information

(a)

(b)

Figure S1. Plots of the mean correlation without noise-suppression (blue) and with
high noise-suppression of  = 0.6 (magenta). For (a) USA, and (b) JPN. USA market
was relatively calm upto 2002 and became turbulent with high mean correlation from
2002 onward; JPN market became turbulent 1990 onward.
Identifying long-term precursor 19

Figure S2. Plots of the intracluster distance as a function of number of clusters

k of USA, for different value of noise-suppression parameter 0.1 6  6 0.7 using k-
means clustering. We used an ensemble of 500 random generated seeds for analyzing
the robustness of different clusters in the k-means clustering. The errorbars are
the deviation of the measure of intra-cluster distances arise due to different random
seeds. The points lie on the boundary of different clusters are subjected to change the
association with the cluster for different initial condition to the centroids in the k-mean
clustering. It changes the measure of intra-cluster distance among clusters. Inset shows
different color lines corresponds to different seed. The value is optimized by keeping
the standard deviation ‘lowest’ and number of cluster ‘highest’, simultaneously, for the
intra-cluster distance. The results are best for  = 0.6 and show minimum deviation
for k = 4 (max) and it grows for k > 4.
Identifying long-term precursor 20

Figure S3. Plots of the intra-cluster distance as a function of number of clusters

k of JPN, for different value of nonlinear suppression parameter 0.1 6  6 0.7
using k-means clustering. We used an ensemble of 500 random generated seeds for
analyzing the robustness of different clusters in the k-means clustering. The errorbars
are the deviation of the measure of intra-cluster distances arise due to different random
seeds. The points lie on the boundary of different clusters are subjected to change the
association with the cluster for different initial condition to the centroids in the k-mean
clustering. It changes the measure of intra-cluster distance among clusters. Inset shows
different color lines corresponds to different seed. The value is optimized by keeping
the standard deviation ‘lowest’ and number of cluster ‘highest’, simultaneously, for the
intra-cluster distance. The results are best for  = 0.6 and show minimum deviation
for k = 5 (max) and it grows for k > 5.
Identifying long-term precursor 21

Table S1. List of all stocks of USA market (S&P 500) considered for the analysis.
The first column has the serial number, the second column has the abbreviation, the
third column has the full name of the stock, and the fourth column specifies the sector
as given in the S&P 500.
S.No. Code Company Name Sector
1 CMCSA Comcast Corp. Consumer Discretionary
2 DIS Walt Disney Co. Consumer Discretionary
3 F Ford Motor Consumer Discretionary
4 GPC Genuine Parts Consumer Discretionary
5 GPS Gap (The) Consumer Discretionary
6 GT Goodyear Tire & Rubber Consumer Discretionary
7 HAS Hasbro Inc. Consumer Discretionary
8 HD Home Depot Consumer Discretionary
9 HRB Block H&R Consumer Discretionary
10 IPG Interpublic Group Consumer Discretionary
11 JCP Penney (J.C.) Consumer Discretionary
12 JWN Nordstrom Consumer Discretionary
13 LEG Leggett & Platt Consumer Discretionary
14 LEN Lennar Corp. Consumer Discretionary
15 LOW Lowe’s Cos. Consumer Discretionary
16 MAT Mattel Inc. Consumer Discretionary
17 MCD McDonald’s Corp. Consumer Discretionary
18 NKE NIKE Inc. Consumer Discretionary
19 SHW Sherwin-Williams Consumer Discretionary
20 TGT Target Corp. Consumer Discretionary
21 VFC V.F. Corp. Consumer Discretionary
22 WHR Whirlpool Corp. Consumer Discretionary
23 ADM Archer-Daniels-Midland Co Consumer Staples
24 AVP Avon Products Consumer Staples
25 CAG ConAgra Foods Inc. Consumer Staples
26 CL Colgate-Palmolive Consumer Staples
27 CPB Campbell Soup Consumer Staples
28 CVS CVS Caremark Corp. Consumer Staples
29 GIS General Mills Consumer Staples
30 HRL Hormel Foods Corp. Consumer Staples
31 HSY The Hershey Company Consumer Staples
32 K Kellogg Co. Consumer Staples
33 KMB Kimberly-Clark Consumer Staples
34 KO Coca Cola Co. Consumer Staples
35 KR Kroger Co. Consumer Staples
36 MKC McCormick & Co. Consumer Staples
Identifying long-term precursor 22

37 MO Altria Group Inc Consumer Staples

38 SYY Sysco Corp. Consumer Staples
39 TAP Molson Coors Brewing Company Consumer Staples
40 TSN Tyson Foods Consumer Staples
41 WMT Wal-Mart Stores Consumer Staples
42 APA Apache Corporation Energy
43 COP ConocoPhillips Energy
44 CVX Chevron Corp. Energy
45 ESV Ensco plc Energy
46 HAL Halliburton Co. Energy
47 HES Hess Corporation Energy
48 HP Helmerich & Payne Energy
49 MRO Marathon Oil Corp. Energy
50 MUR Murphy Oil Energy
51 NBL Noble Energy Inc Energy
52 NBR Nabors Industries Ltd. Energy
53 SLB Schlumberger Ltd. Energy
54 TSO Tesoro Petroleum Co. Energy
55 VLO Valero Energy Energy
56 WMB Williams Cos. Energy
57 XOM Exxon Mobil Corp. Energy
58 AFL AFLAC Inc Financials
59 AIG American Intl Group Inc Financials
60 AON Aon plc Financials
61 AXP American Express Co Financials
62 BAC Bank of America Corp Financials
63 BBT BB&T Corporation Financials
64 BEN Franklin Resources Financials
65 BK The Bank of New York Mellon Corp. Financials
66 C Citigroup Inc. Financials
67 CB Chubb Corp. Financials
68 CINF Cincinnati Financial Financials
69 CMA Comerica Inc. Financials
70 EFX Equifax Inc. Financials
71 FHN First Horizon National Financials
72 HBAN Huntington Bancshares Financials
73 HCN Health Care REIT Financials
74 HST Host Hotels & Resorts Financials
75 JPM JPMorgan Chase & Co. Financials
76 L Loews Corp. Financials
Identifying long-term precursor 23

77 LM Legg Mason Financials

78 LNC Lincoln National Financials
79 LUK Leucadia National Corp. Financials
80 MMC Marsh & McLennan Financials
81 MTB M&T Bank Corp. Financials
82 PSA Public Storage Financials
83 SLM SLM Corporation Financials
84 TMK Torchmark Corp. Financials
85 TRV The Travelers Companies Inc. Financials
86 USB U.S. Bancorp Financials
87 VNO Vornado Realty Trust Financials
88 WFC Wells Fargo Financials
89 WY Weyerhaeuser Corp. Financials
90 ZION Zions Bancorp Financials
91 ABT Abbott Laboratories Health Care
92 AET Aetna Inc Health Care
93 AMGN Amgen Inc Health Care
94 BAX Baxter International Inc. Health Care
95 BCR Bard (C.R.) Inc. Health Care
96 BDX Becton Dickinson Health Care
97 BMY Bristol-Myers Squibb Health Care
98 CAH Cardinal Health Inc. Health Care
99 CI CIGNA Corp. Health Care
100 HUM Humana Inc. Health Care
101 JNJ Johnson & Johnson Health Care
102 LLY Lilly (Eli) & Co. Health Care
103 MDT Medtronic Inc. Health Care
104 MRK Merck & Co. Health Care
105 MYL Mylan Inc. Health Care
106 SYK Stryker Corp. Health Care
107 THC Tenet Healthcare Corp. Health Care
108 TMO Thermo Fisher Scientific Health Care
109 UNH United Health Group Inc. Health Care
110 VAR Varian Medical Systems Health Care
111 AVY Avery Dennison Corp Industrials
112 BA Boeing Company Industrials
113 CAT Caterpillar Inc. Industrials
114 CMI Cummins Inc. Industrials
115 CSX CSX Corp. Industrials
116 CTAS Cintas Corporation Industrials
117 DE Deere & Co. Industrials
Identifying long-term precursor 24

118 DHR Danaher Corp. Industrials

119 DNB Dun & Bradstreet Industrials
120 DOV Dover Corp. Industrials
121 EMR Emerson Electric Industrials
122 ETN Eaton Corp. Industrials
123 EXPD Expeditors Int’l Industrials
124 FDX FedEx Corporation Industrials
125 FLS Flowserve Corporation Industrials
126 GD General Dynamics Industrials
127 GE General Electric Industrials
128 GLW Corning Inc. Industrials
129 GWW Grainger (W.W.) Inc. Industrials
130 HON Honeywell Int’l Inc. Industrials
131 IR Ingersoll-Rand PLC Industrials
132 ITW Illinois Tool Works Industrials
133 JEC Jacobs Engineering Group Industrials
134 LMT Lockheed Martin Corp. Industrials
135 LUV Southwest Airlines Industrials
136 MAS Masco Corp. Industrials
137 MMM 3M Co. Industrials
138 ROK Rockwell Automation Inc. Industrials
139 RTN Raytheon Co. Industrials
140 TXT Textron Inc. Industrials
141 UNP Union Pacific Industrials
142 UTX United Technologies Industrials
143 AAPL Apple Inc. Information Technology
144 ADI Analog Devices Inc Information Technology
145 ADP Automatic Data Processing Information Technology
146 AMAT Applied Materials Inc Information Technology
147 AMD Advanced Micro Devices Information Technology
148 CA CA, Inc. Information Technology
149 HPQ Hewlett-Packard Information Technology
150 HRS Harris Corporation Information Technology
151 IBM International Bus. Machines Information Technology
152 INTC Intel Corp. Information Technology
153 KLAC KLA-Tencor Corp. Information Technology
154 LRCX Lam Research Information Technology
155 MSI Motorola Solutions Inc. Information Technology
156 MU Micron Technology Information Technology
157 TSS Total System Services Information Technology
158 TXN Texas Instruments Information Technology
Identifying long-term precursor 25

159 WDC Western Digital Information Technology

160 XRX Xerox Corp. Information Technology
161 AA Alcoa Inc Materials
162 APD Air Products & Chemicals Inc Materials
163 BLL Ball Corp Materials
164 BMS Bemis Company Materials
165 CLF Cliffs Natural Resources Materials
166 DD Du Pont (E.I.) Materials
167 ECL Ecolab Inc. Materials
168 FMC FMC Corporation Materials
169 IFF International Flav/Frag Materials
170 IP International Paper Materials
171 NEM Newmont Mining Corp. (Hldg. Co.) Materials
172 PPG PPG Industries Materials
173 VMC Vulcan Materials Materials
174 CTL CenturyLink Inc Telecommunications Services
175 FTR Frontier Communications Telecommunications Services
176 S Sprint Nextel Corp. Telecommunications Services
177 T AT&T Inc Telecommunications Services
178 VZ Verizon Communications Telecommunications Services
179 AEP American Electric Power Utilities
180 CMS CMS Energy Utilities
181 CNP CenterPoint Energy Utilities
182 D Dominion Resources Utilities
183 DTE DTE Energy Co. Utilities
184 ED Consolidated Edison Utilities
185 EIX Edison Int’l Utilities
186 EQT EQT Corporation Utilities
187 ETR Entergy Corp. Utilities
188 EXC Exelon Corp. Utilities
189 NEE NextEra Energy Resources Utilities
190 NI NiSource Inc. Utilities
191 PNW Pinnacle West Capital Utilities
192 SO Southern Co. Utilities
193 WEC Wisconsin Energy Corporation Utilities
194 XEL Xcel Energy Inc Utilities
Identifying long-term precursor 26

Table S2. List of all stocks of Japan market (Nikkei 225) considered for the analysis.
The first column has the serial number, the second column has the abbreviation, the
third column has the full name of the stock, and the fourth column specifies the sector
as given in the Nikkei 225.
S. No. Code Company Name Sector
1 S-8801 MITSUI FUDOSAN CO., LTD. Capital Goods
2 S-8802 MITSUBISHI ESTATE CO., LTD. Capital Goods
3 S-8804 TOKYO TATEMONO CO., LTD. Capital Goods
4 S-8830 SUMITOMO REALTY & DEVELOPMENT CO., LTD. Capital Goods
5 S-7003 MITSUI ENG. & SHIPBUILD. CO., LTD. Capital Goods
6 S-7012 KAWASAKI HEAVY IND., LTD. Capital Goods
7 S-9202 ANA HOLDINGS INC. Capital Goods
8 S-1801 TAISEI CORP. Capital Goods
9 S-1802 OBAYASHI CORP. Capital Goods
10 S-1803 SHIMIZU CORP. Capital Goods
11 S-1808 HASEKO CORP. Capital Goods
12 S-1812 KAJIMA CORP. Capital Goods
13 S-1925 DAIWA HOUSE IND. CO., LTD. Capital Goods
14 S-1928 SEKISUI HOUSE, LTD. Capital Goods
15 S-1963 JGC CORP. Capital Goods
16 S-5631 THE JAPAN STEEL WORKS, LTD. Capital Goods
17 S-6103 OKUMA CORP. Capital Goods
18 S-6113 AMADA HOLDINGS CO., LTD. Capital Goods
19 S-6301 KOMATSU LTD. Capital Goods
20 S-6302 SUMITOMO HEAVY IND., LTD. Capital Goods
21 S-6305 HITACHI CONST. MACH. CO., LTD. Capital Goods
22 S-6326 KUBOTA CORP. Capital Goods
23 S-6361 EBARA CORP. Capital Goods
24 S-6366 CHIYODA CORP. Capital Goods
25 S-6367 DAIKIN INDUSTRIES, LTD. Capital Goods
26 S-6471 NSK LTD. Capital Goods
27 S-6472 NTN CORP. Capital Goods
28 S-6473 JTEKT CORP. Capital Goods
29 S-7004 HITACHI ZOSEN CORP. Capital Goods
30 S-7011 MITSUBISHI HEAVY IND., LTD. Capital Goods
31 S-7013 IHI CORP. Capital Goods
32 S-7911 TOPPAN PRINTING CO., LTD. Capital Goods
33 S-7912 DAI NIPPON PRINTING CO., LTD. Capital Goods
34 S-7951 YAMAHA CORP. Capital Goods
35 S-1332 NIPPON SUISAN KAISHA, LTD. Consumer Goods
36 S-2002 NISSHIN SEIFUN GROUP INC. Consumer Goods
Identifying long-term precursor 27

37 S-2282 NH FOODS LTD. Consumer Goods

38 S-2501 SAPPORO HOLDINGS LTD. Consumer Goods
39 S-2502 ASAHI GROUP HOLDINGS, LTD. Consumer Goods
40 S-2503 KIRIN HOLDINGS CO., LTD. Consumer Goods
41 S-2531 TAKARA HOLDINGS INC. Consumer Goods
42 S-2801 KIKKOMAN CORP. Consumer Goods
43 S-2802 AJINOMOTO CO., INC. Consumer Goods
44 S-2871 NICHIREI CORP. Consumer Goods
45 S-8233 TAKASHIMAYA CO., LTD. Consumer Goods
46 S-8252 MARUI GROUP CO., LTD. Consumer Goods
47 S-8267 AEON CO., LTD. Consumer Goods
48 S-9602 TOHO CO., LTD Consumer Goods
49 S-9681 TOKYO DOME CORP. Consumer Goods
50 S-9735 SECOM CO., LTD. Consumer Goods
51 S-8331 THE CHIBA BANK, LTD. Finance
52 S-8355 THE SHIZUOKA BANK, LTD. Finance
53 S-8253 CREDIT SAISON CO., LTD. Finance
54 S-8601 DAIWA SECURITIES GROUP INC. Finance
55 S-8604 NOMURA HOLDINGS, INC. Finance
56 S-3405 KURARAY CO., LTD. Materials
57 S-3407 ASAHI KASEI CORP. Materials
58 S-4004 SHOWA DENKO K.K. Materials
59 S-4005 SUMITOMO CHEMICAL CO., LTD. Materials
60 S-4021 NISSAN CHEMICAL IND., LTD. Materials
61 S-4042 TOSOH CORP. Materials
62 S-4043 TOKUYAMA CORP. Materials
63 S-4061 DENKA CO., LTD. Materials
64 S-4063 SHIN-ETSU CHEMICAL CO., LTD. Materials
65 S-4183 MITSUI CHEMICALS, INC. Materials
66 S-4208 UBE INDUSTRIES, LTD. Materials
67 S-4272 NIPPON KAYAKU CO., LTD. Materials
68 S-4452 KAO CORP. Materials
69 S-4901 FUJIFILM HOLDINGS CORP. Materials
70 S-4911 SHISEIDO CO., LTD. Materials
71 S-6988 NITTO DENKO CORP. Materials
72 S-5002 SHOWA SHELL SEKIYU K.K. Materials
73 S-5201 ASAHI GLASS CO., LTD. Materials
74 S-5202 NIPPON SHEET GLASS CO., LTD. Materials
75 S-5214 NIPPON ELECTRIC GLASS CO., LTD. Materials
76 S-5232 SUMITOMO OSAKA CEMENT CO., LTD. Materials
Identifying long-term precursor 28

77 S-5233 TAIHEIYO CEMENT CORP. Materials

78 S-5301 TOKAI CARBON CO., LTD. Materials
79 S-5332 TOTO LTD. Materials
80 S-5333 NGK INSULATORS, LTD. Materials
81 S-5706 MITSUI MINING & SMELTING CO. Materials
82 S-5707 TOHO ZINC CO., LTD. Materials
83 S-5711 MITSUBISHI MATERIALS CORP. Materials
84 S-5713 SUMITOMO METAL MINING CO., LTD. Materials
85 S-5714 DOWA HOLDINGS CO., LTD. Materials
86 S-5715 FURUKAWA CO., LTD. Materials
87 S-5801 FURUKAWA ELECTRIC CO., LTD. Materials
88 S-5802 SUMITOMO ELECTRIC IND., LTD. Materials
89 S-5803 FUJIKURA LTD. Materials
90 S-5901 TOYO SEIKAN GROUP HOLDINGS, LTD. Materials
91 S-3865 HOKUETSU KISHU PAPER CO., LTD. Materials
92 S-3861 OJI HOLDINGS CORP. Materials
93 S-5101 THE YOKOHAMA RUBBER CO., LTD. Materials
94 S-5108 BRIDGESTONE CORP. Materials
95 S-5401 NIPPON STEEL & SUMITOMO METAL CORP. Materials
96 S-5406 KOBE STEEL, LTD. Materials
97 S-5541 PACIFIC METALS CO., LTD. Materials
98 S-3101 TOYOBO CO., LTD. Materials
99 S-3103 UNITIKA, LTD. Materials
100 S-3401 TEIJIN LTD. Materials
101 S-3402 TORAY INDUSTRIES, INC. Materials
102 S-8001 ITOCHU CORP. Materials
103 S-8002 MARUBENI CORP. Materials
104 S-8015 TOYOTA TSUSHO CORP. Materials
105 S-8031 MITSUI & CO., LTD. Materials
106 S-8053 SUMITOMO CORP. Materials
107 S-8058 MITSUBISHI CORP. Materials
108 S-4151 KYOWA HAKKO KIRIN CO., LTD. Pharmaceuticals
109 S-4503 ASTELLAS PHARMA INC. Pharmaceuticals
110 S-4506 SUMITOMO DAINIPPON PHARMA CO., LTD. Pharmaceuticals
111 S-4507 SHIONOGI & CO., LTD. Pharmaceuticals
112 S-4519 CHUGAI PHARMACEUTICAL CO., LTD. Pharmaceuticals
113 S-4523 EISAI CO., LTD. Pharmaceuticals
114 S-7201 NISSAN MOTOR CO., LTD. Technology
115 S-7202 ISUZU MOTORS LTD. Technology
116 S-7205 HINO MOTORS, LTD. Technology
Identifying long-term precursor 29

117 S-7261 MAZDA MOTOR CORP. Technology

118 S-7267 HONDA MOTOR CO., LTD. Technology
119 S-7270 SUBARU CORP. Technology
120 S-7272 YAMAHA MOTOR CO., LTD. Technology
121 S-3105 NISSHINBO HOLDINGS INC. Technology
122 S-6479 MINEBEA MITSUMI INC. Technology
123 S-6501 HITACHI, LTD. Technology
124 S-6502 TOSHIBA CORP. Technology
125 S-6503 MITSUBISHI ELECTRIC CORP. Technology
126 S-6504 FUJI ELECTRIC CO., LTD. Technology
127 S-6506 YASKAWA ELECTRIC CORP. Technology
128 S-6508 MEIDENSHA CORP. Technology
129 S-6701 NEC CORP. Technology
130 S-6702 FUJITSU LTD. Technology
131 S-6703 OKI ELECTRIC IND. CO., LTD. Technology
132 S-6752 PANASONIC CORP. Technology
133 S-6758 SONY CORP. Technology
134 S-6762 TDK CORP. Technology
135 S-6770 ALPS ELECTRIC CO., LTD. Technology
136 S-6773 PIONEER CORP. Technology
137 S-6841 YOKOGAWA ELECTRIC CORP. Technology
138 S-6902 DENSO CORP. Technology
139 S-6952 CASIO COMPUTER CO., LTD. Technology
140 S-6954 FANUC CORP. Technology
141 S-6971 KYOCERA CORP. Technology
142 S-6976 TAIYO YUDEN CO., LTD. Technology
143 S-7752 RICOH CO., LTD. Technology
144 S-8035 TOKYO ELECTRON LTD. Technology
145 S-4543 TERUMO CORP. Technology
146 S-4902 KONICA MINOLTA, INC. Technology
147 S-7731 NIKON CORP. Technology
148 S-7733 OLYMPUS CORP. Technology
149 S-7762 CITIZEN WATCH CO., LTD. Technology
150 S-9501 TOKYO ELECTRIC POWER COMPANY Transportation & Utilities
151 S-9502 CHUBU ELECTRIC POWER CO., INC. Transportation & Utilities
152 S-9503 THE KANSAI ELECTRIC POWER CO., INC. Transportation & Utilities
153 S-9531 TOKYO GAS CO., LTD. Transportation & Utilities
154 S-9532 OSAKA GAS CO., LTD. Transportation & Utilities
155 S-9062 NIPPON EXPRESS CO., LTD. Transportation & Utilities
156 S-9064 YAMATO HOLDINGS CO., LTD. Transportation & Utilities
Identifying long-term precursor 30

157 S-9101 NIPPON YUSEN K.K. Transportation & Utilities

158 S-9104 MITSUI O.S.K.LINES, LTD. Transportation & Utilities
159 S-9107 KAWASAKI KISEN KAISHA, LTD. Transportation & Utilities
160 S-9001 TOBU RAILWAY CO., LTD. Transportation & Utilities
161 S-9005 TOKYU CORP. Transportation & Utilities
162 S-9007 ODAKYU ELECTRIC RAILWAY CO., LTD. Transportation & Utilities
163 S-9008 KEIO CORP. Transportation & Utilities
164 S-9009 KEISEI ELECTRIC RAILWAY CO., LTD. Transportation & Utilities
165 S-9301 MITSUBISHI LOGISTICS CORP. Transportation & Utilities