Stock Market: Statistical Analysis

Of its Indexes and Its constituents

Priyanshi Singh Abha Thakral

Department of Computer Science and Engineering. Department of Computer Science and Engineering.
Amity University Noida, India Amity University Noida, India
priyan.singh1@gmail.com abhareads@gmail.com

Abstract— The ever-changing realm of the stock market is of that Index and all the stocks and then calculated Standard
constantly thriving under the process of modifications and Deviation on closing prices of the same. For analysis, we
alterations. Thus, making a profit from it is hard and requires compared the respective results followed by result verification.
intensive planning. It is in the context of this fact that makes We used Apache Hive to process this big data, in addition
Stock Market analysis the first and foremost priority for any reaping the benefits of Map-Reduce and parallel processing of
financial investment. Considering the behavioural aspects of Hadoop.
stock prices which have a tendency to rise and fall unexpectedly,
leads to a volatile scenario. However, to acquire some insight, A lot of review and related work is done in this field which
intellectual wit and smartness to extract the best, a thorough and is discussed in the next section, followed by our proposed
consistent analysis is most popular and tested way. This paper methodology and results. The paper is bring to a close by a
aims to determine top high performing stocks having good conclusion.
returns under given index that would be most safe and beneficial
for investment. Using historical data we were able to obtain top
stocks that are advisable for investment. We also verified our
II. LITERATURE REVIEW AND RELATED WORK
results by analyzing contemporary data similarly and found out Volatility indicates the fluctuations of returns, it measures
that the performance and returns of these stocks were still high the risk associated with the stock [1]. Volatility has been of
irrespective of volatility. crucial importance for understanding and learning in finance
markets. It is found to be an evolving process, highly non-
Keywords—Stock Market; Volatility; historical volatility; Stock linear [2]. Garman Klass estimate with Arima time series
Market Indexes; Nifty50; NSE(National Stock Exchange); Big forecasting technique is found out to be more accurate for
Data; Hadoop; Hive volatility forecast amongst various combinations of popular
volatility estimating methods with Arima, Afrima and feed
I. INTRODUCTION forward neural network time series forecasting techniques [1].
A stock market (also known as a stock exchange) has two The Garch models are widely used by financial professionals
basic functionality: First is to facilitate the process for the for estimation of volatility and stock analysis. Indian stock
companies by means of which they can trade. Second is to market is found to have asymmetrical volatility and is mainly
organise and manage the venue, where trade can properly take affected by past negative shocks on applying 3 models of
place. Investing and profiting from the market has never been Garch family i.e. Garch, E-Garch and Aparch on NIFTY and
simple, and that’s due to obvious uncertainty and high volatile BSE data [3]. The Garch effect amongst rest of the other
nature of the market i.e shares/equities have high potential to methods is significantly strong, which indicates the persistence
amplify and fall in value rapidly. Volatility is a statistical of volatility as well [3].The dynamic conditional correlation
measure of the dispersion of returns for a given security or model (DCC-Garch model) was found perfectly fit to figure
market index. Commonly, the higher the volatility, the riskier out the conditional correlations and volatility between different
the security. Historical volatility also ‘known volatility’ is the markets and also optimal for portfolio weights and hedge ratio
volatility of actual prices of underlying stocks. They have in comparison to vector autoregressive moving average
proved to be most challenging yet rewarding and beneficial for (VARMA-Garch) model [4]. The sign and significant change
investment. To new traders and investors, the stock market of return of index or shocks to returns can be significant in
seems to be a bewildering range of options. Understanding figuring out the intensity of the information to which investors
some basic information about how to invest, where to invest pay attention as per search probability measured and conducted
can help in maximising the rate of return on the invested by Google for the several security performance indexes in the
money. In order to find out the most safe stocks listed under a category of attention of the investors and investment. It was
particular Index we collected 8 years (2009-16) historical data also demonstrated that increased investor attention diminishes
return predictability and, therefore, improves market efficiency

978-1-5386-0569-1$31.00 2017
c IEEE 962
[5]. With the help of minute by minute collected data for a or all of the open (O), high (H), low (L) and close (C). These
period of 1 year, an analysis using Toda-Yamamoto are the recorded prices under given category for the day. For
methodology was done regarding the causality relationship of example, opening price is the amount at which the market and
Granger between the trading volumes and prices of around 50 the stocks started trading for that day. Similarly, high, low and
NIFTY companies and illustrated that out of 50 only 29 closing prices denotes the highest, lowest and the price market
companies had bi-directional (two-way), between volume and closed at respectively. Closing prices or close(C) for
price causality. While 15 had a unidirectional causality calculations was used. The methodology has been shown as a
relationship, in it volume did not cause price but vice versa was flowchart in “Fig. 1”.
functional. Also, there are 6 such companies that did not have
any causal relationship at all [6]. Eventually, the Artificial
Neural Network (ANN) model integrated by statistical model
emerged as a solution to the problem of financial data over
single statistical models. It was found better for time sequence
analysis and prediction accuracy for forecasting movements of
the stocks. Likely, Performance analysis of Indian stock market
index using neural network time series model was done and
right parameters like epochs, momentum and learning rate for a
forecast network were found out [7]. It was observed that
normality test can help in getting more precise and accurate
predictions when combined with ANN. Later, dynamic and
hybrid ANNS were proposed for better accuracy and results [8-
9]. The results confirmed that the recurrent neural network
performed near accurate prediction and the hybrid prediction
model outperformed the former [9].
Big Data Analytics is the new technology that is trending
now days and is becoming popular because of various
advantages it comes up with. The most significant being, it
gives the enterprises the advantage of the stored historical data
and as well as fresh data [10]. It is proven for creating accurate
and better predictions for business and hence overcoming the
probability of loss. This emerging technology has slowly
started to make its way to the finance sector, mainly stock Fig. 1. Methodology
exchange market. In order to identify the right software
environment for scientific data analysis, Hadoop was evaluated Step I. Data Collection:
and modified to judge its performance, scalability and fault The historical 8 year (2009-2016) stock data was collected
tolerance. Hadoop, as a result, turned out to be more apt for from NSE website [16]. In this paper, the end of day’s
scientific data analysis in comparison to typical SQL based trading/prices or ‘close’ of the stocks for historical volatility
warehouses [11-13]. Also, the results of the data model taken calculation was considered. The price performance of all
from the GroupLens Research Project revealed that Apache securities on an equity index is based on prices at present close,
Hive, which is a data warehousing package built on Hadoop’s compared with the prices at the historical close.
top, is most appropriate in a low-cost hardware environment
[13]. Lots of other methods and techniques were implemented Also the present data (Jan 2017- April 2017) was collected
on Hadoop platform to process and analyze stock data, and of resultant stocks and index and similar calculations was
obtained satisfying results [14-15]. The stock exchange data is performed in order to verify our result.
typically available in bulk and to process this data into
meaningful information we used Data Analytics to arrive at Step II. Data Acquisition/Presentation
profitable predictions. We applied Hive to process the aforesaid The data is arranged date wise and checked for any
Big Data. missing and redundant values for each company. It is uploaded
to Hive Warehouse (HDFS) for processing. Further, Quarter
III. METHODOLOGY and year number is assigned to each row for the quarter and 4
Our purpose is to find high-performing stocks in order to year-wise analyses respectively.
reap benefits from the investment made. Historical volatility is
essentially a way to tell how far the stock might move in the Step III. Historical Volatility Calculation of Stocks and Nifty
future based on how fast it has been moving in the recent past. 50 Index
The idea is to measure performance in relevance with historical Close to Close measure for calculating volatility as for
volatility. The standard deviation calculated from close prices large dataset it is the best method and only marginal extra
not only indicates the performance but gives us some insight of accuracy is gained for each additional sample above 20 [17].
past volatility of that particular company.There are many Also, bias is directly proportional to sample size in case of this
different measures of historical volatility which can use some method. It is the simplest yet most common type of calculation

2017 International Conference On Smart Technology for Smart Nation 963

 that benefits from using only reliable prices from closing Step I. Results and Analysis of duration: 8 year-wise
auctions. Standard Deviation was calculated on ‘close price’ by “Fig. 2,” 3 Companies with a significant difference in
quarter, year, 4-year and 8-year under this method. standard deviation to that of NIFTY 50 were obtained. Eicher
Motors Ltd with a difference of 6115.23 topped, followed by
Step IV. Comparision of Standard Deviation of Stocks and Bosch Ltd and State Bank of India(SBIN) with 5296.41 and
Nifty 50 Index 2573.6 difference respectively.
After obtaining standard deviation of all the 50 stocks and
Nifty 50 index quarter-wise, year-wise, 4 year-wise and 8 year-
wise, we compared the resultant standard deviation values of
stocks and Nifty 50 by their respective duration range.

Step V. Results and Analysis

The stocks with their standard deviation greater than that of
NIFTY 50 index were picked.

Step VI. Results Verification

To verify the results, we calculated the Standard Deviation
of the newest 4 months data i.e. January to April (2017) and
compared it with the previous results.

IV. RESULTS AND ANALYSIS Fig. 2. Result of 8 Year Analysis.

Considering, X denotes company, then for dataset X = [50],

the results were obtained by calculating Standard Deviation on Step II. Results and Analysis of duration: 4 year-wise
C. The most promising companies are those who have been “Fig. 3,” Eicher Motors and Bosch Ltd are found to have
consistent in their performance despite the presence of grown significantly in recent years whereas SBIN has shown
volatility. The difference of standard deviation of the company steady growth. Furthermore, we noted that Asian paints,
and its index signifies the ratio of performance of that company Dr.Reddy's Laboratories Ltd and Maruti Suzuki India Ltd
to that of the index it belongs to. We broadened the horizon of have performed fairly in recent 4 years and can be taken into
our study by analyzing performance of stocks even for shorter consideration.
interval, quarter being the shortest. It further helped in
revelation of consistency in performance and the effect of
volatility on a particular stock (Table I). Also while going
down the lane, some potential companies having good
performance in the recent years might be found and can be
considered for further analysis.
TABLE I. Key Parameters used for Analysis

DURATION PARAMETERS
• Year
• Quarter-no.
Quarter-Wise
• X’s Standard Deviation
• Index ‘Standard Deviation
• Year Fig. 3. Result of 4 Year-wise Analysis.
Year-Wise • X’s Standard Deviation
• index’s standard deviation Step III. Results and Analysis of duration: year-wise
• Year From the year-wise analysis “Fig 4,” it was found that Eicher
4 year-Wise • X’s Standard Deviation Motors, Bosch Ltd and SBIN are consistent in all the years. It
• Index’s Standard Deviation is noted that Asian Paints Ltd had its last good performance in
• X’s Standard Deviation the year 2013 which is the reason for its promotion in 4-year
8 Year-Wise
• Index’s Standard Deviation analysis. It reflects that it was not consistent and therefore we
drop it. “Fig.5,”Maruti Suzuki had a good performance in the
The stocks from each category were selected on the following year 2015 and 2016 with fair difference i.e. 63.75 and 227.66
rule: respectively. It may be noted that its performance in 2016 may
X’ standard deviation > index’s standard deviation. have dominated 4-year analysis results. Therefore, it remains a
potential consideration. Considering Dr.Reddy, it performed
well in all the years (2011-15) and remained back by 343.5248

964 2017 International Conference On Smart Technology for Smart Nation

in the year 2016. Also, Housing Development Finance Corp Step IV. Results and Analysis of duration: Quarter-wise
Ltd (HDFC) turned out to perform well in the year It was noted that SBIN is most consistent as it performed
2010,12,15,16. However, it has performed well in only 2 well in all 28 quarters consecutively from 2010-16 followed by
recent years in a row. NTPC Limited performed fairly in 2015 Bosch Ltd who performed well in 24 quarters from 2010-16
and 2016 with 182.5 and 31.49 differences respectively. leaving few in between. Eicher Motors performed well in 16
Grasim was another company that outperformed with a good quarters from (2012-16). Maruti performed well in 2016, but
difference of 969.95 in the year 2016 directly after the year there is uncertainty of its steady performance as a whole.
2012. Therefore, it was appropriate to analyse these 4 Dr.Reddy performed well only in 2 quarters (2015-16). HDFC
companies furthermore. and NTPC were noted to perform fair in most of the quarters in
recent years and therefore can be considered. Sample quarter-
wise results are given in Table 2.

Step V. Results Verification

From the “fig. 5,” it can be noted that the companies we got
as a result, have steady performance and significant difference
in the first quarter of 2017. However, the ranking of the
companies are based on their quarter performance and should
not be compared to the ranking of 8-year analysis. Also, one
must not judge yearly ranking of a company based on this
quarter performance as due to the presence of volatility
significant changes in the rankings are expected.

Fig. 4. Result of Year-wise Analysis.

Fig. 6. Result of 1st quarter of 2017.

Based on the analysis it can be concluded that SBIN, Bosch

Ltd and Eicher Motors are the most promising companies and
it is worth noticing that although Eicher Motors secured 1st
position as a result of 8-year analysis it was less consistent in
comparison to other two companies. However, investment in
Fig. 5. Result of Year-wise Analysis. all the above 3 companies is profitable.

TABLE II. Sample Quarter-wise results

Year 2014 2015 2016
Quarter
Company 1 2 3 4 1 2 3 4 1 2 3 4
9 9 9 9 9 9 9 9 9 9 9 9
BoschLtd. ϰϲϴ͘ϳϭ 891.22 718.78 2059.9 2584.2 1666.70 1688.8 1221.42 1222.0 1098.6 759.09 1376.52
9 9 9
Dr.Reddy ϭϮϴϯ͘ϱ 533.13 226.17
9 9 9 9 9 9 9 9 9 9 9 9
Eicher Motors ϰϮϱ͘ϳϭ 536.87 1326.6 1333.8 530.09 1649.35 1217.0 1017.05 1213.7 666.37 1695.3 1878.86
9 9 9 9 9 9 9 9 9 9 9 9
SBIN ϯϲϴϱ͘ϰ 3088.8 3281.8 3858.3 4452.5 4274.88 4439.4 4587.75 4950.5 5225.3 5108.7 5263.45
9 9 9 9 9
HDFC 
531.087 484.88 498.90 607.24 569.393
9
9 9 9 9 9 9 9 9 9 9 9
NTPC ϰϰϱ͘ϴϳ 474.75 506.01 531.17 504.08 536.24
551.38
485.59 483.71 527.75 552.48 575.152
8

2017 International Conference On Smart Technology for Smart Nation 965

HDFC and NTPC can be considered for long time investment Recognition, Informatics and Mobile Engineering (PRIME), 2013
as they show fair market presence. Thus,Digging deep gave us International Conference on. IEEE, 2013.
the best insight of which company is best for investment from [8] Patel, Hiral, and Satyen Parikh. "Comparative analysis of different
statistical and neural network based forecasting tools for prediction of
various aspects and viewpoints of the trader. The Results of stock data." Proceedings of the Second International Conference on
this analysis can be used for less than a year from the date of Information and Communication Technology for Competitive Strategies.
analysis depending on the index performance. ACM, 2016.
[9] Rather, Akhter Mohiuddin, Arun Agarwal, and V. N. Sastry. "Recurrent
V. CONCLUSIONS neural network and a hybrid model for prediction of stock
returns." Expert Systems with Applications 42.6 (2015): 3234-3241.
The ideal approach for a beginner as well as a financial [10] Sagiroglu, Seref, and Duygu Sinanc. "Big data: A
novice is to invest in the best-performing stocks that are less review." Collaboration Technologies and Systems (CTS), 2013
affected by the volatile nature of the market. This minimises International Conference on. IEEE, 2013.
the probability of loss. The methodology used in this paper is [11] Dede, Elif, et al. "Performance evaluation of a mongodb and hadoop
easy and ideal for all the existing indexes and would yield platform for scientific data analysis." Proceedings of the 4th ACM
workshop on Scientific cloud computing. ACM, 2013.
desired top stocks listed under those indexes. The ratio of
standard deviation of resultant stocks to their index would be [12] Dubey, Arun Kumar, Vanita Jain, and A. P. Mittal. "Stock market
prediction using Hadoop Map-Reduce ecosystem." Computing for
significant. It reflects their high performance and will have Sustainable Global Development (INDIACom), 2015 2nd International
high market capitalization, strong cash flows, with consistent Conference on. IEEE, 2015.
growth, high P/E ratios and would be less volatile in nature as [13] Fuad, Ammar, Alva Erwin, and Heru Purnomo Ipung. "Processing
compared to other stocks in that particular index. Therefore performance on Apache Pig, Apache Hive and MySQL
they are considered safe for investment and advisable for the cluster." Information, Communication Technology and System (ICTS),
conservative class of investors. Our approach provides the 2014 International Conference on. IEEE, 2014.
latest insight to develop an ideal investment portfolio in stock [14] Kavitha, S., Raja Vadhana, and A. N. Nivi. "BIG DATA ANALYTICS
IN FINANCIAL MARKET." International Journal of Research in
markets with a time-tested strategy towards successful Engineering and Technology , vol. 4, no. 2, 2015.
investment. [15] Xie, Yonghong, et al. "Implementation of time series data clustering
based on SVD for stock data analysis on hadoop platform." Industrial
Electronics and Applications (ICIEA), 2014 IEEE 9th Conference on.
IEEE, 2014.
REFERENCES
[16] "NSE - National Stock Exchange Of India Ltd.". Nseindia.com. N.p.,
2017. Web. 1 May 2017.
[1] Kumar, Hemanth, and S. Basavaraj Patil. "Estimation & forecasting of [17] Colin Bennett and Miguel A. Gil, “Measuring Historical Volatility”,
volatility using ARIMA, ARFIMA and Neural Network based 2012.
techniques." Advance Computing Conference (IACC), 2015 IEEE
International. IEEE, 2015.
[2] Terzis, John, et al. "Financial Market Volatility.", Columbia University ,
Big Data Analytics , 2014.
[3] Raghunathan, Srinath. "Volatility in Indian stock market." Asian Journal
of Research in Business Economics and Management 5.2 (2015): 298-
311.
[4] Sadorsky, Perry. "Modeling volatility and correlations between
emerging market stock prices and the prices of copper, oil and
wheat." Energy Economics 43 (2014): 72-81.
[5] Vozlyublennaia, Nadia. "Investor attention, index performance, and
return predictability." Journal of Banking & Finance 41 (2014): 17-35.
[6] Abinaya, P., et al. "Measuring stock price and trading volume causality
among Nifty50 stocks: The Toda Yamamoto method." Advances in
Computing, Communications and Informatics (ICACCI), 2016
International Conference on. IEEE, 2016.
[7] Kumar, D. Ashok, and S. Murugan. "Performance analysis of Indian
stock market index using neural network time series model." Pattern

966 2017 International Conference On Smart Technology for Smart Nation