BITS F312 Neural Networks

MID-SEM PRESENTATION Presented by -

Ayaan Aryan
Stock prediction using ANFIS and Harsh Vyas
LSTM and comparison Sarvesh Muttepawar
BITS Pilani
Hyderabad Campus
Introduction

Our project initially employed Long Short-Term Memory (LSTM) models, favored for their ability to capture long-term
dependencies in time series data. However, despite their strengths, LSTMs often require extensive computational
resources and pose challenges in terms of interpretability, which is crucial for adjusting strategies based on model
feedback.

Recognizing these limitations, we transitioned to Adaptive Neuro-Fuzzy Inference System (ANFIS) models. This shift
was motivated by several key benefits that ANFIS offers over LSTM:

● Interpretability: ANFIS integrates neural network learning with fuzzy logic to create a system where the reasoning
behind each prediction can be understood and explained through fuzzy rules. This transparency is invaluable for
financial modeling where stakeholders require clear explanations of predictive behaviors.
● Computational Efficiency: Unlike the deep architecture of LSTMs, ANFIS provides a more streamlined framework
that requires fewer parameters to train, leading to faster and more efficient computation without compromising on
performance.
● Adaptability and Precision: ANFIS models are highly adaptable, allowing for precise tuning of membership
functions to better model the complexities and volatilities of financial markets. This adaptability ensures that the
model remains sensitive to subtle nuances in market dynamics.
● Robust Handling of Uncertainty: The fuzzy logic component of ANFIS makes it inherently better at handling the
uncertainty and noise typical in financial data, providing more stable and reliable predictions.

BITS Pilani, Hyderabad Campus

Objectives

Model Development:

● To develop an ANFIS model that integrates both neural network adaptability and fuzzy logic qualitative analysis.
● To configure and tune this model specifically for the complexities and dynamics of stock market data.

Feature Utilization:

● To effectively use historical stock data features—including open, high, low, along with trading volume—to predict stock
closing prices.
● To explore the impact of different membership functions and rule bases on the prediction accuracy.

Prediction Accuracy:

● To enhance prediction accuracy by fine-tuning the model parameters through advanced hybrid learning techniques
combining gradient descent and least squares methods.
● To compare the performance of the ANFIS model against traditional time series forecasting models to validate its
efficacy.

Real-World Application:

● To provide a reliable tool for investors and financial analysts to forecast stock prices, aiding in better decision-making.
● To establish a foundation for real-time stock price prediction, contributing to more informed and strategic trading
practices.
BITS Pilani, Hyderabad Campus
 Work Done Until Mid-Semester

Data Preparation and Initial Modeling

● Data Acquisition: Collected five years of historical data from the S&P 500 index, focusing on daily open, high, low, and
close values.
● Preprocessing:
○ Cleaning: Addressed missing values and outliers to enhance data integrity and model performance.
○ Normalization: Implemented Min-Max Scaling to ensure equal contribution of features during model training.
○ Windowing Technique: Organized data into overlapping windows to maintain sequential integrity, crucial for LSTM
processing.

Fuzzy Clustering

● Implementation: Employed Fuzzy C-Means (FCM) clustering to segment stock data into distinct market states, enhancing
model input through richer feature sets reflective of underlying market behaviors.

LSTM Modeling

● Architecture: Developed a multi-layer LSTM network to leverage temporal sequences enhanced by fuzzy clustering for
predicting future stock prices.
● Training:
○ Used historical data sequences to train the LSTM model, with Mean Squared Error as the loss function optimized
using the Adam optimizer.

BITS Pilani, Hyderabad Campus

Methodology and Mathematical Formulation

What is ANFIS?
ANFIS stands for Adaptive Neuro-Fuzzy Inference System, a technique that combines the human-like
reasoning style of fuzzy logic with the learning power of neural networks to model complex processes.

ANFIS Architecture

Layer 1 (Fuzzification): Inputs are fuzzified using membership functions. Parameters of these functions
are adjustable and learned during training. We have use Gaussian Functions.

Layer 2 (Rules): Each node in this layer represents a fuzzy rule. The output is the product of all incoming
signals (fuzzy AND operation).

Layer 3 (Normalization): Converts the fuzzy rule outputs into a ratio of the firing strength of a rule to the
sum of all rule strengths.

Layer 4 (Defuzzification): Outputs a single number from each rule’s firing strength, using parameterized
functions.

Layer 5 (Total Output): The final output is the summation of all incoming signals from the previous layer,
which forms the predictive result.

BITS Pilani, Hyderabad Campus

Methodology and Mathematical Formulation

ANFIS Architecture

BITS Pilani, Hyderabad Campus

Methodology and Mathematical Formulation

Layer-by-Layer Mathematical Description of ANFIS

Layer 1: Fuzzification
In the fuzzification layer, each input variable xi​is converted into fuzzy membership values using predefined membership
functions. Commonly used membership functions include Gaussian, generalized bell, and triangular

Layer 2: Application of Fuzzy Rules

This layer multiplies the incoming signals to apply the fuzzy AND operation (also called the product inference rule), producing
a firing strength for each rule

BITS Pilani, Hyderabad Campus

Methodology and Mathematical Formulation

Layer 3: Normalization of Firing Strengths

Each rule's firing strength is normalized to sum to one across all rules:

Where:

● ​ is the normalized firing strength,

● m is the total number of rules.

Layer 4: Defuzzification

Each rule has a function associated with it, typically a linear combination of the inputs,
though non-linear functions can also be used. This layer computes the output contribution
of each rule:

Weighted Contribution of Each Rule:

● The crisp output of each rule is multiplied by its normalized firing strength to
compute its contribution to the final output:

BITS Pilani, Hyderabad Campus

Methodology and Mathematical Formulation

Layer 5: Overall Output

The final output of the ANFIS is the summation of all rule outputs, weighted by their normalized firing
strengths:

BITS Pilani, Hyderabad Campus

Overview of Prediction Model

FIS Models
1. Grid Partitioning:

○ Divides the input space into a grid where each division corresponds to a rule. This method is
straightforward and effective for clear-cut and well-defined data distributions.
1. Subtractive Clustering:
○ Determines the number of rules and membership functions based on the density of data points,
making it suitable for data with overlapping clusters or without a clear separation.
2. FCM (Fuzzy C-Means Clustering):
○ Clusters data by minimizing the weighted within-group sum of squared errors, allowing each data
point to belong to multiple clusters to a varying degree.

Ensemble Approach:

● Combines multiple ANFIS models to enhance predictive performance.

● Ensemble models are built by averaging the outputs of the individual tuned ANFIS models to improve
accuracy and reduce variance.

BITS Pilani, Hyderabad Campus

Model Implementation Details

Building FIS models

Grid Partitioning Method:

● Divides the input variable space into a fine grid, creating a comprehensive framework of rules that
cover all possible scenarios in the dataset.

BITS Pilani, Hyderabad Campus

Model Implementation Details

Subtractive Clustering Technique:

● A data-driven method that assumes each data point as a potential cluster center and calculates the likelihood of each point to
serve as the centroid based on the density of surrounding points. Quickly identifies the number of rules and their parameters
directly from data.

BITS Pilani, Hyderabad Campus

Model Implementation Details

FCM Clustering Process:

● Fuzzy C-Means (FCM) clustering allows each piece of data to belong to multiple clusters with varying degrees of
membership, which are determined by minimizing the distance from any given data point to a cluster center weighted by
the membership degree.

BITS Pilani, Hyderabad Campus

Model Implementation Details`

Ensemble Model Construction Rationale

Concept of Ensemble Modeling:

● Combines predictions from multiple models to improve the accuracy and robustness of predictions by
reducing variance and avoiding overfitting.

Implementation in ANFIS:

● Multiple ANFIS models based on different clustering techniques are trained independently.
● The final prediction is an average of predictions from these models, balancing out individual model
biases.

Advantages:

● Robustness: Reduces the impact of model-specific anomalies and errors.

● Improved Accuracy: Averages out errors across different models, often leading to better overall
performance.

BITS Pilani, Hyderabad Campus

Parameter Tuning and Optimization

Importance of Parameter Tuning

Need for Parameter Tuning:

● Parameter tuning is crucial in modeling to optimize the performance of predictive models, particularly in
complex systems like ANFIS where the interaction of parameters significantly impacts the output.

Benefits of Tuning:

● Accuracy Improvement: Fine-tuning parameters can enhance model precision and reduce prediction
errors.
● Adaptability: Allows models to better adapt to new, unseen data by optimizing the generalization
capabilities of the model.

Approaches Used:

● Use of algorithms like backpropagation and least squares for adjusting membership function parameters
to minimize prediction errors

BITS Pilani, Hyderabad Campus

Parameter Tuning and Optimization

Techniques Used for Tuning

Tuning Techniques:

● Backpropagation: Used primarily for training deep learning models; adjusts the weights of the network by propagating errors back
through the system.

● Least Squares Method: Applies a mathematical approach to find the best fit that minimizes the sum of the squared differences
between observed and predicted values.

Implementation in ANFIS:

● These techniques help in fine-tuning the membership functions' parameters within the fuzzy inference system, critical for accurate
predictions.

Results of Effective Tuning:

● Enhanced Model Performance: Improved metrics such as higher R-squared values and lower MSE and RMSE.

● Reliability: Increases confidence in the model’s predictive power on diverse datasets.

BITS Pilani, Hyderabad Campus

Evaluation Metrics and Performance

R-squared (R^2):

● Measures the proportion of variance in the target variable explained by the model.
● Higher R^2 indicates better model performance (closer to 1 is ideal).
● Helps evaluate how well the model generalizes.

Mean Squared Error (MSE):

● The average squared difference between actual and predicted values.

● Sensitive to large errors, making it useful for identifying outliers.
● A lower MSE indicates better predictive accuracy.

Root Mean Squared Error (RMSE):

● The square root of MSE, expressed in the same units as the target variable.
● Provides an intuitive measure of the average prediction error.
● Lower RMSE reflects more accurate predictions.

Mean Error (Bias) & Standard Deviation of Errors (Error STD):

● Mean Error: Measures systematic bias (overprediction or underprediction).

● Error STD: Indicates consistency of errors; lower values mean more stable predictions.

BITS Pilani, Hyderabad Campus

Evaluation Metrics and Performance

Training dataset and associated errors

BITS Pilani, Hyderabad Campus

Evaluation Metrics and Performance

BITS Pilani, Hyderabad Campus

 Evaluation Metrics and Performance

Performance Highlights

● Daily Models: The untuned daily models (anfis_1_d to anfis_3_d) consistently show high R² values, demonstrating robust
predictive capabilities. Among these, anfis_3_d shows the highest R² (0.9301) and the lowest RMSE (19.3597), highlighting its
superior accuracy and consistency.
● Tuned Daily Models: Similar to their untuned counterparts, the tuned daily models (anfis_1_d (tuned) to anfis_3_d (tuned))
maintain high R² and low error metrics, with minimal improvement in model metrics, suggesting that tuning has refined model
performance slightly without substantial gains.
● Weekly Models: Both tuned and untuned weekly models exhibit significantly lower R² values compared to daily models, with
anfis_2_w and anfis_2_w (tuned) performing best among the weekly setups. This indicates that daily data provides a more
reliable basis for predictions in this context.
● Ensemble Models: The ensemble models (ens_anfis_tuned and ens_anfis_daily (not tuned)) leverage strengths from multiple
models. The ensemble of daily models not tuned (ens_anfis_daily (not tuned)) particularly stands out with an R² of 0.9219 and
the lowest RMSE (20.4715) among all models, validating the effectiveness of combining predictions to enhance accuracy and
reduce error variability.

Model Comparison and Insights

● Effectiveness of Tuning: Tuning generally enhances the model's performance slightly, as indicated by the marginal improvements
in MSE and RMSE. However, the impact of tuning varies, suggesting a need for customized tuning strategies based on specific
model characteristics and data traits.
● Daily vs. Weekly Data: Models trained on daily data consistently outperform those trained on weekly data, which might be due to
the higher granularity and more information available in daily data, capturing more subtle market dynamics.
● Ensemble Advantage: The ensemble approach proves to be highly effective, particularly when combining forecasts from daily
models. This strategy reduces prediction errors and yields more stable and reliable results than individual models.

BITS Pilani, Hyderabad Campus

Evaluation Metrics and Performance

No single metric fully captures a model's performance. Using a combination of metrics provides a
holistic view:

● R-squared evaluates how well the model explains variance in the data.
● MSE and RMSE quantify overall error magnitude.
● Mean Error highlights systematic bias in predictions.
● Error STD assesses prediction consistency and reliability.

These metrics collectively help select and fine-tune the best-performing model for accurate and robust
predictions.

BITS Pilani, Hyderabad Campus

Evaluation Metrics and Performance

Performance of Individual Models

Individual Model Analysis:

● Grid Partitioning Model: Shows specific strengths in certain market conditions due to its structured rule
system.
● Subtractive Clustering Model: Offers flexible adaptation to data with less clear patterns, often showing
improved predictive reliability.
● FCM Clustering Model: Best for complex datasets where data points can belong to multiple categories with
varying degrees.

Test output of best model : FCM Clustering of daily data (not tuned)
BITS Pilani, Hyderabad Campus
Evaluation Metrics and Performance

Performance of Ensemble Models

Ensemble Model Strategy:

● Combines forecasts from multiple models to improve accuracy and robustness.

● Typically, ensemble models average out the biases of individual models, reducing variance.

Test output of best ensemble model : Average of 3 daily models (not tuned)
BITS Pilani, Hyderabad Campus
Discussion of Results

Performance Highlights:
● Improved Accuracy:
○ Ensemble models often outperform individual models by leveraging diverse prediction approaches. In this case, the
ensemble model achieves higher R^2, indicating that it explains more variance in the data compared to any single model.
● Reduced Errors:
○ Metrics such as Mean Squared Error (MSE) and Root Mean Squared Error (RMSE) are consistently lower for ensemble
models, reflecting more precise and reliable predictions.
● Case Example:
○ For daily data, the ensemble model achieves an R2R^2R2 of 0.9219 and an RMSE of 20.47, demonstrating significant
improvement over most individual ANFIS models.
○ For tuned weekly data, the ensemble model also reduces errors significantly, showcasing the robustness of the combined
strategy.

Why Ensembles Work

Ensemble models work by:

1. Balancing Errors: Aggregating outputs from models with different biases neutralizes extreme errors.
2. Reducing Variance: The variability of predictions is reduced, leading to more consistent results.
3. Enhancing Generalization: The combination of multiple models improves the ability to adapt to unseen data.

BITS Pilani, Hyderabad Campus

Discussion of Results

BITS Pilani, Hyderabad Campus

Discussion of Results

This project has conducted a thorough comparative analysis of two advanced modeling techniques for stock market prediction: Adaptive Neuro-
Fuzzy Inference System (ANFIS) with Fuzzy C-Means Feature Integration (FCM-FIS) and Recurrent Neural Networks with Long Short-Term
Memory (RNN-LSTM). Below are the key findings from this comparison:

1. Interpretability

● ANFIS with FCM-FIS: Offers high interpretability as the fuzzy rules used in the model are explicit and comprehensible, allowing users to
understand the reasoning behind the predictions.
● RNN-LSTM: Typically considered a black-box model, offering low interpretability with limited insight into the internal decision-making
processes.

2. Handling Temporal Dependencies

● ANFIS with FCM-FIS: This model exhibits limitations in capturing long-term temporal patterns effectively due to the static nature of fuzzy
rules.
● RNN-LSTM: Excellently captures long-term dependencies inherent in sequential data, which is a fundamental strength of LSTM
architecture.

3. Performance in Noisy Environments

● ANFIS with FCM-FIS: Shows moderate performance when dealing with noisy data, as the fuzzy logic can sometimes smooth over too
much detail.
● RNN-LSTM: High resilience to noise and ability to process complex sequences effectively, making it particularly useful in volatile
environments.
BITS Pilani, Hyderabad Campus
Discussion of Results

4. Scalability

● ANFIS with FCM-FIS: Scalability can be limited by the number of input dimensions, as each additional feature can exponentially
increase the complexity of the fuzzy rule set.
● RNN-LSTM: Highly scalable, capable of handling large datasets and multiple input features without a significant loss of
performance.

5. Training Complexity

● ANFIS with FCM-FIS: Generally requires lower training complexity due to its simpler architecture and fewer parameters.
● RNN-LSTM: Involves a more complex training process with significant tuning and optimization of numerous hyperparameters to
achieve optimal performance.

6. Expected R-Squared

● ANFIS with FCM-FIS: Typically achieves an R-squared between 0.90 and 0.93 in structured environments, indicating high model
accuracy.
● RNN-LSTM: Can reach an R-squared between 0.93 and 0.96, especially in scenarios involving noisy or complex datasets.

Conclusion

The choice between ANFIS with FCM-FIS and RNN-LSTM should be guided by the specific requirements of the application, including the
need for interpretability, handling of temporal dependencies, and the nature of the data environment. While ANFIS offers a more
interpretable and less computationally intensive option, RNN-LSTM stands out in environments where long-term dependencies and noise
are prevalent. This comparative analysis highlights the trade-offs between these models, providing a foundation for selecting the
appropriate approach based on the predictive task at hand.

BITS Pilani, Hyderabad Campus

BITS Pilani, Hyderabad Campus