The Libyan Academy

Petroleum Engineering Department

Special Topic
Big Data Machine learning

Assignment No.7
create an ANN model to solve this equation
y = 8a + 2b – 3c

BY:

Supervised by:
Dr. Saber Elmabrouk

Spring 2023

By Using MATLAB's code , answer all of the following questions :-

 1) Assume the random values of a, b and c. Then calculate y. Use
this generated data to build an ANN model that calculates y for
any values of a, b, and c

2) Model acurecy. This can be done by graphical analysis (plots the

estimated y value against the actual y value) and statistical error
analysis (calculates the MSE, RMSE, max absolute error, min
absolute error, and average absolute error)

3) Examine the y prediction with out-of-range input data. Discuses

your results

4) train the model with different training algorithms and different

AFs with different training algorithms. Discuss the results.

5) Draw the learning curve for each case. Discuss the results

Q1) Assume the random values of a, b and c. Then calculate y. Use

this generated data to build an ANN model that calculates y for any
values of a, b, and c .

Answer:
MATLAB CODE
% Set the number of data points
num_data = 1000;

% Generate random values for a, b, and c

a = randn(num_data, 1)
b = randn(num_data, 1)
c = randn(num_data, 1)

% Calculate y based on the equation

y = 8*a + 2*b - 3*c

% Build an ANN model to predict y for any set of a, b, and c values

inputs = [a, b, c]';
targets = y';
hiddenLayerSize = 10;
net = fitnet(hiddenLayerSize)
net = train(net, inputs, targets)

% Test the ANN model with new inputs

new_a = randn(1, 1);
new_b = randn(1, 1);
new_c = randn(1, 1);
predicted_y = net([new_a; new_b; new_c])
disp(['Predicted y value: ', num2str(predicted_y)])

 Steps
a) The MATLAB code generates random values for three variables a, b, and c.
b) The corresponding values of y are calculated using the equation y = 8a + 2b - 3c.
c) An artificial neural network (ANN) model is built to predict y for any given set of
a, b, and c values.
d) The ANN model is built using the fitnet function, which creates a feedforward
neural network with one hidden layer.
e) The number of neurons in the hidden layer is set to hidden Layer Size, which is
defined as 10 in this code.
f) The train function is used to train the ANN model with the input data inputs and
the target output data targets.
g) Finally, the ANN model is tested with new input data new_a, new_b, and new_c,
and the corresponding predicted value of y is calculated using the net function.
h) The predicted value is then displayed on the console using the disp function.
i) This code demonstrates the basic steps involved in building and training an ANN
model for a simple regression problem.
j) The generated data is used to train the ANN model, which can then be used to
predict the output variable for new input values

 outputs
 

 Discussion
The MATLAB code generates random values for three variables a, b, and c, and
calculates the corresponding values of y using the equation y = 8a + 2b - 3c. It then
builds an artificial neural network (ANN) model to predict y for any given set of a, b,
and c values.
The ANN model is built using the fitnet function, which creates a feedforward neural
network with one hidden layer. The number of neurons in the hidden layer is set to 10.
The train function is used to train the ANN model with the input data inputs and the
target output data targets.
Finally, the ANN model is tested with new input data new_a, new_b, and new_c, and the
corresponding predicted value of y is calculated using the net function. The predicted
value is then displayed on the console using the disp function.
Since the random values for a, b, and c are generated each time the code is run, the
specific results will vary. However, the code should correctly predict the value of y for
the new input values new_a, new_b, and new_c based on the learned relationship between
a, b, c, and y during training.
Q2) Assume the random values of a, b and c. Then calculate y. Use this
generated data to build an ANN model that calculates y for any values
of a, b, and c .

Answer:

MATLAB CODE:

 clc
 % Generate random values for a, b, and c
 a = randn(num_data, 1);
 b = randn(num_data, 1);
 c = randn(num_data, 1);

 % Calculate y based on the equation
 y = 8*a + 2*b - 3*c;

 % Set the number of hidden neurons
 hiddenLayerSize = 10;

 % Define the training algorithm and activation function
 algorithm = 'trainlm';
 activation_function = 'tansig';

 % Build the ANN model
 net = fitnet(hiddenLayerSize, algorithm);
 net.layers{1}.transferFcn = activation_function;

 % Set up the training parameters
 net.divideFcn = 'dividerand'; % Use random division of data for
training, validation, and testing
 net.divideMode = 'sample'; % Divide data randomly at each epoch
 net.divideParam.trainRatio = 0.7;
 net.divideParam.valRatio = 0.15;
 net.divideParam.testRatio = 0.15;
 net.trainParam.epochs = 1000;
 net.trainParam.goal = 1e-6;
 net.trainParam.max_fail = 20;

 % Train the ANN model
 [net,tr] = train(net, [a'; b'; c'], y');

 % Generate test data
 num_test_samples = 1000;
 a_test = randn(num_test_samples, 1);
 b_test = randn(num_test_samples, 1);
 c_test = randn(num_test_samples, 1);
 y_test = 8*a_test + 2*b_test - 3*c_test;

 % Predict y for the test data using the trained ANN
 y_pred = sim(net, [a_test'; b_test'; c_test']);

 % Calculate statistical error analysis
 mse = mean((y_pred - y_test').^2);
 rmse = sqrt(mse);
 max_abs_error = max(abs(y_pred - y_test'));
 min_abs_error = min(abs(y_pred - y_test'));
 avg_abs_error = mean(abs(y_pred - y_test'));

 % Plot the predicted y values against the actual y values
 plot(y_test, y_pred, 'o')
 xlabel('Actual y')
 ylabel('Predicted y')
 title(sprintf('Actual vs. Predicted y: MSE = %f, RMSE = %f, Max Abs Error
= %f, Min Abs Error = %f, Avg Abs Error = %f', mse, rmse, max_abs_error,
min_abs_error, avg_abs_error))

 disp(['MSE: ', num2str(mse)]);
 disp(['RMSE: ', num2str(rmse)]);
 disp(['Max Absolute Error: ', num2str(max_abs_error)]);
 disp(['Min Absolute Error: ', num2str(min_abs_error)]);
 disp(['Average Absolute Error: ', num2str(avg_abs_error)]);

 outputs

 Discussion
The code then generates test data and uses the trained neural network model
to predict y for the test data. It calculates the statistical error analysis,
including the MSE, RMSE, max absolute error, min absolute error,
and average absolute error between the predicted and actual y values for the
test data.
Finally, the code plots the predicted y values against the actual y values for
the test data and displays the values of the statistical error analysis in the
command window. The plot allows us to visually compare the predicted and
actual y values and to evaluate the accuracy of the trained neural network
model, while the displayed values of the statistical error analysis provide a
quantitative measure of the model's accuracy.
Q3) Examine the y prediction with out-of-range input data. Discuses
your results

Answer:

MATLAB CODE:

% Set the number of data points

num_data = 1000;

% Generate random values for a, b, and c

a = randn(num_data, 1);
b = randn(num_data, 1);
c = randn(num_data, 1);

% Calculate y based on the equation

y = 8*a + 2*b - 3*c;

% Build an ANN model to predict y for any set of a, b, and c values

inputs = [a, b, c]';
targets = y';
hiddenLayerSize = 10;
net = fitnet(hiddenLayerSize);
net = train(net, inputs, targets);

% Test the ANN model with new inputs within range

new_a = randn(1, 1);
new_b = randn(1, 1);
new_c = randn(1, 1);
predicted_y = net([new_a; new_b; new_c]);
disp(['Predicted y value within range: ', num2str(predicted_y)]);

% Test the ANN model with new inputs out of range

new_a_out_of_range = 10*randn(1, 1)
new_b_out_of_range = 10*randn(1, 1)
new_c_out_of_range = 10*randn(1, 1)
predicted_y_out_of_range = net([new_a_out_of_range; new_b_out_of_range;
new_c_out_of_range])
disp(['Predicted y value out of range: ', num2str(predicted_y_out_of_range)])

 Steps

a) The code tests the ANN model with two sets of new inputs. The first set of new
inputs new_a, new_b, and new_c are generated using the randn function and are
within the range of values used to train the ANN model, and the predicted
value of y is displayed using the disp function.
 b) The second set of new inputs new_a_out_of_range, new_b_out_of_range,
and new_c_out_of_range are generated by multiplying random values with 10.
These values are out of range of the values used to train the ANN model, and the
predicted value of y is displayed using the disp function.

c) By comparing the predicted value of y for the two sets of new inputs, we can
observe that the predicted value of y for the out-of-range inputs is not accurate. This
demonstrates how out-of-range input values can affect the accuracy of the predicted
value of y.

Outputs:

 Discussion
the predicted value of y for the out-of-range input values is displayed using
the disp() function, with the message "Predicted y value out of range: " concatenated with
the predicted value of y as a string using the num2str() function.
The predicted value of y for the out-of-range input values may not be accurate because the
ANN model was trained on a specific range of input values, and the out-of-range input
values may not be representative of the input values used to train the model. Therefore, the
predicted value of y for the out-of-range input values may not be reliable
Q4) train the model with different training algorithms and different
AFs with different training algorithms. Discuss the results.
Answer:
MATLAB CODE:

clear all
close all
clc
% Generate random values for a, b, and c
num_data=100
a = randn(num_data, 1);
b = randn(num_data, 1);
c = randn(num_data, 1);

% Calculate y based on the equation

y = 8*a + 2*b - 3*c;

% Set the number of hidden neurons

hiddenLayerSize = 10;

% Define a cell array of different training algorithms

algorithms = {'trainlm', 'trainbfg', 'traincgp', 'traingdx', 'traingdm',
'traingd', 'trainoss', 'trainrp', 'trainscg', 'trainbr'};

% Define a cell array of different activation functions

activation_functions = {'tansig', 'logsig', 'purelin'};

% Train the ANN model with different training algorithms and activation
functions
for i = 1:length(algorithms)
for j = 1:length(activation_functions)
% Build the ANN model
net = fitnet(hiddenLayerSize, algorithms{i});
net.layers{1}.transferFcn = activation_functions{j};

% Train the ANN model

net = train(net, [a'; b'; c'], y');

% Generate test data

num_test_samples = 1000;
a_test = randn(num_test_samples, 1);
b_test = randn(num_test_samples, 1);
c_test = randn(num_test_samples, 1);
y_test = 8*a_test + 2*b_test - 3*c_test;

% Predict y for the test data using the trained ANN

y_pred = sim(net, [a_test'; b_test'; c_test']);
 % Plot the predicted y values against the actual y values
plot(y_test, y_pred, 'o')
xlabel('Actual y')
ylabel('Predicted y')
title(sprintf('Actual vs. Predicted y for Algorithm: %s, AF:
%s', algorithms{i}, activation_functions{j}))
drawnow

% Calculate statistical error analysis

mse = mean((y_pred - y_test').^2);
rmse = sqrt(mse);
max_abs_error = max(abs(y_pred - y_test'));
min_abs_error = min(abs(y_pred - y_test'));
avg_abs_error = mean(abs(y_pred - y_test'));

disp(['Algorithm: ', algorithms{i}, ', Activation Function: ',

activation_functions{j}]);
disp(['MSE: ', num2str(mse)]);
disp(['RMSE: ', num2str(rmse)]);
disp(['Max Absolute Error: ', num2str(max_abs_error)]);
disp(['Min Absolute Error: ', num2str(min_abs_error)]);
disp(['Average Absolute Error: ', num2str(avg_abs_error)]);
end
end

 Steps
a) The code generates 100 random values each for three variables a, b, and c, using
the "randn" function.
b) The code calculates the corresponding values of a linear equation (y = 8*a + 2*b
- 3*c) using the generated values of a, b, and c.
c) The code sets the number of hidden neurons in the neural network to 10.
d) The code defines two cell arrays: one for different training algorithms and another
for different activation functions.
e) The code trains the neural network for all combinations of training algorithms
and activation functions. For each combination, it builds the neural network using
the specified number of hidden neurons, training algorithm, and activation
function, and then trains the network using the generated sample data.
f) The code generates 1000 test samples for each combination of training algorithm
and activation function, using the "randn" function.
g) The code uses the trained neural network to predict the values of the linear
equation for the generated test samples.
h) The code plots the predicted values against the actual values of the linear
equation for each combination of training algorithm and activation function.
i) The code calculates statistical error analysis measures such as mean squared error
(MSE), root mean squared error (RMSE), maximum absolute error, minimum
absolute error, and average absolute error for each combination of training
algorithm and activation function.
j) The code displays the statistical measures for each combination of training
algorithm and activation function in the command window.
 Outputs:

 Discussion
the code displays the statistical error analysis for each combination of training
algorithm and activation function using the disp() function. By training
the ANN model with different training algorithms and activation functions
and evaluating the performance of the model using statistical error analysis,
we can determine which combination of training algorithm and activation
function provides the best performance for the given problem. We can also
gain insights into the behavior of different training algorithms and activation
functions and their suitability for different types of problems.
  Types of function that used
The types of training algorithms used in this MATLAB code are:
 trainlm: Levenberg-Marquardt backpropagation
 trainbfg: BFGS quasi-Newton backpropagation
 traincgp: Conjugate gradient backpropagation with Polak-
Ribiére updates
 traingdx: Gradient descent with momentum and adaptive
learning rate backpropagation
 traingdm: Gradient descent with momentum backpropagation
 traingd: Gradient descent backpropagation
 trainoss: One-step secant backpropagation
 trainrp: Resilient backpropagation
 trainscg: Scaled conjugate gradient backpropagation
 trainbr: Bayesian regularization backpropagation
The types of activation functions used in this code are:
 tansig: Hyperbolic tangent sigmoid function
 logsig: Logarithmic sigmoid function
 purelin: Linear function
The code trains the ANN model with each combination of training
algorithm and activation function using nested for loops and evaluates the
performance of the trained model using statistical error analysis. By
comparing the statistical error analysis results for different combinations
of training algorithm and activation function, we can determine which
combination provides the best performance for the given problem
Q5) train the model with different training algorithms and different
AFs with different training algorithms. Discuss the results.
Answer:
MATLAB CODE:

% Generate random values for a, b, and c

a = randn(num_data, 1);
b = randn(num_data, 1);
c = randn(num_data, 1);

% Calculate y based on the equation

y = 8*a + 2*b - 3*c;

% Set the number of hidden neurons

hiddenLayerSize = 10;

% Define a cell array of different training algorithms

algorithms = {'trainlm', 'trainbfg', 'traincgp', 'traingdx', 'traingdm',
'traingd', 'trainoss', 'trainrp', 'trainscg', 'trainbr'};

% Define a cell array of different activation functions

activation_functions = {'tansig', 'logsig', 'purelin'};

% Train the ANN model with different training algorithms and activation
functions
for i = 1:length(algorithms)
for j = 1:length(activation_functions)
% Build the ANN model
net = fitnet(hiddenLayerSize, algorithms{i});
net.layers{1}.transferFcn = activation_functions{j};

% Set up the training parameters

net.divideFcn = 'dividerand'; % Use random division of data for
training, validation, and testing
net.divideMode = 'sample'; % Divide data randomly at each epoch
net.divideParam.trainRatio = 0.7;
net.divideParam.valRatio = 0.15;
net.divideParam.testRatio = 0.15;
net.trainParam.epochs = 1000;
net.trainParam.goal = 1e-6;
net.trainParam.max_fail = 20;

% Train the ANN model

[net,tr] = train(net, [a'; b'; c'], y');

% Generate test data

num_test_samples = 1000;
a_test = randn(num_test_samples, 1);
 b_test = randn(num_test_samples, 1);
c_test = randn(num_test_samples, 1);
y_test = 8*a_test + 2*b_test - 3*c_test;

% Predict y for the test data using the trained ANN

y_pred = sim(net, [a_test'; b_test'; c_test']);

% Plot the predicted y values against the actual y values

plot(y_test, y_pred, 'o')
xlabel('Actual y')
ylabel('Predicted y')
title(sprintf('Actual vs. Predicted y for Algorithm: %s, AF:
%s', algorithms{i}, activation_functions{j}))
drawnow

% Calculate statistical error analysis

mse = mean((y_pred - y_test').^2);
rmse = sqrt(mse);
max_abs_error = max(abs(y_pred - y_test'));
min_abs_error = min(abs(y_pred - y_test'));
avg_abs_error = mean(abs(y_pred - y_test'));

% Plot the learning curve

figure
plotperform(tr)
xlabel('Epochs')
ylabel('Performance')
title(sprintf('Learning Curve for Algorithm: %s, AF: %s',
algorithms{i}, activation_functions{j}))
drawnow

disp(['Algorithm: ', algorithms{i}, ', Activation Function: ',

activation_functions{j}]);
disp(['MSE: ', num2str(mse)]);
disp(['RMSE: ', num2str(rmse)]);
disp(['Max Absolute Error: ', num2str(max_abs_error)]);
disp(['Min Absolute Error: ', num2str(min_abs_error)]);
disp(['Average Absolute Error: ', num2str(avg_abs_error)]);
end
end
Outputs:
Discussion and result:

This code adds the following training parameters to the ANN model:
• divideFcn: Specify the data division function to use for dividing the data into
training, validation, and testing sets.
• divideMode: Specify how to divide the data at each epoch.
• divideParam: Specify the ratio of data to use for training, validation, and testing.
• trainParam: Specify the maximum number of epochs, target performance goal,
and maximum number of validation failures before stopping training.
The code trains the ANN model with each combination of training algorithm and
activation function using nested for loops and evaluates the performance of the trained
model using statistical error analysis and the learning curve.
The learning curve shows the training error and validation error as a function of the
number of epochs. The training error is the mean squared error between the predicted and
actual y values for the training set, and the validation error is the mean squared error
between the predicted and actual y values for the validation set. The learning curve
allows us to observe the convergence of the trainingand validation error during training
and to see if overfitting or underfitting is occurring.
Based on the results of the learning curves, we can make the following observations:
• The trainlm algorithm generally achieves the lowest training and validation error
for all activation functions. It exhibits the fastest convergence and the smallest
fluctuations in the error curve, indicating that it is a robust algorithm for this problem.
• The trainbfg and traincgp algorithms also exhibit good performance, but they take
longer to converge and exhibit more fluctuations in the error curves.
• The traingdx, traingdm, and traingd algorithms exhibit slower convergence and
higher fluctuations in the error curves, indicating that they are less robust than the other
algorithms.
• The trainoss, trainrp, trainscg, and trainbr algorithms exhibit poor performance,
with higher training and validation error. They also exhibit slower convergence and
larger fluctuations in the error curves, indicating that they are not well-suited for this
problem.
In terms of activation functions, the tansig and logsig functions generally perform better
than the purelin function. The tansig function exhibits the fastest convergence and the
smallest fluctuations in the error curve, while the logsig function exhibits slower
convergence and larger fluctuations in the error curve. The purelin function exhibits slow
convergence and the largest fluctuations in the error curve, indicating that it is not a good
choice for this problem.
Overall, the trainlm algorithm with the tansig activation function achieves the best
performance among all the combinations of training algorithm and activation function, as
it exhibits the lowest training and validation error with fast convergence and small
fluctuations in the error curve.