dropout

May 30, 2024

IMPORTING LIBRARIES
[ ]: import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import warnings
warnings.filterwarnings('ignore')

LOADING DATASET
[ ]: df = pd.read_csv("/content/ADL_sonar.csv", header=None)
df.sample(5)

[ ]: 0 1 2 3 4 5 6 7 8 \
20 0.0473 0.0509 0.0819 0.1252 0.1783 0.3070 0.3008 0.2362 0.3830
29 0.0189 0.0308 0.0197 0.0622 0.0080 0.0789 0.1440 0.1451 0.1789
173 0.0329 0.0216 0.0386 0.0627 0.1158 0.1482 0.2054 0.1605 0.2532
43 0.0093 0.0269 0.0217 0.0339 0.0305 0.1172 0.1450 0.0638 0.0740
176 0.0635 0.0709 0.0453 0.0333 0.0185 0.1260 0.1015 0.1918 0.3362

9 … 51 52 53 54 55 56 57 \
20 0.3759 … 0.0193 0.0118 0.0064 0.0042 0.0054 0.0049 0.0082
29 0.2522 … 0.0038 0.0096 0.0142 0.0190 0.0140 0.0099 0.0092
173 0.2672 … 0.0095 0.0151 0.0059 0.0015 0.0053 0.0016 0.0042
43 0.1360 … 0.0212 0.0091 0.0056 0.0086 0.0092 0.0070 0.0116
176 0.3900 … 0.0048 0.0025 0.0087 0.0072 0.0095 0.0086 0.0085

58 59 60
20 0.0028 0.0027 R
29 0.0052 0.0075 R
173 0.0053 0.0074 M
43 0.0060 0.0110 R
176 0.0040 0.0051 M

[5 rows x 61 columns]

DATA PREPROCESSING

1
[ ]: df.shape
df.isna().sum()

[ ]: 0 0
1 0
2 0
3 0
4 0
..
56 0
57 0
58 0
59 0
60 0
Length: 61, dtype: int64

[ ]: df.columns
df[60].value_counts()

[ ]: 60
M 111
R 97
Name: count, dtype: int64

[ ]: X = df.drop(60, axis=1)
y = df[60]
y.head()

[ ]: 0 R
1 R
2 R
3 R
4 R
Name: 60, dtype: object

[ ]: y = pd.get_dummies(y, drop_first=True)
y.sample(5)

[ ]: R
98 False
109 False
163 False
46 True
141 False

[ ]: y.value_counts()

2
[ ]: R
False 111
True 97
Name: count, dtype: int64

[ ]: X.head()

[ ]: 0 1 2 3 4 5 6 7 8 \
0 0.0200 0.0371 0.0428 0.0207 0.0954 0.0986 0.1539 0.1601 0.3109
1 0.0453 0.0523 0.0843 0.0689 0.1183 0.2583 0.2156 0.3481 0.3337
2 0.0262 0.0582 0.1099 0.1083 0.0974 0.2280 0.2431 0.3771 0.5598
3 0.0100 0.0171 0.0623 0.0205 0.0205 0.0368 0.1098 0.1276 0.0598
4 0.0762 0.0666 0.0481 0.0394 0.0590 0.0649 0.1209 0.2467 0.3564

9 … 50 51 52 53 54 55 56 \
0 0.2111 … 0.0232 0.0027 0.0065 0.0159 0.0072 0.0167 0.0180
1 0.2872 … 0.0125 0.0084 0.0089 0.0048 0.0094 0.0191 0.0140
2 0.6194 … 0.0033 0.0232 0.0166 0.0095 0.0180 0.0244 0.0316
3 0.1264 … 0.0241 0.0121 0.0036 0.0150 0.0085 0.0073 0.0050
4 0.4459 … 0.0156 0.0031 0.0054 0.0105 0.0110 0.0015 0.0072

57 58 59
0 0.0084 0.0090 0.0032
1 0.0049 0.0052 0.0044
2 0.0164 0.0095 0.0078
3 0.0044 0.0040 0.0117
4 0.0048 0.0107 0.0094

[5 rows x 60 columns]

[ ]: from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.
↪25,random_state=1)

[ ]: X_train.head()

[ ]: 0 1 2 3 4 5 6 7 8 \
67 0.0368 0.0403 0.0317 0.0293 0.0820 0.1342 0.1161 0.0663 0.0155
14 0.0124 0.0433 0.0604 0.0449 0.0597 0.0355 0.0531 0.0343 0.1052
164 0.0163 0.0198 0.0202 0.0386 0.0752 0.1444 0.1487 0.1484 0.2442
179 0.0394 0.0420 0.0446 0.0551 0.0597 0.1416 0.0956 0.0802 0.1618
19 0.0126 0.0149 0.0641 0.1732 0.2565 0.2559 0.2947 0.4110 0.4983

9 … 50 51 52 53 54 55 56 \
67 0.0506 … 0.0058 0.0091 0.0160 0.0160 0.0081 0.0070 0.0135
14 0.2120 … 0.0078 0.0083 0.0057 0.0174 0.0188 0.0054 0.0114
164 0.2822 … 0.0027 0.0077 0.0026 0.0031 0.0083 0.0020 0.0084

3
 179 0.2558 … 0.0118 0.0146 0.0040 0.0114 0.0032 0.0062 0.0101
19 0.5920 … 0.0153 0.0092 0.0035 0.0098 0.0121 0.0006 0.0181

57 58 59
67 0.0067 0.0078 0.0068
14 0.0196 0.0147 0.0062
164 0.0108 0.0083 0.0033
179 0.0068 0.0053 0.0087
19 0.0094 0.0116 0.0063

[5 rows x 60 columns]

[ ]: import tensorflow as tf
from tensorflow import keras

MODEL A - WITHOUT DROPOUT [0,0]

[ ]: model = keras.Sequential()
model.add(keras.layers.Dense(60, input_dim=60, activation='relu'))
model.add(keras.layers.Dense(30, activation='relu'))
model.add(keras.layers.Dense(15, activation='relu'))
model.add(keras.layers.Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam',metrics=['accuracy'])
history = model.fit(X_train, y_train, validation_split=0.33 ,␣
↪epochs=100,batch_size=8)

Epoch 1/100
13/13 [==============================] - 1s 21ms/step - loss: 0.7040 - accuracy:
0.4904 - val_loss: 0.6818 - val_accuracy: 0.7115
Epoch 2/100
13/13 [==============================] - 0s 4ms/step - loss: 0.6762 - accuracy:
0.7115 - val_loss: 0.6695 - val_accuracy: 0.6923
Epoch 3/100
13/13 [==============================] - 0s 5ms/step - loss: 0.6630 - accuracy:
0.7212 - val_loss: 0.6574 - val_accuracy: 0.7115
Epoch 4/100
13/13 [==============================] - 0s 4ms/step - loss: 0.6503 - accuracy:
0.7500 - val_loss: 0.6438 - val_accuracy: 0.7115
Epoch 5/100
13/13 [==============================] - 0s 4ms/step - loss: 0.6373 - accuracy:
0.7692 - val_loss: 0.6287 - val_accuracy: 0.7115
Epoch 6/100
13/13 [==============================] - 0s 4ms/step - loss: 0.6188 - accuracy:
0.7212 - val_loss: 0.6091 - val_accuracy: 0.7308
Epoch 7/100
13/13 [==============================] - 0s 6ms/step - loss: 0.5990 - accuracy:
0.7981 - val_loss: 0.5787 - val_accuracy: 0.7885
Epoch 8/100

4
13/13 [==============================] - 0s 5ms/step - loss: 0.5799 - accuracy:
0.7500 - val_loss: 0.5565 - val_accuracy: 0.7692
Epoch 9/100
13/13 [==============================] - 0s 6ms/step - loss: 0.5465 - accuracy:
0.7788 - val_loss: 0.5338 - val_accuracy: 0.7500
Epoch 10/100
13/13 [==============================] - 0s 5ms/step - loss: 0.5266 - accuracy:
0.7500 - val_loss: 0.5039 - val_accuracy: 0.8077
Epoch 11/100
13/13 [==============================] - 0s 5ms/step - loss: 0.4878 - accuracy:
0.8269 - val_loss: 0.4880 - val_accuracy: 0.7885
Epoch 12/100
13/13 [==============================] - 0s 5ms/step - loss: 0.4886 - accuracy:
0.7788 - val_loss: 0.4435 - val_accuracy: 0.8269
Epoch 13/100
13/13 [==============================] - 0s 6ms/step - loss: 0.4432 - accuracy:
0.8462 - val_loss: 0.5330 - val_accuracy: 0.7115
Epoch 14/100
13/13 [==============================] - 0s 6ms/step - loss: 0.4302 - accuracy:
0.7981 - val_loss: 0.4017 - val_accuracy: 0.8269
Epoch 15/100
13/13 [==============================] - 0s 5ms/step - loss: 0.3854 - accuracy:
0.8846 - val_loss: 0.5399 - val_accuracy: 0.6923
Epoch 16/100
13/13 [==============================] - 0s 5ms/step - loss: 0.4031 - accuracy:
0.7981 - val_loss: 0.4236 - val_accuracy: 0.8077
Epoch 17/100
13/13 [==============================] - 0s 5ms/step - loss: 0.3657 - accuracy:
0.8558 - val_loss: 0.3833 - val_accuracy: 0.8077
Epoch 18/100
13/13 [==============================] - 0s 5ms/step - loss: 0.3397 - accuracy:
0.9231 - val_loss: 0.4272 - val_accuracy: 0.8077
Epoch 19/100
13/13 [==============================] - 0s 6ms/step - loss: 0.3004 - accuracy:
0.9327 - val_loss: 0.4624 - val_accuracy: 0.8077
Epoch 20/100
13/13 [==============================] - 0s 5ms/step - loss: 0.2882 - accuracy:
0.9327 - val_loss: 0.3947 - val_accuracy: 0.8269
Epoch 21/100
13/13 [==============================] - 0s 4ms/step - loss: 0.2705 - accuracy:
0.9615 - val_loss: 0.3842 - val_accuracy: 0.8269
Epoch 22/100
13/13 [==============================] - 0s 4ms/step - loss: 0.2714 - accuracy:
0.9038 - val_loss: 0.4853 - val_accuracy: 0.8077
Epoch 23/100
13/13 [==============================] - 0s 5ms/step - loss: 0.2463 - accuracy:
0.9519 - val_loss: 0.4631 - val_accuracy: 0.8269
Epoch 24/100

5
13/13 [==============================] - 0s 6ms/step - loss: 0.2263 - accuracy:
0.9615 - val_loss: 0.4559 - val_accuracy: 0.8269
Epoch 25/100
13/13 [==============================] - 0s 5ms/step - loss: 0.2164 - accuracy:
0.9615 - val_loss: 0.4129 - val_accuracy: 0.8269
Epoch 26/100
13/13 [==============================] - 0s 6ms/step - loss: 0.2120 - accuracy:
0.9519 - val_loss: 0.4595 - val_accuracy: 0.8269
Epoch 27/100
13/13 [==============================] - 0s 5ms/step - loss: 0.1852 - accuracy:
0.9712 - val_loss: 0.4864 - val_accuracy: 0.8269
Epoch 28/100
13/13 [==============================] - 0s 5ms/step - loss: 0.1810 - accuracy:
0.9808 - val_loss: 0.4329 - val_accuracy: 0.8269
Epoch 29/100
13/13 [==============================] - 0s 5ms/step - loss: 0.1673 - accuracy:
0.9808 - val_loss: 0.4827 - val_accuracy: 0.8269
Epoch 30/100
13/13 [==============================] - 0s 5ms/step - loss: 0.1664 - accuracy:
0.9712 - val_loss: 0.4680 - val_accuracy: 0.8269
Epoch 31/100
13/13 [==============================] - 0s 5ms/step - loss: 0.1560 - accuracy:
0.9712 - val_loss: 0.4961 - val_accuracy: 0.8269
Epoch 32/100
13/13 [==============================] - 0s 6ms/step - loss: 0.1590 - accuracy:
0.9712 - val_loss: 0.4659 - val_accuracy: 0.8269
Epoch 33/100
13/13 [==============================] - 0s 5ms/step - loss: 0.1383 - accuracy:
0.9808 - val_loss: 0.4940 - val_accuracy: 0.8269
Epoch 34/100
13/13 [==============================] - 0s 6ms/step - loss: 0.1378 - accuracy:
0.9808 - val_loss: 0.4985 - val_accuracy: 0.8269
Epoch 35/100
13/13 [==============================] - 0s 5ms/step - loss: 0.1401 - accuracy:
0.9615 - val_loss: 0.5118 - val_accuracy: 0.8269
Epoch 36/100
13/13 [==============================] - 0s 5ms/step - loss: 0.1316 - accuracy:
0.9808 - val_loss: 0.5214 - val_accuracy: 0.8269
Epoch 37/100
13/13 [==============================] - 0s 5ms/step - loss: 0.1166 - accuracy:
0.9808 - val_loss: 0.5092 - val_accuracy: 0.8269
Epoch 38/100
13/13 [==============================] - 0s 6ms/step - loss: 0.1140 - accuracy:
0.9808 - val_loss: 0.5215 - val_accuracy: 0.8269
Epoch 39/100
13/13 [==============================] - 0s 5ms/step - loss: 0.1089 - accuracy:
0.9808 - val_loss: 0.5274 - val_accuracy: 0.8269
Epoch 40/100

6
13/13 [==============================] - 0s 6ms/step - loss: 0.1030 - accuracy:
0.9808 - val_loss: 0.5326 - val_accuracy: 0.8269
Epoch 41/100
13/13 [==============================] - 0s 4ms/step - loss: 0.1007 - accuracy:
0.9808 - val_loss: 0.5373 - val_accuracy: 0.8269
Epoch 42/100
13/13 [==============================] - 0s 5ms/step - loss: 0.1095 - accuracy:
0.9808 - val_loss: 0.5626 - val_accuracy: 0.8077
Epoch 43/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0963 - accuracy:
0.9808 - val_loss: 0.5173 - val_accuracy: 0.8269
Epoch 44/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0952 - accuracy:
0.9808 - val_loss: 0.5380 - val_accuracy: 0.8269
Epoch 45/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0876 - accuracy:
0.9808 - val_loss: 0.5445 - val_accuracy: 0.8269
Epoch 46/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0855 - accuracy:
0.9808 - val_loss: 0.5294 - val_accuracy: 0.8269
Epoch 47/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0900 - accuracy:
0.9808 - val_loss: 0.5278 - val_accuracy: 0.8269
Epoch 48/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0842 - accuracy:
0.9808 - val_loss: 0.5599 - val_accuracy: 0.8269
Epoch 49/100
13/13 [==============================] - 0s 6ms/step - loss: 0.0770 - accuracy:
0.9808 - val_loss: 0.5555 - val_accuracy: 0.8269
Epoch 50/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0745 - accuracy:
0.9808 - val_loss: 0.5654 - val_accuracy: 0.8269
Epoch 51/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0715 - accuracy:
0.9808 - val_loss: 0.5600 - val_accuracy: 0.8269
Epoch 52/100
13/13 [==============================] - 0s 6ms/step - loss: 0.0750 - accuracy:
0.9808 - val_loss: 0.5516 - val_accuracy: 0.8269
Epoch 53/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0673 - accuracy:
0.9808 - val_loss: 0.5567 - val_accuracy: 0.8269
Epoch 54/100
13/13 [==============================] - 0s 6ms/step - loss: 0.0676 - accuracy:
0.9808 - val_loss: 0.5689 - val_accuracy: 0.8269
Epoch 55/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0645 - accuracy:
0.9808 - val_loss: 0.5966 - val_accuracy: 0.8269
Epoch 56/100

7
13/13 [==============================] - 0s 5ms/step - loss: 0.0608 - accuracy:
0.9808 - val_loss: 0.5794 - val_accuracy: 0.8269
Epoch 57/100
13/13 [==============================] - 0s 6ms/step - loss: 0.0583 - accuracy:
0.9808 - val_loss: 0.5867 - val_accuracy: 0.8269
Epoch 58/100
13/13 [==============================] - 0s 4ms/step - loss: 0.0555 - accuracy:
0.9808 - val_loss: 0.6094 - val_accuracy: 0.8269
Epoch 59/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0535 - accuracy:
0.9808 - val_loss: 0.5846 - val_accuracy: 0.8269
Epoch 60/100
13/13 [==============================] - 0s 6ms/step - loss: 0.0515 - accuracy:
0.9808 - val_loss: 0.6073 - val_accuracy: 0.8269
Epoch 61/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0500 - accuracy:
0.9808 - val_loss: 0.6076 - val_accuracy: 0.8269
Epoch 62/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0480 - accuracy:
0.9808 - val_loss: 0.6213 - val_accuracy: 0.8269
Epoch 63/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0433 - accuracy:
0.9808 - val_loss: 0.6097 - val_accuracy: 0.8269
Epoch 64/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0429 - accuracy:
0.9808 - val_loss: 0.6210 - val_accuracy: 0.8269
Epoch 65/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0394 - accuracy:
0.9808 - val_loss: 0.6270 - val_accuracy: 0.8269
Epoch 66/100
13/13 [==============================] - 0s 6ms/step - loss: 0.0374 - accuracy:
0.9808 - val_loss: 0.6278 - val_accuracy: 0.8269
Epoch 67/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0360 - accuracy:
0.9808 - val_loss: 0.6295 - val_accuracy: 0.8269
Epoch 68/100
13/13 [==============================] - 0s 6ms/step - loss: 0.0348 - accuracy:
0.9808 - val_loss: 0.6639 - val_accuracy: 0.8269
Epoch 69/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0311 - accuracy:
0.9904 - val_loss: 0.6499 - val_accuracy: 0.8269
Epoch 70/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0287 - accuracy:
0.9904 - val_loss: 0.6653 - val_accuracy: 0.8269
Epoch 71/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0274 - accuracy:
0.9904 - val_loss: 0.6671 - val_accuracy: 0.8269
Epoch 72/100

8
13/13 [==============================] - 0s 4ms/step - loss: 0.0259 - accuracy:
0.9904 - val_loss: 0.6922 - val_accuracy: 0.8269
Epoch 73/100
13/13 [==============================] - 0s 6ms/step - loss: 0.0244 - accuracy:
0.9904 - val_loss: 0.6731 - val_accuracy: 0.8269
Epoch 74/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0234 - accuracy:
1.0000 - val_loss: 0.6890 - val_accuracy: 0.8269
Epoch 75/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0216 - accuracy:
1.0000 - val_loss: 0.7165 - val_accuracy: 0.8269
Epoch 76/100
13/13 [==============================] - 0s 6ms/step - loss: 0.0206 - accuracy:
1.0000 - val_loss: 0.7010 - val_accuracy: 0.8269
Epoch 77/100
13/13 [==============================] - 0s 4ms/step - loss: 0.0193 - accuracy:
1.0000 - val_loss: 0.7211 - val_accuracy: 0.8269
Epoch 78/100
13/13 [==============================] - 0s 4ms/step - loss: 0.0176 - accuracy:
1.0000 - val_loss: 0.7220 - val_accuracy: 0.8269
Epoch 79/100
13/13 [==============================] - 0s 6ms/step - loss: 0.0163 - accuracy:
1.0000 - val_loss: 0.7396 - val_accuracy: 0.8269
Epoch 80/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0157 - accuracy:
1.0000 - val_loss: 0.7426 - val_accuracy: 0.8269
Epoch 81/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0143 - accuracy:
1.0000 - val_loss: 0.7615 - val_accuracy: 0.8269
Epoch 82/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0136 - accuracy:
1.0000 - val_loss: 0.7739 - val_accuracy: 0.8269
Epoch 83/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0126 - accuracy:
1.0000 - val_loss: 0.7794 - val_accuracy: 0.8269
Epoch 84/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0120 - accuracy:
1.0000 - val_loss: 0.7789 - val_accuracy: 0.8269
Epoch 85/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0115 - accuracy:
1.0000 - val_loss: 0.8011 - val_accuracy: 0.8269
Epoch 86/100
13/13 [==============================] - 0s 6ms/step - loss: 0.0107 - accuracy:
1.0000 - val_loss: 0.7895 - val_accuracy: 0.8269
Epoch 87/100
13/13 [==============================] - 0s 6ms/step - loss: 0.0101 - accuracy:
1.0000 - val_loss: 0.8169 - val_accuracy: 0.8269
Epoch 88/100

9
 13/13 [==============================] - 0s 6ms/step - loss: 0.0093 - accuracy:
1.0000 - val_loss: 0.8191 - val_accuracy: 0.8269
Epoch 89/100
13/13 [==============================] - 0s 6ms/step - loss: 0.0089 - accuracy:
1.0000 - val_loss: 0.8353 - val_accuracy: 0.8269
Epoch 90/100
13/13 [==============================] - 0s 6ms/step - loss: 0.0084 - accuracy:
1.0000 - val_loss: 0.8363 - val_accuracy: 0.8269
Epoch 91/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0080 - accuracy:
1.0000 - val_loss: 0.8368 - val_accuracy: 0.8269
Epoch 92/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0076 - accuracy:
1.0000 - val_loss: 0.8538 - val_accuracy: 0.8269
Epoch 93/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0071 - accuracy:
1.0000 - val_loss: 0.8588 - val_accuracy: 0.8269
Epoch 94/100
13/13 [==============================] - 0s 5ms/step - loss: 0.0068 - accuracy:
1.0000 - val_loss: 0.8703 - val_accuracy: 0.8269
Epoch 95/100
13/13 [==============================] - 0s 6ms/step - loss: 0.0066 - accuracy:
1.0000 - val_loss: 0.8705 - val_accuracy: 0.8269
Epoch 96/100
13/13 [==============================] - 0s 10ms/step - loss: 0.0063 - accuracy:
1.0000 - val_loss: 0.8722 - val_accuracy: 0.8269
Epoch 97/100
13/13 [==============================] - 0s 8ms/step - loss: 0.0060 - accuracy:
1.0000 - val_loss: 0.8949 - val_accuracy: 0.8269
Epoch 98/100
13/13 [==============================] - 0s 8ms/step - loss: 0.0057 - accuracy:
1.0000 - val_loss: 0.8968 - val_accuracy: 0.8269
Epoch 99/100
13/13 [==============================] - 0s 8ms/step - loss: 0.0054 - accuracy:
1.0000 - val_loss: 0.9003 - val_accuracy: 0.8269
Epoch 100/100
13/13 [==============================] - 0s 11ms/step - loss: 0.0052 - accuracy:
1.0000 - val_loss: 0.9118 - val_accuracy: 0.8269

[ ]: model.evaluate(X_test, y_test)

2/2 [==============================] - 0s 7ms/step - loss: 1.0048 - accuracy:

0.7500

[ ]: [1.0047539472579956, 0.75]

10
[ ]: y_pred = model.predict(X_test).reshape(-1)
print(y_pred[:10])
# round the values to nearest integer ie 0 or 1
y_pred = np.round(y_pred)
print(y_pred[:10])

2/2 [==============================] - 0s 5ms/step

[1.2439683e-13 6.1349976e-01 8.9655834e-01 2.7198162e-08 9.9989498e-01
9.9784505e-01 9.8341465e-01 9.9996066e-01 1.1255453e-05 9.9995947e-01]
[0. 1. 1. 0. 1. 1. 1. 1. 0. 1.]

[ ]: y_test[:10]

[ ]: R
186 False
155 False
165 False
200 False
58 True
34 True
151 False
18 True
202 False
62 True

[ ]: from sklearn.metrics import confusion_matrix , classification_report

print(classification_report(y_test, y_pred))

precision recall f1-score support

False 0.80 0.89 0.84 27

True 0.86 0.76 0.81 25

accuracy 0.83 52
macro avg 0.83 0.82 0.83 52
weighted avg 0.83 0.83 0.83 52

[ ]: print(history.history.keys())
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()
# summarize history for loss

11
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()

dict_keys(['loss', 'accuracy', 'val_loss', 'val_accuracy'])

12
 MODEL B - WITH DROPOUT [0.5,0.9]

[ ]: modeld = keras.Sequential([
keras.layers.Dense(60, input_dim=60, activation='relu'),
keras.layers.Dense(30, activation='relu'),
keras.layers.Dropout(0.5),
keras.layers.Dense(15, activation='relu'),
keras.layers.Dropout(0.9),
keras.layers.Dense(1, activation='sigmoid')
])
modeld.compile(loss='binary_crossentropy',␣
↪optimizer='adam',metrics=['accuracy'])

hist2 = modeld.fit(X_train, y_train, validation_split= 0.33,␣

↪epochs=100,batch_size=8)

Epoch 1/100
13/13 [==============================] - 1s 21ms/step - loss: 1.0139 - accuracy:
0.5000 - val_loss: 0.7045 - val_accuracy: 0.5192
Epoch 2/100
13/13 [==============================] - 0s 5ms/step - loss: 0.8195 - accuracy:

13
0.5288 - val_loss: 0.6810 - val_accuracy: 0.5577
Epoch 3/100
13/13 [==============================] - 0s 4ms/step - loss: 0.7487 - accuracy:
0.5192 - val_loss: 0.6811 - val_accuracy: 0.5962
Epoch 4/100
13/13 [==============================] - 0s 5ms/step - loss: 0.7026 - accuracy:
0.5288 - val_loss: 0.6854 - val_accuracy: 0.6154
Epoch 5/100
13/13 [==============================] - 0s 5ms/step - loss: 0.6941 - accuracy:
0.5096 - val_loss: 0.6879 - val_accuracy: 0.6154
Epoch 6/100
13/13 [==============================] - 0s 6ms/step - loss: 0.6848 - accuracy:
0.5481 - val_loss: 0.6872 - val_accuracy: 0.5769
Epoch 7/100
13/13 [==============================] - 0s 5ms/step - loss: 0.6895 - accuracy:
0.5673 - val_loss: 0.6874 - val_accuracy: 0.5769
Epoch 8/100
13/13 [==============================] - 0s 4ms/step - loss: 0.7016 - accuracy:
0.6058 - val_loss: 0.6863 - val_accuracy: 0.5385
Epoch 9/100
13/13 [==============================] - 0s 5ms/step - loss: 0.7061 - accuracy:
0.5962 - val_loss: 0.6835 - val_accuracy: 0.5962
Epoch 10/100
13/13 [==============================] - 0s 6ms/step - loss: 0.6972 - accuracy:
0.5481 - val_loss: 0.6828 - val_accuracy: 0.5962
Epoch 11/100
13/13 [==============================] - 0s 5ms/step - loss: 0.6978 - accuracy:
0.5288 - val_loss: 0.6814 - val_accuracy: 0.6346
Epoch 12/100
13/13 [==============================] - 0s 5ms/step - loss: 0.6651 - accuracy:
0.6250 - val_loss: 0.6815 - val_accuracy: 0.6154
Epoch 13/100
13/13 [==============================] - 0s 5ms/step - loss: 0.6727 - accuracy:
0.5865 - val_loss: 0.6796 - val_accuracy: 0.6154
Epoch 14/100
13/13 [==============================] - 0s 4ms/step - loss: 0.6857 - accuracy:
0.5865 - val_loss: 0.6797 - val_accuracy: 0.6538
Epoch 15/100
13/13 [==============================] - 0s 5ms/step - loss: 0.6842 - accuracy:
0.5673 - val_loss: 0.6788 - val_accuracy: 0.6731
Epoch 16/100
13/13 [==============================] - 0s 4ms/step - loss: 0.6654 - accuracy:
0.6346 - val_loss: 0.6778 - val_accuracy: 0.6538
Epoch 17/100
13/13 [==============================] - 0s 7ms/step - loss: 0.6809 - accuracy:
0.5577 - val_loss: 0.6809 - val_accuracy: 0.5769
Epoch 18/100
13/13 [==============================] - 0s 7ms/step - loss: 0.6828 - accuracy:

14
0.5673 - val_loss: 0.6858 - val_accuracy: 0.5577
Epoch 19/100
13/13 [==============================] - 0s 7ms/step - loss: 0.7015 - accuracy:
0.5385 - val_loss: 0.6848 - val_accuracy: 0.5385
Epoch 20/100
13/13 [==============================] - 0s 8ms/step - loss: 0.6670 - accuracy:
0.5865 - val_loss: 0.6821 - val_accuracy: 0.5385
Epoch 21/100
13/13 [==============================] - 0s 7ms/step - loss: 0.6921 - accuracy:
0.5385 - val_loss: 0.6819 - val_accuracy: 0.5385
Epoch 22/100
13/13 [==============================] - 0s 8ms/step - loss: 0.6862 - accuracy:
0.5673 - val_loss: 0.6809 - val_accuracy: 0.5577
Epoch 23/100
13/13 [==============================] - 0s 8ms/step - loss: 0.6879 - accuracy:
0.5865 - val_loss: 0.6806 - val_accuracy: 0.5577
Epoch 24/100
13/13 [==============================] - 0s 6ms/step - loss: 0.6572 - accuracy:
0.5769 - val_loss: 0.6831 - val_accuracy: 0.5192
Epoch 25/100
13/13 [==============================] - 0s 7ms/step - loss: 0.6693 - accuracy:
0.5962 - val_loss: 0.6850 - val_accuracy: 0.5192
Epoch 26/100
13/13 [==============================] - 0s 6ms/step - loss: 0.6564 - accuracy:
0.6250 - val_loss: 0.6825 - val_accuracy: 0.5385
Epoch 27/100
13/13 [==============================] - 0s 7ms/step - loss: 0.6830 - accuracy:
0.5385 - val_loss: 0.6788 - val_accuracy: 0.5577
Epoch 28/100
13/13 [==============================] - 0s 8ms/step - loss: 0.6613 - accuracy:
0.6346 - val_loss: 0.6783 - val_accuracy: 0.5769
Epoch 29/100
13/13 [==============================] - 0s 6ms/step - loss: 0.6769 - accuracy:
0.6442 - val_loss: 0.6758 - val_accuracy: 0.6154
Epoch 30/100
13/13 [==============================] - 0s 7ms/step - loss: 0.6828 - accuracy:
0.5865 - val_loss: 0.6768 - val_accuracy: 0.5577
Epoch 31/100
13/13 [==============================] - 0s 7ms/step - loss: 0.6736 - accuracy:
0.5769 - val_loss: 0.6750 - val_accuracy: 0.5577
Epoch 32/100
13/13 [==============================] - 0s 6ms/step - loss: 0.6632 - accuracy:
0.5962 - val_loss: 0.6723 - val_accuracy: 0.5577
Epoch 33/100
13/13 [==============================] - 0s 8ms/step - loss: 0.6321 - accuracy:
0.6346 - val_loss: 0.6682 - val_accuracy: 0.5962
Epoch 34/100
13/13 [==============================] - 0s 8ms/step - loss: 0.6713 - accuracy:

15
0.5962 - val_loss: 0.6568 - val_accuracy: 0.6923
Epoch 35/100
13/13 [==============================] - 0s 8ms/step - loss: 0.6456 - accuracy:
0.6346 - val_loss: 0.6550 - val_accuracy: 0.6923
Epoch 36/100
13/13 [==============================] - 0s 7ms/step - loss: 0.6578 - accuracy:
0.5577 - val_loss: 0.6632 - val_accuracy: 0.6346
Epoch 37/100
13/13 [==============================] - 0s 7ms/step - loss: 0.6648 - accuracy:
0.5962 - val_loss: 0.6576 - val_accuracy: 0.6731
Epoch 38/100
13/13 [==============================] - 0s 7ms/step - loss: 0.6483 - accuracy:
0.6058 - val_loss: 0.6529 - val_accuracy: 0.7115
Epoch 39/100
13/13 [==============================] - 0s 7ms/step - loss: 0.6499 - accuracy:
0.6538 - val_loss: 0.6521 - val_accuracy: 0.6538
Epoch 40/100
13/13 [==============================] - 0s 7ms/step - loss: 0.6258 - accuracy:
0.6827 - val_loss: 0.6436 - val_accuracy: 0.6731
Epoch 41/100
13/13 [==============================] - 0s 7ms/step - loss: 0.6388 - accuracy:
0.6731 - val_loss: 0.6462 - val_accuracy: 0.6923
Epoch 42/100
13/13 [==============================] - 0s 8ms/step - loss: 0.6997 - accuracy:
0.5096 - val_loss: 0.6614 - val_accuracy: 0.5385
Epoch 43/100
13/13 [==============================] - 0s 7ms/step - loss: 0.6445 - accuracy:
0.5865 - val_loss: 0.6616 - val_accuracy: 0.5385
Epoch 44/100
13/13 [==============================] - 0s 8ms/step - loss: 0.6808 - accuracy:
0.5673 - val_loss: 0.6548 - val_accuracy: 0.6538
Epoch 45/100
13/13 [==============================] - 0s 7ms/step - loss: 0.6436 - accuracy:
0.5769 - val_loss: 0.6487 - val_accuracy: 0.6923
Epoch 46/100
13/13 [==============================] - 0s 8ms/step - loss: 0.6574 - accuracy:
0.5962 - val_loss: 0.6504 - val_accuracy: 0.6923
Epoch 47/100
13/13 [==============================] - 0s 7ms/step - loss: 0.6389 - accuracy:
0.5962 - val_loss: 0.6477 - val_accuracy: 0.6923
Epoch 48/100
13/13 [==============================] - 0s 7ms/step - loss: 0.6379 - accuracy:
0.6250 - val_loss: 0.6432 - val_accuracy: 0.6923
Epoch 49/100
13/13 [==============================] - 0s 5ms/step - loss: 0.6677 - accuracy:
0.6731 - val_loss: 0.6442 - val_accuracy: 0.6923
Epoch 50/100
13/13 [==============================] - 0s 5ms/step - loss: 0.6504 - accuracy:

16
0.6250 - val_loss: 0.6387 - val_accuracy: 0.7308
Epoch 51/100
13/13 [==============================] - 0s 5ms/step - loss: 0.6342 - accuracy:
0.6154 - val_loss: 0.6450 - val_accuracy: 0.6923
Epoch 52/100
13/13 [==============================] - 0s 6ms/step - loss: 0.6551 - accuracy:
0.6154 - val_loss: 0.6339 - val_accuracy: 0.7115
Epoch 53/100
13/13 [==============================] - 0s 5ms/step - loss: 0.6348 - accuracy:
0.6154 - val_loss: 0.6239 - val_accuracy: 0.7115
Epoch 54/100
13/13 [==============================] - 0s 4ms/step - loss: 0.6357 - accuracy:
0.6538 - val_loss: 0.6153 - val_accuracy: 0.7308
Epoch 55/100
13/13 [==============================] - 0s 5ms/step - loss: 0.6370 - accuracy:
0.6250 - val_loss: 0.6039 - val_accuracy: 0.7308
Epoch 56/100
13/13 [==============================] - 0s 5ms/step - loss: 0.6220 - accuracy:
0.6442 - val_loss: 0.5916 - val_accuracy: 0.7885
Epoch 57/100
13/13 [==============================] - 0s 6ms/step - loss: 0.6189 - accuracy:
0.6154 - val_loss: 0.6145 - val_accuracy: 0.6923
Epoch 58/100
13/13 [==============================] - 0s 5ms/step - loss: 0.6511 - accuracy:
0.6442 - val_loss: 0.5983 - val_accuracy: 0.7308
Epoch 59/100
13/13 [==============================] - 0s 4ms/step - loss: 0.6576 - accuracy:
0.6442 - val_loss: 0.6014 - val_accuracy: 0.7885
Epoch 60/100
13/13 [==============================] - 0s 5ms/step - loss: 0.6629 - accuracy:
0.6346 - val_loss: 0.6050 - val_accuracy: 0.7692
Epoch 61/100
13/13 [==============================] - 0s 5ms/step - loss: 0.6311 - accuracy:
0.6538 - val_loss: 0.6133 - val_accuracy: 0.7115
Epoch 62/100
13/13 [==============================] - 0s 5ms/step - loss: 0.6054 - accuracy:
0.6731 - val_loss: 0.6139 - val_accuracy: 0.6923
Epoch 63/100
13/13 [==============================] - 0s 4ms/step - loss: 0.6189 - accuracy:
0.6250 - val_loss: 0.6013 - val_accuracy: 0.7115
Epoch 64/100
13/13 [==============================] - 0s 4ms/step - loss: 0.6254 - accuracy:
0.6154 - val_loss: 0.5916 - val_accuracy: 0.7308
Epoch 65/100
13/13 [==============================] - 0s 5ms/step - loss: 0.6274 - accuracy:
0.6058 - val_loss: 0.5942 - val_accuracy: 0.7115
Epoch 66/100
13/13 [==============================] - 0s 5ms/step - loss: 0.6187 - accuracy:

17
0.6346 - val_loss: 0.5948 - val_accuracy: 0.7115
Epoch 67/100
13/13 [==============================] - 0s 5ms/step - loss: 0.6258 - accuracy:
0.6538 - val_loss: 0.5710 - val_accuracy: 0.8077
Epoch 68/100
13/13 [==============================] - 0s 5ms/step - loss: 0.6113 - accuracy:
0.6346 - val_loss: 0.5891 - val_accuracy: 0.7308
Epoch 69/100
13/13 [==============================] - 0s 5ms/step - loss: 0.6008 - accuracy:
0.7115 - val_loss: 0.5499 - val_accuracy: 0.8077
Epoch 70/100
13/13 [==============================] - 0s 5ms/step - loss: 0.5271 - accuracy:
0.7404 - val_loss: 0.5280 - val_accuracy: 0.8269
Epoch 71/100
13/13 [==============================] - 0s 5ms/step - loss: 0.6211 - accuracy:
0.6346 - val_loss: 0.5569 - val_accuracy: 0.7692
Epoch 72/100
13/13 [==============================] - 0s 4ms/step - loss: 0.6073 - accuracy:
0.6442 - val_loss: 0.5590 - val_accuracy: 0.7500
Epoch 73/100
13/13 [==============================] - 0s 4ms/step - loss: 0.5829 - accuracy:
0.6538 - val_loss: 0.5416 - val_accuracy: 0.8077
Epoch 74/100
13/13 [==============================] - 0s 5ms/step - loss: 0.5618 - accuracy:
0.6923 - val_loss: 0.5551 - val_accuracy: 0.7692
Epoch 75/100
13/13 [==============================] - 0s 5ms/step - loss: 0.5865 - accuracy:
0.6346 - val_loss: 0.5575 - val_accuracy: 0.7308
Epoch 76/100
13/13 [==============================] - 0s 4ms/step - loss: 0.6006 - accuracy:
0.6250 - val_loss: 0.5396 - val_accuracy: 0.8077
Epoch 77/100
13/13 [==============================] - 0s 5ms/step - loss: 0.5654 - accuracy:
0.6923 - val_loss: 0.5465 - val_accuracy: 0.7692
Epoch 78/100
13/13 [==============================] - 0s 4ms/step - loss: 0.5706 - accuracy:
0.6635 - val_loss: 0.5339 - val_accuracy: 0.7885
Epoch 79/100
13/13 [==============================] - 0s 5ms/step - loss: 0.5872 - accuracy:
0.6635 - val_loss: 0.5116 - val_accuracy: 0.8077
Epoch 80/100
13/13 [==============================] - 0s 5ms/step - loss: 0.5778 - accuracy:
0.6635 - val_loss: 0.5494 - val_accuracy: 0.7308
Epoch 81/100
13/13 [==============================] - 0s 5ms/step - loss: 0.6312 - accuracy:
0.6154 - val_loss: 0.5567 - val_accuracy: 0.7115
Epoch 82/100
13/13 [==============================] - 0s 5ms/step - loss: 0.5712 - accuracy:

18
0.7019 - val_loss: 0.5053 - val_accuracy: 0.8269
Epoch 83/100
13/13 [==============================] - 0s 4ms/step - loss: 0.5442 - accuracy:
0.7212 - val_loss: 0.5131 - val_accuracy: 0.8077
Epoch 84/100
13/13 [==============================] - 0s 6ms/step - loss: 0.5244 - accuracy:
0.7596 - val_loss: 0.4923 - val_accuracy: 0.8077
Epoch 85/100
13/13 [==============================] - 0s 5ms/step - loss: 0.5272 - accuracy:
0.7404 - val_loss: 0.5045 - val_accuracy: 0.7885
Epoch 86/100
13/13 [==============================] - 0s 6ms/step - loss: 0.5971 - accuracy:
0.6827 - val_loss: 0.4834 - val_accuracy: 0.8077
Epoch 87/100
13/13 [==============================] - 0s 5ms/step - loss: 0.6031 - accuracy:
0.6346 - val_loss: 0.6558 - val_accuracy: 0.5769
Epoch 88/100
13/13 [==============================] - 0s 5ms/step - loss: 0.6892 - accuracy:
0.6250 - val_loss: 0.5133 - val_accuracy: 0.7885
Epoch 89/100
13/13 [==============================] - 0s 5ms/step - loss: 0.6391 - accuracy:
0.7212 - val_loss: 0.4924 - val_accuracy: 0.8269
Epoch 90/100
13/13 [==============================] - 0s 4ms/step - loss: 0.5639 - accuracy:
0.7115 - val_loss: 0.5534 - val_accuracy: 0.7308
Epoch 91/100
13/13 [==============================] - 0s 5ms/step - loss: 0.5956 - accuracy:
0.6538 - val_loss: 0.5312 - val_accuracy: 0.7692
Epoch 92/100
13/13 [==============================] - 0s 4ms/step - loss: 0.5730 - accuracy:
0.6538 - val_loss: 0.5146 - val_accuracy: 0.7885
Epoch 93/100
13/13 [==============================] - 0s 4ms/step - loss: 0.5356 - accuracy:
0.6731 - val_loss: 0.5194 - val_accuracy: 0.7692
Epoch 94/100
13/13 [==============================] - 0s 5ms/step - loss: 0.5306 - accuracy:
0.7115 - val_loss: 0.5211 - val_accuracy: 0.7500
Epoch 95/100
13/13 [==============================] - 0s 6ms/step - loss: 0.5531 - accuracy:
0.7404 - val_loss: 0.4946 - val_accuracy: 0.8077
Epoch 96/100
13/13 [==============================] - 0s 5ms/step - loss: 0.5719 - accuracy:
0.6538 - val_loss: 0.5029 - val_accuracy: 0.7692
Epoch 97/100
13/13 [==============================] - 0s 6ms/step - loss: 0.5505 - accuracy:
0.6923 - val_loss: 0.4853 - val_accuracy: 0.8077
Epoch 98/100
13/13 [==============================] - 0s 5ms/step - loss: 0.5691 - accuracy:

19
 0.6731 - val_loss: 0.4915 - val_accuracy: 0.8077
Epoch 99/100
13/13 [==============================] - 0s 5ms/step - loss: 0.5601 - accuracy:
0.6731 - val_loss: 0.5030 - val_accuracy: 0.8077
Epoch 100/100
13/13 [==============================] - 0s 6ms/step - loss: 0.5395 - accuracy:
0.6731 - val_loss: 0.4902 - val_accuracy: 0.8077

[ ]: modeld.evaluate(X_test, y_test)

2/2 [==============================] - 0s 6ms/step - loss: 0.5050 - accuracy:

0.7885

[ ]: [0.5050378441810608, 0.7884615659713745]

[ ]: y_pred = modeld.predict(X_test).reshape(-1)
print(y_pred[:10])
# round the values to nearest integer ie 0 or 1
y_pred = np.round(y_pred)
print(y_pred[:10])

2/2 [==============================] - 0s 7ms/step

[0.1623204 0.42910007 0.48135084 0.23783027 0.8374161 0.40211114
0.42789772 0.87517387 0.30025247 0.8644154 ]
[0. 0. 0. 0. 1. 0. 0. 1. 0. 1.]

[ ]: from sklearn.metrics import confusion_matrix , classification_report

print(classification_report(y_test, y_pred))

precision recall f1-score support

False 0.71 1.00 0.83 27

True 1.00 0.56 0.72 25

accuracy 0.79 52
macro avg 0.86 0.78 0.77 52
weighted avg 0.85 0.79 0.78 52

[ ]: print(hist2.history.keys())
plt.plot(hist2.history['accuracy'])
plt.plot(hist2.history['val_accuracy'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()
# summarize hist2 for loss

20
plt.plot(hist2.history['loss'])
plt.plot(hist2.history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()

dict_keys(['loss', 'accuracy', 'val_loss', 'val_accuracy'])

21
 MODEL C - WITH DROPOUT [0.9,0.5]

[ ]: modeld2 = keras.Sequential([
keras.layers.Dense(60, input_dim=60, activation='relu'),
keras.layers.Dense(30, activation='relu'),
keras.layers.Dropout(0.9),
keras.layers.Dense(15, activation='relu'),
keras.layers.Dropout(0.5),
keras.layers.Dense(1, activation='sigmoid')])
modeld2.compile(loss='binary_crossentropy',␣
↪optimizer='adam',metrics=['accuracy'])

hist3 = modeld2.fit(X_train, y_train, validation_split= 0.30,␣

↪epochs=100,batch_size=8)

Epoch 1/100
14/14 [==============================] - 1s 19ms/step - loss: 0.8847 - accuracy:
0.4954 - val_loss: 0.6834 - val_accuracy: 0.5319
Epoch 2/100
14/14 [==============================] - 0s 5ms/step - loss: 0.8502 - accuracy:
0.5046 - val_loss: 0.6823 - val_accuracy: 0.5957

22
Epoch 3/100
14/14 [==============================] - 0s 6ms/step - loss: 0.7283 - accuracy:
0.5321 - val_loss: 0.6880 - val_accuracy: 0.6170
Epoch 4/100
14/14 [==============================] - 0s 5ms/step - loss: 0.7210 - accuracy:
0.5138 - val_loss: 0.6916 - val_accuracy: 0.5745
Epoch 5/100
14/14 [==============================] - 0s 5ms/step - loss: 0.7107 - accuracy:
0.5046 - val_loss: 0.6900 - val_accuracy: 0.5319
Epoch 6/100
14/14 [==============================] - 0s 4ms/step - loss: 0.6985 - accuracy:
0.5321 - val_loss: 0.6869 - val_accuracy: 0.6596
Epoch 7/100
14/14 [==============================] - 0s 4ms/step - loss: 0.6964 - accuracy:
0.4128 - val_loss: 0.6897 - val_accuracy: 0.5106
Epoch 8/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6888 - accuracy:
0.5321 - val_loss: 0.6879 - val_accuracy: 0.6383
Epoch 9/100
14/14 [==============================] - 0s 6ms/step - loss: 0.7062 - accuracy:
0.5321 - val_loss: 0.6855 - val_accuracy: 0.7021
Epoch 10/100
14/14 [==============================] - 0s 5ms/step - loss: 0.7154 - accuracy:
0.4679 - val_loss: 0.6843 - val_accuracy: 0.7234
Epoch 11/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6996 - accuracy:
0.5229 - val_loss: 0.6848 - val_accuracy: 0.7660
Epoch 12/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6838 - accuracy:
0.5596 - val_loss: 0.6842 - val_accuracy: 0.7660
Epoch 13/100
14/14 [==============================] - 0s 4ms/step - loss: 0.6853 - accuracy:
0.4862 - val_loss: 0.6834 - val_accuracy: 0.7234
Epoch 14/100
14/14 [==============================] - 0s 4ms/step - loss: 0.6773 - accuracy:
0.5046 - val_loss: 0.6823 - val_accuracy: 0.7234
Epoch 15/100
14/14 [==============================] - 0s 4ms/step - loss: 0.6755 - accuracy:
0.5321 - val_loss: 0.6770 - val_accuracy: 0.5957
Epoch 16/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6869 - accuracy:
0.4312 - val_loss: 0.6729 - val_accuracy: 0.5745
Epoch 17/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6779 - accuracy:
0.5138 - val_loss: 0.6761 - val_accuracy: 0.6170
Epoch 18/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6810 - accuracy:
0.6147 - val_loss: 0.6809 - val_accuracy: 0.5745

23
Epoch 19/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6932 - accuracy:
0.5138 - val_loss: 0.6763 - val_accuracy: 0.6383
Epoch 20/100
14/14 [==============================] - 0s 5ms/step - loss: 0.7138 - accuracy:
0.5321 - val_loss: 0.6725 - val_accuracy: 0.6383
Epoch 21/100
14/14 [==============================] - 0s 4ms/step - loss: 0.6687 - accuracy:
0.5321 - val_loss: 0.6736 - val_accuracy: 0.5957
Epoch 22/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6737 - accuracy:
0.5505 - val_loss: 0.6747 - val_accuracy: 0.6383
Epoch 23/100
14/14 [==============================] - 0s 5ms/step - loss: 0.7042 - accuracy:
0.5596 - val_loss: 0.6696 - val_accuracy: 0.5745
Epoch 24/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6751 - accuracy:
0.5321 - val_loss: 0.6729 - val_accuracy: 0.5745
Epoch 25/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6974 - accuracy:
0.5505 - val_loss: 0.6735 - val_accuracy: 0.6383
Epoch 26/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6901 - accuracy:
0.5321 - val_loss: 0.6772 - val_accuracy: 0.6170
Epoch 27/100
14/14 [==============================] - 0s 6ms/step - loss: 0.6829 - accuracy:
0.5505 - val_loss: 0.6829 - val_accuracy: 0.6170
Epoch 28/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6747 - accuracy:
0.5505 - val_loss: 0.6813 - val_accuracy: 0.6809
Epoch 29/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6810 - accuracy:
0.5321 - val_loss: 0.6786 - val_accuracy: 0.6809
Epoch 30/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6783 - accuracy:
0.5321 - val_loss: 0.6732 - val_accuracy: 0.6383
Epoch 31/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6760 - accuracy:
0.5321 - val_loss: 0.6697 - val_accuracy: 0.5319
Epoch 32/100
14/14 [==============================] - 0s 6ms/step - loss: 0.6619 - accuracy:
0.5413 - val_loss: 0.6610 - val_accuracy: 0.5319
Epoch 33/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6930 - accuracy:
0.5321 - val_loss: 0.6602 - val_accuracy: 0.5319
Epoch 34/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6773 - accuracy:
0.5321 - val_loss: 0.6739 - val_accuracy: 0.5319

24
Epoch 35/100
14/14 [==============================] - 0s 6ms/step - loss: 0.6677 - accuracy:
0.5321 - val_loss: 0.6744 - val_accuracy: 0.5319
Epoch 36/100
14/14 [==============================] - 0s 6ms/step - loss: 0.6777 - accuracy:
0.5229 - val_loss: 0.6682 - val_accuracy: 0.5319
Epoch 37/100
14/14 [==============================] - 0s 6ms/step - loss: 0.6619 - accuracy:
0.5596 - val_loss: 0.6666 - val_accuracy: 0.5319
Epoch 38/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6800 - accuracy:
0.5229 - val_loss: 0.6569 - val_accuracy: 0.5745
Epoch 39/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6392 - accuracy:
0.5596 - val_loss: 0.6426 - val_accuracy: 0.5532
Epoch 40/100
14/14 [==============================] - 0s 6ms/step - loss: 0.6414 - accuracy:
0.5413 - val_loss: 0.6372 - val_accuracy: 0.7021
Epoch 41/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6263 - accuracy:
0.5688 - val_loss: 0.6367 - val_accuracy: 0.7234
Epoch 42/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6527 - accuracy:
0.5321 - val_loss: 0.6349 - val_accuracy: 0.5957
Epoch 43/100
14/14 [==============================] - 0s 6ms/step - loss: 0.6424 - accuracy:
0.5780 - val_loss: 0.6397 - val_accuracy: 0.6596
Epoch 44/100
14/14 [==============================] - 0s 6ms/step - loss: 0.6698 - accuracy:
0.5780 - val_loss: 0.6443 - val_accuracy: 0.7447
Epoch 45/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6523 - accuracy:
0.5321 - val_loss: 0.6507 - val_accuracy: 0.7447
Epoch 46/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6623 - accuracy:
0.5505 - val_loss: 0.6446 - val_accuracy: 0.7660
Epoch 47/100
14/14 [==============================] - 0s 6ms/step - loss: 0.6492 - accuracy:
0.5413 - val_loss: 0.6308 - val_accuracy: 0.7872
Epoch 48/100
14/14 [==============================] - 0s 6ms/step - loss: 0.6894 - accuracy:
0.5046 - val_loss: 0.6224 - val_accuracy: 0.7234
Epoch 49/100
14/14 [==============================] - 0s 6ms/step - loss: 0.6687 - accuracy:
0.5780 - val_loss: 0.6240 - val_accuracy: 0.7447
Epoch 50/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6771 - accuracy:
0.5046 - val_loss: 0.6291 - val_accuracy: 0.7447

25
Epoch 51/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6464 - accuracy:
0.5413 - val_loss: 0.6235 - val_accuracy: 0.7234
Epoch 52/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6620 - accuracy:
0.5413 - val_loss: 0.6325 - val_accuracy: 0.7872
Epoch 53/100
14/14 [==============================] - 0s 6ms/step - loss: 0.6344 - accuracy:
0.5596 - val_loss: 0.6193 - val_accuracy: 0.7234
Epoch 54/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6723 - accuracy:
0.5688 - val_loss: 0.6146 - val_accuracy: 0.7234
Epoch 55/100
14/14 [==============================] - 0s 7ms/step - loss: 0.6570 - accuracy:
0.5413 - val_loss: 0.6231 - val_accuracy: 0.7234
Epoch 56/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6423 - accuracy:
0.5229 - val_loss: 0.6144 - val_accuracy: 0.5532
Epoch 57/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6553 - accuracy:
0.5413 - val_loss: 0.6217 - val_accuracy: 0.5532
Epoch 58/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6209 - accuracy:
0.5780 - val_loss: 0.6215 - val_accuracy: 0.5319
Epoch 59/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6556 - accuracy:
0.5413 - val_loss: 0.6160 - val_accuracy: 0.5532
Epoch 60/100
14/14 [==============================] - 0s 6ms/step - loss: 0.6498 - accuracy:
0.5596 - val_loss: 0.6072 - val_accuracy: 0.6170
Epoch 61/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6561 - accuracy:
0.5413 - val_loss: 0.6167 - val_accuracy: 0.7234
Epoch 62/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6514 - accuracy:
0.5596 - val_loss: 0.6179 - val_accuracy: 0.7447
Epoch 63/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6675 - accuracy:
0.5413 - val_loss: 0.6229 - val_accuracy: 0.7660
Epoch 64/100
14/14 [==============================] - 0s 6ms/step - loss: 0.6526 - accuracy:
0.5321 - val_loss: 0.6158 - val_accuracy: 0.7234
Epoch 65/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6712 - accuracy:
0.5229 - val_loss: 0.6126 - val_accuracy: 0.7021
Epoch 66/100
14/14 [==============================] - 0s 6ms/step - loss: 0.6485 - accuracy:
0.5321 - val_loss: 0.6071 - val_accuracy: 0.5532

26
Epoch 67/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6494 - accuracy:
0.5413 - val_loss: 0.6125 - val_accuracy: 0.5532
Epoch 68/100
14/14 [==============================] - 0s 6ms/step - loss: 0.6363 - accuracy:
0.5413 - val_loss: 0.6078 - val_accuracy: 0.5532
Epoch 69/100
14/14 [==============================] - 0s 4ms/step - loss: 0.6307 - accuracy:
0.5321 - val_loss: 0.6097 - val_accuracy: 0.6596
Epoch 70/100
14/14 [==============================] - 0s 4ms/step - loss: 0.6446 - accuracy:
0.5046 - val_loss: 0.6071 - val_accuracy: 0.6596
Epoch 71/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6201 - accuracy:
0.6147 - val_loss: 0.5978 - val_accuracy: 0.7234
Epoch 72/100
14/14 [==============================] - 0s 6ms/step - loss: 0.6164 - accuracy:
0.6514 - val_loss: 0.5858 - val_accuracy: 0.7021
Epoch 73/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6239 - accuracy:
0.5046 - val_loss: 0.5885 - val_accuracy: 0.7447
Epoch 74/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6248 - accuracy:
0.6239 - val_loss: 0.5911 - val_accuracy: 0.7660
Epoch 75/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6465 - accuracy:
0.5321 - val_loss: 0.5977 - val_accuracy: 0.7447
Epoch 76/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6433 - accuracy:
0.5780 - val_loss: 0.5983 - val_accuracy: 0.7234
Epoch 77/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6507 - accuracy:
0.6330 - val_loss: 0.5982 - val_accuracy: 0.7447
Epoch 78/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6640 - accuracy:
0.5505 - val_loss: 0.6079 - val_accuracy: 0.7872
Epoch 79/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6528 - accuracy:
0.5780 - val_loss: 0.5941 - val_accuracy: 0.7021
Epoch 80/100
14/14 [==============================] - 0s 6ms/step - loss: 0.5965 - accuracy:
0.6330 - val_loss: 0.5827 - val_accuracy: 0.7234
Epoch 81/100
14/14 [==============================] - 0s 6ms/step - loss: 0.5879 - accuracy:
0.6972 - val_loss: 0.5768 - val_accuracy: 0.7234
Epoch 82/100
14/14 [==============================] - 0s 6ms/step - loss: 0.6386 - accuracy:
0.5321 - val_loss: 0.5885 - val_accuracy: 0.7872

27
Epoch 83/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6257 - accuracy:
0.6055 - val_loss: 0.5867 - val_accuracy: 0.7872
Epoch 84/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6268 - accuracy:
0.6239 - val_loss: 0.5789 - val_accuracy: 0.7234
Epoch 85/100
14/14 [==============================] - 0s 5ms/step - loss: 0.5706 - accuracy:
0.7156 - val_loss: 0.5763 - val_accuracy: 0.7660
Epoch 86/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6035 - accuracy:
0.6789 - val_loss: 0.5711 - val_accuracy: 0.7234
Epoch 87/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6339 - accuracy:
0.6422 - val_loss: 0.5828 - val_accuracy: 0.7872
Epoch 88/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6035 - accuracy:
0.6330 - val_loss: 0.5784 - val_accuracy: 0.7234
Epoch 89/100
14/14 [==============================] - 0s 6ms/step - loss: 0.5752 - accuracy:
0.6972 - val_loss: 0.5720 - val_accuracy: 0.7021
Epoch 90/100
14/14 [==============================] - 0s 6ms/step - loss: 0.6319 - accuracy:
0.5872 - val_loss: 0.5813 - val_accuracy: 0.7872
Epoch 91/100
14/14 [==============================] - 0s 5ms/step - loss: 0.5976 - accuracy:
0.6606 - val_loss: 0.5795 - val_accuracy: 0.7021
Epoch 92/100
14/14 [==============================] - 0s 5ms/step - loss: 0.5895 - accuracy:
0.6239 - val_loss: 0.5886 - val_accuracy: 0.6809
Epoch 93/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6320 - accuracy:
0.6697 - val_loss: 0.5777 - val_accuracy: 0.7021
Epoch 94/100
14/14 [==============================] - 0s 6ms/step - loss: 0.6146 - accuracy:
0.7523 - val_loss: 0.6108 - val_accuracy: 0.8511
Epoch 95/100
14/14 [==============================] - 0s 5ms/step - loss: 0.5927 - accuracy:
0.6697 - val_loss: 0.6003 - val_accuracy: 0.8085
Epoch 96/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6618 - accuracy:
0.6330 - val_loss: 0.5809 - val_accuracy: 0.8298
Epoch 97/100
14/14 [==============================] - 0s 6ms/step - loss: 0.6155 - accuracy:
0.6697 - val_loss: 0.5726 - val_accuracy: 0.7447
Epoch 98/100
14/14 [==============================] - 0s 5ms/step - loss: 0.5938 - accuracy:
0.6697 - val_loss: 0.5741 - val_accuracy: 0.8085

28
 Epoch 99/100
14/14 [==============================] - 0s 5ms/step - loss: 0.6048 - accuracy:
0.6147 - val_loss: 0.5728 - val_accuracy: 0.7447
Epoch 100/100
14/14 [==============================] - 0s 6ms/step - loss: 0.5993 - accuracy:
0.6697 - val_loss: 0.5721 - val_accuracy: 0.7660

[ ]: modeld2.evaluate(X_test, y_test)

2/2 [==============================] - 0s 7ms/step - loss: 0.5856 - accuracy:

0.7500

[ ]: [0.5855724215507507, 0.75]

[ ]: y_pred = modeld2.predict(X_test).reshape(-1)
print(y_pred[:10])
# round the values to nearest integer ie 0 or 1
y_pred = np.round(y_pred)
print(y_pred[:10])

2/2 [==============================] - 0s 7ms/step

[0.01930118 0.5140611 0.5122895 0.18600792 0.55231106 0.5317144
0.5179712 0.5591684 0.31827903 0.5558368 ]
[0. 1. 1. 0. 1. 1. 1. 1. 0. 1.]

[ ]: from sklearn.metrics import confusion_matrix , classification_report

print(classification_report(y_test, y_pred))

precision recall f1-score support

False 0.75 0.78 0.76 27

True 0.75 0.72 0.73 25

accuracy 0.75 52
macro avg 0.75 0.75 0.75 52
weighted avg 0.75 0.75 0.75 52

[ ]: plt.plot(hist3.history['accuracy'])
plt.plot(hist3.history['val_accuracy'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()
# summarize hist3 for loss
plt.plot(hist3.history['loss'])
plt.plot(hist3.history['val_loss'])

29
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()

30
[ ]: plt.plot(hist3.history['val_accuracy'])
plt.plot(hist2.history['val_accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('model accuracy')
plt.ylabel(' Validation accuracy')
plt.xlabel('epoch')
plt.legend(['Dropout (0.9,0.5)', 'Dropout (0.5,0.9)', 'Baseline␣
↪(0,0)'],loc='upper left')

plt.show()
print()
plt.plot(hist3.history['accuracy'])
plt.plot(hist2.history['accuracy'])
plt.plot(history.history['accuracy'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['Dropout (0.9,0.5)', 'Dropout (0.5,0.9)', 'Baseline␣
↪(0,0)'],loc='upper left')

plt.show()
print()

31
# summarize hist3 for loss
plt.plot(hist3.history['val_loss'])
plt.plot(hist2.history['val_loss'])
plt.plot(history.history['val_loss'])
plt.title('model loss')
plt.ylabel(' Validation loss')
plt.xlabel('epoch')
plt.legend(['Dropout (0.9,0.5)', 'Dropout (0.5,0.9)', 'Baseline␣
↪(0,0)'],loc='upper left')

plt.show()
print()
# summarize hist3 for loss
plt.plot(hist3.history['loss'])
plt.plot(hist2.history['loss'])
plt.plot(history.history['loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['Dropout (0.9,0.5)', 'Dropout (0.5,0.9)', 'Baseline␣
↪(0,0)'],loc='upper left')

plt.show()

32
33
34
Conclusion:

We have 3 models,
MODEL A (without dropout [0,0]):
• Loss: 1.0048
• Accuracy: 0.7500
MODEL B (with dropout [0.5, 0.9]):
• Loss: 0.5050
• Accuracy: 0.7885
MODEL C (with dropout [0.9, 0.5]):
• Loss: 0.5856
• Accuracy: 0.7500
Since our baseline model is overfit, we use dropout a regulization technique to overcome
overfitting.

35
Answer:
MODEL B has the lowest loss and a relatively high accuracy compared to the other models. Model
B seems to perform better than the Models A and C.
HOW?
• dropout(0.5) - This dropout layer will randomly set 50% of the input units to zero during
training, keeping 50% active. [Moderate dropout rate]
• dropout(0.9) - This dropout layer will randomly set 90% of the input units to zero during
training, keeping only 10% active. [Relatively high dropout rate]
Dropout layers help in regularizing the model by reducing the reliance on specific neu-
rons during training, leading to better performance on new data.
Why should one not use dropout in the validation/testing phase?

• During training, Dropout randomly deactives neurons, which effectively creates an ensemble
of slightly different models. This ensemble helps reduce overfitting.
• However, during validation or testing, you want the model to make consistent predictions
without randomness.
• Using dropout during these phases would introduce randomness into predictions, leading to
inconsistent results.

36