FA 2.

2 – Deep Learning Algorithms

Recurrent Neural Networks
1. Definition
One kind of artificial neural network intended for sequential and time series
data is the recurrent neural network (RNN). These deep learning algorithms are
integrated into programs like Siri, voice search, and Google Translate and are
widely used in language translation, natural language processing (NLP), speech
recognition, and picture captioning. RNNs, as opposed to feedforward and
convolutional neural networks (CNNs), possess a "memory" that enables them to
consider previous inputs when analysing output and current input. RNNs' outputs
are dependent on earlier elements in a sequence, whereas typical neural
networks presume input-output independence. However, when predicting the
future, unidirectional RNNs are unable to account for it.

2. Usage and purpose

Artificial neural networks called recurrent neural networks (RNNs) are utilized
in speech detection and natural language processing. They identify features of
sequential data and employ patterns to forecast future events. RNNs employ
feedback loops to preserve information and are effective in scenarios where
context is crucial. They are especially helpful in language models because the
preceding data determines the word that appears in a phrase. Through the
acquisition of syntax and semantics from their training set, RNNs can mimic
human inventiveness.

Layers of network nodes that process input and forward output make up
artificial neural networks (ANNs), which are built to mimic the structure and
functions of the human brain. Similar to feed-forward neural networks, they can
process information in two directions. However, they loop information back into
the network via feedback loops like backpropagation through time. Sequence-to-
sequence models can benefit from truncated backpropagation using time neural
networks, which restricts the number of time steps in the input sequence.

1
3. Advantages and disadvantages
Sequence modelling tasks including speech recognition, time series analysis,
and natural language processing are well suited for RNNs. They can recognize
connections and relationships within sequences because they can recall
contextual information from earlier inputs. Furthermore, RNNs can handle
variable-length sequences and share parameters, which reduces their memory
and processing needs.

Among the many drawbacks of RNNs are the disappearing and exploding
gradient issues that arise during training. In addition to memory capacity
limitations, bias towards recent data, and challenges with parallelization, these
issues make it more difficult for the network to grasp global context and long-
range dependencies. As a result of sequential data processing, RNNs are longer
to train and have difficulty retaining knowledge over many time steps.

4. Examples of use
Some examples of recurrent neural networks use in the industry include:

1. Sentiment analysis – Putting together training and testing files, importing

datasets, compressing the dataset, constructing the network, fitting and
assessing the model, and visualizing accuracy and loss are all necessary
steps in setting up a Recurrent Neural Network to interpret sentiment from
reviews from your parents' bed and breakfast. The
text_dataset_from_directory Keras utility method is used to load the datasets
during the procedure.
2. Speech recognition – In speech recognition, deep recurrent neural networks
integrate flexible long-range context with numerous layers of representation.
Deep Long Short-Term Memory RNNs obtain the best-ever test set error of
17.7% on the TIMIT phoneme recognition benchmark when trained end-to-
end with regularization. In sequence labelling issues, this method shows
promise.
3. Natural language processing – A branch of computer science and artificial
intelligence known as natural language processing, or NLP, creates and

2
 processes natural language data, mostly using machine learning language
models.

Deep Q-Networks
1. Definition
The first stage in developing a Q-matrix to optimize long-term rewards for an
agent is called Q-Learning. Larger systems with more states and actions,
however, make this approach unfeasible. To solve this problem, Deep Q-
Learning approximates values while preserving their relative importance using a
neural network. By using this method, the neural network receives the initial state
as input and outputs the Q-values for every action that could be taken.

Deep Q-Learning uses a deep neural network in the reinforcement learning to

approximate the Q-function, which establishes the best course of action in a
particular condition. The predicted cumulative reward of adopting a specific
course of action and adhering to a specific policy is represented by the Q-
function. Deep Q-Learning can learn from high-dimensional inputs such as
images or sensor data and handle environments with enormous states and
actions. However, the neural network may have trouble convergent due to the
non-linear character of the Q-function. Stabilizing Q-function updates is aided by
strategies such as target networks and experience replay. Deep Q-Learning has
application in robotics, autonomous cars, and game development.

2. Usage and purpose

An effective algorithm in machine learning and artificial intelligence is deep Q-
learning networks. They work by first executing an initialization procedure that
sets random weights for the neural network and replay memory. After then, the
agent investigates its surroundings by acting in random ways to gain
experiences, accumulating a wide range of encounters for training. Next, the
agent balances exploration and exploitation when choosing an action based on a
policy. In order to update its Q-values, the agent engages with the environment
by acting and gathering feedback. For use in upcoming training, the agent saves
the experience in the replay memory. By using gradient descent to minimize the
difference between the target and anticipated Q-values, the neural network is

3
 trained to become more adept at making decisions. To stabilize the training
process, the target network is updated on a regular basis to match the
parameters of the main neural network. Iteratively repeating the process enables
the agent to become more adept at making decisions. By training the agent to
predict optimal Q-values for every state-action pair, reinforcement learning
improves performance in the given environment and yields bigger rewards.

3. Advantages and disadvantages

Compared to typical machine learning techniques, deep learning algorithms
have several benefits, such as the capacity to handle huge and complicated
datasets, automatic feature learning, enhanced performance, non-linear
correlations, and the handling of both structured and unstructured data. They can
also manage missing data, forecast future events or trends, and process
sequential data like as text, audio, and time series. Deep learning models are
easily deployable on cloud platforms and edge devices, and they can be scaled
to accommodate growing amounts of data. They are a useful tool for a variety of
applications since they can also adapt well to new circumstances or contexts.

The implementation of Deep Q-Learning is hindered by instability during

training, which causes biased updates and poor convergence. To solve this
problem, researchers suggest using experience replay, which lowers correlation
between subsequent events and increases sample efficiency by storing and
sampling experiences from a replay memory. Furthermore, poor exploration and
exploitation tactics and inappropriate decision-making might result from
overestimating Q-values.

4. Examples of use
Some examples of recurrent neural networks use in the industry include

1. Gaming – By teaching agents to perform at a human-level in challenging

games like Go, Atari 2600, and Dota 2, Deep Q-Learning has completely
transformed the gaming business.

4
 2. Autonomous vehicles – With deep Q-learning, autonomous car performance,
safety, and efficiency can all be greatly enhanced by teaching agents to make
decisions in real time based on real-time information.
3. Other uses – Beyond gaming and driverless cars, deep Q-learning has
potential uses in robotics, finance, healthcare, and recommendation systems
for intelligent decision-making across a range of industries.

Reference:

IBM. (n.d.). What are Recurrent Neural Networks? https://www.ibm.com/topics/

recurrent-neural-networks

AIML.com. (2024). What are the advantages and disadvantages of a Recurrent

Neural Network (RNN)? AIML.com. https://aiml.com/what-are-the-

advantages-and-disadvantages-of-a-recurrent-neural-network-rnn/

GeeksforGeeks. (2023, January 23). Deep Q-Learning. GeeksforGeeks.

https://www.geeksforgeeks.org/deep-q-learning/

Laskowski, N.(2021, July 12). Recurrent neural networks. Enterprise AI.

https://www.techtarget.com/searchenterpriseai/definition/recurrent-neural-

networks#:~:text=Recurrent%20neural%20networks%20recognize

%20data's,activity%20in%20the%20human%20brain.

Bento, C. (2022, May 31). Recurrent Neural Networks Explained with a Real Life

Example and Python Code. Medium. https://towardsdatascience.com/

recurrent-neural-networks-explained-with-a-real-life-example-and-python-

code-e8403a45f5de

5
Kvartalnyi, N. (2023, November 24). Deep Q-Learning Explained:Step-by-Step

Guide. Inoxoft |. https://inoxoft.com/blog/deep-q-learning-explained-a-

comprehensive-guide/