Department of Computer Science and Engineering

(Artificial Intelligence and Machine Learning)

NEURAL NETWORKS AND DEEP LEARNING(21AI75)

Presentation Report

Manoj M
4SF21AD026
Under the Guidance of
Dr. Gurusiddayya Hiremath
Associate Professor
Department of Computer Science and Engineering
(Artificial Intelligence and Machine Learning)
SCEM, Mangaluru
Academic Year: 2024-25
Recurrent Neural Networks (RNNs)
1. Definition:
RNNs are a class of neural networks designed for sequential data. Unlike feedforward
networks, they have connections forming directed cycles, enabling them to retain
information from previous inputs.

2. Importance:

o Suited for time-series data, natural language processing (NLP), and speech
recognition.

o Can model temporal dependencies in data.

3. Key Feature:

o Use of hidden states that evolve over time.

Types of RNNs
1. One-to-One

• Description:
This is the simplest type of neural network, where there is a single input and a single
output. Although not strictly an RNN, this structure is included as the base case.

• Example Applications:

o Image classification (e.g., classifying an image into one of several categories).

• Key Characteristics:

o No temporal dependencies.

o Can be implemented using a simple feedforward network instead of RNN.

• Description:
In this structure, a single input generates a sequence of outputs. The input is processed,
and the network predicts multiple outputs over time.

• Example Applications:

o Image Captioning: A single image input generates a sequence of words as output

o Music Generation: A single seed note generates a series of subsequent notes.

• Key Characteristics:

o Useful for generating sequences from static data.

o Requires the network to learn how to expand a single input into a meaningful
sequence.

Many-to-One

• Description:
In this structure, a sequence of inputs produces a single output. The network processes
the entire input sequence and generates a consolidated result at the end.

• Example Applications:

o Sentiment Analysis: A sequence of words (text) is analyzed to produce a single

o Speech Recognition: A spoken phrase is processed to identify the intended

• Key Characteristics:

o Focuses on summarizing information from a sequence.

o The final hidden state carries the combined information from all prior inputs.
Many-to-Many (Two Variants)

This structure processes sequences where both input and output are sequences. It can be further
categorized into:

a. Many-to-Many (Equal Length)

• Description:
The number of outputs matches the number of inputs. Each input corresponds to a
specific output.

• Example Applications:

o Video Frame Labeling: Each frame of a video is tagged with a specific label
(e.g., identifying objects in each frame).

• Key Characteristics:

o The network must maintain a strict mapping between input and output sequences.

b. Many-to-Many (Unequal Length)

• Description:
The input and output sequences differ in length. The network learns to align and process
sequences of varying sizes.

• Example Applications:

o Machine Translation: A sentence in one language is translated into a sentence in

o Speech-to-Text: A continuous audio stream is converted into text.

• Key Characteristics:

o Requires alignment mechanisms like attention for effective learning.

Applications of Different RNN Types

1. One-to-Many:
 o Creative applications like music or poetry generation.

2. Many-to-One:

o Classification tasks (e.g., email spam detection, video summarization).

3. Many-to-Many:

o Complex sequence mapping tasks like translation or chatbot responses.

Challenges in RNN Architectures

• Vanishing and Exploding Gradients: Difficulty in training RNNs for long sequences
due to unstable gradients.

• Alignment Issues: In many-to-many tasks, aligning inputs and outputs is challenging,

• Computational Complexity: Training sequence models can be time-consuming and

Conclusion

The variety of RNN structures enables them to handle a wide range of sequence-based tasks
effectively. However, their limitations, such as difficulty in handling long-term dependencies,
have led to innovations like LSTMs, GRUs, and attention mechanisms to improve their
performance.
PPT SLIDES:
.
Github(CLASS ASSESSMENT):