UNIT: - 4 NATURAL LANGUAGE PROCESSING

SESSION: - 1 INTRODUCTION TO NATURAL LANGUAGE PROCESSING

1.1 INTRODUCTION
NLP is a combination of computational linguistics, machine learning and deep learning. NLP is one of
the domains of AI.
1.2 WHAT IS NLP?
NLP is branch of AI that enables computers to process human language in the form of text or voice
data and to ‘understand’ its full meaning.
Some NLP tasks are:
1. Speech recognition (speech-to-text) is the task of taking input form of audio or voice data
and converting it into text data. e.g. Google assistant, siri
2. Word sense disambiguation is the selection of the meaning of a word with multiple
meanings.
3. Sentiment analysis attempts to extract subjective qualities – attitudes, emotions, sarcasm,
confusion, suspicion –from text.
4. Natural language generations the opposite of speech recognition where the generated or
processed information is reproduced in the form of human language.
1.3 NLP APPLICATIONS
1. Automatic text summarization
2. Sentiment Analysis
3. Text Classification
4. Smart Assistants
5. Digital phone calls
6. Tracking news reports
7. Cognitive Assistant
8. Prediction of diseases

SESSION: - 2 REVISITING AI PROJECT CYCLE (NLP)

2.1INTRODUCTION

2.2 AI PROJECT CYCLE FOR AN NLP APPLICATION

Every AI based project undergoes the phases of AI Project Cycle. So, if you have to develop an
NLP based application, it will also undergo the AI project cycle.

2.2.1 Sample NLP Problem

4Ws Canvas for NLP problem Complaints Addressing Mechanism.

2.2.2 Data Acquisition

1. Database storing past complaints

2. Customer interaction recordings and the speech-to-text of the recordings
3. Customer emails
2.2.3 Data Exploration

1. Pre-processing:- The pre-processing phase ensures that there is no incomplete or repeated or

missing data.
 Inspection and cleaning. It consist of getting rid of the less useful parts of text.
 Annotation. It identifies the text structure and labels and tags the part-of-speech.
 Normalization. It reduce the number of unique tokens present in the text.
2. Data analysis and visualization:- to analyses data and present it visually to find the important
trends or characteristics of data. It consists of statistically probing, manipulating and
generalizing from the dataset for feature analysis.
2.2.4 Modelling
Normalized, cleaned and reduced text is then fed into NLP based chatbot algorithm.
Depending upon the need of the project, different types of chatbot algorithms are available,
based on which different chatbot models are developed in this stage. The dataset of the NLP
is also divided into training dataset and testing dataset.
2.2.5 Evaluation
Once the models developed in the previous phase are trained and tested, they are
evaluated to be picked as one final choice.

2.3 UNDERSTANDING CHATBOTS

A chatbot is a computer program which mimics conversation between users, usually powered
by artifice intelligence. Chatbots today are available on many websites and apps for user’s
interaction.
A chatbot is a piece of technology that allows a computer program to communicate with
people just like conversing through text messages to accomplish specific tasks using a natural
language, say English.
The first chatbot named Eliza was made in the year 1966 at the MIT AI Laboratory. Its purpose
was to give an accurate simulation of a human conversation.

2.3.1 Types of Chatbots

1. Simple Chatbots
Script chatbots are very simple and have limited capabilities, and are usually rule-based
bots.
2. Smart Chatbots
AI-enabled smart chatbots are designed to simulate near-human interactions with
customers.
3. Hybrid Chatbots
Hybrid chatbots combine the features of both the above types of chatbots. Hybrid
chatbots have some rule-based tasks, and they can also understand intent and context
of the chat.
 Difference between Script Bots and Smart Bots.

Script bot Smart bot

Script bots are easy to make and are less Smart bots are flexible, powerful and are
interactive. more interactive.
Script bots have limited functionality. Smart bots have wide functionality.
Script bots work according to a script which Smart bots are made using codes and work
bots work according to script which is on bigger databases and other resources
programmed/ in built in them. directly.
These chatbots are normally free and are Smart bots learn with more data by
easy to integrate to a messaging platform. themselves as the mechanism of AI is in-
built.

SESSION: - 3 HUMAN VS. COMPUTER-LANGUAGES AND NLP

3.1 INTRODUCTION
NLP refers to a method of artificial intelligence that communicates with an intelligent
system using a natural language such as English. Processing of natural language is
required when you want an intelligent system like robot to perform according to your
instructions.
3.2 HUMAN LANGUAGE VS. COMPUTER LANGUAGE
Computer languages are designed to allow humans to interact with computers. They
communicate logical information but do not consider any emotional information.
Human languages were created to interact with other human beings. They
communicate both logical and emotional information.
3.3 CONCEPTS OF NLP
3.2.1 Arrangement of words and meanings
3.2.2 Multiple meanings of words
3.2.3 Perfect syntax but no meaning
3.2.4 Data processing
3.2.5 Text Normalization
1. Sentence segmentation
2. Tokenization
3. Removing stop words, special Characters and numbers
4. Converting text to a common case