DADS402: Unstructured Data Analysis Manipal University Jaipur (MUJ)

MASTER OF BUSINESS ADMINISTRATION

SEMESTER 4

DADS402
UNSTRUCTURED DATA ANALYSIS

Unit 6 : Topic Modelling 1

DADS402: Unstructured Data Analysis Manipal University Jaipur (MUJ)

Unit 6
Topic Modelling
Table of Contents

SL Fig No / Table SAQ /

Topic Page No
No / Graph Activity
1 What is Topic Modelling? - - 3

2 Examples of Topic Modelling - - 4

3 Topic Modelling Vs Topic Classification - - 5
4 Latent Semantic Analysis (LSA) - - 6-8
5 Latent Dirichlet Allocation (LDA) - - 9 - 11

Unit 6 : Topic Modelling 2

DADS402: Unstructured Data Analysis Manipal University Jaipur (MUJ)

Introduction:
Topic modelling is a technique used in natural language processing (NLP) and machine
learning to identify and extract latent topics or themes within a collection of textual data.
The goal is to discover the underlying structure and meaning of the text, by grouping
together similar words and phrases that are likely to appear in the same context.

The most commonly used algorithm for topic modelling is Latent Dirichlet Allocation (LDA),
which assumes that each document is a mixture of topics, and each topic is a probability
distribution over a set of words. LDA works by iteratively assigning words to topics and
adjusting the probabilities until the model converges.

Topic modelling can be applied in various domains, such as social media analysis, content
recommendation, and customer feedback analysis. It allows researchers and analysts to gain
insights into the main themes and trends within a large volume of unstructured text data,
and to explore the relationships and patterns between different topics.

Learning Objectives:
By the end of unit 6, the learners should be able to:

• Define Topic Modeling.

• Know the pre- requisites to understand topic modeling.
• Learn why topic categorization methods are interpreted as supervised machine
learning techniques.
• Understand the concept of topic modeling using examples.
• Understand the concept of Topic Classification.
• Differentiate the concept of Topic Modeling and Topic Classification.
• Know the concept of Latent Semantic Analysis (LSA) and Latent Dirichlet
Allocation(LDA).

Unit 6 : Topic Modelling 3

DADS402: Unstructured Data Analysis Manipal University Jaipur (MUJ)

1. WHAT IS TOPIC MODELLING?

A machine learning approach called topic modelling automatically analyses text data to
identify word clusters for a collection of texts. Since it doesn't need a pre-existing list of
training data or tags that has now been classified by humans, this kind of machine learning
mechanism is known as "unsupervised."

Topic modelling is a rapid and straightforward technique to begin data analysis because it
doesn't require training. However, there is no assurance that the answers you get will be
accurate, that is the reason many companies choose to spend time training a topic
categorization model.

Topic categorization methods are interpreted as "supervised" machine learning techniques

as they require training. Why does that matter? Contrary to text modelling, topic
categorization wants knowledge of a set of texts' themes prior to analysis.

Data is manually attached with these themes so that a topic classifier can learn from it and
subsequently generate predictions on its own.

Say, for instance, that you work for a software company, and you want to examine client
feedback on a new data analysis tool you recently published. The first step would be to
compile a list of categories (themes) that apply to the new functionality, for example. You
would next require using data samples to line up your topic classifier on exactly how to tag
each text applying these predetermined subject tags like Data Analysis, Features & User
Experience.

Although topic classification necessitates more work, it generates more accurate results than
unsupervised methods, which means you'll gain access to more useful insights that would
aid in the development of data-driven decisions.You might say that unsupervised
approaches are a temporary repair, but supervised techniques are more of a long-term
remedy that will aid in the expansion of your company.

Unit 6 : Topic Modelling 4

DADS402: Unstructured Data Analysis Manipal University Jaipur (MUJ)

2. EXAMPLES OF TOPIC MODELLING

To assist you better understand the distinctions between topic categorization and automatic
topic modelling, let's look at a few examples below

By identifying patterns and recurrent words, topic modelling can be used to define the topics
of a collection of customer evaluations. Let's look at how the Eventbrite review below might
be grouped using an “unsupervised" method, for instance:

When you aren't charging for the event, Eventbrite is nice because it is free to use. If you
intend to charge for the event, there would be a fee of 7.5% + $0.98 each transaction.

Topic modelling can associate this review with other reviews that discuss related topics by
detecting phrases and terms like "free to use," "charge," "charging," and "7.5% plus 98 cents
transaction fee" (these may or may not be about pricing).

A topic classification model might also be used to find out what topics’ customers are
discussing via open-ended survey responses, social media posts, and customer reviews, to
mention a few. These supervised procedures, however, take a different tack. Classification
models can automatically tag a review with specified topic tags rather than attempting to
infer to which similarity cluster it belongs. Consider this assessment of SurveyMonkey:

We use our gold level plan extensively and adore its features. It offers the greatest value for
the money.

This review would be categorized under the topics Features and Price by a topic
classification machine that has been trained to comprehend the expressions "gold level
plan," "love the features," and "best bang for the buck."

In summary, topic modelling algorithms produce collections of expressions and words that
they believe to be related, leaving you to decipher the meaning of these relationships,
whereas topic classification algorithms produce topics that are neatly packaged, with labels
like Price and Features that take the guesswork out of the equation.

Short Answer Questions:

Unit 6 : Topic Modelling 5

DADS402: Unstructured Data Analysis Manipal University Jaipur (MUJ)

1. In topic modelling, the goal is to identify and extract ___________ within a collection of
textual data.
2. The most used algorithm for topic modelling is ___________.
3. Topic modelling can be applied in various domains, such as ___________.
3. TOPIC MODELLING VS TOPIC CLASSIFICATION

One thing combines topic modelling and topic classification. They are the methods for topic
analysis that are most frequently applied. Apart from that, they are both highly different, and
your decision between them will likely to be influenced by several variables.

Theoretically, supervised algorithms demand more manual input than unsupervised

machine learning algorithms like topic modelling. This is so they don't require manual data
tags or human training. They do, however, require high-quality data, and not only that, but
they require it in huge quantities, which may not always be simple to obtain.

You will obtain collections of documents that the algorithm has grouped together as well as
groups of words and expressions that it used to infer these relations at the conclusion of your
topic modelling investigation.

On the other hand, supervised machine learning algorithms provide beautifully packaged
findings with topic labels like Price and UX. They do require more setup time because you
must train them by labelling datasets with a specified list of subjects. However, if you
precisely label your texts and refine your criteria, you will be rewarded with a model that
can correctly categorize unknown books according to their topics as well as useful findings.

You'll probably be satisfied with a topic modelling method if you don't have a lot of time to
analyze texts or if you don't need a fine-grained analysis and merely want to know what
topics several texts are discussing.

However, a topic classification method is preferable if you have a list of predetermined

subjects for a batch of texts and want to automatically categorize them without having to
read each one to acquire precise insights.

Unit 6 : Topic Modelling 6

DADS402: Unstructured Data Analysis Manipal University Jaipur (MUJ)

We'll go into more depth about how each of these machine learning algorithms functions
now that we've clarified the distinctions between topic modelling and topic categorization.
Be prepared for things to get a little more technical.

Short Answer Questions:

4. Topic modelling is an ___________ method for uncovering latent topics within a

collection of text data, whereas topic classification is a ___________ method for assigning
predefined topics to individual documents.
5. In topic modelling, the number and nature of the topics are ___________, whereas in
topic classification, the topics are ___________ and predefined.
6. Topic modelling algorithms such as LDA and NMF use a ___________ approach to
identify latent topics, while topic classification algorithms such as SVM and Naive
Bayes use a ___________ approach to assign labels to documents.

Latent Semantic Analysis (LSA) and Latent Dirichlet Allocation are two topic
modelling techniques (LDA).
4. LATENT SEMANTIC ANALYSIS (LSA)

One of the topic modelling techniques that analysts employ most frequently is latent
semantic analysis (LSA). It is founded on the so-called distributional hypothesis, which
claims that by examining the circumstances in which words are used, it is possible to
understand their semantics.

In other words, according to this theory, two words will have similar semantics if they
frequently appear in comparable circumstances.

Having stated that, LSA calculates the word frequencies throughout the documents and the
entire corpus and assumes that documents with a similar subject matter would generally
have a similar distribution of word frequencies. Each document is then handled as a
collection of words, with any syntactic information (such as word order) and semantic
information (such as the variety of interpretations of a specific word) being disregarded.

Unit 6 : Topic Modelling 7

DADS402: Unstructured Data Analysis Manipal University Jaipur (MUJ)

Tf-idf is the commonly used method for calculating word frequencies. This method calculates
frequencies by considering both the frequency of words in a specific document as well as the
frequency of words across the entire corpus of documents. Regardless of how many times
they appear in a single document, terms with a greater frequency in the entire corpus will be
better candidates for document representations than less common words. Tf-IDF
representations are therefore substantially superior to those that merely consider word
frequencies at the document level.

We may develop a Document-term matrix that displays the tf-idf value for each term in every
single document once tf-idf frequencies have been calculated. Every document in the corpus
will have a row in this matrix, and each term under consideration will have a column.

Fig 1: Representing the row and column in the documents.

Ref: https://monkeylearn.com/blog/introduction-to-topic-modeling/

Utilizing singular value decomposition, this Document-term matrix may be broken down
into the product of three matrices (USV) (SVD). The term "Term-topic matrix" refers to the
U matrix, whereas "Document-topic matrix" refers to the V matrix.

Unit 6 : Topic Modelling 8

DADS402: Unstructured Data Analysis Manipal University Jaipur (MUJ)

Fig 2: Sub-division of the row and column into the various documents.

Ref: https://monkeylearn.com/blog/introduction-to-topic-modeling/

Each singular value, or each of the numbers in the main diagonal of matrix S, will be
considered by LSA as a potential topic found in the documents because linear algebra
ensures that the S matrix will be diagonal.

The more common themes found in our original Document-term matrix can now be obtained
if we maintain the greatest t singular values along with the first t columns of U and the first
t rows of V. We refer to this as a truncated SVD because it does not retain all the singular
values from the original matrix, and we must set the value of t as a hyperparameter in order
to use it for LSA.

Unit 6 : Topic Modelling 9

DADS402: Unstructured Data Analysis Manipal University Jaipur (MUJ)

By examining the vectors that make up the U and V matrices, respectively, it is possible to
evaluate the quality of the topic assignment for each document and the quality of the words
allocated to each subject using various methodologies.

5. LATENT DIRICHLET ALLOCATION (LDA)

The distributional hypothesis (i.e., similar topics use similar words) and the statistical
mixture hypothesis (i.e., documents discuss a variety of topics for which a statistical
distribution can be determined) serve as the foundation for both Latent Dirichlet Allocation
(LDA) and Latent Space Analysis (LSA). Each document in our corpus is mapped using LDA
to a collection of topics that encompass most of its words.

Unit 6 : Topic Modelling 10

DADS402: Unstructured Data Analysis Manipal University Jaipur (MUJ)

LDA assigns subjects to word combinations, such as "best player" for a topic relating to
sports, to map the documents to a list of topics.

This is based on the presumption that word choices and word placements influence the
topics of papers. Again, LDA treats documents as a collection of words and ignores syntactic
information, exactly like LSA. Additionally, it presupposes that each word in the document
can be given a likelihood of being related to a particular topic. Having said that, LDA's
objective is to identify the variety of subjects that a text contains.

LDA thus expects that subjects and documents have the following structure:

Ref: https://monkeylearn.com/blog/introduction-to-topic-modeling/

Fig 3: Representation of topics and documents in LDA models.

Additionally, LDA models a new document in the following manner:

Unit 6 : Topic Modelling 11

DADS402: Unstructured Data Analysis Manipal University Jaipur (MUJ)

Fig 4: Representing the new document.

Ref: https://monkeylearn.com/blog/introduction-to-topic-modeling/

LDA assumes that the distribution of themes inside a document and the distribution of words
within subjects are Dirichlet distributions, which is the major distinction between LDA and
LSA. Since LSA makes no assumptions about distribution, the vector representations of
topics and documents become opaquer.

The document and topic similarity are controlled by two hyperparameters, alpha and beta,
respectively. Each document will have fewer subjects assigned to it if the alpha value is low,
whereas the inverse is true if the alpha value is high. When modelling a topic, a low beta
value will utilize fewer words than a high number, making the topics more comparable
between them.

The number of topics the algorithm will detect must be set as a third hyperparameter when
LDA is used because the algorithm cannot choose the number of topics on its own.

A vector containing the coverage of each topic for the document being modelled is the
algorithm's output. It will resemble this [0.2, 0.5, etc.], where the first value represents the
amount of time spent on the first topic, and so forth. These vectors can help you understand
the topical properties of your corpus when they are properly contrasted.

Unit 6 : Topic Modelling 12

DADS402: Unstructured Data Analysis Manipal University Jaipur (MUJ)

You can consult the original LDA paper for further details on how those probabilities are
calculated, the statistical distributions that the algorithm takes for granted, or how to use
LDA.

You may also want to read about the cosine similarity or other similarity measures to learn
more about how to compare vector representations to gain insights into document similarity
or the distribution of themes over a document corpus. The output vectors of both LSA and
LDA can be compared using each of these comparisons to determine how similar they are.

Short Answer Questions:

7. Latent Semantic Analysis (LSA) is a ___________ technique used to analyze relationships

between a set of documents and the terms they contain.
8. LSA uses a mathematical method called ___________ to create a low-dimensional
representation of the documents and terms.
9. LSA can be used for tasks such as ___________ and ___________. Answer: document
similarity, information retrieval
10. LSA assumes that words with similar meanings will appear in similar ___________, and
therefore can be used to identify ___________ between terms.

Summary

1. We had basic understanding of topic modelling.

2. Using an example, we have understood how topic modelling works.
3. We have taken the review from the SurveyMonkey to define how topic classification
model could be used to determine what customers are talking about in customer
reviews, open-ended survey responses and on social media platforms.
4. We understood Topic Modelling Vs Topic Classification.
5. Taking a scenario, we had a clear understanding of Latent Semantic Analysis (LSA).
6. Similarly, we had a brief understanding of Latent Dirichlet Allocation (LDA).

Unit 6 : Topic Modelling 13

DADS402: Unstructured Data Analysis Manipal University Jaipur (MUJ)

Terminal Questions

1. What is topic modelling?

2. How does topic modelling work?
3. What are some limitations of topic modelling?
4. What are some applications of topic modelling?
5. What is latent dirichlet allocation (LDA)?
6. How does LDA work?
7. What are some applications of LDA?
8. What is latent semantic analysis (LSA)?
9. How does LSA work?
10. What are some applications of LSA?

Answers

Short Answer Answers:

1. Latent topics or themes

2. Latent Dirichlet Allocation (LDA)
3. Social media analysis, content recommendation, and customer feedback analysis.
4. Unsupervised, supervised
5. Unknown, fixed
6. Probabilistic, discriminative
7. Statistical
8. Singular value decomposition (SVD)
9. Document similarity, information retrieval
10. Contexts, semantic relationships

Terminal Answers

1. All about topic modelling: Topic modelling is a statistical technique used in natural
language processing to uncover hidden topics or themes within a collection of text
documents. ( Refer to section 1 for more details).

Unit 6 : Topic Modelling 14

DADS402: Unstructured Data Analysis Manipal University Jaipur (MUJ)

2. Procedure of topic modelling: Topic modelling algorithms such as Latent Dirichlet

Allocation (LDA) and Non-negative Matrix Factorization (NMF) identify patterns in
the distribution of words across documents to generate a set of topics. These topics
are represented as a probability distribution over words, and each document is
assumed to be a mixture of these topics. (refer to section 1 for more details).
3. Limitations of topic modelling: Some limitations of topic modelling include the
difficulty of interpreting the output in some cases, the need for a large corpus of text
to generate accurate results, and the potential for overfitting or underfitting the data.
Additionally, topic modelling assumes that each document is a mixture of a fixed
number of topics, which may not always hold true in practice. (Refer to section 1 for
more details).
4. Applications of topic modelling: Topic modelling can be used for tasks such as
document clustering, information retrieval, and content recommendation. It has
applications in fields such as social media analysis, customer feedback analysis, and
scientific literature analysis. (Refer to section 1 for more details).
5. All about LDA: Latent Dirichlet Allocation (LDA) is a probabilistic generative model
used in natural language processing to discover latent topics or themes within a
collection of text documents. (Refer to section 5 for more details).
6. Working of LDA: LDA assumes that each document in a collection is a mixture of a
small number of topics, and that each topic is a probability distribution over a set of
words. The algorithm iteratively assigns words to topics and adjusts the probabilities
until the model converges. The output of LDA is a set of topics, each represented as a
probability distribution over words. (Refer to section 5 for more details).
7. Applications of LDA: LDA can be used for various tasks, such as topic modeling,
document clustering, information retrieval, and content recommendation. It can be
applied in domains such as social media analysis, customer feedback analysis, and
scientific literature analysis. (Refer to section 5 for more details).
8. All about LSA: Latent Semantic Analysis (LSA) is a statistical method used in natural
language processing to uncover the hidden or latent relationships between terms and
documents in a large corpus of text. (Refer to section 4 for more details).

Unit 6 : Topic Modelling 15

DADS402: Unstructured Data Analysis Manipal University Jaipur (MUJ)

9. Working of LSA: LSA uses singular value decomposition (SVD) to convert a large
matrix of term-document frequencies into a lower-dimensional space, where the
rows represent the documents and the columns represent the terms. This allows LSA
to identify the underlying semantic relationships between terms and documents,
even when they do not share exact word matches. (Refer to section 4 for more
details).
10. Applications of LSA: LSA has a wide range of applications in information retrieval,
document classification, and text mining. It can be used for tasks such as document
clustering, topic modeling, and recommendation systems. It has also been applied to
fields such as biology, chemistry, and finance. (Refer to section 4 for more details).

References:
1. Topic Modeling: An Introduction (monkeylearn.com)
2. The Complete Practical Guide to Topic Modelling | by Kajal Yadav | Towards Data
Science
3. Topic Modeling: Algorithms, Techniques, and Application - DataScienceCentral.com

Unit 6 : Topic Modelling 16