Swami Keshvanand Institute of Technology, Management &Gramothan,

Ramnagaria, Jagatpura, Jaipur-302017, INDIA

Approved by AICTE, Ministry of HRD, Government of India
Recognized by UGC under Section 2(f) of the UGC Act, 1956
Tel. : +91-0141- 5160400Fax: +91-0141-2759555
E-mail: info@skit.ac.in Web: www.skit.ac.in

A
Course File
on
(Information Theory & Coding, 5CS3-01)
Programme: B.Tech (CSE)
Semester: Vth
Session: 2021-22

(Shanu Tripathi)
(Assistant Professor)
(Computer Science and Engineering)

Page | 1
 RAJASTHAN TECHNICAL UNIVERSITY, KOTA
Syllabus
III Year-V Semester: B.Tech. Computer Science and Engineering

5CS3-01: Information Theory & Coding

Credit: 2 Max. Marks: 100(IA:20, ETE:80)

2L+0T+0P End Term Exam: 2 Hours
SN Contents Hours

1 Introduction:Objective, scope and outcome of the course. 01

2 Introduction to information theory: Uncertainty, Information

and Entropy, Information measures for continuous random 05
variables, source coding theorem. Discrete Memory less channels,
Mutual information, Conditional entropy.
3 Source coding schemes for data compaction: Prefix code,
Huffman code, Shanon-Fane code &Hempel-Ziv coding channel 05
capacity. Channel coding theorem. Shannon limit.
4 Linear Block Code: Introduction to error connecting codes, coding
& decoding of linear block code, minimum distance consideration, 05
conversion of non-systematic form of matrices into systematic
form.
5 Cyclic Code: Code Algebra, Basic properties of Galois fields (GF)
polynomial operations over Galois fields, generating cyclic code by
generating polynomial, parity check polynomial. Encoder & 06
decoder for cyclic codes.

6 Convolutional Code: Convolutional encoders of different rates.

Code Tree, Trllis and state diagram. Maximum likelihood decoding 06
of convolutional code: The viterbi Algorithm fee distance of a
convolutional code.
Total 28

Syllabus of 3rdYear B. Tech. (CS) for students admitted in Session 2017-18 onwards. Page 2
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Q.2
 TUTORIAL SHEET-2
Q.1

Q.2

Q.3
 Swami Keshvanand Institute of Technology, Management &Gramothan,
Ramnagaria, Jagatpura, Jaipur-302017, INDIA
Approved by AICTE, Ministry of HRD, Government of India
Recognized by UGC under Section 2(f) of the UGC Act, 1956
Tel. : +91-0141- 5160400Fax: +91-0141-2759555
E-mail: info@skit.ac.in Web: www.skit.ac.in

Question Bank (Unit or Chapter Wise)

UNIT NO.1
1. The unit of average mutual information is
a) Bits
b) Bytes
c) Bits per symbol
d) Bytes per symbol
Answer: a

2. When probability of error during transmission is 0.5, it indicates that

a) Channel is very noisy
b) No information is received
c) Channel is very noisy & No information is received
d) None of the mentioned
Answer: c
3. When the base of the logarithm is 2, then the unit of measure of information is
a) Bits
b) Bytes
c) Nats
d) None of the mentioned
Answer: a

4. When X and Y are statistically independent, then I (x,y) is

a) 1
b) 0
c) Ln 2
d) Cannot be determined
Answer: b

5. The self information of random variable is

a) 0
b) 1
c) Infinite
d) Cannot be determined
Answer: c

Page | 1
 Swami Keshvanand Institute of Technology, Management &Gramothan,
Ramnagaria, Jagatpura, Jaipur-302017, INDIA
Approved by AICTE, Ministry of HRD, Government of India
Recognized by UGC under Section 2(f) of the UGC Act, 1956
Tel. : +91-0141- 5160400Fax: +91-0141-2759555
E-mail: info@skit.ac.in Web: www.skit.ac.in

UNIT NO.2
1. Binary Huffman coding is a
a) Prefix condition code
b) Suffix condition code
c) Prefix & Suffix condition code
d) None of the mentioned
Answer: a

2. The event with minimum probability has least number of bits.

a) True
b) False
Answer: b

3. The method of converting a word to stream of bits is called as

a) Binary coding
b) Source coding
c) Bit coding
d) Cipher coding
Answer: b

4. Entropy of a random variable is

a) 0
b) 1
c) Infinite
d) Cannot be determined
Answer: c

5. Which is more efficient method?

a) Encoding each symbol of a block
b) Encoding block of symbols
c) Encoding each symbol of a block & Encoding block of symbols
d) None of the mentioned
Answer: b

Page | 2
 Swami Keshvanand Institute of Technology, Management &Gramothan,
Ramnagaria, Jagatpura, Jaipur-302017, INDIA
Approved by AICTE, Ministry of HRD, Government of India
Recognized by UGC under Section 2(f) of the UGC Act, 1956
Tel. : +91-0141- 5160400Fax: +91-0141-2759555
E-mail: info@skit.ac.in Web: www.skit.ac.in

UNIT NO.3
1. Which among the following is/are the essential condition/s for a good error control coding technique?
a. Faster coding & decoding methods
b. Better error correcting capability
c. Maximum transfer of information in bits/sec
d. All of the above
Answer: d

2. Which among the following represents the code in which codewords consists of message bits and parity
bits separately?
a. Block Codes
b. Systematic Codes
c. Code Rate
d. Hamming Distance
Answer: b

3. In a linear code, the minimum Hamming distance between any two code words is ______minimum
weight of any non-zero code word.
a. Less than
b. Greater than
c. Equal to
d. None of the above
Answer: c

4. The minimum distance of linear block code (dmin) is equal to minimum number of rows or columns of
HT, whose _____ is equal to zero vector.
a. sum
b. difference
c. product
d. divison
Answer: a

Page | 3
 Swami Keshvanand Institute of Technology, Management &Gramothan,
Ramnagaria, Jagatpura, Jaipur-302017, INDIA
Approved by AICTE, Ministry of HRD, Government of India
Recognized by UGC under Section 2(f) of the UGC Act, 1956
Tel. : +91-0141- 5160400Fax: +91-0141-2759555
E-mail: info@skit.ac.in Web: www.skit.ac.in

UNIT NO.4
1.Basically, Galois field consists of ______ number of elements.
a. Finite
b. Infinite
c. Both a and b
d. None of the above
Answer: a

2.According to linearity property, the ________ of two code words in a cyclic code is also a valid code
word.
a. sum
b. difference
c. product
d. division
Answer: a

3.What is the value of leading coefficient of a monic polynomial?

a. 0.5
b. 1
c. 4
d. 16
Answer: b

4. In decoding of cyclic code, which among the following is also regarded as 'Syndrome Polynomial'?
a. Generator Polynomial
b. Received code word Polynomial
c. Quotient Polynomial
d. Remainder Polynomial
Answer: d

Page | 4
 Swami Keshvanand Institute of Technology, Management &Gramothan,
Ramnagaria, Jagatpura, Jaipur-302017, INDIA
Approved by AICTE, Ministry of HRD, Government of India
Recognized by UGC under Section 2(f) of the UGC Act, 1956
Tel. : +91-0141- 5160400Fax: +91-0141-2759555
E-mail: info@skit.ac.in Web: www.skit.ac.in

UNIT NO.5
1.At any given time, the output of an encoder depends on ______
a. Past input
b. Present input
c. Both a and b
d. None of the above
Answer: c

2. While representing the convolutional code by (n,k,m), what does 'm' signify or represent in it?
a. Coded bits
b. Message bits
c. Memory order
d. All of the above
Answer: c

3. During the shifting of bits in an encoder, how are the transitions in the states represented?
a. By lines
b. By circles
c. By summers
d. By squares
Answer: a

4.In Viterbi's algorithm, which metric is adopted for decision making?

a. Hamming distance
b. Galois Field
c. Hamming bound
d. Parity-check
Answer: a

Page | 5
 Swami Keshvanand Institute of Technology, Management &Gramothan,
Ramnagaria, Jagatpura, Jaipur-302017, INDIA
Approved by AICTE, Ministry of HRD, Government of India
Recognized by UGC under Section 2(f) of the UGC Act, 1956
Tel. : +91-0141- 5160400Fax: +91-0141-2759555
E-mail: info@skit.ac.in Web: www.skit.ac.in

List of Text and Reference Books

Text Books

1. R. Bose, Information theory, coding and cryptography, McGraw‐Hill, 3 rd Edition, 2016.

2. Simon Haykin: Digital Communication Systems, Wiley India, 2013.

Reference Books:

1. P S Sathya Narayana, Concepts of Information Theory & Coding, Dynaram Publications, 2005
2. R. M. Roth, Introduction to Coding Theory, Cambridge University Press, 2006.
3. A. B. Robert, Information Theory, Dover Special Priced Titles, 2007.
4. T.M. Cover and J. A. Thomas, Elements of information theory, John Wiley & Sons, 2012.

Page | 1