Scribd
Upload
15 views22 pages

1.4discrete Memoryless Channels

Uploaded by

Kavi Sanjai
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
15 views22 pages

1.4discrete Memoryless Channels

Uploaded by

Kavi Sanjai
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 22

DIGITAL COMMUNICATION

–INFORMATION THEORY
DISCRETE MEMORYLESS CHANNELS
Dr.G.Suchitra,
Assistant Professor,
Department of ECE,
Government College of Technology,
Coimbatore
COMMUNICATION SYSTEM[5]
DISCRETE MEMORYLESS CHANNEL[1]
⚫ It is a statistical model
⚫ Input symbol xi from an alphabet X
⚫ Output symbol yj from an alphabet Y: Noisy version of X
⚫ X and Y are random variables
⚫ Channel is discrete when alphabets in X and Y are of finite
sizes
⚫ Channel is memoryless when present output symbol
depends only on the present input symbol and not on the
previous input symbols
⚫ Graphical
Representation
Notations used[1]
⚫ Input alphabets (Size:J)
X = {x0 , x1 , …, xJ-1 }
⚫ Output alphabets(Size K)
Y = {y0 , y1 , …, yK-1 }
⚫ Transition probabilities for all ‘j’ and ‘k’

⚫ Input and output alphabets need not be of the same


size
DISCRETE MEMORYLESS CHANNEL MATHEMATICAL
REPRESENTATION[1]
CHANNEL MATRIX OR TRANSITION MATRIX
⚫ Each Row corresponds to a fixed channel input
⚫ Each column corresponds to a fixed channel output
⚫ Sum of the elements along any row of the matrix is equal
to one.

⚫ P(yk/xj): Probability that yk is received when xj is


transmitted
CHANNEL MATRIX OR TRANSITION MATRIX [1]
⚫ Inputs to the discrete memory less channel are selected
according to the probability distribution,

⚫ Also known as apriori probabilities of the input symbols

⚫ Joint probability distributions of the random variables X and Y


is,

⚫ Based on Baye’s theorem,


[1]
⚫ Marginal probability distribution of the output random
variable Y:

Marginal probability distribution of the input random variable X:


[1]
⚫ If we are given the input prior probabilities p(xj) and
the stochastic matrix (i.e., the matrix of transition
probabilities p(yk|xj)), then we may calculate the
probabilities of the various output symbols, the p(yk).
⚫ Conditional probability link:
⚫ P(B/A): Event A has already happened; Now what is
the chance of getting event B
⚫ P(A and B) = P(A) P(B/A)
⚫ https://www.mathsisfun.com/data/probability-events
-conditional.html
Different types of channels[5]
⚫ Noiseless binary ⚫ Noisy Channel with
channel Nonoverlapping Outputs
⚫ Input can be recovered from the
output
Noisy Typewriter[5]
⚫ In this case the channel input
is either received unchanged
at the output with probability
½ or is transformed into the
next letter withprobability ½
⚫ If the input has 26 symbols
and we use every alternate
input symbol, we can
transmit one of 13 symbols
without error with each
transmission.
Symmetric channel[2]
⚫ A channel is said to be
symmetric if each row of
the channel matrix
contains the same set of
numbers and each column
contains the same set of
numbers.
⚫ The row and columns are
identical except with some
permutations.
A discrete memory-less channel is SYMMETRIC if
⚫ For all X, the branches leaving the code symbol have the
same set of probabilities, p1, p2,…,pj
⚫ For all Y , the branches entering the received symbol have
the same set of probabilities, p1, p2,…,pj
⚫ Seen from the transition matrix all rows are permutations
of each other and all columns are permutations of each
other.
Binary Symmetric channel(BSC)[2]
⚫ If the source alphabet is
symmetric and channel
is symmetric, then it is a
BSC channel.
⚫ p(0)=α; p(1)=1-α
⚫ Symmetry means that
Channel Matrix the probability of error is
independent of
transmitted symbol.
EXAMPLE OF BSC[5]
⚫ Rows of the probability ⚫ Channel is said to be
transition matrix are Weakly symmetric if every
permutations of each other row of the transition matrix
and so are the columns p(·|x) is a permutation of
every other row and all the
column sums of p(y|x) are
equal.
Binary Erasure Channel(BEC)[2]
⚫ Channel has two inputs {0,1}
⚫ Three output symbols :{0,e,1}
⚫ Information processed
by a channel depends on
the input distribution
p(xj)
REFERENCES
1. “Digital Communication”, Simon Haykin, Wiley India,
2. “Statistical theory of Communication”, S.P.Eugene
Xavier, New Age International Publishers.
3. “Modern Analog and Digital Communication”, B.P.Lathi,
Oxford University Press,
4. https://www.eit.lth.se/fileadmin/eit/courses/eit080/InfoThe
orySH/InfoTheoryPart2b.pdf
5. T. Cover and J. Thomas, Elements of Information Theory,
Second edition, Wiley, 2006
THANK YOU

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy