Answer:
4.2.1 To calculate H(x), we need to use the formula for entropy:
H(x) = -∑ P(xi) * log2(P(xi))
Given the probabilities for the symbols x1, x2, x3, and x4 as follows:
P(x1) = 0.4
P(x2) = 0.3
P(x3) = 0.2
P(x4) = 0.1
Let's calculate H(x):
H(x) = - (P(x1) * log2(P(x1)) + P(x2) * log2(P(x2)) + P(x3) * log2(P(x3)) + P(x4) * log2(P(x4)))
H(x) = - (0.4 * log2(0.4) + 0.3 * log2(0.3) + 0.2 * log2(0.2) + 0.1 * log2(0.1))
Calculating each term:
0.4 * log2(0.4) = 0.4 * (-1.3219) = -0.5288
0.3 * log2(0.3) = 0.3 * (-1.737) = -0.5211
0.2 * log2(0.2) = 0.2 * (-2.3219) = -0.4644
0.1 * log2(0.1) = 0.1 * (-3.3219) = -0.3322
H(x) = - (-0.5288 + (-0.5211) + (-0.4644) + (-0.3322))
H(x) = - (-1.8465)
H(x) = 1.8465
Therefore, H(x) = 1.8465.
4.2.2 To find the information contained in the messages, we can use the formula:
I(xi) = -log2(P(xi))
(i) For the message x1 x2 x3 x4:
I(x1) = -log2(P(x1)) = -log2(0.4) = 1.3219
I(x2) = -log2(P(x2)) = -log2(0.3) = 1.737
I(x3) = -log2(P(x3)) = -log2(0.2) = 2.3219
I(x4) = -log2(P(x4)) = -log2(0.1) = 3.3219
Therefore, the information contained in the message x1 x2 x3 x4 is:
I(x1 x2 x3 x4) = I(x1) + I(x2) + I(x3) + I(x4) = 1.3219 + 1.737 + 2.3219 + 3.3219 = 8.7027
(ii) For the message x4 x3 x3 x2:
I(x4) = -log2(P(x4)) = -log2(0.1) = 3.3219
I(x3) = -log2(P(x3)) = -log2(0.2) = 2.3219
I(x3) = -log2(P(x3)) = -log2(0.2) = 2.3219
I(x2) = -log2(P(x2)) = -log2(0.3) = 1.737
Therefore, the information contained in the message x4