ECSE 310 Thermodynamics of Computing
Homework 4
Each question has the same weight
1. a) Using your own words, define the following:
a) Shannon’s measure of information
b) Joint entropy
c) Mutual information
d) Marginal information
e) Conditional entropy
f) Algorithmic entropy
g) Johnson noise
h) Brownian ratchet
i) Differential entropy
j) Probability density function
k) Uncertainty principle
l) Shannon’s coding theorem
m) Microstate and macrostate
2. Consider the probability that parrots are grey is 0.3, and the probability that they are male or
female is equal. However, the male parrots are 1.5 times more likely than the female parrots
to be grey.
a) You learn that non-grey parrots are male, how much information do you gain from
this knowledge?
b) Rank the following uncertainties in descending order:
i. uncertainty about colour
ii. uncertainty about gender
iii. uncertainty about colour, provided that a parrot is male
iv. uncertainty about gender, provided that a parrot is non-grey
3. The average temperatures in Paris, France in the year 2017 were taken from
www.wunderground.com and are given in the attached excel sheet.
a) What is the uncertainty of the temperature?
b) What is the joint entropy of temperatures and months?
c) How much information is shared between temperatures and months?
d) What is the uncertainty given that month is May?
