Introduction to Information theory recall that the entropy of a discrete random variable x is defIned as h(x) ∆= − x x p(x) log p(x) = − e(log p(x)),