mutual information

Mutual information I(X;Y) between two random variables X and Y is what is left over when their mutual conditional entropies H(Y|X) and H(X|Y) are subtracted from their joint entropy H(X,Y). It can be given by the formula I(X;Y)=-∑ₓ∑_yp_X,Y(x,y) log _bp_X,Y(x,y)/p_X|Y(x|y)p_Y|X(y|x).