The Picture

Figure

  1. MATH, the expected Joint Information in $\QTR{Large}{X}$ and $\QTR{Large}{Y.}$ The Information in $\QTR{Large}{X}$ or in $\QTR{Large}{Y}$

  2. MATHthe red thatched area minus the blue, the expected Information left in $\QTR{Large}{X}$ after we learn about $\QTR{Large}{Y}$ . MATH

  3. MATHthe blue thatched area minus the red, the expected Information left in $\QTR{Large}{Y}$ after we learn about $\QTR{Large}{X}$ . MATH

  4. MATH is the expected Mutual, Information in $\QTR{Large}{X}$ and $\QTR{Large}{Y.}$ If I have Information about $\QTR{Large}{X\ }$ then I also have some Information about $\QTR{Large}{Y}$ MATH.

  5. MATH: Given a Channel , by modifying $\QTR{Large}{X\ }$s distribution, how big can I make the relative overlap, the shared Information.