MATH590: Information theory (INF)

Module overview

Information theory arose from the study of communication operations in Claude Shannon's paper A Mathematical Theory of Communication, documenting research carried out at Bell Labs. Shannon applied recent developments from probability theory developed principally by Norbert Wiener. Later, E.T. Jaynes used this approach to rederive statistical physics from maximum entropy principles.

• Point-to-point communication

• Quantities of information (QIs)

• Maximal entropy principle

• Statistical physics

• Entropy rates of stochastic processes

• Fisher information