Statistical Physics and Information Theory

Statistical Physics and Information Theory focuses on some of the relationships and the interplay between information theory and statistical physics - a branch of physics that deals with many-particle systems using probabilistic and statistical methods in the microscopic level. The author interlaces the physics and the information-theoretic subjects with each other, rather than giving them in two continuous, separate parts. This makes the relations between information theory and statistical physics more apparent. He also shows that, not only are the relations between information theory and statistical physics interesting academically in their own right, but moreover, they prove useful and beneficial in that they provide information-theorists with new insights and mathematical tools to deal with information-theoretic problems. These mathematical tools sometimes prove a lot more efficient than traditional tools used in information theory, and they may give either simpler expressions for performance analysis, or improved bounds, or both. The author provides examples of the techniques and insights. One example is the use of integrals in the complex plane and the saddle-point method. Another example is the analysis technique of error exponents, which stems from the random energy model, along with its insights about phase transitions. Statistical Physics and Information Theory highlights to the reader techniques that have been used in one branch of science which can be applied effectively in another. The point is that it is not the physics itself that may be useful, it is the way in which physicists use mathematical tools. This will bring new insights to all students and researchers in the field of information theory and communications.