Entropy and Information Theory
- ISBN-10
- 1475739826
- ISBN-13
- 9781475739824
- Category
- Computers
- Pages
- 332
- Language
- English
- Published
- 2013-03-14
- Publisher
- Springer Science & Business Media
- Author
- Robert M. Gray
Description
This book is devoted to the theory of probabilistic information measures and their application to coding theorems for information sources and noisy channels. The eventual goal is a general development of Shannon's mathematical theory of communication, but much of the space is devoted to the tools and methods required to prove the Shannon coding theorems. These tools form an area common to ergodic theory and information theory and comprise several quantitative notions of the information in random variables, random processes, and dynamical systems. Examples are entropy, mutual information, conditional entropy, conditional information, and discrimination or relative entropy, along with the limiting normalized versions of these quantities such as entropy rate and information rate. Much of the book is concerned with their properties, especially the long term asymptotic behavior of sample information and expected information. This is the only up-to-date treatment of traditional information theory emphasizing ergodic theory.
Get the book
Other editions
Similar books
-
New Foundations for Information Theory: Logical Entropy and Shannon Entropy
By David EllermanRelatively short but dense in content, this work can be a reference to researchers and graduate students doing investigations in information theory, maximum entropy methods in physics, engineering, and statistics, and to all those with a ...
-
Elements of Information Theory
By Thomas M. Cover, Joy A. ThomasThe Second Edition features: Chapters reorganized to improve teaching 200 new problems New material on source coding, portfolio theory, and feedback capacity Updated references Now current and enhanced, the Second Edition of Elements of ...
-
Mathematical Foundations of Information Theory
By Aleksandr I?Akovlevich KhinchinFirst comprehensive introduction to information theory explores the work of Shannon, McMillan, Feinstein, and Khinchin. Topics include the entropy concept in probability theory, fundamental theorems, and other subjects. 1957 edition.
-
Entropy and Information
By Mikhail V. VolkensteinThis is just...entropy, he said, thinking that this explained everything, and he repeated the strange word a few times. 1 ?
-
Information Theory, Inference and Learning Algorithms
By David J. C. MacKayMACKAY , D. J. C. , and NEAL , R. M. ( 1995 ) Good codes based for blind source separation . In ICA : Principles and Practice , ed . by S. Roberts and R. Everson . Cambridge Univ . Press . on very sparse matrices .
-
An Introduction to Transfer Entropy: Information Flow in Complex Systems
By Terry Bossomaier, Lionel Barnett, Michael Harré... −C given in terms of the Euler constant, C, also called the Euler– Mascheroni constant: C = 0.5772156... (3.61) Kozachenko and Leonenko [166] then extend the estimator in Eqn. 3.58 in that the distances between sorted neighbouring ...
-
The Mathematical Theory of Communication
By Warren Weaver, Claude E ShannonScientific knowledge grows at a phenomenal pace--but few books have had as lasting an impact or played as important a role in our modern world as The Mathematical Theory of Communication, published originally as a paper on communication ...
-
Information Theory for Data Communications and Processing
By Shlomo Shamai, Abdellatif ZaidiThis book, composed of a collection of papers that have appeared in the Special Issue of the Entropy journal dedicated to “Information Theory for Data Communications and Processing”, reflects, in its eleven chapters, novel contributions ...
-
Relative Information: Theories and Applications
By Guy JumarieSome of the most convincing arguments in this regard are in cluded in Fred Dretske's Know/edge & Flow of Information (The M.LT. Press, Cambridge, Mass., 1981) and in this book by Guy lumarie.
-
A First Course in Information Theory
By Raymond W. YeungLangdon, G.G., 396 Lapidoth, A., xv lATEX, xvi lattice theory, 124 Lauritzen, S.L., 396 laws of information theory, xiii, 264,321, 325 leaf, 46.46–57 Lebesgue measure, 278, 305 Lee, J.Y.-B., xvi Lee, T.T., 147, 401 Leibler, R.A., 39, ...