Information theory and statistics / Теория информации и статистика

Information theory, as we shall be concerned with it, is a branch of the mathematical theory of probability and statistics. As such, its abstract formulations are applicable to any probabilistic or statistical system of observations. Consequently, we find information theory applied in a variety of fields, as are probability and statistics. It plays an important role in modern communication theory, which formulates a communication system as a stochastic or random process. Tuller (1950) remarks that the statistical theory of communications is often called information theory. Rothstein (1951) has defined information theory as “abstract mathematics dealing with measurable sets, with choices from alternatives of an unspecified nature.” Pierce (1956, p. 243) considers communication theory and information theory as synonyms. Gilbert (1958, p. 14) says, “Information will be a measure of time or cost of a sort which is of particular use to the engineer in his role of designer of an experiment.” The essential mathematical and statistical nature of information theory has been reemphasized by three men largely responsible for its development and stimulation, Fisher (1956), Shannon (1956), Wiener (1956). <...>