Information theory

From Monoskop
Jump to: navigation, search
Diagram of a general communication system from Claude E. Shannon, A Mathematical Theory of Communication, 1948.

Information theory is a branch of applied mathematics, electrical engineering, and computer science which originated primarily in the work of Claude Shannon and his colleagues in the 1940s. It deals with concepts such as information, entropy, information transmission, data compression, coding, and related topics. Paired with simultaneous developments in cybernetics, and despite the criticism of many, it has been subject to wide-ranging interpretations and applications outside of mathematics and engineering.


Claude E. Shannon, A Mathematical Theory of Cryptography, 1945. View online.
Claude E. Shannon, A Mathematical Theory of Communication, 1948. Download (monograph).
Claude E. Shannon, Warren Weaver, The Mathematical Theory of Communication, 1949. Download (1963 edition).

Shannon's information theory (1948)[edit]

  • Harry Nyquist, "Certain Factors Affecting Telegraph Speed", Journal of the AIEE 43:2 (February 1924), pp 124-130; repr. in Bell System Technical Journal, Vol. 3 (April 1924), pp 324-346. Presented at the Midwinter Convention of the AIEE, Philadelphia, February 1924. Shows that a certain bandwidth was necessary in order to send telegraph signals at a definite rate. Considers two fundamental factors for the maximum speed of transmission of 'intelligence' [not information] by telegraph: signal shaping and choice of codes. Used in Shannon 1948.
  • Harry Nyquist, "Certain Topics in Telegraph Transmission Theory", Transactions of AIEE, Vol. 47 (April 1928), pp 617-644; repr. in Proceedings of the IEEE 90:2 (February 2002), pp 280-305. Presented at the Winter Convention of the AIEE in New York in February 1928. Argues for the steady-state system over the method of transients for determining the distortion of telegraph signals. In this and his 1924 paper, Nyquist determines that the number of independent pulses that could be put through a telegraph channel per unit time is limited to twice the bandwidth of the channel; this rule is essentially a dual of what is now known as the Nyquist–Shannon sampling theorem. Used in Shannon 1948.
  • Ralph V.L. Hartley, "Transmission of Information", Bell System Technical Journal 7:3 (July 1928), pp 535-563. Presented at the International Congress of Telegraphy and Telephony, Lake Como, Italy, September 1927. Uses the word information as a measurable quantity, and opts for logarithmic function as its measure, when the information in a message is given by the logarithm of the number of possible messages: H = n log S, where S is the number of possible symbols, and n the number of symbols in a transmission. Used in Shannon 1948.
  • Claude E. Shannon, A Mathematical Theory of Cryptography, Memorandum MM 45-110-02, Bell Laboratories, 1 September 1945, 114 pages + 25 figures; repr. in Shannon, Miscellaneous Writings, eds. N.J.A. Sloane and Aaron D. Wyner, AT&T Bell Laboratories, 1993. Classified. Redacted and pubished in 1949 (see below). Shannon's first lengthy treatise on the transmission of "information." [1]
  • Claude E. Shannon, "A Mathematical Theory of Communication", Bell System Technical Journal 27 (July, October 1948), pp 379-423, 623-656. Reprinted as Monograph B-1598, Bell Telephone System Technical Publications, 83 pp; repr. December 1957.
    • "Statisticheskaia teoriia peredachi elektricheskikh signalov", in Teoriya peredakhi elektrikheskikh signalov pri nalikhii pomekh, ed. Nikolai A. Zheleznov, Moscow: IIL, 1953. (in Russian, details below)
    • "Matematicheskaya teoriya svyazi", trans. S. Karpov, in Raboty po teorii informatsii i kibernetike, Moscow: IIL, 1963, pp 243-332. (in Russian, details below)
  • Robert M. Fano, The Transmission of Information, Technical Reports No. 65 (17 March 1949) and No. 149 (6 February 1950), Research Laboratory of Electronics, MIT. A similar coding technique like Shannon's, only deducted differently. In 1952 optimised by his student, Huffman (see below).
  • Warren Weaver, "The Mathematics of Communication", Scientific American 181:1 (July 1949), pp 11-15; repr. in Basic Readings in Communication Theory, ed. C. David Mortensen, Harper & Row, 1973, pp 27-38. [2]
  • Claude E. Shannon, Warren Weaver, The Mathematical Theory of Communication, Urbana: University of Illinois Press, 1949, 117 pp; 1963; 1969; 1971; 1972; 1975; 1998. Reviews: Hockett (1953, EN). Consists of two texts: Shannon's 1948 paper (pp 3-91), and Weaver's "Recent contributions to the mathematical theory of communication", an edited version of his July 1949 paper (pp 95-117), and . [3] [4]
    • コミュニケーションの数学的理論, trans. Atsushi Hasegawa, 明治図書, 1969; 2001. (Japanese)
    • La teoria matematica delle comunicazioni, trans. Paolo Cappelli, Milan: Etas Kompass, 1971. (Italian)
    • Mathematische Grundlagen der Informationstheorie, Munich and Vienna: Oldenbourg, 1976. (German). Contents.
    • 通信の数学的理論, trans. Uematsu Tomohiko, 筑摩書房, 2009, 231 pp. (Japanese)
  • Claude E. Shannon, "Communication Theory of Secrecy Systems", Bell System Technical Journal 28 (October 1949), pp 656-715; repr. as a monograph, New York: American Telegraph and Telephone Company, 1949. Unclassified revision of Shannon's memorandum A Mathematical Theory of Cryptography, 1945, which was still classified (until 1957). [5]
    • "Teoriya svyazi v sekretnykh sistemakh" [Теория связи в секретных системах], trans. С. Карпов, in Raboty po teorii informatsii i kibernetike [Работы по теории информации и кибернетике], Moscow: IIL (ИИЛ), 1963, pp 333-402. (Russian) [6]
  • David A. Huffman, A Method for the Construction of Minimum-Redundancy Codes", Proceedings of the I.R.E. 40 (September 1952), pp 1098–1102. Fano's student; developed an algorithm for efficient encoding of the output of a source. Later became popular in compression tools.
  • Claude E. Shannon, "Information Theory", Seminar Notes, MIT, from 1956, in Shannon, Miscellaneous Writings, eds. N.J.A. Sloane and Aaron D. Wyner, AT&T Bell Laboratories, 1993.
Norbert Wiener, Cybernetics: or Control and Communication in the Animal and the Machine, 1948. Download (Second edition, 1965).
Norbert Wiener, "The Machine Age", [1949]. Download (v3).
Norbert Wiener, The Human Use of Human Beings: Cybernetics and Society, 1950. Download (1989 edition).

Wiener's cybernetics (1948)[edit]

  • Norbert Wiener, The Extrapolation, Interpolation, and Smoothing of Stationary Time Series, NDRC Report, MIT, February 1942. Classified (ordered by Warren Weaver, then the head of Section D-2), printed in 300 copies. Nicknamed "Yellow Peril". Published in 1949 (see below). Shannon 1948 mentions it as containing "the first clear-cut formulation of communication theory as a statistical problem, the study of operations on time series. This work, although chiefly concerned with the linear prediction and filtering problem, is an important collateral reference in connection with the present paper" (p 626-7). [7], Commentary.
  • Norbert Wiener, Cybernetics: or Control and Communication in the Animal and the Machine, Paris: Hermann & Cie, 1948; Cambridge, MA: Technology Press, 1948; New York: John Wiley & Sons, 1948, 194 pp; 2nd ed., MIT Press and Wiley, 1961, 212 pp; 1965, PDF; 1980. Reviews: Dubarle (1948, FR), Littauer (1949), MacColl (1950). In the spring of 1947, Wiener was invited to a congress on harmonic analysis, held in Nancy, France and organized by the bourbakist mathematician, Szolem Mandelbrojt. During this stay in France Wiener received the offer to write a manuscript on the unifying character of this part of applied mathematics, which is found in the study of Brownian motion and in telecommunication engineering. The following summer, back in the United States, Wiener decided to introduce the neologism ‘cybernetics’ into his scientific theory. According to Pierre De Latil, MIT Press tried their best to prevent the publication of the book in France, since Wiener, then professor at MIT, was bound to them by contract. As a representative of Hermann Editions, M. Freymann managed to find a compromise and the French publisher won the rights to the book. Having lived together in Mexico, Freymann and Wiener were friends and it is Freymann who is supposed to have suggested that Wiener write this book. Benoît Mandelbrot and Walter Pitts proofread the manuscript. [8]
    • Cibernetica. Controllo e comunicazione nell’animale e nella macchina, Milan: Bompiani, 1953. (Italian)
    • Saibanetikksu: dōbutsu to kikai ni okeru seigyo to tsūshin [サイバネティックス : 動物と機械における制御と通信], Tokyo: Iwanami Shoten, 1957. (Japanese)
    • N. Viner (Н. Винер), Kibernetika, ili upravlenie i svyaz v zhivotnom i mashine [Кибернетика, или Управление и связь в животном и машине], trans. G.N. Povarov, Moscow: Sovetskoe radio [Советское радио], 1958, 216 pp; new ed., 1963; 2nd ed., 1968. (Russian)
    • Cibernética, trans. Miguel Mora Hidalgo, Madrid: Guadiana, 1960, 314 pp; 1971. (Spanish)
    • Kybernetika neboli řízení a sdělování v živých organismech a strojích, trans. O. Hanš, J. Wehle and Z. Wünsch, intro. Karel Winkelbauer, Prague: SNTL, 1960, 148 pp. (Czech)
    • Cibernetica sau Ştiinţa comenzii şi comunicării la fiinţe şi maşini, trans. Alexandru Giuculescu, Bucharest: Ştiinţifică, 1966, 272 pp. (Romanian)
    • Kybernetik. Regelung und Nachrichtenübertragung in Lebewesen und Maschine, rororo, 1968; Econ, 1992. (German)
    • Cibernética ou controle e comunicação no animal e na maquina, São Paulo: Poligono, 1970. (Brazilian Portuguese)
    • Cybernetyka, czyli sterowanie i komunikacja w zwierzęciu i maszynie, trans. Jerzy Mieścicki, Warsaw: Państwowe Wydaw. Naukowe, 1971, 261 pp. (Polish)
    • Kibernetika ili upravljanje i komunikacija kod živih biča i mašina, trans. Nebojša Uzelac and Slobodan Janić, Belgrade: Izdavačko-informativni centar studenata, 1972, 195 pp. (Serbo-Croatian)
    • Sibernetik, trans. İbrahim Keskin, Istanbul: Say Kitap Pazarlama, 1982, 226 pp. (Turkish)
    • Cibernética o el control y comunicación en animales y máquinas, trans. Francisco Martín, Barcelona: Tusquets, 1985, 266 pp; 1998; 2002, 150 pp. (Spanish)
    • Cibernètica, o, Control i comunicació en l'animal i la màquina, trans. Sebastián Xambó Descamps, Barcelona: Edicions Científiques Catalanes, 1986, 280 pp. (Catalan)
    • Kong zhi lun: huo guan yu zai dong wu he ji qi zhong kong zhi he tong xin de ke xue [控制论: 或关于在动物和机器中控制和通信的科学], Zhengzhou : He nan mei zhu chu ban she, 1991. (Chinese)
    • La cybernétique. Information et régulation dans le vivant et la machine, trans. Ronan Le Roux, Robert Vallée and Nicole Vallée-Lévi, Paris: Seuil, 2014, 376 pp. [9] (French)
  • Norbert Wiener, The Extrapolation, Interpolation, and Smoothing of Stationary Time Series with Engineering Applications, Cambridge, MA: Technology Press, 1949; New York: John Wiley & Sons, 1949; London: Chapman & Hill, 1949; 2nd ed., MIT Press, 1966. Earlier printed as a classified NDRC "yellow peril" Report, MIT, 1942. Uses Gauss's method of shaping the characteristic of a detector to allow for the maximal recognition of signals in the presence of noise; later known as the "Wiener filter." Reviews: Tukey (1952).
  • Norbert Wiener, "The Machine Age", [1949]. Unpublished. Written for The New York Times.
  • Norbert Wiener, The Human Use of Human Beings: Cybernetics and Society, Boston, MA: Houghton Mifflin, 1950; 2nd ed., 1954; London: Eyre and Spottiswode, 1954; New York: Avon Books, 1967; New York: Da Capo Press, 1988; London: Free Association Books, 1989; new ed., 1990.
    • Cybernétique et société, Union Générale d'Éditions, 1952; 1971. (French)
    • Mensch und Menschmaschine, Frankfurt am Main: Metzner, 1952; 4th ed., 1972. (German)
    • Materia, maskiner och människor: cybernetiken och samhället, trans. Edvin Thall, Stockholm: Forum, 1952, 210 pp; new ed., Stockholm: Rabén & Sjögren, 1964. (Swedish)
    • Cibernética e sociedade: o uso humano de seres humanos, trans. José Paulo Paes, São Paulo: Cultrix, 1954; 2nd ed., 1968, 190 pp. (Portuguese)
    • Ningen kikairon: Saibaneteikkusu to shakai [人間機械論: サイバネテイックスと社会], Tokyo: Misuzu Shobō, 1954. (Japanese)
    • Kibernetika i obshchestvo [Кибернетика и общество], trans. E.G. Panfilov, Moscow: IIL, 1958, 200 pp. (Russian)
    • Cybernetyka i społeczeństwo, trans. Olgierd Wojtasiewicz, afterw. E. J. Kolman, Warsaw: Książka i Wiedza, 1960, 236 pp; 2nd ed., 1961, 217 pp. (Polish)
    • Kybernetika a společnost, trans. Karel Berka, intro. Arnošt Kolman, Prague: ČSAV, 1963, 216 pp. (Czech)
    • Menneske og automat: kybernetikken og samfundet, trans. Elsa Gress Wright, Copenhagen: Gyldendal, 1963, 196 pp. (Danish)
    • Kibernetika i društvo: ljudska upotreba ljudskih bića, trans. Ljubomir Radanović, intro. Rajko Tomović, Belgrade: Nolit, 1964, 244 pp; 2nd ed., 1973. (Serbo-Croatian)
    • Anashim u-moḥot-mekhonah: ha-ḳeberneṭikah ṿeha-ḥevrah, trans. Ivrit Yaḳov Rabi, Tel Aviv: Poalim, 1964, 199 pp. (Hebrew)
    • Cibernètica i societat, intro. Miquel Masriera, trans. Jordi Monés, Barcelona: Ediciones 62, 1965, 214 pp. (Catalan)
    • Introduzione alla cibernetica. L’uso umano degli esseri umani, trans. Dario Persiani, Torino: Bollati Boringhieri, 1966, 229 pp; 3rd ed., 1970, 240 pp; 1982; 2012, 234 pp. Review. (Italian)
    • Ihmisestä, koneista, kielestä, trans. Pertti Jotuni, Helsinki: WS, 1969, 189 pp. (Finnish)
    • Inimolendite inimlik kasutamine: küberneetika ja ühiskond, trans. Boris Kabur, Tallinn: Perioodika, 1969, 158 pp. (Estonian)
    • Cybernética y sociedad, Buenos Aires: Sudamericana, 1969. (Spanish)
    • Kyvernētikē kai koinōnia: ē anthrōpinē chrēsimopoiēsē tōn anthrōpinōn ontōn, trans. & intro. Giannē Iōannidē, Athens: Papazēsēs, 1970, xxv+151 pp. (Greek)
    • İnsan ve mekanizması, Istanbul: İTÜ, 1973, vii+148 pp. (Turkish)
    • Ren you ren de yong chu: kong zhi lun he she hui [人有人的用处: 控制论和社 会], trans. Bu Chen, Beijing: Shang wu yin shu guan, 1978; 2009. (Chinese)
    • Cybernética y sociedad, México: Fondo de Cultura Económica, 1984. (Spanish)

Macy Conferences on Cybernetics (1946-53)[edit]

  • Cybernetics: Transactions of the Sixth Conference, New York: Josiah Macy, Jr. Foundation, 1949.
  • Cybernetics: Transactions of the Seventh Conference, eds. Heinz von Foerster, Margaret Mead, and Hans Lukas Teuber, New York: Josiah Macy, Jr. Foundation, 1950.
  • Cybernetics: Transactions of the Eighth Conference, eds. Heinz von Foerster, Margaret Mead, and Hans Lukas Teuber, New York: Josiah Macy, Jr. Foundation, 1952.
  • Cybernetics: Transactions of the Ninth Conference, eds. Heinz von Foerster, Margaret Mead, and Hans Lukas Teuber, New York: Josiah Macy, Jr. Foundation, 1953.
  • Cybernetics: Transactions of the Tenth Conference, eds. Heinz von Foerster, Margaret Mead, and Hans Lukas Teuber, New York: Josiah Macy, Jr. Foundation, 1955.
  • Claus Pias (ed.), Cybernetics | Kybernetik 1: The Macy-Conferences 1946–1953. Band 1. Transactions/Protokolle, Zürich/Berlin: Diaphanes, 2003, 736 pp. [10] [11]
    • The Macy-Conferences 1946–1953: The Complete Transactions, Zürich: Diaphanes, 2016, 736 pp. [12] (English)
  • Claus Pias (ed.), Cybernetics | Kybernetik 2: The Macy-Conferences 1946–1953. Band 2. Documents/Dokumente, Zürich: diaphanes, 2004, 512 pp. Introduction. [13] (German),(English)
Louis Couffignal, Les machines à penser, 1952.
Raymond Ruyer, La cybernétique et l'origine de l'information, 1954. Download.

Information theory and cybernetics in France (1950s)[edit]

  • Norbert Wiener, Cybernetics: or Control and Communication in the Animal and the Machine, Paris: Hermann & Cie, Cambridge, MA: Technology Press, and New York: John Wiley & Sons, 1948, 194 pp. Review: Dubarle (1948, FR). (see details and other editions above)
  • Dominique Dubarle, "Idées scientifiques actuelles et domination des faits humains", Esprit 9:18 (1950), pp 296-317. (French)
  • Louis de Broglie (ed.), La Cybernétique: théorie du signal et de l'information, Paris: Éditions de la Revue d'optique théorique et instrumentale, 1951, 318 pp. (French). Contributors include physicists and mathematicians Robert Fortet, M.D. Indjoudjian, A. Blanc-Lapierre, P. Aigrain, J. Oswald, Dennis Gabor, Jean Ville, Pierre Chavasse, Serge Colombo, Yvon Delbord, Jean Icole, Pierre Marcou, and Edouard Picault. [14]
  • Norbert Wiener, Cybernétique et société, l'usage humain des êtres humains [1950], Paris: Deux-Rives, 1952; 2nd ed., Paris: 10-18, 1962; 1971. (French) (see other editions above)
  • Louis Couffignal, Les machines à penser, Paris: Minuit, 1952, 153 pp; 2nd ed., 1964, 133 pp. (French)
    • Denkmaschinen, trans. Elisabeth Walther with Max Bense, intro. Max Bense, Stuttgart: Klipper, 1955, 186 pp; 1965. (German)
  • Colloques Internationaux du Centre National de la Recherche Scientifique 47 (1953). (French). Proceedings of a congress held in Paris in January 1951. Paul Chauchard: the congress was "the first manifestation in France of the young cybernetics, with the participation of N. Wiener, the father of this science." For this congress, organised by the French scientists Louis Couffignal and Pérès, both of whom had visited the U.S. laboratories, and sponsored by the Rockefeller Foundation, a number of foreigners were invited, including Howard Aiken, Warren McCulloch, Maurice Wilkes, Grey Walter, Donald MacKay and Ross Ashby, along with Wiener who was staying in Paris for a couple of months at the Collège de France. 300 people attended; 38 papers were presented; 14 machines from six different countries were demonstrated. [15]
  • Pierre de Latil, La Pensée artificielle, Paris: Gallimard, 1953. (French)
  • Benoît Mandelbrot, Contributions à la théorie mathématique des jeux de communications, Institut de Statistiques de l'Université de Paris 2, 1953. (French). Ph.D. dissertation in mathematics making a connection between game theory and information theory. He showed for instance that both thermodynamics and statistical structures of language can be explained as results of minimax games between ‘nature’ and ‘emitter’. He also made the connection between the definitions of information given by the British statistician Ronald A. Fisher in the 1920s, by the physicist Dennis Gabor in 1946 and the already well-known definition proposed by Shannon. Mandelbrot was at MIT from 1952-1954 and later at the IAS in Princeton. [16]
  • Raymond Ruyer, La cybernétique et l'origine de l'information, Paris: Flammarion, 1954. (French)
  • G.-th. Guilbaud, La Cybernetique, PUF, 1954. (French)
  • Marcel-Paul Schützenberg, "Contributions aux applications statistiques de la théorie de l’information", Publications de l'Institut de Statistique de l'Université de Paris 3:1-2 (1954), pp 3-117; repr. in Œuvres complètes, Tome 3: 1953-1955, Paris: Institut Gaspard-Monge, Université Paris-Est, 2009, pp 56-161. (French). Ph.D. Dissertation defended at the Faculté des Sciences in Paris in June 1953.
  • Marcel-Paul Schützenberg, "La théorie de l'information", in Cahiers d'actualité et de synthèse, Encyclopédie française, Paris: Société Nouvelle de l'Encyclopédie Française, 1957, pp 9-21. (French)
Liapunov, Kitov, Sobolev, "Osnovnye cherty kibernetiki", 1955. View online.
A.N. Kolmogorov, Teoriya peredachi informatsii, 1956. Download.

Information theory and cybernetics in the Soviet Union (1950s)[edit]

  • Claude Shannon (Клод Шеннон), "Statisticheskaia teoriia peredachi elektricheskikh signalov" [Статистическая теория передачи электрических сигналов; The Statistical Theory of Electrical Signal Transmission] [1948], in Teoriya peredakhi elektrikheskikh signalov pri nalikhii pomekh [Теория передачи электрических сигналов при наличии помех], ed. Nikolai A. Zheleznov (А. Н. Железнов), Moscow: Izdatelstvo inostrannoi literatury (ИИЛ), 1953. (Russian). The editor rid the work of the words information, communication, and mathematical entirely, put entropy in quotation marks, and substituted data for information throughout the text. He also assured the reader that Shannon’s concept of entropy had nothing to do with physical entropy and was called such only on the basis of "purely superficial similarity of mathematical formulae". [17]
    • "Matematicheskaya teoriya svyazi" [Математическая теория связи] [1948], trans. S. Karpov, in Raboty po teorii informatsii i kibernetike [Работы по теории информации и кибернетике], Moscow: Izdatelstvo inostrannoi literatury (ИИЛ), 1963, pp 243-332. (Russian)
  • Aleksei Liapunov, Anatolii Kitov, Sergei Sobolev, "Osnovnye cherty kibernetiki" [Основные черты кибернетики; Basic Features of Cybernetics], Voprosy filosofii [Вопросы философии; Problems of Philosophy] 141:4 (August 1955). (Russian). The first Soviet article speaking positively about cybernetics and non-technical applications of information theory, authored by three specialists in military computing—Liapunov, a noted mathematician and the creator of the first Soviet programming language; Kitov, an organizer of the first military computing centers; and Sobolev, the deputy head of the Soviet nuclear weapons program in charge of the mathematical support. They presented cybernetics as a general "doctrine of information", of which Shannon’s theory of communication was but one part. The three authors interpreted the notion of information very broadly, defining it as "all sorts of external data, which can be received and transmitted by a system, as well as the data that can be produced within the system." Under the rubric of "information" fell any environmental influence on living organisms, any knowledge acquired by man in the process of learning, any signals received by a control device via feedback, and any data processed by a computer. [18] [19]
  • A.N. Kolmogorov (А. Н. Колмогоров), Teoriya peredachi informatsii [Теория передачи информации], Мoscow, 1956. (Russian)
  • Igor’ A. Poletaev, Signal: O nekotorykh poniatiiakh kibernetiki, Moscow: Sovetskoe radio, 1958. (Russian). The first Soviet book on cybernetics.

Information theory and cybernetics in Germany (1950s)[edit]

  • Nobert Wiener, Mensch und Menschmaschine [1950], trans. Gertrud Walther, Frankfurt am Main: Metzner, 1952, 211 pp; 4th ed., 1972. (German) (see details and other editions above)
  • Louis Couffignal, Denkmaschinen [1952], trans. Elisabeth Walther with Max Bense, intro. Max Bense, Stuttgart: Klipper, 1955, 186 pp; 1965. (German) (see details above)
  • Werner Meyer-Eppler, Grundlagen und Anwendungen der Informationstheorie, Berlin: Springer, 1959, xviii+446 pp; 2nd ed., eds. Georg Heike and K. Löhn, Berlin: Springer, 1969. (German). Reviews: Tamm (1960), Billingsley (1961, EN), Adam (1965).

Popularisation of information theory in the United States (1950s)[edit]

  • Charles Eames, Ray Eames, A Communication Primer, 16 mm, 1953, 21 min. An educational film aimed at students, sponsored by IBM and distributed by Museum of Modern Art.
  • Francis Bello, "The Information Theory", Fortune, Vol. 48 (December 1953), pp 136-158.
  • The Search, 1954. Documentary film featuring Shannon, Forrester and Wiener, produced by NBC.
  • Stanford Goldman, Information Theory, Prentice-Hall, 1953, 385 pp; New York: Dover, 1968; 2005.
    • S. Goldman (С. Голдман), Teoriya informatsii [Теория информации], Мoscow, 1957. (Russian)
  • Léon Brillouin, Science and Information Theory, New York: Academic Press, 1956. A bestseller rewrite of physics using information theory.
  • Information and Control journal, *1958. Founding editors: Léon Brillouin, Colin Cherry, Peter Elias.
Claude E. Shannon, "The Bandwagon", 1956. Download.

Debate in Transactions (1955-56)[edit]

  • L.A. De Rosa, "In Which Fields Do We Graze?", I.R.E. Transactions on Information Theory 1 (December 1955). Editorial by the chairman of the Professional Group on Information Theory: "The expansion of the applications of Information Theory to fields other than radio and wired communications has been so rapid that oftentimes the bounds within which the Professional Group interests lie are questioned. Should an attempt be made to extend our interests to such fields as management, biology, psychology, and linguistic theory, or should the concentration be strictly in the direction of communication by radio or wire?"
  • Claude E. Shannon, "The Bandwagon", I.R.E. Transactions on Information Theory 2 (1956), p 3. Shannon's call for keeping the information theory "an engineering problem": "Workers in other fields should realize that the basic results of the subject are aimed in a very specific direction, a direction that is not necessarily relevant to such fields as psychology, economics, and other social sciences.. [T]he establishing of such applications is not a trivial matter of translating words to a new domain, but rather the slow tedious process of hypothesis and verification. If, for example, the human being acts in some situations like an ideal decoder, this is an experimental and not a mathematical fact, and as such must be tested under a wide variety of experimental situations."
  • Norbert Wiener, "What Is Information Theory?", I.R.E. Transactions on Information Theory 3 (June 1956), p 48. A rejection of Shannon's narrowing focus and insistance that "information" remain part of a larger indissociable ensemble including all the sciences: "I am pleading in this editorial that Information Theory...return to the point of view from which it originated: that of the general statistical concept of communication.. What I am urging is a return to the concepts of this theory in its entirety rather than the exaltation of one particular concept of this group, the concept of the measure of information into the single dominant idea of all."

Historical analysis and review of the impact of information theory[edit]

In mathematics, engineering and computing[edit]

Written by engineers and mathematicians
N. Katherine Hayles, How We Became Posthuman: Virtual Bodies in Cybernetics, Literature, and Informatics, 1999. Download.
Written by historians and theorists
Bernard Dionysus Geoghegan, The Cybernetic Apparatus: Media, Liberalism, and the Reform of the Human Sciences, 2012. Download.

In the social sciences and humanities[edit]

In other fields[edit]

  • Donna Haraway, "The High Cost of Information in Post World War II Evolutionary Biology: Ergonomics, Semiotics, and the Sociobiology of Communications Systems", Philosophical Forum 13:2-3 (Winter/Spring 1981-82), pp 244-278.
  • Donna Haraway, "Signs of Dominance: From a Physiology to a Cybernetics of Primate Society, C.R. Carpenter, 1930-1970", in Studies in History of Biology, Vol. 6, eds. William Coleman and Camille Limoges, Johns Hopkins University Press, 1982, pp 129-219.
  • Philip Mirowski, "What Were von Neumann and Morgenstern Trying to Accomplish?", in Toward a History of Game Theory, ed. E.R. Weintraub, Duke University Press, 1992, pp. 113-147. Information theory in economics.
  • Evelyn Fox Keller, Refiguring Life: Metaphors of Twentieth-Century Biology, Columbia University Press, 1995, pp 81-118. Information theory in embryology.
  • Lily E. Kay, "Cybernetics, Information, Life: The Emergence of Scriptural Representations of Heredity", Configurations 5:1 (Winter 1997), pp 23-91. Information theory in genetics. [34]
  • Philip Mirowski, "Cyborg Agonistes: Economics Meets Operations Research in Mid-Century", Social Studies of Science 29:5 (1999), pp. 685-718. Information theory in economics.
  • Lily E. Kay, "From Logical Neurons to Poetic Embodiments of Mind: Warren S. McCulloch’s Project in Neuroscience", Science in Context 14:15 (2001), pp 591-614. Information theory in neuroscience.
  • Jennifer S. Light, From Warfare to Welfare: Defense Intellectuals and Urban Problems in Cold War, Johns Hopkins University Press, 2003. Information theory in urban planning.
  • Jérôme Segal, "The Use of Information Theory in Biology: a Historical Perspective", History and Philosophy of the Life Sciences 25:2 (2003), pp 275-281. [35]
  • Jérôme Segal, "L'information et le vivant: aléas de la métaphore informationnelle", in Le zéro et le un: histoire de la notion scientifique d’information au 20e siécle, Paris: Syllepse, 2003, pp 487-534; 2011. (French). Information theory in genetics and biology. [36]

Indirectly related prior work[edit]

  • Jacques Lafitte, Réflexions sur la science des machines [Reflections on the Science of Machines], 1911-32.
  • Marian Smoluchowski, "Experimentell nachweisbare, der Ublichen Thermodynamik widersprechende Molekularphenomene", Phys. Zeitshur. 13, 1912. Connecting the problem of Maxwell's Demon with that of Brownian motion, Smoluchowski wrote that in order to violate the second principle of thermodynamics, the Demon had to be "taught" [unterrichtet] regarding the speed of molecules. Wiener mentions him in passing in his Cybernetics (1948) [37].
  • Hermann Schmidt, general regulatory theory [Allgemeine Regelungskunde], 1941-54.

See also[edit]



Fields and theories: Classics, Art history, History of architecture, Anthropology, Semiotics, Philosophy of technology, Marxist aesthetics, Design research, Humanities computing, Structuralism, Poststructuralism, Mediology, Media archaeology, Cyberfeminism, Cultural techniques, Neuroaesthetics, Posthumanities, Sensory ethnography, Media ecology, Digital humanities, Software studies, Modern periodical studies, Accelerationism.
Concepts: Faktura, Ostranenie, Biomechanics, Commons, Postmedia, Evil media.
Related theories: Systems theory, Information theory, Cybernetics.