Information Theory

how business shennanigans & telephony give rise to information theory

Part 1: The Necessary Competencies

Page effect

Follow the Money

The results of this kind of work implied serious financial benefits to existing telegraphy:

  • different signals, sounding different notes, could be sent down a single wire

    ie, more messages on the same physical infrastructure
  • such devices, if they could be built, were called harmonic telephones

missing piece 1:

variable resistence

missing piece 2:

a theory of language

listen |

see also this page

What does sound look like?

Part 2: The Invention Race

Dates for the 'invention' of the telephone:

  • 1840 - Charles Page
  • 1854 - Charles Bourseul
  • 1854 - Antonio Meucci
  • 1857 - Johann Philipp Reis
  • 1864 - Innocenzo Manzetti
  • 1876 - Bell vs Gray
  • 1878 - Cyrille Duquet
  • 1878 - Thomas Edison and Emile Berliner
  • 1878 - David Edward Hughes

Disreputability of speaking devices

"Fearing ridicule would be attached to the idea of transmitting vocal sounds telephonically... I said nothing of this plan" - Bell

Gray versus Bell

and who did what when to whom how

The receiver worked ok; transmitter was still dodgy

Enter Thomas Edison & Emile Berliner

More legal battles.

Part 3: Social Necessity & Business Model

The supervening social necessity

The breaks & accelerators

What do you do with a telephone, anyway?

Alexander Graham Bell:

In a similar manner, it is conceivable that cables of telephone wires could be laid underground, or suspended overhead, communicating by branch wires with private dwellings, country houses, shops, manufactories... uniting them through the main cable with a central office where the wires could be connected as desired, establishing direct communication between any two places... I believe, in the future, wires will unite the head offices of the Telephone Company in different cities, and a man ...may communicate by word of mouth with another in a distant place.

(prospectus for prospective investors)

Part 4: The Bell Empire

Or, some inside-baseball about the Bell Company, Western Union, and who does what to who when and why

or the battle to control the real money-maker: long-distance calling

And How This Gives Rise To Bell Labs

and thence Claude Shannon and a Theory of Information

(remember, I'm talking US most of the time unless I say otherwise. Good research question for you: what's the situation in Canad?)

Cost to subscribe, annually, 1890, dollars

City Annual Cost
Washington $100
New York $240
Philadelphia $250
Chicago $175
Paris $18
Stockholm $20

Back to main thread:

  • long distance telephony was central to Bell's plan to survive the expiration of its patents
  • seemed also that Bell was directing customers to use telegraph services that it owned/had major stake in
  • antitrust legislation, populist politicians, interstate commerce commission: AT&T in the crosshairs

WWI - AT&T gets nationalised anyway

"After extended conferences between the representatives of the Postmaster General and of the Bell system, covering-that there might be no misunderstanding-painstaking and exhaustive discussion and a frank exchange of views, what constitutes a just compensation for the supervision, possession, control, and operation of the Bell system taken over under the proclamation of the President of the United States, has been agreed upon."

"The representatives of the Bell system throughout the negotiations found nothing but helpfulness; asking no more than they thought ought to be paid by the Government, they found an intent and desire to pay all that ought to be paid, and for the protection of the property to do all that ought to be done and all that has been done in the past.

In taking over the property the Postmaster General also desires to give continuity to the service, and as far as consistent with Government operation, to the personnel which has brought the property to its present degree of efficiency." - NY Times 1918-10-07

Part 5: Bell Labs & Shannon's Breakthrough(s)

1938 A Symbolic Analysis of Relay and Switching Circuits (read it here if you want)

George Boole

At Bell Labs, he comes up with information theory

where a key part is the idea of information as something that can be measured, distinct from its content.

link

link

See also Durojaye, Cecilia, et al. 'When Music Speaks: An Acoustic Study of the Speech Surrogacy of the Nigerian Dùndún Talking Drum'. Frontiers in Communication, vol. 6, 2021. Link

Yoruba linguistic tones

Redundancy is the key to understanding language.

Claude Shannon in 1948 took this idea to its logical conclusion and developed a way to think about a message as being composed of two things:

  • the content
  • the information

'Information' is a measurement of the redundancy of that content, or how 'surprising' the next letter/sound/wave/ is.

Shannon Entropy quantifies this redundancy:

\[H(X) = -\sum_{x \in X} p(x) \log_2 p(x),\]

Where \(H(X)\) is the entropy, \(p(x)\) is the probability of outcome \(x\), and the sum is over all possible outcomes in \(X\).

The 'bit' is the smallest unit of information; it is the amount gained from knowing the outcome of an event with only two possible outcomes

"AAAAAAA" -> low entropy or high entropy?

"JTR45A2" -> low entropy or high entropy?

Shannon Model of Communication

Source: Generates the message (e.g., a sender).

Transmitter: Encodes the message for transmission (e.g., a microphone or encoder).

Channel: The medium through which the message travels (e.g., radio waves, telephone wire).

Receiver: Decodes the message back into its original form (e.g., a speaker or decoder).

Destination: The intended recipient of the message.

Part 6: The Digital Future

Combine information theory - the division of a message from its meaning - with logical circuits and you get the 20th century.