Information & Communication

INLS 180

Spring 2004

Shannon

The technical (engineering) problem

A problem solving strategy: simplify and model

Unit of information is the bit (binary digit)—unit of choice or uncertainty

Rate of information flow—(bits per second) also called the entropy (rate of reducing uncertainty)

Channel capacity--bandwidth

Shannon assumes communication initiated by SELECTING a desired message from a set of possible messages

Then, information is the amount of uncertainty in the SOURCE (not the message). This the amount of entropy (randomness). “Information is a measure of one’s freedom of choices when one selects a message.” P. 9

Weaver discussed three levels of communication problem

Accuracy of transmission (technical problem)

Degree of meaning (semantic problem)

Effect of transmission (effectiveness problem)

Do not confuse information with meaning!!

Examples

1. RSVP yes or no?

When you reply, you reduce my uncertainty by 1/2, requires only 1 bit to achieve—the minimal amount of information

2. A 32 icon language.

When the destination receives/selects one, the uncertainty is reduced by 31/32, requires 5 bits (log32₂=5), five times as much information as the RSVP. So, selecting (or giving a command) a single character/icon in a 32 language reduces uncertainty (provides more information) than selecting a character in a 2 character language.

Assumes independence of each ‘choice’

For more typical settings, conditional probability arises (e.g., if the receiver has received a ‘Q’ in and English word message, the next letter carries 0 information since it does not reduce any uncertainty (we are sure it will be an ‘U.’ This gives rise to coding theory.

Pierce

Suggests that Shannon’s work inspired Chomsky and others to look for semantics in structure (syntax), which in turn inspired psychologists (e.g., Miller) and we might extend to say initiated the cognitive sciences and the debates about whether meaning is constructed by the receiver (context bound) or inherent in the message.

Pierce: Communication is a process of adjusting understandings and attitudes, of making them congruent or ascertaining how and where they agree or disagree. Common language is NOT as important as a common interest. However, perfect common interest (knowledge) makes communication impossible (no information can exchange)—we need to be surprised within our context ala Shannon.

This debate is related to arguments about relevance and indexing that are central to information science.

We aim to balance order and randomness/novelty. Can we data mine the order that escapes us (through biometrics or transaction logging)? What are implications for information design and services?

Pierce on mass communication

Pierce discusses communities of interest and relates to mass communication trends.

This discussion introduces some of the issues of popular (mass) communication. How do his speculations about trends in periodicals (rising at the time) and newspapers (static), look today? Internet impact? See Statistical Abstracts

Pierce on Wiener

Cybernetics introduced homeostasis—balance through feedback.

Pierce relates to politics and social control (consider the time this was written). Is open source software movement a homeostatic movement? ;-)

Homeostasis addresses information deficiency and overload—both issues of our time.


	The technical (engineering) problem
	A problem solving strategy: simplify and model
	Unit of information is the bit (binary digit)—unit of choice or uncertainty
	Rate of information flow—(bits per second) also called the entropy (rate of reducing uncertainty)
	Channel capacity--bandwidth


	Then, information is the amount of uncertainty in the SOURCE (not the message). This the amount of entropy (randomness). “Information is a measure of one’s freedom of choices when one selects a message.” P. 9
	Weaver discussed three levels of communication problem
		Accuracy of transmission (technical problem)
		Degree of meaning (semantic problem)
		Effect of transmission (effectiveness problem)
	Do not confuse information with meaning!!


	1. RSVP yes or no?
	When you reply, you reduce my uncertainty by 1/2, requires only 1 bit to achieve—the minimal amount of information
	2. A 32 icon language.
	When the destination receives/selects one, the uncertainty is reduced by 31/32, requires 5 bits (log32₂=5), five times as much information as the RSVP. So, selecting (or giving a command) a single character/icon in a 32 language reduces uncertainty (provides more information) than selecting a character in a 2 character language.

	Assumes independence of each ‘choice’

	For more typical settings, conditional probability arises (e.g., if the receiver has received a ‘Q’ in and English word message, the next letter carries 0 information since it does not reduce any uncertainty (we are sure it will be an ‘U.’ This gives rise to coding theory.


	Suggests that Shannon’s work inspired Chomsky and others to look for semantics in structure (syntax), which in turn inspired psychologists (e.g., Miller) and we might extend to say initiated the cognitive sciences and the debates about whether meaning is constructed by the receiver (context bound) or inherent in the message.
	Pierce: Communication is a process of adjusting understandings and attitudes, of making them congruent or ascertaining how and where they agree or disagree. Common language is NOT as important as a common interest. However, perfect common interest (knowledge) makes communication impossible (no information can exchange)—we need to be surprised within our context ala Shannon.
	This debate is related to arguments about relevance and indexing that are central to information science.
	We aim to balance order and randomness/novelty. Can we data mine the order that escapes us (through biometrics or transaction logging)? What are implications for information design and services?


	Pierce discusses communities of interest and relates to mass communication trends.
	This discussion introduces some of the issues of popular (mass) communication. How do his speculations about trends in periodicals (rising at the time) and newspapers (static), look today? Internet impact? See Statistical Abstracts


	Cybernetics introduced homeostasis—balance through feedback.
	Pierce relates to politics and social control (consider the time this was written). Is open source software movement a homeostatic movement? ;-)
	Homeostasis addresses information deficiency and overload—both issues of our time.