What Did That Say?
Adapted from Mathematics: A Human Endeavor,
,
Harold Jacobs
We will be comparing the frequency that a letter
occurs in a sample compared to the number of times it is predicted to occur.
For any given letter, this would be a binomial distribution. When
comparing proportions, we may use the normal distribution to approximate
the binomial.
The frequency (from most common to least) of English
letters is as follows:
E T A O N R I S H D L F C M U G P Y W B V K X J Q Z
Their frequencies as a percentage are as follows:
E (13) T(10.5)
A(8.1) O(7.8)
N(7.1) R(6.8)
I(6.3) S(6.1)
H(5.2) D(3.8)
L(3.4) F(2.9)
C(2.7) M(2.5)
U(2.4) G(2)
P(1.9) Y(1.9)
W(1.5) B(1.4)
V(.9)
K(.4)
X(.15) J (.13)
Q(.11) Z(0.7)
SAMPLE #1: From "What Did That Say?" worksheet, Joyce Smart
Of all of the people you could care to meet, Nefrit was the most unusual. A thief of unusual facility, she used the facade of a media reporter, gaining entry into homes by claiming affinity with the Fox news agency. Often, she made off with the entire contents of a home. She made no secret of her success, professing to be a thief of infinite skill.
SAMPLE #2: From How to Lie with Statistics, Darrel Huff, p.133.
The "population" of a large area in China was 28 million. Five years later it was 105 million. Very little of that increase was real; the great difference could be explained only by taking into account the purposes of the two enumerations and the way people would be inclined to feel about being counted in each instance. The first census was for tax and military purposes, the second for famine relief.
SAMPLE #3: From "A Sin of Omission", Games magazine, November/December 1977, also found in Mathematics, A Human Endeavor, 2nd Ed., Harold Jacobs, p. 518.
Around midnight, a sly-looking man slips into a luxury city building.
A woman occupant, watching his actions from a fourth-floor window, grows
suspicious and dials 911 for a patrol car. This lady complains, "A
man in a brown suit, with shaggy hair, a slight build, and a criminal air
is prowling through my lobby."
Fairly soon two young cops, Smith and Jarvis, pull up.
Looking for an unknown vagrant, Smith spots Jim Oats walking out a font
door. Oats, a minor burglar, is bold as brass, arrogant, and calm.
Smith grabs him by his collar...
1. Practice counting: Count the number of f's in Sample
#1. There are _______ f's out of 275 total letters.
Expected proportion:_______ Actual proportion:_________
2. Is the number of f's found in Sample #1 very different from what you would expect? Would it be appropriate to use the normal distribution to approximate the proportion of f's? Why or why not?
3. In Sample #2, there are N= 320 letters (not including numerical
digits). Referring to the frequencies above, the normal approximation
should NOT be used for which letters?
4. Count the number of times the letter "e" occurs in Sample #2.
In order to be more accurate, compare your count with the count obtained
by other members of the class.
e = _______ frequency (as a proportion) of e:
= __________
5. Do a confidence interval and hypothesis test (H0: p
= .13, Ha: p not equal to .13)for the frequency
of of e. Does it seem that the frequency of e in the sample is the
same as the population?
6. Sample #3 is a portion of a mystery found in the Games magazine.
Even without the rest of the text, you should be able to solve the mystery.
The title of the story should give you a clue to the solution. Solve
it.
7. Letter frequencies are used to decode ciphers. Here is a cipher
in which the words are not separated so that there are no clues about their
lengths. For convenience, the message has been written in groups
of five letters each. A graph of the frequencies of the letters
in the cipher is shown below. Using the predicted letter frequencies
and the frequency graph, decode what the cipher says.
B F C O N A N Y K F I X K U S
I X H U C O N G F B C N I A C
H C N A A K S T N I
C O N P H B F Y K I N K M C O
N R H W H F N A N F H T L C O
B A W S H L N I H F
B P W K U C H F C U K S N B F
C O N H P N U B Y H F T B Y C
K U L H C C O N I N
Y B A B T N D H C C S N K M P
B I X H L
Record what each letter stands for here:____ ____ ____ ____ ____ ____
____ ____ ____ ____
A B C
D E F
G H I
J
____ ____ ____ ____
____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____
K
L M N
O P Q
R S T
U V W
X Y Z
Hints:
1. The most frequent letter is probably E, so write
an E underneath each place this letter appears in the cipher. The
second most frequent letter is probably T. write a T under neath
each place this letter appears.
2. A very common word in English is THE. If THE
appears several times in the cipher, you should be able to guess what letter
represents H.
3. A strong clue in solving a cipher is knowledge of
certain words that are likely to appear in it. For example, this
cipher mentions the Second World War. If you can find a place where
the words THE SECOND WORLD WAR can appear, you will know what several more
letters represent.