Undercoverage: population that is actually sampled is not as broad as the population that the researcher desires to sample (the target population)

Nonresponse: missing data, people who hang up, or refuse. People who work long hours.  Unfortunately people who are difficult to contact tend to give answers different from other people's.   Non response is a big problem in mail surveys.
Example: Laramie Wyoming sent out 2000 surveys to Laramie households to pout the citizen's Attitude survey.  Only 481 surveys were returned.  What was the response rate?
 

Don't know, haven't decided: surveyors may interpret these responses to flatter the response they want.

Untruthful answers: people give untruthful answers for several reasons.
  Sensitive questions,
  socially acceptable answers,
  telling the interviewer what he or she wants to hear.
The fix: secret ballots, anonymous surveys, "sensitive question" techniques

Ignorant people who don't want to appear like they know nothing about the subject.
Example: In a study educators were asked how they would rank Princeton's undergraduate business program.  In every case, it was rated among the top 10 departments in the country, even though Princeton doesn't offer an undergraduate business major.

People who don't remember the actual answer.
Example: Students were asked to report their grade point averages.  Researchers then determined the actual GPA's.  Over 17% of the students reported a GPA that was .4 or more above their actual average, and about 2% reported a GPA more than .4 below their actual GPA.  (more inflated their GPA's!)

Timing: in January the National Football League reported a poll that revealed football as the nation's favorite (this is at the time of the Super Bowl)

Statement of questions: Subtle differences in phrasing make large differences in the results
10% said they would support cutting programs involving "aid to the needy"
39% said they would support cutting programs to "public welfare programs"
Try saying no to this question: "Do you favor paying hard-working teachers a little more so that our fine young people can have a decent education?"
Interpretation of people's answers: Large survey organizations require that their interviewers follow very explicit procedures, and state questions exactly as written.

There are two quizzes below, Sampling Bias (1) and Sampling Bias (2), from Beth Chance's AP Statistics: Example Activities and Handouts .  Hand out one quiz to half the class, and the other quiz to the other half of the class.  Then identifying the sources of bias from the remainder of the handout as a class activity.  (From Activity 20 from ESIS.)
Other homework: IPS Section 3.3 p. 262 # 46-50.

_____________________________________________________________________________________________

Name ______________________________                                                                                 Class Period______

2. Social science researchers have conducted extensive empirical studies and concluded that the expression "out of sight, out of mind" is generally true.  Do you find this result surprising or not surprising?
  Surprising_________   Not Surprising_________

3. Suppose that the United States is preparing for the outbreak of an unusual Asian disease which is expected to kill 600 people.  Two alternative programs to combat the disease have been proposed.  Assume that the exact scientific estimates of the consequences are as follows:
   If Program A is adopted, 200 people will be saved.
   If Program B is adopted, there is a 1/3 probability that 600 people will be saved and a 2/3 probability that nobody will be saved.
Which of the two programs would you favor?   Program A________ Program B________
__________________________________________________________________________________________
Identify any sources of bias in each of the following surveys:

1. A Gallup poll found that 81% of U.S. parents say they have spoken with their teenagers about the dangers of drinking and driving.  Only 64% of the teens say they remember such a discussion (USA Today, Dec. 19, 1984).

2.  The U.S. census of 1980 states that 32,194 Americans are 100 years old or older.  However, Social Security figures show only 15, 258 adults of this advanced age (Los Angeles Times, Dec. 4, 1983)

3. In a census in Russia, 1.4 million more women than men reported that they were married (U.S. News & World Report, Aug. 30, 1976).

4. To find out how people reacted to the clothes of vice-presidential candidate Geraldine A. Ferraro, researchers ran a survey shortly after the 1984 Democratic convention in three locations: the Wall Street area of New York City, State Street in Chicago, and Crown Center in downtown Kansas City.  The researchers stopped people at random and asked them if they had seen the Democratic convention on television.  Those who had were not used.  Those who had not "were asked if they would be willing to contribute a minute or two of their time to help a woman candidate choose a suitable picture for a campaign poster.  We wanted to enlist only those who had a positive attitude toward women running for office."  The 347 respondents were then shown picures of women wearing three outfits, and the pictures did not show the women's faces. Then the respondents were asked several questions (Los Angeles Times, Aug., 3, 1984).

5. One year after the Detroit race riots of 1967, interviewers asked a sample of black residents in Detroit if they felt they could trust most white people, some white people, or none at all.  When the interviewer was white, 35% answered "most"; when the interviewer was black, 7% answered "most" (Moore, Statistics: Concepts and Controversies).

6. On a recent deregulation of banking, "[the head of California's Security Pacific Bank] reckons the higher interest accounts, and all the other new financial services, are designed for the most affluent 15% to 20% of Security Pacific Bank's customers.  By extension--as 2 million customers are surely a sample of the general population--the new world of deregulated finance benefits the top-earning 15% to 20% of U.S. households" (Los Angeles Times, Dec. 4, 1983).
 

_______________________________________________________________________________________________
 
 

Name ______________________________                                                                                     Class Period______

2. Social science researchers have conducted extensive empirical studies and concluded that the expression "absence makes the heart grow fonder" is generally true.  Do you find this result surprising or not surprising?
  Surprising_________  Not surprising_________

3. Suppose that the United States is preparing for the outbreak of an unusual Asian disease which is expected to kill 600 people.  Two alternative programs to combat the disease have been proposed.  Assume that the exact scientific estimates of the consequences are as follows:
   If Program A is adopted, 400 people will die.
   If Program B is adopted, there is a 1/3 probability that nobody will die and a 2/3 probability that 600 people will die.
Which of the two programs would you favor?   Program A________ Program B________
__________________________________________________________________________________________
Identify any sources of bias in each of the following surveys:

1. A Gallup poll found that 81% of U.S. parents say they have spoken with their teenagers about the dangers of drinking and driving.  Only 64% of the teens say they remember such a discussion (USA Today, Dec. 19, 1984).

2.  The U.S. census of 1980 states that 32,194 Americans are 100 years old or older.  However, Social Security figures show only 15, 258 adults of this advanced age (Los Angeles Times, Dec. 4, 1983)

3. In a census in Russia, 1.4 million more women than men reported that they were married (U.S. News & World Report, Aug. 30, 1976).

4. To find out how people reacted to the clothes of vice-presidential candidate Geraldine A. Ferraro, researchers ran a survey shortly after the 1984 Democratic convention in three locations: the Wall Street area of New York City, State Street in Chicago, and Crown Center in downtown Kansas City.  The researchers stopped people at random and asked them if they had seen the Democratic convention on television.  Those who had were not used.  Those who had not "were asked if they would be willing to contribute a minute or two of their time to help a woman candidate choose a suitable picture for a campaign poster.  We wanted to enlist only those who had a positive attitude toward women running for office."  The 347 respondents were then shown picures of women wearing three outfits, and the pictures did not show the women's faces. Then the respondents were asked several questions (Los Angeles Times, Aug., 3, 1984).

5. One year after the Detroit race riots of 1967, interviewers asked a sample of black residents in Detroit if they felt they could trust most white people, some white people, or none at all.  When the interviewer was white, 35% answered "most"; when the interviewer was black, 7% answered "most" (Moore, Statistics: Concepts and Controversies).

6. On a recent deregulation of banking, "[the head of California's Security Pacific Bank] reckons the higher interest accounts, and all the other new financial services, are designed for the most affluent 15% to 20% of Security Pacific Bank's customers.  By extension--as 2 million customers are surely a sample of the general population--the new world of deregulated finance benefits the top-earning 15% to 20% of U.S. households" (Los Angeles Times, Dec. 4, 1983).