Determination
33
Minnesota
News Council
In the Matter of the Complaint of
Ronald Jacobson against the Mpls Tribune
Ronald L. Jacobson, assistant professor of neurology
and biometry at the University of Minnesota, complained that the newspaper
used unsound methodology in presenting the results of a survey on university
students' attitudes toward abortion. He complained that the paper's
claim that the survey results were representative of the entire university
student population was statistically invalid. The resulting article,
he said, was therefore inaccurate and misleading.
Background: The paper published an 11-part
series of articles reporting the attitudes of university students
on a variety of public issues. The series, headlined "Think Young,"
was based on the results of a telephone survey conducted for the paper
by an outside marketing research company.
The survey sample was gathered by the following method:
One thousand names were drawn from the student directory by selecting
every 44th name in the book. Students with no phone number were sent
letters and asked to phone the survey center. After making up to three
callbacks to the names on the list and eliminating names not reached,
names who refused to participate, or students with language barriers,
a final sample of 300 was achieved.
Accompanying each article in the "Think Young" series
was a methodology box explaining how the survey was conducted. The
wording in the box claimed that the survey was conducted "among 300
University students," then reviewed the procedure used for contacting
students.
One story in the series reported the students' attitudes
about abortion. Jacobson, who was concerned about the attitudes reported
in the story, contacted the paper for a more detailed explanation
of the survey methodology. Jacobson then complained that since the
original sample size was 1,000, the 300-name sample was not random
and thus not an accurate basis for the story, since the sampling error
might be as large as 37 percentage points in either direction. The
paper argued that the sample selection process was intended to provide
a systematic random sample of only about 300 usable names, that 1,000
had been chosen originally to ensure a final sample of 300, and that
the sampling error was thus only as much as seven percentage points
in either direction.
As a result of Jacobson's original expression of
concern, the paper added mention of the sampling error to the information
in the methodology box for all stories after the abortion attitudes
article.
An expert in social science survey research contacted
by the Council said that there was little effective difference between
a random sample and systematic random sample. But he added that the
enormous nonresponse rate in this case might make questionable the
validity of the survey results. Newspaper representatives argued that
such a nonresponse rate is common in telephone surveys.
The paper argued that the survey methodology used
was scientifically sound and that the survey results were reported
in a fair and nonmisleading manner.
Determination of the Council: One of journalism's
greatest challenges is to produce sound reporting in an age characterized
by increasingly complex problems and issues. The growing use of social
science survey techniques by news organizations to more fully explore
and report on these problems and issues is a positive, indeed necessary,
development.
Testimony at the hearing indicated that survey methodology
involves complex scientific expertise. Both sides of this issue presented
persuasive testimony indicating there is considerable disagreement
among experts as to the degree to which a survey or poll must be scientifically
precise before the results can be considered valid or accurate.
Neither side in this case convinced the Council that
its position is more scientifically sound than the other. Lacking
definitive testimony in either direction, the Council cannot sustain
the complaint.
The paper made a good-faith effort to conduct a sound
survey and showed no evidence of careless disregard for sound survey
procedures. Further, the paper has said it recognizes the importance
of the issues raised in Jacobson's complaint and will continue to
consider ways to increase the precision of its surveys and more fully
explain survey methods to readers.
As a general guide to informative reporting on poll
stories, leading journalists and survey researchers have urged the
news media to include, at minimum, the following eight items of information
in poll stories:
- The identity of the sponsor of the survey.
- The exact wording of the question(s) asked.
- A definition of the population sampled.
- The sample size and, where the survey design makes
it relevant, the response rate.
- Some indication of the allowance that should be
made for sampling error.
- Which results are based on only part of the sample;
e.g., men and women, liberals and conservatives, etc.
- How the interviews were collected: in person,
in homes, by phone, by mail, on street corners, or whatever.
- When the interviews were collected.
August 7, 1978
Read
Determination 34
Back
to Main Determination Index
Want
to comment? Send a message
to the News Council.
