Colleges Ban Wikipedia
The New York Times published the story this week about colleges encouraging students to use sources other than Wikipedia as references for academic work. A professor in Middlebury College's history department initiated the policy after several students wrote on an exam that "the Jesuits supported the Shimabara Rebellion in 17th-century Japan". The professor noted that there were few Jesuits in Japan at the time and they were "in 'no position to aid a revolution'". Middlebury College is not the first to forbid references to Wikipedia¹, it's a growing trend.
Jimmy Wales, co-founder of Wikipedia, said he didn't consider Middlebury's decision "negative". Of course the definition of "encyclopedia" (or -"paedia"), is "course of general education" not, as some would have it -- 'a collection of definitive answers to all questions'. Others point out that Wikipedia is a tertiary source, not a secondary or primary source suitable for college essays.
The New York Times writes that the problem with Wikipedia is accuracy, however others aren't as critical, for instance the courts. Another New York Times article found that, "100 judicial rulings have relied on Wikipedia, beginning in 2004, including 13 from circuit courts of appeal, one step below the Supreme Court". Several studies have concluded that Wikipedia's information is comparable to other sources like Encyclopaedia Britannica.
Nature devised one of the studies, an "expert-led investigation" of 50 entries about scientists and scientific concepts. ("Internet encyclopaedias go head to head", December, 2005. Nature 438, 900-901). The journal appointed experts who deemed 42 of 50 articles surveyed "usable". The unusable articles included four each from Britannica and Wikipedia, which contained inaccuracies like "misinterpretations of important concepts". The review also found articles with "factual errors, omissions or misleading statements: 162 and 123 in Wikipedia and Britannica, respectively".
Internet Time vs. Britannica Time
Jimmy Wales, co-founder of Wikipedia, responded to the results, telling Nature he wanted to recruit more "experts" to write the articles. One reviewer said people would find it "shocking" to know how many errors were in Britannica. Britannica, lifeblood apparently draining, wrote a charged rebuttal (.pdf) to Nature's study. It began: "Everything about the journal's investigation, from the criteria for identifying inaccuracies to the discrepancy between the article text and its headline, was wrong and misleading." Over the next 20 pages it vilified the report as "poorly carried out", "error-laden", "without merit", and "without value". Britannica published the defense on its website three months after Nature's original story and took out an ad in a London paper demanding a retraction.
By the time Britannica got around to it's rebuttal people in and out of the media had digested Nature's results. A few questioned them, for instance one New York Times writer asked one of Nature's experts why he had flagged a fact in an article as inaccurate when his own book contained the same fact. ("The Nitpicking of the Masses vs. the Authority of the Experts". January 3, 2006). Other reporters distilled the results less analytically under titles like this: "'Nature': Wikipedia is accurate" (USA Today Dec. 12, 2005).
Nature wholeheartedly defended its methods and conclusions and refused to retract its article. For whatever reason, the journal was in the middle of an encyclopedia war -- and strangely -- on the open access side. Its article helped convince people that Wikipedia was more than just World Wide Web whimsy.
For its part, Britannica fought the perception that it was seeing its life flash before its eyes like a door in the face of an encyclopedia salesman. The Wall Street Journal (September 12, 2006) hosted an email forum between Mr. Wales of Wikipedia, and Dale Hoiberg, senior vice president and editor in chief of Britannica. Wales cited some links to articles critical of Britannica data. Hoiberg replied that there was ample criticism of Wikipedia too, but he didn't have it handy. Wales emailed back a Wikipedia.com link containing the entire body of criticism on Wikipedia, and took the opportunity to pedantically explain the joys accessing information instantaneously. Hoiberg cited Britannica's "trained editors and fact-checkers" and "more than 4,000 experts", processes, and strict editorial control. Wales taunted that those words were "fitting for an epitaph".
How do we Know?
Is it important that so called tertiary sources are squaring off about who's more accurate, or that colleges are urging students to use primary and secondary sources? Some commentators virtually shrugged. But important questions about how people verify information, what information is trusted, who can publish information and who controls information are at the heart of these debates. When bloggers began producing content, newspapers ranted on and on about how worthless blogs were. Many still do, although they also incorporate blogs into their online content. The PLoS publishing model motivated scientific publishers to hire PR firms who coined deceptive one line: slogans like"Public 'access equals government censorship'; 'Scientific journals preserve the quality/pedigree of science'; and 'government seeking to nationalize science and be a publisher'"
Wikipedia claims that anyone can publish information (with some limits). Many people criticize this model. The New York Times published a piece last month, titled "Anonymous Source Is Not the Same as Open Source". In it, the author said that employing "secondary epistemic criteria" is necessary to verify sources. "Once upon a time, Encyclopaedia Britannica recruited Einstein, Freud, Curie, Mencken and even Houdini as contributors.The names helped the encyclopedia bolster its credibility." The author's quote speaks well, if inadvertently, to the inherent problem. Who's an authority? Houdini may be a font of information but should he be plunked so close to Einstein? Should Freud be slipped next to Curie -- with only a comma separating them? The author continued: "The egalitarian nature of a system that accords equal votes to everyone in the ''community'' -- middle-school student and Nobel laureate alike -- has difficulty resolving intellectual disagreements." To the author perhaps Houdini is an authority to reference. To some he may be an authority on magic tricks of yore but nothing else. But Houdini may also had some insight up his sleeve on some other subject that would be a very valuable addition to Wikipedia. Wikipedia users could judge.
The author says we need to proxies for authority to assess information. Health and science data is especially daunting to assess, therefore we often rely pedigree. So credentials become the proxy for assessing knowledge, occasionally to a fault, as in: Nobel Laureate trumps Professor trumps Associate Professor trumps Assistant Professor trumps MD/PhD trumps Lecturer trumps Resident or PostDoc trumps PhD trumps MS trumps BS trumps Harvard Dropout trumps BA (or something like that). In science the gold standard for research is redoing the experiment, but such testing is usually impractical. These judgments often work, obviously, we will trust our doctor over a spam mail advertising the benefits of herbal health enhancers, but if we put too much faith in credentials or publishing record, we can unwittingly cede our power to evaluate information.
Government as Information Arbiter?
There's some literature out there on this subject and we stumbled across this paper titled: "The problem of online misinformation and the role of schools". The author proposed a two part solution for schools. One was to teach skills to help students assess data, which he fleshed out considerably. Secondly he suggested assigning "intermediaries" to vet sites and "promot[e] reliable sources of online information". For this, he proposed "government-sponsored Web portals and librarians". As far as I know, librarians already do this, so we focused on what he meant by "intermediary". He used medical information as an example of information difficult to evaluate for validity. He cautioned about the potential drawbacks and biases of many types of information sites even those from trusted government sources. He recommended MedlinePlus, part of the National Library of Medicine (NLM) as a good source for students because it was free of bias, amply funded, a well organized, and carried the right pedigree, which he defined as: "18,000 staff, including thousands of physicians and scientists in white lab coats, 106 of whom have been awarded Nobel prizes."².
It's certainly a sound enough recommendation, but nothing is this simple. Frederick Seitz probably wore a "white lab coat". He is a very credentialed PhD physicist, the recipient of the National Medal of Science, the Franklin Medal, the Herbert Hoover Medal, the Defense Department Distinguished Service Award, as well as two NASA Distinguished Service Awards, and The Compton Award. He is a President Emeritus of Rockefeller University Former President of the National Academy of Sciences, Recipient of the Fourth Vannevar Bush Award and the R. Loveland Memorial Award of the American College of Physicians, former President of New York City Commission for Science and Technology, Former Chair of the United States delegation to the U.N. Committee on Science and Technology for Development, as well as over 20 honorary degrees.
Seitz used his stellar credentials to obtain a job working for tobacco companies', and on their behalf he argued for several decades that cigarette smoke was benign. He also used his credentials to rally scientists against climate change evidence. He cited his awards to establish a foundation used to advocate "sound science", that bolstered political positions in order to undermine real scientific evidence. He often inserted himself and his impressive credentials in between business and public health, especially when business interests seemed in conflict with public health risks.
Reference Regulation
This isn't to denigrate the expertise of scientists and doctors and lawyers, but upon occasion experts are as fallible, capable of bias or deception as non-experts. In science and medicine, sycophants to pedigree have enabled huge sweeping, expensive catastrophes and personal tragedies. Renowned scientists have produced false data, and a recent study found that 44,000 to 98,000 Americans a year die from medical errors, many from credentialed doctors.
Nevertheless, everyday as consumers of information we must make decisions; judge the validity of a medical study funded by pharmaceutical companies, learn why science facts are excised out of government science reports, and try to figure out whether the "man on the street" is being candid about the technology or astroturfing while we live our busy lives.
Banning Wikipedia may rightly force students to find alternative sources of information but what data is reliable? Are professors guiding students, elaborating about how history books can be slanted? Do they explain that newspaper articles can distort the facts, as can the evening news? What biases do they bring to their lectures? Are we saying that primary sources don't have opinions, that their value systems are not intermingled with their accounts? We can hopefully vouch for the fact that primary sources said what they said, if they're speaking on camera, but do we know they meant it? Marketing and public relations have altered the landscape and many people have no compunction about standing up and lying on camera. Scientist who do primary research recognize the myriad challenges to designing and conducting experiments to generate and report accurate, relevant data. Pedigree is a very imperfect standard for assessing truth, as is the internet.3
The skill of assessing sources should be honed in college by practice not rules. A professor can be the arbiter of sources for a semester, and a college or librarian may serve that role for a few years, but our future depends on students mastering these skills for life.
----------------------------------------------------------------------
¹We're biased. We like encyclopedias (in general). We love Wikipedia's mission and are forever impressed with the information we find. We often link to Wikipedia to give readers background to subjects we editorialize. We also choose not to link to Wikipedia when articles about controversial medical procedures or public health/policy issues understate risks or read more like product literature.
²Suggesting that the government sanctioned sources is different than the proposed Federal Research Public Access Act of 2006 (FRPAA) which would require federally funded researchers to post papers online after six months.
³ There is a book on this called "Who Controls The Internet?" that's well-argued.
