About Me

My Photo
Australian philosopher, literary critic, and professional writer. Author of FREEDOM OF RELIGION AND THE SECULAR STATE.

Thursday, March 08, 2007

Journalistic accuracy - an oxymoron?

I'm sometimes nasty about news reporters. Perhaps it's easy for me - whatever the pressures I've experienced writing articles, essays, reviews, works of fiction, and indeed blog entries, it must be tough reporting news stories under time pressure. It's a form of writing that I have no experience in, and I can see that it takes a special kind of skill and makes great demands on people's judgment and emotional energy.

That said, it's important to keep news reporters to high standards - their work contributes much to our picture of the world. If news articles are pervasively inaccurate - as they are in my experience - our understanding of the world is incrementally undermined.

Take the currently topical Essjay incident. Among the real facts are that Essjay made about 20,000 edits in his time at Wikipedia.

Of course the number of actual articles he edited was much lower. Blake Stacey gives a figure of about 6000 in his response to my previous post - but even this figure is much too high. As Blake points out in his follow-up post, it includes not just articles but other pages of various kinds. Without checking the figures myself, I know that the number of "entries" in the encyclopedia (which suggests actual articles on topics covered by the encyclopedia) that he worked on is much less than 20,000. Why?

First, no Wikipedia editor works by making only one edit per article. Once you start working on an article you are likely to make a number of edits - sometimes hundreds, though as it happens Essjay did not make a lot of edits per article that he worked on. But even if he'd averaged only two edits per article that alone would reduce the figure to 10,000. Second, many edits are not made to articles but to the talk pages of articles where the content of articles is discussed. Third, much else goes on at Wikipedia, such as exchanges on the talk pages of users who discuss issues one on one. Then there are discussions of policies, discussions of what articles to delete, etc. We also know that Essjay was not just any old editor but a busy Wiki-functionary, who would have been involved in a lot of background administrative tasks requiring edits to the encyclopedia to be made.

Moreover, one news report said that he worked on 20,000 "controversial" articles, but many Wikipedia articles are not really controversial, and there is no reason to think that Essjay worked mainly on the controversial ones.

In other words, from the fact that Essjay had about 20,000 edits, news reporters have basically made stuff up. They had no reason at all to conclude that he had edited anything like 20,000 "entries" or articles, let alone 20,000 "controversial" ones.

I've seen numerous other inaccuracies in news reports on this issue from reputable media organisations.

The thing is, I'm not surprised. In my own past dealings with the press, I've found that there are almost always inaccuracies in news stories, even when the material actually provided to reporters is accurate.

How does this happen? In the Essjay case, time pressure can't have been the reason - it would have been simple for reporters just to report the facts that they had, rather than making up facts - making up enough to give Joe Public an overall distorted impression of what the incident was really about.

If there are any working news journalists out there, perhaps you can respond, illuminating how and why this kind of thing happens. One guess is that the actual facts are often too dry-sounding, so journalists try to reword them in a way that will read better, but in doing so they make unwarranted assumptions, and so draw inferences that are not logically justified from the material they have in front of them. Perhaps jumping to conclusions in this way is just human nature, but it is very dangerous when news reporters do it, because the background facts may often be complex, even somewhat technical, and may not match the assumptions made by the reporter at all.

The Essjay thing is a case in point - you need a bit of semi-technical knowledge about how Wikipedia actually works to be able to see immediately that many of the claims made in news reports cannot be correct. But shouldn't reporters be aware of the risk that they are drawing unwarranted inferences in such cases?

I don't know enough about what goes on to be able to formulate a plausible theory as to how and why it happens; I just know it happens - and this has been a good case study. We'd be better off with more journalistic accuracy in news reporting - if journalistic accuracy in that context were not an oxymoron. Why are things as bad and they are, and what can be done to change it?

2 comments:

Blake Stacey said...

Thanks for posting this!

I should probably point to this section of the "Essjay controversy" Talk page, where several Wikipedians have taken a critical look at the "20,000 articles" claim. One of them points to this tool (programmed by Wikipedia editor Interiot) which computes summary statistics for any user's contributions. It also lists the pages in each category of Wikipedia which the user has edited the most. Looking only at the actual articles, we see the following pages topping Essjay's list, with his number of edits given in parentheses:

Cathedral of the Assumption in Louisville (58)
Roman Catholic Church (24)
Roman Catholic Archdiocese of Louisville (10)
George W. Bush (9)
Mass (liturgy) (9)
Centre College (9)
Deacon (9)
Justin Timberlake (7)
Gay (7)
Real Presence (6)
Confession (6)
Star Wars Episode III: Revenge of the Sith (6)
Transubstantiation (6)
List of cathedrals in the United States (6)
Eucharist (5)

I myself would count only two of these as "controversial" topics (although arguments over Star Wars can get pretty heated — just ask David Brin. . .).

Now, I've been around WP long enough to know where to find edit counters and statistics tools. I don't expect a reporter — even one whose beat is the software world — to know about Interiot's programs or the details of Arbitration Committee policy. However, if you're going to go to the trouble of interviewing Jimmy Wales, you should also poke your head into the WP community. As I said in the previous comment thread, if you're a reporter, your job is still to ask the questions and poke around the dark corners. Any decent investigation would have turned up things like Interiot's edit counter. If Jane Q. Blogger can do a better job than you, then you're not a journalist and aren't worthy of public attention.

Blake Stacey said...

D'oh! The first link I provided in my previous comment is now broken, since that fast-growing Talk page has been archived. Try here.