10 February 2008

Irrationality at the BAFTAs

Atonement, it seems, is very popular - with the British Academy, anyway - it won best film, although Keira Knightley, James McAvoy and Joe Wright missed out on leading actress, leading actor and best director, respectively.

It also failed to scoop the award for the best British film, which reminded me of a logical fallacy I first read about in Stuart Sutherland's excellent book, Irrationality, in which a group of people were read a list of facts about a woman called Linda who "is 31 years old, single, outspoken and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations." They were then asked to rank a series of statements about Linda in order of how likely they were to be true.

The subjects thought it was much more likely that Linda was a feminist bank teller than she was just a bank teller, which is, of course, very illogical but the subjects were obviously looking at the "Linda is a bank teller" statement and thinking, nah, that doesn't sound like Linda, and then reading the "Linda is a feminist bank teller" statement and thinking, ooh, now I just bet Linda is a feminist, so that one is at least half true. As Sutherland explains, the subjects were averaging the probabilities instead of multiplying them.

The scenario at the BAFTAs isn't really the same, although it does rely on faulty reasoning and it does take me back to my days of truth tables in philosophy of language classes:

I bought roses entails I bought flowers
I bought flowers does not entail I bought roses
I didn't buy roses doesn't entail I didn't buy flowers
I didn't buy flowers entails I didn't buy roses

Similarly, the first and fourth lines of this truth table aren't upheld here, although the other two are:

Atonement is the best film should entail Atonement is the best British film (but doesn't)
Atonement is the best British film does not entail Atonement is the best film
Atonement is not the best film does not entail Atonement is not the best British film
Atonement is not the best British film should entail Atonement is not the best film (but doesn't)

Hmm, I'm beginning to think that the only point I have proven is that my logic anxiety really was the reason why I gave up on semantics in favour of pragmatics. Perhaps what is going on here is that different criteria are used to judge the two awards, but this isn't made obvious on the British Academy website. I guess they never anticipated visits from anyone who goes into hyper-pedantry mode in the face of such lacking logic: if it's the best film - the top overall film - and it is British, how can it not also be the best British film? It's like saying a dog is the best example of a pet and a trout is the best example of a pet fish but that the best example of a pet fish is actually a goldfish! Oh wait; I think I just added weight to Jerry Fodor's case....

On the bright side, this wild goose chase did remind me what a great book Irrationality is. Another favourite example involved a 1982 study where a couple of psychologists picked one paper from each of 12 top psychology journals by authors from the most eminent psychology departments in the States. They then removed the names and any potentially identifying information and listed the authors as having fictional names and affiliations (like Dr Ima Fake (no, not really) at "Tri-Valley Centre for Human Potential") and then resubmitted to the same journals. Three of the journals noticed that they had already published the article, but eight of the remaining nine rejected the resubmitted paper. Each of these eight papers was seen by an editor and two referees and all of them noted that the paper didn't merit publication in said journal.

Of course, there could be perfectly rational reasons for the rejection of the second submission of the paper (new developments in the field. Sutherland prefers to assume that these editors and referees are falling victim of a cognitive bias known as the halo effect. The first information the referee reads is the authors' names and their affiliations and if these are prestigious, it is likely to cast a warm glow of awesomeness over the whole paper. If, on the other hand, the paper is authored by Dr (or Ms) Random at Hicksville Polytechnic, the referee is more likely to look for flaws and to be more sensitive to what is wrong than what is right.

This was 26 years ago and thanks to Teh InterWebz and the joys of Google Scholar and, even, text mining, you might think that these results wouldn't be replicated and yet...I can imagine it being all too easy to do so. Irrationality does, after all, prevail, even in the peer review of scholarly journals; perhaps especially in the peer review of scholarly journals.

Edit: I refuse to spell BAFTAs Baftas, which according to Language Log, is a recent development in British newspapers (hence Defra and Nato), one I refuse to acknowledge. However, I do concede that at the very least it may highlight to people that BBC is not an acronym (but an initialism or alphabetism) and so is spelled in upper case rather than the funny-looking Bbc; acronym (abbreviation in which the resulting word can be pronounced as a word) vs initialism (abbreviation where the letters are spelled out one by one). Only an Eastern European could pronounce a word as vowel-starved as sftc...

No comments:

Post a Comment