The "Pie Fallacy", And Why It's Important.
I don't have a lot of visitors here chez poisson – partly because my house is far too small for holding parties or accommodating more than two at the most for anything more than an hour without tripping over each other, and partly because, I'll admit, I tend to keep my place as my escape from the universe, rather than my window on the universe.
That said, when the bloke arrived from the Office for National Statistics yesterday afternoon, I thought I'd better let him in. They're the people who do the once-every-10-years census, due again in 2011, and since efficient governance in part relies on efficient information1 I feel it's my civic duty to participate in their data collection, subject to the usual privacy caveats.
I sat the bloke down with a cup of tea and he worked through a questionnaire on his laptop. After the usual demographical questions [age, gender3, religion, education, income] it became clear what they were collecting data on; firstly, people's attitude to "road pricing" in the UK, and secondly, people's attitude to mental ill health and the workplace, as set against – say – people's attitude to another disabling condition, chronic back pain.
He smiled as he learnt that the second was my "home territory", something I worked directly in towards the start of the last decade, and the questions – should employers be more sympathetic to / more supportive of / less quick to fear mental health conditions – got easy answers [generally along the "things have vastly improved, but there's still some way to go" kind of lines.]
The former was more problematic.
The main question: "Do you think road pricing [charging motorists by their mileage on busy routes] is a good idea, assuming there's tax reductions elsewhere to offset it so generally you'll end up paying the same amount?"
Ooooh, there's one for the Ben Goldacre "I think it's a bit more complicated than that…" category. I put forward my view, that of "generally it's an okay idea, but you have to use the carrot of decent, fairly convenient, and above all affordable public transport alternatives along with the stick and not just hope a new tax will sort out congestion by itself [because it won't: see London], and in rural areas like this you may as well forget it. And that's a gross simplification of all the issues of actual implementation that the idea of road pricing would bring up, and I could go on for a long time yet."
"Ah", he said. "All I've got is a scale of one to ten."
I shrugged and told him to put six.
————————————————–
Many moons ago, when I was doing the stats part of my degree, I talked about the difference between quantitative and qualitative data.
The former gives you a nice number ["54% of respondents in East Anglia were not hostile to the idea"] that you can use in a soundbite, and such data is crucial in things like evidence for medical interventions.
Qualitative, however, which is where the bloke would have written down my whole answer and not just ticked a box, gives you a plethora of people's responses, but no numbers. Someone has to sift through the whole data and spot trends, impressions, caveats, themes, rather than just feed the lot through a computer. The results they give are less hard data and more of a point in the general direction of which way the wind is blowing – often, though, showing things that quantitative data doesn't because you didn't think to ask the question in the way it's been answered.
To give you an idea of what I mean, consider the two approaches to this question: what's your favourite sort of pie?

The Quantitative approach to this question will give you some lovely numbers which you can put into an impressive-looking graph, something like -

[Yes, you guessed it; this is a pie chart. Badum-Tish!!!]
This may be useful, but as I've argued elsewhere, certain corporations [both public and private] seem to think that just because apple is the #1 pie, it's the only pie worth pursuing, and were they in charge of the baking and leisure foods industry you'd all get would be a bog-standard individual apple pie; if you wanted anything else, tough titty.
[The industry I can think of most prone to "the pie fallacy" is UK commercial broadcasting4 - just because reality shows and endless "talent" crap look like the biggest slice of the pie, doesn't mean that's license to fill your networks with them 24/7 to the exclusion of anything else that requires more than one brain cell.]
The Qualitative approach to the question, however, gives you no lovely picture to aim at; instead you'd get a long report on "UK Attitudes To Pies And Pastry-Topped Products" which would be very thick and dull to read, but – crucially – doesn't have that "killer data" you can leap on, instead possibly containing a paragraph along the lines of "43. Although the trend was for 'Apple' to be the most-mentioned pie when people were asked to name their favourite, there is no clear trend of pie preference and many respondents expressed their satisfaction with the range of pies available to them at most retail outlets."
Which is a very different piece of evidence to have at one's fingertips when debating and arguing a case.
——————————————————-
Sorry about all that. Ahem. Back to the subject.
A quantitative answer to the question of road pricing may be convenient for bureaucrats and politicians, but – like much public policy – fails to address consequences, to fully capture people's concerns, and to be able to tell how that other crucial bit of good governance – good implementation – will affect which way the political winds blow.
I'm concerned, then, that my being put down as "six" on the "do you like road pricing?" scale will turn out merely as being in the "fairly positive" percentage of a report, perhaps used as evidence that public opposition to a scheme may be softening, ready to drop on the desk of whoever becomes Transport Minister after this year's election.
Perhaps time for a stiff e-mail or three.
————————————————–
Today's Big Question: What's your favourite sort of pie? [Feel free to answer this question in a qualitative manner.]
============================
1 One of the arguments the revolutionary far-left people I know use is that whilst you can change the politicians, unless you change the Civil Service bureaucrats the status quo will endure2 and the same basic mistakes will happen. Seeing what a mess the Tories made of big IT and data management projects, followed by what a mess the Labour lot made/are making of big IT and data management projects, specifically NHS electronic data, and exactly how much good money has been thrown after bad over the past two decades, gives me quite a bit of sympathy for this view.
2 Cue this. [Sorry.]
3 I decided to cut a long story short and answer "male". I'm not sure he would have had a "genderqueer" box.
4 Though the BBC aren't immune either. "Hey, let's follow the inane dance contest at the end of last year with an inane dance contest at the start of this year!"