Oct. 3rd, 2012 09:32 pm
liv: cup of tea with text from HHGttG (teeeeea)
[personal profile] liv
A couple of days ago, I got the chance to take part in an opinion poll. It's something I'd always wanted to do, cos I like being asked my opinion about stuff, and I sort of want my opinions to be part of the national scene.

So a chap showed up at my door about 7 pm, and said he was from Ipsos MORI. I have somewhat warm fuzzy feelings towards Ipsos MORI because [ profile] shreena used to work for them, and I didn't have any very urgent plans for that evening, so I asked him in.

The whole thing was much less smooth and professional than I expected. The interviewer didn't entirely know what he was doing; he mentioned being recently trained. And he said it would take half an hour, but it was more like an hour, because he was chatty and because he was really slow finding the way round his survey program on his laptop. I quickly got the impression he had taken the job mainly because he's somewhat lonely, and this was rather confirmed when he told me a rambling story about how he'd lost his job as a postman because after going through cancer treatment he was no longer physically fit enough to do it. I'd just been teaching the med students about how to tell the difference between jaundice and variation in skin tone, and with this guy I wouldn't have been prepared to bet either way.

I think a lot of people would probably have been uncomfortable with the amount of personal stuff he was rambling on about, especially in the situation of a woman alone in the house being visited by a strange man. I mainly got a little extrovert buzz from the encounter, I'm interested in people and their stories. But I was still a little bit annoyed, because I felt he was imposing on my time and politeness a bit, and going beyond the bounds of what is expected as part of his job. I'm happy that he was a little chatty and personal, and didn't just robotically read through his questions, but I think he went somewhat too far. He also wasn't very good at avoiding influencing my answers or expressing his own opinions about some of the questions. He even stood in the doorway chatting about nothing much once the interview was over; I only managed to persuade him to leave because my mother phoned while he was still babbling about some benevolently gender-essentialist thing he'd read.

For some of the questions, he was allowed to show me the computer screen, and my goodness the program is clunky as all get-out! I understand that you can't really give your researchers fancy iPads or fast, shiny laptops, because you'd be setting them up as a target for mugging. But even a slow computer can have a decent UI, and this really didn't.

The questions were an utterly random mix, I had expected that a proper polling org would ask mostly political questions, but there were about equal numbers of political ones and more market-research type ones about brands and shopping habits. Having been exposed to qual research recently, I have ideas in my head about survey design and such, and I would say that the questions I faced were above average but not as great as I might have expected from an expert polling company. A few were confusing, such as using multiple negatives or very unclear definitions.

The computer completely choked on the concept that I am married, but live alone. The interviewer himself was totally fine about it, accepting that the problem was in his software, not my reality, but still. Silly narrow-minded demographic categories! So anyway, now my views about elected police commissioners and switching between banks are part of the pile of data that people will look at to decide things, and that's quite cool, even if it was a bit more of a waste of evening than I would really have liked.

(no subject)

Date: 2012-10-03 11:06 pm (UTC)
redbird: closeup of me drinking tea (Default)
From: [personal profile] redbird
I've done opinion polls, though not recently, but they were all either over the phone or online. Maybe American polling companies think (or know) that we won't invite random strangers in to ask those questions?

I am amused by their computer system choking on your living arrangements; even if it has never occurred to them that someone might choose that, they should have noticed that it happens, in part for job-related reasons.

(no subject)

Date: 2012-10-04 10:47 am (UTC)
antisoppist: (Default)
From: [personal profile] antisoppist
I did one of these but the researcher was a woman in her fifties, who I felt comfortable about letting in (and seeing the state of the toy-ridden living room).

(no subject)

Date: 2012-10-04 11:15 am (UTC)
khalinche: (Default)
From: [personal profile] khalinche
This more or less exactly describes my experience of being surveyed by IPSOS-MORI, down to feeling well-inclined towards them because Shreena used to work for them. My interviewer was not a postman recovering from cancer, but he was talkative to the point of me getting annoyed for him taking up my time to tell many stories about his life, rather than actually getting on with the survey. It was a long hour which was mostly me listening to his opinions and family anecdotes, and as you say, the software he had was made out of plywood and Formica. I'd probably still do it again, even so.

(no subject)

Date: 2012-10-04 02:29 pm (UTC)
zhelana: (Default)
From: [personal profile] zhelana
Sorry your wish didn't end up awesome. But I am surprised the postmen have to be physically fit. I mean, they just drive around, right?

(no subject)

Date: 2012-10-04 05:41 pm (UTC)
evilsusan: (Default)
From: [personal profile] evilsusan
Ours cycle from the next village.

(no subject)

Date: 2012-10-13 02:02 pm (UTC)
lovingboth: (Default)
From: [personal profile] lovingboth
Lugging heavy bags around... Both here and in London, they're typically dropped off by van into caches, but 'from there on in, you walk'. Some here cycle, but they're the ones going to the extremities and there's still a significant amount of lifting and walking.

(no subject)

Date: 2012-10-04 04:01 pm (UTC)
pretty_panther: (sw: r2 and 3po in the desert)
From: [personal profile] pretty_panther
I hate when people go off track too much like that. It becomes awkward. I don't like letting people like that in because I have social anxiety but my mum does it sometimes and the variation in how different people conduct themselves is very interesting.

(no subject)

Date: 2012-10-04 07:03 pm (UTC)
tig_b: cartoon from nMC set (Default)
From: [personal profile] tig_b
[soapbox starts]
I don't have warm fuzzy feelings about most market researchers - having battled too many times trying to explain why 'my' data and analyses were better indicators than any 'facts' from than self-selected respondents, poorly designed instruments with leading or biased questions, and excessive interviewer bias.

I do know that some of them do a good job, but s long as we have journalists and politicians who are almost innumerate the bad stuff will get headlines.

[soapbox ends]

(no subject)

Date: 2012-10-04 08:17 pm (UTC)
tig_b: cartoon from nMC set (Default)
From: [personal profile] tig_b
I worked as a Govt Statistician.

One of my first small projects for big boss was to draft a note, to be attached to all copies of a monthly publication, explaining why their 'survey' data shouldn't be used.

I had to visit a couple of the policy people to explain further, as they continued to use the figures for a while.

The 'survey' was of their own members, and with a response rate of between 8% and 12%, but they wrote it up as representing the UK business population.
Very naughty.

Official Govt statistics are not perfect, but we do publish the methods, coverage, etc. and the caveats.
and then we watch the spin drs and others producing catchy headlines - groan.


Date: 2012-10-05 07:46 am (UTC)
From: (Anonymous)
A. The mix of questions is a necessary feature of good surveys, to avoid leading questions.

B. I am mildly concerned by the apparent amateurish style, though your pollster was almost certainly genuine. Did you check his credentials?


(no subject)

Date: 2012-10-13 02:04 pm (UTC)
lovingboth: (Default)
From: [personal profile] lovingboth
The other reason for a mix of questions is that people can pay per question to have them included.

My experience is that it's much easier to say 'let me read it, yes, one, c, d, e, a..' It takes a fraction of the time and they're paid by completed interview, not per hour.

(no subject)

Date: 2012-10-13 09:32 pm (UTC)
shreena: (Default)
From: [personal profile] shreena
Heh. I'm amused that you were well disposed to them because I used to work there!

To respond, slightly haphazardly to a few things in your post/comments -

Actually, having been out with interviewers a few times (researchers do this from time to time to quality assure/gain experience/etc)and from having seen a LOT of fieldwork stats in my time, I'm not convinced that the demographic that let the interviewers in is that unrepresentative. A good random probability survey will get over 50% in response rate, sometimes as high as 65%. A surprisingly high number of people will let interviewers into their homes (though some will make the interviewer stand outside and do it like that.)

Interviewers are more likely to be female than male - the Ipsos MORI fieldforce is about 70% female.

The computer system is clunky but it's quite difficult to make it less so for a few reasons - one is that it's all quite complicated, the interviewers work from home and need to be able to upload the questionnaire scripts, the questionnaire scripts need to be able to export the data sensibly to be able to do quite high level things, the interviewers need to be able to update every day or so on how they're doing, how many interviews they've got/what demographics, etc; and the other thing is that a lot of the very good interviewers (the personable ones that get good results) are, frankly, poor at computers and need something simple.

The interviewer really shouldn't have been leading your answers/etc. If you were to write in/complain, he'd almost certainly get some further training/advice/help from his supervisor.

Questionnaire design - some of it will be poor work from Ipsos MORI staff, no doubt, but, in my experience, a lot of it is that clients are often wedded to asking questions in a particular way and simply won't listen to advice from the researchers.

On questions jumping around - it sounds like to me that you were doing an omnibus survey, i.e. one where different clients have bought questions. Those almost always sound a bit weird as you're going through, the ones which are all one client generally flow better.

One of the issues with interviewers generally is that they have to be personable and good with people to get the interviews but that sometimes leads to them being a bit inappropriately personal with interviewees. It's a fine line.


Miscellaneous. Eclectic. Random. Perhaps markedly literate, or at least suffering from the compulsion to read any text that presents itself, including cereal boxes.

Page Summary

Top topics

October 2017

8 910 11 121314
15 161718192021

Expand Cut Tags

No cut tags

Subscription Filters