I feel like banging my head against the wall (excerpt):
KUALA LUMPUR: Barisan Nasional is expected to win the 13th general election (GE13) which will be held this year, according to a study.
Titled “Study on Feedback of Undergraduate Voters”, it covered 3,000 respondents from Universiti Malaya (UM), Universiti Kebangsaan Malaysia (UKM) and Universiti Putra Malaysia (UPM).
The respondents involved undergraduates aged 21 and above from the Malay, Chinese and Indian communities, some of whom would be voting for the first time.
The outcome of the study, carried out from Dec 9 to 12 last year, also showed that Prime Minister Datuk Seri Najib Tun Razak is a popular leader, has good performance and qualified to be the Prime Minister.
The chairman of Yayasan Hal Ehwal Siswa, Datuk Mohd Radzi Abdul Latif said 57% of the respondents, representing UKM (75%), UPM (67.4%) and UM (41.4%), said the Prime Minister's leadership was good and popular.
“They feel that Najib is the best candidate for Prime Minister.
“The study also found that respondents who are on the fence will be the decisive factor for both sides,” he was quoted by Bernama as saying when announcing the result of the study here yesterday.
At the very least, this is a direct question put to potential voters, with a decent sample size and what looks like a good methodological approach (e.g. the question asked was essentially “who will win”, not “who will you vote for”, which removes some potential bias). The breakdown of the PM’s popularity looks plausible – though I’d dearly love to know what the number, gender and ethnic background of respondents for each university was.
But…when dealing with surveys such as this, you have to be really, really, really careful about making broad inferences. Right at the outset, the survey is already biased – it’s potentially random within the target sample, which is good (though I can’t tell for sure); but only for a subset of a subset of a subset of the population, which is very, very bad.
Can anyone pretend that voting preferences for 21-23 year olds, in tertiary education, in a select public university, is representative of all Malaysian 21-23 years olds? Much less the population at large? Can you say selection bias?
Currently, something like 30% of each Malaysian age cohort actually makes it into a university, and something like half that get into a public university. Presumably they got there because they’re smarter or harder working then their peers (questionable assumptions, I know, especially with quotas), but as such they may have very different viewpoints and preferences on politics and political parties than those who don’t
So you have a target sample that may not be representative of the population subset from which it is drawn. Furthermore, this larger population subset (21-23 year olds) may itself be uncharacteristic of the larger population set (all Malaysian voters).
Unless you know, or can objectively estimate, the extent of these differences (the bias built into the survey by its design), you can’t draw any broad conclusions, only conclusions specific to the target sample and its population subset.
Possibly I’m getting upset just over the headline – this won’t be the first time Bernama’s spinning makes me dizzy. But I’d be a blockhead to accept the conclusions of this survey (or that headline) at face value.
Why, oh why, doesn’t someone do a well-designed, fully representative, randomised, and country-wide survey? Surely there’s enough people interested and money floating around to get that done?