‘Shocking’ data suggested a fifth of the population was considering leaving the country. The National Party even shared it. But how accurate is it?
This story was first published on Stuff.
On July 8, the National Party released a statement. It said: ‘Shocking data released today showing that more than a million New Zealanders are actively considering leaving the country shows the government is pushing Kiwis back, said National Deputy Leader Nicola Willis.
It would certainly be shocking if a fifth of New Zealand packed up and left.
The National Party’s statement was based on a poll by MYOB, an Australian business services company, released the same day. The “Consumer Snapshot,” according to the MYOB website, “surveyed over 500 people from across the country and found that 4% of respondents plan to move overseas to live and work.”
“Based on demographic data from Stats NZ [that is the population of New Zealand], that number is extrapolated to over 200,000 people who are likely to think the same,” reads one blog post. The MYOB story also noted: “In addition, 20% of respondents (equivalent to approximately 1,025,000 people based on demographics) said they were actively considering moving overseas. “
The survey also addressed a number of related questions, such as what made respondents look abroad (money) and what would convince them to stay (more money).
MYOB’s own blog post has the catchy title: Kiwis take flight: Is New Zealand about to face a mass exodus? It is captioned: “New consumer research from MYOB indicates that more than one million New Zealanders may actively consider leaving the country in the next 12 months.”
This particular study was reported by Stuff (not geared towards the one million figure), Newshub (very geared towards the one million figure) and referenced in a column by Deputy ACT Party Leader Brooke van Velden published by the New Zealand Herald.
It was also shared by the National Party with the title “ONE MILLION KIWIS READY TO GO”.
Is there something in all this?
It is true that more people leave New Zealand than enter it.
Data from Stats NZ shows that in the year ending May 2022 there was a net migration loss of 10,700 people – made up of a loss of 8,400 non-New Zealand citizens and 2,300 Kiwi citizens. The latest figures for May are interesting as they suggest more New Zealand citizens are leaving than returning – a major reversal from when borders were closed. In the 12 months to May 2021, for example, some 10,400 Kiwi citizens left and 26,400 returned.
(By the way, another completely independent poll from November 2020 suggested that a million New Zealanders abroad were considering returning home.)
Government officials have suggested around 50,000 people could leave next year as the border reopens. There is, however, significant uncertainty around this prediction.
To be fair, it’s not like National is the only political party to seize on the underlying narratives for political gain. For example, you may recall hearing in 2020 that there was a “deluge of New Zealanders” returning home, driving up property prices. Prime Minister Jacinda Ardern has pushed this narrative under intense pressure over housing affordability. But that ignored the fact that net migration was down at the time.
Either way, the concerns about migration are valid; this is a real problem that risks increasing the pressure on an already tight labor market. But expecting 200,000 (or a million!) to leave is a lot.
Can you discuss the study in detail then?
I thought you would never ask.
These types of eye-catching studies get sent to the media all the time. Some are ignored. Some are picked up.
The July 8 press release contained few details on the methodology behind the poll. He pointed out that the research was conducted by Dynata (a data and information company) between June 3 and June 14. It said 509 consumers were asked to complete the online survey “which was conducted to obtain a nationally representative sample from across New Zealand”.
I asked for more details this week, particularly around the sample of 509 New Zealanders and the question they were asked.
Let’s start with the key question posed to respondents. The consumer snapshot asked all survey respondents: “Now that the borders are reopening, have you thought or considered moving abroad to live and work?”
- 4% answered “Yes, I have decided to move”
- 20% answered “Yes, I thought about it, but I haven’t put it into practice yet”
- 37% answered “No, I didn’t think about it”
- 33% answered “No, I will never move abroad”
- 6% were undecided
I asked Carl Davidson, director of Research First in Christchurch, for his opinion on the framing. According to him, the question that prompted 20% to say “Yes, I thought about it but I haven’t put it into practice yet” was potentially preponderant and could have been structured differently. For example, it would have been better to use the following:
Now that the borders are reopening…
- Have you thought about moving abroad to live and work?
- Or have you considered moving abroad to live and work?
And both of those questions, he points out, had two parts – whether people would live AND work. It is best to narrowly target the survey questions to get good reliable results. There is also, of course, a huge difference between what people say and what they will do. I mean, I’ve personally thought about moving overseas, but I definitely won’t.
But still… 4% said they had decided to leave, right? Well, is the poll representative of all New Zealanders? Dynata and MYOB say yes, it does.
In a statement provided this week, Dynata said the company “conducts surveys that are scripted by customers. As such, we do not make statements, but rather collect data for questions provided by MYOB… We adhere to rigorous standards to ensure that the final composition of survey respondents was consistent with nationally representative statistics by age, gender and region.
MYOB’s response read: “Dynata surveyed 509 New Zealand adults from across the country. To do this, they relied on a sample of the public drawn from Dynata’s online panel. Panel participants choose to participate in individual surveys, in response to an invitation from Dynata.
His statement stressed that polls only take place if the sample is representative of the population. According to Dynata, the poll was statistically robust with a 95% confidence interval.
So what did Davidson think of it?
“In science, the rule is, ‘Extraordinary claims require extraordinary evidence.’ Given that the highest external migration New Zealand has ever seen is around 100,000 people a year, the idea that “a million Kiwis are about to leave” is clearly an extraordinary claim.
“To back up that claim, you would need compelling data, and it’s not clear that the MYOB survey can provide that. Given how survey participants were selected and the questions they asked them and, above all, the huge difference between what people predict they will do and what they will end up doing, the sensible answer is to remain skeptical.
And the National Party?
In a written statement, Willis says his language when commenting on the investigation matched that of MYOB itself. “Respondents gave reasons why they were considering leaving, indicating that they had more than a passing thought,” Willis says.
National, she says, is “deeply concerned” that a significant number of New Zealanders are actively considering leaving.