Voluntary Census long-form questionnaire: Wasting 35 years worth of Canadians’ census effort

Tuesday, 20 July, 2010

Statscan 2011 census

Statistics Canada and Census are not typical summer fireside chats topics but Industry Minister Tony Clement managed to create a firestorm around the 2010 Census by changing the long-form questionnaire from mandatory to voluntary.

Concerns & Oppositions

An ad hoc Census coalition of bankers, economists, medical professionals, academics, pollsters and other census users have expressed their collective concern in a coalition letter to Minister Clement, (emphasis added)

“We are greatly concerned about this decision [to replace the Census long-form questionnaire with a new voluntary questionnaire]. Loss of the long-form Census information will cause considerable economic and social costs.

In a phone interview with Census coalition spokesperson Mel Cappe (mp3) (streaming audio), former clerk of the Privy Council from 1999 – 2002, I chatted with Cappe about the process of picking the 2001 census questions and answers. And discussed his group’s concern of a break in the census data series if the filling of the census long-form questionnaire becomes voluntary.

Wasting 35 years worth of Canadians’ census effort

Cappe stated, “For the last 35 years, people have been filling out this long-form of the census in one form or another. And we have been doing this for over 130 years. And now from 2011 forward, we will not have a data point. That means that all those people who filled out the form in the last 35 years did so for nought. Because we won’t have the next point on the series.

How much time would filling the mandatory census long-form questionnaire take? Cappe explained, “20 percent of the population get asked every five years to fill out this form. […] That means once every 25 years, you got to spend about 30 minutes in answering 41 questions.” To most Canadian citizens, spending about 30 minutes once every 25 years is completely reasonable for the public good of Canada and a reasonable duty.

Impact on Canadians’ healthcare

Canadian Medical Association Journal editorial “Ideology trumps evidence with new voluntary survey” states, (emphasis added)

“[Information from the long-form census] provides accurate and reliable data on social trends and issues, including the determinants of health, such as the relationships among income, gender, education, region, work and other factors that influence access to care and health outcomes. In fact, the long-form census is the only source that brings all these variables together and enables researchers to investigate new ways of understanding the determinants of health.

Opposition from within Statscan

According to Globe and Mail’s “Clement accused of misrepresenting census impact – Statscan insiders say Industry Minister’s comments playing down effects of voluntary survey enraged staff“, (emphasis added)

Mr. Clement has said Statscan officials reassured him the agency can manage the 2011 census effectively without forcing some people to fill out the longer version of the form.

That’s not what Mr. Clement has been told, according to a source close to the story who asked not to be identified, and Statscan officials expect chief statistician Munir Sheikh to come to the agency’s defence by saying so. [K: I hope to see the chief statistician's expert view added in the public discussion real soon.] [...]

Don Drummond, a member of Statistics Canada’s advisory council, said “all of us were shocked” by the news that the mandatory long-form census was being abandoned.

The approximately two dozen members of the advisory council are appointed by the industry minister, and advise the agency on how better to carry out its mandate.

Mr. Drummond, who recently stepped down as chief economist of the TD Bank, said the council unanimously believed that abandoning the mandatory long-form census would skew the 2011 results, causing a statistical break with previous surveys that would it make impossible to read and project trends accurately.

Intrusive questions

To justify his decision, Clement claims some of the census questions are intrusive. But what he should have done was to change the wordings or simply rejecting the questions during the census questionnaire refining process (see 2011 Census content determination process on page 10 of the Census Content Consultation Guide (pdf)).

Provincial governments opposing decision to scrap long-form census in favour of a voluntary application

According to CTV News (with video, emphasis added),

The governments of Ontario, Manitoba, Quebec and Prince Edward Island have all come out against the idea, reminding Ottawa that data collected from the mandatory census enables them to draft policy and deliver services.

Canadians’ personal data collected are anonymous and protected by Statistics Act & Privacy Act

Cappe stated, “There has never been a case, in the history of Canada, in the history of Statistic Canada where someone’s personal census data has been released. All that is released are the aggregation by census track so they add them up.  [...] Statistic Canada has an unblemished record of keeping to themselves – private – all of the returns of the census.” So some of the questions may seem sensitive but they are never used to identified Canadian individually.

Anecdotal support vs. thoughtful statistical understanding

It is unfortunate to see Clement relying on anecdotal encouragement from supporters (via Twitter Julius, Adam, Patrick, Paul, Chris, Tyler + Elizabeth) instead of putting more emphasis on thoughtful statistical understanding of the long term negative impact his decision.

In response to this user’s tweet, Minister Clement tweeted back “Actually 168,000 felt strongly enough last time about mand long form to refuse on pain of jail. Yet that sample was deemed valid.” Census coalition spokesperson Mel Cappe, in response to Clement’s statement, suggested if Clement thinks prison sentence is too harsh, then may be a fine of $500 for non-compliant can be used. Have a listen to Cappe’s full response in my phone interview (mp3) with him.

Concluding thoughts

I think Cappe said it right and worth repeating here, “For the last 35 years, people have been filling out this long-form of the census in one form or another. [...] And now from 2011 forward, we will not have a data point. That means that all those people who filled out the form in the last 35 years did so for nought. Because we won’t have the next point on the series.

It should also be noted that many public and private surveys, including the important Labour Force Survey which tells us employment and unemployment figures in Canada, depend on a statistically valid set of Census of Population.

Now, I think it is safe to say most Canadian citizens, as a duty and for the public good of their country, won’t mind spending about 30-60 minutes once every twenty-five years to fill in a mandatory census long-form questionnaire.

*******

Note: Repeated email questions and phone calls to Statistics Canada have not been returned at press time. It will be interesting to know what has Munir A. Sheikh, Chief Statistician of Canada told Minister Clement? Did the Chief Statistician actually tell Clement that there will be no negative impact by making the long-form voluntary? And previous census results will NOT be less useful as a result this change?

***

July 21 update:

- “In an op-ed for the Sun, Tony Clement manages to twice cite the fact that the long form census includes a question about the number of bedrooms in one’s dwelling.” This Macleans article explains some legitimate use of the bedrooms data: “… government planners and private developers to develop housing communities and projects … Provincial and municipal governments use this information to measure levels of crowding within households and to develop appropriate housing programs.

- Tony Clement answers questions from Globe and Mail in “Tony Clement clears the air on census”. Clement’s answers seem evasive and less than forthright to me.

July 21, 2010 10:48pm MST update:

- “Munir Sheikh, Canada’s chief statistician, resigns to defend integrity of 2011 Census

- “Canada’s Chief Statistician Resigns Amid Row With Government Over Census“, Bloomberg

- “StatsCan chief quits over census furor“, TorStar

- “Federal statistical folly in full view“, Globe and Mail Editorial

- “StatsCan head quits over census dispute“, CBC News


Canada’s mandatory long-form census – Flawed arguments for census changes

Monday, 19 July, 2010

For the record, I agree with the attached Globe Editorial. And I think it is perfect reasonable duty for one in five Canadian citizens to spend 20-30 minutes every five years to share information in an anonymous manner to facilitate government and business decisions. Tell Minister Clement what you think via twitter.

Quoting the editorial, “The elimination of the long-form questionnaire will result in losses across many areas of Canada’s public life, well described by academics and policy-makers.

Yet the federal government has yet to offer any reasons for getting rid of it that can stand up to scrutiny. And that is no basis for public policy of any sort.

Flawed arguments for census changes” (emphasis added)

- The elimination of the long-form questionnaire will result in losses across many areas of Canada’s public life. Yet the federal government has yet to offer any reasons for getting rid of it that can stand up to scrutiny.

From Monday’s Globe and Mail
Published on Sunday, Jul. 18, 2010 8:00PM EDT

The federal government has advanced a number of arguments to justify the abolition of the long-form questionnaire in the 2011 census in favour of a new voluntary “National Household Survey.” Each claim is flawed, faulty or incomplete, and together they display a misunderstanding of the nature and purpose of census-taking, and a worrying approach to governing.

Tony Clement has made three assertions or suggestions. The first is that the National Household Survey, to be sent to 33 per cent of households (the long-form questionnaire goes to just 20 per cent), will be just as accurate.

But while surveys are inherently statistical, the census is primarily a counting exercise.

The census has its own sampling procedures to ensure data accuracy. But those problems can be managed. According to a Statistics Canada technical report on sampling in the 2006 Census, “calibrating sample estimates to known population counts as part of the census weighting procedures helped to reduce the impact of biases.”

Voluntary surveys, on the other hand, are known to create much larger biases, with lower participation from the poor, the very rich and aboriginals. These generate results that do not reflect the population and require adjustments.

These biases are harder to detect without a good base of objective knowledge. Increasing the sample size, from 20 to 33 per cent, is irrelevant; cancelling the long-form census removes that objective base.

The long-form census also creates reliable data that is the basis for almost all other major sampling exercises in Canada.

Ivan Fellegi, chief statistician of Canada for 23 years, said that the sampling for the Labour Force Survey, which generates unemployment statistics, is based on long-form census data.

And if the long-form questionnaire is abolished, it will be hard to compare the data to past censuses.

In other words, the new survey cannot use the same standards as past censuses, nor can it deliver the same level of data accuracy, and it compromises future surveys.

Mr. Clement has stated that the long-form census questionnaire has generated many privacy complaints.

Canada’s Privacy Commissioner has received just three complaints in the last two censuses. The House of Commons has no record of any petition tabled by an MP about these concerns. Nor has it come up in private members’ statements.

In addition, governments still do mandatory “intrusive” surveys. The Labour Force Survey is mandatory. The Canada Revenue Agency requires disclosure of all sources of income.

And completion of the 2011 Census of Agriculture is also mandatory; failure to participate can lead to a fine or prison.

That census requires a report on the area of fields irrigated, the number of live honeybee colonies hosted and the incurred cost of veterinary services, to name just three examples, of every farm in Canada.

Finally, Mr. Clement said that Statistics Canada vouches for this new process.

However, the agency itself refused to go on the record in support, saying, “Statistics Canada is not in a position to answer questions on the advice it gave the Minister in relation to recent statements the Minister has made.”

If it fully supports the measures, Statistics Canada should be forthcoming with its explanation and analysis.

Taken together, the government’s arguments fail to justify the course chosen.

These are not abstract considerations. The elimination of the long-form questionnaire will result in losses across many areas of Canada’s public life, well described by academics and policy-makers.

Yet the federal government has yet to offer any reasons for getting rid of it that can stand up to scrutiny. And that is no basis for public policy of any sort.


Follow

Get every new post delivered to your Inbox.

Join 678 other followers

%d bloggers like this: