ARCHIVED Annual Report 2007-2008

• Previous Page
• Table of Contents
• Next Page

How Public Opinion Research is Managed in the Government of Canada

The Treasury Board Communications Policy of the Government of Canada sets out a framework for conducting public opinion research. The framework is mandatory for departments and agencies identified in schedules I, I.1 and II of the Financial Administration Act. The policy does not apply to other institutions, such as Crown corporations and those authorized to undertake their own contracting under sections 41(1) and 41(2) of the Act. This report focuses on the POR activity of institutions covered by the policy.

Departments are at the centre of the overall process. They are responsible for the quality, content and management of their POR activities including acceptance of all deliverables and payment of suppliers.

The Treasury Board Secretariat (TBS) advises and supports the Treasury Board and its president in the development, management and evaluation of administrative policy. Under the Communications Policy of the Government of Canada, TBS is responsible for developing, evaluating and reviewing government-wide communications policy and advising institutions on the policy interpretation.

Library and Archives Canada (LAC) receives final research reports from departments and agencies and makes them available to the public. It also has a Web site on which it posts executive summaries of research reports and other basic details. This ensures that POR reports are preserved for the use of present and future generations. Library of Parliament (LP) makes the reports available to the members of Parliament and the press.

The Privy Council Office (PCO) has a central role in the coordination and management of government communications.

Public Works and Government Services Canada (PWGSC) works with other federal organizations to serve Canadians efficiently and cost-effectively. The Department is the coordinating and contracting authority for public opinion research within the government. It exercises this authority through two of its directorates, the Public Opinion Research Directorate and the Communications Procurement Directorate, which work closely with each other to provide seamless services to federal organizations.

• The Communications Procurement Directorate (CPD) provides a mandatory common service as the government's contracting authority for public opinion research.
• The Public Opinion Research Directorate (PORD) provides a mandatory common service for the coordination of public opinion research activities. When federal organizations plan public opinion research projects, they must consult the Directorate from the initial stages. The Directorate helps departments and agencies with their research needs. It also facilitates the procurement of services, shares best practices and research results, and coordinates work across federal organizations.

Expanding the Frontiers of Knowledge

Advisory Panel on Online Public Opinion Survey Quality

PWGSC convened a panel of experts to adapt the relevant standards of the market research industry and to develop benchmarks for surveys. The Advisory Panel on Online Public Opinion Survey Quality builds on previous initiatives to develop standards for telephone survey quality.

The standards and guidelines developed for both online and telephone surveys will help ensure greater consistency in the quality of opinion surveys conducted for the Government of Canada. They will be incorporated into future contracts for custom public opinion research studies.

The panel, chaired by PWGSC, consisted of public opinion research experts from the government, the Marketing Research and Intelligence Association (MRIA) and the academic community. The panel conducted its work through in-person, telephone and online discussions. It focused on six areas:

• Pre-field planning, preparation and documentation;
• Sampling;
• Data collection;
• Success rate;
• Data management and processing; and
• Data analysis/reporting and survey documentation.

The results have been shared with departments and the MRIA (PDF Version 210.69 KB) (Help on File Formats).

A Survey of Cellular-Telephone-Only Households - The New Technologies (Web 2.0) and Government of Canada Communications Project

A small but increasing proportion of Canadian households are serviced by cellphones only and are therefore excluded from landline surveys. In this context, PWGSC conducted research to assess the feasibility of surveying cellphone-only households in addition to landline telephone households (PDF Version 112.92 KB) (Help on File Formats).

Because virtually no research exists in Canada on this topic, PWGSC wanted to examine cellphone-only households' interest in and potential barriers to participating in cellphone surveys. PWGSC also wanted to determine how survey respondents in cellphone-only households differed from households that respond using landline telephones, in terms of how they prefer to communicate with the Government of Canada.

The results of this study are being used to provide advice to government departments on how to best reach and conduct surveys with cellphone-only households.

The survey report reached the following conclusions:

• Members of cellphone-only households are among a younger demographic, more likely to be male, live in smaller households and report lower levels of household income;
• Telephone surveys conducted by random sampling of landline telephone numbers may no longer be sufficient in the future to capture a broad range of Canadian opinion. They may need to be supplemented by online panel surveys and cellular telephone surveys; and
• People in cellphone-only households are more frequent users of the Internet than persons with conventional landline telephones and are also more likely to use Web 2.0 applications such as social networks and YouTube. Excluding cellphone-only Canadians could affect the results of surveys on the use of technology and possibly many other topics.

With these results, the Government of Canada can better understand how to reach Canadians so that they can respond to research studies. The report will assist public opinion researchers in the government and the marketing research industry across Canada to make strategic choices about Internet, landline telephones and cellphones as data collection tools.

Advertising Campaign Evaluation Tool

In 2007-2008, PORD, in collaboration with PCO, commissioned a study to review the Advertising Campaign Evaluation Tool (ACET) used to evaluate the impact of the Government of Canada's larger advertising initiatives. The aim of the review was to help determine the extent to which the tool met the government's current evaluation needs, and whether any improvements should be made.

The study involved: 1) a review of literature, including numerous academic and industry publications and government post-campaign evaluation reports; 2) a set of in-depth phone interviews with individuals from government departments and agencies that used the ACET, academics that have expertise related to advertising, representatives of other governments responsible for advertising campaigns, advertising research suppliers, and representatives of private sector firms; 3) testing of both the proposed interviewer-assisted and self-administered versions of the ACET, developed through the first two phases of research; and 4) a review of two pilot tests of the proposed revised tool.

As a result of this study, a new advertising evaluation instrument was produced. The revised version is now shorter and more flexible than the original ACET, saving money on questionnaire design, fieldwork and translation costs. Also, a self-administered version of the instrument has been developed. The self-administered version can be managed via kiosks or the Web and thus has the potential to offer considerable cost savings.

Sharing Results with Canadians

Since 2006, the Communications Policy of the Government of Canada has required institutions to deposit their final research reports with Library and Archives Canada and the Library of Parliament within six months of the completion of fieldwork. Institutions must also send a copy to PORD. In 2007-2008, 440 projects were completed and the reports were submitted directly to Library and Archives Canada and the Library of Parliament. These reports include studies conducted before and during the 2007-2008 fiscal year.

As part of a new initiative, three departments: Environment Canada, Health Canada and Industry Canada, donated survey data files to the Canadian Opinion Research Archive (CORA). The aim is to maximize their usefulness to Canadian society by making the material available to social scientists across the country. Founded in 1992, CORA is administratively accountable to the School of Policy Studies at Queen's University. It makes commercial and independent surveys available to academics, researchers and journalists. The archive contains hundreds of surveys dating back to the 1970s. Its Web site (www) carries results from these and more recent Canadian surveys, as well as general information on opinion research. In the same manner that Library and Archives Canada maintains a collection of all POR studies, CORA keeps data files available for the use of present and future generations of Canadian researchers.

Sharing Knowledge within Government

The Community of Practice

The Public Opinion Research Directorate continued to coordinate the federal Community of Practice and participate in other interdepartmental working groups during the past fiscal year. The community includes POR practitioners from 52 departments and agencies. It met regularly to share information and address issues of common concern, such as survey quality and response rates, procurement issues, emerging research techniques, new survey software, joint initiatives and partnerships, and new legislation and policy requirements.

PORD's Knowledge Management Unit seeks to foster education and capacity-building within the Community of Practice. In 2007-2008, the Unit organized learning sessions on various subjects. They included:

• The MRIA's standard courses on qualitative research and on ethical issues and privacy in marketing research. These courses are part of a series being offered to Government of Canada employees to enable them to attain the MRIA's Certified Marketing Research Professional (CMRP) designation;
• A Statistics Canada course on survey non-response bias. This course was designed to implement one of the recommendations of the Advisory Panel on Telephone Public Opinion Survey Quality, which was, "Every telephone survey should include an analysis of the potential for non-response bias based on information collected during the normal conduct of the survey"; and
• Developing a new program entitled Marketing and Business Intelligence Research at Algonquin College in Ottawa. In 2007-2008, the Public Opinion Research Directorate participated in the Advisory Committee for this program, which will be introduced in the fall of 2009 (www). The program will prepare students to write the examination for the Marketing Research and Intelligence Association's Comprehensive Marketing Research Examination. In addition, students will receive training and preparation for careers in the private and public sectors.

The Unit also organized a speakers program. The major themes of the Public Opinion Research Directorate's 2007-2008 speakers program were: using innovative technology to conduct research; conducting research in a diverse society; and maintaining high standards for data from quantitative and qualitative research.

Under the leadership of PORD, the Community of Practice also participated in three informational meetings with an interdepartmental working group to discuss experiences using online survey software. The working group shared knowledge on required and desired features of this software.

Sharing Knowledge with the Marketing Research Profession

In 2003 the Government of Canada, represented by PWGSC, began meeting with the industry as represented by the Marketing Research and Intelligence Association (MRIA). The result has been a growing and productive working relationship that deals with issues of mutual interest. This has included:

• Ensuring that all Government of Canada surveys contracted through POR standing offers and supply arrangements (expired on December 31, 2007) are registered with the MRIA Survey Registration System;
• Consultations on research quality;
• Identification of best practices in research;
• Participation in research studies to give voice to wide-ranging matters of concern to the marketing research profession;
• Development of new courses in public opinion research that are directly relevant to federal government employees; and
• Participation in the government Advisory Panel on Online Public Opinion Survey Quality.

Best Practices in Public Opinion Research: Improving Respondent Cooperation for Telephone Surveys

In 2007-2008, PORD began work with MRIA's Field Management Group to develop a Web seminar version of the document Best Practices in Public Opinion Research: Improving Respondent Cooperation for Telephone Surveys. A practical guide to achieving and maintaining high response rates in telephone surveys, the webinar will be broadcast to researchers in the industry and the government in the 2008-2009 fiscal year.

• Previous Page
• Table of Contents
• Next Page