ARCHIVED Annual Report 2008-2009
Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.
Definition of Public Opinion Research
Public opinion research (POR) in the Government of Canada is defined as the planned gathering, by or for a government institution, of opinions, attitudes, perceptions, judgments, feelings, ideas, reactions, or views that are intended to be used for any government purpose, whether that information is collected from persons (including employees of government institutions), businesses, institutions or other entities, through quantitative or qualitative methods, irrespective of size or cost.
The information gathering may be associated with a broad range of activities, for example: policy research; market research; communications research; advertising research; program evaluation; quality of service/customer satisfaction studies; omnibus surveys, with the placement of one or more questions; syndicated studies; or product development.
The following research and/or methods for obtaining opinions and/or advice are not considered to be opinion research: literature reviews or reviews of secondary sources, including reviews of already conducted public opinion research; secondary analysis of previously collected public opinion research data; and verification of performance of services or delivery of goods in contract situations.
The definition of public opinion research includes not only entire projects that have a data collection component, but also parts of POR projects, such as the design and testing of a collection method and tools, sampling, data input, coding, primary analysis of data, and online panel development and management.
Role of Public Opinion Research
Public opinion research helps the government to better understand Canadian society and to identify citizens' needs and expectations. It is used to assess the public's response to proposals or to possible changes or initiatives; to assess the effectiveness of policies, programs and services; to measure progress in service improvement; to evaluate the effectiveness of communication activities such as advertising; and to plan and evaluate marketing initiatives, among other applications.
—Communications Policy of the Government of Canada
Public opinion research is an important tool that offers impartial, systematic and reliable methods to collect the opinions of Canadians as a whole or those that could be most affected by a particular Government of Canada activity. POR also provides the opportunity to obtain real-time feedback. This is particularly helpful in times of emergency or rapidly evolving issues such as pandemics.
The need for POR is embedded in a number of the government's administrative policies as one of the tools to help ensure that government activities are achieving intended results for Canadians. For example, it helps the Government of Canada understand how its programs and services are used, how much they are used, by whom, and whether or not these programs and services are considered useful and effective. POR also assists the Government of Canada in understanding whether its communications in support of policies, programs and services are heard, read and understood by Canadians.
Expected results of the government's management of POR ensure:
- research undertaken for projects is aligned with government priorities, policies, programs, services and initiatives;
- bidding processes for contracted public opinion research activities are open, fair, and transparent, and that POR contracts provide value for money; and
- the public's views and needs are well understood and taken into account when the Government of Canada develops policies, programs, services and initiatives.
This 2008–2009 annual report identifies government-wide research contracted to the private sector.
Strengthening Public Opinion Research Management in the Government of Canada
The government has implemented a number of changes to the manner in which POR is planned, managed and released to the public.
The Federal Accountability Action Plan, which was released on April 11, 2006, was the first in a series of changes aimed at strengthening the management of POR to address concerns about the transparency, fairness and value for money of the procurement process in this area.
The Federal Accountability Act, which received Royal Assent on December 12, 2006, fulfilled a commitment of the Action Plan by implementing legislative requirements to:
- Prohibit verbal-only POR reports;
- Require departments to provide written reports to Library and Archives Canada within six months of completion of the data collection; and
- "…promote fairness, openness and transparency in the bidding process for contracts…".
Furthering its commitment to transparency, a new Library and Archives Canada website was developed to house and allow public online access to reports, executive summaries and POR contract information (Public Opinion Research Reports).
In addition, Public Opinion Research Contract Regulations, which standardize the form and content of contracted POR reports came into effect on June 7, 2007.
On February 13, 2008, the Minister of Public Works and Government Services and the President of Treasury Board announced a series of additional measures and committed to reducing government-wide expenditures on POR by $10 million. New measures include: 1) a requirement to obtain ministerial approval of all departmental POR, 2) an immediate freeze on the signing of new subscriptions of syndicated surveys, and 3) an increase in the evaluation threshold for post advertising campaigns from $400,000 to $1 million. In May 2008, these changes, along with a requirement to provide a quarterly POR plan, to be approved by each department's Minister and to be sent to Public Opinion Research Directorate (PORD) and Privy Council Office (PCO), were integrated into the Treasury Board Procedures for Planning and Contracting Public Opinion Research (See Figure 1: 2008 Contract Value Control Measures, Strengthening Accountability).
The implementation of these measures has contributed to more focused expenditures and best value for taxpayers' money.
Figure 1: 2008 Contract Value Control Measures, Strengthening Accountability
Text description of the 2008 Contract Value Control Measures, Strengthening Accountability is available on a separate page.
Business Volume in Public Opinion Research in the Government of Canada
Total Projects Coordinated by the Public Opinion Research Directorate
In 2008–2009, 131 contracted public opinion research projects worth $8.1 million were coordinated through the Public Opinion Research Directorate.
In total, 23 contractors were involved in POR. The firm with the highest volume, Environics Research Group, conducted 23 of the 131 projects, followed by Ipsos Reid Corporation with 21 projects and EKOS Research Associates with 16 projects. See Appendix II - Listing of Contractors Engaged in Public Opinion Research, 2008-2009 for a list of contractors engaged in public opinion research in 2008–2009.
Application of Public Opinion Research by the Government of Canada
Public opinion research undertaken by the Government of Canada can be divided into three areas of application. They are:
- Informing Canadians
- Advertising research
- Development of communication plans
- Communication product testing
- Website testing
- Taking Into Account Canadians' Needs
- Market research
- Policy development
- Quality of service and client satisfaction measurement
- Policy and program review
- Research Supporting Other Government Activities
- Internal organizational research
In 2008–2009, a total of 131 POR projects were conducted, of which 74 were intended to inform Canadians, and 51 were to help strengthen policies and programs that take into account Canadians' needs. The remaining six projects focused on research that supports other government activities (See Table 1: Categories of Custom Research 2008–2009).
Table 1: Categories of Custom Research 20082009
|Contract Value||% Contract Value||Number of Projects||% Number of Projects|
|Taking Into Account Canadians' Needs||$3,731,957.41||46%||51||39%|
|Research Supporting Other Government Activities||$306,449.35||4%||6||5%|
A total of $8.1 million was contracted on Government of Canada research, of which 50 percent was allocated to informing Canadians; 46 percent assessed Canadians' needs as the government develops policies and programs; and an additional four percent was spent on research that supported other governmental activities.
The government has a duty to explain its policies and decisions and to inform Canadians of its programs and services. POR helps ensure that information about policies, programs and services is clear, credible and targeted to appropriate audiences. This includes research to:
- pre-test government advertising;
- evaluate recall and recognition of advertising;
- measure awareness, attitudes and behavioural intentions as the government develops information products and communications vehicles, as well as the concepts for these tools;
- test communication products (including evaluation of concepts, messages, content and creative design); and
- develop and refine other vehicles that provide Canadians with information, such as guides and brochures.
The Government of Canada regularly pre-tests advertising associated with major campaigns to ensure that funds are invested wisely and that messages are well communicated. Pre-testing is mandatory for campaigns with a media buy of $400,000 or more.
Post-testing of advertising is used to attribute recall, attitude and behaviour changes resulting from major campaigns. Post-testing is mandatory for campaigns with a media buy of $1 million or more. The information from these studies is used to improve the planning and development of future information campaigns.
In 2008–2009, 50 percent of all custom research expenditures was allocated to informing Canadians. For an example of this research, see National Defence's post-test of the recruitment advertising campaign Fight described in the Highlights of Public Opinion Research Projects section of this report.
Taking Into Account Canadians' Needs
The government also uses POR to identify the need for implementing policies and programs and to understand and take account of how Canadians relate to them. This includes research to develop programs and outreach initiatives to help Canadians make informed choices about their health, safety and security. This type of research also helps the government both understand the public's perspectives and experiences, and identify the best approaches to reach out to Canadians.
This category includes:
- Policy development research, in which the government collects information on attitudes and opinions to develop public policies that reflect the needs and wants of Canadians or the specific stakeholder groups affected by those policies;
- Market research, which involves collecting information on product attributes, attitudes and opinions that will help in the design and delivery of programs and services and, in some cases, decisions regarding user fees;
- Policy and program reviews, in which the government uses surveys and focus groups to evaluate the effectiveness and usefulness of programs and services and the factors that are related to the utility of programs and services; and
- Quality of service and client satisfaction studies that help the government understand users' experiences, interest in future services, priorities for services, key areas requiring improvement or change, drivers of client satisfaction, sources of information for clients, and current and future trends.
In 2008–2009, 46 percent of all custom research expenditures was allocated to policy development and market research. An example of this research carried out in 2008–2009 is Natural Resources Canada's study Granting a Social License to Operate: Public Opinion and Mining in Remote/Rural Communities (see Highlights of Public Opinion Research Projects).
Program and service evaluation research enables the government to track quality and service indicators, to report on the progress and success of its programs and services, and to identify possible improvements. Parks Canada's study on Special Events in Canada's National Parks is a good example. It too is described in the Highlights of Public Opinion Research Projects section of this report.
Research Supporting Other Government Activities
In 2008–2009, the government focused on ways to improve and strengthen the quality of its research and human resources in the public service. Types of research in support of other government activities included internal organizational research, such as employee surveys or focus groups, which help the government understand the needs and expectations of employees. It also included the maintenance of a Government of Canada online panel, which could be used by all government departments and agencies.
In 2008–2009, four percent of all custom research expenditures was allocated to research supporting other government activities.
Custom and Syndicated Research
Custom public opinion research is commissioned for specific departments or agencies for the exclusive use of the government. Commissioning organizations hold the intellectual property rights to reports and other materials generated by each project. These products are then made available to other federal organizations and the public through Library and Archives Canada. Parliamentarians and the media have access through the Library of Parliament. Custom research accounts for all of the POR work undertaken by the government in 2008–2009.
Syndicated research studies are developed by suppliers who, in turn, make them available to paying subscribers in the private and public sectors. These off-the-shelf products often contain trend information on various topics. The suppliers retain copyright and sole responsibility for managing the content. Subscribers are normally prohibited from distributing the information to non-subscribing parties. As the research costs are shared among the subscribers, syndicated research is sometimes a cost-effective option for meeting a departmental research need and obtaining information on hard-to-reach populations and highly specific target groups. No syndicated research was purchased by the Government of Canada in the 2008–2009 fiscal year.
Approaches to Custom Research
There are two main ways of conducting research: quantitative and qualitative.
Quantitative research follows a systematic approach to collect and analyze information. The information is gathered in carefully structured ways. This includes surveys conducted by telephone, on the Internet, through face-to-face and exit interviews, mailed questionnaires, and self-completed diaries.
A quantitative approach is used when descriptive or inferential statistics are required. The results from quantitative research can in most cases be generalized to the population being studied. The approach can be used to evaluate the effectiveness of a program, service, or advertising campaign, to measure customer satisfaction, to identify market segments, and to track changes in the public's attitudes, behaviour and use of products and services.
As an example, Foreign Affairs' Corporate Communication Survey, described in the Highlights of Public Opinion Research Projects section of this report, was conducted using quantitative research methods.
Qualitative research is used to gain insight into people's intentions and perceptions. It explores their opinions on particular topics in more depth than is generally possible with a survey. Unlike quantitative methodologies, those used in qualitative research do not yield data that can be generalized to the population being studied. Commonly used types of qualitative research include focus groups, personal interviews and small group discussions.
Qualitative research can be used to generate new product, program or service concepts. It can examine clients' knowledge of, and experiences with, a product, program or service. For a publication or website, qualitative research can test clarity and comprehension of the content or format. It can also be used to pre-test concepts for an advertising campaign.
Health Canada's study entitled Exploring Traditional Food Safety Information for First Nations, described in the Highlights of Public Opinion Research Projects section of this report, is a good example of a qualitative public opinion research project.
Procurement Methods in Public Opinion Research
The Government of Canada uses various procurement methods to contract public opinion research. Standing offers were established competitively for the three main approaches to custom research (qualitative, quantitative and a mix of the two). Appendix III - Standing Offers for Public Opinion Research includes a complete list of contractors on the standing offers.
In 2008–2009, 88 percent of the total value of contracts issued ($7,107,755) was awarded through standing offers. The value of contracts for public tenders (MERX) was $832,359 or 10 percent of the total. Specialized research services not offered through the standing offers or public tenders accounted for $163,771.41, or two percent of the total. These are typically low dollar-value requirements below $25,000 for specialized services not covered on the government's existing standing offer list for POR (See Table 2: Procurement Methods 20082009).
Table 2: Procurement Methods 20082009
|Total Contract Value ($)||% of Total Contract Value||Number of Contracts||% of Total Number of Contracts|
|Public tenders (MERX)||832,359||10%||10||8%|
Public opinion research by the Government of Canada touches all Canadians
POR studies undertaken by the government reach out to many segments of Canadian society. Research populations include the general public, users of specific government services, parents, victims of crime, business people, seniors, potential military and public service recruits, youth, Aboriginal people and persons with disabilities. When their opinions are heard through research such as surveys and focus groups, Canadians are able to influence the government decision-making process.
Target Groups Reached
POR is conducted among a wide variety of audiences. Table 3 presents 34 specific target groups reached by POR studies conducted in 2008–2009.
Table 3: Target Groups Reached
- Aboriginal people
- Benefit recipients
- Business people
- Canadian investors
- Canadian passport holders
- Canadian youth
- Clients of PWGSC
- Disability organizations
- Employees of the Government of Canada
- Ethnic communities
- First Nations people
- General population
- Health professionals
- Internet users
- Opinion leaders
- Parents with children living at home
- People with household incomes under $60,000
- Persons with hearing impairment
- Potential Canadian business owners/entrepreneurs
- Potential home buyers
- Trade school students
- Users of networking sites
- Visible minorities
- Wage earners
- Working Canadians
Most Active Departments and Agencies
As health care continues to be a top priority for Canadians, health-related research has predominated for several years. This trend continued in 2008–2009. Contracts awarded on behalf of Health Canada accounted for 17 percent of both the total value of contracts and the total number of projects. Human Resources and Social Development Canada was the second highest with eight POR projects valued at $808,000. Only one department had research in excess of $1 million, compared with five in 2007–2008 and ten in 2006–2007. Appendix I - Listing of Departments and Agencies Engaged in Public Opinion Research, 2008-2009 includes a list of all departments and agencies engaged in POR in 2008–2009.
Geographic Reach of Surveys and Locations of Focus Group Studies Conducted for the Government of Canada in 2008–2009
The map, Government of Canada Public Opinion Research 2008–2009: Geographic Reach of Surveys and Locations of Focus Groups, shows the areas surveyed by the Government of Canada in 2008–2009 and the number and locations of focus groups conducted during the reporting year. Many telephone surveys included all area codes in Canada. Residents of all provinces and territories with landline telephone numbers were included in the samples. The information on focus groups is from studies undertaken in 2008–2009 for which final POR reports were available at the time of publication of this report.
View the Map
Document "ARCHIVED Public Opinion Research in the Government of Canada 2008-2009 Annual Report" Navigation
- Date modified: