Standards for the Conduct of Government of Canada Public Opinion Research—Online Surveys

About the standards

These standards apply to custom online public opinion research surveys conducted for the Government of Canada by contracted third parties. Custom studies are those in which ownership of the data rests with Canada.

This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from Public Services and Procurement Canada for all other uses.

For more information on this publication, please contact Public Services and Procurement Canada at

Catalogue number: P103-8/1-2020E-PDF
International standard book number (ISBN): 978-0-660-32910-9

© Her Majesty the Queen in Right of Canada, as represented by the Minister of Public Services and Procurement, 2020

To download or print a PDF version of the standards [1.1MB]

Help with alternative formats and plug-ins

On this page

Important notes

The following standards are for custom online public opinion research surveys conducted for the Government of Canada Footnote 1. The standards were developed to help ensure that the level of quality of these surveys is consistently high, is transparent to research users, and aligns with the intended use of the research. If you have any questions related to this document, please contact the Public Opinion Research Directorate at Public Services and Procurement Canada (PSPC) by email at

A. Applicability of these standards

These standards apply to custom online public opinion research surveys conducted for the Government of Canada by contracted third parties, including questions added to omnibus/syndicated surveys. Custom studies are those in which ownership of the data rests with Canada. The standards do not apply to syndicated studies (those in which ownership of the data rests with the researcher) unless the Government of Canada contracts to place custom questions in addition to those already included in those studies. They also do not apply to internal surveys, that is, those undertaken directly by Government of Canada departments and agencies without contracting to a third party. Nonetheless, PSPC invites all individuals to view these standards as a resource in support of the planning and execution of their internal online survey research projects.

To ensure that these standards are a primary consideration for researchers as they develop survey proposals, we recommend the following statement be included in all public-opinion-survey-related Statements of Work and Requests for Proposals issued by the Government of Canada:

“The Standards for the Conduct of Government of Canada Public Opinion Research must be respected and applied in whole to all aspects of the conduct of the research. The project authority must be informed should the firm determine that certain elements of the standards cannot be accommodated in the course of the contract. Justification must be provided and permission received from the project authority in writing in these instances.”

These standards should be understood to be in addition to relevant laws, regulations, and policies, for example, the Personal Information Protection and Electronic Documents Act, the Privacy Act, the Official Languages Act, the Accessible Canada Act, the Financial Administration Act, the Public Opinion Research Contract Regulations, and the Policy on Communications and Federal Identity.

B. Applicability of other standards

Where no relevant Government of Canada standards exist, researchers must meet or exceed industry standards, including those identified by the project authority (client).

C. Exceptions

These standards were developed with “typical” studies in mind. It is recognized, however, that public opinion research surveys have individual characteristics, which can sometimes clash with a “one size fits all” approach. Therefore, while departments and agencies are generally encouraged to apply the standards, they can also choose to waive a standard when they believe the characteristics of a study warrant it, or when a standard is not implementable. For example, some Government of Canada (GC) standards may not be implementable for custom questions added to omnibus/syndicated studies.

This document notes some of the more common instances in which it would be appropriate to waive or modify standards.

As departments and agencies are responsible for the quality of the research they undertake, no related procedure is required to waive or modify a standard. In keeping with due diligence, however, they must discuss the intention to waive or modify standards with the Public Opinion Research Directorate. They should also consider specifying which standard is being waived/modified and the related rationale in Statements of Work and other documents, such as researcher proposals and final reports.

D. Five-year review

The following standards were first developed in 2009. Changes in technologies and other factors necessitate their periodic revision, which is carried out approximately every five years. These standards were most recently updated in 2020.

1. Proposal documentation

In this section

Public opinion research contracts are based on the accepted final proposal from a research firm. As such, researchers must ensure that the proposal provides sufficient detail to clearly document all requirements, including those specified in the project authority’s (client’s) Statement of Work. Proposals must include all relevant information described below.

1.1. Introduction

1.1.1. Research purpose

Broadly describe the topic or issue to be examined and the overall intent of the research, including how the Government of Canada intends to use the results.

1.1.2. Research objectives

Detail the specific research objectives that the research project will address (or the key questions that will be answered) in order to achieve the broader purpose.

1.2. Technical specifications of the research

1.2.1. Overview
  1. Provide a brief statement summarizing:
    1. data collection mode(s), method(s) used to select a sample (probability, non-probability or census), and rationale for proposed methodology.
    2. expected final sample size (that is, number of completed interviews).
    3. target population.
  2. State that the recommended research design is cost-effective, will produce quality results, and is appropriate to meet the objectives of the research, along with a rationale for the recommendation.
  3. Describe how accessibility for persons with disabilities will be taken into consideration during research design and implementation.
  4. Describe how respondents will be able to verify the legitimacy of the survey as a research initiative sponsored by the Government of Canada, for example by providing a contact at the department or research firm, or by using an industry-based research registration system.
  5. Identify the individual researcher(s) who will have supervising authority and accountability for the project, and state that these individual(s) will be replaced only in consultation with the client and following written agreement from PSPC.
  6. Identify all subcontractors that will be used for any aspect of the research (for example, for fieldwork/data collection).
  7. State that all field staff directly involved in data collection will be located in Canada, and that survey data will be stored on servers and back-up servers located solely in Canada (as specified in detail in section 12.3).
  8. State that a signed statement of political neutrality will be submitted as part of the final research report.
  9. Agree that the executive summary and report can be posted on the Library and Archives Canada (LAC) website or on any other Government of Canada website.
  10. State that respondents will be informed of their rights under the Privacy Act, the Personal Information Protection and Electronic Documents Act, and the Access to Information Act, and that respondents’ rights will be protected throughout the research process.
1.2.2. Sample / sampling details
  1. Provide details related to the target population and sample size:
    1. the definition of the target population in terms of its specific characteristics and geographic scope, including the assumed incidence of the population and any key sub-groups, and how the incidence was determined (for example, supplied by the client, previous research, or a credible external source).
    2. the total sample size and the sample composition, including the targeted sample sizes of any key sub-groups.
  2. Describe the sampling method, including:
    1. Whether a sample survey or a census will be conducted. To be described as a census, an attempt must be made to collect data from every member of the target population.
    2. If a sample survey is used, describe whether probability or non-probability sampling will be applied. To be described as a probability sample, the sampling method must meet both of the following conditions: (1) respondents are randomly selected from the survey’s target population, and (2) all units in the target population have a known, non-zero probability of being included in the sample.
    3. A rationale for the chosen sampling method (whether census, probability or non-probability) and how it supports the achievement of the research objectives. In the case of non-probability samples, the rationale must take into account the limitations in generalizing the results to the target population and acknowledge that inferential statistics must not be applied when reporting on the collected data.
    4. If non-probability sampling will be used, it must not be described as a representative sampling of the target population.
  3. Describe the sampling frame for the census or sample survey, including:
    1. The type of sampling frame(s) (for example, client list, online directory) and the extent to which the sampling frame covers the target population, including, if applicable:
      1. Identifying under-coverage by describing any segments of the target population that are not covered by the frame (for example, excluded provinces/territories or rural areas, individuals without Internet access; or excluded non-panel members in the case of panel surveys).
      2. The estimated size of non-covered segments or an explanation why a size estimate cannot be provided.
      3. A description of any over-coverage and/or overlap in sampling frames (for example, when a dual sampling frame is used), in the same manner as for under-coverage.
    2. If applicable, the reason why no sampling frame or list will be used.
  4. Describe the origin of the sample for the census or sample survey, including:
    1. The sample source(s) (for example, Government of Canada department or program, online directory, panel). If it will be provided by a third party, provide the name of the sample supplier.
    2. If websites or social media platforms will be used as sources for a sample survey (for example, to collect contact information of potential respondents), the measures that the researcher will implement to confirm respondent eligibility and authenticity.
  5. When more than one sample source or panel will be used to create the sampling frame, also provide the following:
    1. The rationale for sourcing samples from more than one source or panel.
    2. Steps that the researcher will take to ensure that individuals included by more than one of the sources or panels are not contacted to respond to the survey more than once.
  6. Describe the sample design and selection procedures for sample surveys, including:
    1. Sample stratification variables (if any).
    2. Sample selection procedures when selecting from a list (if relevant).
    3. Any multi-stage sampling steps taken (for example, points of service, followed by service categories, followed by individual clients).
    4. Respondent eligibility/screening criteria and all variables used, and targets set, for strata sample sizes or for quota controls (for example, gender, age group), including the rationale for each target or quota control (for example, underlying benchmarks or parameters from Census data or another high-quality data source).
    5. In the case of panel samples, that respondents who are panel members may not have participated in any Government of Canada survey in the previous 30 days as a member of that panel or in a survey on similar subject matter by any mode, except when a study’s research design requires a follow-up survey with respondents who have agreed during the initial survey to be re-contacted for that purpose.
  7. When the research design or selection procedures will not allow the calculation of a response rate or participation rate (for example, when a sample router is used to screen potential respondents and assign them to one of several surveys), describe the design/selection procedure and any potential sampling bias that may result.
  8. Provide an assessment of how likely the targeted final sample size can be achieved by the census or sample survey, taking into account such factors as the expected response rate, and in the case of panels, the active panel size and exclusion rules involving past survey participation.
  9. When the sample will be drawn from one or more panels, describe the following for each panel:
    1. The panel identity and source.
    2. Active panel size in relation to the survey’s target population, that is, the number of panel members that correspond to the target population and that are available to be surveyed or screened.
    3. How the panel is recruited and maintained, including the recruitment sources used to populate it on an ongoing basis, and the proportion of panelists per source, if possible.
    4. Any ongoing rewards or incentives for panel members that are not associated with taking part in a specific research project/survey.
    5. How a sample of the survey’s target population will be drawn from the panel, including whether existing panelist profile data will be used to identify eligible respondents and whether eligibility criteria will be re-validated during survey administration and, if not, how profile data are kept accurate and up to date.
    6. The incentives/honoraria respondents will be offered for taking part in the survey.
  10. Describe any additional sampling limitations or potential for sampling bias in sample surveys.
1.2.3. Response rate / participation rate and sampling error
  1. State expected response rates/participation rates for each survey mode or research stage that is proposed (that is, separate rates for telephone and online components, when both are used, and separate rates, if applicable, for online devices such as laptops, tablets and cell phones). The expected response rate/participation rate is an estimation based on various factors, such as previous response rates/participation rates or established trends. The proposal should include a brief discussion of the factors that might cause the actual response/participation rate to fall short of the stated target. For surveys where it is not possible to calculate a response rate or participation rate, as described in section 1.2.2. (7), disclose this in the proposal.
  2. For a census or attempted census, do not state a margin of sampling error, as no sample will be drawn.
  3. For probability samples, state the level of precision, including the margin of sampling error and confidence interval for the sample(s). (See section 13.2.5. for detailed requirements related to margin of sampling error.)
  4. Describe specific procedures that will be applied to prevent or minimize the impact of potential sampling errors or non-sampling errors (for example, coverage, non-response), on the research findings.
  5. For non-probability samples, research proposals must not include any of the following:
    1. Margins of sampling error.
    2. Alternative measures of precision.
    3. Statements indicating that the sample has a level of sampling error equivalent to a probability sample of similar size.
1.2.4. Data collection
  1. State the mode of data collection.
  2. For multi-mode surveys, that is, where some respondents will complete the survey via one mode (for example, telephone) and some respondents will complete the survey via another mode (for example, online), provide the following:
    1. The rationale for using a multi-mode rather than single-mode method.
    2. The rationale for the specific modes that will be used.
    3. A description of the steps that will be taken to reduce the likelihood of mode biases, and to detect any mode biases.
  3. For multi-stage surveys, where respondents complete the questionnaire in distinct stages, which may also involve switching modes between stages (for example the first part of the questionnaire is completed by phone and then the second part is completed online), provide a rationale for using a multi-stage rather than single-stage method, and a rationale for all modes that will be used.
  4. Provide details on respondents’ incentives/honoraria where applicable, including a rationale for the following (see section 6.6. for considerations regarding incentives):
    1. The type of incentive/honorarium (for example, monetary, non-monetary).
    2. The nature of the incentive (for example, cash, prizes, reward points, charitable donations).
    3. The estimated dollar value.
  5. Describe how language requirements will be met during data collection.
  6. Indicate the number of contact attempts and explain re-contact procedures.
  7. Describe the planned fieldwork validation methods and procedures.
  8. Describe how the rights of respondents/participants will be respected, including those of children, youth and vulnerable respondents, if applicable (see section 5.).
  9. Describe efforts to make the online questionnaire accessible, as per the Treasury Board Standard on Web Accessibility, for all eligible respondents and to facilitate their participation as feasible, for example, using assistive technologies, accessible formats, or alternative modes of data collection (for example, telephone).
1.2.5. Questionnaire design
  1. Indicate that the online questionnaire design will be mobile-friendly, to allow respondents to complete the survey on a range of devices, including a computer (desktop or laptop), tablet or smartphone. If a mobile-friendly design will not be used, describe the rationale, and potential impacts on response rates and coverage of the target population.
  2. Provide the approximate average duration of the questionnaire in minutes.
  3. Specify the maximum number of open-ends.
  4. Describe how the questionnaire will be pre-tested, including:
    1. The objectives of the pre-test.
    2. The method for the pre-test.
    3. The number of pre-test interviews to be completed, and if relevant, the number of pre-tests for specific subgroups of the target population (for example, when questionnaire effectiveness may vary by language, age, level of education or other characteristic).
    4. How the results of the pre-test will be documented and provided to the client.

If no pre-test is to be conducted, a rationale must be provided.

1.2.6. Data processing / data management
  1. Describe safeguards to ensure respondent confidentiality and protection of personally identifiable information throughout data processing and data management.
  2. Where known in advance, describe any proposed weighting.
1.2.7. Data analysis
  1. Briefly describe how the data will be analyzed in order to address the objectives/research questions, including any special analyses (for example, segmentation, conjoint analysis).
  2. For multi-mode surveys, state whether results will be reported separately by mode and/or by combining the data across modes.
1.2.8. Deliverables

List all deliverables, including their language, format, means of delivery, and number of copies, including at a minimum:

  1. Questionnaire(s) and pre-test documentation, if relevant.
  2. Tabulated data.
  3. Report(s) and executive summary(ies).
  4. Presentation(s), including the specific locations, if relevant.
1.2.9. Project work plan

Provide a detailed work plan with scheduled dates, and identify responsibilities.

1.2.10. Project cost

Present cost information in the format designated by PSPC.

2. Questionnaire design

In this section

2.1. General

  1. Survey questionnaires must be designed:
    1. To collect only the information relevant to the stated research objectives of the study.
    2. To minimize respondent burden while maximizing data value and utility.
    3. To be completed in an average duration of 15 minutes. If supported by a strong rationale, an exception to the duration limit could include projects with specialized audiences where the respondent is aware the survey will take longer than 15 minutes.
  2. The technical design of online questionnaires should facilitate respondent participation and minimize the potential for mode effects by optimizing the presentation and functionality of questions for a range of devices, including computer (desktop or laptop), tablet and smartphone. Accordingly:
    1. Alternative approaches that should be considered when designing online questionnaires include the following:
      1. Mobile-first, which uses a mobile-friendly design for all devices, large and small, by optimizing the questionnaire design for the smallest screen (smartphones).
      2. Responsive design, which optimizes question design and layout for each screen size across the range of devices to achieve the best design for each. This option may lead to different designs and layouts across devices.
      3. Combination, which uses a mobile-first design for some questions and responsive design for others, to balance the dual objectives of facilitating respondent participation and minimizing potential mode effects.
    2. “Mobile-friendly” questionnaire design practices should be implemented when possible, for example by:
      1. Avoiding the need for horizontal scrolling.
      2. Minimizing visual clutter.
      3. Minimizing images and other high-bandwidth requirements.
      4. Using short questions and limiting the number of response options.
      5. Avoiding complex interactive elements such as sliding scales or drag-and-drop.
      6. Allocating sufficient space for touch selection of response options.
      7. Limiting the need for respondents to type or enter text.
      8. Using page breaks wisely.
    3. Researchers are required to make online questionnaires accessible for all eligible respondents and facilitate their participation as feasible, using assistive technologies or alternative modes of data collection.

2.2. Questionnaire introductions

  1. The following are required elements of all Government of Canada online survey questionnaire introductions:
    1. In accordance with the Official Languages Act and policies, introductions must include an active offer of both official languages (English and French). The language that appears first in the text will depend on the province in which the respondent resides (French for Quebec and English for the rest of Canada), or will be the respondent’s official language of preference when this information is available (for example, in the sample list). The respondent must be able to easily select the other official language, for example by clicking on a clearly labelled link. When relevant to a specific survey, non-official languages may also be offered.
    2. Identify the Government of Canada or the department/agency sponsoring the survey. When there is a strong rationale, the sponsor may be identified at the end of the survey instead of in the introduction (for example, for advertising post-tests).
    3. Inform respondents of the general subject and purpose of the study. The subject and purpose of the study may be expressed in general terms, as long as the wording used cannot be construed as an attempt to misinform respondents. When the researcher makes a convincing case that stating the subject and purpose of the survey at the outset is likely to bias the sample that chooses to participate and/or to bias participants’ responses, these statements may be deferred to the conclusion of the survey.
    4. Offer alternative modes of data collection and/or formats that are available to make the survey accessible to persons with disabilities.
    5. Inform respondents of the expected length of the interview.
    6. Identify the researcher (research firm).
    7. Inform respondents that their participation in the study is voluntary and confidential.
    8. Inform respondents that their responses will remain anonymous. In exceptional cases where the research objectives require that respondent identity be revealed, the informed consent of the respondent must be obtained.
    9. Inform respondents how they can verify the legitimacy of the survey as a research initiative sponsored by the Government of Canada.
    10. Provide respondents with a means to communicate their specific accessibility needs to enable survey participation, for example through an alternative mode of data collection or assistive technology.
  2. In addition to other material used to communicate with respondents (for example, invitations), the following introduction could be used for online questionnaires (note that the active offer of both official languages must be included as shown below):

    “The Government of Canada is conducting this research survey on subject. Name of firm has been hired to administer the survey. Si vous préférez répondre au sondage en français, veuillez cliquer sur français [Direct the respondent to the French language version]. The survey takes about ## minutes to complete and your participation is voluntary and confidential. Your answers will remain anonymous and the information you provide will be administered according to the requirements of the Privacy Act, the Access to Information Act, and any other pertinent legislation. Click to view our privacy policy [Add link on the words “our privacy policy”]. Click if you need an alternative means of accessing the survey [Add link on the words “alternative means of accessing the survey”].

    Should you have any questions about the survey please contact the Government of Canada / Department name at Generic email address of departmental client or specific contact.

2.3. Demographic questions

  1. For surveys of the general population, the data from the demographic questions (along with the recording of geographic location) should allow comparison with official Statistics Canada population data for the purpose of non-response analysis and facilitate the comparison of results among Government of Canada public opinion research studies. (See section 13.2.5. for further detail on non-response bias.)
  2. The standard set of demographic questions below must be included in all online surveys of the general population, unless a convincing argument is made that the research objectives are better served by excluding one or more of them. This does not preclude the use of additional demographic questions.
  3. When surveying business or organizational populations, these standard demographic questions should be included whenever relevant, such as when the unit of analysis is the individual rather than the business or organization.
  4. The wording used for each demographic question and its response categories must be as provided below, unless a convincing argument is made that particular research objectives require alternative wording. Note that for certain questions (for example, age, income), the response categories may be further broken down if desired, as long as they can be reconstituted to yield the mandatory categories.
  5. For screening purposes, one or more of the demographic questions may be administered near the start of the questionnaire, as required. Prior to administering demographic questions at the end of the questionnaire, respondents should be informed of the purpose of the questions and be reassured that their answers will remain anonymous and confidential.
  6. Although the ordering of questions and the preamble wording may be modified, the following demographic questions must be used for online questionnaires:

[Preamble] These last few questions will allow us to compare the survey results among different groups of respondents. Your answers will remain anonymous and confidential.


What is your gender?

  • Female
  • Male
  • Other Footnote 2
  • Prefer not to answer

In what year were you born? [Record year – YYYY]

[If the respondent prefers not to provide a precise birth year, ask:]

Would you be willing to indicate in which of the following age categories you belong?

  • 18 to 24
  • 25 to 34
  • 35 to 44
  • 45 to 54
  • 55 to 64
  • 65 or older
  • Prefer not to answer

What is the highest level of formal education that you have completed?

  • Less than a High School diploma or equivalent
  • High School diploma or equivalent
  • Registered Apprenticeship or other trades certificate or diploma
  • College, CEGEP or other non-university certificate or diploma
  • University certificate or diploma below bachelor's level
  • Bachelor's degree
  • Post graduate degree above bachelor's level
  • Prefer not to answer
Language Spoken at Home

What language do you speak most often at home? [Accept all that apply]

  • English
  • French
  • Other [Specify or do not specify depending on the needs of the study]
  • Prefer not to answer
Household Income

Which of the following best describes your total household income last year, before taxes, from all sources for all household members?

  • Under $20,000
  • $20,000 to just under $40,000
  • $40,000 to just under $60,000
  • $60,000 to just under $80,000
  • $80,000 to just under $100,000
  • $100,000 to just under $150,000
  • $150,000 and above
  • Prefer not to answer
Geographic Location

May I have the first three digits of your postal code?

  • the first three digits of your postal code
  • Prefer not to answer
  1. Additional demographic questions may be included when required to satisfy the survey objectives. When an optional demographic question from the list below is included in a survey, the researcher is strongly encouraged to use the exact wording provided for both the question and its response categories, in order to facilitate the cross-comparability of surveys commissioned by the Government of Canada.

Do you identify as a person with a disability? A person with a disability is a person who has a long-term or recurring impairment (such as vision, hearing, mobility, flexibility, dexterity, pain, learning, developmental, memory or mental health-related) which limits their daily activities inside or outside the home (such as at school, work, or in the community in general).

  • Yes
  • No
  • Don’t know
  • Prefer not to answer
Sexual Orientation

What is your sexual orientation?

  • Heterosexual (straight)
  • Homosexual (lesbian or gay)
  • Bisexual
  • Other [Specify or do not specify depending on the needs of the study]
  • Prefer not to answer
Mother Tongue

What is the language you first learned at home as a child and still understand? [Accept all that apply]

  • English
  • French
  • Other [Specify or do not specify depending on the needs of the study]
  • Don’t know
  • Prefer not to answer
Employment Status

Which of the following categories best describes your current employment status? Are you… [Accept one answer only]

  • Working full-time, that is, 35 or more hours per week
  • Working part-time, that is, less than 35 hours per week
  • Self-employed
  • Unemployed, but looking for work
  • A student attending school full-time
  • Retired
  • Not in the workforce [Full-time homemaker, unemployed, not looking for work]
  • Other—[Do not specify]
  • Prefer not to answer

2.4. Multi-mode questionnaires

When designing questionnaires for multi-mode surveys, researchers should:

  1. Ensure as much comparability as possible in the wording, ordering and presentation of questions and response options across the different survey modes, to facilitate combining and comparing the collected data.
  2. Include benchmarking questions for which there are high-quality population data available to facilitate the detection of any mode biases, for example, demographic, behavioural or attitudinal questions that may correlate with key measures in the survey.

3. Pre-testing

3.1. General

  1. Pre-test all components of a new or revised survey questionnaire that may influence data quality or respondent behaviour. This includes the online appearance and functionality of the questionnaire.
    1. For mobile-friendly surveys, researchers must also do internal pre-testing on the types and sizes of devices that respondents might use.
  2. Include probing that invites pre-test participants to provide input about their comprehension of, and reaction to, the questions. If requested by the client, a full cognitive pre-test should be conducted.
  3. If the client requires it, target key socio-demographic characteristics when recruiting pre-test participants, to ensure questionnaire effectiveness for key subgroups (by age, level of education, etc.)
  4. Conduct a minimum of 10 pre-test interviews in each language in which the final survey will be fielded. An exception could be projects with small survey populations, in which case the researcher must seek the client’s approval to conduct a smaller number of pre-tests to ensure questionnaire quality; alternatively a cognitive pre-test may be warranted.
  5. For multi-mode surveys, if one mode is interviewer-administered and another mode is self-administered by the respondent, conduct a minimum of 10 pre-test interviews in each language per mode.
  6. Pre-test completions must not be included in the final dataset. Exceptions could be:
    1. projects with hard-to-reach target groups; or
    2. when no changes are made to the questionnaire following the pre-test.
  7. Results of the pre-test(s) must be shared with the client before the questionnaire is finalized and approved. If shared verbally, written documentation of the pre-test results must follow as soon as possible afterwards. The documentation must include (at a minimum):
    1. A description of the pre-test approach and number of interviews completed in each language.
    2. Findings and any resulting modifications.
    3. Average survey completion time.
    4. A statement of whether or not pre-test cases will be retained in the final data set.

4. Retaining public confidence

In this section

4.1. General

  1. Researchers must make every reasonable effort to gain, retain and increase public confidence in their own organizations and in public opinion research conducted for the Government of Canada.
  2. Researchers must act in a manner that respects survey respondents and ensures their rights are protected as they relate to participation in Government of Canada survey research. Survey participation must take place free of embarrassment and in an environment of trust and goodwill.
  3. The researcher must take all reasonable steps to ensure that respondents are not in any way harmed or embarrassed by survey participation, and that they are not in any way adversely affected as a result of it. Researchers must address sensitive subject matter in a way that will minimize discomfort and apprehension for respondents.

4.2. Respondent rights and protections

4.2.1. Informed consent and use of information
  1. In obtaining the necessary agreement from respondents to participate, the researcher must inform them of the study sponsor and of the general subject and purpose of the survey (subject to exceptions described in section 2.2. paragraph 1), and that their participation is voluntary and confidential.
  2. Survey questions must be limited to gathering information relevant to the stated research objectives. The researcher must ensure that the data collected will not be used for any other purpose unless the respondent provides explicit informed consent for that additional use (for example, to help resolve a customer complaint).
  3. The survey data and personal information must not, under any circumstances, be used for direct marketing or other sales approaches to the respondent or the respondent’s household.
  4. Researchers must provide online respondents with a means to ask questions about the research, and answer any questions in a clear, honest and non-deceptive manner.
4.2.2. Right to refuse

Researchers must respect the right of a respondent to refuse to participate in a survey or to terminate the survey at any point.

4.2.3. Accessibility

As indicated in 2.1. 2) c., researchers are required to make online questionnaires accessible for all eligible respondents and facilitate their participation as feasible, using assistive technologies or alternative modes of data collection. Researchers should inform respondents of the available assistive technologies or alternatives.

4.2.4. Identity of research firm

The research firm conducting the survey must be identified, and sufficient information must be provided to allow survey respondents to contact the researcher without difficulty, if desired.

4.2.5. Protection of anonymity and confidentiality
  1. The anonymity of respondents must be preserved unless they have given their informed and explicit consent to the contrary. The researcher must ensure that the information will be used for research purposes only, or, if requested by the respondent, to resolve a customer complaint. The same holds true when respondents’ answers are, with their informed consent, linked or merged with pre-existing data that allows their identification (for example, with administrative data from a Government of Canada program). Information must not be used for any non-research purpose, such as direct marketing, list-building, credit rating, fund-raising, or any marketing activities directed at those individual respondents.
  2. Completed questionnaires or other material (for example, digital media) containing respondents’ identity, or information that might allow respondents to be identified, must not be released by researchers to clients or other third parties.
  3. When verbatim comments are collected, they must not be attributed to the respondent, directly or indirectly, without the respondent’s explicit informed consent. Care must be taken when reporting verbatim comments, to ensure that nothing in the comment, including content, vocabulary and/or style of writing, could be used to identify the individual respondent.
4.2.6. Use of lists
  1. Where lists are used for sample selection, the source of the list must be disclosed to potential respondents, upon inquiry. When the sample comprises individuals who are not employees of the Government of Canada, researchers must ensure that lists are permission-based for research purposes, including, when appropriate, obtaining written assurances from the list provider (who may be the client) that this is the case.
  2. When the sample comprises employees of the Government of Canada, employee lists may be used even if no explicit permission to use that information for survey purposes has been obtained, as long as the survey topic is relevant to the respondents’ employment (for example, an employee satisfaction survey).
  3. Client-supplied lists provided for specific projects must not be used for any other project or for adding names to the researcher’s databases or panels. If they have been updated by the researcher during the project (for example, to correct errors), those updated lists must be returned to the client upon completion of the project. All client-supplied lists still in the researcher’s possession at project completion must be destroyed after any updates have been forwarded to the client.

4.3. Avoiding nuisance or harassment

Researchers must reduce any inconvenience or irritation their invitations (for example, via email or text message) might cause the recipient, by clearly stating its purpose in the first sentence and keeping the total invitation message as brief as possible.

4.4. Privacy issues specific to online survey research

  1. Researchers must have a readily accessible policy statement concerning the use of cookies, log files and, if applicable, software. This statement may be either included in their privacy policy or it may appear in a separate document.
  2. Software must not be installed on respondents’ computers or other devices without their knowledge or consent, and specific care must be taken to ensure compliance, if relevant, with the Canadian Anti-Spam Legislation subsections 10(4) and 10(5). In addition, respondents must be able to remove the researcher’s software easily from their devices (for example, for Windows users, the software must appear in the Add/Remove Programs folder in their Control Panel).
  3. Any links to data protection, privacy policy or cookie policy statements must be provided at the start of the questionnaire.
  4. A respondent’s email address or mobile number is personal information and must be protected in the same way as other identifiers.

4.5. Photographs and recordings

Photographs, video and audio recordings collected from survey respondents must be collected, processed and stored as personal data. They can be shared with a client only after respondents give informed consent, with knowledge of the specific purpose for which it will be used and once the data has been stripped of all personally identifiable information (such as through pixelization or voice modification). The client must agree to make no attempt to identify individual identities.

If respondents are asked to make recordings as part of their survey input, they should be given specific limited tasks (for example, taking images of objects) that do not have the potential to capture the personal data of others.

5. Data collection from children, young people or vulnerable respondents

In this section

5.1. General

  1. Researchers must take special care when collecting survey data from children and young people, and respondents who are otherwise vulnerable. (A "child" is defined as "under the age of 13" and a "young person" as "aged 13-15".) The welfare of children, young people and vulnerable persons must be the researcher’s primary consideration. These respondents must not be disturbed or harmed by the interview experience. Sensitive topics or issues that could upset children and young people (for example, relationships with other children or with parents, sexual activities, use of drugs or alcohol) or vulnerable persons (for example, social embarrassment related to the source of their vulnerability) must be dealt with using special care to minimize any discomfort or apprehension.
  2. The protection of children, young people, and vulnerable individuals is of particular concern for online research. These respondents may be familiar with using the Internet, but more prone to disclosing information about themselves or their households without realizing the implications of doing so. Researchers must exercise a greater burden of care to ensure children, young people, and vulnerable individuals are not exploited as a result of their participation in research.
  3. All explanations related to data protection, privacy policy, adult consent and other notices must be capable of being understood by children.
  4. In addition to these standards, researchers must observe all relevant laws specifically related to children and young people.

5.2. Consent of responsible adult

  1. Before conducting a survey with a child or young person under 16, researchers must obtain the consent of a responsible adult (parent or guardian). Sufficient information must be provided to this responsible adult to enable an informed decision about giving such consent. In particular, the researcher must specify the nature of any potentially sensitive questions when seeking parental/guardian consent for the child’s/young person’s participation in the research. The name of the responsible adult giving the consent must be recorded, as well as that person’s relationship to the child/young person.
  2. The responsible adult’s consent allows the researcher only to invite the child/young person to participate. The child/young person must be given an opportunity to agree or refuse to take part in the survey. To that end, the researcher must inform the child/young person of the purpose of the research in terms that the person is likely to readily understand.
  3. Surveys made openly available on websites must require respondents to give their age before any other information is requested. If the age given is less than 16 years, the child/young person must be excluded from giving further information until the appropriate adult consent has been obtained. Information about how informed consent can be obtained can be provided to the child/young person at that time if their participation is desired.
  4. It might be difficult to determine with certainty the age of the respondent and whether or not the consent of a responsible adult is required. The researcher must make every reasonable effort to comply with the rules set out in these standards.
  5. When it is not possible to obtain parental/guardian consent (for example, street kids) and the client deems that the benefits of the research will outweigh the possible harm to respondents (for example, youth engaged in risky behaviour), the client may choose to waive the researcher’s responsibility to follow this standard.

5.3. Process for obtaining consent

5.3.1. Parent contact details

Researchers are permitted to ask children/young people to provide contact details for a parent/guardian in order for consent to be sought, as long as this purpose is made clear in the request.

5.3.2. Children / young people of online panelists or other approved list members

In cases where survey data are to be collected from children of adult online panelists or children/young people of other online list members, an e-mailed request for consent to the adult panelist or list member must contain:

  1. A notice stipulating that the online survey is intended for the child/young person within the household.
  2. The name and contact details of the researcher and the name of the client (if the client agrees).
  3. An explanation of the objectives of the research and nature of the data to be collected from the child/young person.
  4. An explanation of how the data will be used.
5.3.3. Children recruited through websites

In cases where children are being recruited from websites, the following measures must be implemented:

  1. On websites aimed at persons under age 16, a note to children/young people that informs them of the requirement for adult consent must be prominently displayed in all notices about the survey (that is, before the respondent can access the survey itself). This note must include a clear explanation of the subject matter and objectives of the research, and the name and contact information of the researcher. The notice must request the contact information (for example, email address) of the parent or guardian, and make clear that the authentic responsibility of this individual and their consent will be verified (through a separate follow-up by telephone, regular mail, etc.).
  2. For websites aimed at adults, a notice to parents or guardians seeking their consent to ask their child to participate in the research must be posted on the website. This notice must include:
    1. A heading explaining that this is a notice for parents.
    2. The name and contact details of the researcher and the name of the client (if the client agrees).
    3. An explanation of the objectives of the research and nature of the data to be collected from the child/young person.
    4. An explanation of how the data will be used.
    5. A description of the procedure for giving and verifying consent.
    6. A request for a parent’s or guardian’s contact e-mail address, address or phone number for verification of consent.
5.3.4. Acceptable forms of consent

A return email from a parent or guardian giving consent is acceptable, as long as additional steps are taken by the researcher to ensure that the consent actually came from a parent or guardian — for example, following up with an email, letter or phone call.

5.4. Collection of personal information

Personal information relating to other people (for example, parents) must not be collected from children. Consistent with the Privacy Act and the Personal Information Protection and Electronic Documents Act, “personal information” means information about an identifiable individual. The researcher may collect the contact information of a parent or guardian for purposes of obtaining parental consent (see section 5.3.1).

6. Data collection

In this section

6.1. Collection of email addresses

  1. Researchers are prohibited from using any subterfuge in obtaining email addresses of potential respondents, such as using technologies or techniques to collect email addresses without individuals’ awareness, and collecting email addresses under the guise of some other activity.
  2. Researchers may collect publicly available email addresses from organizational sources for the purpose of business-to-business research (that is, research directed at private-sector firms, non-profit organizations, agencies of other levels of government, professional associations, etc.), when the research topic can reasonably be deemed relevant to organizational responsibilities.

6.2. Data collection and recruitment techniques

  1. Researchers must not make use of surreptitious, misleading or unsolicited data collection or recruitment techniques – including using spambots, spiders, sniffers or other ‘agents’ that collect personal information without the respondent’s explicit awareness.
  2. Passive data collection entails the collection of information about survey respondents without their active input.
    1. Passive detection and collection of the browser characteristics and settings, including the type of device a respondent is using, is permitted for the purpose of optimizing application performance and survey rendering.
    2. Researchers must not use passive data collection of personal information, (for example, web use and browsing history, application usage statistics, geolocation, biometric data, social media data, and other data generated by respondents’ mobile devices or computers) during online surveys or for recruitment purposes unless:
      1. Respondents have previously given their informed consent, or
      2. The passive collection of the data is legally permissible, for example according to the Terms of Use of the website, service or application from which the data are sourced.

6.3. Misleading email return addresses

Researchers are prohibited from using false or misleading return email addresses, including spoofing the “from” label of email messages, when recruiting respondents over the Internet.

6.4. Opt-out

A respondent must be able to refuse participation in the survey via a suitable option, and to refuse further contact in connection with the survey.

6.5. Access panel participation frequency

When recruiting respondents to a Government of Canada survey, researchers must ensure that panel members have not participated in any Government of Canada survey as a member of that panel or any other panel, or a survey on a similar subject matter by any data collection mode in the previous 30 days. An exception would be when a study’s research design requires a follow-up survey with respondents, and the respondent has agreed at the time of the initial survey to be re-contacted for that purpose.

6.6. Incentives / honoraria

6.6.1. General
  1. When including incentives/honoraria in a project, researchers must ensure that it is legal and ethical to do so. Survey incentives must not be coercive or unduly affect a respondent’s choice to voluntarily participate.
  2. Researchers should consider whether incentives/honoraria are necessary for achieving the research objectives within the project timeline, and whether alternative means could be effective for motivating participation.
  3. When selecting the type and value of incentives/honoraria, researchers should minimize their potential to introduce bias or error in the final sample, for example by affecting the response rates of various segments of the target population differently.
6.6.2. Children, young people or vulnerable individuals

When survey respondents are children, young people or vulnerable individuals, the researcher must:

  1. Take ethical considerations into account before including an incentive or determining its type and value (for example, ensure payment is not coercive, and does not expose children, young people or vulnerable individuals to a risk that they would otherwise have avoided).
  2. Decide in advance who will receive an incentive that is, the responsible adult (parent or guardian), the respondent, or both.
  3. Obtain the agreement of the parent or guardian before offering any incentive to the child, young person, or vulnerable individual.
6.6.3. Incentives for Government of Canada employees
  1. Incentives should not be provided to employees of the Government of Canada for research conducted while on duty (that is, during working hours).
  2. Incentives may be offered to employees of the Government of Canada for research studies conducted while off duty (that is, outside of working hours), as long as:
    1. Employees are made aware that their acceptance cannot be misconstrued as an official duty and therefore subject to overtime.
    2. Employees are made aware that they must consult the Treasury Board Directive on Conflict of Interest or their organizational policy on conflict of interest, where applicable, and consider whether their acceptance of the incentive is likely to give rise to a conflict of interest with their official duties, to constrain their capacity to perform their official duties, or to undermine the neutrality of the public service.
    3. If they have doubts as to whether accepting an incentive could breach the Directive, they should discuss the situation with their manager and, if necessary, report and seek written direction from their deputy head or their delegated authority.

6.7. Monitoring of online survey fieldwork

  1. Each online survey must be closely monitored throughout the fieldwork to ensure that responses are valid, that the survey is administered consistently throughout the data collection period, and that the responses are being recorded accurately.
  2. If requested, clients must be provided with interim data tabulation within the first few days and throughout the fieldwork period. Appropriate care must be taken to ensure that no individual can be identified by their responses.

6.8. Detecting and dealing with satisficing

The panel provider shall implement procedures to identify and remove fraudulent and inattentive panel members or respondents, documenting these procedures and actions taken.

6.9. Reminders to non-respondents

  1. Where feasible, researchers must remind non-respondents of the invitation to participate in the survey.
  2. A maximum of three reminders may be sent.
  3. Reminders must not be sent to individuals who have opted out of the survey.
  4. When feasible, reminders must not be sent to individuals who have completed the survey, even if the survey was submitted only partially completed.

6.10. Ensure respondents answer a survey only once

Researchers must implement and document their procedures to limit the possibility that an eligible respondent can answer a survey more than once.

6.11. Data storage

Data must be stored according to the requirements described in section 12.

7. Outcome rates

In this section

7.1. General

  1. Researchers should apply the definitions below when assigning cases to each case category and when calculating outcome rates.
  2. Researchers must calculate the response rate/participation rate for online surveys in the manner described below, and include it in the final report that the Government of Canada’s client will submit to Library and Archives Canada, as per the Public Opinion Research Contract Regulations.
  3. Monitoring of cases dispositions/reasons for non-response must be carried out on an ongoing basis, throughout the entire field period, and this information must be provided to the client upon request.

7.2. Case category definitions

Online survey cases can be broken down into four broad categories:

Invalid Cases
These can include only clearly invalid cases (for example, invitations mistakenly sent to people who did not qualify for the study, incomplete or missing email addresses in a client-supplied list).
Unresolved (U)
These include all the cases where it cannot be established whether the invitation was sent to an eligible or an ineligible respondent or unit (for example, when email invitations bounce back or remain without an answer before the candidate could be qualified).
In-scope non-responding (IS)
These include all refusals, either implicit or explicit; all non-contacts and early break-offs of known eligible cases; and other eligible non-respondents (due to illness, leave of absence, vacation or other).
Responding units (R)

These include cases who have participated but who were disqualified afterwards (for example, when admissible quotas have been reached). It also includes all completed surveys or partially completed surveys that meet the criteria set by the researcher to be included in the analysis of the data.

Unresolved (U), in-scope (IS), and responding units (R) are all included in the broad category of “potentially eligible” cases. However, invalid cases are not included in the calculation of outcome rates.

7.3. Outcome rates definitions

Researchers must calculate and report the response rate/participation rate as defined below. However, the other rates (cooperation, refusal and contact) are optional; researchers are encouraged to include them in reports when the rates help to understand the causes of low response rates or provide context and insight regarding data quality.

Response Rate/Participation Rate
Total number of responding units divided by the total number of potentially eligible cases (including those whose eligibility could not be determined).
Response Rate/Participation Rate = R/(U + IS + R)
The term “response rate” pertains only to probability sample surveys and censuses, while the term “participation rate” pertains only to non-probability sample surveys.
Cooperation Rate
Total number of responding units divided by the total number of known eligible cases (excluding those whose eligibility could not be determined).
Refusal Rate
Total number of refusals (including break-offs) divided by the total number of potentially eligible cases (including those whose eligibility could not be determined).
Contact Rate
Total number of contacts that were made successfully (regardless of whether or not a contact was made with the targeted respondent, for example, when a family member who does not qualify intercepts the survey invitation) divided by the total number of potentially eligible cases (including those whose eligibility could not be determined).

7.4. Outcome rates for multi-mode surveys

When calculating outcome rates for mixed-mode surveys, researchers should calculate the outcome rate for each mode or stage of sampling, and if feasible and appropriate, calculate a single aggregated outcome rate.

8. Coding of open-ended responses

In this section

8.1. Developing code frames

  1. The initial code list/frame must be developed based on a systematic review of a minimum of 10% of open-ended responses and 50% of partial open-ended responses, where a frame does not already exist. The initial code list/frame must be provided to the client for approval.
  2. The researcher must ensure that coders working on the project are fluent in the language of the collected data that they are coding, and provided with instructions and training that must include, as a minimum:
    1. An overview of the project.
    2. Identification of questions or variables to be coded.
    3. The minimum proportion or number of completed questionnaires used to produce code frames.
    4. Where necessary or appropriate, specific segments of the sample to consider when developing code frames (for example, regions, user or non-user).
    5. Guidelines for the inclusion of codes in the code frame (for example, decisions or rules regarding what must be included or excluded from a given code).
    6. Any use to be made of code frames from a previous project or stage.
    7. Any other requirements or special instructions specific to the project.

8.2. Coding procedures

  1. Where “don’t know” and “no response” options have been used during data collection, these must remain distinguishable from each other during coding.
  2. “Other” or catch-all categories must not exceed 10% of responses to any question.
  3. Researchers must ensure that there is a systematic method of verifying the accuracy of coding for a minimum of 10% of questionnaires per project, and that the verification must be undertaken by a second person.

9. Data editing / imputation

9.1. General

  1. Researchers must keep an accurate record of any changes made to the original data set for a minimum of three years, to ensure that any changes undertaken can be replicated at a later date. All imputation processes used, and their logic, must be documented and provided to the client. All edit specifications must be documented.
  2. Data editing/imputation must be used cautiously. The degree and impact of imputation must be considered when analyzing the data, as the imputation methods used may have a significant impact on distributions of data and the variance of estimates.

10. Data analysis

In this section

10.1. Inferences and comparisons

Researchers must base statements of comparisons and other statistical conclusions derived from survey data on generally accepted statistical practice.

10.2. Analysis records

Researchers must keep accurate and descriptive records of the analysis process for a minimum of three years, to ensure that any analysis undertaken can be replicated at a later date.

10.3. Data analysis verification

Researchers must establish and follow procedures to verify the tabulations and other outputs to ensure the following:

  1. Their completeness, that is, that all tables are present as specified, including the results of all reported significance tests.
  2. That abbreviations for headings or open-ended responses accurately reflect the full content.
  3. That the base for each table is correct against other tables or frequency counts.
  4. That the standard breaks/banner points are checked against source questions.
  5. That all derived data items are checked against their source.
  6. That the figures for subgroups and nets are correct.
  7. That weighting is applied appropriately (for example, by test tables).
  8. That frequency counts are run prior to running tables, in order to ensure the accuracy of data and to determine base sizes for subgroups.
  9. Spelling and legibility.
  10. That any statistical analysis used is appropriate and correct, in both its descriptive and inferential aspects.

11. Delivery of data tables

In this section

11.1. Producing standalone data tables

When data are provided to the client, the following must be included, as appropriate:

  1. The source question to which the data pertain.
  2. A description of any weighting method applied to the data.
  3. Clear identification of any subgroups used.
  4. The sample bases for each question (that is, the number of respondents who have actually answered the question).
  5. The number or proportion of respondents who replied “don’t know” or gave no answer.
  6. Both weighted and unweighted bases.
  7. A definition and explanation of all variables used in the analysis of the data, such as any significance testing, indexing, scoring, scaling and calculations of means, medians, modes and standard deviations.
  8. The types of statistical tests being used and their level of precision.
  9. Information on cell suppression and other measures to assure confidentiality.
  10. Warnings on results that may be unreliable because of very small sample sizes.

11.2. Electronic data delivery

Researchers must provide the client with a data file containing individual records in the original language of each interview, which has been checked for the following:

  1. Completeness (that is, all records are in the file).
  2. Inclusion of all appropriate documentation to allow for replication of the data analysis and additional analyses, including, where applicable:
    1. Labelling of the contents of the file, that is, fully labelled variables and value labels.
    2. Identification and description of any computed or recoded variables, and instructions on limitations of use.
    3. Labelled weighting variables and a description of how these were applied.
    4. All personal identifiers per the Personal Information Protection and Electronic Documents Act have been removed from the files (that is, any information that could identify specific individuals).

11.3. Respondent confidentiality

  1. To ensure respondent confidentiality, tabulated data must not show demographic or respondent characteristic categories with a column or row total of fewer than 10 respondents.
  2. Particular care must be taken when respondent verbatim statements are included in data files.

12. Data security and storage

In this section

12.1. Retention of technical data

Researchers must, for a period of three years, maintain the technical data that are not already included in the final report and provide them to the client, if requested. Technical data to be maintained include:

  1. The information pertaining to data processing and analysis, such as:
    1. Raw data files.
    2. Code frames.
    3. Project files, including project management information and survey programming files.
    4. Relevant correspondence.
  2. Documentation of all changes must be kept when the format, content or layout of the raw data has been changed in any way, for example through editing, cleaning, or recoding the data.

12.2. Retention of personally identifiable information

After their contract ends, researchers should retain personally identifiable information or data (whether it was provided by the client for sampling or data collection, or collected during the research) for no longer than one year, without a strong rationale agreed to by the client.

12.3. Protection of data / servers

  1. Researchers must use up-to-date technologies to protect survey data collected or stored on servers or on portable devices (for example, laptops, USB keys) against illegal or unsanctioned access by third parties (that is, “hacking”). The researcher must also control access to all databases on which any data relating to the survey are stored, so that only individuals with the appropriate security clearance are able to access the database, either by using a password or other form of access control (such as biometric controls).
  2. Because some jurisdictions allow their authorities, under certain circumstances, to access all data stored on servers located in that jurisdiction (for example, in the United States under provisions of the Uniting and Strengthening America by Providing Appropriate Tools Required to Intercept and Obstruct Terrorism Act, known as the USA Patriot Act: see the Treasury Board Secretariat’s overview), the researcher must protect data against legally sanctioned access by ensuring that all databases containing any information related to the survey be stored on servers and back-up servers located solely in Canada.
    1. If the client has first consented in writing, these servers may be located in another country where:
      1. Equivalent protections are given to personal information as in Canada under legislation such as the Privacy Act, R.S. 1985, c. P-21, and the Personal Information Protection and Electronic Documents Act, S.C. 2000, c. 5, and under any applicable policies of the Government of Canada.
      2. The laws do not allow the government of that country or any other entity or person to seek or obtain the right to view or copy any information relating to the survey without first obtaining the client’s written consent.
    2. In connection with giving consent to locating a server in another country, the client may, at the client’s option, require the researcher to provide a legal opinion (from a lawyer qualified in the foreign country) that the laws in that country meet the requirements of subsection 2) a) above, or may require the researcher to pay for the Government of Canada to obtain such a legal opinion. The Government of Canada has the right to reject any request to store survey data in a country other than Canada if there is any reason to be concerned about the security, privacy, or integrity of the data. The Government of Canada may also require that any data sent or processed outside of Canada be encrypted with Government of Canada-approved cryptography, and that the private key required to decrypt the data be kept in Canada, in accordance with key management and storage processes approved by the Government of Canada.
    3. The researcher must ensure that all servers, including back-up servers, on which any data relating to the survey are stored, be physically and logically independent (meaning there is no direct or indirect connection of any kind) from all other databases, unless those databases are located in Canada (or in another country approved by the client under subsection 2) a)) and otherwise meet the requirements of this section.
    4. The researcher must ensure that all data relating to the survey be processed only in Canada or in another country approved by the client under subsection 2) a).
    5. The researcher must ensure that all domestic network traffic (meaning traffic or transmissions initiated in one part of Canada to a destination or individual located in another part of Canada) be routed exclusively through Canada, unless the client has first consented in writing to an alternative route. The client will consider only requests to route domestic traffic through another country that meets the requirements of subsection 2) a).
  3. The researcher must not subcontract (including to an affiliate) any function that involves providing a subcontractor with access to any data relating to the survey unless the client first consents in writing.
  4. Protection of data from international surveys: When the target population for a survey comprises individuals residing in a jurisdiction outside Canada, researchers must ensure that data be stored in a manner consistent with the relevant laws of that jurisdiction.
  5. Protection against physical damage to servers: Researchers must also put in place measures to ensure the physical security of data and servers.

12.4. Temporary storage of data on servers

If the temporary storage of data collected takes place on a server that is operated by another provider, the researcher must place the provider under the obligation to take the necessary steps to ensure that the requirements described in section 12.3 be met. Temporary storage of the collected data on the server must be terminated at the earliest possible time.

12.5. Transmission of data internationally

Before data are sent over the Internet to another country, researchers must check with competent authorities that the data transfer is permissible. The recipient may need to provide safeguards necessary for the protection of the data and ensure compliance with the requirements described in section 12.3.

12.6. Disclosure of Respondents’ Emails in Batch Transfers

Researchers must have adequate safeguards in place to ensure that when emails are sent in batches, the addresses of the respondents are not revealed.

12.7. In the event of any data breach

In the event of any data breach, the client must be informed immediately, and provided with details about both the nature and the extent of the data breach.

13. Survey report requirements

In this section

This section describes the survey report requirements that are mandatory under the Public Opinion Research Contract Regulations for all public opinion research conducted by an external research firm for the Government of Canada, as well as other legal, policy-based and administrative reporting requirements. Contracted research firms are responsible for complying with the Public Opinion Research Contract Regulations. Federal institutions must ensure that when contracting for public opinion research, the responsibilities for meeting all other mandatory reporting requirements are clearly assigned among the contractor, the federal institution (client), or other parties.

13.1. Regulatory and administrative mandatory requirements

It is a legal requirement for federal institutions to send written reports to Library and Archives Canada (LAC) within six months of the completion of the data collection for all contracted public opinion research.

Pursuant to the collective requirements of the Public Opinion Research Contract Regulations, the Directive on the Management of Communications (Appendix C: Mandatory Procedures for Public Opinion Research), LAC deposit instructions, and PSPC instructions, final reports must be submitted separately in English and French and include the following, at a minimum:

  1. On the covering page:
    1. The title of the project.
    2. The name of the research firm that entered into the contract.
    3. The contract number, the contract value, and the award date.
    4. The POR Registration Number.
    5. The delivery date (the date the final version of the report was received by the project authority).
    6. The name of the client department or agency sponsoring the research.
    7. The departmental contact information in the form of a generic email address.
    8. The departmental signature and the Canada wordmark.
    9. For the English report, the statement « Ce rapport est aussi disponible en français. »
    10. For the French report, the statement “This report is also available in English.”
  2. Copyright information, including the Government of Canada catalogue number, the International Standard Book Number (ISBN), copyright notice and year of publication, and departmental notice on the rights to reproduce the report.
  3. A narrative executive summary, submitted separately in English and French and consisting of, at a minimum:
    1. A statement of the research purpose and objectives.
    2. A summary of key findings, except where the person who entered into the contract is not responsible for the design, development of the methodology and analysis of the research.
    3. A brief description of the methodology used.
    4. A statement as to the extent to which the findings can be extrapolated to a broader audience.
    5. An outline of how the results were used, if possible, and if not, how the information is expected to be used.
    6. A political neutrality certification signed by the research firm.
    7. The contract value of the POR study.
  4. Appendices containing:
    1. A full set of tabulated data submitted separately in English and French, which, to ensure respondent confidentiality, must in no case show demographic or respondent characteristic categories with a column or row total of fewer than 10 respondents.
    2. Sample size, sampling procedures and dates of research fieldwork.
    3. A discussion of the potential for bias in the research findings due to sampling errors or non-sampling errors (for example, coverage, non-response).
    4. If applicable, weighting procedures, the confidence interval, and the margin of error.
    5. If applicable, the response rate/participation rate and method of calculation.
    6. The research instruments in all languages and, if applicable, the test material in all languages in which they were used and tested.
    7. All other information about the execution of the fieldwork that would be needed to replicate the research initiative.

13.2. Additional reporting requirements

In addition to the reporting requirements described in section 13.1, the following minimum details must be documented in the project report. These allow the reader to understand how the research project was conducted and the implications of its results.

13.2.1. Background
  1. Detailed description of the project background including, at a minimum:
    1. Research purpose.
    2. How the research will be used.
  2. Specific research objectives, research questions.
13.2.2. Sample
  1. Whether a sample survey or a census was conducted.
  2. Detailed description of the following:
    1. The target population for the research project and its geographic location (where applicable), as well as any key sub-groups.
    2. The sample source(s), and if the sampling frame and/or the sample itself was provided by a third party, the name of the sample supplier.
    3. The sampling frame(s) (for example, lists, email addresses) and the extent to which the sampling frame covers the target population, identifying under-coverage by describing any segments of the target population that are not covered by the frame (for example, excluded provinces/territories or rural areas, households without internet, or excluded non-panel members in the case of panel surveys), and the estimated size of non-covered segments or an explanation why a size estimate cannot be provided.
    4. If applicable, over-coverage and/or overlapping sampling frames (for example, when a dual sampling frame is used) must be described in the same manner as under-coverage. If no sampling frame or list was utilized, this must be indicated.
    5. The sampling method, including probability or non-probability, and the sampling design (for example, stratification, clustering) or procedure for selecting respondents.
    6. The rationale for the chosen sampling method, including how it supported the achievement of the research objectives. In the case of non-probability samples, the rationale must take into account the limitations in generalizing the results to the target population and acknowledge that inferential statistics must not be applied when reporting on the collected data.
    7. Screening procedures, including eligibility/screening criteria and any variables used, and targets set for strata sample sizes or for quota controls (for example, gender), including the rationale for each target or quota control (for example, underlying benchmarks or parameters from Census data or another high-quality data source).
    8. If the sample was drawn from a pre-recruited panel or pool, the methods used to recruit the panelists or participants and the procedures for managing the membership, participation, maintenance and refreshment of the panel or pool.
    9. The achieved sample size compared with the projected sample size, both for the overall sample and for any key sub-groups, as well as reasons for not obtaining the projected sample size, strata size targets, or quota controls, if applicable.
    10. The weighting procedures, if applicable, including any underlying parameters or benchmarks used to establish the weights, and their source.
13.2.3. Data collection

Detailed description of the methodology, including:

  1. The data collection method(s), mode(s), and their rationale.
  2. Accessibility provisions for respondents using adaptive technologies, if applicable.
  3. The dates of fieldwork (for each phase/wave, if applicable).
  4. The average interview duration and the range.
  5. The type and amount of incentives, if applicable.
  6. Any relevant stimuli (for example, radio advertisement).
  7. Details of any strategies used to help gain cooperation (for example, advance contact, incentives/honoraria), whether for participation in a panel or for participation in a particular survey.
13.2.4. Quality controls
  1. A description of questionnaire pre-testing and any resulting modifications to improve data quality or respondent experience, if applicable.
  2. The estimating and imputation procedures, and any editing of the data, if applicable.
  3. A brief summary of other quality controls and procedures used. Where applicable, this includes re-contacts to confirm that the interview occurred and/or to verify the respondent’s identity, measures taken to prevent respondents from completing the same survey more than once, and other quality control procedures (for example, logic checks and tests for speeding and patterning). If no such efforts were undertaken, this will be disclosed.
  4. For multi-mode surveys, detailed description of any data quality issues arising from combining data collected via different modes/instruments. The description must also include:
    1. Rationale for decisions about combining or not combining data from different modes in reporting the results.
    2. Description of any steps taken to detect and mitigate mode effects and mode biases in the survey results.
13.2.5. Results

The detailed results in reports must include:

  1. An executive summary of key results and conclusions, linked to the survey objectives and research questions.
  2. Specifications for any indices constructed or statistical modeling in sufficient detail to allow replication.
  3. For probability samples, state the level of precision, including the margin of sampling error and confidence interval for an observed percentage of 50% for yes-no (two option) type questions asked of the total sample and any key subgroups.

    Clearly indicate that the results for subgroups have a larger margin of sampling error than for the overall sample because of their smaller sample sizes. Also, the margin of sampling error is highest for questions where 50% of the respondents gave one answer and the other 50% gave another answer. The margin of sampling error decreases for questions where the observed percentage for a particular response approaches 0% or 100%.

    Results for subgroups must not be reported as significant differences in instances where differences between the groups are within the margin of sampling error for the survey as a whole.

  4. When subsets of the sample (for example, regions, age groups, language groups) have been over- or under-sampled, adjust for the design effect Footnote 3 due to weighting when calculating the margin of sampling error.
  5. For non-probability samples, the report and executive summary must contain a prominently displayed statement in the methodology description regarding why no margin of sampling error is reported, based on the following template: “The results of this survey are not statistically projectable to the target population because the sampling method used does not ensure that the sample represents the target population with a known margin of sampling error. Reported percentages are not generalizable to any group other than the sample studied, and therefore no formal statistical inferences can be drawn between the sample results and the broader target population it may be intended to reflect. [If weighting was done, include a sentence to describe the weighting, for example: “The sample data have been weighted to reflect the demographic composition of (target population).”]
  6. For non-probability surveys, no alternative measures of precision may be cited, and no statements may be made to indicate that the sample has a level of sampling error equivalent to a probability sample of similar size.
  7. For census surveys, the report must contain a statement on why no margin of sampling error is reported, based on the following template: “Because the entire population of [target population] was invited to participate in this study, there is no margin of sampling error to be estimated or reported. The potential impact of non-sampling error due to non-response is discussed in the results section of the report. [If weighting was done, include the following statement: “The data have been weighted to reflect the composition of [the target population (if known) or the sampling frame (for example, client-supplied list)] in relation to its key known characteristics."]
  8. For multi-mode surveys, if the data collected via different modes are combined for analysis and reporting purposes, the report must discuss whether this leads to any data quality issues. This could include, for example, discussion of possible impacts of mode on key survey variables, the impact of any differences in response rate/participation rate by mode, and non-response bias analyses by mode.
  9. The report must provide the final dispositions of cases in sufficient detail to allow secondary analysis of all reported outcome rates. The record of final case dispositions must identify the number of invalid numbers, unresolved cases (U), in-scope non-responding units (IS), and responding units (R).
  10. For all types of surveys (probability sample, non-probability sample, census) the report must state the response rate/participation rate achieved, which must be calculated according to the formula:

    Response Rate/Participation Rate = R/(U + IS + R)

  11. All survey reports must discuss the potential for non-response bias for the survey as a whole and for key survey variables. The report must describe, to the extent possible, the degree to which characteristics of the respondents match or depart from known characteristics (for example, demographics) of the target population or sampling frame, and discuss the possible influence of any resulting non-response bias on the interpretation of survey results.
    1. When authoritative sources of comparison exist, as is the case when conducting a survey of the general population, the analysis of non-response bias will consist of a comparison of at least three variables in the survey sample with the equivalent parameters of the population, normally available from Statistics Canada. No extra surveys or interviews are required.
    2. If no authoritative sources of comparison exist, key variables in the sample could alternatively be compared to variables in the sampling frame. If no such variables exist, only then should the non-response bias discussion be based on comparison of early versus late responders or on observations made during data collection about characteristics of non-responders.
  12. The results that are based on subgroups and the number of cases used in subgroup analysis (that is, unweighted sample sizes for reported subgroup estimates.)
13.2.6. Detailed research instruments and test materials

When appending the research instruments and test materials to the final report, as required by section 13.1 d. vi, the following details must be included:

  1. Study materials, including, when relevant, the recruiting instruments, questionnaires, descriptions, transcriptions or representations of any visual or auditory aids, and other data collection documents, in all languages in which the research was conducted.
  2. A version of the questionnaires displaying the following:
    1. The exact wording and presentation of questions and response options whose results are reported. This includes preceding respondent instructions and any preceding questions that might reasonably be expected to influence responses to the reported results.
    2. Any other instructions (skip, terminate, etc.) needed to understand the logic and flow of the questionnaire.
Date modified: