Public Services and Procurement Canada
Internal Audit: Audit of stakeholder engagement in the PeopleSoft 9.2 upgrade—Stakeholder sign-off process (Phase 1)

Final report
Office of the Chief Audit Executive, Evaluation and Risk Executive

Internal Audit: Audit of stakeholder engagement in the PeopleSoft 9.2 upgrade—Stakeholder sign-off process (Phase 1) (PDF, 412KB)

On this page

Introduction

This engagement was included in the Public Services and Procurement Canada (PSPC) 2019 to 2020 Risk-Based Audit Plan.

PSPC has a legal mandate identified in Sections 12 and 13 of the Department of Public Works and Government Services Canada Act (1996, c.16), to provide payroll services to the employees of the public service of Canada.

The PeopleSoft 9.1 Pay System, also known as Phoenix, has been in operation since 2016. Version 9.1 of PeopleSoft is an end-of-life commercial off-the-shelf product. Standard support for the core system functionality has ended. Extended support for version 9.1 of PeopleSoft has been negotiated and agreed to between Government of Canada and Oracle until December 2021. After this date, PSPC will face 2 key operational risks:

To address these operational risks, PSPC plans to upgrade the existing 9.1 PeopleSoft solution to the new 9.2 version by July 2021. Responsibility for performing the system upgrade will be completed through the PeopleSoft Pay System 9.2 Upgrade Project (referred to as the Project hereafter in this document).

The Project will help achieve pay system sustainability and pay stabilization efforts. The scope of the upgrade focuses on a technical upgrade that will result in minimal changes to integration architecture, data entry, and business processes; however, other benefits will be considered after the upgrade as part of PSPC’s continuous pay stabilization efforts.

Given the large number of stakeholder groups that the upgrade may affect, stakeholder engagement throughout the Project has been identified as a critical success factor to build trust, credibility and strong relationships. This will help facilitate buy-in and successful implementation of this upgrade, as well as future pay stabilization projects.

As outlined in the Project Charter, key stakeholders include:

As a result of the risk assessment of PSPC’s Audit Universe, the Office of the Chief Audit, Evaluation and Risk Executive (OCAERE) determined assurance work related to the Project should be undertaken in 2019 to 2020 and 2020 to 2021. Specifically, an audit of the Project’s stakeholder engagement activities should be performed throughout the Project.

Originally, the OCAERE audit plan included multiple engagements with regular reporting at key project milestones. The first report was issued in February 2020. While completing this first engagement, OCAERE noted significant overlap between its findings and recommendations and those issued by an independent third party, Gartner, engaged by the Project team to perform periodic assessments at key project milestones.

As a result, the OCAERE audit approach has been adjusted to conduct assurance work related to the departmental sign-off process. This critical stakeholder engagement is key to the project’s success . The departmental sign-off process is 1 of a number of measures included in the Project plan that was informed by the Office of Auditor General recommendations, and a series of engagements completed with stakeholders after implementing the Phoenix system. The development and implementation of the sign-off process is part of PSPC’s commitment to stakeholder engagement and represents a key element in the Project’s go-or-no-go decision process.

Definition of internal audit

An audit provides a reasonable level of assurance by designing procedures so that the risk of an inappropriate conclusion being drawn based on the audit procedures performed is reduced to a low level. This includes inspection, observation, inquiry, confirmation, recalculation, re-performance and analytical procedures.

Focus of the audit

The objective of this audit was to assess the adequacy and effectiveness of stakeholder engagement throughout the lifecycle of the Project. The engagement will be completed in multiple phases, with results in multiple engagement reports to ensure that audit findings and recommendations are provided to the Project team in a timely manner. This report contains the findings from the first phase of the engagement. It assessed the Project team’s stakeholder engagement activities as they related to the sign-off process.

The second phase of this engagement will look at the Project team’s execution of the sign-off process. This includes adequacy of stakeholder engagement that will be signing off, and the handling of any issues, related to the sign-off process, according to approved escalation procedures.

Examples of this are providing opportunity for departments to test and completeness of briefing packages.

The scope of the audit focused on relevant stakeholder engagement activities that the project team undertakes. This is from inception to completion of the first phase of the engagement. The audit included an assessment of stakeholder engagement activities as they relate to internal and external stakeholders, including departments and agencies that use the Phoenix system, partners who use Phoenix and/or whose systems interface with Phoenix, and internal and external committees that are required for decisions, endorsement, oversight, project gating, and approvals.

The audit assessed how much stakeholder engagement activities have been performed to meet the needs of the stakeholders. More specifically, audit activities assessed whether the Project team effectively:

The audit assessed the activities that the project team completed between December 2019 and October 23, 2020. Reviews of documentation were completed, including:

Key Project team members were interviewed between July 2020 and October 2020. A sample of 4 stakeholders were interviewed in October 2020. Interviewees were members of the key governance committees that provide oversight of the Project.

The scope excluded an assessment of project activities not related to stakeholder engagement.

More information on the audit objective, scope, approach, and criteria can be found in the section About the audit at the end of the report.

Statement of conformance

This audit engagement was conducted in conformance with the International Standards for the Professional Practice of Internal Auditing, as supported by the results of the quality assurance and improvement program.

Observations

Sufficient and appropriate audit procedures gathered evidence to support the accuracy of the findings and conclusions in this report and to provide an audit level of assurance. The findings and conclusions are based on a comparison of the conditions, as they existed at the time, against pre-established audit criteria that management agreed on. The findings and conclusion apply only to the entity examined for the scope and time period the audit covers.

Stakeholder identification and consultation

The following details the expectations and conclusions of consultations with stakeholders.

Expectations

The project team discussed the sign-off process with stakeholders relevant to it. Stakeholders’ feedback was considered before finalizing the process.

Conclusion

Through the completion of an assessment, the Project team found that the sign-off process is relevant to all departmental stakeholders, in other words all departments and agencies. All such stakeholders would be requested to submit their acceptance and sign-off as part of the Project’s go-or-no-go decision. The Project team talked with many stakeholders on the sign-off process and has obtained feedback on improvements that stakeholders want. This feedback has been incorporated into newer versions of the process.

Note

The project team planned more discussions with stakeholders for November 2020. As such, this criteria was not fully addressed by October 23, 2020, the final day of the audit’s first phase. As the audit covers multiple phases to ensure the timely reporting of findings to the Project team and the Departmental Audit Committee, the assessment of the project team’s activities after October 23, 2020 will be in the report for the subsequent phase of the audit, which will end in spring 2021.

The Project team completed an assessment to identify the stakeholders relevant to the sign-off process . They determined that the sign-off process is relevant to all departments and agencies. As such, responsible executives including Chief Financial Officers (CFO) and heads of human resources (HR) from each department and agency that use the Phoenix system, as well as Chief Information Officers (CIO) from departments that manage Human Resources Management Systems (HRMS), will get a briefing package that includes:

  • attestations from PSPC’s senior management as to the operational and system readiness
  • a project summary
  • a departmental summary along with a request for sign-off as part of the Project’s overall go-or-no-go decision

From December 2019 to October 2020, the Project team completed consultations on the sign-off process with a wide-variety of stakeholders to obtain feedback on the proposed sign-off process, including the decision to include all departments and agencies in the sign-off process. Through discussions with several governance committees including the Upgrade Steering Committee (USC) footnote 1, the Integrated Pay Advisory Board (IPAB) footnote 2, and several other committees, the Project team briefed stakeholder representatives on the draft process and requested feedback regarding changes to the process. Where feedback was noted to have been provided to the Project team by members of these committees, it was confirmed that changes to the draft process were made as required. For example, IPAB noted that, to support PSPC’s request for departmental sign-offs, responsible executives would benefit from PSPC’s senior management providing assurance of the Phoenix systems’ “collective readiness”; specifically for areas where stakeholders’ ability to perform testing is limited. Based on this feedback, the Project team updated the sign-off process to include the completion of several “attestations” by PSPC assistant deputy ministers (ADM), which will be included in the briefing packages for stakeholders when requesting their sign-off prior to system go-live.

In addition to talking with the USC, IPAB, and other committees, the Project team also regularly updated the Upgrade Acceptance Board (UAB) footnote 3, which consists of departmental ADMs responsible for the success of the Project on the status of the sign-off process. After consulting with stakeholders in November 2020, the Project team will present the sign-off process to the UAB for final approval before it is communicated to all relevant stakeholders.

Since all departmental stakeholders are not represented on the committees from which the Project team sought feedback on the draft sign-off process, the project team presented the sign-off 3 times directly to the broader CFO, CIO, and HHR communities in September and October 2020 through these communities’ monthly forums. Through discussion with the Project team and interviews with a sample of stakeholders, it was noted that feedback from these meetings was positive, did not require any updates to the draft sign-off process, and showed that executives responsible for completing sign-offs were comfortable with the process.

The audit team interviewed a sample of 4 stakeholders in October 2020 to corroborate the information provided by the Project team regarding the consultation process that was completed with the USC, IPAB, and UAB committees throughout 2020. A sample of stakeholders that participate in these committees was interviewed. The sample included 2 stakeholders that host an HRMS and 2 stakeholders whose HRMS is hosted by another department. As a result, the selected stakeholders had varying levels of involvement with the Project including some stakeholders that are highly involved with system integration testing activities and participate in several Project-related committees and working groups and other stakeholders who have minimum involvement with the Project.

While discussing with representatives from each of the 4 stakeholders, the following observations were noted regarding the consultation process completed by the Project team through the aforementioned committees:

  • the Project team provided committee members with a detailed overview of the proposed sign-off process
  • where stakeholders provided feedback on the process to the Project team, their concerns were addressed and changes were implemented in future iterations of the process
  • where stakeholders had concerns specific to their organization, the Project team discussed those concerns
  • for departments that host a HRMS for other departments, the host departments communicated the details of the sign-off process to the departments that use their HRMS
  • based on the knowledge of the interviewees from host departments, there are no significant concerns regarding the sign-off process within the hosted departments
  • as the sign-off process was not finalized at the time of the interviews, stakeholders noted that they would be looking to ensure that the final process and briefing packages included sufficient reporting on the metrics
  • these metrics are what departments require to gauge the level of risk that a move to implement the project into production represents
  • although the process has not been finalized, stakeholders felt the Project team will ensure that concerns of stakeholders are addressed
  • the stakeholders indicated that they felt that the briefing packages provided to them will be comprehensive and evidence-based

In September 2020, the Project team determined that although the consultations on the sign-off process were performed at a wide-variety of committees and that the executives that attended the CFOs, CIOs, and Heads of Human resources (HHR) government-wide meetings had been consulted, all departmental stakeholders would be further consulted. The additional consultations would ensure that all departmental stakeholders were given an opportunity to review the proposed sign-off process and give feedback before it was finalized. The Project team decided that a communication would be sent out from the Project sponsor to the CFO, CIO, and HHR for each stakeholder organization with:

  • a copy of the proposed sign-off process
  • an example of a departmental briefing package
  • a link to a recorded video walkthrough of the sign-off process, which will be on GCGollab

As audit fieldwork for the first phase was completed on October 23, 2020, confirmation that the Project team incorporated feedback received from stakeholders after this date will be in the report for the next phase of the audit, which is expected to be completed in spring 2021.

Recommendation

Not applicable (no recommendation is required as this time).

Management action plans

Not applicable (no management action plan is required).

Development of escalation procedures

The following details the expectations and conclusions of escalation procedures.

Expectations

The Project team has developed guidance on escalation procedures in case any stakeholders do not sign off.

Conclusion

The Project team had not completed the development of guidance on escalation procedures in case any stakeholders do not sign off. Through discussion with the Project team and review of relevant documentation, it was found that the Project team was developing this guidance.

Note

The Project team will develop the guidance on escalation procedures and will finalize these procedures before March 2021. This criteria was not fully addressed by October 23, 2020, the final day of fieldwork for the first phase of the audit. Since the audit is being performed in multiple phases to ensure the timely reporting of findings to the Project team and the Departmental Audit Committee, the assessment of activities completed by the Project team after October 23, 2020 will be in the report for the next phase of the audit, expected to be completed in spring 2021.

In case any stakeholders do not sign-off in spring 2021, the project team and the USC developed a minimum threshold for departmental sign-off. The minimum departmental sign-off threshold includes the following minimum requirements:

  • all departments maintaining HR systems interfaced to Phoenix (in other words, host departments) will be required to sign off
  • 70% of the hosted departments will be required to sign off
  • 80% of the direct entry departments (enter payroll data directly into Phoenix) will be required to sign off

In October 2020, the Project team obtained approval of the threshold from the USC and began the development of escalation procedures to accompany the threshold. If the minimum threshold is not met, the escalation procedures will be followed to achieve a resolution and allow the Project to move forward to the implementation phase. As audit fieldwork for the first phase was completed as at October 23, 2020, development of escalation procedures and their incorporation into the sign-off process after this date will be in the report for the audit’s next phase, which is expected to be completed in spring 2021.

Recommendation

The Project team should continue to develop escalation procedures to be followed if the minimum threshold for departmental sign-off is not met. Procedures should be documented and communicated to relevant stakeholders well in advance of the execution of the sign-off process.

Escalation procedures should include items such as:

  • defined escalation stages including the procedures to be completed at each stage and responsibility for completing each procedure
  • specific contacts, including names and contact information, to which issues are to be escalated at each stage
  • conditions to be met and timelines for escalation to each stage
  • communication requirements to be followed at each escalation stage (for instance notification of departmental stakeholders that sign-off threshold was not met)

Management action plans

Develop an escalation procedure that includes escalation stage, process, action, timing, and points of contact.

Action implementation target date: approval via project governance (Upgrade Steering Committee), January 2021

Communicate the escalation process as part of the departmental sign off acknowledgement email to all stakeholder groups.

Action implementation target date: January 2021

Implement as part of the departmental sign-off process.

Action implementation target date: April 2021

About the audit

The following section explains the audit.

Authority

This engagement was included in the PSPC 2019 to 2020 Risk-Based Audit Plan.

Objective

The objective of the audit engagement was to assess the adequacy and effectiveness of stakeholder engagement throughout the lifecycle of the Project. More specifically, the engagement assessed stakeholder engagement activities related to the sign-off and approval process. The engagement will be completed in multiple phases, with results reported in multiple engagement reports so that audit findings and recommendations are provided to the Project team in a timely manner.

Scope and approach

The following explains the scope of the project and how it was approached.

Scope

This audit included the stakeholder engagement activities undertaken by the Project team from its inception through to completion of the Project. The audit assessed stakeholder engagement activities as they relate to internal and external stakeholders, including:

  • departments and agencies that use the Phoenix system
  • partners who use Phoenix or whose systems interface with Phoenix
  • internal and external committees that are required for decisions, endorsement, oversight, project gating, and approvals

The audit started in September 2019 and is ongoing. The early phases focused on examining the foundational elements of the Project’s stakeholder engagement activities including:

  • the performance of stakeholder identification processes
  • the development and communication of a stakeholder management plan and strategy to engage stakeholders throughout the Project
  • implementing project mechanisms that affect how stakeholders are engaged, including
    • defining and communicating roles and responsibilities of the Project team members and stakeholders throughout the Project lifecycle
    • developing a risk management framework and supporting processes to support the identification, escalation, and resolution of identified risks/issues impacting stakeholders
    • developing processes for decision-making and acceptance (sign-offs) throughout the Project
    • conducting performance measurement activities and addressing issues where required

This phase is focused on assessing the degree to which stakeholder engagement activities being performed meet the needs of the stakeholders. More specifically, audit activities will assess the Stakeholder Sign-Off / Approval Process: stakeholder involvement in the go-or-no-go sign-off process in support of the go-live decision.

The audit did not assess project activities outside of those directly related to the engagement of stakeholders (for instance adequacy of technical testing performed by the Project team).

Approach

The audit was conducted in accordance with the Institute of Internal Auditors’ International Standards for the Professional Practice of Internal Auditing.

The audit team has analyzed information, documentation and performed interviews and/or workshops with internal and external stakeholders in order to:

  • validate that the process has been communicated and stakeholders understand what they will be agreeing to when signing off on the project
  • validate that stakeholders are engaged as agreed upon to support the sign-off

The sign-off briefing package will be circulated between July and September of 2020.

In addition, the audit team will continue attending, as an observer:

  • Communication and Engagement Working Group meetings
  • relevant project committee meetings including the Upgrade Steering Committee
  • the Integrated Pay Advisory Board

This is to ensure the full range of stakeholder engagement activities have been considered in the conduct of the audit.

Audit activities were performed concurrently with the Project’s activities. The audit team has communicated audit findings to the Project team on a real-time basis, through regular status reports, at least bi-weekly or more regularly), to facilitate timely implementation of corrective actions. The audit team has remained independent from the Project team and was not responsible for management decision-making as it relates to the Project’s activities and deliverables.

During the examination phases of each assessment area identified above, key PSPC departmental personnel from the Pay Solutions Branch and external consultants working with the PSPC Project team (such as IBM) were interviewed. Furthermore, interviews and/or workshops were conducted with a sample of external stakeholders to obtain a variety of perspectives on the adequacy and effectiveness of the project’s engagement activities. In addition, relevant documentation was reviewed in depth. At the end of the examination phase, key members of the Project team were requested to provide validation of the findings.

During the reporting phase for this individual assessment area, the audit team documented the audit findings, conclusions, and recommendations in a draft audit report. The draft report includes the results of the assessments performed up to the end of the period for which the report relates. The draft report will be submitted for acceptance to the Assistant Deputy Minister of the Pay Solutions Branch in a timely manner following acceptance by the Chief Audit Executive. The draft final report will be tabled at the Departmental Audit Committee meetings.

A management response to the findings in this report and a Management Action Plan in response to the audit recommendations. The draft final report, management response, and Management Action Plan will be tabled at a future Departmental Audit Committee meeting for final approval.

Criteria

The criteria for this review were chosen based on the audit team’s understanding of the Project risks. This, in turn, is based on previous audit work performed and through planning discussions with the Project team.

The criteria were:

Stakeholder sign-off and approval process

  • An assessment has been completed to identify all stakeholders relevant to the sign-off process
  • Stakeholders have been consulted on the signoff process and their feedback have been considered prior to finalizing the process
  • The process contains guidance on escalation procedures if one or more stakeholders decides not to sign off

Audit work completed

Fieldwork for this audit was substantially completed between August 5, 2020 and October 23, 2020.

Audit team

Members of the Office of the Chief Audit Executive and Deloitte conducted the audit, overseen by the Director of Continuous Auditing and Advisory Services and under the overall direction of the Chief Audit Executive.

Date modified: