Public Services and Procurement Canada
Internal Audit: Audit of stakeholder engagement in the PeopleSoft 9.2 upgrade—Stakeholder sign-off process (Phase 2)

Office of the Chief Audit, Evaluation and Risk Executive

May 12, 2021

Do not circulate/Do not copy

Internal Audit: Audit of stakeholder engagement in the PeopleSoft 9.2 upgrade—Stakeholder sign-off process (Phase 2) (PDF, 388KB)

Introduction

Public Services and Procurement Canada (PSPC) has a legal mandate, identified in Sections 12 and 13 of the Department of Public Works and Government Services Canada Act (1996, chapter 16), to provide payroll services to the employees of the public service of Canada.

The PeopleSoft 9.1 pay system, also known as Phoenix, has been in operation since it was implemented in 2016. Version 9.1 of PeopleSoft is an end-of-life commercial off-the-shelf product. Standard support for it has ended. The Government of Canada and Oracle have agreed on extended support for version 9.1 of PeopleSoft until December 2021. After this date, PSPC is faced with 2 key operational risks:

• due to the lack of vendor support, there is a high risk of security vulnerabilities and service degradation impacting PSPC’s ability to produce accurate and timely pay
• failure to apply current tax updates represents a compliance risk and significantly impacts PSPC’s ability to produce accurate pay relating to deductions and tax slips

To address the operational risks, PSPC has developed a plan to upgrade PeopleSoft version 9.1 to the new 9.2 version by July 2021. Responsibility for performing the system upgrade is under the Pay Solutions Branch at PSPC, and is being completed through the PeopleSoft Pay System 9.2 Upgrade Project (referred to as the Project hereafter in this document).

The Project helps the Pay System Sustainability and Pay Stabilization efforts. The scope of the upgrade focuses on a technical upgrade that will result in minimal changes to integration architecture, data entry, and business processes. However, there are other potential benefits to the upgrade, which will be considered post-upgrade as part of PSPC’s continuous Pay Stabilization efforts.

Background

As a result of the risk assessment of PSPC’s Audit Universe, the Office of the Chief Audit, Evaluation and Risk Executive (OCAERE) determined assurance work related to stakeholder engagement within the PeopleSoft Pay System 9.2 Upgrade Project should be undertaken in 2019 to 2020 and 2020 to 2021.

Originally, the OCAERE audit plan included multiple engagements related to stakeholder engagement throughout the Project with regular reporting at key project milestones. The first report was issued in February 2020. During the completion of the first engagement, OCAERE noted significant overlap between its findings and recommendations and those issued by an independent third party, Gartner, engaged by the Project team to perform periodic assessments at key project milestones.

As a result, the OCAERE audit approach was adjusted to conduct assurance work related to the departmental sign-off process, a critical stakeholder engagement activity that is key to the Project’s success . The departmental sign-off process is 1 of several measures in the Project plan that was informed by the Office of Auditor General recommendations, and a series of engagements completed with numerous stakeholders following the initial implementation of the Phoenix system. Developing and implementing the sign-off process is part of PSPC’s commitment to stakeholder engagement. It represents a key element in the Project’s go-or-no-go decision process. The departmental sign-off is a 1-time sign-off based on results from the testing period of the upgrade (often referred to as acceptance). Departments and agencies are intended to sign off and confirm that:

• testing of the human resources (HR) system has been completed
• tested transactions have successfully integrated with the 9.2 system
• technical resources have been or will be allocated to ensure an HR system will connect with the Phoenix 9.2 environments during the dry runs and deployment
• key HR transactions have been successfully tested
• the organization is equipped to support the new platform
• departmental and agency variance is within the approved threshold and tolerance level.

An understood and acknowledged framework will support the process. Departments and agencies will acknowledge the opportunity to test, confirm, and validate the file and/or interface mechanisms is still being executed as per production.

Focus of the audit

This internal audit is in the OCAERE approved Risk-Based Audit Plan 2019 to 2020.

Importance

As outlined in the Project Charter, key stakeholders include:

• departments and agencies that use the Phoenix system
• partners who use Phoenix
• partners whose systems interface with Phoenix
• internal and external committees that are required for decisions, endorsement, oversight, project gating, and approvals

Adequate stakeholder engagement is 1 of a number of measures to support the success of the Project included in the Project plan. After implementing Phoenix, the Project plan was informed by:

• the Goss Gilroy report
• the Office of Auditor General 2018 Report 1 Building and Implementing the Phoenix Pay System recommendations
• a series of engagements completed with numerous stakeholders

Independent assessments and, along with addressing issues identified in these assessments are part of PSPC’s commitment to stakeholder engagement.

Given the large number of stakeholder groups that the upgrade may affect, stakeholder engagement throughout the Project has been identified as a critical success factor in order to build trust, credibility and strong relationships. This will help facilitate buy-in and successful implementation of this upgrade, as well as future pay stabilization projects.

Objective

The objective of this audit was to assess the adequacy and effectiveness of stakeholder engagement throughout the Project’s lifecycle. The engagement will be completed in multiple phases, with results reported in multiple engagement reports to ensure that audit findings and recommendations are provided to the Project team in a timely manner.

This report contains the findings from the second phase of the engagement. It assessed the Project team’s stakeholder engagement activities relating to the sign-off process, including the development of an escalation process to be followed if one or more stakeholders do not sign off or respond within the established timeframe.

The third and final phase of this engagement will look at the Project team’s execution of the sign-off process including adequacy of the engagement of stakeholders (for example, providing adequate opportunity for stakeholders to test and completeness of briefing packages) that will be signing off, and the handling of any issues related to the sign-off process, in accordance with approved escalation procedures. Appendix A: Audit criteria and methodology provides a summary of the scope of each phase and which criteria are assessed.

Scope

The period covered in this audit engagement covers December 1, 2020 to March 8, 2021.

This assessment reviewed activities performed by the Project team up to March 8, 2021. The audit assessed the degree to which stakeholder engagement activities have been performed to meet the needs of the stakeholders. More specifically, audit activities assessed if the Project team:

• effectively consulted with the relevant stakeholders on the process
• obtained and incorporated feedback into the process
• obtained acknowledgement of the process from relevant stakeholders
• ensured that escalation procedures have been developed to address a scenario in which one or more stakeholders do not sign off

The scope excluded an assessment of project activities not related to the engagement of stakeholders.

Audit criteria, sources and methodology

The internal audit was conducted in accordance with the International Standards for the Professional Practice of Internal Auditing, as supported by the results of the quality assurance and improvement program.

The audit criteria, sources of criteria and methodology are in Appendix A: Audit criteria and methodology.

High level observations and findings

Overall, we found that the Project team has sufficiently consulted with a wide-variety of stakeholders on the sign-off process and obtained feedback from stakeholders. The Project team has developed guidance on escalation procedures to be followed if one or more stakeholders do not sign off; however, the audit has provided several recommendations to the Project team related to the opportunity to:

• increase both the rigour of obtaining the sign-offs
• reinforce the importance of the departmental sign-off process within the stakeholder community
• clarify how some of the sign-off activities will be executed

Observations and findings

The recommendations in this report address issues of high significance or mandatory requirements.

The Pay Solutions Branch agrees with the recommendations in this report and has developed related action plans. OCAERE has determined that they appear reasonable to address the recommendations.

Sign-off process

The departmental sign-off process represents a key element in the Project’s go-or-no-go decision process. To support the stakeholder sign-off process, stakeholders must be consulted on the sign-off process and that feedback is considered before finalizing the process. Also, once the sign-off process is formalized it needs to be communicated to all relevant stakeholders.

Stakeholder consultation

The sign-off process was developed by the Project team to provide a framework to execute the process and clarify responsibilities between the Project and stakeholders. Between October and November 2020, the Project team engaged with the Chief Information Officers (CIOs), Chief Financial Officers (CFOs) and Heads of Human Resources (HHR) of all stakeholders to obtain feedback on the proposed sign-off process before presenting it for approval at the Upgrade Acceptance Board (UAB). Documentation given to stakeholders included a video presentation of the sign-off process and the presentation deck on the sign-off process. It was noted that where feedback was provided to the Project team by stakeholders, changes to the draft process were made as required. For example, 1 stakeholder suggested providing a 14-day window for stakeholders to respond during the sign-off process. Based on this feedback, the Project team updated the sign-off process to include this 14-day window.

In addition to completing consultations with the relevant stakeholders, the Project team has participated in several governance committee meetings in the stakeholder communities to present the departmental sign-off process. These committees included the Assistant Deputy Minister (ADM) HR-to-Pay Steering Committee, the ADM HR-to-Pension Steering Committee, the Public Service Management Advisory Committee, and the Union Management Consultation Committee. No concerns were raised at these meetings, and the proposed sign-off process received a positive response from Committee attendees.

Formalization and communication

The departmental sign-off process was presented to the UAB on November 12, 2020. It was approved as presented.

Per the designed process, a briefing package will accompany the request for departmental sign-off. This package will include:

• a summary of the project with the major testing activities that took place
• the issues identified during the testing phases
• how the issues were resolved as the project progressed

Furthermore, the Project team maintains an Assumptions, Constraints, Risks, Issues and Decisions (ACRID) log, where risks and issues identified in key documentation are integrated. The Project team has biweekly meetings to ensure all risks and issues are shown as resolved over the life of the project. This will help to ensure there are no known major issues outstanding when the departmental sign-off process will take place.

Acknowledgement of stakeholder understanding

A final consultation period occurred from January to March 2021 to formally inform the stakeholders of the process. It highlighted the expectations for departments and agencies and to request acknowledgement of the approved departmental sign-off process. Emails were sent out in mid-January and February to the CIOs, CFOs, and HHR for all stakeholder departments and agencies. The purpose of the email was to communicate several updates, which included:

• a briefing package that included an overview of departmental sign-off
  • context within the Project
  • the Departmental thresholds
  • the escalation process
• the departmental sign-off process approved by the ADM-level Project UAB on November 12, 2020
• the departmental sign-offs (April 2021) supported by evidence that will be accumulated during the upgrade project

From examining a sample of acknowledgements received from stakeholders, it was noted that some acknowledgements came on behalf of the executive or by the departmental contact assigned to the project on behalf of the department or agency.

Lastly, at the time of the audit, the Project had received responses from approximately 40% of departments and agencies. The Project team is still receiving these acknowledgements as no specific timeline or target completion rate was stated.

Through review of the sign-off process and consulting the Project team, it was noted that exactly what is a complete departmental sign-off response is not determined. Since 3 individuals (CFOs, CIOs, and HHR) in each organization must sign off, the Project team has not determined if all 3 individuals must sign off, or if only one or two of the three will be considered an acceptable response.

It has not yet been determined if sign-off received from delegates of the CFO, CIO and HHR will be accepted as sign-off.

Escalation process

The departmental sign-off is a 1-time sign-off (also referred to as acceptance) by user departments and agencies, based on evidence accumulated over the testing period of the upgrade. The sign-off consists of obtaining a sign-off by CIOs (where applicable), CFOs and HHR of each stakeholder organization. This sign-off will include the following elements:

• complete testing of the HR system
• tested transactions successfully integrated with the 9.2, with technical resources allocated to ensure an HR system is available to connect with the Phoenix 9.2 environments during the dry runs and deployment
• key HR transactions have been successfully tested
• organization is ready and equipped to support the new platform
• departmental variance that falls within the approved threshold and tolerance level

The Project team has developed guidance on escalation procedures in case any stakeholders do not respond.

In case any stakeholders do not respond, the Project team and the Upgrade Steering Committee (USC) developed a minimum threshold for departmental sign-off. The minimum departmental sign-off threshold includes the following minimum requirements:

• all departments and agencies maintaining HR Systems interfaced to Phoenix must sign off (17 organizations)
• 70% of the hosted departments and agencies must sign off (62 organizations)
• 80% of the direct entry departments and agencies (enter payroll data directly into Phoenix) must sign off (18 organizations)

In October 2020, the Project team obtained approval of the threshold from the USC and began developing escalation procedures to accompany the threshold. The following escalation process was developed and approved by the UAB on November 12, 2020:

• departments and agencies will have 14 days to provide a sign-off
• the Project team will send a reminder within the 7 days
• on day 15, the Project team will send an escalation email, with 3 days to respond
• on day 18, the ADM of Pay Solutions Branch (PSB) or the ADM of Pay Administration Branch (PAB) will follow-up with a phone call to discuss options before proceeding to the UAB for assessment of sign-offs against the approved threshold

The Project team also identified that a lack of response from a stakeholder will be treated as an “assumed” sign-off if the organization has participated in testing, submitted all acceptance forms, and have reported no issues or risks to the Project team.

The escalation procedures identify that when the minimum threshold is not met, the escalation procedures will be followed to achieve a resolution and allow the Project to move to the implementation phase. Reviewing the escalation procedures and consulting with the Project team revealed 3 observations related to the escalation process.

First, some stakeholders might not respond to the sign-off request but meet specific criteria:

• participated in system testing
• submitted testing acceptance forms
• identified no issues or risks

Second, their sign-off may be “assumed.” Although stakeholders are aware of this approach and will be made aware if their sign-off is being ‘assumed’, there is a risk that the go-or-no-go may be based on these assumed signoffs. This increases the risk that the legitimacy of the process by the stakeholder community in question, especially if issues with the system occur. These risks increase with each assumed sign-off.

Third, the process does not contain procedures for escalating beyond the executives responsible for signing off within their respective organizations or central agencies:

• Office of the Chief Human Resources Office (OCHRO)
• CIO for the Government of Canada
• Office of the Comptroller General

Given the short timeframe for stakeholders to sign off, the inclusion of processes for escalating issues (no response from stakeholders, major stakeholder issues or concerns, or other factors) beyond these executives is recommended.

The process focuses on how the Project team must follow up with stakeholders who do not respond; however, no procedures have been identified with respect to the potential that stakeholders provide a no-go response.

Recommendations

Please read the following recommendations and action plans.

Recommendation 1

The Project team should clearly define how the departmental sign-off process will be finalized and evaluated for the purposes of the go-or-no-go decision. Specifically, it is recommended that the Project team:

• determine if each of the identified individuals in a department/agency (CFO, CIO and Head of HR) are required to sign-off, in order for an organization to be deemed to have signed-off
• determine if acceptance of the sign-off process by delegates of the CFO, CIO and Head of HR are acceptable responses from each organization
• ensure that mitigation measures are in place to manage the risk associated with acceptance of responses as assumed sign-offs
• clear communication to governance bodies with respect to the number of sign-offs received and relevant trends, which are sign-off by type of executive (CFO, CIO, Head of HR), number of assumed sign-offs
• communicate to individuals that will be performing the go-or-no-go decision

Furthermore, we recommend that any updates to processes as a result of this recommendation be included in Project documentation and be communicated to stakeholders.

Action plan 1

Departmental sign-off continues to be identified as a project risk and the Project team continues to increase the visibility during various engagement and project governance meetings and committees.

Information sessions were held on April 7 and 8, 2021 with all the department Chief Information Officers (CIOs), Chief Financial Officers (CFOs), and HHR to further communicate the departmental sign-off process which includes timelines, roles and responsibilities and expectations leading up to go live. Outreach and education efforts are actively underway.

Presentations to various engagement, governance and working group meetings and committees are being scheduled. Communications and reminders to CFOs, CIOs and HHR continue to ensure key stakeholders are familiar with the expectations for departmental sign-off.

Additional actions

Communicate that all CFOs and Heads of HR are required to sign-off. Organizations with HR systems that are hosted by another department will not require the CIO signature as this is delegated to the service provider (host). The CIO for direct entry departments will not be required to sign-off as they are not integrated with Phoenix. The remaining CIOs whose HR systems are integrated with Phoenix will be required to sign-off. A detailed list has been developed that identifies who is required to sign off by department (CFO, CIO or Head of HR) in order for the organization to be deemed to have “signed off.”

Communicate that the expectation is for Assistant Deputy Minister (ADM) to sign-off. Delegates’ approvals will be accepted.

The escalation process is clear on acceptance of responses as “assumed.” However, the expectation is to achieve all sign-offs and the 9.2 Upgrade project team has engaged ADMs from Pay Solution Branch (PSB), Pay Administration Branch (PAB), Office of the Chief Human Resources Officer (OCHRO), Office of the Chief Information Officer (OCIO) and if required PSPC Deputy Minister (DM) to call departments with no response. Continue to track and follow up with departments / agencies as required leading up to the sign-off.

Communicate tracked sign-off response rates at the governance bodies and include as evidence in the go-or-no-go process.

The Management Action Plans will be added to the Continuous Improvement Plan and all documents including tracking and contact lists are posted on GCdocs, PSPC’s official repository for corporate information, as part of the project’s official documentation.

The target completion date for this action plan is June 3, 2021.

Recommendation 2

The Project team should continue refining the escalation procedures or other procedures as necessary, to be followed if the minimum threshold for departmental sign-off is not met and if one or more stakeholders decide not to sign-off. Considerations include:

• procedures for escalating beyond the executives responsible for signing-off within their respective organizations and/or to central agencies to follow-up with respective departments (for instance OCHRO, Government of Canada (GOC) CIO, Office of the Comptroller General (OCG))
• procedures to manage the potential that stakeholders may provide a no-go sign-off on the upgrade

Action plan 2

The escalation process is clear and the expectation is to achieve sign-off from all departments/agencies.

Detailed tracking of the sign-off response rates will be monitored and compared to the approved thresholds.

We will continue to track and follow up with departments/agencies as required leading up to the sign-off.

Departments will be categorized based on the criteria identified in the departmental scorecards. For Departments in red, they will be contacted prior to the sign-off process to resolve any issues that prevents the sign-off.

Established PeopleSoft 9.2 Upgrade project-specific governance (Upgrade Steering Committee (USC) and Upgrade Advisory Board (UAB)) and government-wide governance (Government of Canada Enterprise Architecture Board (GCEARB) and Public Service Management Advisory Committee (PSMAC)) will be utilized in addition to the escalation procedures.

Treasury Board Secretariat (TBS) has offered to provide assistance with communications in support of the sign-off process. The project team is working with TBS to have the assistance if/as required.

Deputy Ministers were made aware of the departmental sign-off process and expectations on April 16, 2021 at PSMAC and, if needed, in will be briefed again in June.

The target completion date for this action plan is June 3, 2021.

Recommendation 3

The Project team should continue its efforts to collect the formal acknowledgement of the stakeholders’ understanding of the process. In performing these follow ups, the Project team should define a date by which acknowledgements are to be received by, and an appropriate response rate.

Action plan 3

Acknowledgement and reminder emails have been sent on January 14, February 19, March 30, and April 12, 2021.

Detailed tracking of the departmental acknowledgement response rates are monitored and compared to the approved response thresholds.

The cut-off date for acknowledgement is April 12, 2021 and was communicated on March 30, 2021 to all CIO, CFO and Heads of HHR. This was also communicated at the information sessions on the departmental sign-off process, April 7 and 8, 2021 and in a follow-up email to the information sessions on April 12, 2021. The project team also continues to have one on one discussions with departments/agencies as requested.

The target completion date for this action plan is April 30, 2021.

Conclusion

Through the completion of an assessment, the Project team determined that the sign-off process is relevant to all stakeholders and that all such stakeholders would be requested to submit their acceptance and sign-off as part of the Project’s go-or-no-go decision. The Project team has consulted with stakeholders on the sign-off process. Stakeholders have responded with feedback . The Project team has developed guidance on escalation procedures to be followed if one or more stakeholders do not sign off. However, they can:

• increase both the rigour of obtaining the sign-offs,
• reinforce the importance of the departmental sign-off process within the stakeholder community
• be clear on how some of the sign-off activities will be executed

Acknowledgement

In closing, we would like to acknowledge, recognize, and thank the Project team for the time and information provided during this engagement.

Audit team

Lori-Lee Flanagan

Director

Gilles Trahan

Audit Principal

Jhaal Fergus

Auditor

Carter Wilmsmeyer

Audit intern

Deloitte (LLP)

Consultant

Appendix A: Audit criteria and methodology

This table includes lines of enquiry and information on which report each criteria is addressed in.

Methodology

Assurance engagement to be completed by the OCAERE for the remaining of the Project fall under the following categories:

Departmental sign-off procedures

The audit team continued attending, as an observer, communication and engagement working group meetings and relevant project committee meetings including the USC, and the Integrated Pay Advisory Board. This is to ensure the full range of stakeholder engagement activities have been considered in the conduct of the audit.

The audit team used the following methodology to complete reviews of documentation including meeting minutes, records of decisions, presentation material, communications, responses and other items from the Project team. This was to assess the consultation process for the departmental sign-off and escalation process.

The audit team also interviewed key Project team members between January and February 2021.

Audit activities were performed at the same time as the Project’s activities. Biweekly or more often, the audit team has communicated audit findings to the Project team on a real-time basis, through status reports. This was to facilitate timely implementation of corrective actions. The audit team remained independent from the Project team and was not responsible for management decisions related to the Project’s activities and deliverables.

During the examination phases of each assessment area identified above, interviews were conducted with key PSPC departmental personnel from PSB and external consultants working with the PSPC Project team (such as IBM). In addition, relevant documentation was reviewed in detail. At the conclusion of the examination phase, key members of the Project team were requested to provide validation of the findings.

During the reporting phase for each individual assessment area, the audit team documented the audit findings, conclusions, and recommendations in a draft audit report. The draft report includes the results of the assessments performed up to the end of the period for which the report relates. The draft report was submitted for acceptance to ADM of PSB in a timely manner following acceptance by the Chief Audit, Evaluation and Risk Executive. The draft final report will be tabled at the Departmental Audit Committee meeting in May 2021.