Public Services and Procurement Canada
Audit of the Real Property 1 and 2 contract oversight control framework (phase 1—Expenditures)

Audit of the Real Property 1 and 2 contract oversight control framework (phase 1—Expenditures) (PDF, 966Ko)

Executive summary

i. As the Government of Canada's real property (RP) and real estate expert, Public Services and Procurement Canada (PSPC) manages a large and diverse real estate portfolio across the country. PSPC must therefore ensure the provision of building maintenance services such as the operation, maintenance and management services for all mechanical, electrical systems and architectural components of the facilities and sites.

ii. Starting in 1998, PSPC has gradually transitioned from delivering the day to day property management operations and project delivery services of crown assets to using Real Property Management Contract mechanisms which leverage private sector contractors for the delivery of these services.

iii. Since PSPC first began using such mechanisms, audits of these major service delivery contracts identified a number of concerns and recommendations. In response to the recommendations of the various audits and reviews, PSPC has implemented improvements to strengthen the management and controls of the Real Property Services Management contracts.

iv. These improvements are encompassed within the third generation of contracts which are currently made up of 2 mechanisms: 1) Property Management and Project Delivery Services and 2) National Capital Area Property Management and Project Delivery Services known as RP-1 and RP-2 respectively.

v. In May 2013, PSPC awarded the RP-2 contract for building management services to Brookfield Johnson Controls. This contract includes providing real property services such as property management, project delivery and optional services to the Carling Campus, Tunney's Pasture and the Graham Spry Building, which are all located in the National Capital Area. The contract operational start date was August 15, 2013 at the Carling campus.

vi. In November 2014, PSPC announced the award of the RP-1 contracts to Brookfield Johnson Controls for the delivery of diverse real property services. RP-1 is composed of 6 contracts, each for a corresponding PSPC region. The contract operational start date was April 1, 2015.

v.ii According to documentation reviewed as part of the planning phase of the audit, total expenditures under the RP-1 contracts, from their start dates up to March 31, 2018, were approximately $1.789 billion, while total expenditures under the RP-2 contract from its start date up to the March 31, 2017 were approximately $429 million. We were unable to obtain the RP-2 contract expenditure information up until March 31 2018 as the audited cost statement for RP-2 for the 2017 to 2018 fiscal year will only be available at the end of the third quarter of fiscal year 2018 to 2019.

v.iii An oversight framework called the RP-n Oversight Framework was introduced in April 22, 2016. It was designed to ensure the efficient and consistent management of the RP-1 and RP-2 contracts, as well as any future contracts of a similar scale. The framework was designed based on the Community of Sponsoring Organizations of the Treadway Commission framework, which is a commonly used internal control framework.

ix. This audit was conducted to assess whether processes and operational control activities over expenditures under the RP-1 and RP-2 contracts were designed and implemented as intended. We reviewed data for the fiscal year 2016 to 2017 as well as relevant documentation up to August 2018.

x. Overall, we found that the processes and operational control activities identified for review were generally developed and being followed. Our audit results indicated that the RP-1 and RP-2 contracts are appropriately administered. However, some opportunities for further improvement exist.

xi. We found processes and procedures were documented.  Further many key controls were found to be functioning, including controls related to work authorization; invoice certification; payments related to contractor performance management; quality monitoring; and data integrity.

xii. We noted that a few processes identified in the RP-n framework were not fully developed. We noted 2 key controls over the RP-1 and RP-2 expenditures that were designed, but not fully implemented. These included the National Project Management System, which was not consistently applied by PSPC and the quality monitoring process, which did not report results and action plans to senior management or oversight bodies. PSPC contractors are not required to follow the National Project Management System when providing services to PSPC.

Management response

Management has had the opportunity to review the report, and agrees with the conclusions and recommendations found therein. Management also developed a Management action plan to address these recommendations.

Recommendations and management action plan

Recommendation 1

The Assistant Deputy Minister, Real Property Services should ensure that processes and procedures related to expenditure management for the RP-1 and RP-2 contracts contained in the RP-n Oversight Framework Reference Library are developed, finalized and implemented.

Management action plan 1.1

Initiate and finalize reviews of all oversight processes/procedures specific to expenditure management.

Management action plan 1.2

Develop new and/or update existing expenditure management processes/procedures as required.

Management action plan 1.3

Circulate draft expenditure management processes/procedures for national review and consultation.

Management action plan 1.4

Present expenditure management processes/procedures requiring approval to the Real Property Services Policy Steering Committee (PSC).

Management action plan 1.5

Schedule and deliver national training to complete the implementation of expenditure management processes/procedures.

Management action plan 1.6

Provide direction to the National Capital Area (NCA) to ensure project documentation processes, including milestone billing schedules, are actioned and documented appropriately. Training to be provided if necessary.

Recommendation 2

The Assistant Deputy Minister, Real Property Services should ensure that the quality monitoring plan is approved and results and action plans are reported to branch oversight committees or senior management.

Management action plan 2.1

Future quality monitoring plans will be submitted to a senior management level branch oversight committee for review and the Director General of the Strategic Sourcing Sector for approval. The 2019 to 2020 quality monitoring plan will be presented and approved by March 2019.

Management action plan 2.2

A quality monitoring update process will be implemented to provide on-going regular feedback to senior management about the progress of quality monitoring activities.

Management action plan 2.3

Future quality monitoring results and action plans will be submitted to a senior management level branch oversight committee for information and feedback.  The 2018 to 2019 quality monitoring results and action plan will be presented by July 2019.

Introduction

1. This engagement was included in the Public Services and Procurement Canada (PSPC) 2017 to 2020 Risk-Based Audit and Evaluation Plan.

2. As the Government of Canada's real estate expert, Public Services and Procurement Canada (PSPC) manages a large and diverse real estate portfolio across the country. PSPC must therefore ensure the provision of building maintenance services such as the operation, maintenance and management services for all mechanical, electrical systems and architectural components of the facilities and sites.

3. In the mid-1990s, in response to multiple challenges such as fiscal constraints, downsizing objectives resulting from program reviews and other reform initiatives, PSPC chose to focus on its strategic advisory and service management roles, while identifying a more efficient method of providing day-to-day operational services. Since 1998, the first generation of procurement mechanisms (previously known as Alternate Forms of Delivery contracts) set PSPC on a course that began moving the department away from delivering day-to-day property management and project delivery services in crown assets, to focus more on the strategic aspects of its business. Numerous audits on the second generation of Alternate Forms of Delivery contracts played a large role in shaping the third generation of contracting mechanisms.

4. Audits of PSPC's previous major service delivery contracts, Alternate Forms of Delivery, identified a number of concerns, including: inappropriate charges to PSPC for a number of areas and services; internal audit recommendations not being fully implemented; and the need for improving the oversight regime.

5. In response to the recommendations of the various audits and reviews completed over the past few years, PSPC has implemented improvements to strengthen the management and controls of the Real Property Services Management contracts. These include the enhancement of the oversight framework; the development and application of tools to ensure that contractual obligations are met; the implementation of a service delivery regime; and the use of monitoring and training activities to ensure consistency across regions.

6. These improvements are encompassed within the third generation of contracts which are currently made up of 2 mechanisms: 1) Property Management and Project Delivery Services and 2) National Capital Area Property Management and Project Delivery Services known as RP-1 and RP-2 respectively.

7. In May 2013, PSPC awarded the Real Property-2 (RP-2) contract for building management services to Brookfield Johnson Controls. This contract includes providing real property services such as property management, project delivery and optional services to the Carling Campus, Tunney's Pasture and the Graham Spry Building, which are all located in the National Capital Area and are all crown-owned. RP-2 was implemented first due to the timing of the expiry of a previous contract. The contract operational start date was August 15, 2013 at the Carling campus.

8. In November 2014, PSPC announced the award of the RP-1 contracts to Brookfield Johnson Controls for the delivery of diverse real property services, including the management of approximately 3,800 crown-owned and leased buildings and assets across Canada. RP-1 is composed of 6 contracts, each for a corresponding PSPC region, and one solely for the National Capital Area due to its quantity of buildings. The contract operational start date was April 1, 2015.

9. According to documentation reviewed as part of the planning phase of the audit, total expenditures under the RP-1 contracts, from its effective date to March 31, 2018, were approximately $1.789 billion, while total expenditures under the RP-2 contract from its effective date to March 31, 2017 were approximately $429 million. We were unable to obtain the RP-2 contract expenditure information up until March 31 2018 as the audited cost statement for RP-2 for the 2017 to 2018 fiscal year will only be available at the end of the third quarter of fiscal year 2018 to 2019.

10. The RP-1 contracts include both leased buildings and crown-owned buildings, where either PSPC or another government department is the custodian. The majority of expenditures for RP-1, representing approximately 98% for fiscal year 2016 to 2017, are for crown owned buildings. The National Capital Area is the largest region with respect to expenditures, accounting for approximately 51% of total RP-1 expenditures for fiscal year 2016 to 2017.

11. An oversight framework called the RP-n Oversight Framework was introduced in April 22, 2016 and it was designed to ensure the efficient and consistent management of the RP-1 and RP-2 contracts, as well as any future contracts of a similar scale. The framework was designed based on the Community of Sponsoring Organizations (COSO) of the Treadway Commission framework, which is a commonly used internal control framework.

12. The RP-n Oversight Framework includes a number of elements such as leadership and direction, risk management, oversight and quality monitoring. It applies across all sectors in Real Property Services and all regions, and its effective implementation is a joint responsibility of Real Property Services and the regions. The Strategic Sourcing Sector within Real Property Services exercises national RP-1 and RP-2 oversight accountability. The Strategic Sourcing Sector developed and facilitates the implementation of the framework and acts as the integrator of all oversight activities related to the RP-1 and RP-2 contracts.

13. In its service delivery role, Real Property Services is structured around 7 service lines, each offering a variety of services both to PSPC and to other government departments. The current RP-1 and RP-2 contracts include services that touch all service lines. The 7 service lines are: Infrastructure Asset Management, Real Estate Services, Portfolio & Asset Management Services, Technical Services, Property and Facility Management Services, Project Management Services and Workplace Solutions Services.

14. Key information systems related to RP-1 and RP-2 expenditures include the Alternative Forms of Delivery System and PSPC's financial system, SIGMA. Contracting and expenditure information is input and maintained by the contractor in the Alternative Forms of Delivery System, although the system is owned by PSPC and PSPC has access. 

Focus of the audit

15. The objective of this audit was to assess whether processes and operational control activities over expenditures under the RP-1 and RP-2 contracts were designed and implemented as intended.

16. During the planning phase, we met with key stakeholders from the Strategic Sourcing Sector of Real Property Services in order to gain a better understanding of the administration and monitoring of expenditures under the RP-1 and RP-2 contracts. We reviewed relevant documentation such as the RP-n Oversight Framework, the RP-n Oversight Framework Reference Library and the procedures it referenced, relevant past engagements, oversight committee terms of reference, documents provided by the monitoring functions, and financial information related to expenditures. A risk-based approach was then used in establishing the objectives, scope and approach to this audit engagement.

17. The audit assessed key controls over expenditures spanning the period of April 1, 2016 to September 30, 2017, for all 6 regions. However, we reviewed some additional documentation until August 2018 to obtain a more complete understanding of certain activities.

18. Scope inclusions were determined based on the analysis of expenditure data for fiscal year 2016 to 2017. Expenditures for services provided within leased and crown-owned buildings were included in the scope of this engagement.

19. The scope of the audit focused on day to day internal controls within PSPC relevant to the management of expenditures under the RP-1 and RP-2 contracts. All 6 regions (Atlantic, Quebec, National Capital, Ontario, Western and Pacific) had responsibility for managing expenditures under the RP-1 contract, and the RP-2 contract covered one area in the National Capital.

20. We selected a sample of 74 invoices for testing, from a population of 1024 invoices recorded in the Alternate Forms of Delivery Management System. We also sent questionnaires and document requests to key personnel in the regions. We did not audit the operations of the service provider.

21. Governance and risk management were reviewed at a high level during the planning phase of the audit, and the findings have been documented in a management letter.

22. More information on the audit objective, scope, approach and criteria can be found in the section “About the audit” at the end of the report.

Statement of conformance

23. The audit conforms with the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program.

24. Sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of the findings and conclusions in this report and to provide an audit level of assurance. The findings and conclusions are based on a comparison of the conditions, as they existed at the time, against pre-established audit criteria that were agreed on with management. The findings and conclusion are only applicable to the entity examined and for the scope and time period covered by the audit.

Observations

25. Audit observations were developed through a process of comparing criteria (the correct state) with condition (the current state). The following observations may note satisfactory performance, where the condition meets the criteria, or they may note areas for improvement, where there was a difference between the condition and the criteria.

Documented processes and procedures generally exist, but some require finalization and implementation

26. Processes and procedures provide an authoritative reference for employees concerning the proper administration of the RP-1 and RP-2 contracts. Well-developed processes and procedures contribute to the effective management of the contracts.

27. We expected to find that processes and procedures relating to the management of expenditures under the RP-1 and RP-2 contract were developed and communicated to the relevant stakeholders.

28. We noted that the RP-n Oversight Framework Reference Library was the main reference source for processes and procedures within the Strategic Sourcing Sector of Real Property Services. We reviewed the procedures and processes identified within this library.

29. We noted that some processes and procedures had not yet been created, such as the reconciliation of information between financial systems, and Audited Cost Statement acceptance and reconciliation. In addition, some of the procedures and guidance documents in the RP-n Oversight Framework Reference Library were not finalized and were still in draft form.

30. We confirmed that the RP-n Oversight Framework Reference Library was communicated and available to staff. 

31. Overall, we found that documented processes and procedures generally existed and defined how to conduct RP-1 and RP-2 expenditure activities. We also noted that these processes and procedures were communicated and available to staff.

32. Adequately developed and implemented procedures play a key role in supporting the effectiveness of internal controls by enabling PSPC to apply consistent standards and practices across all regions for its management of expenditures.

Key controls are designed and implemented, but could be strengthened

33. Controls are any actions taken by management or other parties to manage risks that may impede the achievement of established objectives. The objective is the proper administration of expenditures related to the RP-1 and RP-2 contracts. This includes ensuring that PSPC pays the correct amount for services received, as well as the amounts paid are in accordance with the terms of the contract. The audit reviewed project management controls and expenditure controls over billings and performance payments.

Project management controls

34. The National Project Management System is a methodology that is applied to project management activities to establish and maintain efficient, effective and standardized project management practices. Real Property Services is expected to follow the National Project Management System for all projects, as outlined in the Departmental Policy on the National Project Management System. This includes projects for which delivery is outsourced to the contractor, according to guidance issued by the PSPC Project Management National Centre of Expertise in the fall of 2017. PSPC contractors are not required to follow the National Project Management System when providing services to PSPC.

35. We expected to find the requirement to follow the National Project Management System had been communicated, and that the methodology was being followed by Real Property Services for RP-1 and RP-2 project delivery. 

36. We were informed that some of the requirements with respect to National Project Management System control points and deliverables for projects delivered by the Contract through RP-1 and RP-2 contracts had not been well understood within Real Property Services. Thus, at the beginning of the contracts the National Project Management System was not being used.

37. We were also informed that the Project Management National Centre of Expertise clarified these requirements in the fall of 2017, and that work was underway to formally update a directive on the National Project Management System to formally communicate these requirements. The centre also indicated that future National Project Management System conformance monitoring would take place, and would include review of conformance with this requirement.

38. Application of the National Project Management System by Real Property Services during project management activities is important for ensuring that projects are managed in a manner that maximises the chances of successful project delivery with respect to scope, time, and budget.

Work authorization controls

39. A work authorization is a tool used to authorize the contractor to commence work on an “as and when requested” basis, within an identified upper limit of work costs, and in accordance with terms and conditions of the contract. The contractor must not begin any work before receiving such work authorization from PSPC, to ensure that only admissible work is carried out.

40. We expected Real Property Services to implement and consistently apply a work authorization process to ensure that the property management services, project delivery services, and additional services provided by the contractor are only provided for authorized services and amounts. More specifically, we expected work authorizations to be consistent with identified requirements per the planning process, and that the authorizations were properly approved.

41. We were provided reports of all invoices issued by the RP-1 and RP-2 Service Provider for fiscal years 2016 to 2017 and 2017 to 2018 and 74 invoices were randomly and judgmentally selected from all regions for detailed review. This included a random sample of 63 positive value invoices using a 90% confidence interval and 10% margin of error, and 11 judgementally selected invoices based on materiality and professional judgement.

42. We found that for 15 out of 74 samples, work authorizations were not applicable because they were either for services which were not identified through the departmental processes, or because the same work authorizations applied for more than one sample. For the remaining 59 applicable work authorizations, 58 were in concurrence with the expected planning documents such as program of projects and/or with the most recently accepted operations, maintenance and utilities plan (as applicable) with the estimated costs concurring with the applicable planning documents.

43. For 6 out of 74 samples, the testing of delegation of authority was assessed as not applicable, because the same work authorizations applied for more than one sample. For the remaining 68 applicable work authorizations, we found 57 were appropriately approved. For 11 samples, we either found errors or could not confirm delegation via appropriate delegation of authority forms.

44. We concluded the work authorizations processes were in place and were generally consistently applied across all regions to ensure that work related to the RP-1 and RP-2 contracts was appropriately authorized.

45. Consistent application of these processes is necessary for ensuring the services provided are requested and approved by PSPC before work begins.

Expenditure controls over billings and payment

46. The Real Property contractor submits invoices to Real Property Services for payment on a monthly and annual basis. The invoice certification process consists of amounts invoiced being reviewed for accuracy and admissibility and approved. Payment is trigged by entry into the PSPC financial system.

47. We expected to find Project Invoice Delivery Reports, which contain a detailed breakdown of project costs, as well as milestone billing schedules on file, as evidence the expenditure had been reviewed for accuracy and admissibility. We also expected that each invoice would be appropriately approved, which is further evidence the expenditure was deemed to be accurate and admissible.

48. To confirm that controls exist and are functioning and correct payments were made, we expected that each invoice submitted by the contractor was accurately recorded in the Alternate Forms of Delivery Management System and SIGMA. The Alternate Forms of Delivery Management is a repository of data that is entered and submitted electronically by the contractor. SIGMA is the Department's contractual and payment information system that houses the expenditure data for the RP-1 and RP-2 contracts.

49. Lastly, to confirm the work being paid for was authorized in accordance with the work authorization process, we expected only authorized work and their related billings were paid for. 

50. Using the same sample of 74 invoices, we found there was sufficient evidence provided to support the conclusion that the Project Invoicing Detail Reports were being reviewed prior to payment. We noted that the Project Invoicing Detail Reports were reviewed by the appropriate manager in 25 out of the applicable 27 invoices.

51. We noted there was inconsistency with respect to the review of milestone billing schedules. We found that milestone billing schedules were on file for 53 of the 66 applicable selected invoices. After 2 follow-up inquiries, we were unable to obtain a milestone billing schedule for 13 of the selected invoices, with 12 of those being from the same region.

52. We reviewed documentation to determine whether the invoices were appropriately approved by a person with delegated authority, and whether milestone billing schedules and project invoice detail reports were on file. We found that all of the 74 selected RP-1 and RP-2 invoices had the appropriate approvals.

53. We found that for each invoice selected, the appropriate information had been entered into the Alternate Forms of Delivery Management System and SIGMA. The data presented in each system was accurate and adequately documented.

54. Finally, we also matched the authorized cost and fee amounts on the work authorizations to the invoices, to determine whether only authorized amounts were paid for. We found that the amounts billed for work were authorized via an approved work authorization for all but 1 sample.

55. We concluded that controls over the administration of expenditures exist and are generally functioning to support the payment of RP-1 and RP-2 invoices. Overall, amounts paid were exclusively for services provided, were reflective of actual costs incurred by the service provider and were for authorized services only. 

56. Consistent application of these processes is necessary for ensuring that the services provided are accurately and adequately invoiced by the contractor.

Expenditure controls over performance related payments

57. Performance measurement allows management to incentivize the Real Property contractor to ensure key objectives are attained. Carefully selected Key Performance Indicators are used for measuring progress over time and performance against targets, and to help identify areas for increased attention.

58. The RP-1 and RP-2 contract terms outlined that contractor fees are only payable for each invoice at the agreed upon percentage for each contract (75% for RP-1 and 85% for RP-2), with the remaining fees being identified on a separate line, withheld, and only paid out upon a successful year-end result of Key Performance Indicators of the Real Property contractor.

59. We expected that Real Property Services withheld a percentage of the contractor's fees when paying invoices throughout the year. We also expected that Real Property Services developed Key Performance Indicators to measure the contractor's progress and success, reviewing and approving them at the beginning of the fiscal year. We also expected that Key Performance Indicators were monitored throughout the year.  Further, we expected performance against the Indicators was assessed at the end of the fiscal year.  And finally, we expected year end invoices related to payment were subject to reconciliation and approved before the fees withheld were released.

60. Using the same sample of 74 invoices, we tested whether the contractor's fees were withheld adequately throughout the year. We found that all applicable invoices tested contained and clearly identified the appropriate fees and fee holdbacks based on the type of invoice. We also found they were all approved by the appropriate Section 34 approver.

61. We also determined that the indicators were established at the beginning of the year and that they were validated for each of the regions, at year end, by an appropriate authority.

62. While evidence of monitoring of the indicators at monthly meetings was provided by 5 of the regions, 2 of the regions did not provide evidence of monthly review of the indicators, and compliance could not be assessed.

63. Year-end invoices submitted by the contractor requesting the payment of fees withheld throughout the fiscal year were submitted after the PSPC validation of Key Performance Indicator results.

64. We also noted that 3 regions were performing reconciliations between fees withheld throughout the year and fees invoiced following the approval of the indicators at year end. All invoices had been appropriately approved.

65. We concluded that the release of performance-related amounts is being supported by an adequate assessment of the Service Provider's performance, is appropriately approved and is in accordance with the contracts.

66. A well implemented performance management process is essential for ensuring the government is obtaining value and the expected outcomes from its transactions with the real property contractor by incentivizing the contractor to ensure key objectives are attained.

Monitoring activities at the operational level are occurring, but reporting to the senior management level could be improved

67. Monitoring and reporting mechanisms support the observation and collection of tangible evidence that objectives are on track to be achieved, and that desired outcomes are being met.

68. We expected for Real Property Services to have designed and implemented mechanisms to report on and monitor the financial accuracy and technical quality of the services provided by the contractor. We also expected mechanisms to monitor PSPC's conformance to internal polices, processes, and procedures and its administration of expenditures under the RP-1 and RP-2 contracts.   We expected the plan would be approved by the oversight bodies, and the results would be reported as well.

69. We found that Real Property Services designed a monitoring framework  which included:

• risk based contractor quality monitoring of the contractor's quality management system and conformance with the terms and conditions of RP-1 and RP-2 contracts. These engagements reviewed the technical and administrative aspects of the contractor's quality management system. There was no prescribed volume or frequency for such engagements under the framework. The contract also allows Real Property Services to review the contractor's performance regarding the procurement expectations via more in-depth quality monitoring engagements
• conformance monitoring of PSPC's consistent application of internal processes and procedures by PSPC employees. There was no prescribed volume or frequency for such engagements under the framework
• financial quality monitoring to provide reasonable assurance that costs and fees, including labour costs, claimed by the contractor for property management and project delivery services were in accordance with the RP-1 and RP-2 contracts. The intended frequency of such engagements was quarterly review of project delivery services transactions, and monthly review of property management transactions

70. Planning: We found that the Real Property Services Quality Monitoring Centre of Expertise developed an annual plan that contained all contractor quality monitoring, conformance monitoring and financial quality monitoring engagements. We found the 2016 to 2017 and 2017 to 2018 plans did not receive the approval or endorsement of senior management or any oversight body. While monitoring activities and engagements identified in the annual plans did take place, the plans identified multiple planned or proposed engagements which did not end up being conducted within the suggested timelines. 

71. Conduct—Contractor Quality Monitoring: We found that PSPC conducted contractor quality monitoring engagements focused on the contractor's quality management system and the contractor's conformance with the contract requirements. These engagements were led by the Quality Monitoring Centre of Expertise, with support from quality monitoring staff from the regions.

72. We found that the quality monitoring activities related to contractor conformance have been effective at identifying financial and operational non-conformities of the contractor with terms and conditions of the RP-1 and RP-2 contracts.

73. The engagements generally found that the contractor met the expected requirements. Minor exceptions, non-conformities and opportunities for improvement were identified.

74. An engagement was conducted in the fiscal year 2017 to 2018 to review how well the Contractor managed non conformities, and the overall conclusion was that the contractor adequately managed the resolution of non-conformities. Minor exceptions were also reported to the technical authorities, who have the authority to decide on the best course of action to address them.

75. An in-depth quality monitoring engagement to review the contractor's performance regarding the procurement expectations was conducted in the summer of 2018. We noted the financial quality monitoring engagements identified some instances for which the contractor did not respect its own subcontracting policy with respect to the RP-1 and RP-2 contracts.  We did not see any evidence of these findings being communicated to senior management.

76. The RP-1 and RP-2 contracts identify expectations for the contractor to use subcontracting practices that are accessible open, fair and transparent, that provide opportunities for participation by industry, including small and medium enterprises, and that result in competitive bidding for subcontracts. However, the contracts do not prescribe or dictate how the contractor is to achieve these expectations.

77. Conduct—Conformance Monitoring: We determined one engagement was conducted to evaluate compliance by PSPC employees with the PSPC processes and procedures during the period under audit. The engagement reviewed regional conformance with the directive on the RP-n delegation of administrative authority.  We noted the findings were positive overall, and only minor exceptions were identified. No findings were deemed significant enough to require an action plan.

78. Conduct—Financial Quality Monitoring: We found that financial quality monitoring engagements related to property management services and project delivery services were conducted by Real Property Services' Financial Quality Monitoring Unit.

79. We found that the intended frequency of monthly review of samples of property management transactions was consistently met. We also found that the intended frequency for quarterly samples of project delivery services transactions was not consistently met for the time period under review.

80. Financial monitoring findings included late payment charges that were charged to PSPC as allowable costs (the majority of instances were detected and reversed by the contractor before the audits discovered them), instances of work order/purchase order amounts being lower than the invoice from suppliers, instances of costs submitted that were not incurred in the month in which they were submitted, incorrect processing of accruals, allocation of costs to the wrong portfolio and inconsistent allocation methodology for direct labour overhead costs.

81. Also noted by the Financial Quality Monitoring Unit were instances where some of the administrative fees charged by the sub-contractor to the contractor were not in accordance with the contract, and instances where some of the direct labour fees charged by the sub-contractor to the contractor were not in accordance with industry rates.

82. The identification of late charges being charged to PSPC were the result of late payments by the contractor to its suppliers. For example, audits of property management services transactions identified multiple instances of late payments for utilities.

83. While RP-1 contracts do not contain explicit expectations in relation to the contractor's timelines and timeliness for payments to its subcontractors, it is implied that the contractor will respect the terms and conditions outlined in the contracting documents between the contractor and its subcontractors. The RP-2 contract does outline some expectations regarding the timeliness of payments to suppliers and subcontractors in the context of expenditure management. The RP-1 and RP-2 contracts stipulate that the Government of Canada is not responsible to any subcontractor. The contractor is responsible for paying any subcontractors for any part of the work performed.

84. While the Real Property contractor is not an agent of PSPC, the timeliness of payments to subcontractors by the contractor can have an influence on PSPC's reputation. It is important for Real Property Services to continue to use its monitoring mechanisms to ensure that no systematic issue exists in regards to timeliness of payments to subcontractors.

85. Reporting: The results of the contractor quality monitoring engagements were shared with the contractor and the PSPC technical authorities responsible for ensuring findings are addressed for their respective regions. However, the results were not provided to any senior or oversight bodies.

86. For compliance monitoring engagements, the regional results were distributed to the technical authority. There is no mechanism to report and monitor action plans to address performance or conformance gaps. The results of these engagements were not provided to any senior or oversight bodies.

87. We found the results of financial quality monitoring of costs and fees were primarily communicated to technical authority organizations responsible for managing the work under the RP-1 and RP-2 contracts in their respective regions. However, they also were not provided to any oversight bodies or senior management.

88. We concluded there are mechanisms in place to monitor performance of the contractor and PSPC employees. We identified opportunities for improvement related to the development of action plans for findings related to PSPC staff and regular reporting and monitoring to senior management.

89. Reporting and monitoring activities support management in ensuring that the expenditures under the RP-1 and RP-2 contracts are managed a manner that facilitates continuous improvement. Development of action plans and monitoring implementation contributes to continuous improvement. Periodic reporting of results to oversight bodies and senior management provides an opportunity for exercising oversight and providing strategic advice, which enables continuous improvement.

Data related to Real property-1 and Real property-2 expenditures in Public Services and Procurement Canada's information systems is accurate and complete

90. SIGMA is the Department's contractual and payment information system that houses the expenditure data for the RP-1 and RP-2 contracts. The Alternate Forms of Delivery Management System is a repository of data that is entered and submitted electronically by the contractor. Total invoiced amounts are recorded in both systems.

91. We expected to find that SIGMA and the Alternate Forms of Delivery Management System contained complete and accurate information, and that amounts recorded in the systems are compared and reconciled regularly.

92. Using the same sample of 74 invoices, we noted that for the 72 applicable invoices, the financial information was consistently and accurately recorded in both systems.

93. An additional 15 invoices were reviewed to determine whether reconciliations between invoiced amounts and amounts recorded in SIGMA and the Alternate Forms of Delivery Management System were performed. We noted that reconciliation activities were being conducted and results were being recorded and followed up on for all 15 invoices. We also noted that there was no documented procedure for the reconciliation of expenditure amounts between financial systems.

94. Accuracy of data in the information systems is essential for ensuring sound stewardship of public funds, as having accurate expenditure information allows management to track and monitor spending and make informed decisions in support of successful project delivery.

Conclusion

95. Overall, we found that the processes and operational control activities identified for review were generally developed and being followed. Our audit results indicated the financial administration of RP-1 and RP-2 contracts is appropriate. However, some opportunities for further improvement exist.

96. We found that documented processes and procedures existed, and defined how to conduct RP-1 and RP-2 expenditure activities. These were documented in the RP-n Oversight Framework Reference Library, which was communicated and available to staff.

97. We noted that several key controls over the RP-1 and RP-2 expenditures were designed and implemented but improvements could be made.

98. We were informed that the requirement that Real Property Services use the National Project Management System for the RP-1 and RP-2 contracts was not well understood at the implementation of the contracts. This requirement was clarified in the fall of 2017 and work is underway for formally communicating the requirement. There are plans to begin monitoring compliance following the implementation. PSPC contractors are not required to follow the National Project Management System when providing services to PSPC.

99. We noted that controls over work authorizations were in place and consistently applied across all regions. This ensured that expenditures related to the RP-1 and RP-2 contracts were appropriately administered.

100. Controls over the management of expenditures, which support the payment of RP-1 and RP-2 invoices, exist and are functioning as intended. Overall, amounts paid by PSPC were for services provided and were reflective of actual costs.

101. We found that Real Property Services appropriately withheld fees throughout the year and developed key performance indicators to measure progress during the year. The key performance indicators were monitored throughout the year and assessed at year end before fees were released to the contractor.

102. Formal financial and quality monitoring of activities delivered through the RP-1 and RP-2 contracts have been designed and implemented in the regions. Real Property Services employs reporting and monitoring mechanisms over the financial and technical quality of services provided by the contractor at the operational level. We also identified a missed opportunity for continuous improvement, with respect to reporting of monitoring activity results to oversight committees, the oversight, approval, coverage, and full execution of the annual quality monitoring plan. This would also include tracking and resolution of internal conformance findings.

103. Data related to RP-1 and RP-2 expenditures in PSPC's information systems is accurate and complete. We found that invoiced amounts were recorded accurately in the SIGMA and Alternate Forms of Delivery Management System, that reconciliation activities were being conducted and that results were being recorded and followed up on when required. We found that there is no documented procedure for the reconciliation of expenditure amounts between financial systems.

Management response

Management has had the opportunity to review the report, and agrees with the conclusions and recommendations found therein. Management also developed a Management action plan to address these recommendations.

Recommendations and management action plan

Recommendation 1

The Assistant Deputy Minister, Real Property Services should ensure that processes and procedures related to expenditure management for the RP-1 and RP-2 contracts contained in the RP-n Oversight Framework Reference Library are developed, finalized and implemented.

Management action plan 1.1

Initiate and finalize reviews of all oversight processes/procedures specific to expenditure management.

Management action plan 1.2

Develop new and/or update existing expenditure management processes/procedures as required.

Management action plan 1.3

Circulate draft expenditure management processes/procedures for national review and consultation.

Management action plan 1.4

Present expenditure management processes/procedures requiring approval to the Real Property Services Policy Steering Committee (PSC).

Management action plan 1.5

Schedule and deliver national training to complete the implementation of expenditure management processes/procedures.

Management action plan 1.6

Provide direction to the National Capital Area (NCA) to ensure project documentation processes, including milestone billing schedules, are actioned and documented appropriately. Training to be provided if necessary.

Recommendation 2

The Assistant Deputy Minister, Real Property Services should ensure that the quality monitoring plan is approved and results and action plans are reported to branch oversight committees or senior management.

Management action plan 2.1

Future quality monitoring plans will be submitted to a senior management level branch oversight committee for review and the Director General of the Strategic Sourcing Sector for approval. The 2019 to 2020 quality monitoring plan will be presented and approved by March 2019.

Management action plan 2.2

A quality monitoring update process will be implemented to provide on-going regular feedback to senior management about the progress of quality monitoring activities.

Management action plan 2.3

Future quality monitoring results and action plans will be submitted to a senior management level branch oversight committee for information and feedback. The 2018 to 2019 quality monitoring results and action plan will be presented by July 2019.

About the audit

Authority

This engagement was included in the Public Services and Procurement Canada (PSPC) 2016 to 2019 Risk-Based Audit and Evaluation Plan.

Objective

The audit's objective was to determine whether the Procurement Branch has developed and implemented an oversight and control framework to achieve its transformation objectives.

Scope and approach

The audit focused on the governance, risk management, and the control processes of the Procurement Branch's Acquisitions Program Transformation (the program). The audit assessed the program's progression, spanning from September 2015 to June 2017. Included in our assessment was the design, implementation and effective operation of the Smart Procurement, the Supporting Our People strategies, and preliminary accomplished to date on the e-Procurement solution within the Leveraging Digital Technology. A detailed examination of the e-Procurement solution project is planned to be the subject of future audit.

This audit covered the period from September 2015 to June 2017. However, we reviewed documents outside of this time range to obtain an understanding of prior procurement transformation Solutions to draw on lessons learned, as well as items and documentation which supports the program activities prior to September 2015.

This audit was conducted in accordance with the Institute of Internal Auditors' International Standards for the Professional Practice of Internal Auditing.

Audit Criteria

The criteria used to assess the program's oversight and control framework were derived from the results of the detailed risk assessment, and risk areas with risk levels of moderate or above were included in this audit. The audit criteria were developed based on guidance from the Treasury Board Secretariat's Management Accountability Framework.

Control process: Processes and control activities are in place to ensure that expenditures related to the RP-1 and RP-2 contracts are appropriately administered.

1. Documented processes and procedures exist, and are available to staff involved in the management of RP-1 and RP-2 contracts
2. Key controls are designed and implemented to ensure that amounts paid are in accordance with the terms of the contracts:
   1. amounts paid are exclusively for services provided, are reflective of actual costs incurred by the Service Provider and are for authorized services only
   2. release of performance related amounts is determined by an adequate assessment of the Service Provider's performance and in accordance with the contracts
3. Formal monitoring and reporting mechanisms are in place to provide management with information which allows it to monitor expenses under the RP-1 and RP-2 contracts
4. Data related to expenses in PSPC's information systems is accurate and complete

Audit work completed

Audit fieldwork for this audit was substantially completed on August 27, 2018.

Audit Team

The audit was conducted by members of the Office of the Chief Audit Executive, overseen by the Director of Procurement Auditand under the overall direction of the Chief Audit Executive.

The audit was reviewed by the quality assessment function of the Office of the Chief Audit Executive.