2006-747 - Audit of the Contract Related to the Business Transformation Initiative, Final Report and Management Action Plan

2007-05-18

Table of Contents

• Main Points
  • What was examined
  • Why it is important
  • What was found
  • Management Response
• Introduction
• Background
  • Key Features of the Contract
• Focus of the Audit
• Observations
  • Section 1 Activities prior to contract award
  • Section 2 Activities after contract award
    • 2.1 Project Management
      Case Study on eAuctions
    • 2.2 Contract Management
    • 2.3 Financial Management
  • Section 3 Procurement Transformation Issues
    • Highlights of Procurement Transformation Achievements
• Conclusions
• Recommendations
• About the Audit
  • Objectives
  • Scope and Approach
  • Criteria
  • Audit work completed
  • Audit Team
• Subsequent Events
• Appendix 1 - Management Action Plan

Please note that under the Access to Information Act a limited amount of text within these documents may not be disclosed, and will be shown as [ * ].

Main Points

This Main Points is not a stand alone document and must be read in conjunction with the body of this report.

What was examined

This audit examined the overall planning and management of a contract related to the Business Transformation Initiative - the A.T. Kearney Ltd. service contract, including compliance with government authorities, financial controls and reporting. The audit also examined the contract for examples of results achieved to date, and requirements under the contract such as knowledge exchange. The audit noted impediments and advancements in procurement transformation. The period under review for project management was from contract award in November 2005 till October 2006.

Why it is important

Procurement transformation is a complex, multi-faceted multi-stakeholder, and multi-year initiative. PWGSC Senior Management indicated that it was imperative for the department to demonstrate significant progress towards key objectives during fiscal year 2006/07 in order that the savings projected for 2007/08 could be realized.

The federal government spends approximately $20 billion annually contracting for goods and services, and PWGSC is contracting authority for 9% of the transactions that make up 90% of this total value. The Government believes that major improvements can be made in the way procurement is done, and under the PWGSC departmental plan called the "Way Forward", has set ambitious improvement targets that began to take effect in 2005-06. The contract that is the subject of this audit forms a crucial part of the current initiative to transform the government's procurement functions. The purpose of the contract was to obtain high-level procurement transformation knowledge and experience. During the 8-10 month period covered by the audit (January to October 2006), $19.8 million (GST not included) was spent within the approved contract limit of $24 million (GST included). As procurement business transformation is still under way, this audit provided an opportunity to examine progress to date and to identify some lessons for the future.

What was found

The procurement transformation initiative, under this contract, was undertaken within a very dynamic environment where priorities and assigned resources were adjusted to meet rapidly evolving needs. This strategic context placed a high-demand on effective project management. It was in this context that major project management challenges occurred with regards to this contract. Notwithstanding, as noted in Section 3 Procurement Transformation Issues, significant progress did occur on procurement transformation.

The requirement for procurement transformation, under this contract, was originally expected to last from four to seven years. The pace of the contract work was accelerated to develop and implement procurement tools that would assist the department in meeting the savings target for 2007/08. This resulted in the total approved contract funds being spent in less than one year. The contract ceased to be a normal type of service contract as the role of the Contractor shifted from consultant/advisor to very active developer and implementer of procurement transformation activities, jointly with PWGSC. As a result, a higher proportion of contract senior level staff was used than originally estimated in the solicitation.

The contract Task Authorizations were approved as a shared level of effort between PWGSC and the Contractor. Task activities were complex, often deferred or cancelled, and underestimated in level of effort. The outcomes identified in the Statement of Work for the four-year contract were not materially altered. As a result, not all of the deliverables could be completed within the new time lines.

The Project Office did not demonstrate due diligence in the exercise of effective project management and controls. Project Management controls were inadequate, decisions were taken without adequate planning and without documented analysis of the risks and benefits. In some instances, existing financial controls did not ensure full and consistent adherence to policies on commitments and payments. Documentation on oral interactions and decision points was too limited to provide assurance that Senior Management had accurate and complete information. The expertise and advice of departmental oversight groups on critical contract decisions or procurement transformation strategies was not sought early nor were concerns fully addressed. Individuals responsible for the project changed and their roles were not clearly articulated. Some of the project management responsibilities were transferred to the Contractor, whose staff assumed important planning, monitoring, and reporting tasks. In addition, some of the normal checks and balances used to ensure the integrity of the procurement process were weakened due to the close organizational relationships of the Contracting Authority and the Project Authority prior to January 2006.

PWGSC Senior Management's position was that the risks were commensurate with the benefits of the contract. They asserted that PWGSC needed to act quickly to obtain better information about procurement spending, in order to identify specific savings opportunities and convince federal departments and central agencies of their feasibility. Rather than follow the original timetable and perform all the necessary steps for due diligence, PWGSC decided to speed up and accepted the short-term risk in exchange for greater benefits over the long term.

Acquisitions Branch did not have an adequate project and a resource-management plan in place to help it absorb the knowledge exchange and operationalize the new ideas and methods of work that were being introduced by this contract. As well, an effective change management process was required to take advantage of the procurement strategies introduced by this contract and to advance changes to department and government procurement policies, training and upgrading skills of procurement officers, and modifications of procurement systems. At the time of the audit, the Department was continuing to address procurement transformation activities, including internal capacity-building and change management strategies.

Although there were major project management challenges related to the contract, the audit noted that improvements in government procurement tools have occurred during the period of the contract. Although it is too early to assess the overall results of the procurement transformation, important groundwork has been completed during the period of the contract and the Contractor has made an important contribution to the work. In the initial stages of this contract, departments were not fully engaged and the transformation process was hampered by limitations in department's financial systems and their recording of expenditures. Nonetheless, a comprehensive database of government procurement ('spend cube') has been created and government spending trends analyzed, extensive engagement of departments has been undertaken, new procurement sourcing strategies have been piloted with varying degrees of success, and some of the planned savings have been achieved.

The audit makes five recommendations:

• The Assistant Deputy Minister, Corporate Branch should put departmental practices/policies in place that prior to the solicitation process of any major service contract for which PWGSC is the client, an independent assessment of the requirement be undertaken, which includes a clear governance structure and definition of roles and responsibilities, and identification of any major obstacles that could affect achievement of the expected results.
• The Assistant Deputy Minister, Acquisitions Branch should put practices in place, which ensure that management decisions that substantially alter the scope, timing, or costs of active contracts for which PWGSC is the client are reviewed by departmental oversight groups and that analysis of impacts, risks, options, and costs-benefits are prepared and decisions documented.
• The Assistant Deputy Minister, Acquisitions Branch should re-examine the number and use of Standard Acquisition Contract Clauses to ensure that the terms and conditions are appropriately used and consistent with the current procurement environment.
• The Assistant Deputy Minister, Acquisitions Branch, with the support of the Department, should develop an explicit new change management strategy and a resource-management plan that takes into account the risks and issues identified herein, and enable the Department to continue successfully advancing the Acquisitions Business transformation.
• The Chief Financial Officer should communicate to line management staff about any errors, including non-critical ones found in the Post Payment Verification, whenever there is a possibility that the knowledge would help prevent a recurrence of errors.

Management Response - Summary

Management accepts the findings of the Report as being fair and accurate. The ATK contract, as noted in the Report was awarded in an open, fair and competitive process and complied with financial authorities and Government Contracting Rules. The Audit notes that "Although there were major project management challenges related to the contract ... improvements in government procurement tools have occurred during the period of the contract". The Report provides further information on results achieved in Exhibit 3.

In managing the contract, while there was less emphasis placed on administrative procedures such as monitoring of time sheets, there was frequent (weekly and sometimes daily) management oversight and review of outputs and achievements. We are pleased to report that, savings of over $81M were obtained during 2005-06, and we are on track to obtain $204M in 2006-07. The foundation has been put in place to generate incremental savings for the Government of Canada in future years. This will rise as we implement additional Standing Offers resulting from this project. Going forward we have benefited from many of the methods and processes introduced by ATK during the course of this contract.

The audit also noted the importance of knowledge transfer by ATK to PWGSC during the course of the contract. ATK delivered a wide range of methods and related processes to help PWGSC improve in a number of areas, including Information Management, Client Engagement and Commodity Management. Delivery included detailed training packages, guide documents and supporting tools to assist PWGSC. PWGSC has applied the knowledge and tools developed by ATK to improve delivery of services to client departments and suppliers.

PWGSC is satisfied that it received good value from the expertise provided by ATK to develop the methodology and ongoing processes being used to implement and deliver improved procurement (demonstrated in the December 2006 ATK Executive Summary Report provided at the conclusion of the contract).

We agree with the recommendations of the Audit Report and will be implementing corrective action as indicated.

Introduction

The audit was undertaken to provide timely assurance and information on a contract related to Business Transformation Initiative - A.T. Kearney Ltd. service contract by PWGSC.

Background

The federal government conducts approximately 2.2 million transactions and issues approximately 470 thousand contracts and purchase orders annually. PWGSC manages 9% of the total procurement transactions that make up 90% of the total contract value within the total government procurement spend of approximately $20 billion annually.

There has been a long history of procurement studies within the Government that identified the need for procurement reform. The most recent was the Parliamentary Secretary's Task Force on Government-wide Review of Procurement in December 2003.

In the 2004 Budget, the Government announced it was launching several initiatives to strengthen financial management and operational integrity. It created the Expenditure Review Committee, whose mandate was to review and reprioritize government expenditures.

In November 2004, PWGSC presented to Expenditure Review Committee a strategy on procurement opportunities and potential savings for the government. The strategy proposed a target of $2.59 billion in savings over 5 years (2005-2010), and an investment of $90 million. To achieve these savings, the strategy identified 4 key elements: mandatory use of standing offers; obtaining better pricing; travel modernization; and using better management tools. (Exhibit 1)

Exhibit 1 - Expenditure Review Committee Savings

	2005-06 (M $)	2006-07 (M $)	2007-08 (M $)	2008-09 (M $)	2009-10 (M $)	Total (M $)
Use of existing Standing Offers	53	122	267	311	311	1,064
Better Prices	6	29	228	361	370	994
Travel Modernization	0	50	80	115	130	375
Administrative Savings	0	3	23	54	77	157
Total Savings	59	204	598	841	888	2,590
Investment needed at PWGSC	(20)	(25)	(25)	(20)	0	(90)
Net Savings	39	179	573	821	888	2,500

Expenditure Review Committee also approved PWGSC's 'Way Forward' strategy. The 'Way Forward' is a wide-ranging strategy led by PWGSC to find innovative ways to deliver services smarter, faster, at a reduced cost, and to improve how the Government of Canada does business. There are two fundamental, interrelated objectives at the core of 'Way Forward': saving money by maximizing efficiencies; taking advantage of economies of scale and leveraging existing expertise; and doing these things in an open, transparent way so as to preserve the confidence of Canadians in the integrity and fairness of the Government of Canada.

In February 2005, the Expenditure Review Committee identified almost $11 billion in new savings over 5 years across federal departments and agencies, including PWGSC's strategy and savings, which were incorporated in the Budget for 2005. The identified monies were transferred from existing departmental budgets for the 2005/06 and 2006/07 fiscal years and reinvested in Government priorities. The Government also approved an action plan to 'transform' procurement and generate savings of $2.5 billion through the: mandatory use of 'standing offers' negotiated by PWGSC for ten commodities categories; amending PWGSC and Financial Administrative (FAA) Acts; streamlining Canadian International Trade Tribunal regulations regarding disputes with vendors; developing higher service standards to reduce time; and strengthening the government's rights within contracts and the vendor performance policy.

While procurement transformation was championed by PWGSC as the common service provider, the success of the initiative required the full and active support of all stakeholders, departments, and central agencies. Several committees were identified to provide oversight and to resolve government-wide issues related to procurement transformation - the senior level interdepartmental committee of Deputy Ministers, the Treasury Board Portfolio Advisory Committee, and an interdepartmental Assistant Deputy Minister Steering Committee.

In May 2005, PWGSC submitted a budget for Treasury Board expenditure authority that identified the allocation of the $90 million investment over five years. There were 10 investment activities included in the budget: Spend Analysis; Commodity Management Training and Mentoring; Communication; Commodity Reviews and Business transformation Management; Value Management Office; Project Management; Policy process and governance; Office of Small and Medium Enterprises; Government of Canada Market Place; and Acquisitions Information Management Systems. Costs included monies for contracting dollars and the Acquisitions Business Transformation's internal operations. PWGSC stated it considered the multi-year renewal of procurement similar to a Major Crown Project and would implement comparable management and oversight mechanisms.

The funding allocation for the investment was approved for 2005/06 and 2006/07 with conditions of future reporting and effective project approval, which included substantive cost estimates and a business case. The Business Transformation Initiative for Government Procurement is identified as high-risk, as it is significant in its size, scope, and importance to government. Factors contributing to the risk were noted in financial and organizational issues, stakeholders expectations, and administrative/policy complexity.

PWGSC sought external private sector expertise to help with the procurement transformation through both contracting and recruitment. PWGSC posted a Letter of Interest on the Canadian electronic tendering system (MERX) to solicit the Industry's comments on its requirement for professional services contracts to support the review and improvement of federal government's procurement. The staffing process for Special Advisors started in February 2005, and two individuals were originally recruited in the fall of 2005. One Special Advisor was for Acquisitions and the other for Real Property Services. In January 2006, a new Special Advisor to the Deputy Minister, responsible for Acquisitions transformation was named.

By January 2006, PWGSC realized there was a need to adjust its initial 2004 strategy, in which the five-year savings of over $1 billion could be achieved through the mandatory use of existing Standing Offers. PWGSC, through a review of the state of procurement data on commodity spending and in consultation with departments and agencies believed that savings in 2007-08, and beyond, would not be achieved simply by increased compliance to the use of existing Standing Offers. For the savings to be achieved, there had to be better pricing via new commodity strategies.

In February 2006, PWGSC created a Strategic Transformation Branch that was responsible for coordinating and monitoring the implementation of the 'Way Forward' agenda and providing direction to branches on issues related to the changes. Strategic Transformation Branch built on the earlier work of the Strategic Transformation Group and was to strengthen the department's analytical function and integrated reporting capacity in the context of business transformation. The Deputy Minister chaired the first meeting of the Business Transformation Committee on March 27, 2006. The Committee was established to raise, discuss, and resolve issues that affect the implementation of the 'Way Forward' and met, on average, once a month.

Key Features of the Contract

At the outset of the solicitation for professional services in support of the Acquisitions 'Way Forward' implementation project, PWGSC identified that a core of knowledge and experience related to federal government procurement and relevant transformation initiatives already resided within the department. Therefore, the intent of the solicitation was to supplement that knowledge by procuring the services of a Contractor who demonstrated procurement transformation knowledge, experience, tools, techniques, and methodologies that have been developed and used in procurement transformation programs within other large organizations.

In response to the July 2004 Letter of Interest (LOI) posted on MERX, thirty-three (33) firms expressed interest. Of the thirteen firms who provided comments, five of them ultimately submitted bids in the subsequent Request for Proposal stage. At that time, EDS Canada Inc. partnered with A.T. Kearney Ltd to present their comments.

In November 2004, the Assistant Deputy Minister, Acquisitions Branch approved a procurement plan to competitively solicit a firm for expertise in procurement transformation. The services were to be paid on an 'as and when requested' basis. The Request for Proposal (RFP) was dated April 28, 2005 and posted on MERX on April 29, 2005 and closed July 20, 2005. The following bidder instructions formed the basis of the contract:

• The total estimated cost of the contract would be $15 million (GST included) over 4 years and 3 option-years.

• The contractor was to provide staff resources that met the minimum years of experience in the field of work for which the resources would be utilized.

• Prices were firm, all inclusive per diem rates for 3 personnel categories - Project Manager, Senior, and Intermediate. The estimated percentage of total level of effort attributable to each personnel category was 10/45/45. This percentage was used as a weight in evaluating the prices of the bids.

• Costs for the Contractor's staff travel and living expenses, as well as the firm's overhead and profit were included in the contract per diem rates.

• The Contractor was responsible for providing technical, administration and human resources support necessary to accomplish the work.

• The definition of a normal day worked was 7.5 hours. However, the contract permitted the charging of and payment for actual time worked if more or less than 7.5 hours. Payment was calculated based on the per diem rates.

• Task Authorization (TA) was to be the contractual means of identifying the required work. Normally, a TA is an administrative process for service contracts where a definite need for the service exists but the precise nature and timing of the need cannot be set out in advance. Project Authority provides a Statement of Work for each TA, including the required deliverables and schedule of delivery. The Contractor proposes a cost breakdown and detailed level of effort to carry out the task. The Contractor is not to carry out, nor be reimbursed for any work until it receives a signed TA from the Contracting Authority. The Contracting Authority may amend the contract, if required.

• The TA identified the Basis of Payment for the services. Payment could be firm price, ceiling price, or expenditure limitation. The Contractor was permitted to submit monthly invoices for payment.

• Subcontractor expenses and other direct expense, authorized through a TA were to be paid at actual cost without mark-up.

• Contract clauses for Time Verification and Discretionary Audit were included.

• In the event that PWGSC did not request work under the contract, the Crown's liability was limited to a minimum contract value of $750,000 (GST included).

• The Statement of Work identified three broad categories of required services: 1) Project Management and Change Management Support Services; 2) Value Chain Analysis and Process Improvement, including Project and Program Governance, Contract and Vendor Relationship Management, and Information Management and Performance Measurement; and 3) Training, Knowledge Transfer, and Implementation Support.

Fifty-five (55) solicitation packages were requested and five (5) bids were received. The solicitation's technical evaluation criteria consisted of two components - an evaluation of past project experience and an oral presentation. Five bids met the project experience criteria and each delivered an oral presentation to a Senior Executive panel. The final ranking of bidders was based on a weighting of 75% for technical competence and 25% for price calculated on the basis of average per diem rates.

On November 22, 2005, the Minister of PWGSC authorized the department to enter into a contract with the winning bidder. The approved contract value was $19 million (GST included). The cash flow for the contract was estimated at $12.6 million for a 4-year contract, $2.4 million for 3 option years, and $4 million for contingency. In addition, the Contracting Authority obtained advance authority from the Minister to "reprofile" funds between the initial contract period and the three option years, providing that the total cash flow did not exceed the total amount approved. This approval to shift funds recognized that the amount and rate of expenditures against the contract could be affected by unforeseen activities, which would have an impact on the government's ability to achieve procurement savings targets within the prescribed time frames.

On November 24, 2005, A.T. Kearney Ltd. was awarded a contract for $1.75 million (GST included). It is affiliated with its parent company, A.T. Kearney, Inc. As well, at the time of award, the parent company was a wholly owned subsidiary of Electronic Data Systems Corporation (EDS), which employs 117,000 persons around the world.

Normally, the roles and responsibilities of managing a contract such as this one are divided between two groups. The Contracting Authority is responsible for contracting issues such as: overview of progress and payments; amending the contract if necessary; and negotiating contract disputes. The Project Authority is responsible for the day-to-day management of the contract such as: monitoring performance of the TAs; ensuring compliance with requirements of the contract and specific TAs; paying invoices in accordance with the contract and financial rules; and monitoring total expenditures to ensure expenditures and commitments are within the maximum contract value and that the spending rate is appropriate.

In this instance, the overall day-to-day direction of procurement transformation and the guidance and management for the tasks, Contractor, and departmental staff was the responsibility of several groups working together as the 'Project Office'.

• Steering Committee consisting of the Deputy Minister, the Assistant Deputy Minister, Acquisitions Branch to provide project governance. The Associate Deputy Minister chaired the meetings in the absence of the Deputy Minister.

• Special Advisor to the Deputy Minister for Acquisitions Business Transformation as the Champion for procurement transformation.

• Strategic Transformation Branch as the group responsible to coordinate and monitor the implementation of the 'Way Forward' agenda. The Business Transformation Committee, chaired by the Deputy Minister, was to raise, discuss, and resolve issues that affect the implementation of the 'Way Forward'.

• Project Authority(s) named in the contract as the authority to recommend that Contracting Authority amend the contract, and as the Responsibility Center Manager authorized to sign for financial commitments and contract payments.

• Stream Leads for up to 7 activity Streams - Strategic Sourcing, Client Engagement, Information Management, Program Infrastructure, Performance Management, Process and Organization Improvement, and Communication.

• Departmental Teams for procurement commodities.

In late December 2005 and early January 2006, there were two changes in the governance of the 'Way Forward' and the Project Office. A new acting Assistant Deputy Minister, Acquisitions Branch, and a new Special Advisor on Acquisitions Business Transformation who reported to the Deputy Minister were named.

The Director General, Acquisitions Renewal Sector was the first Project Authority (PA#1). His authority was from the solicitation until the contract was amended in April 14, 2006. At that time, the Chief of Staff in the Special Advisor's office assumed the role of Project Authority (PA#2).

In January 2006, however, leadership in procurement transformation was transferred from the Director General, Acquisitions Renewal Sector to the Special Advisor on Acquisitions Business Transformation. The Special Advisor had been hired from the private sector, through the Public Service Executive Interchange; as a senior-level individual with extensive experience in the private sector in acquisitions, and possessing a strong expertise in change management and business transformation. The Special Advisor reported directly to the Deputy Minister and was the Champion for procurement transformation.

In a proposed accountabilities document, the responsibilities for procurement transformation were divided between the Assistant Deputy Minister, Acquisitions Branch and the Special Advisor. The main accountabilities of the Special Advisor were to: develop tools and processes to enable procurement transformation; provide advice and guidance on implementation to the Deputy Minister and Assistant Deputy Minister; direct the ATK Kearney contract; direct management and control of transformation resources, and integrate and coordinate with the Acquisitions Renewal Sector. The Assistant Deputy Minister's accountabilities were to: provide overall leadership and accountability for the Acquisitions Branch; focus on integration of the 'Way Forward' and day-to-day operations; build stakeholders relationships; rebuild capacity, capability and organization structure; deliver submissions and related presentations; and in collaboration with Special Advisor identify, implement, and report on 'Pathfinder' projects.

It was at this point that the acceleration to the services provided by the Contractor occurred. Specific portions of the original requirement were accelerated such as: commodity sourcing; client engagement; and procurement spend analysis (spend cube). The contract ceased to be a normal type of service contract as the role of the Contractor shifted from consultant/advisor to very active developer and implementer of procurement transformation activities jointly with PWGSC. The funding for a 4-year contract plus 3 option-years was consumed in 8-10 months of work (January- October 2006).

The department used the advance approval authority from the Minister to 'reprofile' the funds. The Project Authority and Steering Committee interpreted the wording of the 'reprofile' such that all the funds could shift into one year of the contract rather than the funds from the three option years could be moved into the four contract years. To meet the requirement for expert capacity in procurement transformation, the Special Advisor or Project Authority(s) approved the rapid sourcing and deployment of additional qualified resources from the Contractor's affiliated companies or through sub-contractors. PWGSC was unable to provide sufficient and dedicated resources to the procurement transformation activities due to lack of resources expert in a particular commodity, staffing delays, and involvement in day-to-day operations of procuring goods and services for the Crown.

In a strategy to control the contract cash flow, the Project Authority limited the contract value to the value of work identified in the Task Authorizations (TA). For each approved TA there were increases to the value of the contract through contract amendments. These increases were drawdowns from the approved funding source of $19 million (GST included). Of the eight (8) contract amendments - two (2) were administrative and zero value, five (5) were drawdowns from the $19 million, and one (1) was an amendment to increase the authority of funding to $24 million (GST included).

By March 2006, the Project Office discussed strategies to identify sources and obtain additional funding. Sources of funds considered were: a reallocation of departmental funds from other Branches; a business case to access the Deputy Minister reserve; and a submission to Treasury Board for its expenditure authority to access future investment monies for the transformation program. As identified in Exhibit 1, the $90 million investment was allocated over four of the five years. Increased monies could be made available by changing the distribution of funds. Increasing the dollars in 2006-07 and decreasing the dollars in future years would not change the total approved investment. As well, there was discussion to seek Treasury Board contracting authority to increase the value of this contract. By May 2006, the Steering Committee's decision was not to pursue the modification to the investment monies nor the additional contracting authority.

By August 2006, the term for the Special Advisor, who had been hired in January 2006, was completed and the Project Authority role in the contract was passed to the acting Assistant Deputy Minister, Acquisitions (PA#3).

The value of invoices billed under this contract, as of August 31, 2006, was $19.8 million (GST excluded). For the work completed as of that date, the Contractor had billed 57.4% on Strategic Sourcing, 10.9% on Client Engagement, 10.8% on Project Infrastructure, and less than 5% on each of the remaining streams.

The Contractor completed certain contract obligations after the end of the audit fieldwork. As of November 27, 2006, PWGSC has received invoices totally $21.7 million (GST excluded) for work completed from January to October 2006.

Focus of the Audit

The focus of the audit was to examine the overall planning and management of this contract. There were four specific audit objectives dealing with project management, compliance, financial controls, and reporting. In addition, the audit reviewed the department's results achieved to date, examined knowledge transfer requirements under the contract and noted impediments and advancements in procurement transformation.

This audit engagement did not examine the entire PWGSC 'Way Forward' or the Government-wide initiative of Procurement Transformation. More information on the objectives, scope, approach, and criteria can be found in the section: 'About the Audit'.

Observations

Section 1: Activities prior to contract Award

1.1 Contract value was within PWGSC's authority.

The audit expected that the Contracting Authority would comply with the limits set out in the Government Contracts Regulations for entering into or amending a contract without the approval of Treasury Board.

According to the Government Contracts Regulations, for a service contract that was procured through electronic bidding the PWGSC Minister has the authority to enter into a contract up to $20 million (GST included) and to amend the contract by $10 million. In this instance the total value of this contract was $19 million and the one amendment was $5 million.

Therefore, in the case of this contract, the audit found that the authority to contract complied with the Government Contracts Regulations and was within the delegated authority of the Minister.

1.2 Lack of Arms-Length Relationship.

It was expected that the controls for the project would demonstrate a clear and appropriate segregation of duties between the Project Authority and Contracting Authority.

PWGSC normally has an arms-length relationship with client departments. Detailed departmental and government contract policies and practices are designed to ensure an appropriate segregation of duties and due diligence for the client as Project Authority and the PWGSC Contracting Authority. This division of duties is an important characteristic of internal control in that different departments/persons are responsible for different aspects of contract activities; so there is an increase in control. Effective internal controls prevent contracting irregularities and correct minor errors as they occur.

In this instance, prior to January 2006, both the Project Authority and Contracting Authority were in a direct reporting relationship to the Assistant Deputy Minister, Acquisitions Branch.

The resulting structure of reporting relationships and procurement expertise residing in both parties contributed to weakening some of the normal checks and balances used to guarantee the integrity of the procurement process.

The following three points (1.2.1 - 1.2.3) are examples of the results of the dilution of controls between Project Authority and Contracting Authority that were observed by the audit.

1.2.1 Unfunded requisition accepted.

Contrary to the intent of PWGSC Supply Manual section 3.020, the Contracting Authority accepted an unfunded requisition to start and continue the procurement process.

The normal practice is that a solicitation will not occur until funding is secured. This means the monies are committed or held in reserve for a potential contract. Given that a solicitation may take six to twelve months to complete the funds cannot be used for other projects or operations during that time. The rationale for this practice is the risk that the solicitation will be completed, the Suppliers will have expended funds in the preparation of a proposal and in the expectation of a contract, and the Project Authority discovers it does not have the funds within its authority and cannot commit to the contract award. Variances to the practice are made for exceptional or emergency requirements. In those circumstances the Contracting Authority should add a clause to the Request for Proposal stating that it is an unfunded solicitation.

In this instance, the Contracting Authority accepted an unfunded or zero value requisition in 2004 as commitment from Project Authority to engage in the procurement process. This requirement was neither exceptional nor an emergency. No funds were committed until one month before award (2005). It was felt that since the requisition was within the department and this sourcing for expertise was a priority to support the Acquisitions Business Transformation, there was no risk that the funds would not be available at the point of award. However, this practice was inconsistent with normal treatment of client departments.

1.2.2 Policy interpretation permitted selectively advising firms of an impending procurement.

Contrary to the intent of PWGSC Supply Manual section 5.41, six additional firms were selected and advised of this impending procurement. Providing such advice risked contravening the procurement practices of fairness and transparency.

It is normal procurement practice to notify firms who had expressed interest at the Letter of Interest stage of an impending solicitation. This is usually done with the posting of a Notice of Proposed Procurement. This provides Bidders the opportunity to consider the requirement and to obtain any security or certification that is required for the solicitation.

In this instance, prior to posting the competitive procurement on MERX, the Contracting Authority posted, in February 2005, a Notice of Planned Procurement. The Project Authority requested that the Contracting Authority add six selected firms, in addition to the thirty-three firms who had taken part in the Letter of Interest stage, to the list of potential bidders who would be notified of the impending procurement. It was felt that sending out additional notices was in the best interest of Canadians and PWGSC would maximize competition by notifying international firms, who were known to have core competency in procurement transformation but who may not regularly access MERX. There were numerous e-mails between the Contracting Authority's Director General and the Project Authority's Director General on sending the Notice of Proposed Procurement to the additional firms. Finally, the Project Authority (PA#1), wrote: "As the Director General responsible for Policy, your interpretation of section 5.041 of the Supply Manual will be revised as per a Policy Notification shortly, as it is not justified nor in-line with delivering the best value demanded by the Deputy and Assistant Deputy Minister". The Contracting Authority acted in accordance with the policy interpretation provided.

The Notice of Proposed Procurement and the Request for Proposal were posted on MERX on April 29, 2005. Copies of the Notice of Proposed Procurement were mailed to the list of potential bidders on May 06, 2005. Of the six firms identified by Project Authority only one, the incumbent Contractor, submitted a bid. As of the Audit fieldwork, no Policy Notification was completed.

As a result of the Project Authority's decision, the solicitation received special exclusion to a policy requirement that is not normally permitted or encouraged by Contracting Authority. As well, it had re-opened the interpretation that advising a few selected firms is an open, transparent and fair process.

1.2.3 Solicitation posted before completion and receipt of all evaluation activities.

Contrary to the intent of PWGSC Supply Manual section 6B.142, the Contracting Authority posted the solicitation on MERX before the proposed bid evaluation grid for technical evaluation and the oral presentation evaluation methodology were completed, although actions were taken to mitigate risks.

It is a standard contracting practice that the Contracting Authority reviews the Project Authority's proposed evaluation of supplier's bids in detail. This is important given that the most significant group (25%) of complaints brought to the Canadian International Trade Tribunal are for improper evaluations. However, there are exceptional or emergency requirements in which the evaluation is not available at the time of solicitation posting. It is normal in those types of requirements for the Contracting Authority to proceed with the posting but to take actions to mitigate the risk.

In this instance, the former acting Assistant Deputy Minister, Acquisitions Branch felt that it was critical for the Acquisitions Business Transformation agenda that the solicitation be posted on MERX immediately, even though Project Authority had not finalized its evaluation grid and all the decisions regarding the oral presentation evaluation methodology. Contracting Authority proceeded with the posting but in an effort to safeguard the fairness and transparency of the solicitation process and to mitigate the risks it opened the bids only after completion and receipt of all the evaluation activities.

PWGSC did not abide by its normal contracting practices. This was not an emergency or exceptional requirement. Although, the step to not open any bids until the evaluation activities had been finalized preserved the overall fairness of the process; potentially the department could have received complaints and as a result the award process and the critical expertise to support the Acquisitions Business Transformation could have been postponed. An appropriate segregation between the Project and Contracting Authorities would have decreased the likelihood of this preferential treatment occurring.

1.2.4 Non-competitive audit clause in contract.

It was expected that the Contracting Authority would structure the contract to reduce or eliminate the costs or consequences of potential risks.

One method of reducing risk is including a clearly worded, specific provision for the Crown's right to undertake an audit within the contract terms and conditions. The Contracting Authority selects standard contract terms and conditions from the Acquisitions Branch's Standard Acquisitions Clauses and Conditions (SACC) Manual. It is a standard practice that some contract terms and conditions are stated by reference only and others are stated in full text. This practice reduces the length of a contract document, but does not limit the obligation of a Contractor to be in compliance with all terms and conditions during the period of the contract. A limitation of the Standard Acquisitions Clauses and Conditions Manual is the number of different clauses related to one subject that a Contracting Authority may select. As an example, there are 4 different Discretionary Audit clauses and 3 different Time Verification clauses. Each clause is different in its Contractor's obligations and benefits to the Crown.

In this instance, the contract was a Fixed Time Rate basis of payment. It identified, by reference, three clauses on the Crown's right to audit. The three references were: General Conditions for services; Time Verification; and a Discretionary Audit for non-competitive contracts. General Conditions required that a Contractor keep proper records and make such records open to audit, inspection, and examination. The Time Verification clause stated that the time charged and the accuracy of the Contractor's time recording system may be verified by Canada's representatives before or after payment is made to the Contractor. The Discretionary Audit clause's purpose was to protect the Crown in the event that there is only one technically compliant bidder. In the event of only one bidder, the bidder must sign a certification that it is providing its best price to the Crown. Although this clause served a function in the solicitation, the events that unfolded with more than one compliant bidder's proposal made this clause non-applicable. The audit examined the other discretionary audit clauses and noted that none would provide an appropriate reduction of risk for contracts in which limitation of expenditure or ceiling price is used.

Neither the Project Authority nor Contracting Authority identified the weakness in the referenced audit clause. It was only during the audit, that it was identified that the clause was not applicable. This created a serious impediment for validating the time charged, an element of effective project management.

As a result of the non-availability of an appropriate Discretionary Audit clause, the Crown has limited its right to ask for supporting information on the accuracy of time and expenses charged. Notwithstanding the non-applicable audit clause, the Contractor had allowed the department access to some of its records.

Section 2: Activities after Contract Award.

2.1 Project Management.

The standard of project management was identified in the contract statement of work: "The project office will be managed in accordance with project management principles and methodologies consistent with current best practices such as those defined by the Project Management Institute and the Treasury Board Secretariat's (TBS) Enhanced Management Framework."

The audit expected that PWGSC as the project originator and as the Project Authority would have evidence to demonstrate how it exercised effective project management and control. This would include support for: monitoring by governance or oversight groups; clear roles and responsibilities, planning of goals, risks, tasks, resources and timing; documenting decisions, changes, actions, and results; monitoring of planned tasks to actual achievements; identifying critical issues or risks and proposing solutions; regularly tracking and reporting of progress; communicating with all stakeholders; and managing scope, time, cost, quality, and human resources of the project.

In this instance, the audit found that many decisions and approvals were undocumented. These included verbal instructions, meetings with high-level notes but no minutes or record of decisions, transitory e-mails, and as the auditor was told, pin-to-pin transmission in personal palm pilots or blackberries. This lack of an audit trail of information restricted the auditor's ability to depend on primary evidence. As well, it would have limited the ability of future project authorities to follow past approvals, decisions and actions.

Management's position was that the risks were commensurate with the benefits of the contract. They asserted that PWGSC needed to act quickly to obtain better information about procurement spending; in order to identify specific savings opportunities and convince federal departments and central agencies of the feasibility of savings. Rather than follow the original timetable and perform all the necessary steps for due diligence, PWGSC decided to speed up and accept the short-term risk for larger benefits in the longer term.

The following ten points (2.1.1 - 2.1.10) are examples of weaknesses in project management that were observed by the audit.

2.1.1 Project Management roles and responsibilities were not clearly defined or implemented.

As noted in the section "Key Features of the Contract", project management for the overall procurement transformation was the responsibility of a Project Office. Project management was complicated by several factors:

• It was a dynamic environment in which changes in the Task Authorizations (TA) scope and deliverables were agreed to, but not well documented;

• TAs were planning documents for project work rather then contractual documents. Work and deliverables identified in the TA were not the responsibility of the Contractor but rather a joint cooperative effort of both PWGSC and contract resources;

• Many of the resources, both contract and PWGSC, changed during the period of the contract. When individuals changed positions, their responsibilities sometimes changed or they carried out their responsibilities in different ways;

• The Special Advisor to the Deputy Minister, as the Champion for procurement transformation played an increasingly important role;

• The division of responsibilities between Stream Leads/commodity teams and contract staff was not clear. For example, while the Stream Leads were aware of what needed to be accomplished in their specific stream activity, due to rapidly shifting priorities, they were not always aware of who or how many contract resources would assist them and of the changes made to the proposed and approved TAs; and

• The Contractor assumed some project management functions as outlined in the TA.

These factors underlie some of the specific observations that follow.

2.1.2 Acceleration from a contract of 4-years to 8-10 months led to major Project Management challenges.

It was expected that the Project Office's decision to accelerate the level of effort in the contract would be well documented and supported by assessments of impacts to the contract and procurement transformation project in terms of risk, benefits and opportunities.

Contracting policies require that sufficient records be maintained on the contract file to provide evidence of critical decisions and the justification or support of such decisions/actions. As well, the Treasury Board Secretariat's Enhanced Management Framework questionnaire to Project Managers states: "Is there a change management process in place to ensure that changes are analyzed quickly to determine their impact (risk, cost and time) and that this information is brought to the attention of the appropriate level of management as soon as possible?"

Less than one month after the contract was awarded, the Steering Committee decided to accelerate the contract work requirements. The TA#1 scope (December 2005) included critical work on spend analysis, strategies and action plans for major commodities, client engagement, performance management, and the development of key planning documents.

Later in January 2006, the Special Advisor advised the Steering Committee to further accelerate the pace of the contract work in order to meet the Expenditure Review Committee savings target for 2007-2008. The TA#3 scope further accelerated the contract work on Strategic Sourcing Streams and actions to identify opportunities for savings. The end-result was the compression of a portion of the contract work requirement of 4-years plus three one-year options into 8-10 months of work.

During this contract, it was normal practice that only high-level notes were taken during the meetings of the Steering Committee and no records of decisions were taken during one-to-one meetings with the Deputy Minister. No explicit supporting documentation such as risk analysis, cost-benefit analysis, or adjustment in the expected contract scope was found. Also, interviews and documentation indicate that at the time of the decision, the counsel of oversight groups, such as Risk, Legal, or Finance was not sought nor were they made aware of the impacts of the accelerated contract work.

Examples of increased risk exposure and impacts that are directly linked to the accelerated timelines are:

• 30% of the contracted individuals never received the required contract security clearances;
• Five of the 40 activities identified in TA #1 were incomplete and deferred into TA#3. Others spanned the authorizations for TA#1/3/8;
• Some of the services outlined in the contract's statement of work, such as training, knowledge transfer and implementation support, could not effectively be achieved in such a short time-frame and would have benefited from the longer contract duration;
• PWGSC was unable to provide sufficient, dedicated resources to facilitate timely decisions, avoid delays, and ensure the most efficient use of contract staff;
• Change in cash flow meant that the opportunity costs of carrying out other critical departmental projects were not recognized; and
• Mitigation strategies or compensatory controls were not implemented to deal with the risks identified by the Contractor in the TAs and by both the Contractor and PWGSC in the status reports.

There should have been an explicit understanding of the department's risk tolerance on this decision to accelerate. The failure of the Project Office to deal explicitly with the change to the contract's time frame limited the timely recognition of the financial burn-rate, impacted the deliverables, and increased the risk to the department.

2.1.3 Contract Task Authorizations limited in value for contract management.

It was expected that the Project Authority(s) and Special Advisor would manage Task Authorizations as contract obligations that tasked the Contractor with deliverables (outcomes or outputs), which were achievable within the timeframe of the TA, and that could be validated or tracked.

As stated in the section on Key features of the contract, a TA was a contracting tool that identified the scope of the services or work to be undertaken by the Contractor. Normally, TAs define the precise nature and timing of the required service. Contractual amendments were required to incorporate the TA value into the contract.

The audit performed a general overview of all nine TAs and selected the largest two (TA#1, TA#3) for a detailed review of the deliverables. Some of the deliverables in the TAs were specific outputs such as preparing a commodity strategy or new templates for Request for Standing Offer documents. Others were described in more vague terms such as review, validate, enhance the development of a 'tool-kit', facilitate participation, or support a program to track results.

The audit team compared the deliverables or outcomes in the TAs to the percentage completion of results reported in the weekly status reports provided by the Project Office. It was expected that this comparison would provide a means of measuring contract performance. However, in this instance the TAs were approved by PWGSC as planning documents for procurement transformation project work rather than contractual documents. Work and deliverables identified in the TA were not the responsibility of the Contractor, but rather a joint cooperative effort of both contract and PWGSC resources. As well, in the dynamic environment of the project there was limited documentation on those activities, identified in a TA, which were modified, deleted, or deferred as other critical work arose. As a result, the Contractor's contribution and performance could not be segregated or separately measured.

During the audit review of TA#1 and TA#3, Management indicated that the original estimation of time to complete the TAs was not realistic given the acceleration, pace of work, and circumstances of procurement transformation. As an example, analysis of departmental procurement data took longer then expected given the data issues raised in section 3.3 of the audit. TA #2 for the urgent review of 4th quarter spending meant that work on commodity reviews was delayed in TA#1. The audit found no evidence that knowledge of poor data and other delays or circumstances was taken into consideration in the preparation of the later TAs.

As stated above, some of the deliverables were described in more vague terms. The deliverable for TA #5 and TA #9 was to provide expert advice. The Contractor was requested to provide the services of two senior consultants whose responsibilities were to provide strategic advice to the Special Advisor. The Special Advisor authorized the hours charged by the senior consultants in the invoices. Since there were no concrete deliverables expected the audit examined e-mails and other sources of information to try to validate the expenditure of $457K. The audit could not determine how these expenditures were tracked and validated.

In summary, the audit found the TAs as a contract management tool did not add-value in the management of the project and contract. The structure of the TA meant that the Contractor's contribution and performance could not be measured; the scope of the TA was optimistic with a wide breadth of services and expertise, and the achievement of an activity could not be validated in an environment of continuous changes without documentation. As a result, the procurement transformation project was not able to achieve all the services originally identified in the 4-year contract nor was the Project Office able to fully validate all the services provided. The impact on the project was that some of the services and deliverables identified in TAs were deferred into later tasks until finally no time or funds remained.

2.1.4 Arms-length relationship of Contractor weakened.

It was expected that the Project Authority and Special Advisor would abide by the contracting policies and regulations regarding the selection of resources contracted for under this service arrangement.

The Project Authority approved TA#5 for the services of two senior consultants who would provide strategic advice to the Special Advisor. The estimated cost of this task was $1.44 million (GST not included). TA#5 was reduced in time and cost, and a second TA #9 was created. As of July 2006, the expenditures for the two tasks were $457K.

In March 2006, it was suggested that the Contractor consider the services of two named individuals. In February, the Special Advisor had already made initial contact with one of those individuals and asked the Project Authority (PA#2) to work out the details.

After reviewing the resumes and interviewing the individuals, the Contractor prepared a TA document and proposed the use of the recommended individuals. The Contractor has informed the PWGSC Chief Risk Officer that he was not instructed to hire the suggested individuals and only used them after he was fully satisfied with their professional capabilities and had ensured their rates were within accepted industry norms. Within the TA document, the Contractor chose to communicate clearly and forthrightly that the consultants had a pre-existing relationship to persons in the department.

Under the terms of the contract, the Contractor could hire any resource that met the experience requirement. Notwithstanding this, suggesting names to the Contractor is not the normal departmental practice. Such actions may give the perception of influencing the selection of resources hired under this type of service contract.

2.1.5 Mandatory contract Project Planning documents were not fully developed.

It was expected that the Project Office would have ensured that the mandatory project planning documents, outlined in the contract, were developed to support the effective transformation of procurement over the next four years.

The Project Authority (PA #1) prepared a Statement of Work for the solicitation and contract, which proposed that the Contractor prepare key project planning documents within the first three-months of the contract. These documents would have assisted in managing the procurement transformation project over the 4-year life span of the service contract.

There were four mandatory and key project planning deliverables in this contract: develop a Relationship Governance Model Agreement; provide Base-Line Analysis report; develop a 4-year Project Management Master Schedule and Budget; and provide an annual procurement transformation plan for Phase 2. Within the deliverables there were other sub-activities or planning factors. As an example, the 4-year Project Management Master Schedule and Budget stated that the Contractor must provide project and contractor performance measures, risk management control mechanisms, communication and change management strategies, and critical success factors.

The scope for TA#1 (December 2005) included the development of key planning documents, as well as critical work on strategies and action plans for spend analysis, major commodities, client engagement, and performance management. One of the key planning documents the Contractor was tasked to undertake was the development of a 4-year Project Plan and Budget. By February 2006, a detailed 3-month work plan was submitted to the Project Office and by March 30th a high-level, 4-year work plan was submitted. Although certain elements of project planning factors were found to be incorporated into commodity and client engagement strategies, these 3-month and 4-year work plans did not address the project planning factors outlined in the contract; such as: performance, risk, communication, change, and success. These planning factors were critical for a procurement transformation project of this size, scope, and importance to the government.

As observed in section 2.1.1 this was a dynamic environment in which changes to scope were agreed to but not documented. No TA or contract amendment was created that decreased or removed the project planning requirements outlined in the contract, such as a 4-year Project Plan and Budget.

This audit concluded that the Project Office lacked a key instrument - project planning documents, which were mandatory deliverables under this contract and critical to the successful delivery of the procurement transformation project over a multi-year period.

2.1.6 Failure to manage the level of assigned contract resources impacted financial burn-rate.

It was expected that the Project Authority(s) or Special Advisor would validate and approve additional numbers and levels of resources proposed by the Contractor. There should have been documentation of resumes or of discussions regarding the skills, experience, and security required by the resources to undertake the work. This approval would be consistent with the levels estimated in the solicitation or supported by financial analysis of a change in levels and the impact of the change on contract expenditures.

The contract identified three personnel categories. The resources proposed by the Contractor had to meet the minimum experience requirements stated in the contract: 'Project Manager' - five (5) years as project manager/leader; 'Senior' - five (5) years of experience gained in the field of work; and 'Intermediate' - two (2) years of experience gained in the field of work.

The Contractor selected resources from a pool of employees within its parent company. Within the Contractor's pool of resources there were approximately 850 persons in North America and 2000 worldwide. The Contractor looked for resources with the right skills and experience, as well as their availability. North America was the primary pool, but some individual resources were sourced from other countries.

Project Authority(s) stated that they or the Special Advisor met with the Contractor to discuss the resources. There are resumes on file of some of the thirty-six resources employed in TA#1. However, there were ninety (90) full-time or part-time individuals who worked on this contract at various times. As well, there was minimal documentation on the approval of the level, skills, or security for any of the resources charged to the contract.

The solicitation for this contract estimated the percentage of total level of effort attributable to each of the three personnel categories at 10/45/45. This percentage was used as a weight to evaluate the bids and to determine the winning bid. This meant that, at the time of solicitation, the Project Authority estimated that, for a 4-year contract requirement, 10% of the contract resources would be expensed as a Project Manager, 45% of the resources would be expensed at a Senior level, and 45% of the resources would be expensed at a Intermediate level.

In fact, due to the acceleration and critical nature of the task activities, many more project management and senior level staff was actually employed than had been originally estimated in the solicitation. The percentage of estimated resources authorized in the TA varied. (E.g. TA#1 - 26/56/17; TA#3 - 20/73/17; TA#8 - 21/54/25) The total actual percentage of level of effort attributable to each of the three personnel categories charged to the contract was 25/55/20. The difference in per-diem rate between the lowest and highest category is a factor of 4 times. Thus the total costs of the contract escalated more quickly then planned.

The Project Authority(s) and Special Advisor failed to demonstrate adequate project management by documenting their approval, controlling the resources proposed by the Contractor, and modifying the rate of contract expenditures. As a result the financial burn-rate of the contract was approximately 4 times as great as originally envisioned in the solicitation, and the achievement of all the benefits originally outlined in the contract was not possible.

2.1.7 Limited tracking of contract performance.

It was expected that the Project Authority(s) would have a system in place to track and report regularly on the progress and financial burn-rate of the TAs. As a minimum, the system should monitor planned compared to actual work completed, and identify critical barriers or risks to the achievement of the task.

Project Authority (PA#1) tasked the Contractor, within the first month, to develop a means of reporting to the Project Office on the status of procurement improvements. The Contractor used existing information to create status or "Flash" reports. The Contractor updated the reports weekly, based on input by the contract resources and by the Stream Lead for each category of services or activity stream. The reports identified, by each activity stream: the names of the individuals, both contract and departmental doing the work; the percentage completion of deliverables; work completed for the current week and work arising for the next week; risks or barriers; and a 'red, yellow, or green' progress indicator. Each week the Project Office met with the contract resources and the Stream Leads to discuss the validity of information in the report. Extensive discussions were held with the Steering Committee on the status of the work.

The audit found that the weekly status reports tracked the progress of activities and some selected deliverables from the TA, but there were no Departmental mechanisms put in place for the explicit tracking of the performance of the Contractor. Activities were a joint effort of PWGSC employees and the Contractor's resources. The reports provided an overview of the procurement transformation activities going on in the department, and tracked the progress of each overall stream or activity. While this may have been an appropriate approach to activity-based reporting, additional mechanisms should have been put in place for reporting on purely contract-related activities.

The Contractor modified an existing system for tracking the tasks in TA#1 using Microsoft Project. The initial Project Authority (PA#1) used this for tracking of task activities and reporting to Project Office. However, there was no evidence in the latter tasks that the system continued to be used.

The Stream Leads worked closely with the contract resources. The Stream Leads were responsible for validating the hours that contract resources worked in a month, before the progress payment would be paid. Although the Stream Leads could verify the reasonableness of the hours, they were not able to track the Contractor's performance because they were not fully aware of the changes made to the proposed and approved deliverables of each TA. The Stream Leads had knowledge of their specific stream, but not of the total services approved in each TA. They indicated to the auditor that they felt that they were not expected to track the Contractor's overall deliverables.

In late March 2006, after amending the contract for TA#3, the Contracting Authority told the Project Authority that actions were needed; as there were limited funds remaining in the approved contract value. Although, the Contractor provided weekly financial expenditures to the Project Authority, it was not until late April 2006 that the Project Authority (PA#2) raised concerns to the Special Advisor that the burn-rate of contract dollars was too high. The estimated time period for TA#3 was March to November 2006, but at the current burn-rate the estimated costs would have been totally expended by late June. To resolve this would have meant decreasing the ongoing tasks and not undertaking new tasks.

The Project Authority's tracking of task deliverables and timely recognition of the financial burn-rate of the contract was limited. The impact on the project was that the Project Office was unable to maintain control over a rapidly evolving implementation schedule.

2.1.8 Changes in scope limited the benefit of innovative procurement strategies.

It was expected that changes to the scope of the work would have been analyzed by the Project Office to determine their impact on the contract in risk, cost, and time.

The pace of the contract work was accelerated to develop and implement procurement tools that would assist the department in meeting the savings target for 2007-2008. Given that it normally takes 8-12 months to put in place a new procurement tool, such as a Standing offer or Supply Arrangement, the procurement tools had to be developed and implemented in a more rapid rate.

Initially, the approach outlined in the Sourcing Strategy Stream of TA#3, tasked the Contractor to work with PWGSC to develop and implement enhanced procurement processes and instruments in 3 groups of the 4 commodity areas, based on the seven steps of the Contractor's Strategic Sourcing Methodology. The seven steps were: develop Category Profile; generate Supplier Portfolio; develop Sourcing Strategy; select Implementation Path; negotiate and select Suppliers; operationalize New Agreements; and sustain the Results.

This Sourcing Strategy task was revised and expanded to 3 groups of 15 commodity areas, and knowledge sharing with a 4th group of 8 commodity areas.

By the end of TA#8 (October 2006), the Contractor was able to provide, on average, development and knowledge on commodities up to step 4 and important parts of step 5. This meant that the department gained broader knowledge and experience on supply market analysis and sourcing strategies, but failed to gain extensive experience in negotiation strategies and ongoing performance measurement. These latter strategies in steps 5-7 are ones in which PWGSC also required expertise and support from the Contractor.

In the first three steps of the strategic sourcing methodology the Contractor provided the department with: commodity analysis on demand and supply; identification of the major suppliers; and recognition of future trends in the industry. This assisted the department in examining its transactional-based procurement and in moving toward a strategic-based procurement. Examples of the strategic-based procurements considered were: buying fuel at the low seasons and storing it rather than only buying during the peak and costly seasons; providing a guarantee or commitment to a supplier that a number of chairs would be bought in a given year, which permits the supplier to work through non-peak times and warehouse the stock until required; and recognizing the trends in certain type of printers or cellular phones and adjusting the supply arrangements to increase this type. The steps through five to seven were to assist the department in establishing a solicitation document, negotiating with bidders or suppliers, and then monitoring the performance of the supplier.

A similar increase in contract scope occurred in the Client Engagement Stream of TA#3. Initially the approach of the TA focused on 6 client departments for completion of procurement assessment workshops and development of departmental specific procurement plans. However, the Acquisitions Branch's 2006-2007 to 2008-2009 Business Plan, stated: "Acquisitions will complete, by spring 2006, 22 client department procurement strategies to be incorporated into interdepartmental service level agreements". The revisions to the Client Engagement Stream targeted 24 departments. By the end of September 2006, through the joint effort of the Contractor and Department, 8 major departments were engaged in the process, 5 departmental procurement strategies completed, and 4 Service Level Agreements drafted.

The expansion into the Real Property Branch was yet another change in the contract scope. The departmental 'Way Forward' was focused on generating efficiencies in three key areas - purchasing, real property and information technology. This contract was to provide professional services in support of the Acquisitions Branch. But TA #6 and #7, at a total cost of $893 thousand (no GST), expanded the scope into Real Property Branch. At the time of the audit report, the department had not yet actioned the recommendations of the business case and options for improvements and savings in Real Property Branch.

While there is no question that the Project Authority and Steering Committee had the authority to modify the contract scope, the audit expected that there would be clear analysis justifying the need to use this contract for the Real Property Branch work rather than using a separate competitive contract. Furthermore, there is no documentation of the rationale for the decision taken by the Special Advisor to approve the tasking of the Contractor's services for work outside Acquisitions Branch.

As a result of committing the funds for the expanded scope, the free-balance of the contract and the availability of contract resources for Acquisitions Branch were reduced. Even though the money was later refunded from Real Property Branch to the Responsibility Centre Manager for Acquisitions Business Transformation there was limited means of using the transferred funds for sourcing strategies. When the financial burn-rate for critical sourcing strategies in Acquisitions Branch was high and the achievements were critical, it was not clear why the Project Office chose to expend limited contract resources in Real Property Branch.

The audit concluded that the Project Authority and Steering Committee did not effectively manage the changes in scope such as increasing the number of commodity areas and client departments, and expanding the work to Real Property Branch. As a result, the department benefited less than it could have from the Contractor's depth of knowledge and experience of innovative sourcing strategies for procurement.

2.1.9 Insufficient PWGSC Resources to facilitate knowledge exchange.

The audit expected that the Project Office would ensure there were sufficient and capable internal resources dedicated to support and assist the Contractor in its delivery of procurement transformation and to ensure a smooth transfer of knowledge.

The contract stated that the Project Office, supported by the Contractor on an 'as and when requested' basis, shall pursue knowledge transfer as a primary objective of the Acquisitions implementation project. A major service that the Contractor was to provide over 4 years was the transfer of required skills in procurement transformation to government staff. For this transfer to occur, PWGSC procurement officers had to work directly with the contract resources. The Contractor was to provide procurement consultancy services in the broad category of Knowledge Transfer and Implementation Support. The work was intended as a key support to the continuity of improvements and the realization of savings from procurement.

In preparing the requirements for TA documents and the weekly status sheets, the Contractor noted that a critical success factor and a risk for procurement transformation was the availability of Acquisitions Branch personnel.

Prior to January 2006, an organizational chart of those involved in the project indicated 23 dedicated department resources within the activity streams. The plans were to increase this number by 70 persons. This increase did not include the resources from the Electronic Acquisition Business Directorate and the Standing Offer Management Office, both of which supported the procurement transformation.

After January 2006, a proposed organizational chart of the project indicated a mix of departmental dedicated resources and contracting resources within each activity stream. The chart also indicated plans to increase the dedicated department resources to over 100 persons, and included an additional 40 resources from multiple service contracts. Within the activity streams, the percentage of departmental to contract resources ranged from 14% to 75%. The proposed organization was presented but not approved and there was limited implementation particularly with reference to the staffing of departmental resources.

As previously indicated, during the acceleration period the Contractor was able to obtain additional resources from the pool of persons within its affiliates and sub-contractors. The numbers of persons employed under the Contractor rose from 35 persons in TA#1 to 70 persons in TA#3. PWGSC, however, was unable to significantly increase its number of dedicated resources.

The audit observed that the Contractor took on many tasks, as described in the TAs, of an administrative or project management nature, which are normally the responsibility of the Project Office. Examples of such tasks are: creating the requirements for a new Task; tracking performance of the activity streams; reporting performance through updates to weekly status reports; providing weekly financial reports of usage; creating communication strategies; creating presentations; and preparing business cases and submission documents.

Another impact of PWGSC resource limitations was that the Contractor took on the role of monitoring itself. Although one of the Contractor's activities was project management, it was expected that the overall role and responsibility would rest with the Project Office. The Contractor identified the task requirements, monitored their own performance, reported on the status of teamwork, and communicated any resulting savings.

By July 2006, the weekly status progress tracking indicators of the activity streams were moved from 'green' to 'yellow' or 'red'. In the reports, specific references to the risk of limited PWGSC resources were: "Knowledge Sharing team members from PWGSC are not yet identified, causing these work streams to be delayed"; "Contractor is expected to roll-off. Current team cannot support the Quick Wins and longer-term process improvements"; "Upcoming cessation of Contractor support will impact schedule of remaining departments". These types of statements by the Stream Leads of the activity streams were a basic indicator that the Contractor was the major driver of procurement transformation activities and that knowledge transfer was being impacted by a lack of PWGSC resources.

The Contractor was required to provide research, advice, information, and strategies, which were to be incorporated into a departmental document. (Client workshop, Client Service Level Agreement, or Commodity strategy). The Contractor provided this service for most activity Streams, conducting knowledge transfers as work progressed with assigned PWGSC staff and conducting formal training sessions. The Contractor provided extensive discs and documentation of the work that was undertaken during the period of the contract.

In summary, the Project Office encountered challenges in ensuring that there were sufficient, experienced, and dedicated PWGSC staff to work in the activity streams; particularly in strategic sourcing and client engagement. This resource challenge increased the administrative activities of contract resources and limited the potential for knowledge exchange in some activities.

2.1.10 No assurance that status reports provided complete and accurate project information.

The audit expected that those responsible for procurement transformation would provide authorized, complete, and accurate project information to departmental Senior Management. Such project information would assist Senior Management in making critical decisions, in analyzing the impact of external changes and developments, and in generating new knowledge and future directions.

The audit found that a number of venues were used to provide procurement transformation information to departmental Senior Management. These include: monthly Business Transformation Committee meetings; weekly status reports of Spend Analysis and discussions with the Strategic Transformation Branch; weekly status reports or 'flash reports'; numerous ad-hoc presentations and documents on specific commodities or activities streams; and meetings on a daily to weekly basis.

In review of the weekly status reports it was noted that the summary of weekly status reports occasionally remained 'green' or on schedule, even when the details within the report indicated risks and/or delays in achieving the deliverables. Interviews with Stream Leads indicated a lack of consistent understanding of the progress indicators. Some felt as long as the issues were being dealt with, it stayed green; others that if the baseline of expected completion was adjusted then after a week of yellow it should return to green; and others believed if there was an indicator of risk, the progress should be 'yellow or red'. For example, one weekly status report on the Client Engagement Stream indicated 'green' even though the task for assessing the procurement function for six (6) focus departments was not completed and there was only one week left in the TA period.

Management indicated that a review of the documentation couldn't fully gauge the level of Senior Management's engagement in the status of the procurement transformation project. The weekly status reports served as a management tool for the Stream Leads, procurement teams, and Contractor to validate the status and progress of activities to date. The primary purpose of the status reports was to foster discussion, review strategies, and receive guidance. Although no records were taken, the Special Advisor and members of the Steering Committee met on a daily to weekly basis to discuss the details of the status and progress of work to date. When required, they invited the Stream Leads and/or Contractor to discuss the progress.

As a result of a lack of records of decisions on oral interactions and decision points, the audit could not conclude if Senior Management had complete and accurate project information that would assist them in risk mitigation or modification of the course of actions.

Exhibit 2 Case Study on pilot of new procurement sourcing techniques
Evolution of eAuctions as a tool for PWGSC to achieve savings

This case study, illustrates a failure of the Project Office to think through the process of procurement reform and to consult with key stakeholders and deal with the risks of regarding new procurement strategies. This failure caused expenditure of time and contract dollars, as well as risks for the Department. EAuctions were one of several innovative sourcing strategies that were considered and tried in this contract, as part of the transformation of government procurement. Although this case discusses only eAuctions, the findings could apply to other sourcing strategies such as limiting suppliers, negotiations, volume commitments, and alternative supply arrangements.

EAuctions or Reverse auctions are live, on-line competitions where suppliers bid against one another to provide the lowest price. It is most effective when used for goods where each unit has exactly the same quality and inputs of labour and materials. Governments and Private Industry have used eAuctions to achieve savings. For example, the United Kingdom government saved about 20% buying IT hardware. However, critics of eAuctions argue that the savings are illusionary, as bidders submit artificially inflated prices knowing there will be an opportunity later to resubmit a more competitive price. They also argue that eAuctions create a greater likelihood of supplier disputes, bad faith, and increased risks.

In August 2004, the suppliers in response to the PWGSC Letter of Interest for this procurement transformation contract identified eAuctions as one sourcing strategy, among others, to achieve procurement savings. Again, in October 2005, bidders who responded to the PWGSC Request for Proposals identified eAuctions.

In November 2005, the contract was awarded, and in January 2006, the Contractor began work on sourcing strategies. In March 2006, the Steering Committee met with the Contractor to discuss a pilot test of eAuctions in five upcoming solicitations. Two of those solicitations were buying printers, and temporary help services. The Contractor recommended, and the Project Office decided to test eAuctions in commodities that were most familiar with the concept, such as buying printers.

In April 2006, the Special Advisor authorized the Contractor's staff to travel to Vancouver to finalize a number of procurement strategies and to brief Regional staff on the recommended use of eAuctions for the upcoming Request for Standing Offers (RFSOs) on printers. Shortly thereafter, the Contractor worked with the Regional staff to create a plain-language template for the RFSO.

At that time, Acquisitions procurement staff at Headquarters and some Regional staff raised concerns regarding the use of procurement tools such as eAuctions. Furthermore, up to that point, no one within the Project Office had sought legal advice on the government's right to use such approaches or to revise standard template documents. The Project Office believed there were no legal or policy barriers in place, and that eAuctions could be conducted within the boundaries of current policies and legislation.

[ * ]

[ * ]

[ * ]

At the bidders conference on June 12, 2006, suppliers voiced concerns regarding the new process, which eventually reached the media and politicians. On June 14, the RFSO was cancelled. In July 2006, the PWGSC Minister announced that eAuctions were off the table entirely and that Industry and supplier consultations would be held before proceeding further with certain elements of procurement reform. Later, the Project Office reiterated that eAuctions were never a central strategy for achieving the Government's savings targets.

This was the first of the 'Way Forward' solicitations leading to procurement transformation in the government. The case of eAuctions illustrated a failure by the Project Office to think through the process of procurement reform and to consult with key stakeholders and deal with the risks regarding new procurement strategies. As a result, the Contractor and departmental staff spent time and contract dollars on modifying RFSO templates and dealing with suppliers and media, the RFSO was delayed, and PWGSC's reputation as the contracting expert and leader in procurement transformation was weakened.

2.2 Contract Management

2.2.1 Contracting Authority proceeded with some amendments before receipt of requisition.

Contrary to the intent of Supply Manual section 6A.040, the contract value was amended without the appropriate Financial Administrative Act (FAA) section 32 signature of the Project Authority. The Contracting Authority believed that sufficient compensatory documents, such as an exchange of e-mails and TAs, existed to justify the approval of a drawdown amendment without a formal signed requisition. Although formal section 32 authority was subsequently provided, Part III of the FAA indicates that section 32 certification is mandatory, and as a result, the responsibility to receive appropriate certification was not met at the time of the contract amendments.

2.2.2 Contract work started before Task Authorization signed.

Contrary to the intent of the Supply Manual (section 11.001) and the terms of the contract, the Project Authority permitted three tasks to start before a formal TA was signed.

The Project Authority is the primary entity responsible for the day-to-day receipt of performance or goods. Sometimes, for reasons of urgency or importance, a Project Authority has asked a supplier to start work before a contract was in place, or requested terms in work/scope/payment that were not within an existing contract. If justification was provided, PWGSC Contracting Authority would address these situations through a confirming order or a contract waiver; after seeking legal advice and obtaining approval at the Director level.

In this instance, under the contract term and condition for basis of payment, the Contractor was not to perform or be paid for work without the prior written approval of the Contracting Authority. TAs were the normal method of providing such written approval. Contract amendments were initiated to modify the contract value or scope based on estimated costs and level of effort outlined in a TA. In review of the invoices for costs relating to existing TAs, the Contracting Authority noted that work was started prior to written approval and requested functional oversight advice. Since the contract identified that work was outlined in TAs it was agreed to revise the Task document. Of the nine TAs, three (#3, #7, #8) were revised to include a waiver. Payment of work started prior to written approval breached the conditions of the contract and normal PWGSC practices.

2.3: Financial Management

2.3.1 Weakness in process to approve progress payments.

The audit expected that the Project Authority would validate invoices and authorize payments in accordance with the FAA section 34 financial delegations and government/departmental policies.

According to FAA section 34, no payment is to be made unless an authorized person certifies: that the work has been performed, the goods supplied or the service rendered, and that the price charged is according to contract, or if not specified by the contract, is reasonable; that the payment is according to the contract, or where made in advance of verification, that the claim for payment is reasonable; and, that the payee is eligible for or entitled to the payment.

One of the Project Authority's roles and responsibilities was to validate the invoices prior to signing section 34. To comply with the FAA, the Project Authority was to certify that the service was rendered, and the price charged and payment made was according to the contract.

In this contract, the Project Authority received the invoices by-hand or e-mail. In order to confirm the resources the Project Authority tasked the Stream Leads to review their applicable section of the invoice. They were to review the named resources and validate that such persons were employed in the delivery of their activity Stream, and to review the hours charged by said persons. Their acceptance of the resources and hours was evident by their initials on the invoice documentation or by e-mail. Once confirmation and acceptance was received from the Stream Leads, the Project Authority would sign section 34 for payment of the invoice.

The Stream Leads could not verify all the actual hours worked since: most TAs approved the Contract resources to work 11 hours/day; some of the resources worked off-site; and some of the resources worked on the week-ends. Instead, Stream Leads initialled that the hours charged were reasonable when compared to the work completed. The Contractor had provided evidence that 97% of the hours charged was for work completed during the Monday to Friday timeframe.

The Project Office placed greater reliance on the Contractor to maintain accurate and appropriate time recording systems since the actual hours could not be verified by PWGSC. Later, these were subject to a separate cost audit to verify the time charged. The cost audit report has not yet been completed.

A further constraint in reviewing the invoices was that, while the Stream Leads were aware of what needed to be accomplished in their specific activity Streams they were not always aware of the rapid changes made to the proposed and approved TAs. As a result, they approved resources and hours for work in advance of contract amendments. For example, the work of cleaning and validating the financial information for the Spend Cube was a critical and integral component of the Client and Commodity Streams started in TA#1. The Stream Lead for Information Management indicated that he had requested that the work on the Spend Cube be clearly identified in subsequent task documents. In fact, the draft TA #4 does outline those requirements. However, with rapid changes in the task activities and financial constraints the draft TA#4 was not approved. As a result, the work for resources in the installation of the servers for the Spend Cube was completed in May 2006 and $3,800 was approved for payment on July 14, 2006. On the same date, TA#8 authorized work on the Spend Cube and incorporated it into the contract. However, TA#8 was later revised to include a waiver that provided approval and authority for the work that started earlier.

As a result, in this particular instance, the Project Authority approved resources and hours for work that were not authorized until later in the contract, and therefore was not compliant with the FAA section 34.

2.3.2 Poor financial practices in committing funds not yet transferred to the Responsibility Centre.

The audit expected that the Project Authority would authorize the increase in the value of the contract through appropriately signed and timely requisitions.

The Project Authority must sign requisitions to modify the value of a contract under FAA section 32. In advising the Contracting Authority to increase the value of the contract, the Project Authority is certifying that there are sufficient unencumbered balances available with their Responsibility Centre budget. In this instance, requisitions were signed against the Acquisitions Business Transformation budget for which funds had not yet been transferred from the Assistant Deputy Minister's or Deputy Minister's budget. Although the risk was small, this practice weakened financial controls.

2.3.3 Commitments were not recorded in CDFS as required by PWGSC policy.

The audit expected that the Project Authority would commit funds for the work arising from the contract, in accordance with the FAA section 32 financial delegations and government/departmental policies. According to FAA section 32, no contract is to be entered into unless there is a sufficient unencumbered balance to discharge any debt that will be incurred during that fiscal year.

One of the Project Authority's roles and responsibilities was to manage its expenditures. The project was assigned a Responsibility Centre number, with a reference level, budget and monthly financial reporting.

The departmental policy on Commitment Control requires that Responsibility Centre managers recognize and record a commitment in the Common Departmental Financial System for each individual contract or legal obligation of $5K or over. This ensures that sufficient balances are available in their budget to discharge all obligations and that Responsibility Centre managers are appropriately exercising their spending authority. The policy requires that a periodic review (at least monthly) of the commitment information be carried out by the Responsibility Centre manager.

In this instance, the financial monthly Free-Balance budget reports did not record a commitment equivalent to the multiple TAs signed by the Project Authority.

The lack of reporting of million-dollar commitments limited the ability of senior departmental management to administer the department's availability of funds (free-balance) that could be used to address changing priorities.

2.3.4 Integrity of financial data impacted by inaccurate recording.

It was expected that Project Authority's activities with regard to payments would be in accordance with government and departmental policies.

The Project Authority's responsibility was to ensure that an accurate receipt date was stamped on the invoice, particularly if the invoice was presented in person or sent directly to their attention. Once the invoice charges are validated, the Project Authority authorizes the invoice for payment by signing a section 34 stamp, and then provides the invoice to an administrative clerk. The clerk is responsible to enter the payment into the Common Departmental Financial System on behalf of Acquisitions Branch. Batches of payments in the Common Departmental Financial System are then approved by Finance Branch clerks and processed for release of cheques.

The Common Departmental Financial System has certain checks and controls built into the system that ensure accurate and complete payment information. As well, there are some controls that address government regulations and policies. In accordance with the government's cash management strategy, as defined in the Treasury Board Policy on Payment Requisitioning and Payment on Due Date, the Common Departmental Financial System defaults to a standard payment term of 30 days from stamped receipt date of an invoice or acceptance of the goods or service, whichever is later. This ensures that payments are cost-effective and in compliance with the government policy on payments and interest charges.

In this instance, Project Authority(s) failed to stamp a receipt date on the day they received an invoice, presented in person. This receipt date was a marker in which the Crown had 30 days to review the invoice and to initiate payment procedures. Instead Project Authority(s) validated the information and then provided the invoice to an administrative clerk, who stamped the receipt date as the date the invoice was received in the financial section. As an example, an invoice for $1,299,090 was dated February 17, 2006, presented to Project Authority, in person, on February 20, 2006, stamped as received on March 2, 2006, paid March 9, 2006, and deposited in the Contractor's account on March 21, 2006.

The impact is that the Common Departmental Financial System recorded erroneously that the system default was overridden and that two of the eight contract progress payments were issued in advance of the 30-day standard payment term.

The audit noted evidence that Project Authority asked for and received the Contractor's agreement to waive their right to interest payments. According to the Treasury Board policy on Payment on due Date, the Contractor is to receive interest from the Crown for payments that are not received by the first day after 30 days, from receipt of an invoice or acceptance of the goods or service, whichever is later.

Project Authority(s) were not compliant with government/department policies regarding payments, which ensure that the Crown met its contractual obligation of fair and timely payments.

2.3.5 Post-Payment Verification does not mitigate early payments.

The audit expected that the departmental Finance Branch would have a system in place that identified any payments made prior to the 30-day standard payment term and corrected any future actions.

The Treasury Board's Account Verification Policy allows departments to use a sampling methodology, in order to provide assurance that accounts for payment and settlement are verified in a cost-effective and efficient manner while maintaining the required level of control. The departmental sampling methodology classifies transactions into two categories - those requiring financial verification prior to payment, and those selected post-payment and based on three risk groups - High/Medium/Low. High-risk transactions include those transactions that are $1 million and over. On a quarterly basis, Accounting and Controls Directorate selects a sample of post-payment transactions in order to provide assurance of the adequacy of the section 34 account verification. The established procedures require that the selected transactions are examined and that evidence obtained demonstrates that: the work had been performed, the goods supplied or the services rendered or in the case of other payments, the payee was entitled to, or eligible for the payment; and the relevant contract or agreement terms and conditions had been met, including price, quantity and quality. Upon identification of a critical error in a selected transaction, a memorandum is sent directly to the responsible section 34 officer, providing them with an opportunity to correct the error and ensure it is not repeated.

In this instance, a Quality Assurance review was conducted on the payment issued in March 2006. Although the early payment was identified as an error during this Quality Assurance review, this type of error fell into the category of a non-critical error and therefore, was not communicated to the input clerk's operational unit.

As a result of identifying an early payment as a non-critical error, the opportunity to correct any future errors was limited.

Section 3 Procurement Transformation Issues

The Audit observed that important progress in procurement transformation had occurred in the period of January to September 2006, through the joint efforts of the Contractor and PWGSC.

3.1 Significant progress did occur during the eight-ten month period.

By 2005, it was a departmental priority to achieve the savings commitment of $2.5 billion over five years, and to transform procurement in order to deliver services smarter, faster, and at reduced cost. Earlier contracted research had identified the potential for savings, if the Government consolidated its buying power and leveraged the best prices. To achieve this, Acquisitions Branch needed better information on what and how the Government buys, support from Client departments and Central Agencies, improved tools, and new strategies for procurement. Given the scheduled savings were due to start accruing in 2005-06, it was important that actions started immediately. PWGSC Management realized that it was critical for PWGSC to have access to specialists in procurement transformation. Since the expertise was not readily available in-house, it was decided to obtain it under contract from the private sector.

The audit compared the status of the procurement transformation in the department prior to January 2006 and what progress had been achieved by September 2006. As a result of the Contractor's support and advice to the Project Office, a number of significant elements of procurement transformation did occur during that period. Although it is too early to assess the overall results of the procurement transformation, the department received benefits and made important progress on procurement reform and government wide savings. (Exhibit 03)

As well, client departments benefited from the analysis of procurement spending in examining efficiencies in their operations and administration. Lessons were learnt from the consideration of new procurement sourcing techniques, such as reducing the number of suppliers, negotiations and eAuctions. (Case study on eAuctions - Exhibit 2) Sufficient and skilled resources have not yet been deployed within the department to flesh out and implement these new tools and ideas.

3.2 Extensive consultation increased engagement and understanding of Stakeholders.

There were government expectations that PWGSC would provide complete, accurate, and timely information to stakeholders regarding the improvements to the federal government procurement function. Information on the 'Way Forward' or procurement transformation should have included: identification of potential savings; legislative, policy, legal or industry barriers; and new innovations or directions in commodity management or procurement strategies.

Based on interviews and reviewed documentation, the audit found evidence of extensive information being provided on the 'Way Forward's' potential savings during the period of January to September 2006. Through presentations, consultations, and open forums, PWGSC communicated with federal Client departments and Central Agencies on the new approach to transform the procurement function and to generate savings. Information included the identification of potential savings in commodity procurements and operations. As well, it identified the use of alternative or new procurement strategies for future savings.

As a result of the extensive consultation, there was increased engagement and understanding by Client departments and Central Agencies regarding the improvements to the federal government procurement.

3.3 PWGSC's ability to accurately determine savings opportunities was hindered by lack of consistent Government financial data.

In order for the Project Office and Contractor to accurately determine the opportunities for savings from procurement it was necessary for departments and agencies to supply their financial transaction data to PWGSC. To leverage the information for procurement decisions, it was critical that the data extracted from the department's Accounts Payable, Acquisition cards and Purchasing systems be complete, accurate, detailed and timely. The audit found that of the 130 departments and agencies that form the population, the 2004/05 financial data from 51 departments was analyzed and compiled into a unified spending database or 'spend cube'. These 12 million transactions represented over 83% of total government spending on procurement.

Interviews and documents indicated that support was not readily forthcoming from departments and agencies. This included client departments requesting exclusions from the mandatory use of Standing offers, and a willingness to share only high-level spend information rather than providing PWGSC with detailed spend information.

The audit noted a lack of consistent government-wide financial systems or procedures had hindered the extraction and analysis of the data. As a result, the Project Office departmental Lead and team for Information Management and the Contractor spent a significant amount of time and resources in validating the data. The following are examples of government data issues:

• The federal government does not use one central financial system. Extraction from multiple financial systems was time consuming and created inconsistencies.

• The use of a single supplier number, such as the Procurement Business Number, was inconsistently used in financial transactions. As a result the matching of supplier names increased the time to validate the data.

• The use of economic codes rather than a Goods and Services Identification Number provided less detailed information and increased the time to analyze the data.

• There was a high incidence of mis-categorization of commodities. This resulted in more time to validate the information and modify the coding.

• Not all suppliers identified in the database were registered in Supply Registry Information system. Those registered in the on-line free registry were easier to match with their commodity speciality and to provide confirmation of the analysis.

• Many of the agencies and departments multiple internal systems were developed independently of one another and consequently do not link their information on purchases and suppliers. This increased time and analysis since data had to be extracted from each separate system, and then cross-related.

A key aspect of procurement reform was the ability to identify what the government buys, where and when. To support advanced reporting and ongoing strategic commodity sourcing, PWGSC will continue to refresh financial and commodity information every quarter. Therefore, the lack of consistent practices in the recording of financial information on procurement spending across the federal government's departments and agencies will demand increased time and cost for PWGSC to retrieve and analyze the information for government savings opportunities.

Exhibit 3 Highlights of Reported Procurement Transformation Achievements January to October 2006

Savings:

• For fiscal year 2005/06, the Department has reported realized savings in excess of $81 million.

Information Management:

• PWGSC now manages a sophisticated database (Spend Cube) located on a server that stores and integrates government procurement information. The existing baseline is the 2004-05 financial data on 12 million transactions from 51 departments. The database can present the financial data by clients, suppliers, commodities, and expenditures. The Contractor defined the logic structure for the database and trained 10 departmental analysts to manage future extracts of information. Previously, only 10% of the IT business requirements for analysis had been defined, and about 75% of Government spending had been analyzed for 2002-03, and that had been only after extensive manual intervention.

• Training on information management and spend cube tools was provided to PWGSC staff, and consultation of the benefits of the spend cube with other government departmental procurement staff.

Client Engagement:

• Client Engagement frameworks assist PWGSC in its future engagements of clients. These frameworks included procurement savings opportunity reports, business cases, client procurement strategies, and service level agreements. The Contractor leveraged its knowledge of best practices in procurement to support PWGSC in evaluation of departmental processes, spending, and procurement strategies.

• Training was provided to more than 60 PWGSC staff in client consultation. As well, 24 other government departmental workshops were completed.

Strategic Sourcing:

• The Project Office, supported by the Contractor, developed new Requests for Standing Offers, which covered over $4.9 billion in annual spending and targeted total savings of $1.4 - $1.9 billion over five years, as well as other benefits.

• The Project Office will be building on the task led by the Contractor to address the 4th Quarter 2005/06 spending by repeating the process in 2006/07. The Project Office reported that the 2005/06 agreement of over 30 client departments to pool their purchases, achieved $3.8 million in volume discount savings and identified $5.3 million more in incremental savings.

• Training on strategic sourcing was provided to over 100 participants.

Performance Management:

• PWGSC used the seven robust and consistent rules and methodologies, developed by the Project Office and the Contractor to track and report on savings for the current and future years.

• Over 90 individual contract authorities were trained in calculating savings.

• PWGSC was addressing the bottlenecks and structural improvements that were identified by the Project Office and Contractor in order to increase efficiency, reduce cycle times for high-value contracts, reduce requisition fulfillment cycle times and improve management's ability to control risk.

Other Improvements:

• The Contractor supported Real Property Service Branch in the development of an overall procurement framework, strategy, and business case for providing property management services, project management services, professional and technical services, and options for an Integrated Workplace Management system.

Conclusions

The audit reviewed the actions of the department to ensure compliance with contracting and financial policies and practices, and to validate that the contract was managed in accordance with the best practices of project management. The main findings were:

Project Management:

• Procurement transformation was more than just achieving a savings target of $2.5 billion. And yet, the Project Office's primary focus was on a single aspect - savings targets. There was no documentation on consideration of other options or alternatives, or the examination of risks and cost-benefits.

• Contracting for an external service provider brought two advantages: a high level of skills and new ideas that the Department did not have within its portfolio of in-house resources; and the capacity and staff to undertake work immediately.

• Task Authorizations were not approved as a contractual document to regulate the work and expenditures of a contract for services on an 'as and when requested basis'. Task activities were: a joint responsibility between the Contractor and PWGSC; ambitious in scope within the agreed time period; and modified/deleted/deferred without documentation, as other significant work arose.

• The Project Office transferred some of its project management responsibilities to the Contractor, whose staff assumed important planning, monitoring, and reporting tasks.

• Effective project management and control was not exercised. During the duration of the contract significant areas of weakness were noted in: planning; tracking of performance; and managing scope, time, cost, quality, and human resources of the project.

• Critical decisions were not documented and supported with assessments of impacts in terms of risk, cost, time, and scope.

• The expertise and advice of oversight groups, such as Finance, Risk, and Legal, on critical contract decisions or procurement transformation strategies were not sought early nor were concerns fully addressed. This resulted in: resources and monies spent on responding to multiple queries from suppliers, media, and politicians; delays in solicitation documents; and decrease in PWGSC's reputation as the contracting expert and lead in the Government's procurement transformation and savings.

Reporting

• Records of decisions on oral interactions and decision points were too limited to provide audit assurance that Senior Management had accurate and complete reporting, and thus was fully aware of issues that would assist them in risk mitigation or modification of the course of actions.

Contract Management

• The Contracting Authority and the Project Authority did not maintain an expected arms-length relationship prior to January 2006. This contributed to weakening the normal checks and balances, which normally ensure the integrity of the procurement process.

• Subsequent to January 2006 structural changes to the Project Office were introduced, which resulted in some improvement to the arms-length relationship.

• The existing Standard Acquisitions Clauses and Conditions (SACC) Manual contained many variations of similar terms and conditions. This increased the risk that an inappropriate clause would be included in a contract. And as noted in the example of the discretionary audit clauses, none of the clauses would have provided an appropriate reduction of risk for contracts in which limitation of expenditure or ceiling price was used.

Financial Management

• PWGSC did not overexpend its authority under the contract. All expenditures were authorized within the value of the contract.

• Although Finance Branch provided departmental financial training to ensure that all responsible parties understood and accepted their roles and responsibilities, this audit noted areas of weakness in commitments, approval of invoices, and payments.

• Even when the payment of greater than $1 million was classified as a high-risk group, Post Payment Verification did not prevent a reoccurrence of a recorded early-payment.

Procurement Transformation

• The Department received benefits, made important progress on procurement transformation, and obtained government-wide savings during the duration of this contract.

• This government initiative to reform procurement required the full support and engagement of all departments and agencies in order to achieve significant savings and process improvements. In the initial stages of this contract, departments were not fully engaged and the transformation process was hampered, particularly by limitations in department's financial systems and their recording of expenditures. If these limitations continue they will hamper the Government's ability to forecast and achieve procurement savings.

• PWGSC did not have an adequate project and a resource-management plan in place to help it absorb the knowledge exchange, and operationalize the new ideas and methods of work that were being introduced by this contract. The pace of future work on procurement transformation will depend on having the necessary resources and capacity to achieve the desired results.

• The Department did not have an effective change management process in place to take advantage of the procurement strategies introduced by this contract, and to advance changes to department and government procurement policies, training, and upgrading skills of procurement officers, and modifications of procurement systems.

In this instance, the Project Office did not demonstrate due diligence in the exercise of effective project management and controls. The close organizational relationships of the Contracting Authority and Project Authority diluted the independence and authority of the contracting function. And the existing financial controls were not ensuring adherence to policies on commitments and payments. Nevertheless, the audit observed that important progress in procurement transformation had occurred in the period of January to September 2006, through the joint efforts of the Contractor and PWGSC. While a number of key foundation pieces have been done, it is however, too early to assess the overall results of the initiative to date.

Recommendations

The audit recommends that:

• The Assistant Deputy Minister, Corporate Branch should put departmental practices/policies in place that prior to the solicitation process of any major service contract for which PWGSC is the client, an independent assessment of the requirement be undertaken, which includes a clear governance structure and definition of roles and responsibilities, and identification of any major obstacles that could affect achievement of the expected results.

• The Assistant Deputy Minister, Acquisitions Branch should put practices in place, which ensure that management decisions that substantially alter the scope, timing, or costs of active contracts for which PWGSC is the client are reviewed by departmental oversight groups and that analysis of impacts, risks, options, and costs-benefits are prepared and decisions documented.

• The Assistant Deputy Minister, Acquisitions Branch should re-examine the number and use of Standard Acquisition Contract Clauses to ensure that the terms and conditions are appropriately used and consistent with the current procurement environment.

• The Assistant Deputy Minister, Acquisitions Branch, with the support of the Department, should develop an explicit new change management strategy and a resource-management plan that takes into account the risks and issues identified herein, and enable the Department to continue successfully advancing the procurement transformation.

• The Chief Financial Officer should communicate to line management staff about any errors, including non-critical ones found in the Post Payment Verification, whenever there is a possibility that the knowledge would help prevent a recurrence of errors.

About The Audit

Objectives

The objective of this audit was to provide assurance on the following aspects of a contract related to Business Transformation Initiative - A.T. Kearney Ltd. service contract:

• Project Management: PWGSC planned and managed the project's time, cost, and performance criteria in accordance with the requirements of the contract.

• Reporting: PWGSC provided authorized, complete, and accurate project information to Senior Management.

• Contract Management: PWGSC complied with delegated authorities, Government Contracts Regulations, and government and departmental policies/guidelines/practices.

• Financial Management: PWGSC validated invoices and authorized payments in accordance with Financial Administrative Act's financial delegations and government/departmental policies.

Scope and Approach

The Internal Audit Services Directorate auditors undertook the detailed examination phase during September and October 2006. The audit's primary focus was on Project Office's contract and project management after award of the contract.

The audit requested documentation and e-mails from the Project Authority (PA#1, PA#2) and from primary persons within the Project Office and Contracting Authority. Specific examples of results and achievements, based on the weekly status reports, was requested from the Stream Leads and examined. Interviews were conducted with PWGSC officials and the Contractor. The audit did not interview the Special Advisor, as his term with the Department was complete, and relied on documentation within the Project Office as evidence of actions taken.

The audit performed a general overview of all nine Task Authorizations and selected the largest two (TA#1, TA#3) for a detailed review of actions and deliverables.

While this audit contains the name of the Contractor, it must be noted that the conclusions about management practices and actions refer only to those of public servants. The rules and regulations referred to are those that guide public servants in the exchange of their duties. The actions and records of the private sector Contractor were not examined as part of this audit. However, in accordance with the terms and conditions of the contract, Audit Services Canada's Cost Auditors are verifying the accuracy of time claimed by Contractor.

This audit was conducted in accordance with the Treasury Board Policy on Internal Audit and the Institute of Internal Auditors' Professional Practices Framework that require internal auditors have sufficient and appropriate evidence and analysis to support their conclusions and audit results.

The auditors prepared audit findings and validated the findings and conclusions with appropriate departmental senior management and the Contractor, prior to preparation of the final report.

Criteria

The Criteria used in this audit were:

• Project Management. The project should be managed in accordance with principles and methodologies consistent with current best practices, such as those defined by the Project Management Institute and the Treasury Board Secretariat's Enhanced Management Framework.

• Reporting. Project Management should provide complete, accurate, and timely project information to Senior Management for the purpose of sharing of information and/or undertaking of appropriate actions or decisions.

• Contract Management. Government contracting should be compliant with Government Contract Regulations and policies, such that it will stand the test of public scrutiny in matters of prudence and probity, facilitate access, encourage competition, and reflect fairness in the spending of public funds.

• Financial Management. Prior to payment, PWGSC should attest that the level of effort of work, completed in hours and by resources, is accurate and/or reasonable, and that the work undertaken is in accordance with the contract terms and conditions.

Audit Work Completed

Audit fieldwork for this report was substantially completed on October 31, 2006.

Audit Team

The audit was conducted by members of the Audit & Evaluation Branch under the overall direction of the Chief Risk Officer, PWGSC who during the period of the audit also served as the Acting Director General of the Audit and Evaluation Branch.

Subsequent Events

As of September 28, 2006, the solicitation process to seek the services of a Contractor to continue support of the Acquisitions Business Transformation has been suspended. The Department has completed further procurement strategies and departmental service level agreements.

On November 23, 2006, as part of the Economic and Fiscal Update, the Government announced that it was adjusting its savings forecast for Procurement Transformation. The Government determined that the project savings would total $1.4 billion less than originally anticipated. This revised fiscal target was reflected in the Government's spending projections for the period starting in year three of the Procurement Transformation (2007-08) to year seven (2011-12).

In Budget 2007, the Government announced that it had determined that the savings for 2008-09 and beyond will need to be achieved by departments through overall cost efficiencies, including continued efforts to reduce procurement costs. Accordingly, departmental budgets are to be reduced to reflect the fiscal targets projected in the November 2006 Economic and Fiscal Update.

Along with this audit, and in accordance with the Time Verification clause of the contract, PWGSC initiated a Cost Audit. This action of due diligence will enable the Crown to verify the accuracy of time and billing claimed by the Contractor. The Cost Audit is currently underway.

Appendix 1 - Management Action Plan

Recommendation 1

The ADM, Corporate Branch should put departmental practices/policies in place that prior to the solicitation process, for any major service contract for which PWGSC is the client, an independent assessment of the requirement be undertaken, which includes a clear governance structure and definition of roles and responsibilities and identification of any major obstacles that could affect achievement of the expected results.

OPI

Director General - Health, Safety, Security, Emergencies and Administration Sector

Implementation Actions

Develop a process that will provide the required national oversight for major service contracts for which PWGSC is the client.

Action Implementation Date(s)

Second quarter 2007-08.

Implementation Actions

Implement approved approach.

Action Implementation Date(s)

Third quarter 2007-08.

Recommendation 2

The ADM, Acquisitions Branch should put practices in place, which ensure that management decisions that substantially alter the scope, timing, or costs of active contracts for which PWGSC is the client are reviewed by departmental oversight groups and that analysis of impacts, risks, options, and costs-benefits are prepared and decisions documented.

OPI

Director General, Risk, Integrity and Strategic Management Sector

Implementation Actions

Pursuant to implementation action for recommendation 1 above, take action as necessary.

Action Implementation Date(s)

Third quarter 2007-08.

Recommendation 3

The ADM, Acquisitions Branch should re-examine the number and use of Standard Acquisition Contract Clauses (SACC) to ensure that the terms and conditions are appropriately used and consistent with the current procurement environment.

OPI

Director General, Risk, Integrity and Strategic Management Sector

Implementation Actions

Complete review of existing clauses and notes to contracting staff.

Action Implementation Date(s)

Fourth quarter 2007-08.

Implementation Actions

Modify or redraft clauses in consultation with appropriate groups. (Legal, Cost Accounting, Contractual Risk and Quality Control (CRQC) etc.)

Action Implementation Date(s)

Fourth quarter 2007-08.

Implementation Actions

Issue Policy Notification (PN) informing contracting staff of changes to clauses and instructions.

Action Implementation Date(s)

Bulk of changes is done through quarterly SACC releases. If required in specific circumstances PNs or instructions are issued on an ad hoc basis. Documents issued since audit completion include:

Policy notifications

PN 71 - Departmental Plain Language Standard Procurement Documents: Low Dollar Value and Medium Complexity Requirements - March 2005.

PN 73 - Notification of Bidders November 1, 2006.

PN 79/79U - Departmental Plain Language Standard Procurement Documents: Standing Offers - Goods or Services - August 2006.

PN 81 - Bid Price Evaluation - January 23, 2007.

PN 82 - Client Review of Elements of a Bid Solicitation - January 23, 2007.

PN 83 - Departmental Standard Procurement Documents for Higher Complexity Requirements for Goods, Services or both - March 19, 2007.

Instructions / Reminders to all Contracting Staff

Appropriate Clause Usage - January 12, 2007

Notification to DGs/RDGs re Bid Solicitation Review Process and Checklist - April 2, 2007.

Implementation Actions

Incorporate changes to Standard Acquisitions Contract Clauses and Supply Manual.

Action Implementation Date(s)

Fourth quarter 2007-08.

Recommendation 4

The ADM, Acquisitions Branch, with the support of the Department, should develop an explicit new change management strategy and a resource-management plan that takes into account the risks and issues identified herein, and enable the Department to continue successfully advancing the Acquisitions Business transformation.

OPI

Acquisitions Branch

Implementation Actions

Explicit Change Management Strategy. The Acquisitions Branch (AB) is currently in the process of finalizing a comprehensive implementation plan (IP) for procurement transformation. This plan will ensure that PWGSC has an effective change management process to implement new procurement strategies and other advances to policies, training and systems.

Action Implementation Date(s)

Second Quarter 2007/08.

Implementation Actions

Resource Management Plan: The IP contains detailed information about the required resources to achieve the transformation. The IP will identify areas where the aid of consultants will be or may be necessary. In those instances, the IP will ensure there are sufficient AB or PWGSC resources to absorb outside expertise.

Action Implementation Date(s)

Third Quarter 2007/08.

Implementation Actions

AB will interface with representatives from Finance, Risk, Legal, and Corporate Policy to ensure that:

• Critical decisions on procurement transformation are fully documented in terms of risk, cost, time, and scope.
• The procurement transformation defines success more broadly than savings.
• An appropriate 'arms-length' relationship between the project authority and contract authority in any subsequent contracts for external assistance with transformation.

Action Implementation Date(s)

In place, on-going.

Recommendation 5

The Chief Financial officer should communicate to line management staff about any errors including non-critical ones found in the Post Payment Verification, whenever there is a possibility that the knowledge would help prevent a recurrence of errors.

OPI

Chief Financial Officer

Implementation Action

Will ensure that non-critical errors found in the Post Payment Verification be communicated to line management staff, whenever there is a possibility that the knowledge would help prevent a recurrence of errors, in addition to reporting on critical errors which is currently done under the National Accounts Verification Framework.

Action Implementation Date(s)

Fourth quarter 2006-07.