2005-729 Directed Audit of the Validation of the Proposal Values for the NFTC Settlement, Final Report

September 08, 2006

Table of contents

• Executive Summary
• 1. Introduction
  • 1.1 Authority for the Project
  • 1.2 Objectives
  • 1.3 Scope and Approach
  • 1.4 Background
• 2. Findings
• 3. Conclusions
• Appendix A - Terms of Reference

Please note that under the Access to Information Act a limited amount of text within these documents may not be disclosed, and will be shown as [ * ].

Executive Summary

The objective of the audit was to provide assurance of Departmental due diligence by assessing the validation, conducted by Acquisitions Branch, of the compensation values in the proposal from Bombardier Inc. for the Settlement of Missed Training for the NATO Flying Training in Canada (NFTC) program. Audit criteria identified in the Terms of Reference (Appendix A - Terms of Reference) were applied to the assessment.

Table 1. Missed Training and Table 2 (below) present the values for which the Internal Audit Services Directorate (IASD) audit team assessed the validation conducted by Acquisitions Branch. The missed training and proposal compensation values presented in these tables have been accepted by the Department of National Defence (DND) Project Authority and the Acquisitions Branch – NFTC Contract Authority. These values were applied by a reputable firm in its Assessment of the Proposed Settlement (October 2004). The values have been presented in a Draft Memorandum to the Minister.

Based on the application of the audit criteria to its assessment, IASD concludes that due care was exercised by Acquisitions Branch in the validation of the values of the Missed Training ($89.1 million) and Compensating Values (totaling $60.0 million). The aforementioned values were validated in the context of the contract basis of payment in a manner consistent with Government of Canada Contract Cost Principles. The Acquisitions Branch validation is supported by a detailed, quantified, and logical reconciliation using corroborating information where appropriate.

After applying the total of the Proposal's compensating values ($60.0 million) to the calculated value of the missed training ($89.1 million) there remains a residual $29.1 million which is an amount to be assumed by the Crown, with acceptance of the proposal, as stated in the Draft Memorandum to the Minister. There is no apparent objective manner to produce a quantitative reconciliation of the residual $29.1 million against contributing factors. A rationalization for acceptance of an amount to be assumed by the Crown is put forward in the Draft Memorandum to the Minister, however, an analysis of the rationalization was beyond the scope of this audit.

Table 1

1. Missed Training

Total missed training to Dec 2002 - a quantification of the amount of missed training used by Acquisitions Branch and DND in the assessment of the adequacy of the proposal compensation. This value was calculated by Acquisitions Branch, based on information provided by DND regarding enrollment and proficiency flying hours, using rates derived from the contract schedules. 89.1 million

Table 2

2. Mitigating amounts and compensation contained in the Bombardier Inc. (BI) Proposal assigned a tangible value. These amounts have been validated by Acquisitions Branch (AB) supported by a detailed quantified rationalization.

2.1 Mitigating Amounts:

• Liquidated Damages held by MILIT-AIR. This amount is fixed and will come to the Government. 6.0 million
• Extra Interest which accrued in MILIT-AIR due to delivery delays and will come to the Crown. Although the AB validation of interest is based on a model, it is detailed and thorough with comparisons to audited statements. AB cost analysts reviewed/ accepted this work. 11.5 million
• Waiver of claim by BI against surplus in MILIT-AIR (allowed for in the Canada Services Agreement (CSA). 3.0 million
• [ * ]

2.2 Flight School Administration compensation:

This service is not in the contract, however, it has been provided by BI since start of the NFTC. BI proposes to continue this service at no additional cost until end of the contract - 2021 with acceptance. 2.2 million

2.3 Foreign student credit compensation:

BI will train foreign students for a reduced price, for up to 10.3 million savings, with acceptance. DND has Memoranda of Understanding (MOU) in hand and will receive the $ directly. 10.3 million

2.4 Training and proficiency flying compensation:

Services delivered over and above that scheduled through the contract already provided by BI and not paid. Although contract administration requirements call for a formal amendment prior to delivery of services of additional or revised services, DND had authorized the training and flying therefore an implied financial obligation. This value was calculated by AB, based on information provided by DND regarding enrollment and proficiency flying hours, using rates derived from the contract schedules. 27.0 million

Summary

The total of mitigating amounts and compensation contained in the BI Proposal assigned a tangible value by AB supported by a detailed quantified rationalization is 60.0 millions.

1. Introduction

1.1 Authority for the Project

The audit project was undertaken by the Internal Audit Services Directorate (IASD), Office of Audit and Evalution (OAE), Public Works and Government Services Canada (PWGSC) in response to a request from the Deputy Minister, PWGSC, and the Chief Risk Officer.

1.2 Objective

To provide assurance of Departmental due diligence by assessing the validation, conducted by Acquisitions Branch (AB), of the compensation values in the proposal from Bombardier Inc. for the Settlement of Missed Training for the NATO Flying Training in Canada (NFTC) program.

The IASD applied the following criteria in its assessment:

• The validation processes applied are documented and consistent with the Government of Canada Contract Cost Principles as detailed in DSS 1031-2.
• The settlement proposal was validated in the context of the contract basis of payment.
• Appropriate competencies in contract cost auditing, contracting and the NFTC contract and program were applied to the validation.
• There is an adequate itemization of costs and benefits and calculations are correct and logical to support the proposal and business decisions.
• Information independent from Bombardier and/or audited Bombardier information including Department of National Defence (DND) acceptance of training enrollment and flying hours was used to corroborate the proposed compensation values.

1.3 Scope and Approach

The Terms of Reference (ToR) for this audit (Appendix A - Terms of Reference) were provided to A/Assistant Deputy Minister Acquisitions Branch and Chief Risk Officer for comment prior to commencement. The ToRs were also shared with the DND NFTC Project Authority and the DND Chief Audit Executive.

The audit team conducted interviews, collected and analysed relevant data to confirm the criteria, and assessed the work conducted by AB and DND to validate the compensation values in the Bombardier proposal.

Interviews were conducted with the AB Contract Authority Manager, the AB Procurement Officer, the AB Contract Cost Officer, and the DND NFTC Project Authority. In addition, the PWGSC Chief Risk Officer and the Office of the Auditor General were consulted to determine potential concerns.

Key documents were reviewed including the following: the NFTC Briefing to ADMs which includes draft Memorandum to the Minister with supporting documentation including the Bombardier proposal and the DND request to effect the Settlement; the Canada Services Agreement (CSA) (which is the subject contractual document between Canada and Bombardier Inc.) and its schedules; the Assessment Report of the Proposed Settlement; as well as AB supplied documentation describing the NFTC program and its validation process. Corroborating documentation and correspondence from DND used in the validation process was obtained and reviewed. Copies of electronic spreadsheets which were used by AB and DND in the validation of the proposal values were obtained along with a written description from AB of the contents and how the contents were applied in the validation including assumptions.

The audit examined and assessed the processes and information used by the AB to validate key values related to the Settlement proposal. These key values are displayed in Tables 1 and 2 presented in Section 3. Conclusions.

The audit team developed findings against each audit criterion which were validated with appropriate managers in Acquisitions Branch. Findings and Conclusions as presented Section2. Findings and 3. Conclusions were validated with the A/ADM, Acquisitions Branch prior to presentation to the Chief Risk Officer and the Deputy Minister.

Audit examination work commenced on October 28, 2005 and was completed on November 3, 2005.

1.4 Background

The NATO (North American Treaty Organization) Flying Training in Canada (NFTC) is a $3.46 billion program for the provision of military pilot training on the behalf of DND. Bombardier Inc. is the prime contractor for the program.

There was an outstanding issue related to an approximate $89.1 million of training paid for but not received up to December 2002. Subsequent to negotiations with PWGSC and DND, Bombardier submitted a settlement proposal, in 2004, for the missed training. The DND Project Authority has requested that the Bombardier proposal be accepted.

Acquisitions Branch has advised that, in conjunction with DND, they have validated the values in the proposal and recommend acceptance of the proposed settlement. They have provided supporting documentation with this recommendation. Acquisitions Branch advised that acceptance will require a contract amendment to effect the settlement.

[ * ]. The Chief Risk Officer was involved in meetings on this issue. The Office of the Auditor General (OAG) and the Standing Committee and Public Accounts are aware of the issue and have recommended resolution.

2. Findings

This section presents each audit criterion followed by a finding:

Criterion: The validation processes applied are documented and consistent with the Government of Canada Contract Cost Principles as detailed in DSS 1031-2.

Finding: The validation process applied is documented in the CA's Briefing Presentation slides and written description accompanying spreadsheets. The validation process is consistent with the Government of Canada Contract Cost Principles as detailed in DSS 1031-2.

Criterion: The settlement proposal was validated in the context of the contract basis of payment.

Finding: There is evidence that the CSA and supporting rate schedules were applied by AB in the validation. The Briefing Presentation indicates the contract basis of payment was applied in the validation where appropriate. The audit team participated in interactive sessions led by the AB CSA Contract Authority and Procurement Officer which demonstrated the validation process and included a walkthrough of each of the spreadsheets. During the walkthrough session the audit team referred to supporting documentation including the CSA schedules and NFTC student enrollment numbers and proficiency flying hours to verify key values for each of spreadsheets.

Criterion: Appropriate competencies in contract cost auditing, contracting and the NFTC contract and program were applied to the validation.

Finding: The AB Contract Authority and the AB Procurement Officer are experienced and are knowledgeable in terms of the Program and contract. They have been involved since 2000 and 2002, respectively, and have worked since receipt of the proposal in February 2004 on the validation. Several independent groups and individuals were involved in the validation including a AB Cost Analyst used to review interest validation. In addition, DND Project Authority financial personnel participated in the validation. Senior managers in AB and DND have reviewed the settlement validation. A reputable firm was engaged to review the validation as part of its Assessment of the Settlement.

Criterion: There is an adequate itemization of costs and benefits and calculations are correct and logical to support the proposal and business decisions.

Finding: Although the audit team did not independently test the accuracy of the spreadsheet contents and calculations, the audit team participated in interactive sessions led by the AB CSA Contract Authority and Procurement Officer which included a walkthrough of each of the spreadsheets. There was evidence of considerable detail and itemization of costs and benefits to support the validation of proposal values. The process applied was documented, logical and referenced to the CSA. Intangible and future benefits were noted but were not assigned compensation values by AB.

Criterion: Information independent from Bombardier and/or audited Bombardier information including DND acceptance of training enrollment and flying hours was used to corroborate the proposed compensation values.

Finding: There is evidence that DND NFTC Project Authority accepts the proposal. There is evidence for certain key numbers such as enrollment and extra flying hours that confirmation of the numbers was obtained from DND as part of the validation. The DND validated NFTC student enrollment numbers and extra flying hours are the key underlying basis to the "Missed Training" (Table 1. Missed Training) and the "Training and proficiency flying over and above that scheduled through the contract already provided by BI and not collected by BI" (Table 2 - 2.4). The audit team limited its examination to the correspondence from DND confirming these numbers since the underlying source documents for the student enrollment numbers and proficiency flying hours are the property of DND and were not available to the IASD audit team.

Independent corroborating information, available during this audit, such as annual audited and quarterly unaudited financial statements for MILIT-AIR was used to compare interest figures and liquidated damages as well as consultation with the MILIT-AIR external auditors where appropriate.

3. Conclusions

Table 3 and Table 4 (below) present the values for which the IASD audit team assessed the validation conducted by Acquisitions Branch. The missed training and proposal compensation values presented in these tables have been accepted by the DND Project Authority and the Acquisitions Branch – NFTC Contract Authority. These values were applied by the firm engaged to review the Settlement (2. Findings) and reported in its Assessment of the Proposed Settlement (October 2004). The values have been presented in a Draft Memorandum to the Minister.

Table 3 Missed Training presents the value of payments to December 2002 made by the Crown for NFTC training which was not received. This value was calculated by Acquisitions Branch based on DND provided information and the NFTC CSA contract and has been used by Acquisitions Branch and DND in the assessment of the adequacy of the proposal compensation.

Table 4 presents the mitigating amounts and compensation values contained in the Bombardier Proposal for Settlement which have been assigned a tangible value by Acquisitions Branch. The validation of these values conducted by Acquisitions Branch is supported with a documented detailed and quantified rationalization. The Acquisitions Branch indicates the Bombardier proposal, if accepted, offers future potential, if taken or realised, which could provide further compensation. These future potential amounts are not included as part of Table 4, Compensating Values.

Based on the application of the audit criteria set out in the Terms of Reference (Appendix A - Terms of Reference) to its assessment of the validation of the compensation values conducted by Acquisitions Branch, IASD concludes that due care was exercised by Acquisitions Branch in the validation of the values of the Missed Training ($89.1 million) and Compensating Values (totaling $60.0 million). The aforementioned values were validated in the context of the contract basis of payment in a manner consistent with Government of Canada Contract Cost Principles. The Acquisitions Branch validation is supported by a detailed, quantified, and logical reconciliation using corroborating information where appropriate.

After applying the total of the Proposal's compensating values ($60.0 million) to the calculated value of the missed training ($89.1 million) there remains a residual $29.1 million which is an amount to be assumed by Crown, with acceptance of the proposal, as stated in the Draft Memorandum to the Minister. There is no apparent objective manner to produce a quantitative reconciliation of the residual $29.1 million against contributing factors. A rationalization for acceptance of an amount to be assumed by the Crown was put forward in the Draft Memorandum to the Minister, however, an analysis of the rationalization was beyond the scope of this audit.

Table 3

1. Missed Training

Total missed training to Dec 2002 - a quantification of the amount of missed training used by Acquisitions Branch and DND in the assessment of the adequacy of the proposal compensation. This value was calculated by Acquisitions Branch, based on information provided by DND regarding enrollment and proficiency flying hours, using rates derived from the contract schedules. 89.1 million.

Table 4

2. Mitigating amounts and compensation contained in the Bombardier Inc. (BI) Proposal assigned a tangible value. These amounts have been validated by Acquisitions Branch (AB) supported by a detailed quantified rationalization.

2.1 Mitigating Amounts:

• Liquidated Damages held by MILIT-AIR. This amount is fixed and will come to the Government. 6.0 million
• Extra Interest which accrued in MILIT-AIR due to delivery delays and will come to the Crown. Although the AB validation of interest is based on a model, it is detailed and thorough with comparisons to audited statements. AB cost analysts reviewed/ accepted this work. 11.5 million
• Waiver of claim by BI against surplus in MILIT-AIR (allowed for in the Canada Services Agreement (CSA). 3.0 million
• [ * ]

2.2 Flight School Administration compensation:

This service is not in the contract, however, it has been provided by BI since start of the NFTC. BI proposes to continue this service at no additional cost until end of the contract - 2021 with acceptance. 2.2 million

2.3 Foreign student credit compensation:

BI will train foreign students for a reduced price, for up to 10.3 million savings, with acceptance. DND has Memoranda of Understanding (MOU) in hand and will receive the $ directly. 10.3 million

2.4 Training and proficiency flying compensation:

Services delivered over and above that scheduled through the contract already provided by BI and not paid. Although contract administration requirements call for a formal amendment prior to delivery of services of additional or revised services, DND had authorized the training and flying therefore an implied financial obligation. This value was calculated by AB, based on information provided by DND regarding enrollment and proficiency flying hours, using rates derived from the contract schedules. 27.0 million

Summary

The total of mitigating amounts and compensation contained in the BI Proposal assigned a tangible value by AB supported by a detailed quantified rationalization is 60.0 million.

Appendix A - Terms of Reference

Project Title and Number

2005-729 Directed Audit of the Validation of the Proposal Values for the NATO Flying Training in Canada (NFTC) Settlement

Authority for the Project

The audit project was undertaken by the Internal Audit Services Directorate (IASD), Office of Audit and Evalution (OAE), Public Works and Government Services Canada (PWGSC) in response to a request from the Deputy Minister, PWGSC, and the Chief Risk Officer.

Objective

To provide assurance of Departmental due diligence by assessing the validation, conducted by Acquisitions Branch (AB) of the compensation values in the proposal from Bombardier Inc. for the Settlement of Missed Training for the NATO Flying Training in Canada program.

The IASD will apply the following criteria in its assessment:

• The validation processes applied are documented and consistent with the Government of Canada Contract Cost Principles as detailed in DSS 1031-2.
• The settlement proposal was validated in the context of the contract basis of payment.
• Appropriate competencies in contract cost auditing, contracting and the NFTC contract and program were applied to the validation.
• There is an adequate itemization of costs and benefits and calculations are correct and logical to support the proposal and business decisions.
• Information independent from Bombardier and/or audited Bombardier information including DND acceptance of training enrollment and flying hours was used to corroborate the proposed compensation values.

Scope

The assessment will examine and assess the processes and information used by the AB supported by the Department of National Defense (DND) to validate the compensation values in the Bombardier proposal. Interviews will be conducted with AB and DND personnel, as required, as well as with the PWGSC Chief Risk Officer. The contract, related audits, studies and documentation maintained by AB and DND to support the validation will be examined and considered as required.

The IASD audit team will limit its examination to the documentation available from AB and DND and will not request additional information from Bombardier.

Background

The NATO (North American Treaty Organization) Flying Training in Canada (NFTC) is a $3.46 billion program for the provision of military pilot training on the behalf of DND. Bombardier Inc. is the prime contractor for the program.

There is an outstanding issue related to an approximate $89 million training paid for but not received up to December 2002. Subsequent to negotiations with PWGSC and DND, Bombardier submitted a settlement proposal, in 2004, for the missed training.

Acquisitions Branch has advised that, in conjunction with DND, they have validated the values in the proposal and recommend acceptance of the proposed settlement. They have provided supporting documentation with this recommendation. Acquisitions Branch has advised that acceptance would require a contract amendment to effect the settlement.

[ * ]. The Chief Risk Officer has been involved in meetings on this issue.

The Office of the Auditor General (OAG) and the Standing Committee and Public Accounts are aware of the issue. The OAG is finalizing its audit work for a report on this issue in February 2006.

Approach and Methodology

The assessment will be conducted in accordance with applicable generally accepted auditing standards.

The audit team will conduct interviews, collect and analyse relevant data to confirm the criteria, and assess the work conducted by AB and DND to validate the compensation values in the Bombardier proposal against the criteria.

The audit team will prepare audit findings against the audit criteria and develop a conclusion against the objective which will be validated with appropriate managers in Acquisitions Branch.

Key findings and conclusions, along with any resulting recommendations will be documented in a Draft Report. The Draft Report will be validated with the A/ADM, Acquisitions Branch prior to presentation to the Chief Risk Officer and the Deputy Minister.

Time Frame

Due to contract related time constraints the assessment will be commenced immediately with a target Draft Final Report November 10, 2005.

Resources

Audit Team

André Auger

Director General, Audit and Evaluation Branch

B. Orsini

Director, Internal Audit Services

N. MacDowall

Audit Principal

D. Borys

Professional Services CAC

Professional Services and Audit Team Budget: Approximately $10,000 and 3 Audit Weeks