Fixed-Wing Search and Rescue Aircraft Replacement Project - Independent Review of Evaluation Plans, Methods and Tools

November 21, 2013

The observations included in this report are limited to the scope described in Section 2. This report must be considered in its entirety; selecting specific words or portions of this report without an understanding of the review timeframe and coverage could result in misinterpretation of our observations.

Table of Contents

• 1. Background
• 2. Objective and scope
  • 2.1 Review objective
  • 2.2 Review scope
  • 2.3 Scope exclusions
• 3. Approach
• 4. Limitations of this report
• 5. Review observations
  • 5.1 Comments on review observations related to the unbiased procurement objective
  • 5.2 Comments on review observations related to the best value procurement objective

Acronyms

The following acronyms are used in this report:

APAT

Aircraft Performance Assessment Tool

CDRE

Capability Delivery Risk Evaluation

CFO

Chief Financial Officer

DND

Department of National Defence

FWSAR

Fixed Wing Search and Rescue

FWSAR Project

Fixed Wing SAR Aircraft Replacement Project

IRB

Industrial and Regional Benefits

ISS

In-Service Support

NRC

National Research Council

OAG

Office of the Auditor General

PWGSC

Public Works and Government Services Canada

QA

Quality Assurance

RFP

Request for Proposal

RS

Requirement Specification

SAR

Search and Rescue

SOR

Statement of Operational Requirements

SOW

Statement of Work

1. Background

Search and Rescue (SAR) in Canada is a shared responsibility among federal, provincial/territorial and municipal organizations, which requires ground, air and maritime capabilities, and must be able to accommodate the varying climates across the country. The Department of National Defence (DND) holds primary responsibility for providing aeronautical SAR services (search for downed aircraft), while the Canadian Coast Guard holds primary responsibility for maritime SAR services. DND's aeronautical SAR services are delivered by both rotary and fixed wing aircraft, each with a distinct purpose.

DND's Fixed Wing SAR (FWSAR) operations are presently carried out by aircraft that are nearing the end of their service life. DND has a fleet of 6 CC115 Buffalo that are 45 years old, as well as a fleet of 13 CC130H Hercules that are between 20 and 39 years old. As part of the Canada First Defence Strategy, the replacement of Canada's aging FWSAR aircraft has been identified as a Government priority.

The Fixed Wing SAR aircraft replacement (FWSAR) project (referred to in this report as “the FWSAR Project”) was initiated in 2002 to replace the aging fleet of FWSAR aircraft. In the fall of 2009, following industry consultation undertaken by the FWSAR Project, the Government of Canada engaged the National Research Council (NRC) to conduct an independent review of the FWSAR Project's Statement of Operational Requirements (SOR). In its report, the NRC recommended improvements to the technical requirements, as well as to the assumptions and constraints that were used to derive the requirements for the FWSAR Project.

Following the NRC's recommendations, the FWSAR Project was restarted with the development of a new SOR. With this restart, the Government of Canada aimed to shift greater focus to the required outcomes (capabilities) of the FWSAR aircraft, rather than the required characteristics of the aircraft product (or “platform”). The intent of this shift was to address NRC's recommendations and allow bidders to devise solutions that would best meet the required outcome. The increased focus on capability-based procurement has given rise to bid evaluations plans, methods, processes, decisions and tools that are new for DND aircraft acquisitions.

Delivery of the new SAR aircraft is currently planned for 2018. The Air Force has extended the operational lives of the Hercules and Buffalo aircraft. As highlighted in the Office of the Auditor General's (OAG) 2013 spring report, maintenance costs have risen and the availability of the FWSAR aircraft has been reduced due to maintenance of the aging fleet.

The replacement of FWSAR aircraft represents a multi-billion dollar acquisition for the Government of Canada. We understand that Public Works and Government Services Canada (PWGSC), Industry Canada and DND are working toward releasing the Request for Proposal (RFP) for the FWSAR Project in 2014 with expected delivery of the first new aircraft in 2018 and Full Operational Capability for the new aircraft in 2020. The FWSAR Project is seeking to promote best value for Canadians by considering the capital, operating and maintenance costs over the full life of the aircraft. The FWSAR Project's procurement approach also seeks to achieve industrial benefits.

2. Objective and scope

In July 2013, through a competitive procurement process, PWGSC contracted Orbis Risk Consulting Inc. to conduct an independent review between July 2013 and November 2013. Our review objective and scope are detailed in the following sections:

1. Review objective;
2. Review scope; and
3. Scope exclusions.

2.1 Review objective

The objective was to perform an independent review of FWSAR Project evaluation plans, methods, processes, decisions and tools, which were at varied stages of development, and provide observations in relation to two of the Government of Canada's procurement objectives, that they be:

• unbiased, and not inadvertently favour one bidder over another; and
• promote a best-value procurement approach.

We considered the broader framework of FWSAR Project bid evaluation plans, methods, processes, decisions, and tools as they were being developed and, for a subset of these which were nearing completion, we examined them in greater depth.^{Footnote 1} The aspects examined in greater depth are described in section 2.2. Section 2.3 provides greater clarity on aspects specifically excluded from the scope of our review.

A review does not constitute an audit and consequently we did not seek to express an opinion on whether the FWSAR Project evaluation plans, methods, processes, decisions and tools are free of bias and will achieve best value. 

2.2 Review scope

Our review was conducted between 9 July 2013 and 21 November 2013 and considered activities, decisions and documents from the period of 1 April 2012 to 21 November 2013. Considering that bid evaluation plans, methods, processes, and tools were at varied stages of completion at the time of completing our review, the scope of our work was limited to the information as it existed when it was provided to the review team. We were asked to examine in greater depth, a subset of plans and tools that were further along in their development, which are listed below. Our review also considered, but did not examine in depth, the broad framework of plans and tools being developed by the Government of Canada to support bid evaluation for the FWSAR Project.

In recognition that the Government of Canada was evolving and refining various aspects of the bid evaluation plans, methods, processes, decisions and tools at the time of this report, coupled with the need for a timely independent review report, we were limited to reviewing information when it was provided to the review team and were reliant on DND, PWGSC and Industry Canada to provide us with current, relevant, complete and accurate information.

The plans and tools that were examined in greater depth included:

• planned high-level structure for relative weighting between cost and capability criteria;
• methods employed to assign weights to ranked aircraft acquisition requirements;
• conceptual and functional design of the Aircraft Performance Assessment Tool (APAT), which was developed by the Government of Canada to evaluate the SAR incident response performance of bidders' proposed solutions;
• change, quality assurance and security controls supporting the APAT;
• framework for engaging industry throughout the procurement; and
• industry consultation processes for soliciting feedback on the draft RFP between 30 August 2013 and 18 October 2013. At the time of writing this report, the workflow to manage incoming industry feedback was being refined to manage the volume of information received from bidders in response to the draft RFP. While we examined the consultation processes to solicit feedback on the draft RFP between 30 August 2013 and 18 October 2013, the scope did not examine this industry consultation process in its entirety.

2.3 Scope exclusions

The following elements were not included in the scope of our independent review:

• Full procurement process: Our review did not look at the end-to-end FWSAR Project procurement process and excluded all aspects of the procurement processes unrelated to the subset of bid evaluation plans, methods, processes, decisions, and tools outlined in Section 2.2. Examples of processes not covered in our review included budgeting, project management, aircraft lifecycle management, end-to-end industry engagement throughout the procurement process, end-to-end quality assurance, monitoring and oversight, as well as requirements management, among others.
• Definition of requirements: Establishment of requirements for the aircraft acquisition, response performance, In-Service Support (ISS) and capability delivery risk was excluded from the scope of our review, including the processes by which the requirements were developed, documented, classified (as mandatory or rated), prioritized, articulated and iteratively refined. For the aircraft acquisition requirements, while the scope excluded the iterative selection, classification, prioritization (i.e. ranking in order of importance) of requirements, we did examine the methods employed to assign weights to ranked rated requirements. While we could not examine the ranking process used for the aircraft acquisition requirements due to interdependence with requirements selection and classification processes, we did consider how the FWSAR Project employed facilitated consensus meetings to mitigate risks of bias by drawing on the expertise of internal specialists during ranking decisions.
• Other reviews: While the framework for soliciting and integrating reviews completed by internal and external subject matter specialists was considered, specifically how these reviews intersected with the subset of bid evaluation plans, methods, processes, decisions and tools within our scope and the processes employed in completing these reviews was not considered, nor did our independent review duplicate the intended coverage of these reviews. These reviews included, but were not limited to, a legal and quality assurance review, fairness monitoring, a costing model assessment and a "Red Team" end-to-end review of the draft RFP, all of which were underway while we performed our work. Our independent review did not assess the completeness, accuracy or alignment of these other reviews.
• Full draft RFP document: The FWSAR Project's Statement of Work (SOW), Requirement Specification (RS) and the entirety of the draft RFP shared with industry on 30 August 2013, along with supporting documentation, was excluded from the scope of our review. Only those elements of the draft RFP related to the subset of bid evaluation plans, methods, processes and tools within scope were examined.

3. Approach

Our independent review consisted primarily of enquiry, analytical procedures, and discussion related to information supplied to us by representatives of DND, PWGSC and Industry Canada, as well as subject matter experts external to the Government of Canada. Over 750 documents were examined and 33 interviews conducted. We had no contact with known potential bidders throughout the independent review.

A review does not constitute an audit and consequently, our work was not intended to be exhaustive and we did not seek to express an opinion on whether the FWSAR Project evaluation plans, methods, processes, decisions and tools are free of bias and will achieve best value. Our independent review relied on the accuracy of information provided by DND, PWGSC and Industry Canada.

Our independent review was conducted in the following two phases:

• Phase I, conducted from July 2013 to August 2013 – The first phase entailed examining a subset of bid evaluation plans, methods, processes, decisions and tools prior to the release of a draft RFP to industry on 30 August 2013. As part of Phase I, Orbis identified preliminary observations pertaining to the Government of Canada's procurement objectives of promoting best value and unbiased procurement.
• Phase II, conducted from October 2013 to November 2013 – The second phase included a review of applicable changes to the draft bid evaluation plans, methods, processes and tools reviewed in Phase I. Phase II also entailed examining the consultation processes to gather industry feedback on the draft RFP between 30 August 2013 and 18 October 2013. As outlined in Section 2, the workflow to manage incoming industry feedback was being refined in reaction to the volume of information received from bidders in response to the draft RFP. The subset of draft bid evaluation plans, methods, processes, decisions and tools within scope continued to be under development at the time that our independent review was performed, and we expect them to change following the completion of our review.

This report comments on observations resulting from the two-phased review conducted between 9 July 2013 and 21 November 2013.

4. Limitations of this report

This report is provided for the sole and exclusive use and benefit of the Government of Canada to inform decisions pertaining to the design of bid evaluation methods, processes, and tools. No other person is authorized to rely on, use, copy, duplicate, reproduce or disseminate this report, in whole or in part and for any reason whatsoever, without the express prior written consent of Orbis Risk Consulting Inc. This report should be considered in its entirety; selecting specific portions of this report may result in the misinterpretation of the content. Any person using this report, other than the persons to whom it is addressed, does so entirely at its own risk. Orbis Risk Consulting Inc. assumes no responsibility or liability in connection with the reliance made by any party on this report or the observations and comments contained within this report. Orbis Risk Consulting Inc. reserves the right, but will not be obligated, to review or revise this report if deemed necessary by Orbis Risk Consulting Inc.

Our independent review and this resultant report are limited to the scope described in Section 2. This report should not be acted upon without performing further analysis. The Government of Canada is responsible for decisions made in respect of the FWSAR Project and such decisions will influence the realization of intended outcomes. The value ultimately achieved by the FWSAR Project and the extent to which the bid evaluation methods, processes and tools are unbiased will be impacted by future decisions of the Government of Canada.

5. Review observations

This section details our review observations.

Based on the work performed in our review, as set out in the scope described in section 2, nothing came to our attention that would cause us to believe that the FWSAR Project plans, methods, processes, decisions or tools are biased, where one bidder was inadvertently favoured over another. Also, we noted methods that are consistent with the objective of promoting a best value procurement approach. Our observations are presented under the following headings:

1. Comments on review observations related to the unbiased procurement objective; and
2. Comments on review observations related to the best value procurement objective.

5.1 Comments on review observations related to the unbiased procurement objective

Our observations related to the unbiased procurement objective are detailed under the following three sections:

1. Engaging industry;
2. Developing evaluation techniques; and
3. Validating decisions.

5.1.1 Engaging Industry

Our review highlighted measures taken by the Government of Canada, which are consistent with the objective of managing risks related to bias, as follows:

• Early engagement: Industry was engaged by the Government of Canada early in the procurement process. Seeking early feedback from potential bidders positioned the Government of Canada to identify aspects of the procurement which needed to be closely managed to mitigate the risk of inadvertently favouring one bidder over another.
• One-on-one sessions: Consistent with the Government of Canada's Smart Procurement Principles, one-on-one sessions were held with known potential bidders at different stages of the procurement process. These sessions served to establish a dialogue between the FWSAR Project and industry, which can be helpful in understanding and proactively managing risks related to bias.
• Involvement of a Fairness Monitor: The Government of Canada engaged a Fairness Monitor to participate as an objective advisor throughout the FWSAR Project, including during industry consultations and one-on-one meetings with potential bidders. The Fairness Monitor provided feedback to the FWSAR Project in support of the objective of managing instances of potential inadvertent bias.
• Training for industry: Recognizing that some potential bidders have had more exposure to Canadian FWSAR activities than others, we understand that the Government of Canada offered training to potential bidders to promote consistent understanding of the requirements.

As procedures for industry consultations are finalized and feedback is assessed and responded to, it is important that processes employed and decisions made be well preserved to position the Government of Canada to respond promptly to future scrutiny.

5.1.2 Developing Evaluation Techniques

We observed the following tools and techniques developed and applied by the Government of Canada to promote an unbiased procurement approach:

• Tools to support a capability-based evaluation: The Government of Canada developed the Aircraft Performance Assessment Tool (APAT) to allow potential bidders to input their optimal basing and aircraft solutions and calculate the expected response times of their aircraft to a set of historical SAR incidents. Through our review, we observed that, while designing and refining the APAT, the Government of Canada sought to mitigate inadvertent bias toward one aircraft solution over another, which is consistent with the NRC's recommendations in its 2010 report. Subject matter specialists advised the Government of Canada on the design of the tool and on the reasonableness of underlying data and assumptions. The Government of Canada also strengthened the security controls around this tool following its draft release to industry.
• Empirical methods used to support weighting decisions: The Government of Canada undertook empirical methods to develop some aspects of the evaluation tools and techniques, including for the APAT incident data. In particular, a stability analysis of the supporting data was performed by a party external to the project. We also observed proactivity by the Government of Canada in selecting methods and establishing processes for weighting the prioritized aircraft acquisition requirements.
• Governance structures and processes: We observed meeting minutes, terms of reference and presentation decks for multiple oversight bodies in place to support oversight of the FWSAR Project. We anticipate that the direction of these oversight bodies contributes to the identification and proactive mitigation of possible risks related to bias.
• Involvement of experts and representatives: To support the FWSAR Project, the Government of Canada engaged a diverse group of specialists, which helped to identify and mitigate risks related to bias. These specialists included pilots, procurement specialists, costing experts, SAR technicians, engineers and diverse other experts from across DND, PWGSC and Industry Canada.

During our review, we observed that a discretionary clause was being developed for inclusion in the RFP to address clerical errors in the proposals of bidders. The discretionary clause is intended to be used as required by the Government of Canada to permit bidders to correct certain clerical errors in their proposals, thereby avoiding bidder disqualification and improving the likelihood of the procurement being successful. We observed that processes and protocols for administering the discretionary clause remained unclear at the time of writing our report. The Government of Canada could mitigate risks of perceived bias in its application of the discretionary clause by clearly documenting processes and protocols by which discretion will be exercised during bid evaluation.

Within the evaluation framework, the Government of Canada plans to employ new procurement methods for evaluating certain rated criteria, such as within the Capability Delivery Risk Evaluation (CDRE) and the Flight and Ground Test Assessment. During our review, we observed that processes and protocols for conducting on-site and field assessments were not yet finalized. The Government of Canada could mitigate the risks associated with bias during on-site and field assessments by establishing clear, documented protocols detailing the methods, tools and processes to be employed.

As the Government of Canada finalizes its bid evaluation plans, methods, processes and tools, continuing to strengthen records management, quality assurance and change management, including for the APAT and its input parameters, would position it to promptly respond to future scrutiny.

5.1.3 Validating Decisions

We observed the following decision validation methods applied by the Government of Canada to promote an unbiased procurement approach:

• Seeking expert reviews: The Government of Canada sought reviews from subject matter experts on design decisions made with respect to evaluation tools and techniques to be employed. These reviews included independent expert reviews on the APAT code and the APAT incident data and underlying assumptions, as well as examinations of sensitivity analysis performed on bid weightings. Multiple other reviews were performed on various aspects of the FWSAR Project, as outlined in Section 5.2.3.
• Involving experts in discussions: The Government of Canada involved subject matter experts throughout the FWSAR Project, including in consensus meetings and in the planned reviews of industry feedback.
• Involvement of a Fairness Monitor: As previously stated, the Government of Canada engaged a Fairness Monitor to provide independent and objective validation of decisions impacting the fairness of the FWSAR Project.
• Governance over decisions: We observed governance structures in place to review key decisions on the bid evaluation plans, methods, processes, decisions and tools. We anticipate that the advice of these oversight bodies contributes to the identification and proactive mitigation of possible risks related to bias.

We found that, for many of the reviews performed by and for the FWSAR Project, no clearly defined terms of reference were readily available. Retaining clear records and evidence of how decisions were validated through third-party reviews could help the Government of Canada manage the risks of knowledge loss and incomplete quality control.

While the FWSAR Project established specific wording to articulate the project objective in key documents, we noted that this wording was not consistently used across documentation and in the draft RFP of 30 August 2013. The Government of Canada could mitigate risks of misinterpretation and potential perceived bias by performing end-to-end quality assurance (QA) for the final documentation prior to its release to industry.

5.2 Comments on review observations related to the best value procurement objective

In framing our observations with respect to the best value procurement objective, we structured our observations in accordance with the four PWGSC "Smart Procurement" principles. These principles reach beyond the objective of achieving best value, also enforcing values of integrity, transparency, and openness. The Smart Procurement principles are as follows:

1. Early Engagement establishes a two-way conversation between industry and client departments to better understand both needs and available solutions. Early Engagement mitigates risk, identifies innovative solutions and enhances competition. In conducting early engagement, PWGSC connects buyers and suppliers through industry days, one-on-one supplier consultations and online collaboration tools.
2. Effective Governance sets ground rules for industry discussions and creates an effective framework for managing project decisions, risks and other issues.
3. Independent Advice sought during the procurement process, allows PWGSC to be informed by an objective third party (typically an industry expert) who assesses and validates decisions.
4. Smart Procurement Benefits all Canadians by weighing potential risks and opportunities for both Canada and all Canadians throughout the procurement process. In this way, Smart Procurement supports job creation in Canada, innovation, increased opportunities for suppliers (including small- and medium-sized businesses), and reduced barriers for Aboriginal Canadians.

PWGSC's Smart Procurement approach is designed to align to Government of Canada procurement policy, which seeks, among other objectives, to obtain best value for Canadians.^{Footnote 2}

We identified methods that are consistent with the objective of promoting a best value procurement approach and aligned with the Smart Procurement principles. While the Smart Procurement principles did not serve as the basis against which our review was conducted, we have used them as the structure against which we present our observations related to the objective of achieving best value. As explained earlier, the Smart Procurement principles reach beyond best value; our comments are limited to observations related to the objective of achieving best value.

5.2.1 Early Engagement

We observed that several iterations of industry engagement were initiated by the Government of Canada for the FWSAR Project at various points in the procurement process. We noted that this industry engagement was leveraged by the project to better inform estimates, assess technical feasibility, and identify areas requiring greater clarity in the RFP. The FWSAR industry engagement process included industry days, one-on-one supplier consultations and online notifications supporting the exchange of two-way written communication.

To manage the high volumes of industry feedback gathered through the engagement process, we observed heightened risk associated with records management and change management that should be managed closely by the Government of Canada as the procurement continues.

While we observed several practices in place to document industry correspondence, including one-on-one meetings, we did not observe a consistent level of detail in the meeting minutes and records of decision maintained for industry consultations. To promote knowledge transfer and position the Government of Canada in promptly responding to future scrutiny, it is important that communications with industry, as well as the rationale for all decisions, be well preserved. As the procurement process advances and the Government of Canada responds to the industry feedback received, it is important that responses to industry and the rationale for decisions made in reaction to industry feedback (e.g. changes to bid evaluation plans, methods, processes, and tools) be well documented. Specifically, decision rationale should demonstrate how the interdependencies between the various elements of the procurement approach were considered and weighed to achieve best value by the Government of Canada in advance of making changes. From the perspective of knowledge transfer, managing records and changes throughout the industry consultation approach can help inform future similar procurements.

5.2.2 Effective Governance

Governance structures and processes for the FWSAR Project were not within our scope; however, we did observe how governance impacted decision making as it relates to the elements of our scope. In this regard, we observed general consistency in the direction provided to the FWSAR Project in support of the development of evaluation plans, methods, processes and tools as well as the use of industry engagement.

We also observed the following attributes of effective governance during our review:

• responsiveness and openness to our information requests of DND and PWGSC as part of our independent review, demonstrating a strong commitment to transparency and thoroughness;
• within a complex governance structure involving several government departments, multiple governance bodies were instituted to guide decision making and oversee the project; and
• involvement of subject matter experts in the development and review of the bid evaluation plans, methods, processes, decisions and tools, including the APAT.

The wording used to communicate project objectives was refined over the course of the project. While a clear objective for this procurement has been established, it is not yet consistent across all documentation. To avoid potential misunderstandings that could impact decisions related to best value, it is important to enforce consistency in communication of objectives as this procurement advances. This risk could be reduced through quality control and monitoring processes for documents prior to dissemination and concerted internal communication of the procurement objective.

5.2.3 Independent Review

While other reviews were not re-performed or assessed for completeness, accuracy or alignment within the scope of our independent review, we observed and intersected with many other reviews being undertaken by subject matter specialists to advise on other aspects of the procurement. Also worthy of note is the National Research Council's independent review (2010) which served as the catalyst for a course change, redirecting the FWSAR Project toward an increased focus on capability-driven evaluation with a view toward achieving best value for Canadians. Some examples of other reviews being undertaken include:

• a Fairness Monitor is engaged to review documentation, attend one-on-one meetings with industry and provide objective feedback to the Government of Canada
• a "Red Team" review of the draft RFP was performed in advance of the 30 August 2013 release, with a team of 11 subject matter specialists from DND, PWGSC and Industry Canada;
• financial costing analysis was sought by DND's Chief Financial Officer (CFO) sector from an internal DND review function ("DCostS") and we understand this analysis will be continuing and culminating in a final review in 2014;
• external subject matter experts were engaged to review the coding of the APAT tool and the statistical relevance of the underlying SAR incident data as well as underlying assumptions;
• we understand that legal advice was sought by PWGSC from the Department of Justice on some aspects of the procurement; and
• we understand that a quality review of the draft RFP was sought from PWGSC's Contract Quality Control function.

We also observed a proactive approach by DND to seek independent advice, gather objective input to promote best value and implement recommendations from independent reviews. This observed proactivity helps to enable a more effective independent review process.

In addition to the observations above, we identified an opportunity to improve the clarity of the scope of independent reviews, including how they intersect with, and reinforce, one another. To promote efficiency and consistency of understanding among internal stakeholders, and ultimately maximize value, the Government of Canada should continue to strengthen the protocols for establishing, documenting and managing the scope of reviews, including retaining fulsome descriptions of the coverage achieved by quality reviews.

5.2.4 Benefit to all Canadians

Our review was not intended to include, nor did it include, an assessment of whether the procurement is designed to maximize benefit to all Canadians. Rather, in completing our review of the subset of bid evaluation plans, methods, processes, decisions and tools within scope, we considered the extent to which related decisions promoted the Government of Canada's objective of achieving benefits for Canada.

The FWSAR Project requires that all bids include a Canadian ISS integrator and methods intended to both meet Canada's Industrial and Regional Benefits Policy (IRB Policy)^{Footnote 3} and maximize the industrial benefits associated with the Canadian ISS provider(s). While assessing IRB Policy compliance and the evaluation of industrial benefits was outside the scope of our review, we observed that Industry Canada was actively involved in advising the FWSAR Project on the achievement of industrial benefits and the establishment of an IRB evaluation plan in support of intended outcomes. Further, we observed consistency between the use of the APAT to factor projected SAR incident response times in the evaluation of bidders and the NRC's recommendations to shift focus toward evaluating solution capability.

While outside of the scope of the FWSAR Project, we observed that limited SAR incident data was available to support procurement decisions. By acquiring and leveraging additional SAR incident data beyond historical SAR incident coordinates, the FWSAR Project could have, to a greater extent, supplemented expert opinion with data to support decision making.

Consistent with observations identified in other sections of our report, we observed instances in which the documentation supporting underlying plans, methods, processes, decisions and tools could be strengthened to enhance knowledge transfer within the FWSAR Project and to those responsible for future defence procurement initiatives.