Headquarters shelter systems

Fairness monitor final report, February 28, 2017

Addendum to final report, May 25, 2017

Submitted to Director, Fairness Monitoring Program

Submitted by Samson & Associates

On this page

1. Introduction

Samson & Associates was engaged as a Fairness Monitor (FM) on October 17, 2012 to observe the procurement process for the Headquarters Shelter Systems' (HQSS) requirement, issued by Public Services and Procurement Canada (PSPC) as solicitation #W8476-155245/A, on behalf of the Department of National Defence (DND). Samson & Associates is an independent third party with respect to this activity.

The initial Letter of Interest (LOI) and the first four amendments were issued prior to the involvement of Samson & Associates. We reviewed all of the information provided and observed all relevant activities from October 19, 2012 onward and had full access to all aspects of the procurement process. We had the full co-operation of all stakeholders.

We hereby submit the Final FM Report, covering the activities of the FM, commencing after the initial LOI posting and continuing through the Request for Proposal (RFP) evaluation phase.

This report includes our attestation of assurance, a summary of the scope and objectives of our assignment, the methodologies applied and relevant observations from the activities undertaken.

2. Attestation of assurance

The Fairness Monitor hereby provides the following unqualified assurance statement concerning the procurement process for the Headquarters Shelter Systems requirement, commencing after the initial Letter of Interest posting and continuing through the close of the Request for Proposal evaluation phase.

It is our professional opinion that the competitive acquisition process for the above activities we observed, was carried out in a fair, open, and transparent manner.

Original signed by
Mathieu Farley, Chartered Professional Accountant (CPA) auditor, Chartered Accountant (CA), Certified Internal Auditor (CIA),
Certified Information Systems Auditor (CISA)
FM Team Leader, Partner, Audit & Advisory Services
Samson & Associates

Original signed by
Ted Pender
FM Specialist

3. Project requirement

The HQSS project is aimed to dramatically improve the Canadian Forces' ability to plan, coordinate and command its forces during training and deployed operations. The HQSS project is to replace DND Tent Expandable Modular System (TEMS) with modular, tactically mobile, soft-walled shelter systems. HQSS may be used as tactical command posts, medical shelters and temporary accommodating units in austere environments.

This project includes integral environmental protection and auxiliary equipment such as heaters, air conditioners, vehicle connectors, semi-rigid flooring, internal tactical lighting, solar shades and, shelter insulation. It also calls for integrated logistics support to the HQSS over its lifecycle.

Included in the procurement is the provision for spares, ancillary equipment, publications, training, field service support and In-Service Support (ISS) for the HQSS. The requirement will result in the issuance of two contracts to one successful Bidder: the HQSS Acquisition Contract and the HQSS ISS Contract.

The period of the HQSS Acquisition Contract will be for an estimated four years. The period of the HQSS ISS Contract will be for five years, with four five-year option periods, for a total of 25 years.

4. Methodology

Samson & Associates was engaged as a FM to observe the competitive acquisition process for the HQSS requirement, and to attest to the fairness, openness and transparency of this monitored activity.

In accordance with the terms of our engagement, we familiarized ourselves with the relevant documents (note: the initial LOI and related first four amendments were issued prior to the involvement of Samson & Associates) and participated at the Industry Days and one-on-one meetings, and reviewed responses to submitted questions from potential bidders, identified fairness-related matters to the contracting and technical authorities, and ensured that actions were reasonable and appropriate.

4.1 Activities and observations: Initial letter of interest phase

The LOI for the HQSS was posted on MERX on August 17, 2012 and closed on March 28, 2013. There were 8 posted amendments, the last of which was posted on November 30, 2012.

Proponents were invited to attend an Industry Day, after which they were able to have a one-on-one meeting with the Technical Authority of DND. They could then submit their comments and suggestions for consideration.

The Industry Day was held on October 25, 2012. Thirty-two firms were in attendance. Four firms were denied attendance as they missed the deadline for registration or chose not to sign the required Industry Engagement Agreement. The CA and the FM attended the Industry Day.

In all there were 77 questions submitted during the process. The first 23 questions were submitted before the Industry Day and were addressed at the event. The remaining questions were addressed through amendments.

There were a total 13 LOI response packages submitted. One-on-one session events with industry to discuss their LOI responses were not conducted at this stage but were incorporated as part of the scheduled Draft RFP process.

Observation

The LOI document was written in a clear and understandable manner. All questions submitted were responded to in a clear and understandable fashion and posted on MERX. The FM participated at Industry Day and reviewed the summary results of the assessment of LOI responses/information submitted by proponents. As a result of respondent feedback, the start-up and operational temperature for the Tactical Lighting Fixtures went from -51 degrees Celsius to -40 degrees Celsius in order to promote competition.

The review of submissions/comments from proponents was carried out by seven DND reviewers and signed off by the Project Manager and the DND Procurement Authority. The FM and the PSPC CA reviewed the final summary of submissions/comments from proponents.

While no fairness issues were identified, all changes to the requirement specifications resulting from industry feedback was formally tracked with an associated rationale for acceptance or rejection of the proposed change in the draft RFP as part of the second LOI Phase.

Summary of Findings on the Initial LOI Phase: It is the opinion of the FM that the initial LOI Phase for the HQSS contract was conducted in a fair manner. In this context, fairness refers to decisions being made objectively, free from bias, favouritism or influence and conform to established rules.

4.2 Activities and observations: Second letter of interest phase

The Second LOI for the HQSS was posted on buyandsell.gc.ca on April 16, 2014 which included a draft RFP for comments. The initial closing date of July 15, 2014 was extended through amendments until March 3, 2015.

An Industry Day was held on May 26, 2014. Subsequently there were two streams of one-on-one commercially confidential meetings with proponents who wished to participate (May 26 to 30, 2014). One stream was to review the technical requirements and the other was to review the value proposition of the acquisition. Attendance at the event(s) was not mandatory and would not prevent suppliers from bidding on a possible future solicitation. Seventeen firms chose to participate in the one-on-ones. The FM attended the industry day and all one-on-one sessions.

There were 16 posted amendments, the last of which was posted on March 2, 2015.

Enquiries were to be submitted 15 days before the LOI closing date or they might not be answered. The responses to 423 questions and comments on the technical specifications and evaluation process were provided through the amendment process. The FM reviewed all questions and comments received, as well as the responses provided.

Observation

The Second LOI document was written in a clear and understandable manner. All questions submitted were responded to in a clear and understandable fashion and posted on buyandsell.gc.ca.

No fairness issues were identified.

Summary of Findings on the Second LOI Phase: It is the opinion of the FM that the Second LOI Phase for the HQSS requirement was conducted in a fair manner. In this context fairness is understood to encompass the elements of openness, competitiveness, transparency and compliance.

4.3 Activities and observations: Request for proposal phase

The RFP for the HQSS was posted on buyandsell.gc.ca on April 15, 2015. A Bidder's Conference was held on May 29, 2015 to provide an overview of the HQSS Project and requirement and give bidders an opportunity to ask clarification questions on the HQSS RFP documentation already posted on buyandsell.gc.ca.  The FM attended the Bidder's Conference. There were presentations made by each of the three departments involved in the acquisition process.

The FM reviewed the RFP prior to its posting as well as the presentations from each of the three departments prior to presentation at the Bidder's Conference.

The initial closing date of July 15, 2015 was extended through amendments until September 4, 2015.

There were 15 amendments issued, the last of which was posted on August 27, 2015.

Enquiries were to be submitted 10 days before the RFP closing date. The responses to 531 questions and comments on the technical specifications and evaluation process were provided through the amendment process. The FM reviewed all questions, answers and related amendments.

Observation

The RFP document was written in a clear and understandable manner. All questions submitted were responded to in a clear and understandable fashion and posted on buyandsell.gc.ca.

No fairness issues were identified.

Summary of Findings on the RFP Phase: It is the opinion of the FM that the RFP Phase for the HQSS requirement was conducted in a fair manner. In this context fairness is understood to encompass the elements of openness, competitiveness, transparency and compliance.

4.4 Activities and observations: Evaluation phase

Evaluation orientation meetings were held on September 11, 2015 (DND Technical evaluation) and September 21, 2015 (Industry Canada Value Proposition evaluation) with the evaluators to review roles and responsibilities. The CA and the FM were in attendance.

Canada carried out a two-phased bid evaluation process for this requirement. Phase 1 consisted of a two-step bid evaluation process. Step 1 was a review of the bidder's proposal and, if necessary, the release of a Preliminary Evaluation Report to all Bidders indicating whether or not they were compliant. If they were deemed non-compliant, they were invited to submit additional or different information. Step 2 was the evaluation of the proposals which included Bidders responses to the Preliminary Evaluation Report. For point rated criteria having a minimum pass standard, if the additional or different information provided by a Bidder rendered the criteria responsive, those criteria were deemed responsive. However, there was no change to the Bidder's original evaluated score for the purposes of determining the lowest cost per weighted point rating for financial evaluation purposes.

Bids were evaluated from a Technical perspective and had to meet mandatory criteria as well as point rated criteria. The technical criteria accounted for 80% of the available points. In parallel, bids were evaluated from a Value Proposition perspective on how their bid would benefit Canadian industry. Bidders had to meet several mandatory criteria as well as several point rated criteria. The Value Proposition accounted for 20% of the available points.

Phase 2 consisted of an HQSS Performance Verification Test. Only those Bidders whose bids were deemed to be responsive at the completion of Phase 1 were invited to participate in Phase 2.

Four bids were submitted in response to the RFP.

Bids were evaluated individually by the evaluators. The evaluators then met on several occasions to arrive at consensus on the results of the evaluation. The FM and the Contracting Authority were in attendance for the consensus discussions.

Based on the results of the initial evaluation, Preliminary Evaluation Reports were issued to all bidders and some additional information was provided by the proponents for further consideration. The FM reviewed the Preliminary Evaluation Reports prior to their distribution.

In addition there were individual requests for clarification submitted to some bidders, which were reviewed by the FM.

As a result of the final evaluation consensus meetings attended by the CA and the FM, which considered the responses to the Preliminary Evaluation Report process, as well as the responses to requests for clarification, all four proponents were deemed to have met the mandatory and minimum rated criteria stated in the RFP prior to the third party testing.

Subsequently, and in accordance with the RFP, bidders were required to submit sample shelter systems to a third party to undergo Performance Verification Testing. A total of 63 requirements (26 mandatory and 37 point rated) were tested. Testing was conducted from December 2015 until February 2016. The FM reviewed the testing plan and procedures in advance and was made aware of results throughout the testing period.

As a result of the third party testing, two bidders were deemed to meet the Phase 2 mandatory and minimum rated requirements stated in the RFP.

Bidders were aware of the testing results as they participated in and witnessed the testing and signed the test result sheets. The two unsuccessful bidders submitted objections to PSPC. After PSPC responded, denying the basis of their objections the two unsuccessful bidders submitted challenges to the Canadian International Trade Tribunal (CITT). The CITT determined that both bids had failed to meet the mandatory requirements stated in the RFP. The result of the last challenge under review was released by the CITT on October 8, 2016. The FM reviewed the published communications by the CITT.

Following the results of the CITT review, the final financial evaluation based on the criteria and methodology stated in the RFP determined the ranking of the proponents. A successful proponent was identified on October 8, 2016.

No fairness issues were identified.

Observation

The instructions and tools of the evaluators were complete, respected the terms of the RFP and were consistent with the principles and practices of fairness, and they were executed fairly. Evaluation criteria, weighting and points awarded complied fully with what had been established in the RFP. The evaluation was performed as described by the RFP and the scoring was performed in accordance with the criteria. Consensus discussions were open and a consensus was reached for all the criteria.

FM Summary of Findings: It is the opinion of the FM that the RFP Evaluation Phase for the HQSS requirement was conducted in a fair manner. In this context, fairness refers to decisions being made objectively, free from bias, favouritism or influence and conforming to established rules.

5. Reference documents

The following documents referenced by number in this report and unless otherwise indicated, are all available through PSPC.

Letter of interest documents by publication date
No. Documents Date
1 Letter of Interest (LOI) Published on MERX on August 17, 2012
2 Amendment no. 001 Published on MERX on September 19, 2012
3 Amendment no. 002 Published on MERX on October 5, 2012
4 Amendment no. 003 Published on MERX on October 5, 2012
5 Amendment no. 004 Published on MERX on October 19, 2012
6 Amendment no. 005 Published on MERX on November 2, 2012
7 Amendment no. 006 Published on MERX on November 8, 2012
8 Amendment no. 007 Published on MERX on November 23, 2012
9 Amendment no. 008 Published on MERX on November 30, 2012
Request for proposal documents by publication date
No. Documents Date
1 Letter of Interest (LOI) Published on buyandsell.gc.ca on April 16, 2014
2 Amendment no. 001 Published on buyandsell.gc.ca on April 17, 2014
3 Amendment no. 002 Published on buyandsell.gc.ca on May 8, 2014
4 Amendment no. 003 Published on buyandsell.gc.ca on May 12, 2014
5 Amendment no. 004 Published on buyandsell.gc.ca on May 16, 2014
6 Amendment no. 005 Published on buyandsell.gc.ca on June 5, 2014
7 Amendment no. 006 Published on buyandsell.gc.ca on July 29, 2014
8 Amendment no. 007 Published on buyandsell.gc.ca on August 29, 2014
9 Amendment no. 008 Published on buyandsell.gc.ca on September 29, 2014
10 Amendment no. 009 Published on buyandsell.gc.ca on October 3, 2014
11 Amendment no. 010 Published on buyandsell.gc.ca on October 15, 2014
12 Amendment no. 011 Published on buyandsell.gc.ca on November 19, 2014
13 Amendment no. 012 Published on buyandsell.gc.ca on December 10, 2014
14 Amendment no. 013 Published on buyandsell.gc.ca on December 16, 2014
15 Amendment no. 014 Published on buyandsell.gc.ca on January 14, 2015
16 Amendment no. 015 Published on buyandsell.gc.ca on February 13, 2015
17 Amendment no. 016 Published on buyandsell.gc.ca on March 2, 2015
Request for proposal 2015 documents by publication date
No. Documents Date
1 RFP published Published on buyandsell.gc.ca on April 15, 2015
2 Amendment no. 001 Published on buyandsell.gc.ca on April 23, 2015
3 Amendment no. 002 Published on buyandsell.gc.ca on May 8, 2015
4 Amendment no. 003 Published on buyandsell.gc.ca on May 21, 2015
5 Amendment no. 004 Published on buyandsell.gc.ca on May 27, 2015
6 Amendment no. 005 Published on buyandsell.gc.ca on June 4, 2015
7 Amendment no. 006 Published on buyandsell.gc.ca on June 12, 2015
8 Amendment no. 007 Published on buyandsell.gc.ca on June 26, 2015
9 Amendment no. 008 Published on buyandsell.gc.ca on July 9, 2015
10 Amendment no. 009 Published on buyandsell.gc.ca on July 16, 2015
11 Amendment no. 010 Published on buyandsell.gc.ca on July 23, 2015
12 Amendment no. 011 Published on buyandsell.gc.ca on July 30, 2015
13 Amendment no. 012 Published on buyandsell.gc.ca on August 10, 2015
14 Amendment no. 013 Published on buyandsell.gc.ca on August 20, 2015
15 Amendment no. 014 Published on buyandsell.gc.ca on August 24, 2015
16 Amendment no. 015 Published on buyandsell.gc.ca on August 27, 2015

6. Addendum to final report, May 25, 2017

Addendum to fairness monitor final report dated February 28, 2017, on the competitive selection process for the Headquarters shelter system.

This Addendum to the FM Final Report covers the period following the conclusion of the evaluation phase.

Fairness monitor activities related to contract award and debriefings

There were four bids submitted in response to the RFP. Two bids were deemed responsive. The successful bidder was announced and the contract awarded on January 13, 2017. On the same day the successful bidder was provided with a confirming letter, which included their total score and cost per point.

Concurrently each unsuccessful bidder was provided with a letter, which included their total score and cost per point as well as the total score and cost per point of the successful bidder. The FM reviewed all communications to bidders.

On March 13, 2017, each unsuccessful bidder was provided with a written debrief letter. The successful bidder received a similar letter on March 17, 2017.

The debriefing process concluded March 28, 2017. The FM reviewed all debriefing communications and no FM issues were noted.

Fairness monitor attestation of assurance

The FM hereby provides the following unqualified assurance statement concerning the competitive selection process for Headquarters Shelter System.

It is our professional opinion that the competitive selection process that we observed, was carried out in a fair, open and transparent manner.

Original signed by
Mathieu Farley, CPA auditor, CA, CIA, CISA
FM Team Leader, Partner, Audit & Advisory
Samson & Associates

Original signed by
Ted Pender
FM Specialist

Date modified: