Defence Resource Management Information System: Integrated support services and SIGMA integrated support services—Request for proposal 2

Fairness monitor final report, April 22, 2016

Addendum to final report, September 6, 2016

Submitted to Director, Fairness Monitoring Program

Submitted by PPI Consulting Limited

Defence Resource Management Information System: Integrated support services and SIGMA integrated support services—Request for proposal 2PDF version (412KB)
Alternative formats and plug-ins

On this page

1. Introduction

PPI Consulting Limited (PPI) was engaged by Public Services and Procurement Canada (PSPC) on October 12, 2012 as the Fairness Monitor (FM) contractor to observe the procurement process related to PSPC's acquisition of Support Services for the Department of National Defence (DND)'s Defence Resource Management Information System (DRMIS) and PSPC's SIGMA System.

PPI is an independent third party with respect to this activity.

We reviewed all of the information provided and observed all relevant activities.

We hereby submit our Final Report, covering the activities of the Fairness Monitor, commencing with the Letter of Interest/Request for Information process, Solicitation No: W8474-126279/A issued on November 16, 2012 continuing through an initial request for proposals, Solicitation number W8474-126279/D dated June 12, 2014, which was subsequently cancelled, and the re-issued request for proposals process, Solicitation No: W8474-126279/F issued on December 10, 2015 and subsequent evaluation consensus meetings.

This report includes our attestation of assurance, a summary of the scope and objectives of our assignment, the methodologies applied and relevant observations from the activities undertaken.

2. Project requirement

The Department of National Defence (DND) has an SAP-based integrated information system for the support of Materiel Acquisition and Support and Financial and Managerial Accounting business processes, as well as a number of other related processes. Implementation of this integrated information system involved the replacement of a large number of Materiel Acquisition and Support Information System (MASIS) and the Financial and Managerial Accounting System (FMAS). In April 2010, these two systems were merged to establish a single core SAP ECC 6.0 based solution, known as DRMIS. Further consolidation of business processes has occurred with the implementation of an SAP solution for supply chain functionality in DRMIS, as well as the incorporation of other business processes including those related to real property management.

As the Department continues to leverage DRMIS to meet evolving requirements, and as functionality is increased and organizational coverage is widened (for example, the integration into DRMIS of supply chain functionality and other business processes such as those related to real property management) it is expected that DRMIS will grow to over 30,000 users in the coming years. These future opportunities will allow the organization to achieve maximum benefit from its Enterprise Resource Planning (ERP) platforms.

PSPC has implemented an integrated system that supports its financial, procurement and real property business lines using SAP ECC, known as SIGMA which includes SAP Business Intelligence (BI, SAP Productivity Pak and other SAP tools and applications).

SIGMA is a complex SAP-based system. It was initially implemented in 2008 to meet departmental financial and material management requirements and is currently being used by over 6,200 users located within PSPC and in client departments.

The Department continues to leverage SIGMA to meet evolving requirements in order to achieve savings and operational efficiencies. With the integration of real property functionality into SIGMA, it is expected that SIGMA will grow to over 8,700 users in the coming years.

PSPC continues to identify opportunities that would allow the Department and the Government of Canada (GC) to realize maximum benefit from its ERP platform.

3. Attestation of assurance

It is our professional opinion that the solicitation process for the procurement of Support Services for DND’s Defence Resource Management Information System and PSPC’s SIGMA System, that we observed, was carried out in a fair, open and transparent manner.

Original signed by
John Davis
CEO
PPI Consulting Limited

Original signed by
Ian Brennan
CSCMP
FM Team Leader

Original signed by
Robert P. Tibbo
FM

4. Fairness monitor engagement and observations

The Proposal Evaluation Phase began when the request for proposal (RFP) solicitation period closed on the amended bid closing date of August 26, 2014. PWGSC received four (4) proposals by the closing date and time. The following activities were monitored by the Fairness Monitoring Team.

4.1. Request for information process

During the RFI Phase, the FM Specialist attended the following meetings:

  • Commercially Confidential Meetings with 10 interested suppliers (held July 16, 17, and 20, 2015)
  • Project team meetings/discussions to review status, evaluation approach, issues resolution, evaluation criteria, specifications etc

During the RFI Phase, the FM Specialist was provided with the following documents for review:

  • Notices of Proposed Procurement
  • Draft RFP documents
  • Correspondence with industry e.g. containing clarifications to questions raised during the Commercially Confidential Meetings
  • Draft Evaluation Plan
  • Industry Engagement Summary

Interested suppliers provided their responses to the draft RFP. Participation in the Industry consultation process was not a mandatory requirement and suppliers that did not participate in the consultation process were advised that they would remain eligible to submit a proposal in response to any future RFP relating to the DRMIS procurement.

The documents and activities completed in this phase complied with the fairness principles below:

  • All potential proponents were provided with the same opportunity and the same information at the same time
  • Potential proponents were afforded the same opportunity to provide feedback with respect to the requirements and proposed procurement approach
  • Potential proponents’ feedback received due consideration by the client based on an assessment against the client’s operational and business requirements
  • Requests for further explanation or clarification were provided to all participants

4.1.1. Fairness monitor specialist observations

The FM Specialist reviewed the draft solicitation documents and attended the Commercially Confidential Meetings with interested suppliers.

Any requests for clarification by the suppliers during the Commercially Confidential Meetings were addressed by the DND / PSPC project team and by the contracting authority either during the meetings or provided following the meetings. The same information was provided to all participants.

Any observations or requests for clarification made by the FM Specialist were addressed by the DND / PSPC project team and by the contracting authority.

Both the DND / PSPC project team and the contracting authority were responsive in providing clarification when requested by the FM Specialist.

4.2. Request for proposals process

An RFP for the DRMIS & SIGMA In-Service Support Services was initially posted on the buyandsell.gc.ca website as solicitation number W8474-126279/D on June 12, 2014 with a bid closing date of August 26, 2014. This RFP process was subsequently cancelled. The FM attests that all activities observed by the FM for this cancelled process were carried out in a fair, open and transparent manner.

An RFP for DRMIS and SIGMA Integrated Support Services was subsequently re-issued on the buyandsell.gc.ca website on December 10, 2015 as bid solicitation number W8474-126279/F, with a bid closing date of February 29, 2016. The re-issued RFP document replaced the previous version entirely.

During the re-issued RFP Phase, the FM Specialist attended the following meetings:

  • Project team meetings/discussions to review status, evaluation approach, issues resolution, evaluation criteria, specifications etc

During the RFP Phase, the FM Specialist was provided with the following documents for review:

  • Draft and final RFP documents
  • Correspondence with industry e.g. containing clarifications to questions raised during the Commercially Confidential Meetings
  • Evaluation Plan
  • Evaluator Briefing / Instructions to Evaluators

4.2.1. Bid open period

The RFP for DRMIS was issued on the buyandsell.gc.ca website as Solicitation No. W8474-126279/F, on December 10, 2015 with an original closing date of January 21, 2016. During the Bid Open Period 15 amendments were posted on the buyandsell.gc.ca website, responding to 74 questions from industry. In amendment 12 dated February 17, 2016, the closing date was extended to February 29, 2016.

Four proposals were received by the submission closing date and time.

4.2.2. Fairness monitor specialist observations

The FM Specialist reviewed the draft and final solicitation documents.

Any requests for clarification during the Bid Open Period were addressed by the DND / PSPC project team and by the contracting authority. The same information was provided to all Proponents by means of addenda posted on the buyandsell.gc.ca website. Both the DND / PSPC project team and the contracting authority were responsive in providing clarification when requested by the FM Specialist.

Any observations or requests for clarification made by the FM Specialist were addressed by the DND / PSPC project team and by the contracting authority. Both the DND / PSPC project team and the contracting authority were responsive in providing clarification when requested by the FM Specialist.

4.2.3. Technical bid evaluation

The FM Specialist attended the following meetings:

  • Evaluator Briefing / Kick-off meeting held March 2, 2016
  • Evaluation consensus meetings held March 16-23, 2016
  • Evaluation Peer Review meeting held April 5, 2016
  • Evaluation consensus follow-up meeting held April 6, 2016

The following process was undertaken to evaluate the bids:

  • Each evaluator conducted his/her own independent review and scoring of the proposal in his/her evaluator workbook
  • Once the independent reviews were completed, the evaluators participated in meetings to agree on the consensus assessment for both mandatory and rated requirements.
    • Technical mandatory requirements were reviewed for compliance with the information provided in both the compliance matrix and in the substantiating information in the proposal. A record of compliance or non-compliance was recorded in the master record along with the rationale
    • Each Bid’s responses to the rated requirements were reviewed separately, on an item-by-item basis against the identified evaluation criteria. For each requirement, the individual evaluator scores and the rationale for the scores were reviewed and discussed by the team. The evaluators used a consensus approach to determine a single score and rationale for each rated requirement that all evaluators agreed to. The consensus score and associated rationale was recorded in the master record
  • Evaluators recorded and initialled any changes agreed to during consensus in their individual workbooks
  • The FM confirmed with each member of the evaluation team that they were in agreement with the final consensus scores and rationale
  • All evaluators confirmed their agreement with the recorded scores and rationale
  • All evaluators confirmed that they had treated all four Bidders equitably and fairly
  • The evaluation team signed the consensus record

The FM’s review of documents and monitoring of activities in this phase considered the principles of fairness, openness and transparency. This included the following:

  • Evaluation Plan was consistent with the published procurement documents
  • Evaluation team members were chosen and confirmed prior to the receipt of proposals
  • Evaluation training was provided to all evaluators. This included informing evaluators of the following:
    • identity of the proposals received and requesting evaluators to declare any conflict of interest
    • confidentiality protocols
    • document control
    • clarification process
    • overview of scoring workbooks and method for individual assessment
    • explanation of the concept of hidden criteria
    • explanation of fairness and the need for objectivity, consistency and equitable treatment of all proposals
    • guidance with respect to only assessing the information that is in the proposal – not information from previous contracts, personal knowledge, or any other information
  • The scoring criteria and assessment tools were established prior to the receipt of proposals and were consistent with the RFP. The scoring criteria contained no hidden criteria
  • The submissions were logged and recorded upon receipt, clearly identifying that these were submitted on time
  • The pricing was contained in a separate envelope (as applicable)
  • The proposals complied with the mandatory submission requirements
  • The process for establishing one score from a team of evaluators was established prior to the receipt of proposals (consensus scoring was used)
  • The same team of evaluators evaluated all proposals
  • The scoring assessment was applied consistently and equitably by the evaluation team with no evidence of bias
  • A secure location for the evaluation exercise was established for the period of the evaluation
  • Proposal documents were physically secured within a secure location

All four proposals were deemed to have met the technical bid requirements of the RFP. The FM had no fairness concerns related to the technical bid evaluation.

4.2.4. Financial proposal assessment

PSPC had assigned a separate individual to complete the assessment of financial proposals. This individual had no contact with the technical evaluation team, and had no knowledge of the technical evaluation nor its results. Similarly, the PSPC contracting authority who attended the technical evaluation team meetings had no knowledge of the financial proposal assessment, nor its results.

The PSPC individual assigned to complete the assessment of financial proposals reviewed and verified the financial proposal of each bid. The individual verified that each financial proposal was complete and complied with the requirements of the RFP.

The PSPC individual completed the assessment of all financial proposals using the process defined in the RFP.

The FM met with the PSPC individual on April 4, 2016 to review and verify the financial proposal process and results. The FM was shown all financial spreadsheets and was able to verify that the process defined in the RFP had been used. The FM verified that the financial assessment results were consistent with the proper application of the process defined in the RFP.

The FM had no fairness concerns related to the assessment of financial proposals.

4.2.5. Fairness monitor specialist observations: Request for proposals process

The FM witnessed the technical bid evaluation meetings and confirms that the process was conducted appropriately in accordance with the process established in the RFP and in a fair, open and transparent manner.

The FM reviewed the assessment of financial proposals and confirms that the process was conducted appropriately in accordance with the process established in the RFP and in a fair, open and transparent manner.

The DND / PSPC team and the contracting authority were responsive in providing clarification when requested by the FM Specialist.

The evaluation teams were knowledgeable and diligent in their evaluation of proposals. All four proposals were evaluated consistently and fairly.

5. Reference documents

Documents pertaining to the request for proposals process for Support Services for DND’s DRMIS and PSPC’s SIGMA (Solicitation # W8474-126279/F) can be found on the buyandsell.gc.ca website.

6. Addendum to the final report, September 6, 2016

Addendum to fairness monitor final report dated April 22, 2016

This Addendum to the Fairness Monitor final report covers the period following the conclusion of the evaluation phase up to and including contract award.

Contract award

As a result of solicitation # W8474-126279/F for Defence Resource Management Information System – Integrated Support Services and SIGMA Integrated Support Services, of the four bidders, three successful bidders were selected and contracts were awarded on June 21, 2016. Of the four bidders, three requested and received a debriefing meeting including the one unsuccessful bidder. The fourth bidder requested and was provided debriefing comments via email. The FM reviewed all debriefing comments and attended the three debriefing meetings that were held in person.

It is our professional opinion that the Defence Resource Management Information System – Integrated Support Services and SIGMA Integrated Support Services procurement process that we observed, was carried out in a fair, open and transparent manner.

Original signed by
John Davis,
CEO,
PPI Consulting Limited

Original signed by
Ian Brennan,
CSCMP,
FM Team Leader

Original signed by
Robert P. Tibbo,
FM

Original signed by
Mairi Curran
FM Specialist (Backup)

Date modified: