Consultant and Contractor Performance Evaluations
The Real Property Branch (RPB) and Real Property Contracting (RPC) in conjunction with the Canadian Construction Association and the Professional Associations of architects and engineers have established a generic framework with specific evaluation criteria to objectively assess the quality of services provided by consultants and contractors on federal real property projects in an effort to:
- improve the effectiveness of project delivery and client service at reasonable cost;
- ensure that the firms with which RPB contracts perform their obligations in a fully satisfactory manner;
- demonstrate to firms that there are consequences to poor performance;
- improve the performance of consultants and contractors on real property projects;
- ensure to the fullest extent possible the use of qualified consultants and contractors;
- establish and maintain a continuous record of consultant and contractor performance for the purpose of recognition and penalties.
Performance evaluations are concerned with both the quality of project deliverables and the manner in which they are delivered. They are critical to the success of real property procurement. The results of performance evaluations may affect a firm's eligibility for future real property contracts, therefore they must be completed promptly and accurately to reflect the work that was completed. It is the responsibility of RPC to ensure that the Consultant and Contractor Performance Evaluation Forms developed for this purpose are completed by project managers (CPERF 2913 and 2913-1 available in ELF).
A Contract Performance Review Committee has been established to manage the contract performance evaluation process. The evaluation reports must be completed for all real property contracts regardless of value. RPB is responsible through its PM's and in-house design managers to ensure that all consultants and contractors are held responsible to provide quality services and products.
Performance Evaluation Criteria
A firm's performance is assessed for both 'product' and 'services' through the use of generic performance evaluation forms. Quality of the 'product' assesses the quality of the completed facility/project in terms of how successfully it fulfills the stated objectives for the project. In the case of studies and briefs, where there is no built work involved, the product refers to the actual work produced by a consultant. Quality of 'services' assesses the firm's ability to manage and deliver professional services in a comprehensive, effective, reliable and timely manner throughout each stage of the project. Individual evaluation criterion are pre-established and are as detailed on the CPERF.
To evaluate consultant and contractor performance, five generic criteria have been established for each group. These criteria apply to the majority of projects. A detailed listing of ratings within each of the five criteria can be found on the adjoining pages of the appropriate CPERF. A weight factor of 20 points is assigned to each of the five main criteria. On some consultant contracts, not all criteria are necessarily evaluated as they may not be applicable to the specific work being performed. Non-applicable evaluation criteria are assigned a weighing factor of zero and the SELECT system calculates the assessment on a percentage (pro-rated) basis. The Instruction and Additional Information Sheet(s) of the CPERF forms provide more explanation related to these criteria.
Performance Evaluation Criteria for Consultants
The following evaluation criteria have been established for evaluating consultant performance and apply to the majority of consultant contracts. Three criteria are optional, two are mandatory.
- Quality of designFootnote 1
- Quality of the results of the deliverables throughout each stage of the project
- Management of the project including the project delivery and the overall services
- Time planning and schedule controlFootnote 1
- Quality of cost planning and control during the life of the project
This table is a grid that is used to evaluate the performance of the consultant.
|Points||Design - Conception||Quality of Results||Management||TimeNote 2||Cost|
|6-10||Not Satisfactory||Not Satisfactory||Not Satisfactory||Late||Not Satisfactory|
|17-20||Superior||Superior||Superior||Ahead of Schedule||Superior|
Performance Evaluation Criteria for Contractors
The following evaluation criteria have been established for evaluating contractor performance and apply to the majority of construction contracts. Two criteria are optional, three are mandatory.
- Quality of WorkmanshipFootnote 1
- TimeFootnote 1
- Project Management
- Contract Management
- Health and SafetyFootnote 1
This table is a grid that is used to evaluate the performance of the contractor.
When evaluating performance for work scheduled on a daily or weekly basis, (e.g.: for building cleaning/maintenance services), points should be deducted from the quality of workmanship criterion for those instances where a particular task is not performed. However, judgment and prudence must be well exercised. If a firm's employee was absent for a legitimate reason and the firm made a reasonable effort but was unable to perform the work, then an incomplete task may be ignored when completing the CPERF. If a firm is late in completing less frequently scheduled portions of work (monthly, quarterly, semi-annually) then points should be deducted from the time criterion. Again, good judgment must be exercised as the firm may appeal the rating given.
Delays caused by sub-contractors/consultants are the prime consultant/general contractor's responsibility. However, if the latter has taken all possible action to expedite the sub-consultant/sub-contractor's work, the effectiveness of this effort should be considered when evaluating the firm's performance.
The time criterion deals with the firm's performance up to the actual completion date. Typically, on small contracts the actual completion date is the date on which the work is 100% complete. However, where the contract provides for the issuance of an Interim Certificate of Completion, the completion date is the date shown on the Interim Certificate. In this case, the firm's performance on the clean-up of deficiencies, for example, should be taken into account under the criterion 'project management'.
Some questions to consider when assessing the 'time' criterion (not an exhaustive list):
- Did Public Services and Procurement Canada (PSPC) contribute to the delays?
- Was appropriate follow-up action and monitoring done by the PM?
- Was the firm notified of concerns on an ongoing basis?
- Were the delays beyond the control of the firm or were there any extenuating circumstances that need to be considered?
- Did the prime appropriately deal with subs?
- Can the PM demonstrate that he/she has exercised good judgment at all times and that the project records can justify their position?
Performance Evaluation Process
A CPERF is required to be completed for all real property contracts regardless of the dollar value. The evaluation supplements and reinforces standard written communications that a PM would initiate when confronted with unsatisfactory performance. It is essential that PMs gather supporting/reference material throughout the course of the project. The PM must diligently document all instances of performance other than 'satisfactory' or 'on time'. This is particularly important when performance is less than satisfactory. The success of the system depends upon the quality of each evaluation and the accuracy of its supporting material. The PM must strive for an evaluation that is based on facts directly related to the project and which is fair to both the firm and PSPC. The impact of scope creep on the schedule must not be underestimated.
During the execution of the work, the firm is to be notified immediately, in writing, if the work is not proceeding in a satisfactory manner. This notification would normally be further documented in job meeting records, or in the PM job diary. The firm can be directed to provide an action plan detailing a remedy to the situation. The PM would take notes and stay abreast of the overall performance of both the firm and the department as the job progresses. Quantification of documents is discretionary. One might assume that at least two warning letters are provided prior to corrective action being taken. PM's should contact RPC to devise contractual strategies.
Roles and Responsibilities
Project Manager (PM):
- prepares and signs a CPERF at the time of the issuance of payment of the final invoice (for the majority of projects). However, in certain cases, particularly dealing with poor performance, the CPERF may be completed prior to completion of the work, i.e.:
- prior to completion of the work an interim CPERF can be issued as required for cases of clear lack of performance, including inordinate delay in the clean-up of project deficiencies. An interim CPERF carries the same weight as a final CPERF.
- at specific key stages in the project delivery process as pre-established in the proposal and contract documents (e.g. for consultants this could be at the 33%, 66%, and 99% completion of design and construction documents).
- at any time an interim CPERF is requested to be issued by the Contract Performance Review Committee (CPRC).
- in exceptional cases, such as: at the time the work is taken out of the consultant/ contractor's hands, termination of contract, abandonment of the work, etc.
- ensures the SPM initials all CPERFs which have a rating other than "satisfactory" or "on time".
- where (RPC) awarded the original contract, forwards the completed CPERF to the appropriate Contracting Officer (CO) for entry into the system.
- where RPB awarded the original contract, and has direct access to SELECT, completes the CPERF and enters it into the system or when no direct access is available, forwards the completed CPERF to the SELECT Business Manager for entry into the system.
- forwards the completed CPERF to the applicable CO for entry into the system for all other contracts which were entered outside of the SELECT system.
NOTE: When completion of a project is delayed by project circumstances, a CPERF may be completed prior to final completion if the final costs and the results of the performance review can be forecast with a high degree of confidence. Typically, this would occur at substantial completion of a project.
Contracting Officer (CO):
- advises PM on all aspects of the CPERF process, particularly non-satisfactory evaluations.
- enters the CPERF directly into the SELECT system where PM does not have direct access for all SELECT contracts issued by the CO.
- forwards the completed CPERF to the SELECT Business Manager for all non SELECT real property contracts issued by the CO.
SELECT Business Manager (BM):
- forwards all CPERF's rated as 'satisfactory' with a covering letter to the firm advising them of their evaluation. The letter includes a notice to the firm that they may rebut the evaluation and/or related consequences within a specified period. (Where the CPERF was entered in SELECT, the system will automatically generate a standard letter for the BM's signature; for all others the BM will draft the appropriate letter).
- reviews all CPERF's rated as 'non-satisfactory'. Where the evaluation is borderline, the BM may decide to review the evaluation with the PM. A decision may be taken to improve the score to a 'satisfactory' rating based upon several factors including, but not limited to: the PM's assessment, the firm's previous performance, the project specifics, market considerations, time and cost considerations, implications upon the delivery of the project, etc. In cases where the PM and BM agree to amend a borderline evaluation from a 'non-satisfactory' to 'satisfactory' level, the revised CPERF is forwarded to the firm. The letter accompanying the CPERF includes a notice to the firm that they may rebut the evaluation and/or related consequences within a specified period.
- ensures ratings and consequences are/have been recorded in SELECT
- reviews and forwards all CPERF's rated as 'superior', 'unacceptable' or 'non-satisfactory' to the Contract Performance Review Committee for action.
- receives and acknowledges rebuttals from firms contesting their rating. Refers rebuttals to the PM and if no resolution can be obtained, to the CPRC.
Contract Performance Review Committee:
- ensures due diligence in the management of private sector contract performance from a variety of points of view and solicits recommendations from Committee members and other advisors to reward or suspend bidding privileges of private sector firms based on their actual performance
- reviews all CPERF's rated as 'superior', 'unacceptable' or 'non-satisfactory'and determines appropriate action, ratings and consequences to be taken based on recommended actions incorporated for use in SELECT (see chart below.)
- chairperson forwards these CPERF's with the appropriate covering letter to the firms following the deliberations of the Committee advising firms of the Committee's decisions. The letter includes a notice to the firm that they may rebut the evaluation and/or related consequences within a specified period.
- ensures ratings and consequences are recorded in the SELECT system.
- chairperson reviews all rebuttals with a view to obtaining agreement on the performance of the firm.
Contract Performance Evaluation Ratings and Consequences
This table provides recommended actions to be carried out based on a firm's contract performance evaluation overall rating.
|Overall Rating||Recommended Action|
|85% or higher - (Superior)||Reward
If an opportunity point was assigned in SELECT for the contract, the number of opportunity points assigned to the firm in SELECT is reduced by one. Contracts outside of SELECT do not have a mechanism for reward.
|51% to 84% - (Satisfactory)||No action. A standard letter is issued|
|30% to 50%Footnote 3
The letter sent with the CPERF to the firm indicates that the next time they receive 50% or less they may be suspended.
|Less than 30%Footnote 4 -
Firm suspended to all rotational and Real Property public tenders for one year
Contract Performance Review Committee (CPRC)
Each regional office has a CPRC to manage the performance evaluation process. Since the SELECT system is utilized as a repository for the registration of individual local firms within each region, there is no need for the establishment of any permanent national committee for such purposes. However, in cases where one region is considering suspending a firm that has offices in another region(s), they are strongly advised to contact the applicable region(s), prior to the suspension, in order to ensure that one region's action is not undermining another region's position. An ad hoc inter-regional CPRC may need to be established in certain cases.
The CPRC advises and supports the RPB and RPC in the review of consultant and contractor performance in the delivery of real property projects. The Committee manages the day-to-day internal processing and reporting of consultant and contractor performance evaluations, the interface with industry including the rebuttal process and the establishment and enforcement of penalties levied against non-satisfactory consultant and contractor performance.
Committee Membership (NCA sample)
- Chairperson: Manager of Regional Operations and Commodity Management
- Secretary: Regional SELECT Business Manager
- RPB representative: Applicable Senior Project Manager
- RPB representative: Head, Claims Prevention and Management
- RPCD representative: Applicable Manager Construction/Maintenance/A&E
- Defense Construction Canada: Applicable Manager
- Others: The presence of various other participants may be required from time-to-time to provide advice, or to present or justify a particular position. Participants may include the PM, a firm's representative, other specialists from a wide variety of internal, professional or technical disciplines, industry representatives, client representative etc. Selection is at the discretion of the Chair and such participants may only attend that portion of the meeting required to present their opinion.
The CPRC is primarily concerned with managing and addressing private sector performance on real property projects. The existence of this Committee provides for an effective separation between the contracting parties to help ensure impartiality, integrity and transparency. The Committee typically deals with the following issues:
When reviewing a firm's performance, the firm's overall performance record on previous real property projects should be considered before determining the consequences of the evaluation. The firm's current project history should also be considered.
Superior performers are recognized by issuing a letter of acknowledgment. If the contract in question had resulted in the firm being assigned an opportunity point in SELECT then the firm may be rewarded by having this opportunity point removed. This will improve the likelihood of the firm being invited to bid on future rotational selections.
Rebuttals are reviewed with a view to obtaining agreement on the performance of the firm. This will usually involve receiving input from the PM as well as the firm. If disagreement remains, the firm's views are retained on file. Where appropriate, the final CPERF ratings and consequences are to be recorded in the SELECT system. The rating and consequences are also entered into the firm's hard-copy record as well as into the project and contract files.
The CPERF contains "protected" information and cannot be disclosed to the public, including other firms, unless the firm concerned agrees in writing to the release of the information. The completed CPERF can be made available to officials of other government departments on a "protected" basis. The completed CPERF must always be provided to the firm being evaluated.
Suspension letters are issued identifying the reasons why the firms are suspended, the duration of the suspension, and the consequences of being suspended. It is important to ensure that the suspended firms are not awarded any real property contracts, regardless of value, during the suspension period.
Once a suspension period has elapsed a firm wishing to be considered for rotational selections must re-apply to SELECT. In the case of construction or maintenance firms, references provided in the new application must be for contracts completed since the start of the suspension period.
Meeting Location and Frequency
Meetings are held on an ad hoc basis at the call of the Chairperson
- Maintains a record of decisions and follow up as required
- Prepares and distributes minutes within a one week period following each meeting to all Committee members
- Footnote 1
Mandatory evaluations required - footnote 1
- Footnote 2
If timely completion is critical, the 'unacceptable' performance rating can be defined as zero overrun, or a maximum acceptable overrun established. The time requirement would have to have been clearly identified in the solicitation and contract documents, and should have been emphasized to the firm as the work/services progressed. The normal risks associated with contracting are not to be considered as causes beyond the firm's control. Note: The time requirements are clearly identified in the award letter, the articles of agreement and section 01 00 10 of the specifications. Furthermore, General Conditions clause C7 of the contract states that 'time is of the essence of the contract', so by definition, timely completion is critical. The CPERF's adjoining pages give good indications of what should be considered when evaluating the time aspect, especially when the contract is not completed on time.
- Footnote 3
OR 50% or less (if firm has already been sent a warning letter within the last 2 years).
- Footnote 4
OR 5 or less on any one criterion.
- Date modified: