Annex C: Benchmark Testing

National Master Standing Offer
EZ107-060001/001/VAN to EZ107-060001/006/VAN
Desktop and Network Printers   including Accessories, Supplies and Services

Annex C: Benchmark Testing

1. Overview

Benchmark tests on offered equipment are mandatory and must be conducted as part of the evaluation. The Benchmark tests must be conducted by the testing authority, NSTL Canada Corp. (NSTL), at the NSTL test site located in the National Capital Area. The benchmark procedures have been established in cooperation with NSTL. All questions and concerns about these procedures must be addressed to the Contracting Authority. Failure to comply with the benchmark test procedures may result in disqualification of an Offer without further notice.

2. Nature of Testing

The benchmark testing procedure is not a pass/fail evaluation, unless the benchmark testing reveals that a printer does not meet a mandatory requirement of the solicitation. It is used to generate a score based on the performance (speed), quality of output, features and usability of the printer. This score is used in the determination of Value Factor. The data generated through the benchmark testing will be used for evaluation and posted on the IPMG site.

3. Benchmark Test Schedule

Canada will submit a list of products to be benchmarked to NSTL. NSTL, in cooperation with the PWGSC Contracting Authority, will then draw up the Benchmark Test Schedule. Offerors will be notified by e-mail and/or facsimile of the Benchmark Test Schedule, at least seven (7) calendar days in advance. The notification will include the schedule of delivery dates for each Product to be tested, and detailed instructions regarding delivery of test units, supplies, documents and payments required for the benchmark tests. Failure to deliver the equipment and other items on the specified date(s) and time will result in disqualification if NSTL, in its sole discretion, determines that extending the deadline will have an impact on the overall testing schedule.

4. Use of Previous Test Data for Evaluation of this RFSO

If a printer offered under this RFSO has previously been tested by NSTL for other requirements (including the NMSO#EZ107-060001/001/VAN to EZ107-060001/006/VAN030001/XXX/VIC series of Offers), any resulting test data may be used if NSTL determines that the data applies to this requirement. Decisions by NSTL will be final in this respect. Although an administration fee will be required to cover the costs of recompiling and posting the data for the new requirement, this fee will not exceed the benchmark test fee identified below for the relevant Sub-Category. It is the Offeror's responsibility (at the time testing is scheduled with NSTL) to indicate where a printer has already been tested by NSTL so that the previous test data can be verified and testing for any missing data can be scheduled.

5. Equipment Delivered for Testing

(a) Equipment delivered for testing should be accompanied by:

(i) full payment for all tests;

(ii) the completed NSTL Features Questionnaire (available through the Internet before the test period begins);

(iii) a qualified technician to set up and configure the equipment; and

(iv) a complete set of the user manuals normally provided by the Offeror or the OEM, in both English and French (or bilingual), both for verification and for use during the testing.

(b) Each printer should be delivered with a 10' cable for the Centronics parallel interface. Printers that have both a USB and a parallel interface should be delivered with USB and parallel cables. USB-only devices must incorporate a USB port and include a USB cable.

(c) The Offeror should also supply two (2) sets of appropriate imaging consumables (not including paper).

(d) Printers submitted into network categories should include a card and cable for Ethernet 10/100 or 100 Base T.

(e) Any Products delivered without all of the above may not be tested by NSTL, in which case they will be disqualified as non-compliant.

(f) Each Product supplied for benchmark testing must meet or exceed the Technical Specifications in Annex A and the Product Details and must be identical to the Product described in the Offer.

6. Installation and Configuration for Benchmark Testing

(a) It is the Offeror's responsibility to install, set up and test the equipment at the NSTL testing site before benchmark testing.

(b) It is the Offeror's responsibility to ensure that equipment is delivered to the Benchmark test site on the appointed date by 2PM EST.

(c) Offerors setting up more than 1 machine are expected to bring sufficient staff to complete all set-ups according to the following guidelines:

(i) 4 hours for up to 8 machines;

(ii) 8 hours for up to 16 machines; and

(iii) 12 hours for over 16 machines.

(d) Failure to set up the Products within these time frames will result in disqualification if NSTL, in its sole discretion, determines that extending the deadline will have an impact on the overall testing schedule.

7. Inadequate or Malfunctioning Equipment

(a) If the equipment set up for benchmark testing does not function in compliance with the Technical Specifications in Annex A and/or the technical specifications described in the Offer, or is found to be different from the printer described in the Offer, it will be removed, and a new appointment will be made to deliver an operational and/or compliant printer within 2 working days.

(b) If a printer is unable to complete benchmark tasks with the offered memory and additional memory is required, the Offeror will be given an opportunity to increase the memory for further benchmark testing within 2 working days. If the printer completes benchmark testing with the increased memory, the Offeror must furnish this added memory for all Call-ups for the printer without increasing the price (if it is listed in a resulting Standing Offer) or withdraw the printer from consideration.

(c) If, at the end of this 2-working-day period:

(i) the equipment or its replacement exhibits the same fault a second time;

(ii) the equipment or its replacement exhibits another fault;

(iii) the addition of memory does not allow the printer to function in accordance with the Technical Specifications; or

(iv) the Offeror has failed to make the repair or submit the replacement or additional memory, the equipment will be eliminated from further consideration in all relevant Sub-Categories.

(d) Upon request, NSTL will demonstrate any fault found during the benchmark tests.

8. Return of Equipment after Testing

Equipment provided for testing may be required for up to 60 working days. After the Product has been tested, NSTL will contact the Offeror with a date and time for retrieving the Product from the test site. The Offeror must retrieve the Product from NSTL within an hour of the appointed date and time. Offerors are strongly advised to inform NSTL as soon as they become aware that they might not be able to make an appointment to deliver or retrieve printers.

9. Contact Information

Offerors may contact the following NSTL contact (collocated at PWGSC) for any matters concerning the logistics of benchmark testing (scheduling, etc.):

NSTL Canada Corp.
c/o PWGSC/STAMS
11 Laurier Street, Place du Portage, 4C2
Gatineau, PQ K1A 0S5

Contact: Jon Drummond
jon.drummond@pwgsc.gc.ca
Phone: 819.956.8355
Facsimile: 819.956.1156

10. Benchmark test fees

Benchmark tests must be conducted by NSTL Canada Corp. NSTL is not part of the Government of Canada. The Offeror may be required to enter into an agreement with NSTL regarding the testing. All costs associated with the benchmark testing are the Offeror's responsibility. The cost for benchmark testing each printer will be in accordance with the Schedule of Benchmark Costs identified below, and is payable by certified cheque made out to NSTL. The fees below will apply during initial evaluation, but may be subject to change during the Standing Offer Period. The fees below include GST.

A certified cheque in the amount required must be delivered to NSTL before benchmark testing commences. Offerors submitting more than one printer may present one cheque for all printers or one cheque for each group of printers submitted for testing. Printers that are not paid for within 24 hours of set-up will not be tested.

Group 1: Core Printers - duplex standard

Group 2: Specialty Printers - duplex standard

Exceptions:

(a) Products offered in the Monochrome Network Laser Printer Category that are also offered in any Monochrome Standalone Laser Printer Sub-Categories will be subject to an additional cost of $901.00/unit per additional Sub-Category tested (instead of the Sub-Category fee set out above).

(b) Products offered in Network Colour Page Sub-Categories that are also offered in any Standalone Colour Page Printer Sub-Categories will be subject to an additional cost of $901.00/unit per Sub-Category tested (instead of the Sub-Category fee set out above).

(c) Where an otherwise identical printer is tested in a Multifunction Sub-Category, then in a Network Sub-Category, and then finally as a standalone printer, the Offeror will first pay the benchmark test fee for the multifunction printer Sub-Category, then an additional $1060.00/unit test fee for the network Sub-Category, and finally a $901.00/unit test fee for the standalone Sub-Category (instead of the Sub-Category fees set out above).

(d) For any printer model being tested for more than one Sub-Category, if the model is configured identically in each Sub-Category for which it is offered (so that the benchmark tests can be run just once) to obtain data for all Sub-Categories, the benchmark fee for that printer will only be payable once, with an additional administrative fee of $530.00 per additional Sub-Category in which it is offered.