Quality Management - Knowledge Area
National Project Management System
Quality Management Knowledge Area
1. EFFECTIVE DATE:
This policy related document is issued under the authority of the Deputy Minister, Public Services and Procurement Canada (PSPC).
This knowledge Area is to be implemented in conjunction with the PSPC National Project Management System (NPMS) Policy.
The purpose of this document is to describe the components and requirements of the NPMS as applied the quality management of Business Projects-IT-Enabled.
In October 2008 a departmental definition of quality and a quality framework were approved.
The PSPC definition of quality is a departmental offering (product or service) with characteristics that consistently:
- meet or exceed client requirements;
- comply with legislative/regulatory/policy requirements; and
- meet established standards.
PSPC's Quality Management (QM) Framework consists of seven interrelated elements that are required to become a quality driven organization. They are as follows:
- People Focus
- Partner / Supplier Focus
- Process Based
- Organizational Performance
Descriptions of these elements are provided in Annex D - The PSPC Quality Management Framework.
This quality management lens has been integrated into the Quality Management Knowledge Area as well as the other NPMS knowledge areas. It is applied to all processes, products and services. Quality management is the vehicle through which the department ensures successful repeatability in project delivery and continuous improvement of its project management framework, policies, objectives, standards, responsibilities, practices and procedures. Quality management also allows organizations to reinforce a corporate culture of service quality, so that their employees, suppliers and partners will endeavour to achieve and maintain high client satisfaction and to implement a philosophy of continuous improvement.
With respect to NPMS and PMBOK®, "Quality Management is the identification of the Quality Assurance and Control process to ensure that the end product of the project will satisfy the need for which it was intended." At a project level, the quality management process involves the development and implementation of policies, objectives, standards, responsibilities and procedures to ensure that the products and deliverables produced by the project meet quality standards. Quality management applies to both project deliverables and project work processes. Plan Quality is the process of identifying quality requirements and / or standards for the project and product, and demonstrating how the project will demonstrate compliance.
Perform Quality Assurance is the process of auditing the quality requirements and the results from quality control measurements to ensure quality standards and operational definitions are used.
Perform Quality Control is the process of monitoring and recording the results of executing the quality activities to assess performance and recommend necessary changes.
The purpose of the Project Quality Management process is to ensure that:
- quality planning activities are conducted and result in the required metrics, tools, standards and processes that will be used by the project to attain its quality objectives;
- products are built to agreed-upon standards and requirements;
- products and processes are built in accordance with all relevant client, project, product development, and branch, departmental, GC governance review and approval gating criteria;
- work processes are performed efficiently and are documented;
- non-conformances are identified and appropriate corrective action is taken;
- necessary quality documentation is prepared and retained as part of the project record; and
- project delivery program and NPMS Business Project-IT-Enabled deliverables and processes are subject to continuous quality improvement.
Please refer to Annex A - Quality Management Process, Annex B - Quality Forums, Annex C - Quality Control Activities and Annex D - The PSPC Quality Management Framework for additional information on Quality Management.
Quality Management Process by NPMS Stage and Phase
Quality management during the Inception and Identification Stages means ensuring the quality of the Identification Stage processes and deliverables.
The project conducts a preliminary assessment and documents a business need or opportunity in the form of a Statement of Requirements (SoR) document. A high-level statement regarding quality requirements may be included.
Provide a high-level preliminary quality statement in the Preliminary Project Plan (PPP) along with a set of proposed milestones and deliverables. During Initiation, projects define the business requirements and may document a high-level target business vision that incorporates quality. The target business vision and the business requirements are the framework upon which quality requirements are built.
Projects may conduct an environmental scan to review available technical solutions to the business problem and to assist in the feasibility assessment process. The Feasibility Report identifies all of the viable options and potential high-level activities related to each option. These will be subject to further analysis. A Conceptual Architecture Solution document is produced during the Feasibility Phase. A Concept of Operations document is either produced or an existing document is refined to reflect how a solution may operate once implemented. These documents support the identification and definition of quality criteria ultimately incorporated within scope definition.
During the Analysis Phase the Business Projects-IT-Enabled objective is to produce a Business Case which includes a recommended technical solution and a Project Charter which will contain a high-level quality statement. This forms the basis upon which further quality detailing occurs. Quality criteria are also normally detailed in the Evaluation Criteria which will be used in the procurement process and in the Migration Strategy and Testing Strategy documents that support the technical solution.
Identification Close Out Phase
The purpose of the Identification Close Out Phase is to ensure an appropriate level of assessment, reporting, evaluation, handover and administrative closure has taken place. A formal close out provides enough directional detail for the (delivery organization) Project Manager to seamlessly proceed to the Delivery Stage.
In light of the Preliminary Project Approval (PPA) decision, obtained in the Analysis Phase, the project team ensures that the document records prepared in this phase include high-level quality requirements.
Subsequent to the PPA decision, the quality planning process is repeated in the Planning Phase. Quality planning in the Delivery Stage must include quality procedures, metrics and tools to ensure the quality of both project management and product deliverables and processes required to produce and implement the recommended solution.
The project team uses the following inputs to define its Quality Management Plan processes, metrics and tools:
- Scope Baseline;
- Stakeholder Register (outputs of Stakeholder Analysis);
- Cost Performance Baseline;
- Schedule Baseline;
- Risk Register; and
- Other tools, processes and information existing within the organization.
The Quality Management Plan results from the quality planning process. It is typically captured in the Project Management Plan. Associated components of the plan include quality assurance, quality control and continuous improvement approaches, such as quality metrics, quality checklists and process improvement plans.
During the Design Phase the Logical Architecture document is further updated to reflect the physical architecture design with quality components. A migration plan is produced to outline how existing elements will be incorporated into the enterprise architecture as final products and processes. As these products are produced and then updated, it is essential to conduct both scope verification and control activities.
Scope verification differs from quality control in that scope verification is primarily concerned with acceptance of the design as it relates to scope while quality control is primarily concerned with meeting the quality requirements specified for the deliverables. Quality Control is generally performed before scope verification, but these two activities can be performed in parallel.
Perform Quality Assurance
Quality assurance activities create standards and measurement tools. The project team then monitors and verifies that the processes used to manage and create the deliverables are followed and are effective.
Internal quality control activities are exercised via:
- the Project Steering Committee, as it reviews the effectiveness of project management and product development practices and processes;
- the Project Management Team, through its regular scheduled meetings and discussions and through the implementation of the PMP;
- the project's creation of effective quality measurement standards, deliverables and tools such as:
- Quality Standards
- Quality Metrics
- Quality Checklists
- System Testing Strategy
- System Testing Plan
- User Acceptance Testing Plan
- Requirements Traceability Matrix
- Acceptance Criteria
- the Project Change Management Process (see Integration Management Knowledge Area).
Conduct Process Improvement Activities
Projects continually improve the effectiveness of their quality management process through the management of quality non-compliance, implementation of lessons learned, analysis of performance data, and management review.
The continuous improvement team is responsible for managing improvements to the NPMS. The Project Management Advisory Council approves all changes.
For external audits and reviews see Annex B - Quality Forums.
For more information on process improvement activities see Annex C - Quality Control Activities.
Perform Quality Control
Quality control executes and implements quality measures and collects data. Activities monitor and verify that project deliverables meet defined quality standards. The following is a sampling of quality control activities:
- Peer Reviews
- Design Reviews
- Technical Reviews
- Control of Design and Changes
- Product Testing
For external audits and reviews see Annex B - Quality Forums.
For more information on quality control activities see Annex C - Quality Control Activities.
Effective performance monitoring is a key to the Quality Management Process within projects and at the NPMS framework level. Reporting provides information on how well the project (and NPMS) is working and provides feedback on the degree to which project management, technical and product development teams and deliverables meet the defined standards to enable continuous improvement.
Performance reporting also supports internal project quality control processes. Additional key performance indicators are used as part of the ITSB status report process. ITSB, the IT delivery organization, has identified additional indicators to be used in its project status reports. Monitoring is done through the project governance structure which includes the sponsor, the Project Steering Committee and an independent review function via the Monthly Project Delivery Management process. Monitoring and control processes ensure quality services and solutions and enhance the consistent management and delivery of the Department's Business Projects-IT-Enabled.
Manage Quality Non-Compliance
Documented processes, plans and standards define how project activities should be performed. Project non-compliance is the degree to which the actual performance of project activities deviates from how they should be performed.
The quality non-compliance management process involves:
- identifying, analyzing and documenting non-compliance;
- defining corrective treatments;
- referring change requests to the project's Change Management Process;
- authorizing and monitoring corrective actions; and
- closing non-compliance.
The quality non-compliance management process applies to project management and product development deliverables, processes, activities and to project governance structures.
Manage Project Lessons Learned
Lessons learned are the knowledge gained from the positive and negative experiences encountered during the execution of a project. The objective of capturing lessons learned is to support a knowledge base of project management information. This experience-based information is used to continuously improve project management processes by ensuring that positive project experiences are repeated and negative project experiences are avoided.
Managing lessons learned is not a static process. Lessons learned are captured at the end of the Identification and Delivery Stages and also collected formatively during the Analysis, Planning and Design Phases to support internal project improvement. The lessons learned collected at project close out are used to improve the delivery of the enterprise project portfolio. Changes are also submitted through the NPMS continuous improvement process.
The lessons learned management process includes:
- identifying and documenting lessons learned;
- compiling and analyzing lessons learned;
- reporting and sharing lessons learned;
- implementing lessons learned (individual projects and corporate methodologies); and
- monitoring the implementation of lessons learned corrective actions.
Final lessons learned are collected at the end of the Identification Stage Close Out. They are collected formatively during the Project Delivery Stage Planning, Design and Implementation Phases. During Project Close Out, a separate Lessons Learned Report is prepared for larger projects; for smaller projects, project lessons learned are included as a component of the Project Close Out Document.
The lessons learned process is an essential component of the Continuous Improvement Process. In addition to being a key element of good quality management, it also allows the NPMS continuous improvement teams and the monthly project delivery management teams the opportunity to evaluate and learn from the many activities that took place throughout the project lifecycle and to propagate these lessons learned in similar projects.
Delivery Close Out Phase
Once the project is complete, the project team prepares the Project Close Out Document, and the lessons learned report. It also conducts the administrative and contract Close Out activities, documenting the process thoroughly. The degree to which the project met its quality assurance and quality control standards and followed its quality management are thoroughly described in both documents. Approved changes to these processes that were implemented also need to be documented. Note: In a majority of projects, lessons learned are incorporated into the Project Close Out Document.
The Project Close Out Process is used to tie up loose ends at the end of the project, such as transferring unresolved issues and action items appropriately. The operational enterprise Change Advisory Board also conducts a post-implementation review to tie up loose ends from an operational change management perspective. For the same reason, the business line owner conducts a post implementation review to capture lessons learned and to begin the benefits realization assessment. Refer to Annex B - Quality Forums for more information.
Recommendations generated by the Project Quality Management Processes should be submitted through the appropriate channel, including process changes related to the NPMS Framework.
This Knowledge Area applies to all PSPC Business Projects-IT-Enabled.
For projects carried out for and funded by other government departments (OGDs), the NPMS practices are to be applied in keeping with client approvals and governance, as per the OGD procedure.
Quality Management and Required Deliverables for « Full » and « Lite » Projects
Projects >$1M, or projects requiring increased project management rigour as a result of the NPMS Business Projects-IT-Enabled Screening Tool assessment for complexity, risk or sensitivity. These projects use the « Full » NPMS control point deliverables, including a Feasibility Report and a separate Lessons Learned Report that is generated at project close out.
In addition, a full PMP will be created during the Project Delivery Stage, Planning Phase. One of the components of this PMP will be a Quality Management Plan. The Delivery PMP will define how the project will be managed; the quality management plan will define how quality is integrated into project management and product development deliverables and processes.
Projects <$1M, or projects valued at less than $1M and which do not meet one of the override criteria as determined by the NPMS Business Projects-IT-Enabled Screening Tool, will not normally produce a Feasibility Report. Lessons Learned will be integrated as a component of the Project Close Out Document.
The Inception and Identification Stage are the responsibility of the OGD, as are Treasury Board (TB) approvals. The OGD's delegated authority is determined by its TB Organizational Project Management Capacity Assessment.
Project Delivery organizations will produce a full PMP for OGD projects during the Planning Phase (Project Delivery Stage). Quality Management will be reflected in this PMP. The PMP will define how the project manager will manage the project; the quality management plan will define how quality is integrated into project management and product development deliverables and processes.
- Key Performance Indicator "On Time"
- The development/delivery status of the project is in accordance with the approved schedule.
- Key Performance Indicator "On Budget"
- The current financial status of the project and estimated budget to completion is in accordance with the approved budget objective.
- Key Performance Indicator "On Scope"
- The current scope of the project is in accordance with the approved scope objective.
- Key Performance Indicator "Overall Status"
- The Project is in accordance with the approved objectives for each of the On Time, On Budget and On Scope indicators.
8. ROLES AND RESPONSIBILITIES:
All parties responsible for developing Quality Management Plans are strongly encouraged to consult with other Project Leaders/Managers and Senior Project Managers when developing the Quality Management Plan. It is also recommended that Project Managers and Quality Managers seek advice from technical experts and other SMEs within PSPC when producing/updating quality plans.
Project Lead (Business Side)
The Project Lead (Business Side) is responsible for the following quality management activities:
- ensuring that a quality management process is implemented and respected;
- ensuring that quality assurance & quality control processes are implemented by the project and technical teams;
- monitoring quality measurement activities from a business perspective;
- signing off on product and project management deliverables in accordance with the quality requirements of the business,
- respecting the project quality management processes;
- actively participating in quality management activities;
- endorsing decisions to improve existing processes; and
- reporting quality management lessons learned.
Note: A specialized dedicated resource may be required to fill this role on larger projects. On smaller projects this role is filled by the (delivery) Project Manager.
The Quality Manager is responsible for the overall quality management on the project. Specifically, the Quality Manager is responsible for the following quality management activities:
- Ensuring that a Risk Management Plan is created, maintained and implemented as documented, using the Risk Log created during the Initiation Stage and updated during the Identification Stage as the basis for the Risk Management Plan;
- defining and establishing the project's quality objectives and standards;
- preparing and maintaining the quality management plan;
- acting as the OPI for ensuring that the quality management plan aligns to the departmental QMF;
- implementing the quality management processes;
- ensuring that all project personnel are following the established best practices for the project;
- establishing and maintaining the quality management records/logs/files;
- reviewing deliverables for compliance with requirements/standards;
- establishing and maintaining the continuous quality review process and ensuring that all non-compliance issues are tracked in accordance with the approved process;
- attending project management meetings; and
- participating in quality management audits/project reviews and tracking action items.
The project manager is responsible for the following quality management activities:
- ensuring that a client-focus is maintained by the project team;
- ensuring that quality management best practices are integrated into all processes and deliverables;
- providing resources to quality management activities (personnel, tools);
- ensuring that all personnel follow the established quality management processes;
- reviewing quality management audit reports and taking corrective action, where required;
- ensuring quality management activities and processes are included in the project plan and schedule and monitoring for deviations from planned activities;
- resolving project quality non-conformance issues;
- reviewing and sharing project lessons learned reports and implementing applicable recommendations; and
- reporting on quality management.
Project Management Team
The project management team (Project Manager, Technical Manager, Business Manager, others) is responsible for the following quality management activities:
- maintaining a client-focus;
- reviewing and approving project quality management plan and processes;
- reviewing quality control reports and taking corrective actions, when required;
- reviewing quality management audit reports and taking corrective actions, where required;
- reporting on ways to improve present processes; and
- reporting on quality management lessons learned.
The project team is responsible for the following quality management activities:
- maintaining a client-focus;
- following the project quality management plan and processes;
- supporting approved testing programs;
- reporting on ways to improve existing processes;
- reporting quality management lessons learned;
- reporting quality non-compliance issues; and
- conducting quality control activities and collecting metrics when required.
Client / Business Line Owner
The client / business line owner is responsible for the following quality management activities:
- ensuring that quality management process objectives are incorporated into client acceptance documents and that these objectives are met before acceptance is agreed;
- ensuring that a client-focus is maintained by business line and project teams;
- respecting the project quality management processes;
- actively participating in quality management activities;
- reporting on ways to improve existing processes;
- reporting quality management lessons learned; and
- signing off on product quality.
- TBS Policy on the Management of Projects
- International Standards Organization ISO 9001:2008
- PSPC Quality Management Framework
- PSPC Continuous Improvement Process Standard
- Project Management Institute. A Guide to the Project Management Body Of Knowledge, 4th Edition, 2008
- Document Naming and Metadata Standard
- Performance Monitoring and Reporting Standard
- Integration Management Knowledge Area
- Annex A - Quality Management Process
- Annex B - Quality Forums
- Annex C - Quality Control Activities
- Annex D - The PSPC Quality Management Framework
Please direct enquiries about this Knowledge Area to the Director, Centre of Excellence, ITSB Project Delivery Office.
Annex A - Quality Management Process
Quality Management consists of the following processes:
- Quality Planning
- Quality Assurance
- Quality Control
As the project team performs its quality functions during the project lifecycle, outputs will include Status Reports, metrics, updated project documents including the Business Case, the Project Charter and the Project Management Plan. Other outputs will be an updated Risk Log and Issue Log as well as lessons learned and inputs to both the project and the NPMS continuous improvement process.
Quality standards will be detailed in the requirements definition process. Quality will then be confirmed as the result of product and process testing. In this context testing includes both formal IT testing processes and procedures as well as alternative quality testing processes such as peer review) during the Design and Implementation Phases. In addition, deliverables and processes will be reviewed by the Monthly Project Delivery Management process, which is an ongoing component of the Product Delivery Stage. The process supports project management practice and process oversight.
The following principles are based on an excerpt from A Guide to the Project Management Body of Knowledge (PMBOK® Guide)
Modern quality management complements project management. Both disciplines recognize the importance of:
Understanding, evaluating, defining and managing expectations so that customer requirements are met.
Prevention over Inspection
Quality is planned designed and built in – not inspected in. The cost of preventing mistakes is generally much less than the cost of correcting them.
Success requires the participation of all members of the project team, but remains the responsibility of management to provide the resources needed to succeed.
The PSPC continuous improvement process is closely modeled on the industry standard Plan/Do/Check Act (PDCA) Process (The Deming Cycle). A key component of the NPMS framework is the NPMS Continuous Improvement Process which is captured in the continuous improvement standard document and reflected in these quality management processes.
Annex B - Quality Forums
The following forums can serve both quality assurance and quality control functions:
External Audits & Reviews
External audits and reviews will be performed on Business Projects-IT-Enabled as and when required by the audit and review organizations. Audits and reviews examine both the project management processes and the project's product. Each audit and review authority defines its own audit and review scope and process. Examples include:
- External Audits – Auditor General of Canada / PSPC Audit & Evaluation;
- Departmental Contract Reviews – ITSB Contract Review Board, AB Procurement Oversight Committee;
- Treasury Board Independent Verification and Validation Reviews; and
- Treasury Board Submission Reviews.
Treasury Board submissions will be reviewed throughout their development in order to ensure that the scope and quality of the submission adheres to the Treasury Board standards. PSPC corporate level governance bodies, including the TB Submission Committee, will undertake these reviews.
The Project Steering Committee
In terms of quality management, the Project Steering Committee's role is to:
- Receive and review project metrics and tracking reports; and
- Recommend remedial action, if required for the projects product or its processes.
IM/IT Architecture Review Board
The IM/IT Architecture Review Board reports to the Departmental Business Operations Committee, which has the mandate to approve enterprise IT projects. The IM/IT Architecture Review Board reviews projects to ensure that project initiatives align to the Departmental Investment Plan and are consistent with the Enterprise Architecture. Enterprise quality management standards are used. The Board's review and impact assessment process begins as soon as the appropriate client executive is informed by the accountable branch that a statement of requirement has been approved.
The Enterprise Change Advisory Board (eCAB) provides authority and management controls for all IT changes deemed to be either in the major or in the significant category. These changes include but are not limited to business applications and IT Infrastructure services relating to the PSPC enterprise production environment. eCAB has a quality assurance role in that it verifies that the project's product, its design and build fit operational quality standards and can be implemented without degrading operational standards. Quality is confirmed by Client Acceptance Testing (CATE) and Release Acceptance Testing (RATE) results.
The eCAB Change Management Process and Request for Change (RFC) documentation is available.
Monthly Process Delivery Management
The Monthly Project Delivery Management is a regular review of project risks, quality and performance. The purpose of the Monthly Project Delivery Management is to provide an independent review of projects in order to ensure that every project managed by the Project Delivery Office is run in an efficient, consistent and repeatable manner and achieves a high level of customer satisfaction. The Monthly Project Delivery Management focus is on incorporating best practices and ensuring projects adhere to established organizational governance. The Project Monthly Project Delivery Management is integrated with the ITSB Dashboard process. Monthly Project Delivery Management is used by both ITSB's project delivery organizations: the Project Delivery Office (PDO) and Service Transitions and Major Projects (STAMP).
NPMS Process Evaluation
Under the direction of the Real Property Project Improvement Steering Committee, reviews of success, relevance, and cost effectiveness of the National Project Management System are to be conducted on a 3-year basis, in order to ensure effective feedback to enable the continuous improvement. The reviews are intended to be evaluated using an external evaluator and following the TB Project Management Policy - Standard for Organizational Project Management Capacity together with an evaluation of a representative sample of projects in-progress and completed. These are to be assessed in order to verify proper compliance with the National Project Management System.
Project Close Out Post Implementation Reviews
Business (Client Evaluation)
The client conducts a Post-Implementation Review to look at, among other things, the effectiveness of the solution implementation process and the degree to which the solution has addressed the business need.
Under the auspices of the Enterprise Change Advisory Board authority, a Post-Implementation Review (PIR) is conducted, specifically focusing on lessons learned, as well as release, change and configuration management processes and activities. This review also addresses capacity and business recovery issues as required.
Annex C - Quality Control Activities
Whenever possible, the documents developed during the project will be reviewed by a peer. A list of deliverables to be produced along with the OPI and the reviewer(s) will be maintained.
The Project Manager schedules design reviews in order to evaluate the ability of the design to fulfill requirements, to identify any problems and propose corrective actions. Participants in such reviews include the project team as well as business and functional representatives concerned with the design stage(s) being reviewed. The results of the reviews and any necessary actions will be maintained as part of the project record.
Before going to EPA, the project manager and technical manager review and approve the technical design and solution documentation internally. Normally SMEs from within the project and operational in-service organizations participate in the review. Internal testing and / or quality review ensure that mandatory requirements are met before the technical review can take place.
Control of Design and Development Changes
Design and development changes will follow this project's Change Management Process, which is documented in the Integration Management Knowledge Area. The purpose of change management is to control changes to the established project baselines.
The project will produce and implement detailed and comprehensive product development test strategies and plans in order to determine whether the product meets user requirements. All test strategies and plans are reviewed and approved by the Project Management Team. The Project Team will use the Requirements Traceability Matrix and System Test Plan as the basis for constructing test sets and system test procedures and for documenting results.
The Project Team will test the components in stages prior to integrating them into a system. When the system is integrated and the external test procedures documents are ready, a test readiness review is conducted to determine if the system is ready for external testing. Independent reviewers conduct verification and validation testing prior to release. Independent testing includes functional, non-functional, performance, capacity, security, installation and interoperability test components.
Independent Verification and Validation (IV&V) testing will be conducted for all internal PSPC technical solutions through the standardized Client Acceptance Testing (CATE) Process and the Release Acceptance Testing (RATE) Process. Design and Product Testing Results will be reviewed by the operational implementation authority, Enterprise Change Advisory Board (eCAB), for acceptability of the design, and to confirm readiness for solution implementation. The eCAB process also ensures that a post implementation review is conducted.
The client is responsible for the conduct of the user acceptance tests and the results of these tests will substantiate the case for implementation approval. The project is responsible for assembling and documenting test results and producing summaries for the approval process. Project, testing, product and operational handover documentation is updated in preparation for the implementation approval gating process.
Results of testing are captured in the Requirements Traceability Matrix and Testing Results Summaries. The test results above are used not only to confirm the quality of the product; they may also be used to make changes in the quality control process including its standards, metrics and measures.
Annex D - The PSPC Quality Management Framework
Seven interrelated elements of a quality-driven organization
- Leadership Long term senior management commitment to quality through shared vision, appropriate governance and a common framework
- Planning Overall quality strategy and integrated plan with specific management goals
- Client Focus Clear understanding of client expectations
- People Focus Environment conducive to quality through staff participation and personal and organizational growth
- Partner / Supplier Focus Best value to clients through effective and efficient suppliers and partners arrangements
- Process Based Facilitate quality transformation through the continuous improvement of key processes
- Organizational Performance Use of performance information (financial, human resource, program, client, partner / supplier performance) to improve enabling support systems, products and service delivery
- Date modified: