Archived publication
This publication is no longer current or has been superseded.
Annex 6: Quality Assurance and Quality Control
During preparation of the New Zealand 2002 national greenhouse gas inventory, work started on the implementation of the New Zealand National Greenhouse Gas Inventory Quality Control and Quality Assurance Plan. Specific checks have been completed as time and resources have allowed.
A6.1 QC procedures implemented in preparation of the 2002 inventory
Tier 1 quality checks for key sources (as identified in the level analysis of the 2001 national inventory) were completed on the 2002 data. Two of these checklists with the results are included as examples in this Annex. Quality control checks on data in the energy sector for 2002 also checked previous year's data as the energy worksheets (provided by MED) are linked. Checks on Table 10 in the CRF are used to identify any outliers or inconsistencies in the trends from 1990-2002.
In addition to the QC checks undertaken with the Tier 1 QC checklists, a number of informal checks on the data are completed including checking sectoral spreadsheets and methodology reports for inconsistencies as well as visual checks once data were entered into the CRF. Checks on the energy sector data for anomalies and inconsistencies are carried out by officials both in the MED and the NZCCO. Trends from 1990 to 2002 are examined using data from 2002 (using new emission factors for energy) and data from 2001 for the energy and industrial processes sectors.
Some specific findings included:
- PFC emissions data reported in previous inventories (1980-1999) were found to be inconsistent with company supplied emissions. This was corrected for the 2002 inventory and previous CRF.
- An outlier identified in energy data for 2001 (using 2002 data). This was traced to the wrong emissions factor being used (petrol instead of gas) for liquid fuels. This was subsequently corrected.
- An entry in the solvent sectoral spreadsheet for 2000 was shifted up one line causing emissions not to line up with the source. This was corrected.
- Activity data for asphalt roofing were found to be inconsistent in previous years (1999 and some earlier years) with the values stated in 2000-2002 sectoral worksheets. This was traced to some confusion in previous years over total bitumen and asphalt roading activity data. It was confirmed with the consultant who carried out the industrial processes work that the value stated in the 2002 worksheet was the correct one.
- Several cell reference errors in energy files were picked up by MED officials and fixed by MED and NZCCO officials. A detailed document listing these errors and when they were fixed is filed on the computer network at the MfE.
A6.2 QA procedures implemented
A checklist of recent reviews of specific sectors (including emission factors) undertaken as part of the quality assurance of the New Zealand national inventory is included in this Annex. It shows that during 2003 a review was undertaken of emission factors used in the energy sector. Most of the recommendations are implemented. A review of the industrial processes sector was also undertaken and the recommendations are being addressed.
Documentation and archiving of important reports and resources have begun including e-mail correspondence related to clarification of methodologies used by external agencies.
Plans for the future are to implement the procedures outlined and scheduled in New Zealand's inventory QA/QC plan (refer 6A.3), specifically to:
- Hold a copy of CVs of people from external agencies who provide uncertainty estimates and who undertake expert reviews of part of the national inventory (expert qualifications).
- Quality control checks (Tier 1) extended to include some non key sources in the 2005 NIR (2003 data).
- Tier 1 checks to be revised where necessary to include all relevant checks on the NZ data (where possible) and this is to be reflected in the checklist.
- Start quality control (Tier 2) checks on some of the key sources.
A6.3 Quality control and quality assurance plan
Purpose of Plan
New Zealand's greenhouse gas inventory quality control and quality assurance plan is designed to improve the transparency, consistency, comparability, completeness and confidence in the New Zealand national greenhouse gas inventory. New Zealand's plan closely follows the definitions, guidelines and processes presented in Chapter 8 'Quality Assurance and Quality Control' of the Good Practice Guidance and Uncertainty Management in National Greenhouse Gas Inventories (IPCC, 2000).
Definitions
- Quality Control: Quality Control (QC) comprises the routine technical activities used to measure and control the quality of the inventory as it is being developed.
- Quality Assurance (QA) activities include a system of review procedures conducted by personnel not directly involved in the inventory compilation/development process.
Principles of the QC/QA Plan
The QC/QA plan is intended to ensure transparency and quality of New Zealand's greenhouse gas inventory. The principles of the plan include:
- Applying greater QC effort for key source categories and for those source categories where data and methodological changes have occurred recently.
- Periodically checking the validity of all information as changes in sample size, methods of collection or frequency of data collection occur.
- Conducting the general procedures outlined in General QC Procedures (Tier 1) on all parts of the inventory over a period of time.
- Balancing efforts between development and implementation of QC/QA procedures and continuous improvement of inventory estimates.
- Customising the QC procedures to the resources available and the particular characteristics of New Zealand's greenhouse gas inventory.
- Confirming that national statistical agencies and other agencies supplying data to the inventory have implemented QC/QA procedures.
Responsibilities
The inventory agency responsible for co-ordinating the QC/QA plan is the New Zealand Climate Change Office (NZCCO). The NZCCO is also the lead agency for compiling New Zealand's greenhouse gas inventory.
The QC/QA Plan
This document specifies the QC/QA procedures to be used in New Zealand's greenhouse gas inventory. The QC/QA plan also includes reporting, documentation and archiving procedures.
The plan is to have a 'living' document that can be modified as appropriate i.e. when changes in processes occur or on advice from independent reviewers.
QC/QA Procedures
The QC procedures comprise general (Tier 1) procedures and source category-specific procedures (Tier 2).
Tier 1 procedures: the focus of Tier 1 QC techniques is on the processing, handling, documenting, archiving and reporting procedures that are common to all the inventory source categories. Tier 1 QC checks will be used routinely throughout the preparation of the annual inventory and will usually involve cross-checks, recalculation and/or visual inspections.
The results of the Tier 1 activities and procedures will be documented as described in the 'Documentation, Archiving and Reporting' section in this Annex.
Tier 2 procedures: Tier 2 QC procedures are directed at specific types of data used for individual source categories. The source category-specific measures will be applied on a case-by-case basis focusing on key source categories and on source categories where significant methodological and data revisions have occurred. Source category-specific QC activities will include QC of emissions data, activity data and uncertainty estimates.
The QC procedures that need to be implemented by the inventory agency will depend on the method used to estimate the emissions for a given source category. Where emissions estimates are developed by outside agencies and the QC activities performed by the outside agency meet QC requirements, the QC activities of the outside agency will be referenced as part of the inventory QC.
The results of the Tier 2 activities and procedures will be documented as described in the 'Documentation, Archiving and Reporting' section in this Annex.
Quality Assurance: quality assurance procedures will involve external reviewers conducting an unbiased review of the national inventory or parts of the inventory.
Experts conducting reviews will complete the Documentation of Experts' Qualifications. The expert's qualifications will be reviewed by the NZCCO prior to commencing the review.
The results of the QA activities and procedures will be documented as described in the 'Documentation, Archiving and Reporting' section in this Annex.
QC/QA Schedule:
The QC/QA procedures will be included in the 2002 inventory onwards.
Tier 1 QC: all key sources identified in the NIR of the previous year should undergo a Tier1 QC process every year. A selection of non-key source categories will be reviewed each year with the goal of ensuring all sectors are periodically reviewed.
Tier 2 QC: a selection of key sources identified in the NIR of the previous year will undergo a Tier 2 QC each year. The selection of sources will be determined from the contribution of the key source to New Zealand's national inventory, comments from the UNFCCC reviews of New Zealand's NIR and CRF tables, uncertainty in the emission estimates from categories and whether the category has undergone significant change in methodology or data.
QA: expert reviews of sectors will be undertaken as resources permit. It is anticipated to undertake at least one expert review every year. The priority for QA review is determined from the contribution of the key source or sector to New Zealand's national inventory, comments from the UNFCCC reviews of New Zealand's NIR and CRF tables, uncertainty in the emission estimates from categories, availability of expert reviewers and whether the category has undergone significant change in methodology or data.
Documentation, archiving and reporting
All information required to produce the national emissions inventory and the inventory itself (the NIR and accompanying CRF tables) will be documented and archived by the inventory agency.
Electronic copies of information will be stored in a clearly identified directory on the MfE's networked computer system and be archived by the MfE's standard network backup routine. No files necessary for the inventory will be stored on local computer hard drives.
Where emission estimates are developed by outside agencies, a copy of the necessary documentation will be held by the inventory agency.
Documentation will include sufficient information to enable all activity data, emission factors, uncertainty calculations, expert judgements and QC/QA information to be recalled, reproduced, justified or traced to the referenced source.
A summary of implemented QA/QC activities and key findings will be included in New Zealand's NIR.
A6.4 Example worksheets for QC procedures in 2002
The worksheets document the quality control procedures used on data entered in the 2002 inventory. Two examples are provided: enteric fermentation and mobile combustion from road vehicles. The quality assurance worksheet documenting the quality assurance procedures (sector reviews, methodology reviews, inventory reviews) applied to the inventory and resulting actions taken by the NZCCO.
Download example worksheets for QC procedures in 2002 (PDF 113 KB)
