Model evaluation

See also:

Experimental data sets: Top-level page in hierarchy on data sets. Note that there is a forum for communication of experiences with data sets.

Pitfalls related to quality assurance of dispersion models: A brief list.

---

Introduction

Model evaluation studies abound in the technical literature. Still, it is not easy for a decision-maker to determine whether model A is better than model B for his particular purpose.

Typically, modellers have difficulty in coming up with definite answers. They can present some investigations related to model A, others related to model B, but seldom comparable.

The problem of model evaluation is intrinsically difficult.(See Why is model evaluation difficult?).

There have been endeavours to construct common frames of reference for model evaluation, so several models can be compared in a fair manner.

The present page presents annotated links to such toolkits or common exercises. A related set of activities are pure model intercomparisons, whereby models are intercompared, but not verified against measurements.

Links to frames of reference for model evaluation

• Within the framework of the Initiative on Harmonisation within Atmospheric Dispersion Modelling for Regulatory Purposes (http://www.harmo.org) the so-called Model Validation Kit has been developed and made freely available to the modelling community. The Kit is a collection of four field data sets as well as software for model evaluation. It addresses the classic problem of dispersion from a single point source. The official web site of the kit is http://www.harmo.org/kit. The present Wiki contains a page for Postings by users of the Model Validation Kit.
• Within the framework of the ASTM (American Society for Testing and Materials), a Standard Guide for Statistical Evaluation of Atmospheric Dispersion Models, D6589-00 has been developed. The ASTM methodology is general in the sense that it does not assume that one deals with a certain type of model or with a certain concentration variable. However, there is an appendix to the guide, which specifies an example where the framework is used. This example deals with the classic problem of a plume being emitted from an isolated point source. More info can be found at http://www.harmo.org/astm
• The COST action 732 (running 2004-2009) is a European collaboration network working with quality assurance of micro-scale flow and dispersion models that are applied in urban environments. COST 732 is developing quality assurance procedures with guidance to model developers and users. The target models are mainly - but not exclusively - CFD models. Reports with guidance are on model evaluation are available from the COST 732 web site. The official title of COST 732 is "Quality Assurance and Improvement of Micro-Scale Meteorological Models". Note that a brief list of Pitfalls related to quality assurance of dispersion models, was assembled during the course of the Action and is available from the present Wiki. Additions to that list are welcome.
• Within COST 732 a model validation exercise concerning the MUST data set (Mock Urban Setting Test) was undertaken. Results from this exercise can be explored with a comprehensive set of Excel tools. Read more on the page COST_732_forum

---

More links should be added. The exercise with Göttinger Strasse, Street Emissions Ceiling exercise etc.

To visit the Model Evaluation category and read the explanation of how to use that category, click on the Model Evaluation link just below: