Model evaluation

Introduction

Model evaluation studies abound in the technical literature. Still, it is not easy for a decision-maker to determine whether model A is better than model B for his particular purpose.

Typically, modellers have difficulty in coming up with definite answers. They can present some investigations related to model A, others related to model B, but seldom comparable.

The problem of model evaluation is intrinsically difficult.(See Why is model evaluation difficult?).

There have been endeavours to construct common frames of reference for model evaluation, so several models can be compared in a fair manner.

The present page presents annotated links to such toolkits or common exercises. A related set of activities are pure model intercomparisons, whereby models are intercompared, but not verified against measurements.

Links

• Within the framework of the Initiative on Harmonisation within Atmospheric Dispersion Modelling for Regulatory Purposes ([http://www.harmo.org]) the so-called Model Validation Kit has been developed and made freely available to the modelling community. The Kit is a collection of four field data sets as well as software for model evaluation. It addresses the classic problem of dispersion from a single point source. More info can be found at [http://www.harmo.org/kit]

• Within the framework of the ASTM (American Society for Testing and Materials), a Standard Guide for Statistical Evaluation of Atmospheric Dispersion Models, D6589-00 has been developed. The ASTM methodology is general in the sense that it does not assume that one deals with a certain type of model or with a certain concentration variable. However, there is an appendix to the guide, which specifies an example where the framework is used. This example deals with the classic problem of a plume being emitted from an isolated point source. More info can be found at [http://www.harmo.org/astm]

More links should be added. The exercise with Göttinger Strasse, Street Emissions Ceiling exercise etc.