Experiment Naming

(ticket:930)

One of the difficulties with marrying the Metafor description of experiments with the  DRS and  Taylor et al descriptions (which differ) are that the word experiment is used in different ways by CMIP5 and Metafor.

So starting with some helpful concepts (from metafor):

• An experiment is a description of an activity that needs to be carried out. It is independent of whether or not anyone has carried out such an activity. So, in the case of a numerical experiment, it could exist, whether or not someone has "run" a "simulation" which conforms to it or not.
  • In the case of CMIP5, the metafor experiment description should correspond as closely as possible to the description in Taylor et al ...
• A simulation (in metafor) describes one or more (in which case it is an ensemble of ) "runs" which conform to the experiment.

(In truth I think we need to rename simulation in metafor to become "ensemble" and accept that it might have only one member - since any simulation can be turned into an ensemble at some later date. We should consider that as part of ticket:920.)

In the questionnaire, we enter an ensemble by defining as simulation, and then adding modifications for ensemble members ...

So

• In the case of CMIP5, we probably want simulations to correspond to ensembles within specific Taylor et al experiments.

Now, let's take the DRS view of the world:

/CMIP5/output1/UKMO/HadCM3/decadal1990/day/atmos/day/r3i2p1/v20100105/tas/ 
tas_day_HADCM3_ decadal1990_r3i2p1_199001-199012.nc

corresponding to

<activity>/<product>/<institute>/<model>/<experiment>/<frequency>/<modeling realm>/
<MIP table>/<ensemble member>/<version number>/<variable name>/ <CMOR filename>.nc

In this string, the experiment concept conflates the idea of an experiment, a simulation, and to an extent, the concept of an ensemble.

To a great extent that conflation is Bryan's fault: he took his eye off the ball at a crucial time, and we are where we are. We need to fix it though ...

It has been compounded by further issues, typified by the confusion with experiment names in the DRS and Taylor et al.

1. In Taylor et al, experiment names are, for example
   1. 1.1 "Ensemble of 10-year hindcasts and predictions"
   2. 1.2 "Ensemble of 30-year hindcasts and predictions"
   3. 1.5 is really just a variant of 1.1 and 1.2 with different initialisation
   4. 7.1 historical with natural forcings
   5. 7.2 historical with GHG only
   6. 7.3 other historical forcings
2. Which are, in DRS language:
   1. experiment name decadalXXXX where XXXX should be the initialisation year.
   2. experiment name decadalXXXX where XXXX should be the initialisation year.
      • i.e. no different from the first, so to find 1.2 experiments, you need to search for simulations which went for 30 years ...
   3. same as above, with different i<M> value (see below)
   4. "historicalNat":
   5. "historicalGHG"
   6. "historicalMisc"

Each of these can of course be carried out with an ensemble, and the following is mandatory:

r<N>i<M>p<L>

in all DRS names, where:

1. p<L> is used to identify perturbations in physics (from the same base model).
2. i<M> is used to identify perturbations in initialisation method, and
3. r<N> the "realisation" number, is used to identify ensemble members which differ in other characteristics.

Issues

So some issues, to be clear, are:

1. Are the experiment names decadal1960 etc or are they decadal?
2. What should we do with extension/contraction experiments such as 1.2 and 3.1S?
3. Should we have experiments with suffixes (such as E or I)?
4. Should we ever expect to see any of r.i.p appearing in an experiment description?

Before we answer the question, it's worth thinking about the fact that Taylor et al, and particularly the DRS, were written with the modeller in mind: what experiments should I run simulations for? How should I organise the data? i.e. they were written for the producer. (Yes, there was an element of using the DRS to help the consumer ...)

The metadata and gateways need to organise data for the consumer! How will they approach these problem?

Let's ask two obvious consumer orientated questions:

1. Find me all the thirty year hindcast runs (1.2) (or similarly, all the 500 year control runs, 3.1)
   • the answer I'd want would be 1.2 plus some of 1.5
2. Find me all the ten year hindcast runs (1.1) (or similarly, all the 100 year "short" control runs, 3.1S).
   • the answer I'd want would be 1.1 plus some of 1.5

Currently, if we follow the DRS, and collapse 1.1 and 1.2 (and collapse 3.1 and 3.1S), then there is no way to distinguish between these questions in the portal! So, if I ask either question, I'll get a lot more data than I want!

How could we handle that? Well, ideally we'd ask the experiment question based on looking for 1.1 and 1.2 (or in the other use case 3.1 and 3.1s)? Can we do that?

Yes, if we have 1.1 handled separately from 1.2, but runs for 1.5 put where-ever appropriate in 1.1. and 1.2.

Will we be able to handle this on the metadata trackback side of ESG? Dunno yet!

So, what's our recommended answer to the issue questions above:

The first should be resolved as follows:

1. The official 1.1 experiment name should be "decadal", and that's what the questionaire should show.
   • Metafor should provide guidance that folks doing the 1960 ensemble should create a simulation with start date 1960, and an ensemble within that, similarly a simulation for 1965 etc ...
   • The questionnaire to XML code should conflate the experiment name (decadal) with the start year, and write that into the "target DRS name" attribute for each ensemble member thus allowing
   • The portal to go from ensemble member to the right metadata.
   • Issue: do we believe the ESG portal is handling ensembles sensibly yet? Is there a ticket on that? (At one point they wanted all ensemble members to be handled as simulations, in curator language, which would result in very unwieldy lists following searches.)
2. The second issue can be handled via the same mechanism effectively. Folks should see "tri-decadal" as the experiment. They should enter metadata for their simulation FIRST in the decadal experiment, copy that to the tri-decadal experiment, and change the duration, and mark it as extended from the first simulation.
   • However, in their filenames on disk, they follow the DRS naming, and we deal with it in metadata.
   • Note that from a questionnaire point of view, both simulations will have the same DRS string ... but that's what you want!
3. E and I experiments are really just changes in the size and shape of the ensemble. Leave them be ... but we need to know how ESG is handling ensemble members!
4. The answer to the last question is unambiguously no. And we can't expect folks to assume that any given p number will correspond to the same forcing between experiments or between modelling centres. Folks will need to go to metadata (both sorts) to get to that!

Charlotte's Comment
This is what I understand from Bryan's recomendation above:
When users enter information about their decadal1960 simulations in the questionnaire they can assign them to four experiments but the xml output from the questionnaire will represent the four as one single experiment.

Questionnaire Experiment Name	Experiment name in xml output (after conflation)
decadal	decadal1960
tri-decadal	decadal1960
decadal-E	decadal1960
decadal-I	decadal1960

So the distinct decadal experiment are there to guide the user - the user sees familiar experiment names that match with the concepts in the Taylor et al CMIP5 experiment design document. The distinct decadal experiment names also help the tools to answer Bryan's questions above.