Madrigal Experiments

In the madrigal database system, the data are organized by experiment. An experiment consists of data from a single instrument, and, as a rule, is meant to address a particular scientific goal. Most often an experiment will correspond to a particular set of operating modes run for a contiguous interval of time. For example, the data collected by an incoherent scatter radar during a World Day would constitute an experiment. More complicated situations exist, and judgement may have to be exercised in determining what constitutes an experiment. For example, an experiment might be interrupted temporarily in order to use a different set of operating modes in support of a satellite overpass. In this case there would be two experiments which overlap in time.

To each experiment there corresponds a directory. These directories are of the form {root}/year/instrument/start_date{expt}, where root is the root directory for madrigal experiments, year is the year, instrument is the mnemonic for the instrument, start_date is the start date of the experiment in the form ddmmmyy and expt is a character to distinguish different experiments with the same start date. For example, /opt/madrigal/experiments/1997/son/06jan97 contains Sondrestrom data for an experiment beginning on 6 January, 1997, where root=/opt/madrigal/experiments, year=1997 instrument=son, start_date=06jan97 and there is no expt because there is only one Sondrestrom experiment starting on this date.

Each experiment directory must contain:

The Experiment Table entry (expTab.txt) for this experiment.
The Data Availability Table entry (dataTab.txt) for this experiment.

Each experiment directory may contain:

The File Table entry (fileTab.txt) for this experiment. This table must have one entry for each madrigal format dataset in the experiment.
One or more datasets in madrigal (or any valid Cedar) format.
Catalog and/or header files, in madrigal format, corresponding to the datasets.
Sudirectories containing an html file named index.html. Links to these files will show up the experiment page.
Html pages, and again links to these files will show up the experiment page.
Plots or other files related to individual records in the dataset. Links to these files appear next to the appropriate record in the summarizeCedarFile.cgi page. These files must be located in the subdirectory "plots/[dataset name]/records" under the experiment directory. In order for the script to determine which file goes with which record, the files must include a five digit number somewhere in its name. For example, plot00027.gif would appear as a link next to record 27 in summarizeCedarFile.cgi. More than one file can be associated with a given record.

A complete description of the various ways to add new experiments to a Madrigal database is given here.

Note that an experiment need not contain any madrigal format datasets. If a dataset is accessible through the Web, it may be cataloged by and accessed through the Madrigal Database by creating an experiment directory, and placing an experiment table, a data availablility table, and an html file pointing to the data in the experiment directory.

At Millstone, Madrigal format dataset names are constructed from the instrument mnemonic, experiment start date, experiment designator expt and an extension to distinguish datasets produced by different analysis assumptions or programs, or containing different kinds of data. This naming convention is not required or used by the Madrigal database.

Note that there is no requirement that the experiment directory actually contain any datasets. If data from an instrument have not been installed in the database, but are available through a Web page, an html file pointing to that Web page can be included in the experiment directory, thereby providing access to the data. The organizational structure described above also permits the madrigal database itself to be distributed. The madInvent.cgi optionally displays experiments from the current Madrigal site only, or a complete list of experiments at all Madrigal installations.

Revised: March 28, 2003