Paul Kanciruk, Manager
Environmental Information Analysis Program
ORNL Environmental Sciences Division
Most of the readers of CDIAC Communications are familiar with how CDIAC operates. Scientists, usually at the end of a study, submit their data to CDIAC. CDIAC quality-assures and documents the data, in many cases adding value to the data by further analyzing or combining the data with related data, and then makes the data available for distribution. There is another data-management support function, which until recently was not supported by ORNL. That is support for field data that were not mature but nevertheless had value to active scientists working on the same project. Supporting the (limited) distribution of these constantly changing, dynamic data sets, among researchers on three continents, was a task that the National Aeronautics and Space Administration (NASA) was facing for its Large Scale Biosphere-Atmosphere Experiment in Amazonia (http://lba-ecology.gsfc.nasa.gov/lbaeco/). NASA wanted a system able to make field data quickly available that did not put too many demands on scientific investigators producing these data and that could be built and maintained quickly and inexpensively.
At the ORNL Distributed Active Archive Center (DAAC), a NASA-sponsored data center (http://www-eosdis.ornl.gov/), we developed a web-based, distributed data system we call Mercury. Mercury provides quick information delivery by supplying a template for creating detailed catalog entries, furnishing an engine that gathers data from linked web sites, and allowing users to search linked web sites for data. Researchers just need to make their metadata (information about their data sets) available on the Internet and, at night, an automated 'harvester' travels between the linked web sites retrieving the information and finally depositing it in an organized fashion at ORNL.
If a scientist put a data set out at night, it would be part of the system by the next morning. Each night, after the system has harvested the data, the database is rebuilt from scratch. Mercury takes advantage of the powerful XML (Extensive Markup Language) standard so it can support focused, fielded searches unlike many search engines. If the principal science investigator chooses to make a data set invisible to other users, it can be taken out of the database by the next day. This is important because it allows people to pull data from programmatic view if they later realize that there is some problem with them.
An initial test of Mercury began in late spring 1998. The International Geosphere-Biosphere Programme (IGBP) tested it until September of 1998, when they adopted it as their system (http://mercury.ornl.gov/servlet/igbp/). Currently, NASA's Earth Observing System Land Validation team has also adopted Mercury. ORNL is looking to work with NASA to expand the system to NASA's Earth Science Information Partners program. The Mercury home page is http://mercury.ornl.gov/. For a tutorial on Mercury see http://mercury.ornl.gov/PK/Mercury/tutorial.ppt. Open it as a presentation in PowerPoint.
Paul Kanciruk, Manager
Environmental Information Analysis Program
ORNL Environmental Sciences Division
kng 09/99