ICAT

ICAT is an open source meta-data management system designed for large facilities. ICAT development is a collaboration involving ISIS, the Scientific Computing Department of STFC, ILL and the Diamond Light Source. PaNdata WP4 is aiming to provide use cases and requirements for the ICAT development to produce a product which meets the functionality of the proposed virtual labs. Important tasks of PaNdata WP4 include the deployment of ICAT and service verification; this work is progressing well. Currently, ICAT is in production at ILL, ISIS and DLS and several labs (e.g. ALBA, DESY, ELETTRA and ESRF) have deployed ICAT prototype instances, which have been tested during service verification.

An important consideration in a meta-data catalogue is the meta-data schema. To satisfy PaNdata's needs, a number of modifications have been proposed and implemented in ICAT. The schema is closely related to the standardized meta-data schema implemented in NeXus. NeXus provides a complete description of the experimental conditions during an experiment, and covers essentially all of the data required for meaningful searches in a meta-data catalogue. Ingestion of NeXus into ICAT is straightforward; the ICATs in production at ILL, ISIS and Diamond ingest thousands of NeXus files every day.

ICAT has many attractive features, such as the programmable web service interface, the integration of provenance information and the registration of Data Object Identifiers. The developments at the moment include the ICAT Data Service (IDS) and a rich collection of authentication mechanisms.


Documentation


Resources


Deployment status

Currently, nine of the participating facilities operate an ICAT instance. A quick overview can be found on the latest ICAT service verification. Currently, ICAT is in production at ISIS, ILL and DLS (at some sites not involved in PaNData ODI). An instance at DESY will be in production in Fall 2013. Prototype instances of ICAT are operational at ELETTRA, ESRF, HZB, JCNS and SOLEIL. STFC e-science in addition operates a number of virtual ICAT instances to demonstrate the federation of ICATs via TopCat and the federated search across catalogues.


ICATs in production

Currently, four sites run an ICAT in production to archive and manage all experimental data generated on site:


Demonstrator

There are a number of demonstrators available for various aspects of the ICAT and TopCat services. All of them can be found through the ICAT project pages


Roadmap

Alistair Mills has been assigned the release manager for PaNdata ODI to

  • write a plan for the release management;
  • be the main contact between the work packages and the test and the meta-data service work package;
  • define and deliver a schedule of test events involving most of the partners of Pandata;
  • coordinate the test events;
  • ensure that the materials provided for test are in a testable state;
  • provide feedback to the work packages on their materials;
  • define the contents of a release;
  • ensure that software releases are coherent and appear in a timely and reasonable form.

A detailed account of theses efforts can be found here.

Schedule of service verification

The schedule for the service verification is in the following table. A later draft of this plan will contain further refinement of the contents of the service verification. Each service verification has detailed instructions and the report and can be found below.

Schedule of service verification (SV)
SV Date Major topics
4 10 May Use of the bigv federation to verify the download manager.
Use of the bigv authentication service.
5 21 June Initial test of the development federation.
Verification of an download service.
Verification of an ingest service.
6 October Initial test of the production service.
Integration of materials from WP 5, 6, 7, 8.
7 November Verification of the Umbrella authentication service.
Integration of materials from WP 5, 6, 7, 8.
8 December Verification of the production service.
9 February To be defined.
10 March To be defined.
Note: These releases will not be packaged as a single package. The release will be a stack of components from a variety of sources. We will provide information in the service verification report of the versions and their performance.


Software Releases
Release Date Contents Verification
0.0 May ICAT, authn_db, authn_ldap, DM, Topcat.
Low grade authentication service using Facebook. 		SV[4]
0.1 June As 0.0, plus development federation SV[5]
0.2 October As 0.1, plus materials from WP 5, 7 and 8 >SV[6]
0.3 November As 0.2, plus integration with Umbrella SV[7]
0.4 December As 0.3; A technology preview release SV[8]
0.5 February Release candidate SV[9]
1.0 March Production release SV[10]