wiki:WP4/ServicesPlanningDocument
Last modified 2 years ago

Metafor Services (WP4, WP5, & WP6): Deliverables, Dependencies, and Designs

  1. Metafor Services (WP4, WP5, & WP6): Deliverables, Dependencies, and …
    1. Deployment Architecture as of end of year two
    2. RELEVANT YEAR TWO MILESTONES AND DELIVERABLES
    3. Relevant Milestones and Deliverables: Architecture and Sequencing
      1. 1. D4.1 Single-Sign-On Evaluation (delivered approx 15 June)
      2. 2. M4.1 Service Identification (M15)
      3. 3. M4.2: OAI Prototype (M18)
      4. 4. D6.1 Automatic Capture Tools (M18)
      5. 5. M6.3 Prototype Human Gui (M18)
      6. 6. D5.1 Query Tool (M21) (ticket)
      7. 7. M5.3 Difference Engine Requirements (M21)
      8. 8. D5.2 Difference Engine Implementation (M24)
      9. 9. D4.2 OAI System Deployed (M24)
      10. 10. D5.3 Presentation Tools
      11. 11. D5.4 Data Transformation Tools
      12. 12. D6.2 Revised Capture Tools (M24)
      13. 13. M4.3,M4.4, M4.5,M4.6 Portal Prototype and underlying services deployed …
    4. Appendix A: Dependency Chart
    5. Appendix B: Relationships with IS-ENES

This document is to be used collaboratively between developers of WP4, WP5, and WP6 to understand the interactions and dependencies among their various milestones and deliverables and to document the technology options and, ultimately, technology choices to be used in implementing those deliverables.

Deployment Architecture as of end of year two

RELEVANT YEAR TWO MILESTONES AND DELIVERABLES

  • WP4: D4.1 Single-Sign-On-Evaluation (M12->M15 by agreement).
  • WP4: M4.1 Service Identification (M15)
  • WP4: M4.2 OAI Prototype (M18)
  • WP4: D4.2 OAI System Deployed (M24)
  • WP5: M5.2 Agree Features of the Query Interface (M15)
  • WP5: D5.1 Query Tool (M21)
  • WP5: M5.3 Difference Engine Requirements (M21)
  • WP5: D5.2 Difference Engine Implementation (M24)
  • WP5: D5.3 Presentation Tools (M24)
  • WP5: D5.4 Data Transformation Tools (M24)
  • WP6: D6.1 Automatic Capture Tools (M18)
  • WP6: M6.3 Prototype Human GUI (M18)
  • WP6: D6.2 Revised Capture Tools (M24)

Relevant Milestones and Deliverables: Architecture and Sequencing

It's probably helpful to review the original basic architecture diagram:

The Original Architecture Vision

(Note that data services are out of scope for Metafor per se, but absolutely in scope for IS-ENES.)

In each of the following, there are three sections: a brief description of what we expect, the relationship/dependencies between metafor work packages, and the expected dependencies of IS_ENES where they exist (there are NO expected dependencies on IS-ENES from Metafor).

They are presented in date order

1. D4.1 Single-Sign-On Evaluation (delivered approx 15 June)

Description
This deliverable outlines the choice of authentication and authorisation for metafor, which we expect to be used to control editing etc. This work has been done in conjunction with ESG.
Internal Dependencies
The data creation tools should eventually use these technologies to allow editing access to CIM creation instances. The eventual Metafor portal will allow metadata annotation using this solution.
External Dependencies
We expect IS-ENES to exploit this, so this is a key interface with IS-ENES.

2. M4.1 Service Identification (M15)

Description
This milestone identifies which services will

(or could) be deployed in Metafor. Current expectations are:

  1. An aggregation service (that harvests CIM compliant metadata from CIM producers, initially via OAI and then possibly via provider atom feeds).
  2. A restful query API, which exposes the query interface (M5.2)
  3. An OpenSearch Interface
  4. An Atom feed of new CIM instances harvested to the portal
  5. An annotation interface to CIM instances (restful + metafor security)
  6. An Atom feed of user annotations to CIM instances
  7. Vocabulary services exploiting the BODC vocabulary RESTful API (including editing services to allow the management of controlled vocabularies)
  8. A WFS interface (once the CIM is recast as an Application Schema of GML).
  9. A service to allow differencing CIM documents (i.e. exposing D5.2)
  10. A service to expose CIM Transformation Tools (i.e. exposing D5.4)
  11. A portal which utilises and/or provides the above interfaces (cf D5.3)
  12. It might be possible to run a UML-> XSD and/or RDF service to expedite the ongoing development of the CIM, but this depends on as yet unmade decision to make Metafor GML compliant. If so, this would be one Data Transformation Tool.
Internal Dependencies
Nearly all WP4 activities depend on this milestone. A number of WP6 activities impact on theses services.
External Dependencies
If, as seems likely, IS-ENES build their own portals, then they will depend on these services (as well as other data manipulation services).

3. M4.2: OAI Prototype (M18)

Description
Essentially this was about starting to put in place the mechanisms for moving CIM documents around the partners. At this stage it seems most simple for BADC and MPIM to all create Geonetwork instances in order to store “local” copies of CIM instances, and for BADC to use a standard OAI tool (not Geonetwork, in order to demonstrate multi-vendor capability) to harvest CIM instances from both sites into a repository. At this stage it doesn't matter what the repository is. The point is to get OAI working.
Internal Dependencies
While we might want to use Atom in the longer run, in the first instance, this should work easiest, and then the portal can use the harvested CIM documents. The CIM creation tools will need to get their data into local Geonetwork instances via some sort of batch interface.
External Dependencies
Ideally none.

4. D6.1 Automatic Capture Tools (M18)

Description
Internal Dependencies
External Dependencies

5. M6.3 Prototype Human Gui (M18)

Description
Geonetwork + Questionnaire.
Internal Dependencies
CIM
External Dependencies
CMIP5, editing CIM instances exported from anywhere else.

6. D5.1 Query Tool (M21) (ticket)

Description
Allows users to perform queries on CIM records. Should support faceted search. Standard search can be done via a set of xquery definitions. Faceted search may require RDF technologies. Using Pylons and rdflib I can run SPARQL queries on RDF representations of the CIM. However, this requires converting the CIM from XSD/XML to RDF.
Internal Dependencies
The portal will depend on xqueries defined into CIM documents
External Dependencies

7. M5.3 Difference Engine Requirements (M21)

Description
Internal Dependencies
External Dependencies

8. D5.2 Difference Engine Implementation (M24)

Description
Internal Dependencies
External Dependencies

9. D4.2 OAI System Deployed (M24)

Description
It seems that it will be easiest to simply use the Geonetwork instances to establish the Metafor “system”. So the enhancement from the prototype would be for ISPL to deploy Geonetwork, and potentially,despite the name of the package, to modify all the (or utilise) Geonetwork atom feeds to expose their content (what would we do about exposing “deletions”?), and utilise something based on a feed aggregator to populate the “portal repository”.
Internal Dependencies
The development of the portal depends on this, as do all the services.
External Dependencies
No services for IS-ENES can be exposed without this activity. If we want to interoperate with GENESI-DR we might need the atom feeds (in order to allow them to harvest from us).

10. D5.3 Presentation Tools

Description
I think this could be either genshi(pylons) or django templates
Internal Dependencies
need to make a decision: django or pylons for the portal?
External Dependencies

11. D5.4 Data Transformation Tools

Description
Should include the production of GEOS documents, and INSPIRE metadata. Atom Feeds. These are all essentially xqueries on CIM content (or database queries on ORM content if we load into django).
Internal Dependencies
External Dependencies

12. D6.2 Revised Capture Tools (M24)

Description
Internal Dependencies
External Dependencies

13. M4.3,M4.4, M4.5,M4.6 Portal Prototype and underlying services deployed (M27)

Description
This is the first “official” metafor portal and service deployment, but we expect to have deployed as much as possible before this point. In particular, we expect to have some CIM presentation tools and limited CIM querying available by M21 (approximately the end of 2009).
Internal Dependencies
All subsequent services depend on this, and probably the success of the project. This depends on all the 2009/2010 milestones which outline capability and tools to be deployed. Depends on CIM V2.0 (the M24 version!), so we can expect a tool upgrade between month 24 and month 27, which in practice means we need all the underlying tools deployed with CIM V1.x by the end of Metafor year 2.
External Dependencies
will be legion on the services, and it's likely that BADC and MPIM will build on this for deploying CMIP5 services.

Appendix A: Dependency Chart

Gant or Pert ..

Appendix B: Relationships with IS-ENES

The key issue is that there a number of dependencies on Metafor in the IS-ENES project plan. In this section we identify those dependencies.

TBD

 1CSML: Climate Science Modelling Language  http://csml.badc.rl.ac.uk/

 2NCML: NetCDF Markup Language  http://www.unidata.ucar.edu/software/netcdf/ncml/

Attachments