Skip to end of metadata
Go to start of metadata

ImageIO Metadata Support Evaluation.

In this brief chapter I would like to investigate a bit more in depth the opportunity given by the Java ImageIO package to handle data sources metadata using XML.

First of all let's provide the definition of metadata in this context.

with the term metadata we can refer to all the data stored in a data source file which does not represent real pixel values plus some other information that can be derived from the file structure itself (tipically width and height).

The ImageIO package distinguishes between the concept of stream metadata which is used to report information about the whole data set we are referring to like, as an instance, the number of images (coverages) there contained and the concept of image metadata which is used to report information about an single image (coverage). A stream can contains an arbitrary amount of images (however, not all image formats have this capability).

We are interested in using both, for different purposes, let us try to describe them.

Stream Metadata.

When talking about stream metadata for a coverage data source we intend to describe (or at least this is my understanding) information which relates to the whole data set we are referring to. This include a quite wide range of things we can describe, like the compound coordinate reference system for the whole set of coverages that compose the data set, information about the point of contact for this data set, information about acquisition, etc. All these elements are not usually directly related to a single coverage but to the all set therefore it is, in my opnion, very suitable to gather them together and present them in a unified manner by exploiting the concept of stream metadata.
During my investigation on OGC specifications I read a lot of them and I found that the discussion paper OGC 05-015 "Imagery Metadata" clearly present an evaluation over the metadata presents in the "OGC Abstract Specification Topic 11 OpenGIS(tm) Metadata (ISO/TC 211 DIS 19115) version 5" (Dublin core) for the imagery case and over ISO 19139 geographic metadata implementation model. Some example of how to use the metadata schema proposed in the latter specification are there introduced and discussed and I think that they would be a good point of start for our discussion about stream metadata.

Image Metadata.

When talking about image metadata for a coverage we intend primarly identification of its geometry and CRS in a deterministic way. There are quite a few OGC papers covering these topics, all of them add something to the discussion but my impression is that, looking at them as a whole, we are still far away from having a clear picture of what we need and how to describe it.
The more important OGC publications related to these issues are Recommendation Paper OGC 05-027r1 "Recommended XML/GML 3.1.1 encoding of image CRS definitions", Discussion Paper OGC 05-014 "Image CRSs for IH4DS", Document Number 04-107 "The OpenGIS® Abstract Specification Topic 7: The Earth Imagery Case Version 5" and Discussion Paper OGC 04-071 "Some image geometry models".
Difficulties in describing geometry and CRS are comprehensible since we are moving in a fast-growing field which deals with terabytes of data daily coming from a wide range of sensor and devices (satellites, aerial photograps, models, etc...) but neverthless I think that we need to start with something which is widely agreed in our project and maybe after having gained some experience with these issues make some proposals to the OGC.
I can anticipate that I did not like these specifications very much, I found them quite a bit unclear and sometimes confusing. It is of course my personal impression but especially the first one which should be maybe the more important and which should provide the developer with a well defined template for CRS and geometry information still needs to be refined a bit (and maybe I need to read it again some more times (smile)).

Update on OGC specifications

Icon

As of August 2007, the following OGC specifications have been deprecated:

  • 05-027r1 - Recommended XML/GML 3.1.1 encoding of image CRS definitions

The following OGC specifications still in "discussion paper" stage:

  • 04-071 - Some image geometry models
  • 05-014 - Image CRSs for IH4DS
  • 05-015 - Imagery Metadata

The following are abstract specifications:

  • 04-107 - Topic 7 - Earth Imagery

The following are final OGC specifications:

  • 05-047 - GML in JPEG 2000 for Geographic Imagery Encoding Specification

ImageIO and Metadata.

In the following I will try to explain my understanding of the possibilities given by ImageIO package with respect of handling metadata in XML form.
Classes used by ImageIO plugins to represent image or stream metadata should be very plug-in dependent since they should be able to store and represent as much metadata as possible for their format. These classes should implement the abstract class IIOMetadata which expose the metadata structure as an XML DOM structure which should be described by a class implementing the IIOMetadataFormat interface which states a set of contraints on the type and number of child elements that an element may have, on the names, types and values of its attributes.
There are some rules to respect when designing the XML structure of these metadata which stand in between the restrictions introduced by DTD and XML schema. These rules are as follows:

  • Elements may not contain text or mix text with embedded tags (i.e. Java objects are allowed, but not mandatory).
  • The children of an element must conform to one of a few simple patterns, described in the documentation for the CHILD_* constants:
    • CHILD_POLICY_ALL A constant returned by getChildPolicy to indicate that an element must have a single instance of each of its legal child elements, in order.
    • CHILD_POLICY_CHOICE A constant returned by getChildPolicy to indicate that an element must have zero or one children, selected from among its legal child elements.
    • CHILD_POLICY_EMPTY A constant returned by getChildPolicy to indicate that an element may not have any children.
    • CHILD_POLICY_REPEAT A constant returned by getChildPolicy to indicate that an element must have zero or more instances of its unique legal child element.
    • CHILD_POLICY_SEQUENCE A constant returned by getChildPolicy to indicate that an element must have a sequence of instances of any of its legal child elements.
    • CHILD_POLICY_SOME A constant returned by getChildPolicy to indicate that an element must have zero or one instance of each of its legal child elements, in order.
  • The in-memory representation of an elements may contain a reference to an Object. There is no provision for representing such objects textually.
  • Namespaces are not supported.

It is worth to point out what Brian Burkhalter (one of the SUN lead engineers for JAI, ImageIO) about the first rule.

Useful Information

Icon

"May" refers to permission whereas "might" refers to probability. My reading of the foregoing specification is that elements are NOT permitted to contain NOR ARE THEY permitted to mix text with embeddded tags.

This clearly means that, we are not allowed to store textual data as object values for an IIOMetadataNode.
However, as he pointed out, the rules that I introduced above are not enforced anywhere, therefore we could, if we want just ignore them as far as it concerns with our native stream and image metadata. This fact is also reinforced here in the javadocs of the IIOMetadataNode class.

Experimentation with IIOMetadata framework.

I did some experiment with writing a small ImageIO plugin for grib files (which can be 5D as we need) and I have played a bit with image metadata to see how these rules can represent a limitation. It is worth to point out that the OGC papers for coverage metadata and CRS description propose some quite complex XML-SCHEMAs which do NOT abide by the aforementioned rules (you can find more on this topic here TBD).

The IIOMetadata subclass you are supposed to define for your plugin's specific image metadata is basically used to extract the metadata you need in a requested format (usually there is at least one native format specific to each plugin) and to present it using an XML-DOM structure. At this level there is NO checking for the rules I mentioned above, I tried myself to specify an XML-SCHEMA which was clearly in violation of them for the grib plugin I worked with and the result was that I was still able to request the DOM tree (using getAsTree(String formatName)) without any problems. Here below you can see an example of it for a grib record which in my vision represent a JAI RenderedImage:

The above XML code is basically trying to describe a small but meaningful set of metadata about a record of a grib file (which can be seen as a GridCoverage2D). It is obvious from this small excerpt that the IIOMetadata mechanism is pretty powerful and that it does not put any limits on what we can handle since it relies on XML-DOM. Here below the code I wrote to build it is reported:

Problems, I guess, should arise when we try to implement the IIOMetadataFormat interface since it should be basically used to describe the legal structure of the XML-DOM our plugin has to follow. An example of usage can be found here below where I reported a small excerpt of the ImageIO BMPMetadataFormat class:

what basically happens here is that we are describing the following XML-SCHEMA using a Java class which provides methods for checking object values, attributes cardinality and types, etc...

At this level the rule for IIOMetadata subclasses are enforced by not allowing the users to specify only element values which are references to objects, no other values are contemplated here as you can infere from the avalabile methods of IIOMetadataFormatImpl and from the fact that the possibility to specify the data types is avalaible only at the attribute level (these topic needs more investigation).
The only possible values for an Element are given by the return values for the method getObjectValueType(String elementName) and as you can see from here they comprehend only references or list of references ( see also getObjectClass(String elementName)).

It is finally worth to point out that namespaces are not supported in this framework, which would prevent us from support any qualified XML output, like as an instance GML. I tried myself to specify a namespace for an IIOMetadataNode, but as you can infer from here, there is no support for them in the IIOMetadata framework.

Which object to put in an IIOMetadata

In XML, an element usually contains text only. In IIOMetadata, it may be text but can also be a Java object (for example a CoordinateReferenceSystem object). In order to reduce Image I/O plugins dependencies toward Geotools, IIOMetadataFormat should not ask for any Geotools object type. But we can ask for GeoAPI object types.

So we have a choice: an IIOMetadata can stores CRS information as plain text (using a XML tree with textual elements only), as a full fledged Java object, or we can let the choice to developer. Using a Java object has the following advantage and inconvenient:

Advantage:

  • Plugins don't need to format a potentially complex object as XML.
  • No need to write a XML parser for CRS objects in the referencing module.
  • Plugins can create potentially complex CRS objects, including custom implementation that can't be encoded in XML.
  • May be more efficient (at least when the plugin returns a hard-coded CRS).

Inconvenient:

  • On the user side, it introduces a GeoAPI dependency.
  • On the implementation side, it introduces a Geotools dependency (or any other library providing CRS implementations). However, the users don't need to know that. Users only see the GeoAPI dependency.
  • No labels

11 Comments

  1. Even if GridCoverageExchange were stable, writing J2SE Image I/O plugin would still a very good idea anyway. I'm strongly in favor of coding a maximal amount of work as Image I/O plugin, and only the things very specific to GridCoverage as a GridCoverageExchange or Format plugin.

    I support the proposed plan. I agree that it is important to avoid any dependency to Coverage in Image I/O plugin at this time. However, this is less strict for CRS, since CRS should be much more stable than Coverage. Remember than CRS is an ISO specification and ISO is very reluctant to any changes unless there is very good reasons (I was in some OGC meeting where CRS changes were discussed, so I can witness for that). Current GridCoverage API is not the ISO one, which is why it is a much less committed API.

    So my suggestion is: let express CRS as XML if practical. Otherwise, I don't see usage of CRS interfaces in IIOMetadata as a problem. Coverage API should be avoided however.

    Note also that CRS parsing from XML is not yet implemented in Geotools, so some work may be needed there.

  2. Good. (smile) I figured this would be a little less controversial than multidimensional data selection. My reason for wanting the ImageIO plugin to not have any Geotools dependencies is largely so that the plugin will work outside of Geotools. I really only want to write my HDF MODIS ImageIO plugin once. I don't want to write it once for Geotools and once for a lightweight display/processing app that doesn't know anything about geospatial concepts. The stability issue is just what kicked me into action.

    Now on to how one might communicate CRS information through IIOMetadata... I'm doing a little poking around in the javax.imageio.metadata package, and I may have found a solution. It seems that one can define a "metadata format" in a way very similar to XML Schema. Here is an excerpt from the IIOMetadataFormat Javadoc page:

    The structures that may be described by this class are a subset of those expressible using XML document type definitions (DTDs), with the addition of some basic information on the datatypes of attributes and the ability to store an Object reference within a node. In the future, XML Schemas could be used to represent these structures, and many others.

    The differences between IIOMetadataFormat-described structures and DTDs are as follows:

    • Elements may not contain text or mix text with embedded tags.
    • The children of an element must conform to one of a few simple patterns, described in the documentation for the CHILD_* constants;
    • The in-memory representation of an elements may contain a reference to an Object. There is no provision for representing such objects textually.

    I do not know if IIOMetadataFormat objects are tied to specific file format, but my initial read of these pages indicates that a single format object could be used with a number of different file formats. It is also my impression that these format objects are very similar to XML Schema: they only specify the sequence/contents of the "tags" (or IIOMetadataNodes). The "format" objects do not contain code which produce valid metadata.

    I stumbled across something else which is interesting. To get metadata, you call the "getAsTree(String formatName)" This formatName is presumably the same string that is returned by the IIOMetadataFormat object's getRootName() method. From this, I presume that CRS-aware imageIO plugins could be asked for CRS information via a call to getAsTree("CRS"). Also presumably, this call would return only CRS information. We can distinguish between CRS-aware and not-CRS-aware plugins because each plugin should report the metadata format names supported by the plugin.

    The Javadocs explicitly say that the tags can't contain text, so passing a single tag with GML CRS would seem to be out of the question...

    So let me recap, since I did a lot of typing while I was trying to figure stuff out:

    1. Provided GML CRS abides by the constraints in the IIOMetadataFormat javadocs, we can describe GML CRS in terms of IIOMetadataFormat objects.
    2. We should only describe this format once, and not in each individual plugin.
    3. This description has nothing at all to do with parsing XML, nor does it constrain the manner in which the data is stored inside the image file.
    4. ImageIO plugins which do not support CRS information will not list our "formatName" as being supported.
    5. Requesting CRS information from an ImageIO plugin which does support it should filter out everything which is not CRS.

    From this, it would seem that a single GCE ImageIO wrapper could be made to adapt the IIOMetadataFormat CRS data to the Geotools CRS framework. Each plugin would be responsible for adapting between the file format representation and the IIOMetadataFormat representation. This is a pleasing separation of responsibilities.

    Big remaining questions:

    1. where should the GML CRS IIOMetadataFormat implementation live? (GeoAPI? Geotools? Deegree?)
    2. Is GML CRS implementable as IIOMetadataFormat objects?

    Is there anyone out there who has played with IIOMetadataFormats?

    Bryce

  3. GCE it is not just for files for formats, there is a Web Map Server GCE implementation.
    Each layer is available as an entry, and the format parameters communicate the available styles via metadata etc ...

    I am not saying it is good, just that we proofed the API with two implementations not one.

    If you guys need to make a simple API for access to Raster File Formats then please feel free. Note the existin API supports the handling of GC information in a single pass (as during a file upload to a web server).

  4. I've been seeing various emails go by. This is a good summary of some alarms and a response to those alarms. The non-quoted material below is from Martin.

    Reynolds, Greg a écrit :

    I spoke with one of our engineers that has worked with gridded data before, and he informs me that what you are trying to with rasters and raster metadata would fall within the GridCoverage API and whatever metadata mechanisms it exposes. You should certainly familiarize yourself with GridCoverage and possibly the GO-1 FeatureStore API's before delving deeper into proposing a new standard or JSR.

    Yes, yours engineers is right. But Bryce's proposal is largely driven by the fact that we have not yet switched GridCoverage interfaces to ISO 19123. The switch is documented as GEO-59 task on JIRA:

    http://jira.codehaus.org/browse/GEO-59

    As also added warning in GeoAPI javadoc:

    http://geoapi.sourceforge.net/snapshot/javadoc/org/opengis/coverage/Coverage.html

    Some peoples want to read rasters now. How can we support that without exposing them to too much compatibility break when we will implement ISO 19123? Bryce's proposal is to put as much dependencies on J2SE ImageIO framework as possible. His idea seems appealing to me:

    • Instead of writting a GridCoverageExchange and/or Format plugin for each raster format, write only one implementation for all raster formats. This implementation works with an arbitrary J2SE Image I/O plugin and looks for the CRS under some standard XML element in IIOMetaData.
    • Write one Image I/O plugins for each raster format. This plugin put the CRS (and other geographic metadata) in some standard XML elements in IIOMetaData, to be understood by the above-cited GridCoverageExchange plugin.

    This approach seems close to the spirit of the ongoing work at OGC, about GML and JPEG 2000 (as mentioned by James). JPEG 2000 can stores XML informations in an image, so the natural choice is to put CRS informations as GML right in a JPEG 2000 image. This is where OGC is heading (in my understanding). Bryce's proposal is to expose the GML information bundled in a JPEG 2000 image as a J2SE's IIOMetaData, and apply the same concept to all other raster formats (the Image I/O
    plugins would have to do the conversion itself).

    Advantages of this approach:

    • Insulate the raster plugin from any change in GridCoverage's API.
    • Fits naturaly with the ongoing OGC work on JPEG 2000 + GML.

    Note that one approach doesn't exclude the other. Nothing prevent a developper to write directly a GridCoverageExchange plugin if he wish so.

    Martin.

  5. James Macgill asked:

    How, if at all, does this relate to the recent work with GML and JPEG2000. i.e. is there a defined subset of GML for image metadata that we could leverage?

    They are two different animals.

    1. GML + JPEG 2000 is a particular file format. While I don't know the specifics of what will come out of that, the specification will concern how to store XML in JPEG 2000 and what that XML should contain. It will only apply to this new format they concoct.
    2. Defining a Java-consistent way of expressing CRS information as an IIOMetadataFormat does not constrain the representation of CRS data in any file. It applies to GeoTIFF, ArcGrid ASCII, and World + Image just as much as to JPEG2000. It also makes Geotools, Degree, and OpenMap capable of exchanging ImageIO plugins, as the reliance of a particular implementation of CRS intelligence has been completely factored out of the plugin. The plugin merely needs to know how to express CRS information as a collection of IIOMetadataNodes.

    The "lucky coincidence" that IIOMetadataFormat and XML Schema have very similar definitions merely makes GML a "convenient" target.

    As to the other question: GML 3.1.1 definately has an "entry point" (predefined subset) for CRS information. I don't know how it handles other image metadata.

  6. Martin Observed:

    Because of OGC's work on JPEG 2000 + GML, I don't believe that there is a need for a new OGC group or JSR specification. I think that it should be possible to leverage their work.

    It's certainly not necessary that it become a specification, but I do not think that OGC working on JPEG 2000 + GML will result in anything like a standard expression of CRS information in Java for all file formats.

    The OGC is very data-compatibility oriented. The new file format will be accessible from multiple platforms having different byte-orderings and word lengths from different languages. But it is only one file format.

    My suggestion concerns only one platform (J2SE), but all file formats for which an ImageIO plugin is written. ImageIO plugins are accessible to all Java code, so standardizing the expression of CRS metadata within the ImageIO framework is a natural step: especially when the hooks to do it are already provided by Sun and are available everywhere.

  7. In my understanding, "getAsTree(String formatName)" is not really for fetching an element like "CRS". It is rather for complying to different standards.

    CRS is not the only information needed. We also need the bounding box, sample dimensions, etc. So "CRS" would be a little bit to restricted as a name.

    For a standard way to organize the informations in a XML-like tree, we should look at the OGC's working group on JPEG2000 + GML. Some possible relevant documents on http://www.opengeospatial.org/specs/?page=baseline are:

    • Recommended XML/GML 3.1.1 encoding of image CRS definitions
    • GML in JPEG 2000 for Geographic Imagery

    About question 1 (where should the implementation live), we can exclude GeoAPI. This project is for interfaces only. Other possibilities (Geotools, Degree, other) is at the implementor's choice (smile).

  8. This is true that the JPEG2000 + GML working group target specifically one format, but it seems to me that their work should be applicable to IIOMetaData for arbitrary raster. If the image information is going to be encoded as GML, even if the XML is going to be bundled in a JPEG2000 image, I would expect large part of this XML format to be applicable to other image raster as well... Anyway, it should be close to identical to what you planned to do, since the XML in JPEG2000 should be GML 3.1.1 (or close to that). But I have not looked at the GML + JPEG2000 specification, so I can't speak much about that.

  9. There has been much talk so far about the GML + JPEG 2000 specification. So far, I think that this is Martin thoughtfully provided a link and sure enough, the specs are right there on the OGC website. THey weren't there the last time I checked.

    Two specs interested me enough to make me download them and think about them this weekend:

    1. 05-027r1: Recommended XML/GML 3.1.1 encoding of image CRS definitions
    2. 04-045: GML in JPEG 2000 for Geographic Imagery

    05-027r1 doesn't even have the word "JPEG" in it (I did a "find".) It should be used for anything we do with IIOMetadata, as it has been decoupled from file format. Presumably, this is the vocabulary used inside JPEG2000, as well as by WCS implementations. 04-045 describes the GML + JPEG 2000 file format, which should be referenced for writing a GML+JPEG 2000 plugin. There's another paper which might be useful: RP 05-011, "Recommended XML/GML 3.1.1 encoding of common CRS definitions", however, this paper asks for a password.

    I need to read/absorb and let this percolate.

  10. I looked at these two specs over the weekend. I'll expound on GML + JPEG 2000 elsewhere and add a link. In short, the term "everything including the kitchen sink" would seem to apply. The GML associated with the image includes annotations, other features, styling, you name it. Also, with the exception of Clause 8, I believe, there's nothing specific to the JPEG 2000 format. As long as a handy way of binding an XML file to an image can be found (PkZIP, tar, files in same directory, etc.) most of the spec applies to the process of arbitrary image annotation with GML.

    The other spec (05-027r1) contains examples of how to use GML to encode CRS information. Basically, you make an XML file with the root node being the CRS tag of interest (ImageCRS, ProjectedCRS, GeographicCRS, DerivedCRS) and provide all the supporting information. There are some finer points I don't quite get yet. For instance, the paper exists to provide "reference CRS instances", not just examples of how one might typically approach the problem. The reason for this eludes me.

    In any case, 05-027r1 is much closer to my vision of how this IIOMetadata thing might work than GML + JPEG 2000. 05-027r1 doesn't encode "associated features", nor does it fiddle with styling, nor does it label anything, it just communicates CRS information. There's something to be said for keeping it simple.

    There is a subtle difference in how the two papers handle CRS information, however. 05-027r1 recommends an XML file with a CRS tag as the root element. GML + JPEG 2000 recommends an XML file with a FeatureCollection tag as the root element, which contains a "Coverage" tag, which contains CRS information. It would seem that GML + JPEG 2000 would violate the OGC's own recommendation for supplying CRS information, were it to be accepted as is.

  11. As of August 2007, 05-027 ("Recommended XML/GML 3.1.1 encoding of image CRS definitions") has been deprecated while 05-047 ("GML in JPEG 2000 for Geographic Imagery Encoding Specification") graduated to final OGC specification stage.