Skip to end of metadata
Go to start of metadata

Project Status Board

Icon

For Review and Comment: Coverage Plugin Use Cases
Design and Implementation: A chart of personnel and a list of major tasks: OpenOffice or PDF

Summary

This effort can be defined as an attempt to provide a unified data access API to an n-Dimensional dataset where some subset of the dimensions contain a known spatial and/or temporal definition.

This page should be regarded as a touchstone for the coverage augmentation project. It collects into one location many diverse and detailed elements of design, experimentation and implementation.

Motivation and Scope

This page describes why we are exploring the augmentation of GeoTools' coverage functionality. It also explains that we are limiting ourselves to 4D coverages with bands, and why.

Overall Design Guidelines

Much pondering, many discussions, and some experiments have taken place over the past months. Some of these are captured on the related pages. The result of these activities is a fairly accurate notion of the required functionality and desired characteristics of an nDimensional subsystem. Here is a summary:

  1. To the maximum extent possible, plugins for specific data formats should remain ignorant of spatial or temporal significance of the data they transport. When dealing with gridded data, plugins should deal only with indicies into the grid. When storing or retrieving spatial referencing information, plugins should concentrate on copying descriptive parameters from metadata object to file or from file to metadata object.
  2. Data should be accessible by either it's location in the grid, or by it's spatio-temporal location. The conversion from grid location to spatio-temporal location should be exposed to the interested user.
  3. The index into the gridded data should be unified and symmetric.
  4. In order to leverage the performance and intelligent tile caching of JAI, it is highly desirable to design a system compatible with the J2SE ImageIO framework.
  5. Plugins developed for our nD framework should be accessible and usable outside of GeoTools, and maybe even outside a geospatial framework. (e.g., A good netCDF plugin which can extract data slices might be usable in any 2D imaging program, even without spatial meaning.) If this is not possible, then a minimum set of classes and interfaces should be collected into a module which does not depend on the remainder of GeoTools.

Project Organization and Status

The following people and organizations have expressed interest in or are contributing to this project:

Organizations

  • USDA Forest Service / Fire Science Lab : USDAFS
  • NURC / Nato Undersea Research Centre : NURC
  • IRD / Institut de Recherche pour le Développement : IRD
  • ITC / International Institute for Geo-Information Science and Earth Observation : ITC

USDA Forest Service / Fire Science Lab.

Our group is developing an air quality forecasting model which takes real time fire detections from MODIS as input. We have co-developed and validated a prototype burn scar algorithm to refine the perimeters generated from hotspots alone. Geotools is used in our processing chain, largely because it's extensible. We have commercial products available to us, but they haven't implemented the functionality we need and we can't add it ourselves.

Having gained some experience programming using Geotools, we adopted Geoserver as our display mechanism. Our philosophy in doing so was that we wanted everything to be capable of being displayed on a single map. As we are integrating satellite imagery, fire perimeters from incidents, existing database sources of MODIS Hotspots (which we also developed), and the output of numerical weather prediction models (WRF), we surmized that existing solutions capable of ingesting all of this data were rare. These were usually in the category of visualization programs which could produce a static output or interactively view the data on a local workstation. We required a web based solution, capable of distributing not only finished map products to the public, but also the underlying data to collaborators and associated modeling centers. Our appraisal was that it was easiest to satisfy all our requirements by adding the missing parts to Geoserver/Geotools.

NURC / Nato Undersea Research Centre.

Our group is comprised of people with various backgrounds, oceanographers, eletronic engineer, telecommunication engineers, software engineers. Our area of interest currently covers various aread including remote sensing applications related to ocean colors products and remote surveillance, sea bottom type classification, tactical decision aid applications and, of course, OGC Web Services implementations in order to support the management of the vaste amount of data we acquire, process and produce during our research activities.

Development of OGC-enabled Web Services using GeoTools and GeoServer came in the path after having evaluated various Open Source products like Minnesota MapServer with GDAL/OGR and Deegree and astime goes by we get more and more confident about this choice!

ITC / International Institute for Geo-Information Science and Earth Observation

The Geo Software Development unit supports research and education at ITC. Since 1985 we developed our own, mainly raster based, GIS/RS software ILWIS. Datasets are becoming larger and larger in number of bands and in time, standards are getting mature, and more non-professional software developers want to develop themselves. Thus we stop the evelopment in C++ and move to Java. To prevent reinventing the wheel and to promote reuse we think it useful to cooperate in GeoApi/GeoTools. With con terra and the university of Muenster we are cooperating in 52°North.

Personnel

Institution

Name

Availability

USDAFS

Bryce Nordgren

Very task specific. Available 100% for mission-critical items. Available 0% when occupied with unrelated emergencies.

USDAFS

Alex Petkov

Available approx 50% perpetually for all aspects pertaining to our geospatial infrastructure.

IRD

Martin Desruisseaux

Task specific. Available 100% of the time for some issues like ISO 19123, port of legacy implementation, etc. But often distracted by unrelated emergencies.

ITC

Wim Koolhoven

Limited.

Project Tasks and Progress

Loading
Type Key Summary Status Assignee

Macro params are invalid

Alessio does not have a JIRA account that I'm aware of, or more stuff would be assigned to him. Some of the things assigned to Simone in the previous table really belong to Alessio.

Initial Design

Click here to view/download an initial summary of design considerations, implementation components, and a synthetic interaction between a GeoTools client, the GeoTools framework, and a GeoTools grid coverage plugin.

An initial use case document from the perspective of a Coverage Plugins has been prepared. Click here to download the OpenOffice document. Edit and save your changes, then email to bnordgren@fs.fed.us. I can use the "Compare Documents" function to see what you had to say. Alternatively, you can turn Change Tracking on.

This problem decomposes nicely into two equally important, yet separate topic areas: data and metadata. Each topic is going to have it's own wiki page to collect thoughts and design notes.

Wiki pages for major design components:

Standards under consideration

  • We are considering the use of ISO19115 as the common language which plugins and the GeoTools framework can speak. There is a comprehensive vocabulary for the description of coverage geometry and CRS. See the metadata page for more information.
  • We are considering ISO19123 as the coverage framework. It offers a separation of access-by-grid-value and access-by-geospatial-coordinate. See the data access page for more details.

Interoperability Note

The design of this project is going to occur with commonly available open-source or community-edition (e.g., Free-as-in-lunch) tools. UML diagrams will be generated with Poseidon Community Edition and design documents will be written with OpenOffice 2.0. These tools are sufficiently powerful and available to address our immediate design needs. Where appropriate, important documents will be exported to PDF format and placed on the WIKI. However, the original documents will (eventually) be checked into Subversion for others to edit. (NOTE: This does not constitute an endorsement of these tools by GeoTools, NATO, or the USDA Forest Service.)

Community Discussion thus far

Coverage Augmentation Wiki Space

  • No labels

3 Comments

  1. Actually, data sources don't need to choose which 2D slice a grid coverage represents, since grid coverage can have an arbitrary number of dimensions. Coverages have the same number of dimensions than the underlying coordinate reference system (CRS), and CRS can have an arbitrary number of dimensions. I have seen somewhere the misconception that CRS attached to the earth surface can only be two-dimensional. It would be almost accurate if CompoundCRS didn't exist. But with CompoundCRS, user can really build a wide range of CRS including time and vertical axis.

    The list given above (third spatial dimension, time, channel wavelength, etc.) can be separated in two groups:

    --------------------------------
    Group 1: CRS dimensions
    --------------------------------
    CRS (and consequently grid coverage) can be 3D, 4D, etc. with third spatial dimension and time axis. The Geotools GridCoverage2D implementation is a special case that represents a 2D slice. But as the "2D" suffix suggests, this is really a special (but common) case; the GridCoverage interface is absolutely not limited to 2D. It is worth to point out that Geotools also provides other implementations, like CoverageStack. CoverageStack takes a list of n-dimensional coverages where each coverages is a slice in a (n+1) dimensional space, and represents them as a single coverage in a (n+1) dimensional space.

    Given the above, I would say that a data store should not cut any slice (unless the user asks explicitly for that). If the data source as a dimensionality higher than 2, its default behavior should returns a GridCoverage of that dimensionality. If one or more axis do not produce data on regular intervals (e.g. a logarithmic distribution), then the data source should returns a plain Coverage instead of a GridCoverage.

    -------------------------------------
    Group 2: Sample dimensions
    -------------------------------------
    Channel wavelength and named parameters could fall in the "sample dimension" group I think. "SampleDimension" is a generic term. For a GridCoverage, a sample dimension is a band in an image. Each wavelength channel is a band. Same can applies to wind velocity components: in our laboratory, we have geostrophic current computed from sea level anomaly (SLA). Our GridCoverages have 3 sample dimensions: SampleDimension 0 contains sea level anomaly, SampleDimension 1 contains the U component of geostrophic current, and SampleDimension 2 contains the V component of geostrophic current. Those data as represented as a series of 2 dimensional GridCoverage, each of them at a different instant in time. The set of all GridCoverage2D are put in a CoverageStack, and the whole set of SLA,U,V data appears as a big 3 dimensional Coverage (where the third dimension is time).

    One more note about named parameters: All the above description applies to current coverage API. However, this API is under revision. Stephane Fellah (who worked at PCI geomatic, the ones who published the GridCoverage implementation specification 1.0) is working on a new proposal with a more elaborated API for named axis.

  2. The topic of data selection arises because I don't believe there is a standard for it. Also, I think that if we're going to take the route of making J2SE ImageIO plugins, the plugin should make a pretty picture even when it's not used with Geotools. I think people should be able to control which 2D slice they are seeing (even outside of Geotools). Not to mention the fact that if you really are only interested in a single 2D slice, you do not want to load the entire cube in, then pick out what you want later!!! S L O W. I think you can pretty much assume that any dataset which requires this kind of control is going to be very large. In C, processing these datasets tends to be IO bound, so you use every trick you can to reduce the amount of IO.

    What I would like to advocate is some sort of extension to IIOParam/ImageReadParam which would select a 2D slice out of an nD dataset. The problem with IIOParam/ImageReadParam is that they currently assume that the plugin knows which dimension is width and which is height. It provides no facility for changing/specifying that assignment. Everything else seems to be in place:

    • Source band selection
    • Rectangular region selection
    • Remapping from source to destination bands.

    I kind of envision a separate package, with no dependence on Geotools, which has the following components:

    • An extension to ImageReadParam allowing for slice selection.
    • Some abstract classes which implement some of the common functions:
      • Calculating strides
      • Calculating bounding indicies
      • Calculating geometry (width/height)
      • Instantiating an appropriate RenderedImage implementation based on what's available (TiledImage, PlanarImage, BufferedImage.
      • Recognizing whether the user is using an ImageReadParam or the "extension".

    Our Geotools nD plugins should be implemented on top of this package, and the Geotools GCE wrapper should know enough to use the extension to ImageReadParam when it is beneficial. Of course, if more than 2 dimensions are desired, (for processing or whatnot) they can all be loaded in. Inside Geotools, users should interact with the GCE/ImageIO wrapper and they specify the bands they want based on
    some Geotools standard for selecting a slice out of nD data. At the Geotools level, the slice doesn't have to be 2D, but at the ImageIO level, it does (needs to assign a "width" and "height" dimension.)

    Am I making more sense now? (smile)

  3. Excerpt from the OGC Web Coverage Service (WCS) implementation specification.

    This excerpt I am reporting here should clarify a bit more what we intend for a multidimensional grid.

    Grid coverages have a domain comprised of regularly spaced locations along the 1,2 or 3 axes of a spatial coordinate reference system. Their domain may also have a time component, which may be regularly or irregularly spaced. A coverage with a homogeneous range set defines, at each location in the domain, either a single (scalar) value or(such as elevation), or a series (array/tensor) of values all defined in the same way (such a s brightness values in different parts of the electromagnetic spectrum).

    Basically a multidimensional grid (which could be represented as an OGC GridCoverage as Martin pointed out) is made by a domain set comprised by a spatial domain and (optionally) a temporal domain which defines its 4 physical dimensions (x,y,z,t). The fifth dimension is represented by the range set or sample dimensions which (excerpt from the same specification):

    defines the properties (categories, measures or values) assigned to each location in the domain. Any such property may be a scalar (numeric or text) value, such as population density, or a compound (vector or tensor) value, such as income by race, or radiances by wavelength.