Skip to end of metadata
Go to start of metadata

Overview

This page outlines the project for bringing the ISO Feature implementation laying on the GeoTools fm branch and the ComplexDataStore module laying on the GeoTools complex-features branch as unsupported modules on GeoTools trunk, which are meant to be ran in some flavor of GeoServer build, either on its own branch (discouraged) or through a community plugin in GeoServer trunk.

The purpose of this capability is to serve data in formats specified by a community of use, not dictated by whatever random persistence schema or file format the data is in. 

This is not radical (and the ISO concept of Feature is quite general enough to handle real world data), but there is a legacy of GIS systems oversimplifying data into denormalised "flat" tables with scalar value attributes. We need to support time-series, objects with 0, 1 or more geometry attributes, complex data types and relationships between Features, such as when one feature has a property that associates another feature (a junction joins two or more road segments).

Project proposal

Goals

This project has three major goals:

  1. The availability of the ISO Feature implementation from the fm geotools branch on trunk, ensuring the Filter implementations plays well with it.
  2. The port of the ComplexDataStore module from the complex-features geotools branch as an module on trunk, using the new ISO Feature Model and providing the same functionality than in complex-features. Target is working, unsupported module, with the goal of seeking supported status in the future.
  3. The availability of some sort of GeoServer build that provides the same business driver community schema mapping capabilities than the geoserver cpx branch, preferably through a community module on trunk.

Background

Before stating the concrete scope of this project, let present some background on the current state of play:

ISO Feature model status.

The ISO Feature model provided by GeoAPI trunk has yet to be stressed out with real world use cases, hence it cannot be taken as final API, but its subject to corrections, if proven needed. While geotools's legacy feature model was extended for the complex-features branch copied the business goals of that initiative, it proved too much GML oriented and a group of volunteers took the work of designing the new Feature Model inspired on the OGC General Feature Model, which by all means is a more appropriate approach given the nature of the GeoAPI project. As a result, though, some structural modeling power was lost, specifically the ability to create sequence, choice, and all structural descriptors, which might well appear in a GML schema but are not concerns for the General Feature Model. So, this separation of concerns will have to be addressed later when we need to create GML documents respecting the externally defined schema.

ISO Feature model implementation status

The ISO Feature implementation laying on the fm branch was used to test the API, and was intended as an eventual replacement for the somewhat crippled geotools Feature API and implementation. Provision was taken during its implementation to provide meaningful ways of dealing with the general complexity of the new model, like the existence of builders for types and complex attributes, as well as the open ended userData properties on the type definitions.

Data access API

GeoTools provides its own Data (feature) access API, with the DataStore interface as the entry point. At a first glance, it would be almost perfectly suitable for accessing ISO Feature instead of geotools Feature, provided that it is modified accordingly to provide the former instead of the later.
The known limitation for the DataStore API regarding the goals of the community schema requirements is the lack of separation between the concepts of TypeName and AttributeDescriptorName. Translated to GML jargon, this means DataStore does not differentiates between the name of a Feature (an element in the _Feature substitution group), and its type (a type derived from AbstractFeatureType. At the DataStore level, they use to be the same thing, generally the name of a table in a RDBMS or the name of a file in disk.
At the GeoServer side, for example when a WFS DescribeFeatureType is issued, GeoServer responds with a XML schema fragment with the DataStore provided type name as the Feature's element name, and encodes the feature type with the same name plus a _Type suffix.
So, for example, for a postgis table names ROADS, the DescribeFeatureType response contains a ROADS element declaration and a ROADS_Type type definition.
As another example, consider the one used in the WFS 1.1 spec: a DescribeFeatureType for the TypeName}}s {{myns:TreesA_1M and myns:RoadL_1M returns a schema with the myns:TreesA_1M and myns:RoadL_1M element declarations and the myns:TreesA_1M_Type and myns:RoadL_1M_Type type definitions respectively.

Though this situation wasn't a major concern for the kind of content GeoServer is used to serve through DataStore, it becomes important for the community schema project to be able of distinguishing between Feature name and Feature Type name.

So it becomes important that DataStore, or whatever the ISO Feature data access API ends up being, is able to access content through Feature name

GeoServer trunk

TODO!!!

Restrictions

With the new modularity approach, and improvements in abstracting Filter and Expression, it is possible now to inject these capabilities into the geotools trunk. Well, in trunk/unsupported, with the following and perfectly understandable limitations:

  • Do not add any burden to any supported module

That is, don't touch anything but our unsupported modules, except for fixing a bug that don't allow us to go ahead.

Scope - Relation to broader and cross project planning

The Community Schema Road Map page outlines the PMC proposed plan to bring home (aka, trunk and supported) both the ISO Feature implementation and the ComplexDataStore module. General consensus is that such an effort shall be preceded by the availability of the resources necessary to fulfill the estimates at the end of the page.

On this run, however, we're under a much more restricted scenario, with specific goals, tight deadlines and restrictions:

  • the port of ISO Feature implementation as an unsupported and self contained module, not its integration with the rest of the library (where integration means the merge of both models, as proposed by Justin and agreed by other developers).
  • the port of ComplexDataStore as an unsupported module, maybe using an alternative "data access API" to overcome the limitations of the current one in functionality and in Feature model usage, not fixing the current DataStore API, nor formally proposing a new one or an extension of the current one.
  • to provide the ComplexDataStore port with the same functionality as the one in commplex-features branch to the maximum extent possible, not adding any new functionality. (Some bug fixes will be undertaken post porting however)
  • to provide a means of accessing the same business driven community schemas as the ones used to test the GeoServer cpx branch, through a community plugin in trunk or in the worst case on a branch, for the WFS GetFeatures and DescribeFeatureType operations, not to make it work with transactional operations.
  • to allow ComplexDataStore to be used by a WMS GetMap request only if possible, or to identify and report back the work needed for it to function.

Benefits

Among the business benefits for the project's stake holders, the GeoTools and GeoServer communities will benefit from this project in two ways:

  • first, the project artifacts will serve to identify the actual suitability of the geotools and geoserver codebases for the incorporation of a major api shift. Work has been done in the past months to prepare the floor for the next big step, mostly on the Filter front at the geotools side, and on the dispatching and plugin system on the geoserver side.
  • second, the availability of a working prototype both for the feature model implementation as for the community schema support module on trunk, though unsupported, will bring up the concern to the wider developer community which will help to identify the implications of the shift throughout the whole library, and allow for a more accurate planning.
  • Geoserver will be seen as a viable platform for the many Spatial Data Infrastructures appearing nationally, locally and internationally, all of which are predicated on being able to deliver specific data products.

Project Plan

Schedule

Phase

Deliverables

start

end

status

1

  • ISO Feature port to trunk

2007-02-02

2007-02-15

(tick)

 

  • ComplexDataStore port to trunk and ISO Feature

 

 

(tick)

2

  • GeoServer build with ComplexDataStore running the community schemas from previous project

2007-02-16

2007-03-01

 

3

  • GeoServer build tested over various schemas over geometryless DataStore

2007-03-02

2007-03-15

 

 

  • Project report feeding back general planning

 

 

 

Issue Tracking

GeoTools

http://jira.codehaus.org/browse/GEOT-1303

GeoServer

http://jira.codehaus.org/browse/GEOS-1145

Project reports

ISO Feature port to trunk

Overview

The FM was ported as a new module,unsupported/community-schemas/fm. The port was pretty straight forward simply because we keep it completely isolated, so just needed to ensure the unit test suite completely passes.
(warning) Attention shall be given to the fact that actually integrating the FM implementation on the supported codebase means affording the effort described in the Feature Model Preparation page, included as the first step in the Community Schema Road Map.

Packaging

So we need the FM implementation on trunk with no burden on the existing stuff. To overcome the possible naming conflicts with the current stuff in geotools main we used the java packaging system to isolate it, inside the java package org.geotools.feature.iso.

Filter support

A small deviation of the specific porting job has to be done in order to get the geotools Filter implementations being able to evaluate ISO Features. Though the grunt work were already done by other developers, at the time of asking a Filter instance to evaluate an ISO Feature instance the issue described in the following jira issue arose and needed to be fixed.
The new way in geotools to support the evaluation of PropertyName expressions over whatever Feature model is being used is the PropertyAccessor interface. An implementation for ISO Feature was added and registered through the usual geotools factory system.

ComplexDataStore port to trunk

Porting ComplexDataStore as an unsuported module on trunk involved:

  • an API shift from the legacy feature model in complex-features branch to the new ISO Feature model and
  • an API shift from the old geotools Filter interfaces to new new GeoAPI ones
  • the use of an alternative Data Access API
  • the development of an eclipse emf based GML/xsd to FeatureType parser

Adaptation to the new Feature Model

The feature model used in complex-features, though not appropriate as a General Feature Model, were specifically designed for mirroring a GML schema, and almost appropriate for any XML Schema. Switching to the new ISO Feature model wasn't any hard, though it meant loosing the ability of modeling structural particles like sequence, all and choice. See the Schema parser section bellow to see how we expect to overcome this limitation at gml encoding time.

Filter

Switching to the GeoAPI Filter interfaces was pretty trivial since the GeoTools implementation on trunk is already up to the shape. Among the trivial work of the api shift, it was needed:

  • developing a new filter to extract the property names in a Filter: FilterAttributeExtractor used in the process of translating the attribute mappings between the community schema and the surrogate one. (warning) This should replace the one with the same name the main module since that one does not works well with GeoAPI Filters.
  • and the reimplementation of UnmappingFilterVisitor based on GeoAPI Filter, used to translate a Filter meant to work over a community schema to the schema of the underlying surrogate FeatureType.

Data Access

The need for an alternate Data Access method at least to achieve this specific project goals was described above in the Background section.
Fortunately, on the past months a temptative new general DataAccess API were introduced, with a FeatureAccess flavor meant for feature access. Though aware of the decision is only going to last for this run of the project, it seemed like a perfect fit for our needs, since FeatureAccess extends the old and good DataStore, so it serves for our purpose of being able to use any existing DataStore as a surrogate of ComplexDataStore without the need to actually porting the DataStores (and thus the whole library) to the ISO Feature API. To do that we're providing a DataStore->FeatureAccess adapter, which is pretty easy since DataStores uses only Simple Feature.
So, as ComplexDataStore is and works with FeatureAcccess, DataAccessFinder (the counterpart of DataStoreFinder), gets a DataAccess of any existing DataStore out of the box.

Schema parser

At the time the complex-features branch was created, the were no an existing parser to get {{FeatureType}}s out of a community schema in XML Schema files, so we used a quick, dirty and ugly one with lots of limitations.
Now the scenario is completely different, GTXML, an outstanding technology developed by Justin Deoliveira for the use of real world amounts of data through XML came to help. It is based on the Eclipse Modeling Framework, and provides us ready to use incarnations of the GML 3.1.1 schema and its dependencies.
On the other hand, EMF fits perfectly as FeatureType metadata so we don't lose a single bit of the actual community schema we're working with, since the emf type definition object in the schema can be stored as a "userData" property for each FeatureType, as well as the emf element declaration for each AttributeDescriptor. (warning) This information could well serve at a later step to actually be able to creating valid GML documents for the community schema by the use of a schema assisted encoder.
So we just have thrown away the old and ugly schema parser in favor of a new, emf based one, that reuses the GML to FeatureType bindings in GeoTools. Note that since these bindings exists for the current GeoTools Feature Model, we're again using adapters to go from geotools FeatureTypes to ISO FeatureTypes. This situation is temporary though, and should last only until the ISO Feature model is adopted, in which case the GML bindings are going to be ISO FeatureTypes by them selves.

Support for mapping the target model to the persistence layer

(warning) TODO

The ability to work with a accurate in-memory representation of the target feature allows styles and queries and encoding rules to be predictable. The task is then to map data from a surrogate data store, reflecting the local persistence layer, into the feature model, and to map queries back to the data store's native capabilities. 

This was achieved in the complex-features branch  

GeoServer build

(warning) TODO

  • No labels