One long standing request of the GeoTools code base is to offer an operations api for working on Features (similar to what is available for grid coverage).
The idea here is to have a low level interface to handle a very simple kind of operations on data.
Here are examples of what I call "usual operations on data" :
Previous attempts have been limited in scope to lack of collaboration and focus. Often they were created in isolation by people wanting to write feature hacking stuff; rather than by desktop applications.
- ogr2ogr binded in a process
FWtools gives a set of .exe files very interesting, ogr2ogr is one of them. This exemple is to demonstrate a process can be linked to native executables.
- shoal technology to send process on different processors
this exemple is to raise the problem of dispatching processes anywhere, so process should not have relation with threads.
Hot-Swap and Dynamically Defined Process
We have the request from Simone to support hot swapping of process implementations; as such assume he would use OSGi to swap out an old plug-in and swap in a new one. A method to kick the ProcessFinder into searching the classpath again would thus be needed.
The problem of dynamically defined processes (say as a groovy script) is a bit more tricky since we are a Java program and like to work with classes and instances. I hope that Dynamic Proxy can be used here by whatever developer wants to try the idea out.
Eclesia has three main reasons:
- Offer a structure for usual operations
- Once a low level process is written, Swing or SWT Widgets can be made separately.
- Thoses processes can be quickly adapted in ETL applications
Jody has two reasons:
- Kick valuable code out of the uDig codebase for wider use (such as the reshape operation)
- Make use of Eclesia's enthusiasm
- Want to make sure this stays simple enough to work; the last four attempts failed to catch on
Relationship to Web Processing Service:
- no really we did not look at it first; consider this API as raw ability that you could wrap up as a WPS
- you could also hide an external WPS behind this API
Requirements and Acceptance Tests
Acceptance tests are the best definition of scope, the final API we present here will meet the following requirements.
- The best solution is the ones that is easiest for implementors to make a new Process for us; that is simply measured as number of lines of code (the burden can be eased with a nice abstract super class).
Controlling the scope of "process description":
- The ability to create a Swing user interface based just on the Process description
- The ability to create a WPS GetCapabilities and DescribeProcess
Controlling the scope of "process":
- The ability to execute a process
- The ability to track a process that is executing
- The ability to interrupt a process that is executing
Not quite in scope:
- The ability to split a process in two, and merge the result (see FeatureVisitor for examples)
- The ability to chain several processes together
While our design will address these concerns, we will only implement what we need at this time. It is way better to wait until someone has a real live problem in hand in order to test the solution.
The API is currently being defined using code - the final API will meet several acceptance tests.
First Attempt - ProcessFactory and Parameter
- Write down an initial starting point
- When implementing a process you have a strange dance where the inputParameters have to be provided along with you factory as part of the constructor You also need to run over to your ProcessFactory and make references to the static final Parameter implementations it has in order to lookup keys in your inputParameters, and store your resultParameters
- Not having the inputParameters as part of the Process interface made it impossible to use just a Process object on its own in normal Java code
- Not able to handle multiplicity
- unclear that you can only call process once? or can you call it multiple times ...
Single thread example (say from a main method):
Multiple threads example (say from a Swing Button):
Second Attempt - Process and Parameter
- makes the process method accept input parameters; as a result is is much easier to code up this method, the inputParameters are visible in our interface and can be documented
- this has the side effect of making processfactory / process split non useful (since the process method is not stateful)
- strictly admit to isNillable, minOccurs, maxOccurs in the Parameter api (leaving metadata to address user interface concerns)
- name has been provided for Process; to allow for dynamicly generated processes that cannot be strictly identified using a classname
- panic that there is not an explicit Process object around that can be manged in a queue (this was a surprise to me since whatever job system is used you will need to write a wrapper around the bundle of inputParameters and Process; even if it is just a SwingWorker or Runnable).
To repeat the design goal: we need to make Process easy to implement - wrapping the result up for a job system or dispatch to a grid system is not our concern.
- need to handle up multiplicity
Third Attempt - ProcessFactory and Parameter with a standalone Process
- make Process a stand alone object so that it can be more easily managed / wrapped by job systems such as Runnable, SwingWorker, Eclipse Jobs, etc...
- Create an AbstractProcess that takes care of a lot of the grunt work so that the poor implementer has a small number of lines of code to write
- Strictly seperate out value constraints (type,isNillable,minOccurs,maxOccurs) used by the framework from the general metadata used to assist the end user
- Define a subclass of Process that has the ideas of Splitting and Merging in order to handle distribution of a process across multiple threads (the alternative is some more control metadata at the Process level)
4th Attempt - Process and Beans
- Benefit all the bean property description to build user interface "on the fly"
- Less code for implementers to write
- NetBeans RCP : really easy to embed
- Down side is that minOccurs, maxOccurs kind of information cannot be provided, poor integration with List<Geometry> and internationalization support is poor (based on GeoServer experience)
- desruisseaux asked : why not extending the runnable interface (java standard) and replace method process by run ?