The meeting that did not fit into the meeting ... conversations on DataAccess and GridAccess with jgarnett, acuster, simone and grabiel. This conversation will be documented as a proposal for a an IRC breakout later in the week.
simboss: jgarnett
simboss: got 5 minutes?
desruisseaux: Maybe a few minutes while I'm shuffing down my other applications...
desruisseaux: I need to leave. Bye all!
pramsey left the room (quit: Read error: 104 (Connection reset by peer)).
jgarnett: hi
desruisseaux left the room (quit: "ChatZilla 0.9.80 Firefox 2.0.0.11/2007121614").
pramsey n=pramsey@mail.refractions.net entered the room.
jgarnett: I am here simboss; sorry this meeting was such a mess.
jgarnett: I really wish the xml binding stuff was solid.
simboss: np
simboss: quick question
simboss: on coverage data access
simboss: I have been willing to start writing up a proposal for years
simboss: byut anyway
simboss: I have been thinking lately
simboss: about it for toher reasons
simboss: if you factor out dataaccess
simboss: you envision
simboss: the possibility that coveragestore
simboss: will be a child of dataaccess?
jgarnett: nope; I tried that and people did not like it enough to give me feedback.
jgarnett: I think gabriel's idea of
jgarnett: taking what WCS needs may be better.
aaime: simboss, something based on wcs spec could have various benefits
aaime: 1) shared understanding (the spec is one and it's there, no need to write one)
aaime: 2) easy to code wcs on top of it
aaime: 3) wcs extraction is powerful
jgarnett: we can take the same general apporach; a CoverageAccess class that matches what a WCS service can describe; and a class
for accessing a specific coverage; with optional interfaces to reveal additional capabilities (such as overlays)
aaime: 4) we could have complex stuff like PostGrid sit behind it
aaime: (at least, I hope so)
jgarnett: yeah you guys are all on the same page.
simboss: andrea knows more or les what I have in mind
simboss: and I share his ideas
simboss: my intent is to say two things
simboss: 1>do we start pushing that a coverage is a feature subclass
simboss: 2>I would not mind having something similar to gdal/ogr dataset
simboss: I.e. a single entry point for data access
simboss: t1 and 2 can be orthogonal though
simboss: with respect to the dataaccess proposal and what aaime said
aaime: simboss, I think to have some chance of getting work done
aaime: we should try to keep the scope of a "coverageStore" as small as possible
aaime: and be sure to cover the needs of PostGRID, Oracle and SDE rasters
aaime: (and a sample directory based one too)
simboss: nothing else?
simboss: 
aaime: Well, if you wanto to design a interface
aaime: you need usually at least 3 different examples no? 
simboss: I think we should try and have data access as much unified as possible
aaime: hmmm... and how do you filter over coverages? using ogc filter?
aaime: (band extraction, resampling...)
simboss: that's why I am asking
simboss: the data access proposal
simboss: is too feature centric
simboss: as most part of the geotools proposal
simboss: proposals
jgarnett: thinking ...
aaime: I'd keep the two separate and have a single class implementing both if it has both capabilities
simboss: that's not dataaccess
simboss: that' feature access
aaime: but yeah, maybe some top level methods could be shared
jgarnett: I think it is a bit premature to push for coverate extends feature (but I would like to see us try).
jgarnett: something similar to data set is a good idea; I settled for a common ResourceInfo - as it is what I need today.
simboss: (I do not only think it is premature, it is something tht I do not actually like at all )
jgarnett: good thought about them being orthogonal.
simboss: long story short
jgarnett: no worries.
simboss: I don't care (actually I don't like) coverage as a feature
jgarnett: I think it has only ever been stressed since RobA had visions of using WFS query to screw around with coverages.
simboss: but I would like to have 1 single entry point for data access
simboss: that was just my humble opinion
jgarnett: well tell you what; take a hard look at gabriels data acess super class; and see if a CoverageAccess subclass based on
WCS makes sense.
simboss: I am not saying it is the best one
simboss: as a baseline datacess
jgarnett: (at the very least I don't want to hold up gabriel waiting for coverage if we don't have time to follow through)
simboss: should just tell you how to access your data
simboss: my goal is just to raise the issue
mgrant n=mgrant@norbu.plus.com entered the room.
jgarnett: gak this is too facinating a conversation not to do well. I have considered this issue twice before:
jgarnett: a) for catalog Sevice / GeoResource split (it went really well)
jgarnett: b) for DataAccess (it dies due to lack of interest)
jgarnett: gabriel what do you think?
jgarnett: andrea I like your idea;
jgarnett: there is a bit of a conflict with list of contents.
jgarnett: but you are on to something.
jgarnett: let me rant for a second.
jgarnett: if we had something like getNames(); that listed a bunch of Name
jgarnett: and a getInfo( Name ) that described the contents
jgarnett: we could have subclasses cough up additional methods
jgarnett: like
jgarnett: getFeatureSource( Name )
jgarnett: getFeatureStore( Name )
jgarnett: getCoverageSource( Name )
jgarnett: the FeatureSource / FeatureStore / CoverageSource interfaces would match up with their spec, WFS, WFS-T, WCS.
jgarnett: end rant.
jgarnett: aaime is that what you had in mind?
Eclesia: jgarnett, Idea : could there be a method in the ServiceInfo : public Map<String,Object> getConnexionParameters() ? this
way I could start to save MapContexts in files, and open them again later.
simboss: exact
simboss: that' mny idea
simboss: push down the getFeature and getSchema thing
simboss: and you have something you could use
simboss: for coverage as well
simboss: or at least
simboss: that could give you either a list of coverage names as well as a list of featuretypes
jgarnett: Eclesia; maybe. I would just as soon as see you store your own connection parameters (you need to keep datastores
around for reuse anyways). There is a class around in geotools to do this for you.
simboss: aaime, what do you think?
jgarnett: simboss I like it.
jgarnett: it is more agressive than what gabriel needs; but I like it.
simboss: well
simboss: it is more general
simboss: and it leaves room for imprvements
simboss: if we don't start thinking in a more cross-field manner
simboss: we will always end up
simboss: with having
simboss: people working on coverage which do not know anything about people working on feature
simboss: and viceversa
jgarnett: the hard part is giving enough information out of the getInfo( Name ) method that people can tell what method should
work; ie getCoverage, getFeatureSource, getFetureStore? Or perhaps that is handled via an instance of check on the DataAccess
object.
simboss: this is a small leap forward I think
jgarnett: agreed; yep.
Eclesia: There is a class around in geotools to do this for you. ? there is something to save a MapContext ?
jgarnett: nope there is something to hold onto your datastores.
simboss: I am fine with either approaches
jgarnett: I have code in udig that saves out an industry standard map context somewhere.
simboss: it is something like having getcapabilities for all services
simboss: and specific methods that distinguish them
jgarnett: okay simboss what can we do from here? we should chat with gabriel in his IRC breakout. Or maybe hack on that page
before hand ...
simboss: hack and chat
simboss: is he around now?
sfarber left the room.
acuster: call it gridcoverage unless you want to prevent vector and analytic coverages
jgarnett: groldan ping?
simboss: what do you mean acuster?
groldan: pong
jgarnett: acuster++
acuster: you are talking about access to 'image' like data right?
jgarnett: coverage is a tree data structure; similar to feature. that completely covers an area of the world.
simboss: nope
cbriancon left the room (quit: ).
simboss: whatever is a coverage
jgarnett: grid coverage is the same thing except all the shapes are rectangles.
acuster: 'coverage' in the gis world, is a very general idea
simboss: coverage like in iso 19123
acuster: which is, for example, what udig calls a 'layer'
simboss: yeah
acuster: so your 'access to image data' is not really coverage, but a subset
acuster: the modules in geotools are misnamed
groldan: catching up...
simboss: (hold on a sec, phone call.. )
acuster: jgarnett, coverage doesn't have to have any data structure
jgarnett: got it; silly question acuster; does WCS serve up coverage? or only grid coverage?
acuster: GravtiationalModelObject
acuster: good be a coverage
acuster: jgarnett, not sure about WCS
aaime: jgarnett, grid ones
aaime: georectified or not
aaime: but grid based afai
jgarnett: still; back to the general approach outlined by simboss; what do you think gabriel?
jgarnett: can we write that up before your breakout IRC; and should I post these logs since we are making progress.
groldan: ok
groldan: still catching up, but so far...
groldan: I guess what we need to filter coverages is not Filter
groldan: but rather DataAccess should focus in a Query interfaces
groldan: whichs a superclass of the current, featutre focused, query
jgarnett: still whatever we did need woudl be from the WCS spec; and would be part of a CoverageAccess interface.
groldan: yup, just that abstracting out Query gives you room to specialize as you want for coverages
jgarnett: oh I see your point gabriel; but Query does not show up in DataAccess; only in FeatureSource right ?
groldan: it shows up in Source
jgarnett: what does WCS do; what are the operations andrea?
aaime: fully different
groldan: and keep in mind I'm ignorant about coverages, so just ranting
simboss: ok back
jgarnett: aaime do we have a list of the opreations on WCS ?
aaime: rangeset extraction (covers band and subset of bands when the band is of vector type instead of, say, scalar type)
aaime: spatial subsetting: bbox, temporal
simboss: I don't think we need query or filter fo coverage data access
aaime: resampling and grid to world transformation
aaime: yep, they do not seem to fit
simboss: (or better I never thought about it )
jgarnett: robA did; it was why he wanted Coverage < Feature
aaime: for example, there is nothing like "return the pixel whose value is between x and y"
jgarnett: not sure why he wanted it; so lets ignore it for now.
groldan: ok, those are operations, not predicate based queries
simboss: yeah
groldan: but does a bbox query fit as a query?
aaime: that would be the only one that fits
aaime: but the bbox is n-dimensional
aaime: and there is a separate time dimension
groldan: and asking for just a subset of bands?
aaime: it's a lot more complicated
groldan: ok
jgarnett: catalog had a similar query; using an n dimensional bbox. to account for time.
aaime: jgarnett, in wcs time and space are separate
aaime: space can be n.dimensional
aaime: time sits on the side
aaime: they are not integrated
jgarnett: a specific Query object for this would be good then.
jgarnett: thinking; not sure how well they were intergrate for catalog; but bounds and time range were needed as part of the
query.
aaime: A CoverageQuery that does not have anything to do wtih feature one? yeah
jgarnett: agreed.
groldan: that's what I'm asking, can all that be encapsulated in a CoverageQuery or such?
jgarnett: different "Parameter Object" for a different service.
simboss: actually
simboss: overcaming the current parameter based read
simboss: is a goal for sure
simboss: it would be great having a single object
simboss: probably something like abean
simboss: to describe the actual query
aaime: yep, that would be quite an improvement
simboss: much like ImageRead param for ImageIO
groldan: cool, that sounds good to me, as feature Query and coverage Query don't even need to be in the same hierarchy
simboss: my idea was to build on that
groldan: we can parametrize them at the DataAccess level
simboss: would you write up your idea
simboss: so that I can give it a look
simboss: ?
aaime: guys, it's late for the old man
aaime: I'm gonna go bed
aaime: see you tomorrow
acuster: ciao
simboss: ciao aaime
aaime left the room.
acuster: groldan, not sure that's right. /me is getting confused by coverage vs. grid coverage
acuster: but feature Query and coverage Query are going to be similar when the day comes
simboss: grid coverage should be a coverage back by gridded data
groldan: acuster: ok, I'm on the hands of the ones that know coverage here
acuster: some day, indeed
groldan: I never took the time to understand it
simboss: much of the confusion IMHO comes from the fact that people are using terms mixed from various specs
acuster: but grid coverage today is nothing of the sort
jgarnett: guys silly suggestion
simboss: the okde one has moreor less only gridcoverage
jgarnett: would "raster" work?
simboss: old
simboss: ISO 19123
simboss: uses them all with different meaning
acuster: "grid" would work
groldan: coverage is confusing to me too, as the concept I have of it is the Arc/Info one
acuster: a coverage is a function with a domain and a range
jgarnett: cool; acuster has it; lets write this up as GridAccess ...
acuster: where all values in the domain are guaranteed to have a value in the range
groldan: understood
acuster: so arc/info was in the ball park
acuster: I'll have to think of how we query grids from a general standpoing
groldan: ok, though its not a grid
acuster: standpoint
acuster: coverages are not grids
acuster: grids can be coverages
groldan: thanks!
acuster: if they behave correctly
groldan: so are we actually sticking to grid here?
jgarnett: coverages (real coverages) are a seperate problem for another day; I am sure a DataAccess class can implement
CovreageAccess (when we make it up) and GridAccess which we define today ...
jgarnett: stick to Grid.
simboss: acuster have you seen examples of coverages backed by GML in action?
jgarnett: it is like "Raster" but shorter to type.
simboss: (curious ...)
acuster: simboss, we have the core of one coverage in geotools , the referencing3d model
acuster: a spherical harmonic model of gravitation
acuster: it's something that is in most gps units
acuster: and we have no real coverage infrastructure right now
simboss: I was curious about people experiences with coverage nto backed by matrices but by something else
acuster: but if you have a grid and a pre-defined interpolation method, all nicely bundled behind an interface
acuster: you start having a real coverage
simboss: your use of proper terminology pushed me to ask
simboss: people are trying to used fast way to encode coverage (or better gridcoverage) as GML
acuster: a shapefile can be though of as a coverage
acuster: which is why it works raw as a layer in uDig
simboss: since binary data make BPEL engines choke!
simboss: I have used shapefiles as coverage
simboss: (but it was not geotools work unfortunately )
simboss: (that was 
)
simboss: but I am curious if someone has started to work out coverage and GML encoding
acuster: sorry, I don't know
acuster: galdos are the folk to ask
simboss: exact
acuster: they just announced their gmlsdk
acuster: a C++ lib to chew gml
simboss: I have seen some results in one of the project s I am working
simboss: and they showed some results about encoding coverage in GML
simboss: which sounded a bit optimistic to me
acuster: we have a long way to go to handle gml
groldan: encoding coverage in gml means a lot of numerical values?
simboss: yeah
acuster: well, at the 'shapefile' like level you could do a gml coverage
simboss: they claim to be quite efficient
simboss: actually
groldan: cool, you should talk to RobA for some insights
acuster: why numeric?
acuster: ah, you are talking about grids again?
simboss: I can tell you that they stram GML files really big as a charm
groldan: and wait till I finish the BXML encoder for eficient gml
groldan:
***acuster missed that
simboss: groldan, tell me about it
simboss: where does it sit?
groldan: I'm on a project implementing the bxml spec
groldan: its a standalone library, and I'm encoding GML3 where posList for example are saved as plain double arrays
simboss: for geoserver?
***acuster curses terminology and all those who don't use exactly the same terms in exactly the right places i.e. everyone else
who doesn't share all the (wrong) ideas as his brain cells
groldan: the sponsors are the gvSig guys, though the lib has no dependency other then the jdk
simboss: that was my second guess
groldan: it's meant to work for/in geoserver in the long term, yes
simboss: that's cool
acuster: groldan, it's valid gml3?
groldan: actually I'm meant to provide a geoserver build serving bxml encoded gml
acuster: that is cool
simboss: yea it is!
groldan: its valid, since posList is of type list<double>
acuster: maybe that's what the gvSIG were trying to show me in Italy
simboss: do you know if there is the possibility to get binaryxml decoded at javascript level?
groldan: may be, as by that time the work were too in an early stage
***acuster needs to look at their work someday
acuster: or at gabriel's
groldan: I don't think at the javascript level
jgarnett: guys I need to go take a break; I will leave this on and post the good bits as "Grid IRC breakout 1"
groldan: at least the browsers makes it transparent
simboss: you mean like gzip?
***acuster needs to sleep
simboss: I did not think that would possible yet
acuster: good night all
groldan: that is, the encoding is such a detail, in the same way you can get the stream gzip encoded. If the browser supports
bxml that's it
simboss: that for sure
murray42 left the room (quit: Client Quit).
simboss: but is it supported right now?
simboss: I am not aware of any browser supporting that
groldan: although for browser support its more possible they'll support a real standard encoding some day, like the one the w3c
is working on
***Eclesia annotations +1, xml -1 
Eclesia: +++ good nigh
Eclesia left the room.
groldan: right now not afaik, like in the ogc bxml spec is just a best practices document
groldan: that cubewerx defined and use
groldan: and I guess Galdos is doing its own thing
simboss: yeah
simboss: I read that
simboss: I am curios since we mov a lot of features for measurements around
simboss: and hacing BXML would be a plus
acuster left the room (quit: "ciao all").
groldan: yup
groldan: as long as the parser leverages the binary encoding (that is, no need to translate to string and back to the primitive
types) parsing times are great
ggesquiere left the room (quit: "Bye").
vheurteaux left the room (quit: ).
groldan: though space consumption is not generally as much as gzip compressed text, it really depends on the data structures
simboss: what about
simboss: gzip(binaryxml))
groldan: the most xml tags you have, the most space you save. Same thing for number of ordinates
simboss: I have heard that=
groldan: yeah the spec allows to gzip the contents
groldan: though the impact in both parsing and encoding is high
groldan: it may be worth when bandwidth is the main concern
groldan: yet, for an OWS, I'd let the web server handle the gzip encoding
groldan: ie apache does a much better job than java
groldan: I looked around for a well performing java zlib alternative but found nothing
groldan: and GZipInput/OutputStream suck
groldan: that not to say I'm using java.nio instead of java.io
simboss: yeah
groldan: and, although still need to implement namespace support, which will impose some processing overhead, with java.nio I'm
getting better performance than the CubeWerx C implementation
simboss: cool =
groldan: jgarnett question:
jgarnett: sure
groldan: I'll be doing some datastore implementation based on a gpl library
CIA-22: robatkinson 2.4.x * r29068 geotools/modules/unsupported/community-schemas/community-schema-ds/src/test/ (34 files in 9
dirs): Updated to match final published OGC versions of schemas
groldan: guess that can't sit on the geotools svn?
jgarnett: which gpl library ?
groldan: a gvsig one
jgarnett: can it be used as a dependency?
jgarnett: gvsig people I think will sue your ass
groldan: it'll be a dependency, yes
jgarnett: but sure try it and see
groldan: no, actually they're asking me to do that
jgarnett: hrm
groldan: but I wasn't sure if possible at all
jgarnett: well it may be an interesting one; the construction of a GPL library is an odd beaste.
groldan: agreed
groldan: though I'm hands tied for the time being
jgarnett: ie what does it mean to use it? classpath exception is nice and clear for java.
groldan: repeat: agreed
jgarnett: FSF would like to see gpl library only used in gpl programs; but that is hard for dynamic libraries (and jars)
jgarnett: one easy thing to do
groldan: yup
jgarnett: is punt the library into a maven repository; be sure to include the licene info
jgarnett: and make a community plug-in that has a gpl license.
jgarnett: bit it is a slippery slope.
jgarnett: maybe make the plugin lgpl; and force it to be gpl if they ask.
jgarnett: as long as the plug-in is not providing any api; the geotools core library does not care.
jgarnett: that is the approach I used for uDig
jgarnett: when I want to create a GPL plugin.
groldan: I don't know if a lgpl plugin can use a gpl lib?
groldan: understood
jgarnett: well even so; a gpl plug-in can use a gpl library.
jgarnett: but yeah you are on a road to hell; enjoy!
jgarnett: what gvsig library?
groldan: the question is, even if the plugin is lgpl, the lib could only be used on gpl clients (ie geoserver)
jgarnett: their port of the xml binding stuff?
jgarnett: not sure about that gabriel
groldan: not a port, but yeah, that one
groldan: k, thanks for the feedback
jgarnett: if it is a problem you can always host the plug-in in the geoserver code base.
groldan: aha, that makes sense too
jgarnett: so gabriel is your "break out IRC" done
jgarnett: there are a lot of ideas there to write up.
groldan: indeed, though not sure if that'll avoid the break out
groldan: though surely will provide a lot more feed
groldan: we still miss Justin
groldan: and I don't know what mad plan he has in mind
jgarnett: well I can tell you that part
groldan: but so far mine is to make ResourceCollection NOT extend java.util.Collection
jgarnett: he wants me to admit that FeatureReader exists
jgarnett: and have FeatureSource.getFeatureReader( Query )
groldan: and make geotools FeatureCollection extend GeoAPI FeatureCollection
jgarnett: yeah we are all going down the same road here
ahughes n=andrew@202.189.75.62 entered the room.
jgarnett: he finally made a FeatureCollection that delegates EVERYTHING to feature source.
jgarnett: and the result can be resued everywhere.
groldan: yet, the very existence of FeatureCollection as the result of a query seems like an unneeded indirection
jgarnett: so hopefully at the end of the day all the ResourceCollection stuff will die.
groldan: since it ends up being a cursor api anyways
groldan: you still have to ask for an iterator
jgarnett: with one difference!
jgarnett: you can reuse it; ask for the iterator twice.
groldan: yup
zzorn left the room (quit: "Leaving").
groldan: already though about that
jgarnett: that is valuable; you can pass that thing around
jgarnett: still two goals:
jgarnett: 1. less to implement (ie just feature store)
jgarnett: 2. not make feature collection impossible (one implementation that delegates to feature source accomplishes this)
jgarnett: that is his plan
jgarnett: he would love to kill FeatureCollection;
groldan: hmmm my last two featurecollection implementations rely completely on featurereader...
jgarnett: I want to make sure that something like feature collection is posisble
groldan: is that what you mean justin is doing?
jgarnett: since that is how you will need to do assocaitions (for joins)
jgarnett: ie a lazy collection.
jgarnett: I think he was depending on FeatureSource
jgarnett: if you just dependend on FeaureReader you would need to cache the results of the first read
jgarnett: like a LazySet
jgarnett: you need to talk to justin more; sounds like you guys are duplicating work.
groldan: yeah, that
a
jgarnett: I think his stuff is in H2
groldan: it wouldn't be the first time, sadly enough
jgarnett: aside: Eclisia did give us a thumbs up on getInfo stuff
jgarnett: so I have jesse's, yours and eclisias review. time to bring over postgis getInfo etc...
groldan: I've a possible modification over ResourceInfo
jgarnett: what is it?
groldan: getName():Name instead of getName():String
jgarnett: before you go much further; look at the other information there.
groldan: there's duplication with Source.getName() afaik
jgarnett: is there not a get Namespace or something?
groldan: I guess not, at least looked for it
jgarnett: I am torn
jgarnett: I would like this one to be simple.
jgarnett: so that serviceURL#name
jgarnett: works as a URI
jgarnett: really feature type, service and resource need seperate identifiers
groldan: but its not that easy
groldan: uri#topp:states works as id
groldan: though still need the Name to hold the qualified name
groldan: so you're not doing dark tricks with the names
jgarnett: question
jgarnett: does not topp
jgarnett: stand for the namespace?
groldan: nope
groldan: just for the prefix
groldan: though
jgarnett: right the prefix; indicating the namespace uri
jgarnett: and that is available already in getInfo()
groldan: doing real namespace work, you can change the prefix
jgarnett: so you have everything you need.
groldan: where's a ns by prefix lookup in getInfo()?
jgarnett: ie name and namespace uri are already available.
jgarnett: there is none; prefix is something that only lasts for the duration of an xml document
jgarnett: it is not part of the formal model.
jgarnett: you can have a global one for geoserver if you want.
jgarnett: (honest answer; may not be useful)
groldan: no I don't want, I want to play fair
groldan: ie, a WFS request changes the prefix, its resolved to the correct namespace from the same WFS query, and my featuresouce
can be asked
jgarnett: still
jgarnett: what?
jgarnett: sorry reading again.
groldan: agreed prefix is out of the equation
jgarnett: yeah I get it; the wfs request can specify any prefix; you need to resolve it to a namespace. got it.
jgarnett: sorry gabriel I don't have the info code in front of me.
jgarnett: but I did try to make the namespace and name available somewhere in there.
groldan: np, I need to go bed anyway
groldan: thanks for the chat, gonna update the proposal tomorrow
groldan: and call Justin to chime in
Labels