Skip to end of metadata
Go to start of metadata

59 <jgarnett> I have a question (while I wait for my checkout) - what do people think of that change of policy I mentioned in email? 1) devel on a branch 2) bug fix on trunk 3) when you merge in new capabilities you make a milestone release 4) PMC will release a dot release on a regular schedule
00 <jgarnett> Hi martin - we are trying to restore the build (smile)
00 <jgarnett> Glad you could make it ...
00 <bnordgren> Simone, Martin, check out
00 <Martin_> Hello all
00 <bnordgren> (Jody) My stupid email has about a five hour delay. Ain't seen it yet.
00 <jgarnett> oh.
00 -->| andrewW ( has joined #geotools
00 <jgarnett> For all I know it has a poor subject line - it basically came up when geoserver was trying to figure out what was going on ...
01 <normanbarker> hi Andrew
01 <jgarnett> (There has been so much email it was probably missed)
01 <andrewW> Hi Norman, i made it (wink)
01 <simone> hello martin...
01 -->| Jesse_Eichar ( has joined #geotools
02 <bnordgren> hello martin
02 <simone> hello everybody
02 -->| appetkov ( has joined #geotools
02 <jgarnett> Just removing all traces of Java 5 from my windows box so I can be sure of a clean build...
02 <bnordgren> Excellent, Alex is here too. I'm going to drop out and walk over to his terminal.
02 |<-- bnordgren has left ("Disconnecting")
03 <jgarnett> opps we lost him
03 <appetkov> he's here
03 <appetkov> and I 'm here too(smile)
03 <jgarnett> just a sec have to restart my IRC app
03 |<-- jgarnett has left ("Chatzilla 0.9.68a Firefox 1.0.4/20050511")
04 <simone> does anybody tried the link bryce provided?
04 <appetkov> i did...
04 <simone> does it work for you?
04 <appetkov>
04 <cresques> It doesn't work for me (sad)
04 <simone> the same here
04 <rgould> I will perform a linux build on 1.4
05 <simone> this one works
05 <appetkov> cool
05 <appetkov> he suspected that he tytped it wrong...
06 -->| jgarnett ( has joined #geotools
06 <normanbarker> what is the 3d processing?
06 <jgarnett> Hi again -
06 <jgarnett> what version of maven are people running?
06 <Martin_> 1.0.1
06 <appetkov> Projection of pictures onto the landscape.
06 <appetkov> Maven 1.0
06 <jgarnett>
07 <jgarnett> lists 1.0.2 ...
07 <jgarnett>
07 <jgarnett> also lists that as the most current ...
07 <cresques> Projection of pictures onto the landscape == image sading by means of a DEM ?
08 <appetkov> Does 1.0 not work? Worked for me yesterday.
08 <jgarnett> It is not really a question of work/not work - I am trying to figure out what is known to work. Basically I want to ensure that
08 <appetkov> Negative. Georeferencing of an unregistered high-oblique image (contains the horizon).
08 <jgarnett> someone can follow the focs..
09 <jgarnett> typo: focs = docs
09 <cresques> sounds great.
10 <jgarnett> Q: I am I in the middle of the great Martin / IRC meeting? Or is everyone trying to get the build going ...
10 <appetkov> I upgrade when things stop working. (smile) I followed the docs when I downloaded maven originally. (smile)
11 <appetkov> I'm here for IRC mtg.
11 <jgarnett> got it. I assume everyone is here to talk to Martin - cause he is wonderful (smile)
11 <appetkov> We think he's great.
11 <Martin_> I'm reading the Bryce's page right now (it sound like pretty good Bryce!)
11 <jgarnett> Cool. I may ask questions about the build ... but I will try and keep it down to a dull roar.
11 <jgarnett> Martin were you able to read the IRC about OSSIM getting involved?
12 <jgarnett> They sounded quite confortable with JAI, and interested in providing an implementation behind a GridCoverage.
12 <appetkov> It needs more work. (smile) Feel free to edit to put your own requirements in. (Just not everyone at once. Might break it.)
12 <Martin_> I have read the IRC, but didn't saw OSSIM. Maybe I read it too fast and need to read it again.
13 <Martin_> What is OSSIM?
13 <andrewW> Have just seen Bryce's page for first time...
14 <andrewW> Our reqts closely echo WRF, but need in general 4-d subsets from 4-d: ...BBOX=-180,-90,179,90,0,1000&CRS=some3dCRS&TIME=1996/2005/1...
14 <appetkov> There is certainly an inherent need to be able to select a subset out of an nD box.
15 <simone> Ossim is basically a very powerful raster library
15 <andrewW> WCS supports up to 5-d (if you include PARAMETER=...)
15 <simone> with the ability to read various formats
15 <simone> and to do 3d
15 <appetkov> Let's start off organized like. Simone & Alessio have just done a bunch of work along these lines (including grib support). Can you two brief us in?
16 <simone> ok
16 <simone> alessio is not around
16 <jgarnett> (For Martin: )
16 <simone> let me just call him on the phone
16 <simone> otherwise I will brief everything myself
17 <simone> just a sec...
17 <jgarnett> While we wait - martin OSSIM is a C++ library that has lots of raster support. They have some generated Java bindings, and would like to start playing with the geotools crowd.
17 <--| Jesse_Eichar has left #geotools
17 <normanbarker> are the java bindings for ossim available?
17 <jgarnett> This represents a significant outreach effort so I would like to make them feel welcome when they show up (in about 3 weeks due to a release).
18 <jgarnett> Apparently they "are", but they lost them during a website move. They promiss to have them again as part of their release.
18 <normanbarker> also, I thought ossim used gdal as a driver, and that is restricted to 2D and bands
18 <jgarnett> They did sound like they knew of JAI, and were quite happy to hear about GridCoverage.
18 -->| Jesse_Eichar ( has joined #geotools
18 <Martin_> If we are getting more help for external contributors, it give me the feeling that switching to ISO 19123 interfaces is becoming more and more urgent...
19 <jgarnett> It very well might. My impression is there strength is setting up chains of processing.
19 <jgarnett> If the end of a chain was a GridCoverage it would work nicely with geotools...
19 <simone> alessio is not answering
19 <appetkov> Amen. One of the things we want to do is minimize the amount of re-writing we have to do when we adopt a new gridcoverage scheme.
19 <jgarnett> I am sorry I don't know more Martin.
20 <simone> we can proceed as you'd like.
20 <Martin_> Thanks for the information Jody.
20 <--| Jesse_Eichar has left #geotools
20 <appetkov> Go ahead, Simone
21 <simone> Ok
21 <simone> small brief
21 -->| Jesse_Eichar ( has joined #geotools
21 <simone> I have been playing with geotools since oct 2004
22 <simone> and geoserver as well
22 <simone> because we had been asked to build a prototype for a complete server
22 <simone> wcs+wms+wfs+(possibly)catalog
22 <simone> we evalutaed various products
23 <simone> but we ended up with GeoTools+Geoserver
23 <simone> Alessio worked on the Server part
23 <simone> he studied GeoServer for a couple of weeks in january
23 <simone> in order to leverage the preexisting structure
23 <simone> to build a skeleton for a WCS
24 <simone> initially the goal was to stay on 2D
24 <simone> and support
24 <simone> georeferenced images
24 <simone> geotiff
24 <simone> grib files
24 <simone> usgs dem
24 <simone> but also Ascii grids came for free
24 <simone> since it was just a matter of porting the legacy code
25 <simone> The skeleton alessio built
25 <simone> integrates perfectly with Geoserver
25 <simone> the preexisting one
25 <simone> that was one of the goal
26 <simone> since we had only three months to have a decent prototype
26 <simone> Since I had more experience with geotools
26 <simone> I worked on the GridCoverage support from GeoTools
26 <simone> I had no experience with JAi at the beginning
26 <simone> and the GridCoverages were under port
26 <simone> therefore
27 <simone> as I said before
27 <simone> the support we set up
27 <simone> was an initial prototype
27 <simone> but it worked pretty fine
27 <simone> I jhave been playing with it for a month in a NATO exercise during may
27 <simone> and we had no problems
27 <simone> the point now
27 <simone> is
28 <simone> ok
28 <simone> we know that we need more
28 <simone> we know that we can do more
28 <simone> we learnt how we could do more
28 <simone> let's try to do more !
28 <simone> Just a note
28 <simone> WMS raster support
28 <simone> that is something
28 <simone> that was not planned to be done
29 <simone> but thanks to a bright alessio intuition
29 <simone> we quickly hacked it
29 <simone> and it worked fine
29 <simone> for source that were not too big
29 <simone> (in size)
29 <simone> the point now is to enhance that support
30 <simone> using tiling, pyramids, and raster symbolizer
30 <simone> As far as we are concerned with WCS
30 <simone> the goal is to go to more than 2 dimension
31 <simone> ( I have been thinking about for months and I have some ideas about it...)
31 <simone> but support also
31 <simone> 3 and 4
31 <simone> plus the band selection
31 <simone> which basically means
31 <Martin_> (note: band selection should be on trunk now)
31 <simone> 5th dimension = parameter
32 <simone> (yeah... we saw that... thanks Martin!)
32 <simone> I think that as brief it was long enough!
32 <cresques> band it's a parameter on WCS, isn't it?
32 <simone> yeah
32 <simone> in the GridCoverage spec
32 <simone> dimensions are 4
33 <simone> x,y,z,t
33 <simone> but you also have bands
33 <simone> which can be pretty anything
33 <cresques> and parameters has no limit on number, in specs, I think
33 <simone> like for example in multipsectral images
33 <appetkov> Simone: tomorrow morning, can you repeat that summary on that web page?
34 <andrewW> Simone,I think there will be a big userbase for x,y,z,t+ parameter
34 <simone> I can make a better one
34 <appetkov> Yes, I think there's a big user base too.
34 <simone> there is a sensor Hyperion we are thinking about using which can retrievew more 100 bands....
34 <appetkov> Simone, where is the stuff you've already written (grib, etc.) ?
35 <simone> grib right now
35 <cresques> (sorry to disturb) did you take a look to ?
35 <simone> is on my pc!
35 <normanbarker> is this going to link into GALEON
35 <appetkov> can we collect your work into a branch, so we can all look at it?
36 <appetkov> That will help us try to move forward, I think...
36 <simone> well
36 <simone> we can talk about it
36 <simone> after that exercise we had on june
36 <simone> we started thinking abut how to improve everything
36 <simone> therefore we are working on it daily
37 <simone> but not committing anything
37 <simone> since it is unstable
37 <simone> we are playing with various things
37 <simone> pyramids
37 <simone> tiling
37 <simone> JAI integration
37 <simone> I think after the meeting
37 <andrewW> Do you stack GRIB records vertically for 3-spatial-d?
37 <simone> we can start talking about how to reuse what we did
37 <appetkov> Branches can be unstable. I think the main value is to start other people thinking along the same lines as you.. (smile)
38 <simone> I started working on that
38 <simone> using martin's classes
38 <simone> CoverageStack
38 <simone> in order to handle
38 <simone> 5d grib files
39 <simone> so, what about the agenda for this meeting?
39 <jgarnett> That is a fun demo - thanks (smile)
39 <appetkov> Well, we covered your intro. My intro was on the web page. Who else is here?
39 <normanbarker> isn't the demo MapServer?
40 <simone> it is
40 <simone> martin, bryce...
40 <appetkov> Martin?
40 <Martin_> Not in any particular order, we may talk a bit about plan for ISO 19123, and plan for IIOMetadataFormat (if we decide that it is a path to follow)
41 <simone> We could start with the issue ISO 19123
41 <Martin_> The good new is that I believe that it will not break a lot of compatibility.
41 <appetkov> ding ding ding. I'd add discussing a strategy for specifying multi-d datasets.
42 <Martin_> (sure)
42 <appetkov> ISO 19123
42 <Martin_> The only problem right now is to find the time to finish interface creation in GeoAPI.
42 <Martin_> There is at least 2 (or 3) steps to follow:
42 <Martin_> - Commit all interfaces in the pending directory
42 <Martin_> (looking for URL...)
43 <Martin_> (sourceforge CVS is down right now...)
43 <Martin_> I would said that about 30-40% of ISO interfaces are done right now.
44 <Martin_> Any help for finishing them is welcome, as usual (smile)
44 <simone> I agree
44 <Martin_> Once all interfaces are done, the next step is to merge them with Stephane Fellah's proposal.
44 <appetkov> Does 19123 specify a method for nD datasets or is that external to the spec?
45 <appetkov> for selecting slices out of nD datasets, I meant
45 <cresques> is 19123 available for reading?
45 <simone> Yeah
45 <simone> iso 19123
45 <Martin_> If "D" is spatio-temporal or spectral dimension, I believe that ISO is general enough. For selecting slices, I'm not sure. But ISO 19123 interfaces are generally more powerfull than OGC's one.
46 <simone> does take into account spatio temporal coverages
46 <simone> agree
46 <andrewW> 19123 unfortuinately has fundamental limittion to regularly spaced grids (but agree its a good sart)
47 <jgarnett> As long as the "standard" can be represented as a subset of your agreeded interface you should be good.
47 <Martin_> About availability for reading. ISO 19123 is a very particular case. This ISO specification is not yet published in the list of OGC abstract specification (like ISO 19111), but everyone at OGC use.
47 <andrewW> Martin: STephane Fellah's proposal?
48 <jgarnett> Martin you and I have "access" to that spec. Can we make interfaces that are good enough for others to code against?
48 <cresques> andrewW: you mean coverage suport shoud include TINs ?
48 <Martin_> Furthermore, ISO 19123 is at least partially in some derived form (XML) there:
48 <andrewW> No, just not a grid defined by offset vectors.
49 <andrewW> eg lat-lon, but increased resolution over (e.g.) UK
50 <Martin_> I believe that wWe can make interface from ISO 19123, since we are allowed to uses and the later is derived from ISO 19123 (and other ISO specificatiions).
51 <Martin_> Continuing: Stephane Fellah was the editor of the legacy OGC GridCoverage specification. He published in GeoAPI pending directory his own set of interfaces derived from ISO 19123, and modified for better WCS supports.
51 <andrewW>
51 <Martin_> Thanks Andrew! I was not aware of this link.
52 <cresques> thanx too!
53 <andrewW> It's Rob Atkinson et al at SEEGrid - has he been involve in this coverage/wcs/geoserver discussion?
53 <Martin_> Stephane Fellah's interfaces can'n be adopted as is, because there is some work needed (javadoc, annotations, etc.). However, once we have finished a "clean" set of interfaces derived from ISO 19123 as directly as we can, we should take in account Stephane Fellah changes. He have a lot of experience with GridCoverage (he worked at PCI geomatic).
54 <appetkov> What about persistence? Does it replace GCE too or is that an open issue?
55 <Martin_> I didn't saw anything about GCE in ISO 19123 (but I have not finished reading it).
55 <Martin_> And GCE in OGC GridCoverage is insuffisient.
55 <simone> I read it all, even if quickly
55 <simone> and there was nothing about gce
56 <simone> that is my main concern about ISO 19123
56 <appetkov> So we may have another issue to tackle in addition to the internal represenatation of grids...
56 <Martin_> My feeling is that on the particular case of GCE, we should forget about GeoAPI interfaces for now, do our own in Geotools only, and maybe put the interfaces back in GeoAPI when we feel they are good enough.
57 <simone> I agree
57 <cresques> sorry for my ignorance ... GCE is what?
57 <simone> gce is somehow poor for 2d (since GridCoverages from OGC are essentially 2D)
57 <Martin_> GridCoverageExchange: the part that read and write images.
58 <simone> but is quite useless for ND (N>2)
58 <simone> to work with 3d, 4d...
58 <simone> we need something else, I think....
58 <appetkov> I think the multidimensional problem spans several areas: internal representation, persistence, etc.
59 <appetkov> We're going to have to bring the internal representation and the persistance mechanisms online at the same time, I fear.
00 <Martin_> Once we feel that ISO 19123 interfaces are good enough, we can implement them as an experiment in Geotools. If it work, we can proposes them to OGC adoptions. They will need to be voted at some OGC meeting (there is 4 meeting by years).
00 <normanbarker> I quite the like the idea of ingesting into PostGIS, e.g.
00 <cresques> by persistence on this topic, what do you means exactly?
00 <simone> reading writing multidimensional coverages
01 <cresques> tx
01 <appetkov> Storage of a grid into a file (or server or something.)
01 <appetkov> The interface should be format independent.
01 <appetkov> sorry
01 <andrewW> 19123 is OK for n-D in my reading
01 <simone> actually I did some experiments with using Postgis for persistence
02 <cresques> simone: tiling your data?
02 <simone> what do you mean?
02 <simone> let me rephrase
03 <simone> how tiling comes into account now?
03 <cresques> caus of my experience, sorry
04 <Martin_> I don't know for postgis. For 2D images in files, it should be handled by JAI.
04 <normanbarker> for postgis, I was referring to N-D
05 <cresques> at gvSIG project we've bing working for a while with raster support
06 <cresques> we read with JAI, jmrsid, jecw and jgdal. I've foud JAI far to slow to be used for big amount of data
06 <Martin_> On GridCoverageExchange topic. Do we agree about using GeoAPI interfaces as a target of opportunity, but to not try to hard if it is inconvenient? Until we come with better proposal.
07 <Martin_> Which JAI instruction do you use for reading image?
07 <Martin_> And how much memory do you allocate to yours tile cache?
07 <cresques> wait, I'll check
07 -->| andrewW_ ( has joined #geotools
08 <appetkov> Running notes at:
08 <appetkov> Squawk on error.
08 <cresques> basically we open straight tiffs with no tiling/overlaing, that about 100 Mb each
09 <cresques> JAI was really slow, and loaded full image to memory
09 <simone> quick not
09 <simone> to use tile cache
09 <simone> you need to tile image on disk
09 <simone> otherwise
09 <cresques> after jgdal writing, same image was really usable
09 <simone> you will load it all as a whole single tile
10 <simone> i have been playing with 400 mb tiff from quickbird
10 <simone> tiling them on fisk
10 <simone> adding overviews
10 <simone> and then displaying them
10 <cresques> but was not our target, to change image formats. We wanted better speed without touching data. It was a client side approache
10 <simone> the result was astonishing
10 <Martin_> Additionnaly, the default JAI's tile cache is only about 16 Mb if my memory serve me right. You may need to increases it (JAI.getDefaultInstance().getTileCache().setMemory(...) or something like that).
10 <cresques> I work with this size QB images, without tiling
11 <simone> (yeah, 16 mb )
11 <cresques> initial load takes about 20 seconds (400x200 px). zoom in are quicker
11 <simone> (and enable tile recycling...=
12 <andrewW_> To be competitive with legacy DODS, WCS should handle up to O(100MB) for data exchange
13 <simone> I know...
13 <simone> I am working on that relly hard....
13 <appetkov> So, JAI gurus, is there any way to load an untiled tiff (like cresques') faster, without first loading it in and re-writing it as a tiled image?
13 <simone> the only way to avid loading it all in memory
13 <simone> is to use decimation on reading
13 <Martin_> Can we talk about JAI tips on mailing list and come back to a plan for GridCoverage evolution?
13 <simone> sure
13 <simone> sorry
14 -->| pramsey ( has joined #geotools
14 <cresques> I think main problem on Jai it's tiff reader strategy
14 <Martin_> (They are very valuable tips Simone; it would be nice to put them in a wiki page)
14 <simone> when I will have time
14 <simone> I will do that
14 <Martin_> ISO 19123: Do we have any volunter for helping to finish interfaces?
15 <cresques> as it loads on memory before anything, wen you may starts to work you've 100 Mb less of RAM, and 10 seconds later
15 <appetkov> My time is going to free up in the fall, unfortunately, we're cranking on a project right now.
16 <cresques> as gdal reads only what you specify (it's remote sensing oriented) is far more efficient
16 <cresques> sorry
16 <Martin_> Okay. So I think we may have ISO 19123 completed around September (sooner if we find volunter).
16 <appetkov> Simone, what is your availability in the short and long term?
17 <simone> well
17 =-= appetkov is now known as bnordgren
17 <Martin_> (correction: just realizing that we are almost in August: I will target october for ISO 19123 interfaces).
17 <Martin_> Second topic: GridCoverageExchange. Who is working on it?
18 <simone> my time is pretty bounded from what they ask me to do at the center
18 <Martin_> I did a little bit of work, Simone did some. Bryce did some I believe?
18 <simone> but I think I could give some of time on this
19 <Martin_> Nice! Please just send me an email when you feel ready to do some work on ISO 19123 interfaces.
19 <jgarnett> I worked on it - I think that it is a good high level service (but we should throw it away and have something easier/more direct)
20 <bnordgren> I implemented a reader, but I'm not working GCE per say.
20 <bnordgren> Jody says not to keep it alive on his account.
20 <bnordgren> Oh. Hi jody.
20 <bnordgren> That's the problem with looking at the keys when typing.
21 <jgarnett> Now the GCE implementations that are there:
21 <bnordgren> So: What are the parameters of a New, Improved GCE?
21 <jgarnett> 1) WMS
21 <jgarnett> 2) File
21 <jgarnett> The File one processes a SDI meta-inf thing to find the available formats.
22 <jgarnett> The WMS was really a hard-core expercise in understanding parameter (sad)
22 <jgarnett> Well it is missing something right now.
22 <jgarnett> "Discovery"
22 <jgarnett> there is no way to tell what is "in" a GCE.
23 |<-- andrewW has left (Read error: 113 (No route to host))
23 <jgarnett> So I would see if you can just do something similar to DataStoreFactory / DataStore ... we got some interfaces in uDig that you can have if you want.
23 <jgarnett> But all of this is 2nd place to agreeing on a GridCoverage interface (which I think you guys already covered)
24 <Jesse_Eichar> There is a stream implementation too.
24 <Jesse_Eichar> reads and writes to and from arbitrary streams.
24 <Martin_> The final GridCoverage interfaces is pending ISO 19123 interfaces, but I believe that it will be compatible with current interfaces for the most important methods.
25 <bnordgren> Ok, New GCE needs to retain ability to persist to: stream, WMS, and files.
25 <bnordgren> Needs nD functionality.
25 <bnordgren> Needs plugin framework.
26 <Martin_> I would like to process steps by steps: 1) Get a Steam-only GCE working for n-D coverages, 2) Add a WMS version, 3) Add discovery (this one may be pending a catalog service), if possible as a DataStore
26 <Martin_> Does this plan make sense?
26 <jgarnett> How about reading? If you don't have to read from a stream it becomes easier to do a lot of the tricks like mmaps.
26 <andrewW_> Agree - we need file read
26 <bnordgren> I like stepwise implementation. We just gotta keep the big picture in mind from the start. (smile)
27 <cresques> and to DB (is not a stream, or is it ?)
27 <jgarnett> That was the questions: streaming file read or tiled file read? Or both as needed ...
27 <Martin_> I was including File read in Stream read, since they are almost the same from javax.imageio.ImageReader user point of view.
27 <jgarnett> I like stepwise too
27 <andrewW_> ok
27 <simone> the same here
27 <jgarnett> good point cresques - not sure.
28 <cresques> I think tiled shoud be transparent
28 <Martin_> Lets focus on Stream/File first, and see DB after.
28 <Martin_> Tiling should be transparent, yes.
28 <normanbarker> I can see requesting on parameters (big picture) in the WCS to be more difficult without a DB
29 <Martin_> So lets do DB after File/Stream and before WCS, as step 1.5)
30 <Martin_> For a grid coverage reader implementation, I would like to process in the way suggested by Bryce (IIOMetadataFormat).
30 <Martin_> Rational:
30 <jgarnett> I would also point out that you can leave WMS out of it - WMS, WCS etc can be their own client as is happening with WMS and WFS now.
30 <Martin_> GridCoverage2D is basically a wrapper around java.awt.image.RenderedImage, adding supports for geographic metadata like CRS.
31 <jgarnett> The client implementation of WCS can simply use the streaming Reader/Writer ...
31 <Martin_> Thanks; so we can set WCS support as a "target of opportunity".
31 <bnordgren> Updated notes
32 <bnordgren> Should I remove WMS?
32 <jgarnett> I think you should use a plugin/WCS as a common platform to make sure that your implementation is capable.
32 <jgarnett> I would think so bnorgren.
32 <Martin_> continuing: like GridCoverage is a wrapper around java.awt.image.RenderedImage, GridCoverageExchange can be basically a wrapper around javax.imageio.ImageReader in the 2D case.
32 <jgarnett> You can just change the implementations to use your readers as a "sanity check" that we are on the right path.
33 <bnordgren> This IIOMetadata format thing is Yet Another Area where we need to be able to communicate nD concepts.
34 <Martin_> We may not need to write a GridCoverageFormat for each format. A more general approach may be to agree on a way to transfert metadata (CRS, envelope) from ImageReader to GridCoverageExchange, and IIOMetadata may be used for that.
34 <bnordgren> J2Se is 2D + bands. We need to be able to specify which slice to take out of a 4D hypercube and have our IIO plugin produce the correct image.
34 <Martin_> I believe that IIOMetadata is general enough. We basically can put whatever we want there.
35 <simone> about this
35 <simone> I am trying to do something like that with GribFiles
35 <bnordgren> Is IIOMetadata bidirectional? Or is it plugin -> client only?
35 <simone> I am adapting my grib library to work under ImageIO readers
36 <simone> as an experiment on how to leverage this approach....
36 <bnordgren> I like IIOMetadataFormat for reporting from the plugin to the app, and would suggest the OGC's 05-027r1 for that. (Image CRS using GML)
36 <Martin_> IIOMetadata would need to be kind of bidirectional ImageReader --> GridCoverageExchange --> ImageWriter.
36 <simone> it is that way
37 <simone> IIoimage
37 <simone> wrap
37 <simone> image+iiometadata
37 <simone> ImageWriter
37 <simone> uses IIomage
37 <bnordgren> But can we use it bidirectionally with the reader? (e.g., The app needs to specify the Time and Height to get the correct slice of data from the IIO plugin.)
37 <cresques> supports ImageWriter floats or doubles?
38 <cresques> (so ImageReader)
38 <simone> that is the way
38 <simone> I wrote the Geotiff writer
38 <simone> i build a IIometadata object from crs and envelope
38 <simone> than i wrap
39 <simone> renderedimage+iiometadata
39 <simone> and write does the trick
39 <Martin_> In order specify the Time and Height to get the correct slice of data from the IIO plugin: I think it is ImageReadParam's job rather than IIOMetadata's job.
39 <simone> imagereader should support float
40 <simone> and also double
40 <simone> i should check though
40 <Martin_> Simone: looks like a good aproach. What I would like to do is to put the IIOMetadata part in a common package for all plugins.
40 <simone> well
40 <Martin_> I mean, we probably need at least a common IIOMetadataFormat.
40 <simone> the approach should be
40 <simone> 1>read a file format or a stream
40 <simone> 2>check the metadata
40 <simone> then select the right image by index
40 <bnordgren> Are we looking at making a standard set of ImageReadParams which we would expect nD aware plugins to supports?
41 <simone> there are two kind of metadata
41 <simone> stream metadata
41 <Martin_> I suggest that a standard set of ImageReadParams should be planned for Geotools.
41 <bnordgren> Martin, Simone: I like the idea of a common definition of CRS IIOMetadataFormat.
41 <simone> one for each file or stream
42 <simone> and iiometadata
42 <bnordgren> Also a standard set of ImageReadParams.
42 <simone> one for each image (or slice as you like)
42 <simone> imagereadparams control, 2d subsetting, decimation, offsetting and tiling..
42 <bnordgren> Part of the common package could be the ability to construct a CRS from the IIOMetadata regardless of which plugin it came from.
43 <simone> for each slice-image
43 <Martin_> Bryce, agree.
43 <simone> me too
44 <bnordgren> typing in notes now....
44 <Martin_> Simone, can you extracts the part of yours IIOMetadata work that are general enough (not specific to Geotiff) and put this generic part in the package?
44 <Martin_> (do we agree about a package for such common parts?)
45 <bnordgren> notes updated
45 <simone> in JAI
45 <simone> there is GEOTiffMetadata
45 <bnordgren> Martin Sure, that sounds like a good package.
46 <jgarnett> (Aside: almost have the build back again)
46 <bnordgren> Simone: that's the native metadata for GeoTIFF. we're going to want a format-neutral version.
46 <bnordgren> (Yea Jody)
46 <simone> yeah
46 <simone> I wanted to use that as a base
47 <simone> I am missing something, sorry
47 <Martin_> Having GEOTiffMetadata doesn't prevent having a neutral version too. Thankfully, javax.imageio supports more than one IIOMetadataFormat.
47 <simone> this IIometadata
47 <simone> in the case of a ND (N>2)
47 <simone> coverage
47 <simone> stored, let's say, in a file
47 <simone> would relate to the overall file
48 <simone> or there would be an instance of these metadata for each 2D slice of it?
48 <simone> this
49 <bnordgren> I think J2SE allows for metadata associate
49 <Martin_> If we take in account that two kind of IIOMetadata exists in ImageReader ("stream" and "image), I guess that stream IIOMetadata would be for the ND case, and each image IIOMetadata would be for a particular 2D slice.
49 <CIA-10> 03jgarnett * r14969 10geotools/gt/plugin/wfs/test/org/geotools/data/wfs/ ( Patch to WFSDataStoreReadTest to allow testing against optional testing against localhost
49 <simone> that is what I wanted to hear
50 <bnordgren> with both file and image.
50 <simone> we need to take into account this from the beginning
50 <bnordgren> yes
50 <simone> stream metadata <---> whole dataset (2d,3d,4d,5d)
50 <simone> iiometadata <----> 2d slice
50 <simone> that would general
51 <simone> be
51 <simone> it is the way ImageIO works by the way
52 <Martin_> So, do we have a volunter for putting some generic IIOMetadata skeleton in
53 <bnordgren> I think right now, we're still identifying needs.
53 <Martin_> Right.
53 <Martin_> An other need is for an ImageReadParam with parameters like time.
53 <bnordgren> I mentioned this to Simone, but my intent is to outline work throughout the winter into next spring.
54 <simone> and z-level
54 <Martin_> Good plan Bryce. Thanks for doing that (smile)
54 <bnordgren> Yupper. refresh the notes page.
55 <Martin_> IIOMetadataFormat: need to contains CRS, Envelope, formulas for translating pixel integer values to "real world' (or geophysics) value. Something else?
56 <bnordgren> Any thots on OGC 05-027r1 fro IIOMetadata format?
56 <bnordgren> IIOMetadata is a DOM structure, right?
56 <simone> yeah
56 <Martin_> Yes.
56 <simone> whic spec is that one, you ask too much to my memory!
57 <bnordgren> That's a recommendation paper for Image related CRS transport in GML3.1.1
57 <Martin_> OGC 05-027r1 (Imave CRS in GML) may a good candidate; I would need to read it.
57 <bnordgren> Should lend itself pretty well to a DOM formulation.
57 <Martin_> However...
58 <Martin_> My understanding of Image CRS is that it is a kind of CRS before georectification (I means a CRS for raw image, oblique, with horizon on it, etc.).
59 <Martin_> After a rectification, the CRS attached to an Image may not be an ImageCRS, but rather a "normal" CRS like GeographicCRS.
59 <Martin_> So OGC 05-027r1 is certainly an important specification, but maybe only a part of what we need.
59 <simone> a look at gml in Jpeg2k?
00 <simone> it has been on my desk for months now...
00 <Martin_> An other candidate is "GML in JPEG2000" specification. However, this one is probably too much complex for our need. I suggest that we retains only a subset of "GML in JPEG 2000".
00 <bnordgren> My read was that it was capable of representing post-rectification CRS too.
00 <cresques> metadata of a rectified image should contain CRS of the original one?
00 <bnordgren> I'll have to look again.
00 <simone> agree
01 <bnordgren> I gotta split. (Wife's coming.) Who's going to take notes from here on out?
01 <Martin_> I will take some.
02 <Martin_> (just tell me when you have commited yours last notes to the wiki page)
02 <bnordgren> All right, good start! I'll read 'em tomorrow.
03 <bnordgren> (I'm not editing)
03 <bnordgren> later, guys...
03 |<-- bnordgren has left ("ircII EPIC5-0.0.5 – Are we there yet?")
03 <andrewW_> Nice to meet everyone, also have to leave. Will stay tuned, this (n-D WCS on netCDF/GRIB/...) is very important for us..
04 <cresques> didn't had a problem on the future, if we don't try to think too on streams and DN r/w before to start designing?
04 <jgarnett> (aside: The last commit has trunk working for windows & Java 1.4, I will wait for the results of a linux build before sending out the all clear email)
04 <--| andrewW_ has left #geotools
04 <Martin_> Thanks for fixing the build Jody.
05 <jgarnett> Well do need that linux and Java 5 test
05 <jgarnett> (if anyone has either of those env)
05 <jgarnett> And I am going to get in "trouble" with dzwiers for commiting the fix without permission.
07 <Martin_> So we talked about IIOMetadata and ImageReadParam. For IIOMetadata, the requirements are CRS, Envelope, sampleToGeophysics informations. For ImareReadParam, requirements are time and z-values. Any other requirements to write down?
09 <CIA-10> 03jgarnett * r14970 10geotools/gt/demo/referencing/src/org/geotools/demo/referencing/ is this how we want to use @version
09 <simone> I have a doubt
09 <simone> (sorry to bother)
10 <normanbarker> just thought about model time and forecast time for data
11 <simone> so far with this structure what we have been talking about was how to get 2d slices, that is grid coverages
11 <jgarnett> That was a good meeting ...
11 <simone> am I right
12 <simone> ?
12 <Martin_> Yes, it was basically about 2D slices. ND is not excluded however, but the path may be different.
13 <Martin_> I means, IIOMetadata is very likely to be pertinent for 2D. It may be pertinent for ND too, but maybe not as strongly.
13 <jgarnett> (someone is posting the logs from todays meeting?)
13 <Martin_> A ND GridCoverage reader may not be backed by javax.imageio.ImageReader.
13 <simone> basically the readers we are talking about
13 <Martin_> I will post the log.
13 <simone> are able to read 2D slices
14 <normanbarker> can we quickly discuss the N-D approach
14 <simone> but some other entity (Discovery? Catalog) will take into account ND by selecting slices?
14 <simone> yeah
14 <normanbarker> galeon is my driver, 5D is a requirement
15 <Martin_> It would also be able to read 3D slices if a file contains more than one image. A simple GridCoverageReader implementation could assumes that all images in a stream is a 2D slice, and consider the whole stream as a 3D coverage. From the user point of view, we would have a 3D coverage reader.
16 <normanbarker> I am not sure that is that different from a bands approach
16 <Martin_> Similar; the difference is a little bit arbitrary.
17 <cresques> 3d slices is a must, for satelite images
17 <Martin_> GridCoverage make a distinction between CRS dimensions and sample dimensions.
17 <Martin_> A band is a SampleDimension, but I agree that the difference is a little bit arbitrary.
17 <normanbarker> the thing with 3d, is that it would be nice to take vertical slices from a 3d cube (atmospheric data say)
18 <simone> Let me try to explain this, therefore I can see if I am understanding this or not
18 <cresques> there are too a more complicated data samples, as a landsat 7 scene, by example. 8 images with 3 different resolutions
19 <simone> let's say I have a data set which is air pressure and air temperature ina certain zone on the earth at different altitudes over a time range
19 <simone> this means 5D
19 <simone> 2 params
20 <simone> + 4D x,y,z,t
20 <Martin_> It would be a 4D GridCoverage with 2 SampleDimensions.
20 |<-- jgarnett has left ("Chatzilla 0.9.68a Firefox 1.0.4/20050511")
20 <simone> than 5D
20 <simone> then
20 <simone> even if your definition is more precise
21 <simone> in such a situation
22 <simone> the reader would be aware of the multidmensional nature using Stream meatadata
22 <simone> and then it would select the slices to use using ImageReadParams set using the slices' IIometadata
22 <normanbarker> vertical slices?
23 <Martin_> Not all work need to be on GridCoverage reader side. Lets me explain:
24 <simone> vertial
24 <simone> and temporal
24 <simone> and based on parameters
24 <simone> at the very end
24 <simone> a slice
24 <simone> would be 2D (xy)
24 <simone> but would have also
24 <simone> a time
24 <simone> a level
25 <Martin_> Lets use javax.imageio.ImageReader as an exemple. ImageReader is capable to performs decimation while reading the image. However, it doesn't performs interpolations; this is beyond the scope of ImageReader. ImageReader read datasubset only when it can be done without involving significant computation process.
25 <Martin_> If a user want to performs a subsampling using an interpolation method, he is better to read the full image and uses a JAI operation.
25 <Martin_> For vertical slice, I would said:
26 <Martin_> If the data layout on disk is organized in such a way than a vertical slice can be extracted in a natural way (for example the 3D coverage is already organized as a stack of vertical slices), then it is GridCoverageReader's work.
27 <simone> I am working on grib
27 <simone> since they a re a good example of this
27 <Martin_> Otherwise, we may consider that it is the job's of a GridCoverage's operation to be applied on the 3D GridCoverage after the reading, hoping that deferred execution and tiling will keep memory requirement at an acceptable level.
27 <simone> in a grib file
27 <simone> you can store
27 <simone> 5d dataser easily
28 <simone> and the format is well defined
28 <normanbarker> 3d coverage is a stack of vertical levels, I am thinking of a slice from top to bottom, bbox = x,y,z?
29 <Martin_> Yes, I understand that. Since a slice from top to bottom do not fit the "natural" layout of data and need processing power, it may be GridCoverageProcessor's work rather than GridExchangeWork.
29 <cresques> to read the full image it's not a good approach. They must be ways of choose not to do so, even for processes with high computation needs
29 <Martin_> We should not turn GridCoverageExchange into a GridCoverageProcessor.
30 <simone> basically it would result in applying a GridCoverageProcessor to each xy slice that fits the requested 2d levels
30 <simone> am i right Martin?
30 <simone> 3d
30 <Martin_> Maybe something like that. It need investigation...
31 <simone> I think it should not be big deal
32 <simone> subsetting 3rd 4th and 5th dim
32 <simone> all the hard work
32 <simone> (reprojections etc..)
32 <simone> is on xy==2d
32 <simone> and there is strong support for that in geotools
32 |<-- pramsey has left ()
32 <Martin_> We have to keep in mind that even when we request the whole 3D GridCoverage, it doesn't means that it will be read fully in memory. We should try to get deffered execution to work as much as possible.
33 <simone> yeah
33 <simone> I agree
33 <simone> that is a matter of writing correctly the plugins for readers
33 <simone> I am thinking about
34 <simone> doin 3d
34 <simone> x,y,t with a series of tiled tiff
34 <simone> right now we could load only what we need
34 <simone> seamlessly
34 <simone> leveraging JAI readers
34 <simone> that should be the goal for all the plugins we wrote
35 <Martin_> Agree.
36 <Martin_> We are going late. Should we close this IRC, try to put some note on Bryce's pages and decide for a next IRC meeting?
36 <cresques> yes, it sounds the right way
36 <simone> cool
36 <cresques> here it's 1:35 AM
37 <simone> same here
37 <simone> but it was worth it
37 <Martin_> Wow! Time to close.
37 <normanbarker> thanks, Andrew Woolf and myself have found it interesting
37 <simone> let's schedule next one
37 <simone> or at least try to
37 <Martin_> In 2 weeks?
38 <simone> sure
38 <simone> conclusions?
38 <simone> 1>volunteering for geoapi
38 <simone> 2>iiometadata
39 <simone> ??
39 <Martin_> 3>ImageReaderParam
39 <Martin_> 4>Put common parts in package
40 <Martin_> 2.5>Looks at OGC spec for an IIOMetadataFormat
40 <simone> I can start working on 2-2.5-3 and write something on the RnD
41 <Martin_> 3.5>Find some way to express a parameter for requesting a slice in a N-dimensional GridCoverage at least when it match the natural data layout.
41 <Martin_> Nice! Thanks.
41 <simone> I am trying to get some prototype using Grib files
41 <simone> they are nice to do that
41 <simone> they are usually small in size
41 <simone> but with a lot of level and time instants and params! (smile)
44 <simone> i think you send an email asking for 1
44 <simone> should send
45 <Martin_> Will send one on the GeoAPI mailing list.

  • No labels