Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 4.0

<simboss> ciao mbedward
<simboss> or michael (smile)
<aaime> Hi
<mbedward> hi simone, andrea
<mbedward> I'm not sure how to start - do you have something you would like to begin with
<aaime> Is this a meeting? (smile)
<mbedward> no just a chat (smile)
<mbedward> about jai-related things
<simboss> well, the floor is yours
<simboss> daniele is coming
<simboss> not sure if antonello is around
<simboss> looks like is not
<simboss> around
<simboss> checked on the udig channel
<mbedward> I can wait a few minutes - that's no problem
<simboss> if you did not sent around a reminder
<simboss> it might be that he has forgotten
<simboss> (smile)
<mbedward> yes - I'm afraid I forgot about reminding anyone (smile)
<mbedward> perhaps we can just start chatting - we can always go back if he joins us or rehash later
<aaime> if you want any chance of him showing up send a mail to him and Silvia
<mbedward> ok, I'll do that now
<moovida> morning folks
<moovida> thanks for the email
<mbedward> no worries
<moovida> I was working on a bug and forgot (smile)
<moovida> what did I miss?
<mbedward> we haven't started
<moovida> oh, great!
<mbedward> we couldn't start without you !
<moovida> (big grin)
<mbedward> perhaps I'll begin my just describing my immediate project needs that have led me down this path
<moovida> good for me
<mbedward> for an app that I'm working on at the moment I need to be able to do two things
<mbedward> 1. allow a user to create a grid coverage based on mathematical expressions
<mbedward> where the user just provides a relatively simple script
<mbedward> and the expressions have spatial position and/or other coverage data as inputs
<aaime> just for the record, Sextante has some raster algebra support built-in
<mbedward> ok - I'd be interested to hear about that in a minute
<moovida> mbedward: what do you mean by: the expressions have spatial position?
<--| ggesquiere has left #geotools ("Bye")
<mbedward> basically functions of x,y coords
<moovida> not row/column, but instead coordinates?
<mbedward> could be row/col, or proportion of width/height etc
<moovida> can you make an example in which you would want to use coordinates inside an expression?
<moovida> ah, ok, got it, wanted to be sure
<mbedward> sure: my app is about animals in landscapes
<mbedward> responding to various resources
<mbedward> imagine I want to create an artificial resource map
<mbedward> with a north-south gradient
<mbedward> I might do that with a simple linear function plus a noise component
<moovida> ok, nice
<moovida> so you are talking about functions really or more neighbour cell touching?
<mbedward> ???
<moovida> I mean you won't be able to embed a function based on position in a static approac
<moovida> you will have to use relative positions between cells?

  • moovida has difficulties to explain himself
    <moovida> so the gradient will be based for every cell on values of the cells around it
    <moovida> is that what you mean by function?
    <mbedward> that's not what I was think about in the example...
    <mbedward> although neighbourhood stuff certainly comes in lates
    <mbedward> later
    <moovida> ok, good, how would a hipotetic function look like?
    <moovida> you have an idea already?
    <moovida> the linear function for example
    <mbedward> value = xpos * alpha + beta + random(sigma)
    <mbedward> where xpos could be position along coverage axis scaled as 0 -> 1
    -->| mauricio (n=mauro@128.Red-80-36-0.staticIP.rima-tde.net) has joined #geotools
    <moovida> sounds good
    <mbedward> so very simple
    <mbedward> this was the idea behind 'jiffle'
    <mbedward> the ANTLR grammar that I'm fiddling with
    <mbedward> it's essentially a simple expression evaluator
    <mbedward> I've put the current ANTLR parser and tree grammars on to the project site
    <mbedward> they are very preliminary but maybe give you some idea when you look at them
    <moovida> ok
    <mbedward> but it isn't joined to images yet
    <mbedward> now that I've got basic arithmetic, trig functions etc in jiffle
    <mbedward> the next thing I'd like to do is work out how to plug input image data into the interpreter
    <mbedward> I was interested in the r.mapcalc page that you pointed me to yesterday
    <aaime> again for the record, what about Janino? http://www.janino.net/use.html It can be used to compile expressions into bytecode. Makes the evaluation very fast
    <moovida> r.mapcalc is what I will need and for which I could work
    <mbedward> I don't know this at all - would be very interested to know more - have you used it ?
    <aaime> nope
    <moovida> and I think it contains also what you need
    <moovida> so, how could I/we help you?
    <mbedward> you mentioned that you were using jep previously ?
    <aaime> sextante does too
    <aaime> (use jep)
    <moovida> yes, I did for a while
    <moovida> but it is now closed source
    <moovida> and so I would not want to use it any more
    <moovida> or better, I do not use it any more
    <aaime> yep, that's why I looked into janino a little bit, to suggest Sextante an escape route
    <mbedward> I understand jep was highly optimized (question)
    <moovida> yes, it was, but from that moment on they closed the source (smile)
    <mbedward> at the moment speed is less important for my needs than just ease of use and flexibility
    <mbedward> and having something that I can understand !
    <moovida> ok, but you would not want to use jep, right?
    <mbedward> what are your r.mapcalc needs ?
    <mbedward> no - I want to use open source
    <moovida> great
    <moovida> my needs are well inside yours
    <moovida> functions would be great
    <moovida> because they contain also functions based on several maps
    <moovida> and conditions are important
    <mbedward> yep
    <moovida> things like:
    <moovida> if in map1 the value is null, do that, else take value from map2
    <moovida> but I think on this we have the same ideas
    <mbedward> yes
    <moovida> so one question from an ignorant antrl person
    <moovida> antrl is used as a functional language "compiler"?
    <moovida> and then you need something to evaluate the transformed language?
    <mbedward> it is general parser building tool
    <mbedward> similar in aim to lex, yacc, bison...
    <mbedward> for example
    <moovida> ok, so thorught that you get your java language for the math you write
    <mbedward> you can
    <mbedward> or C, or python or other things
    <mbedward> it can be used to build a translator
    <mbedward> but what I am interested in is an interpreter
    <mbedward> so two steps...
    <mbedward> first provide a grammar that describes the language elements (similar to EBNF)
    <mbedward> from this ANTLR helps you to build a lexer and parser that can accept language statements
    <mbedward> detect syntax errors
    <mbedward> re-arrange things for faster processing
    <mbedward> the output of this first step is a tree representation of your input statements
    <mbedward> Abstract Syntax Tree in the jargon
    <mbedward> second step is to build a 'tree-walker'
    <mbedward> this knows how to take navigate the tree-representation of your 'program'
    <mbedward> and you can embed the code for actions into the walker
    <mbedward> that's a really terrible explanation (smile))
    <mbedward> I'll give an example
    <moovida> (smile) I understand, the JGrass console is built like that
    <mbedward> phew...
    <moovida> now I am very sorry we didn't use antrl
    <mbedward> antlr is really cool
    <mbedward> I'm only learning
    <mbedward> but it is possible to get quite a bit done in a small amount of time
    <moovida> the developer created an own compiler, and now every time I have to changes something, I get mad (smile)
    <moovida> yes, that was my feeling of antrl
    <moovida> sound really good
    <mbedward> in the past I've written parsers by hand and this is **much** easier
    <moovida> what timeline did you think of for this?
    <moovida> also, were are you based?
    <mbedward> I'm already very late (smile)
    <moovida> (big grin)
    <mbedward> I'm in Sydney
    <moovida> argh, I found one in London and thought we could meet for an initial push (sad)
    <mbedward> I think he works for the BBC
    <mbedward> I saw his name on TV once
    <moovida> hmmm, then let's get back to the timeline, Sydney is out of budget for this project (smile)
    <moovida> just to understand if we could coordinate
    <moovida> I also have to study antrl first
    <moovida> so it would be good to see if we are at least a bit in sync
    <mbedward> well, it's very fuzzy for me and there are many strands to my project that I can also work on when it's not convenient to be in sync
    <mbedward> sync in time I mean
    <mbedward> there are good intros on the antlr site
    <moovida> yep
    <moovida> great, I'll check them out
    <mbedward> I want to find a good starting approach for inputting image data into the interpreter...
    <mbedward> r.mapcalc had the simple convention of var names
    <mbedward> if not functions etc. they were assumed to be maps - is that right ?
    <moovida> yes, and keeping those would help me get JGrass -> GRASS compatibility
    <moovida> yes, exactly
    <mbedward> mmm...
    <mbedward> I was thinking of something slightly more indirect
    <mbedward> though it might still be compatible
    <moovida> users have huge scripts for mapcalc to do the most powerfull things
    <mbedward> I thought that would be the case
    <mbedward> so it would be great to be compatible
    <moovida> good you think so
    <moovida> at least to start like that (smile)
    <mbedward> I want to keep this all as modular as possible
    <moovida> then there are always possibilities to get out of compatibility
    <moovida> what do you mean?
    <mbedward> well the interpreter is one module
    <mbedward> it 'knows' (will know) how to access data from some provider interface
    <mbedward> but the provider implementations could be quite diverse
    <mbedward> my thinking is very fuzzy...
    <mbedward> in r.mapcalc is there any sort of definition section for variables / inputs
    <mbedward> or do they just appear in statements ?
    <moovida> I don't understand exactly what you mean
    <mbedward> is there anywhere in the script where you have to declare var names prior to using them ?
    <moovida> you can have also variables evaluated in steps and then used inside the script
    <moovida> no, var=eval() is enough
    <mbedward> I need to go study r.mapcalc while you look at ANTLR (smile)
    <moovida> nice
    <moovida> (smile) then we have a sort of plan
    <mbedward> yes !
    <moovida> even if I feel it will take me some longer
    <moovida> the mapcalc module can do very complex things
    <moovida> did you think to make a constatly usable progress?
    <mbedward> definitely !
    <moovida> I mean, start to work and get a version that supports some functions and son and on?
    <mbedward> otherwise giant mess !
    <moovida> great, that sounds good
    <moovida> same for me here, and also the project then can grow
    <moovida> slowly and stable
    <mbedward> there is also a second strand to this work - JAI operators
    <mbedward> especially for neighbourhood map calculations
    <mbedward> I need to be able to do the sort of things that, in the old, old days I might have done in arc-info (smile)
    <mbedward> esp. focal functions
    <mbedward> e.g. average neighbourhood value, dominant value, number of discrete values...
    <mbedward> I was a bit surprised that these were not already in JAI operators
    <moovida> are you sure there aren't?
    <mbedward> also being able to constrain these and other calculations with mask / ROI
    <moovida> sounds strange to me also
    <mbedward> I think some of these things could be useful to geotools etc. generally
    <mbedward> and I would like to open source my code for new operators
    <mbedward> so I plan to put them on the jai-tools site also
    <moovida> seems to be a lot of work to be done (smile)
    <moovida> what was your schedule again?
    <mbedward> I'm very very late (sad)
    <moovida> for yesterday or so?
    <mbedward> but I am also the project manager
    <moovida> (smile)
    <mbedward> unfortunately the project does not have enough money to employ a real programmer...
    <mbedward> so an ecologist is doing the job
    <mbedward> me
    <moovida> (big grin)
    <moovida> ecologist and environmental engineer (smile)
    <mbedward> mmm... (smile)
    <mbedward> obviously there is overlap between ANTLR side and jai-operators side
    <moovida> any other person that wants to join the brigade?
    <mbedward> I think / hope simone is interested in the jai operators strand
    <moovida> simboss, dany_r ? What about you guys?
    <dany_r> yes he's interested
    <moovida> mbedward: just to have an idea, you need a version that does x,y,z for date a,b,c
    <moovida> can you tell about x,y,z and a,b,c
    <moovida> if you are allowed, and just a more-or-less
    <moovida> is enough
    <mbedward> yep, roughly anyway...
    <moovida> this job is resources absorbing and if I help I want to be able to really do so
    <mbedward> in next month I would like to have the scripting working to the point where
    <dany_r> our aim would be allowing handling, chaining math operations and functions in JAI operators.
    <mbedward> it can do arithmetic and basic math/trig functions already in grammar
    <mbedward> plus conditional expressions
    <moovida> hmmm, I guess that will not be exactly the case
    <moovida> am I right?
    <mbedward> and access data from input images
    <mbedward> (will get to that in a sec)
    <moovida> doing it mbedward's way will mean that no chaining occurs?
    <mbedward> if I can get that working by end of Feb that would be very good for me
    <mbedward> ok - in terms of output
    <mbedward> I have thought about two approaches, each of which I'd like to implement
    <dany_r> in this right moment Simone is not here but he can provide you more details about its idea very soon.
    <mbedward> first is interpreter produces output that can be linked to a JAI image function
    <mbedward> great
    <moovida> brb
    <moovida> sorry guys, I have to run, could anybody post the logs, or send me the logs, if much is said after this moment?
    <mbedward> no worries
    <moovida> mbedward: I am very interested to join the efforth, but I have to understand if I am ablr to contribute that early
    <mbedward> sure
    <moovida> to do so, I will study antrl in the next days
    <moovida> and study what you already did
    <mbedward> you can then tell me all the mistakes
    <moovida> would be glad to (smile)
    <moovida> and let's keep the discussion in the list active please
    <mbedward> will do
    <moovida> thanks for the chat
    <mbedward> cheers
    <moovida> ciao
    <--| moovida has left #geotools
    <mbedward> dany would you like to keep going when simone comes back ? or I can pen some thoughts and send them to you...
    <simboss> ciao michael
    <simboss> I am back
    <mbedward> ciao simone
    <simboss> sorry, but I have someone here, so I had to talk to him
    <simboss> (smile)
    <simboss> I am all yours now (big grin)
    <mbedward> (smile)
    <mbedward> I think we were only just getting to teh things you are most interested in
    <mbedward> jai operators plus
    <simboss> ah great
    <mbedward> possibility of using an interpreter to help make image functions etc.
    <simboss> well, at this stage
    <simboss> my interest is in having a good project
    <simboss> where we can drop JAI operators
    <simboss> so that we can releases
    <simboss> have
    <simboss> releases
    <simboss> in a more stable manner
    <simboss> JAI is not dead
    <simboss> but it's a zombie
    <mbedward> why has that happened ?
    <mbedward> it's hard to understand
    <simboss> the members of the prject are heavily involved in other projeects
    <simboss> well, it's simpler than it seems
    <simboss> JAI was created under bih push from JPL
    <simboss> the Jet Propulsion lab fa NASA
    <mbedward> yep
    <simboss> they used in a few mission
    <simboss> and they are still using
    <simboss> and it's pretty good the way it is
    <mbedward> but there are some surprising gaps
    <simboss> they are not interested in moving it forward
    <simboss> or in babysitting
    <simboss> it
    <simboss> exact
    <simboss> but it's an empasse
    <simboss> they do not want to spend time on it
    <simboss> but they are not releasing it
    <simboss> as OS
    <simboss> that's why I started imageio-ext
    <simboss> and I am happy to see something like JAI-operators
    <mbedward> are there tricky licensing issues ?
    <mbedward> for instance
    <mbedward> I have written a modifed convolve operator
    <mbedward> that can use an ROI
    <mbedward> so it's really just a hack of the existing image op
    <simboss> my advice about code from JAI is use it as a template
    <simboss> but rewrite it all
    <mbedward> I see
    <simboss> the license is too messy to reuse that code
    <simboss> for imageio
    <simboss> you can reuse it, but that's another story ....
    <simboss> anyway
    <simboss> I am hoping that sooner or later SUN will OS JAI and Imageio
    <mbedward> I have a number of operators that I need for my current project
    <mbedward> so I would like to open source these
    <simboss> if we show them that there is a community able to ensure the continuity of the projects that might help them to make up their minds
    <mbedward> that would be great
    <mbedward> but surely they must know that jai is very actively used already ?
    <simboss> yeh they know very well
    -->| dany_r_ (n=chatzill@host177-41-dynamic.30-79-r.retail.telecomitalia.it) has joined #geotools
    <simboss> but their team, while very very good, is small
    <mbedward> ok
    <mbedward> I should compile a list of the operators that I will be needing soon
    <mbedward> and send it to you
    <mbedward> perhaps some are already available and I have overlooked them
    <mbedward> otherwise it will be the list of things I will be coding in the next month or two
    <simboss> there is one thing
    <simboss> that I might want to bring up
    <simboss> I did the same thing with antonello a while ago
    <simboss> are your operators geospatial-aware (e.g. they need some parts of geotools torun)
    <mbedward> not so far
    <simboss> or do they run in the pure raster space, where there is no notion of geospatial
    <mbedward> yes
    <simboss> I would recommed to be careful with this in order to not introduce circular dependencies between the teo projects
    <mbedward> I understand
    <simboss> usually I try, whenver possible, to clearly separate geostuff (geotools) from pure raster stuff (jai or imageio)
    <mbedward> yes
    <simboss> the key for doing this is usually the gridtoworld transform
    <simboss> (probably I am going to far, but I think it is important to say at least once)
    <mbedward> yes, that's good point to bring up early
    -->| acuster (n=acuster@mtd203.teledetection.fr) has joined #geotools
    <mbedward> so far
    <mbedward> for example with the convolve operator I mentioned
    <mbedward> that works in raster space
    <mbedward> then there is a wrapper for geo-spatial part
    <mbedward> but I agree with you that keeping those things separate is best
    <mbedward> and I also find it easiest
    <mbedward> anyway, regarding the project and site
    <simboss> following you...
    <mbedward> I am very open to
    <mbedward> approaches to managing it / handling releases etc
    <mbedward> I don't have fixed ideas or
    <mbedward> requirements in that regard
    <mbedward> and to begin with
    <mbedward> I will just be gradually uploading my operators
    <mbedward> and working on the ANTLR related strand
    <mbedward> we are probably done for now - yes ?
    INFO Preference "collapseMsgs" is "on".
    <simboss> I think
    we would like to port one operator
    that is right now inside geotools
    which we could probably somehow join with your stuff
    we should develop also a simple operator

    <-- acuster has left irc.freenode.net ("Leaving")
    <simboss> thatcan be used
    to transform a single color
    into a trasparent color
    <mbedward> which is the operator in geotools ?
    <simboss> it is buried inside the renderer since there was not complete consensu about where to put it
    <CIA-21> aaime * r32368 /trunk/modules/ (4 files in 3 dirs): GEOT-2316 jdbc-ng should advertise query capabilities
    <simboss> it can be used to create a piecewise transformation
    <mbedward> have to go in minute - sorry
    <simboss> np, I think we are done

    <-- dany_r has left irc.freenode.net (Read error: 113 (No route to host))
    <mbedward> thanks simone !
    ciao
    <simboss> thanks to you man
    ciao ciao