<brett> ok, anyone up for dependency mediation?
<jdcasey> bring it on
<brett> jdcasey: you said you had some feedback on that?
<jdcasey> it's about the whole resolve-to-concrete thing on release
<brett> yep, that's important.
<brett> I had a couple of questions myself in the yellow box there ;)
<brett> what's your Q?
<jdcasey> what about creating either (a) a special release-POM with a=
ll deps resolved concretely, or (b) a special tagged version of the POM wit=
h that info, followed by another non-tagged commit of the user-defined POM
<jdcasey> this is something it seems that you're wrestling with in th=
<jdcasey> I'm -1 on losing user info in favor of resolved deps...but =
that's an obvious statement
<brett> yeah, I didn't think of recreating the pom with resolved stuf=
f, I was just going to mark it up
<brett> which also facilitated doing it in SCM
<jdcasey> mark it up how?
<brett> that extra resolvedVersion tag
<brett> maybe it could add a profile :)
<jdcasey> that's an idea, but it'll lead to some serious bloat, and f=
or something that's essentially a snapshot in time
<jdcasey> IMO so would the resolvedVersion stuff
<jdcasey> resolvedVersion would get stale pretty quick, and probably =
would be removed before starting development of the next release anyway
<jason> what is scope=3Dnone? only resolved for the project?
<jason> the example on the top there
<jdcasey> but wow, this represents a big shift in maven philosophy wr=
t versioning...now we're starting to enforce a versioning scheme of sorts
<brett> jason: that was how I figured to exclude transitive deps you =
<jason> when would you want to do that?
<jason> is the jdbc thing a real example?
<brett> ideal world: never
<brett> but we can't leave projects at the mercy of their dependencie=
<brett> so this would be how they get to have final say on their deps=
<brett> rather than turning of transitivity altgoether
<brett> which would just be abused
<jdcasey> but if a dependency declares that it needs such-and-such, c=
an we really second-guess that? it seems like it would lead to bugs/errors
<jason> did someone have a case where scope=3Dnone helped?
<brett> jason: you'd use that example on velocity, for example
<trygvis> the continuum plexus app
<brett> when you have a dep on velocity, and it gives you jdbc
<trygvis> and logkit+avalon framework api
<brett> jdcasey: we're not second guessing as such, its just making s=
ure the project gets to have the final say
<brett> I think that's important, both on inclusion and version
<jason> yah, that's a crappy build
<brett> right, and there will -always- be crappy builds
<jdcasey> so if I define a dep with scope=3Dnone, it'll just blow it =
out of the deps list wherever it encounters it?
<brett> jdcasey: right
<brett> jason: I know, but...
<jason> i think some extra deps there might be better then scope=3Dno=
<jason> might it lead to some problem, where you did that and have an=
other dep that needs jdbc
<brett> yes, it requires some intelligent use
<jason> just playing devil's advocate and what's better, some extra d=
eps or something that potentially doesn't work
<brett> not when they get bundled
<brett> not when they don't exist in the repo
<jason> sorry, not following
<brett> not if they are not licensed appropriately (when we start che=
<brett> examples when it is not better to just get the deps
<jason> what do you mean "not when they get bundled"?
<brett> way I see it is that the project should always have the final=
say, but this is somewhat annoying enough that they will try and get the s=
ourc eof the problem fixed
<brett> look at continuum's lib directory
<brett> I know they could be excluded in other ways
<brett> but that's the sort of thing we're trying to avoid
<jdcasey> brett: how will the scope=3Dnone stuff work when it's in a =
project that's brought in through transitive resolution?
<jason> yah, not ideal but i can live with that knowing that the velo=
city build will improve
<evenisse> If I use a dep that define a dep with scope =3D none, woul=
d I obtain it or is it excluded only for the project?
<brett> jason: you can. other's won't
<jason> i think the scope=3Dnone could be potentially cause problems
<brett> ok, alternatives to solve the same problem?
<brett> or is it solving the problem that causes problems?
<trygvis> a better application for managing the repository so stuff l=
ike this can be fixed in a better and more permanent way
<jdcasey> trygvis: the avg. user won't have access to such
<jason> i don't think getting some extra bits is a huge problem
<brett> ok, jason volunteers to answer that question on the user list=
every day ;)
<jason> i will get loggy to do it!
<jdcasey> brett: I think one thing that would help immensely is knowi=
ng what implies what...seeing the dependency graph, in other words
<trygvis> we'll just use bayesian filtering on the user list to point=
people to the FAQ
<trygvis> like we talked about jason
<brett> jdcasey: I agree
<jdcasey> a tool to do that, plus a local-only way to "fix"=
poms might do the trick
<brett> local fixes is problematic
<jdcasey> b/c of portability?
<trygvis> because it's local :)
<brett> rm -rf local_repo breaks stuff
<brett> repoman is better, but we really want to hear about it
<brett> everyone using scope=3Dnone emails the maven developers autom=
<jdcasey> you mean setup a trigger in maven to actually notify the li=
st (or whatever) that this pom is broken? :)
<brett> ok, lets put it aside as problematic. I think we need a solut=
ion. I don't think it is acceptable to just drop in extra deps for the reas=
ons given above. We'll come back after more thought
<jdcasey> brett: [repost] how does scope=3Dnone factor into poms reso=
<jason> it just gets taken out of the graph entirely
<jdcasey> ignored (I hope)?
<brett> not sure
<jdcasey> I mean resolved poms with a scope=3Dnone in it
<jdcasey> ok, just file it away then :)
<brett> depends where depMgmt is applied on transitive poms
<brett> will also be affected by versioning, so I'll just note it
<brett> I think "changes to dep mgmt" actually describes ho=
w it works now
<brett> except for the last point, which is just scope=3Dnone
<brett> changes to scope element can be skipped
<brett> everyone agree "resolve dependencies once only" is =
<jdcasey> I like that one, actually
<brett> ok, something more tricky
<brett> version ranges
<ben> oh that'll be fun
<jason> would be good to get one of the SAS guys here
<jason> i'm going to mail one of them, they probably have some ideas =
<jason> having 300 builds
<jason> and if they would go for a standard versioning scheme
<brett> will gwaltney one of them? he was here 1/2 hr ago (I assume t=
hat was him)
<jdcasey> jstrachan might have a problem with it ;)
<jason> yah, he's one and jared
<jdcasey> active-* always causes problems on conversion with version =
<brett> what did you think of the versioning scheme?
<jdcasey> brett: what's the diff between '1.0' and '(1.0)'?
<jdcasey> okay, then
<brett> "Soft requirement, equivalent to 1.0 on its own
<brett> but [1.0] would be absolute
<brett> does the syntax suit people?
<brett> very math oriented :)
<brett> kind of ugly though
<jdcasey> nah, syntax is good for me
<jason> i think it's good, being math oriented we know it already wor=
<jason> set notation good
<jason> no one can argue with us
<brett> I bet someone has a version alpha(2) :)
<ben> not on windows they don't
<brett> ok, we'll stick with that for now
<brett> I like Jason's point too
<ben> oops I tell a lie, ( is valid on windows
<jdcasey> brett: will 1.0-RELEASE and 2.0-RELEASE be ok together?
<brett> no such thing
<jdcasey> you can't be maintaining two release cycles on a product?
<brett> I really prefer RELEASE be silent and behind the scenes
<brett> that's not what it was intended for...
<brett> we might need to re-evaluate, its mainly to support the plugi=
ns right now
<jdcasey> misunderstood the finer points of RELEASE, I guess
<brett> you're right, which is why I want it hidden
<brett> if that makes sense
<jdcasey> I think so
<brett> ok, are we happy with the technique of version comparison, an=
d the version layout?
<jdcasey> I think those types of notations (like -SNAPSHOT) might be =
useful eventually, but for now, that's no biggie
<jdcasey> +1 here
<brett> shall we change our snapshots to have _build then?
<jason> i pinged walt to see if he wanted to show up
<brett> cool, we'll skip over the next bit until then as that's the i=
<jdcasey> brett: I think so
<jason> as long as you stipulate deployment to one repository or else=
you won't be able to increment accurately
<jason> which we are doing now yes?
<jason> then that's cool
<brett> except under profiles, but that's intentional
<jdcasey> brett: we may have to move the artifact classifier too, jus=
t to make the whole thing more parseable
<brett> ie, you are incrementing different builds
<brett> jdcasey: not intending to have to parse it in the repo
<brett> you just need group ID and artifact ID
<jdcasey> actually, strike that anyways
<brett> that identifies the project, and that has all the info
<jdcasey> I meant for the repo mgt tools
<brett> yep, it can find the project.
<brett> next point was "preventing RELEASE dependencies", w=
hich I explained earlier. Let them specify an open ended version instead
<brett> so [2.0) is 2.0-RELEASE
<jdcasey> is resolvedVersion on hold pending further thought?
<brett> yeah, will do that last if SAS guys turn up
<brett> hmm, did I screw up set notation?
<brett> I think ) actually means "don't include this"
<jason> look on mathworld
<jdcasey> should've known it would be a wolfram site
<brett> ok, I'll check later
<brett> lot of good that maths degree did me
<brett> agree on parent versions?
<brett> "They do not need to incorporate ranges, and are general=
ly treated as unversioned."
<jdcasey> brett: I suspect you're right on being wrong, tho... ;)
<jdcasey> what do you mean by unversioned?
<jdcasey> they can change, no?
<brett> yes, to the user they generally want the latest
<brett> but it gets resolved on deploy
<brett> they can specify it, but generally they don't want to
<brett> actually, that's part of the release mgmt discussion, and JAs=
on and I spent a fair while on it before, so maybe we should come back to i=
<brett> nearly to the end of this document, then might be time for an=
other break :)
<brett> agree that version ranges wouldn't match a snapshot unless it=
is specified as a boundary?
<brett> ok, so all that is left is conflict resolution, and reproduci=
<jdcasey> I'm interested in what would create a warning message as be=
ing potentially workable, but outside the specified range(s)
<jdcasey> is that just any dep that is specified by group/artifactID,=
but where the version is incompatible?
<brett> that may be redundant, I had it written earlier
<brett> where is it again?
<jdcasey> maybe I made it up...can't find it
<jdcasey> disregard it, then
<brett> no, I said it
<brett> oh, its just in the original reqs
<brett> they are not part of the design, just the inspiration for it
<jdcasey> well, that would apply in "use-nearest" after a f=
<brett> yeah, I think using soft requirements solved that
<jdcasey> if you use the nearest, and another transitive dep had that=
artifact with a different version spec, you might want to drop a warning
<jdcasey> ok, cool
<jdcasey> where is that strategy best defined?
<jdcasey> I might argue for the settings.xml, personally
<brett> needs to be consistent for different people building the proj=
<jdcasey> per-project might leave it open to changes injected by POMs=
of transitive deps, though...
<jdcasey> I agree on that
<brett> I don't think you'd honour what a transitive pom did
<jdcasey> guess you could specify to use the strategy in place, and i=
gnore those specified by POMs of transitive deps
<brett> I'd like to start with just the one strategy so we get a feel=
for how it works, and see whether others are really needed
<brett> easier to give than to take away
<jdcasey> so start with the range-based?
<jason> just a note, if we're going to start using a graph proper the=
n let's make a separate project of that tool because i want to be able to v=
isually represent the graph with decorations on the nodes for all the items=
we're talking about
<jason> i want to use the exact same tool for visualization
<jason> jung.sf.net is very cool btw
<jason> i've settle on thta
<brett> ok, I'll take a look
<brett> do we think we'll take a crack at the reproducibility questio=
<jdcasey> I think we have to, if we're going to get into compatibilit=
y ranges at all
<brett> yeah, I just mean now :)
<jdcasey> we have to have a concrete, reproducible build for release
<jason> if we're doing ranges are we implicity going to promote our v=
<brett> I think we only support our versionining strategy for ranges =
<jason> i would like
<jason> i think if we have something reasonable, people will use it
<ben> could you not emit a file containing the decisions the automate=
<ben> then use that as an optional input at another point in the futu=
<jdcasey> ben: like a properties file or something of resolved versio=
ns, as a separate file?
<brett> jason: it encompasses most of the standards now, although cha=
nging build to _ differs from RPM
<jdcasey> the only problem with that might be portability
<brett> what do you mean jdcasey?
<jdcasey> was talking to ben :)
<brett> I know
<jdcasey> I mean the resolved-versions file would have to be deployed=
<jdcasey> unless we have a fully-resolved pom that we release to the =
<jdcasey> that means a pom with all transitive deps included and pegg=
<brett> ok some objectives:
<jdcasey> actually that might be a really good thing to do...flatten =
it all out for a release
<brett> - capture all of the information used in a release
<brett> - retain the user's original specifications
<brett> - store at release point in SCM, and repo
<brett> the question I have is, when do we use the resolved versions,=
and when don't we?
<brett> hi jared, willgwal
<brett> brb, grabbing coffee. feel free to discuss
<jdcasey> I'd say that you're going to have a problem any time you're=
unable to depend on a release-quality POM...you don't know how it'll resol=
ve those transitive deps
<brett> that's whats getting to me
<jdcasey> I think that on release you should build a release-quality =
POM that has all transitive deps included in it, with concrete versions for=
each. This POM shouldn't replace the pom.xml for the project, but should p=
robably be put into version control for future reference
<brett> if you always take what the release used, you are losing the =
ability to make a smarter decision knowing all the factors it had available
<brett> if you always take the other factors, you might get a differe=
<brett> jdcasey: sure, that's for building that project. I agree 100%=
<jdcasey> but do you want to make a different decision? that's not re=
producing the build, and could mean that debug/test builds fail to reproduc=
e a bug
<Arnaud> +1 for jdcasey and POMs
<brett> here's another point: shouldn't it always make the same decis=
<brett> the released poms don't change
<jdcasey> aren't you assuming that all the transitive deps will be re=
<brett> for a reproducible build, yes
<brett> the release plugin should just freak out at snapshots
<jdcasey> but as a user you don't always have that kind of control
<jason> releases or just non snapshots?
<jdcasey> jason: what's the difference?
<brett> is there a difference?
<ben> but lets say commons-foo depends on commons-clown [1.*]
<ben> and commons-clown 1.1 is out for your original build, but after=
than, commons-clown 1.2 came out
<evenisse> I'm sorry, but I must leave.
<jdcasey> what about the timestamped deploys that go with snapshots? =
can't you depend on those for a release?
<jason> jdcasey, that's what i mean
<jdcasey> yeah, just got that :)
<jason> it doesn't necessary have to be a proper release, just a time=
<jason> sorry, i have twins running around at my feet
<jdcasey> that's why we can't depend on all transitive deps to supply=
<ben> jason: I have a newfound appreciation for the distraction that =
children cause :)
<jason> oh yah
<jason> that's why i borrow them :-)
<jason> i'm enough of a scatter brain
<brett> the problem with timestamped snapshots (at least now) is that=
the deployed poms may contain unresolved snapshots
<jdcasey> right, which means that when your project does a release, i=
t has to resolve and then peg those snapshots for its own purposes
<jdcasey> so it has to collapse the transitive deps into its own pom
<jdcasey> it'd be nice to annotate them as to their origin
<jason> what if decided that anything deployed had to be resolved?
<brett> hold up
<brett> 3 questions I can answere here
<brett> actually, 2 I forgot what I had to say about clowns
<trygvis> they rock?
<brett> folding down transitivity:
<brett> this means you can't effectively use the ranges any more
<brett> unless you retain the spec to use later
<jdcasey> I see what you mean
<brett> [1.1) might resolve to the latest 2.0, but your project wants=
[1.2] exactly, no way to find out what was really wanted
<brett> jason: resolving snapshots on deploy means you don't get upda=
tes any more, basically same reason
<brett> you lose info
<brett> is that right?
<trygvis> this is something that all the linux distros have fixed, we=
should perhaps look more into what they've done
<jdcasey> maybe we need a built-with metadata file, then...more of an=
audit than anything else
<jason> what are you losing if the POM is fully resolved for deploy?
<brett> trygvis: have they?
<jdcasey> something that we could use as input
<jdcasey> jason: you lose the compatibility statement provided by tha=
<jdcasey> for other projects to use
<jdcasey> range on the dep, I mean
<jdcasey> or snapshot, yeah
<brett> a SNAPSHOT depends on b SNAPSHOT (1), c depends on a SNAPSHOT=
and hence b
<brett> a gets published, is now (1) with dep on b (1)
<brett> new b (2) is published
<brett> c ->a (1) -> b(1)
<brett> but you want b(2)
<jason> yah but for a release how are you ending up with a snapshot d=
epending on a snapshot?
<jdcasey> brett: I'd agree, with one thing: major version changes are=
meant to signal API incompatibility, no?
<jason> oh, the ranges
<jason> jdcasey, typically
<jdcasey> so maybe [1.0) would mean anything in the 1.x range, but no=
<brett> I don't think so
<brett> they might be backwards compat
<jdcasey> [1.0,2.0) would be correct for that case, then?
<brett> interesting thought there
<brett> hard to retrofit onto something that used to declare [1.0)
<trygvis> on debian they would change the artifact id if they change =
<brett> typically a good practice, I think
<jdcasey> so it'd go from foo-1 to foo-2 artfactID?
<brett> but we can't rely on it. People break stuff between 1.1 and 1=
<trygvis> that's how they do it
<brett> though that was prettty ugly when you showed it before :)
<brett> ok, noting the retrofitting as an issue
<brett> actually, the dependency management would just be used to ove=
rride the over-enthusiastic pom
<adc> Get an error running the m2 tutorial
<brett> adc: I forgot to update it I think
<brett> with the last release
<brett> which was dumb of me
<brett> will take a look later
<brett> ok, trygvis has a good point
<brett> I think the packagers take the approach of "one version =
to rule them all"
<trygvis> they do
<brett> and if they don't play together, its a different package
<brett> but that has come with its own set of problems, too
<jason> yes, that is one thing etrade specifically didn't like about =
<jason> and why they didn't use it
<brett> right, we don't have to do that. we can have every version kn=
own to man around for use
<jdcasey> yeah, like try tracking an artifact across all the differen=
<brett> trygvis: your point?
<trygvis> just showing how they specify deps
<brett> I'd like to use >=3D syntax, but not in XML :)
<trygvis> kaffe (>=3D 2:1.1.4.PRECVS8-2)
<jdcasey> I think we need to get the version-range syntax nailed down=
, but I think this stuff will work.
<trygvis> they also have some issues with they version stating althou=
gh they're making it more flexible
<brett> the | is interesting, kind of like specification deps. I've r=
uled them out for this release though
<jdcasey> (having a syntax makes it a little easier to explore it all=
<trygvis> I like what brett proposed, yay mathematical synatx
<brett> as long as I get it right
<jdcasey> and we all learn it ;)
<jdcasey> I didn't have set theory in college
<brett> I did a bit of everything. A lot of pure maths is good for co=
mputer science, if useless in the real world :)
<jdcasey> yeah, been reading that in spare moments, trygvis ;)
<trygvis> I like the power set character
<trygvis> looks like a super fancy P
<jason> brett: that's not true. pure math is very handy
<jason> you wait till we play graph visualization :-)
<brett> yeah, it was tongue in cheek, mostly
<brett> ok, let me summarise where we are at...
<brett> I think we've agreed on everything in that doc
<brett> but with the following issues
<brett> - strategies are out for release 1, adding complexity to a pr=
oblem not fully tested yet. likewise spec deps
<brett> we'll just use the range
<adc> brett, ping me when you're ready to help me out, tks.
<brett> adc: will do
<brett> - we need to get to the bottom of what gets resolved, how, an=
d when it is used
<brett> did we all agree that resolving versions in the transitive po=
ms outright would limit the ability to do the right selection later?
<brett> I'm thinking we operate in two modes, basically. use resolved=
versions (historical builds) and used everything
<brett> ok, I think we can keep this simple
<brett> I think we can ensure the reproducible build is self containe=
<brett> that eliminates all of the guess work
<brett> I don't think that needs to go into the repo at all, only SCM=
<jdcasey> might also put it in the jar itself, for reference
<brett> this is essentially what ben said at the start
<jdcasey> sort of a "here's how I was built" thing
<brett> jdcasey: right. We actually agreed to do that already last we=
ek, didn't get around to it. This takes it a step further.
<jason> so effecitvely a manifest for reproduction?
<brett> right. if you are using a source build, transitivity is turne=
d off, and all the versions are resolved
<brett> not sure about the parent
<jason> so that pom would enumerate the entire list of artifacts requ=
ired to build?
<brett> makes it immune from repo changes, changes in the app code, e=
<brett> jdcasey: good point :)
<brett> I can't see a use case for deploying fully populated poms to =
the repo - we went through that the other day and we agreed it was a bad id=
<jason> really with ranges there is no other way
<brett> and meanwhile, normal source code builds will always use the =
<brett> its nicely paralleled to the snapshots, so I like it
<jason> so for a source drop that POM could be placed in there so it =
would always work, what would we do with the SCM as far as the build being =
<brett> drop it in, tag, revert back and bump version.
<brett> no more commits than we have now
<brett> bigger diffs :)
<brett> I'm open to an alternate way of doing that though
<jason> so fully resolved pom gets checked in and revert back again
<brett> maybe a separate file
<brett> so, if we agree on that, outstanding issues are:
<jason> so tag with the full resolved POM and then the perform will b=
uild using that fully resolved POM?
<jason> that makes sense
<brett> ok, issues:
<brett> - how do we resolve the parent?
<brett> - how do we store the information? (in the pom, replacement p=
om, side by side file)
<brett> parent, I guess we also fold in.
<jason> fold in how?
<brett> I was thinking we needed to keep it dynamic because something=
s can change (eg the company URL), but a release is a release
<jason> there won't be any ranges there and we could always retrieve =
<brett> fold in, I mean write out the model at the end of assembly, a=
nd keep that
<brett> the parent has deps that would affect the final pom
<jason> so fully resolve parent and all
<jdcasey> brett: I'd be +1 for making it totally flat for historical =
purposes...nothing to lose that way...
<brett> yeah, I think so
<brett> ok, last decision in this marathon topic... where do we store=
<brett> I'm in favour of a separate file actually
<jdcasey> yeah, me too
<jdcasey> seems cleaner
<jason> you mean the fully resolved POM?
<jason> the full-monty-pom.xml
<brett> well, the profiles doc already called for profiles.xml
<jdcasey> nice side effect is to be able to diff releases of that fil=
e for differences between releases
<jason> so in a source drop or tag it's just pom.xml but where is thi=
s uberpom going?
<brett> next to it
<brett> if it exists, use it instead (unless overridden by switch?)
<jason> the same file?
<brett> not sure what you mean?
<jdcasey> no, we'd leave the original pom.xml untouched, except for p=
ossibly incrementing the version (?)
<jason> if i just checkout a tag and go "m2 install"
<jdcasey> this is instead of replace-tag-revert,isn't it?
<jason> so the tag will have the original pom.xml and to reproduce th=
e build the release-pom.xml would be used
<jdcasey> that's what I think
<brett> yep, added bonus is that it would remove the need for release=
<brett> I think
<jason> ok, but i think everyone will check out the tag and just &quo=
<jason> which won't use what we intend
<jason> ok, i'm confused
<jason> the pom.xml in a tag will be full resolved on or not?
<brett> I was thinking not, so it didn't have such a wild change hist=
<jason> that's fine, we could probably have the cli look for the rele=
ase pom so people don't inadvertently build with the wrong pom
<jason> they will naturally just check out and go "m2 install&qu=
<jason> or at least warn them there is a release pom there that can b=
<brett> do we want to keep the release pom there, or only have it on =
<jdcasey> I'd think only on the tags
<jdcasey> since it's only valid for that release
<jason> i think only in the tags and source balls
<jdcasey> that file will go stale as soon as new development starts
<brett> ok, so release process is:
<brett> - set release version in pom.xml, write out release-pom.xml
<brett> - tag
<brett> - delete release-pom.xml, bump version in pom.xml