This summer, Piotr Tabor and Peter Kolbus will create (as Google Summer of Code participants - with Jason van Zyl and Carlos Sanchez as mentors respectively) a software system that will allow graphs of Maven project information to be created, both interactively and during the build process of a project. This includes:
- APIs to define graph structures (graphs, nodes, edges, and metadata), and implementations for class and artifact diagrams
- APIs for producers and consumers of graph information (connectors and listeners), and implementations
- A graphical editor to browse graphs and save configuration (positioning, exclusions, etc.)
- A Maven plugin to automatically generate static class and artifact diagrams
- A tool to troubleshoot artifact dependency problems
The WORK IN PROGRESS code connected to this proposal is in the mojo sandbox: http://svn.codehaus.org/mojo/trunk/sandbox/maven-diagram-maker/
See http://docs.codehaus.org/display/MAVEN/Dependency+Graphing for previous work and discussions, to reuse as much as possible
Why "Maven Diagram Maker"?
There are no good, free tools to prepare diagrams - especially such that would automatically generate diagrams from the source code and project descriptors. On the other hand, good and up-to-date documentation is a very important part of software development process - particularly in the Open Source world, where documentation is critical to help many people work on projects. On the other hand, nobody likes preparing documents and diagrams. To make the matters worse, using standard one-time diagrammers helps only for a short period of time. In fast growing projects, the pictures become obsolete very quickly. They start to mislead. Therefore I think that adding possibility to generate actual up-to-date diagrams in the build process is a very important idea - and Maven is the perfect environment to do that.
Java developers can nowadays use very good text documenting tools, such a JavaDoc and Doxia, but the lack of methods to prepare and incorporate actual visual information in the documents is a very serious shortcoming of those tools.
Why "Dependency Troubleshooting"?
In larger software projects, it becomes difficult to ensure versions of dependencies are consistent throughout the component hierarchy. The dependency troubleshooter will provide the software developer with the information needed to find and fix dependency problems:
- Identify dependency exclusions. Is it excluded, and where did exclusion originate.
- Identify dependency version decisions. If maven picks another version, which one, and why?
- Identify dependency owner. If dependency is present in project, how did it get there? By current project, by transitive dependency, or by parent pom?
- Identify transitive dependency complexity. Multi module project is getting out of hand, what dependencies can i collapse safely into one dep?
- Identify relocated dependencies.
- Identify missing or bad dependencies.
- Filter unneeded information, and find specific artifacts in the tree
- Get more information about artifacts
- Answer "why was this artifact included?
- Answer "why it changed scope or version?".
Graphs of Java Classes
Graphs of Artifact Dependencies
Graph Connectors and Listeners
A Connector provides a source of information about the structure of a graph; a Listener receives this information and creates a representation for output. The connector/listener model is used because graph structure information can often only be learned incrementally; for example, Maven's artifact resolution process never provides the overall structure of all dependencies, it only fires events as each artifact is scanned for dependencies. The connector model also allows
Events fired by a Connector include:
The configuration of a connector will be described in XML format, via a dataSource tag. The dataSource describes the groupId and artifactId of the connector, as well as a container tag for the details of the configuration. The specific tags used in the configuration tag will be different for each connector, but common tag names will be used whenever appropriate.
<dataSource> <groupId>org.apache.maven.diagram.connectors</groupId> <artifactId>classes-diagram</artifactId> <configuration> <!-- details for a particular connector --> </configuration> </dataSource>
The following connectors will be implemented as part of this project:
- Class diagram - to visualize the inheritance and composition aspects of the class hierarchy. The information will be obtained by reflection.
- Dependencies diagram connector - to visualize the chosen project's subset of the dependencies (internal or external, up to a configureddepth or/and included in a given scope). It will be also possible to exclude some dependencies from the diagram.
- Dependency difference connector - this will take two dependency sources and merge them, setting properties to identify whether a node or edge is in the new, old, or both diagrams
Of course the mechanism of the connectors will be universal enough to make the addition of a new aspect of a project as simple as possible.
Class Diagram Connector
The class diagram connector will inspect the classes in a directory, *.jar or *.war file and allowing to limit them. The information will be obtained from the classes by reflection.
<dataSource> <groupId>org.apache.maven.diagram.connectors</groupId> <artifactId>classes-diagram</artifactId> <version>1.0</version> <configuration> <dependencies> <dependency> <groupId></groupId> <artifactId></artifactId> <classifier></classifier> <type></type> </dependency> ... </dependencies> <includes> <include>org\.codehaus\.ClassName</include> <include>org\.apache\.**</include> <include><!--regexp--></include> </includes> <excludes> <!-- keepEdges means: nodes: A,B,C; graph: A-B-C, result of "exclude B" with keepEdges: A-C. result of "exclude B' without keepEdges: A-C The first rule wins. (default: false) --> <exclude keepEdges="true/false">org\.apache\.maven.*<\exclude> </excludes> <nodes> <!--if a class contains x,getX, setX, it will be represented only as "<property> x" --> <compressJavaBeanProperties>true/false (default true)<compressJavaBeanProperties> <!-- if a superclass of a class contains (public or protected) methods - should the methods be visible in the inherited class --> <propagateInheritedMethods>true/false (default false)</propagateInheritedMethods> <!-- if a superclass of a class contains (public or protected) fields - should the fields be visible in the inherited class --> <propagateInheritedFields>true/false (default false)</propagateInheritedFields> </nodes> <edges> <inheritance/> <implemmenting/> <aggregating/> </edges> </configuration> </dataSource>
Providing data about project dependencies and allowing to limit them. In implementation it will be very similar to the "dependencies" task of maven-help-plugin
<dataSource> <groupId>org.apache.maven.diagram.connectors</groupId> <artifactId>dependencies-diagram</artifactId> <configuration> <scopes> <scope>compile<scope> <scope>test<scope> <scopes> <excludesTransitives> <!-- Do not expand transitive dependencies of the artifacts:--> <exclude> <artifactId></artifactId> <groupId></groupId> <type></type> <classifier></classifier> </exclude> </excludesTransitives> <configuration> <filtering> <includes> <include>.*<!-- default --> </includes> <excludes> <exclude>org\.bad_groupId:.*</exclude> </excludes> </filtering> </dataSource>
Dependency Diff Connector
The Graphical Editor
'The Graphical Editor will offer live (WYSIWYG) preparation of a graphical presentation of the project by setting such properties as:
- Type and subset of data, we want to present,
- What part of the data is to be presented,
- The method of presenting of each type of diagram nodes (that is, what attributes of the item will be displayed), and the general 'style' of the presentation. I would like to provide UML-like design.
- The general layout algorithm
- he positions of selected locked nodes - which we want to put in a fixed area. All other nodes will be positioned automatically.The schema of the diagram created by the editor will be saved to an XML file. Additionally the editor may be used as a graphical browser (explorer) of chosen aspects of the project. The editor will use the Prefuse library (http://prefuse.org).
Planned editor features
The Graphical Editor will help the user create and edit the visualization schema XML file. The file will contain two types of information:
- The <dataSource> section will describe which data will be presented, that is - how to acquire the information to present. The classes and attributes of connectors, which will extract the data from the project, will be given. No actual data will be stored in the XML file itself.
-The <view> section will describe how to present data. In details it
- Perspective (coordinates of the visible area in the picture) or the information that the whole diagram has to fit into the picture.
- Information about objects pinned (locked) to specified positions of the picture. This is necessary because the layout algorithms cannot cope with every situation - and in such a situation help from the user is needed. The objects are identified by business identifiers.
- Selection of information from single node to be presented
- Selection and parameters of the general layout strategy
"View" tag schema proposition (example):
<view> <layoutStrategy> <class>prefuse.action.layout.graph.ForceDirectedLayout<!--or other layout class--></class> <configuration> <duration>1000</duration> <!-- how long (how many iterations) the forces have to be simulated--> </configuration> </layoutStrategy> <labels> <label x="10" y="15">Label1</label> <label x="20" y="25">Label2</label> </labels> <lockedNodes> <lockedNode> <id>org.funny.package.TheBestClass</id> <x>25</x> <y>40</y> </lockedNode> <lockedNode> <id>org.funny.package.TheBestClass2</id> <x>45</x> <y>20</y> </lockedNode> </lockedNodes> <crop> <!--perspective (coordinates of the visible area in the picture) f omited whole diagram has to fit into the picture --> <x></x> <y></y> <width></width> <hight></hight> </crop> <shows> <show>ClassName</show> <show>PublicField</show> <show>PublicMethods</show> </shows> </view>
The editor will be using a Prefuse library (http://prefuse.org/). In particular, all strategies of automatic object positioning provided by the library will be available for the user. The only connection between Maven and the editor will be the data source component (the connector). Out of the two connectors I wish to create only one (the dependencies view) will be dependent on Maven, the other (class hierarchy view) will be Maven-independent. Thus the editor may be viewed as a universal tool to browse and explore all kinds of data through the Prefuse library and to prepare visualization template files. This means the editor could be used in a wide variety of cases, provided an appropriate connector is supplied.
The Maven Plug-in (maven-graph-plugin)
The Maven Plug-in will be using the XML file prepared by the Graphical Editor and the current state of project to prepare images in various graphic formats (JPG, TIFF, PNG and others
supported by the Sun JAI library (http://java.sun.com/javase/technologies/desktop/media/jai/). The plug-in will be able to prepare the HTML <map> tag for the picture too (to create an active area on web page containing the image). The resulting file will be ready to use in the next Maven phases by Doxia or other documenting tool.
The maven-graph-plugin will cope with changes in the project made after generating the schema XML file. This means that if an object from the XML file is no longer exists in the project, it won't be visible in the picture. It also means that if the object is new, it will be placed in the picture (in a place calculated by the general layout strategy set in the XML file). Additionally the user will be warned about any discrepancies between the expected and actual data and problems with finding space for a new object. The diagram generation will not modify the XML file, but will only try to interpret it for the current situation.
The Maven-graph-plugin will be a tool to join the visualization XML file created by the editor with current project data (or other information described in <dataSource> section) and as a result to collection of graphical files in chosen formats (provided by Sun JIMI library: http://java.sun.com/products/jimi). The result of the plug-in would be also a HTML <map> tag - to support active links on the generated image. Configuration of result file formats, sizes in pixels, file names and destinations directory will be provided in the plug-in configuration.
TBD: See http://docs.codehaus.org/display/MAVENUSER/Interactive+Dependency+Graphs for now.