Work In Progress Please post your comments on the Groovy developer list!

Introduction

This text here is based on ideas from Jochen Theodorou (see chitchat-with-groovy-compiler and ast-macros-and-mixins)

Right now, there is no good macro preprocessor for Java. Annotations somehow come close but they don't really fit the bill. In the Java VM, annotations are a runtime feature. You cannot enhance an existing class; only create new ones. This means that you cannot add setters and getters to a class.

When you look at OR Mappers, they even do this at runtime, so there is no way to see what is actually happening: When the error happens, the code which is executed can be completely different than what you see in the source. Even decompiling the class file will not help anymore because the information isn't there, yet. It's only added when the classloader reads the file.

So from a certain point of view, Sun's solution is the worst of all worlds: Your code is changed at a point in time when you can't see it anymore and you cannot move the modification in the compile cycle because the API simple doesn't allow it.

The Goal

To know where you want to go, you must have a goal. The goal here is to reduce the amount of code to write for a certain feature. Specifically, the idea is to be able to move common, repeated code into a single place and be able to reference it easily.

The code must be more flexible than a method call and easier to manage than cut&paste.

Example 1: Bound Properties

A bound property in a Java bean is a field which sends notifications to listeners when it is changed. This means it is made up of these parts:

• There is a list of listeners who are interested in changes
• The field itself
• Methods to add and remove listeners
• Setter code which changes the value and notifies the listeners if the value has changed

Example 2: Merge Java and SQL

OR Mappers will only get you so far. While they will solve many or all problems, they also introduce new ones:

• You have to learn to use the mapper
• You must still understand how a database works
• The mapper will try to connect the Java semantics to the Database semantics. This is not always straightforward. For example, a table might define a non-unique primary key. There is no way to map this to a Java map where primary keys must be unique or you will overwrite objects.
• When a mapper doesn't support a specific corner case, you're in trouble. Mappers are often quite monolithic and they shove their tendrils in many places. Usually, you can't split it apart, injecting your own code in certain places.

So what do we expect from AST Macros in this case?

• They shouldn't get in the way of the developer. If she choses to use an OR Mapper, she should be free to do so.
• It should help to manage all the cases which the OR Mapper doesn't handle well.
• It should give aid to interface the Groovy code with the OR Mapper.
• It should allow to write code which directly interfaces the DB (for example, when you have to execute some special SQL which the OR Mapper simply can't do).

Some simple examples:

• counting rows
• loading objects from a DB

Examples

Before we look at solutions, let's look at what the code ought to do in the end.

Example 1: Bound Properties

class A {
    @BoundProperty int x
}

should become

import javax.beans.*;

class A {
    // the following is added only once per class
    PropertyChangeSupport propertyChangeSupport

    void addPropertyChangeListener(PropertyChangeListner listner) {
        propertyChangeSupport.addPropertyChangeListner(listener)
    }

    void addPropertyChangeListener(String property, PropertyChangeListner listner) {
        propertyChangeSupport.addPropertyChangeListner(property, listener)
    }

    void removePropertyChangeListener(PropertyChangeListner listner) {
        propertyChangeSupport.removePropertyChangeListner(listener)
    }

    void removePropertyChangeListener(String property, PropertyChangeListner listner) {
        propertyChangeSupport.removePropertyChangeListner(property, listener)
    }

    PropertyChangeSupport[] getPropertyChangeListeners() {
        return propertyChangeSupport.getPropertyChangeListeners
    }

    // the following is added per each annotated proeprty
    private int x

    void setX (int x) {
    	propertyChangeSupport.firePropertyChanged('x', this.x, this.x = x)
    }

    int getX() {
        return x;
    }
}

specifically:

• When using code completion, the additional fields and methods should be visible.
• The added code should be lean and fast
• It should not get in the way of existing user code, for example, it should be possible to define a custom setter which gets wrapped by the code above
• If a custom setter exists, it should be possible to invoke it before the comparison (so you can define setters which convert the value before assigning it)

This leads to a couple of demands which an AST Macro Processor (AMP) must met:

• It must be able to see all fields and methods, no matter if the user supplied them or an AMP added them.
• It must be able to add new methods and fields and static code to an existing class and presumably also to classes created by an AMP
• It must be able to reorder source code or at least AST Nodes.

In a perfect world, an AMP should be able to modify the code on a source level and pass it back to an IDE, for example, so that I can see (and debug) what is actually compiled (instead of only seeing the Annotation).

Example 2: Merge Java and SQL

SQL enhanced code is pretty similar to bound properties but more code is generated. The first step is to define the class which maps a database table to a Java object:

@Entity
class Foo
{
    @Id
    @Column (type:java.sql.Types.INTEGER)
    int id
    @Column (type:java.sql.Types.CHAR, size:20)
    String name
}

def columns = [Foo.SQL.value, Foo.SQL.name]
def cond = Sql.WHERE () { Foo.SQL.id >= 5 && Foo.SQL.name != null }
def list = Sql.SELECT (columns, table:Foo.SQL.TABLE, where:cond, class:Foo.class)

def list = []
def _sql = "SELECT id, name FROM foo WHERE id >= ? AND name IS NOT NULL"
_sql = Sql.eachRow (_sql, [5]) {
   Foo o = new Foo ()
   o.id = it[0]
   o.name = it[1]
   list << o
}

def foo = Foo.SQL.load (5)

Open Issues

Java 1.4/5

Groovy 1.x must run on Java 1.4. We must decide what to do with non-macro annotations, whether we want to support a switch to generate Java 5 classfiles (so Groovy can generate code for third party APTs like Hibernate)

It seems that it is possible to write annotations into Java 1.4 classfiles (see Commons Attributes). But the questions is: Is this futile? There are only a few tools which support annotations and Java 1.4.

In this light, it makes more sense to add a switch to allow Groovy to write Java 5 classfiles, so users stuck to 1.4 can still use it and Java 5 users can upgrade when they want to.

Expand or Pass On

The compiler needs a way to decide what to do with an official Java 5 annotation like javax.persistence.Entity which is defined in EJB3: Expand it as a macro or pass it on into the class file so a third party library/tool can process it later.

Here, the user might want to decide differently per class (i.e. handle most of these cases with Hibernate and some corner cases with her own AST macro).

For Groovy-specific macros, the solution is to add a marker interface to the macro annotation.

Options

• Add a config file to the compiler
• Users have to use a different annotation which implements both Groovy's marker interface and javax.persistence.Entity

Links

• Java 6 Annotations
• Getting Started With APT (Java 6)
• Annotation processing in Core Java (7th Edition)
• OOPSLA paper on which Annotations are based