Groovy 3 semantics and new MOP










Jochen "blackdrag" Theodorou



Last modification:


Last update by:


For quite long we are thinking about a new MOP for Groovy, without finding the right way  to do it better this time. It is difficult to find the right path between being user-friendly and efficient. Thus the new MOP will contain some features the old did not, but will also remove some old features for the sake of a more straight and more easy to understand MOP.


Removing default null argument

The default null argument is used for method calls that have one parameter, but the call is done without an argument for the parameter. Groovy will here use null, as long as the type of the parameter is not a primitive. In case of a primitive the call fails. This feature was mainly added in the early Groovy pre 1.0 years before Groovy supported default values for parameters. Another limitation of this logic is that it works only with single parameter methods. There have been multiple cases of confused users by this logic. Performance wise this logic doesn't cost much, and certainly not more than a call with a real argument. But since this is a confusing feature of limited use, it should be removed.

Removing automatic list expansion

A method call done with a list that finds no matching method for that list (a method with one parameter of type List, Collection, Object, etc), will cause a second method selection iteration. This time the list is "unpacked" and all elements of the list are taken as if the method call had been done with the elements rather than the list. Groovy also supports spreading of lists by an syntax element, making this automatic feature not needed. In fact this can be quite surprising for users and is a problem for performance. A spread version might be still not good in performance, but at least the user will have to use an extra symbol and thus have the visual indicator. As of why this feature was originally added is unclear. Looking at user code you will find barely intended usages of this. Thus it should be removed.

Changing Safe Navigation to stop evaluation

Currently an expression like a?.b.c will fail if a is null. It will not evaluate b, but it will try to evaluate c on null. This defies the intend of safe navigation to avoid a NullPointerException. Thus this should be changed to stop the evaluation of the path expression.

User Wish List

Some of the leading design ideas of this MOP

The plan is to orientate us a lot on the open call site caching the JVM provides with invokedynamic. For this to work all parts of the MOP should no longer be seen as places that do invoke something, but as places that return something, that then will be invoked. An invokeMethod then will for example return instead an object that acts as a kind of handler, which can be invoked. Groovy will then store it and avoid the reselection unless you invalidate it. In the old MOP such caching often did not happen once you interact using meta programming. The tasks to be solved in this are to provide an "extension point" for intercepting methods and to react to missing methods, as well as being able to invalidate a cached version and/or to make an uncached version possible.

MOP2 outline

in meta class of x in pseudo code:

  cachable = getMethodInterceptor(...)
  if (cachable==null) cachable = getFittingMethod(...)
  if (cachable==null) getMethodMissingHandler(...)
  if (cachable==null) throw new MethodMissingException(...)

as an explanation to the methods:

Context Meta Class

For the MOP2 Groovy will use a system of meta classes with a context element and an always existing default. At each call site only one such view will be valid and it will be constant. Those views can be used to defined "sealed" meta class, which won't get influenced by outside meta classes or to allow for example calling private methods and not allowing them in other cases. This makes 3 possible meta class variations currently:

  1. Default meta class
    Behaves pretty much like what Java allows in regards of method calls (same class private methods are available) and all meta programming changes are visible
  2. Sealed meta class
    Like the default meta class, but meta programming changes from outside the class are not visible. This can be especially useful to library writers.
  3. Test meta class (un)selaed
    Like the two before but with the addition that private members will always be available.

Getting a Meta Class

Because of the concept of context a class does not have one direct meta class that can be generated without its context. The call site defines the place of the context. How the context itself is defined is a TODO. As an implementation strategy it is possible to for example use ClassValue to store a table with the context being a key. The key would probably have to be available as static information, or as easily computable information. Since the resulting meta class could be stored later in the call site object context changes are to be avoided, since it implies the invalidation of the call sites using that context.

General dispatch rule for methods (and properties)

To define the dispatch rules correctly we need to define some terms first:
Static Sender Class (SSC):  This is the static information about the class a call is made from. If there is for example a class A and a class B extends A, and a call in a method in A, then even if your instance is actually a B the SSC will still be A.
Inheritance based Multimethods (short multimethods from now on): Given a class A and a class B extends A, a call made from within A may see a method defined on B as long as the method is visible (not private). Groovy defines a special  exception to this though. If the method call from within A is calling a method of the name m, then a m from B can only be visible if there is no private m defined in A.

Given those two definitions a method call in A will select the set of method to decide from based on this:
A call m() with the SSC A and done on an instance of B (extends A) will be using the methods defined in A, if A has a private m, otherwise the call is done using B.

Calls to Super:
A call to super in B extends A will have the SSC B, but for the method selection process the super class of SSC (super(SSC)) will be used. In super calls mutimethods are not visible. Thus we can directly use the meta class super(SSC), but we will dispatch only on the public methods of that meta class.

Module Extension Methods Shadowing Rules

Module Extensions Methods are in the old and new MOP defined by the DefaultGroovyMethods related classes and module extension, like groovy-swing. In the definition here we will use the terms of from "inside" and from "outside" to define a callsite, that lies in the same class as the target method (inside) or not (outside). The general rules are:

Subclasses of the class the module extension method has been applied to have these extended rules:

Open Blocks are not seen as separate classes.

Property Discovery

Currently MetaClass discovers properties based on the Java Beans conventions. It also allows pseudo properties matching a convention on java.beans.EventSetDescriptor. This allows the following trick in SwingBuilder for example

button(actionPerformed: { println it })

The pseudo property actionPerformed is inferred from the single method exposed by ActionListener, a type of listener that can be registered on a JButton. The code responsible for discovering these properties is buried in MetaClassImpl and is not accessible to the outside. It would be great if this mechanism be made pluggable.

The Realm concept

In MOP2 a Realm is a tree like structure containing the meta classes. There is a root realm, used as default, but there can be any number of lower realms.  A meta class change is visible in a realm, if the change is done to the meta class in the same realm or to a meta class in a higher realm. Script execution engines are able to set a realm for example to prevent them changing meta classes they should not change. This can be used for unit tests to isolate meta class changes done during the tests as well. A library can have its own realm (defined through an annotation) to prevent other classes to leak their changes into that library, while the library can still use a higher realm to make changes more public visible, if the realm allows that. Realms can have a setting that prevents code executed from there to make changes to higher realms. Calling a method is always done using the meta classes from the current realm, even if the called class then calls other classes using its own realm. A realm is thus not thread local structure, it is more of a lexical scope. A realm can also use a different meta class flavor, to for example allow access to private methods and fields.

Mailing-list discussions

JIRA issues


Work Items

This part is to guide the implementors with the course of action and planning of the subtasks.

Breaking changes trace

groovy.lang.MetaObjectProtocol (currently in groovy.mop.MetaObjectProtocol):

groovy.lang.MetaMethod is split into a public interface groovy.mop.MetaMethod and an internal default implementation groovy.mop.internal.DefaultMetaMethod.

Differences to groovy.mop.internal.DefaultMetaMethod:

groovy.lang.MetaProperty is split into a public interface groovy.mop.MetaProperty and an internal default implementation groovy.mop.internal.DefaultMetaProperty.

Differences to groovy.mop.internal.DefaultMetaProperty:


Useful links

The work branch for this: