Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 5.3

Tweaking the compiler configuration

Whether you are using groovyc to compile classes or a GroovyShell, for example, to execute scripts, under the hood, a compiler configuration is used. This configuration holds information like the source encoding or the classpath but it can also be used to perform more operations like adding imports by default, applying AST transformations transparently or disabling global AST transformations.

Compilation customizers

Introduction

Before Groovy 1.8.0, doing tasks like adding imports transparently (for DSLs) was a bit complicated. It involved writing a custom GroovyClassLoader and lots of trickery. The goal of compilation customizers is to make those common tasks easy to implement. For that, the CompilerConfiguration class is the entry point. The general schema will always be based on the following code:

Code Block
import org.codehaus.groovy.control.CompilerConfiguration
// create a configuration
def config = new CompilerConfiguration()
// tweak the configuration
config.addCompilationCustomizers(...)
// run your script
def shell = new GroovyShell(config)
shell.evaluate(script)

Compilation customizers must extend the org.codehaus.groovy.control.customizers.CompilationCustomizer class. A customizer works:

  • on a specific compilation phase
  • on every class node being compiled

You can implement your own compilation customizer but Groovy includes some of the most common operations.

The import customizer

Using this compilation customizer, your code will have imports added transparently. This is in particular useful for scripts implementing a DSL where you want to avoid users from having to write imports. The import customizer will let you add all the variants of imports the Groovy language allows, that is:

  • class imports, optionally aliased
  • star imports
  • static imports, optionally aliased
  • static star imports
Code Block
import org.codehaus.groovy.control.customizers.ImportCustomizer
def icz = new ImportCustomizer()
// "normal" import
icz.addImports('java.util.concurrent.atomic.AtomicInteger', 'java.util.concurrent.ConcurrentHashMap')
// "aliases" import
icz.addImport('CHM', 'java.util.concurrent.ConcurrentHashMap')
// "static" import
icz.addStaticImport('java.lang.Math', 'PI') // import static java.lang.Math.Pi
// "aliased static" import
ica.addStaticImport('pi', 'java.lang.Math', 'PI') // import static java.lang.Math.PI as pi
// "star" import
icz.addStarImport 'java.util.concurrent' // import java.util.concurrent.*
// "static star" import
icz.addStaticStar 'java.lang.Math' // import static java.lang.Math.*

The AST transformation customizer

The AST transformation customizer is meant to apply AST transformations transparently. Unlike global AST transformations that apply on every class beeing compiled as long as the transform is found on classpath (which has drawbacks like increasing the compilation time or side effects due to transformations applied where they should not), the customizer will allow you to selectively apply a transform only for specific scripts or classes.

As an example, let's say you want to be able to use @Log in a script. The problem is that @Log is normally applied on a class node and a script, by definition, doesn't require one. But implementation wise, scripts are classes, it's just that you cannot annotate this implicit class node with @Log. Using the AST customizer, you have a workaround to do it:

Code Block
import org.codehaus.groovy.control.customizers.ASTTransformationCustomizer
import groovy.util.logging.Log
def acz = new ASTTransformationCustomizer(Log)
configuration.addCompilationCustomizers(acz)

That's all! Internally, the @Log AST transformation is applied to every class node in the compilation unit. This means that it will be applied to the script, but also to classes defined within the script.

If the AST transformation that you are using accepts parameters, you can use parameters in the constructor too:

Code Block
def acz = new ASTTransformationCustomizer(Log, value: 'LOGGER') // use name 'LOGGER' instead of the default 'log'

As the AST transformation customizers works with objects instead of AST nodes, not all values can be converted to AST transformation parameters. For example, primitive types are converted to ConstantExpression (that is 'LOGGER' is converted to new ConstantExpression('LOGGER'), but if your AST transformation takes a closure as an argument, then you have to give it a ClosureExpression, like in the following example:

Code Block
final expression = new AstBuilder().buildFromCode(CompilePhase.CONVERSION) {-> true }.expression[0]
customizer = new ASTTransformationCustomizer(ConditionalInterrupt, value:expression, thrown:SecurityException)
configuration.addCompilationCustomizers(customizer)
def shell = new GroovyShell(configuration)
shouldFail(SecurityException) {
    shell.evaluate("""
                // equivalent to adding @ConditionalInterrupt(value={true}, thrown: SecurityException)
                class MyClass {
                    void doIt() { }
                }
                new MyClass().doIt()
            """)
}

The SecureASTCustomizer

This customizer will allow the developer of a DSL to restrict the grammar of the language, to prevent users from using some constructs, for example. It is only "secure" in that sense only and it is very important to understand that it does not replace a security manager. The only reason for it to exist is to limit the expressiveness of the language. This customizer only works at the AST (abstract syntax tree) level, not at runtime! It can be strange at first glance, but it makes much more sense if you think of Groovy as a platform to build DSLs. You may not want a user to have a complete language at hand. In the example below, we will demonstrate it using an example of language that only allows arithmetic operations, but this customizer allows you to:

  • allow/disallow creation of closures
  • allow/disallow imports
  • allow/disallow package definition
  • allow/disallow definition of methods
  • restrict the receivers of method calls
  • restrict the kind of AST expressions a user can use
  • restrict the tokens (grammar-wise) a user can use
  • restrict the types of the constants that can be used in code

For all those features, the secure AST customizer works using either a whitelist (list of elements that are allowed) or a blacklist (list of elements that are disallowed). For each type of feature (imports, tokens, ...) you have the choice to use either a whitelist or a blacklist, but you can mix whitelists and blacklists for distinct features. In general, you will choose whitelists (disallow all, allow selected).

Code Block
org.codehaus.groovy.control.customizers.SecureASTCustomizer

def scz = new SecureASTCustomizer()
scz.with {
    closuresAllowed = false // user will not be able to write closures
    methodDefinitionAllowed = false // user will not be able to define methods
    importsWhitelist = [] // empty whitelist means imports are disallowed
    staticImportsWhitelist = [] // same for static imports
    staticStarImportsWhitelist = ['java.lang.Math'] // only java.lang.Math is allowed
    // the list of tokens the user can find
	// constants are defined in org.codehaus.groovy.syntax.Types
    tokensWhitelist = [
            PLUS,
            MINUS,
            MULTIPLY,
            DIVIDE,
            MOD,
            POWER,
            PLUS_PLUS,
            MINUS_MINUS,
            COMPARE_EQUAL,
            COMPARE_NOT_EQUAL,
            COMPARE_LESS_THAN,
            COMPARE_LESS_THAN_EQUAL,
            COMPARE_GREATER_THAN,
            COMPARE_GREATER_THAN_EQUAL,
    ].asImmutable()
	// limit the types of constants that a user can define to number types only
    constantTypesClassesWhiteList = [
            Integer,
            Float,
            Long,
            Double,
            BigDecimal,
            Integer.TYPE,
            Long.TYPE,
            Float.TYPE,
            Double.TYPE
    ].asImmutable()
    // method calls are only allowed if the receiver is of one of those types
    // be careful, it's not a runtime type!
    receiversClassesWhiteList = [
            Math,
            Integer,
            Float,
            Double,
            Long,
            BigDecimal
    ].asImmutable()
}

If what the secure AST customizer provides out of the box isn't enough for your needs, before creating your own compilation customizer, you might be interested in the expression and statement checkers that the AST customizer supports. Basically, it allows you to add custom checks on the AST tree, on expressions (expression checkers) or statements (statement checkers). For this, you must implement org.codehaus.groovy.control.customizers.SecureASTCustomizer.StatementChecker or org.codehaus.groovy.control.customizers.SecureASTCustomizer.ExpressionChecker.

Those interfaces define a single method called isAuthorized, returning a boolean, and taking a Statement (or Expression) as a parameter. It allows you to perform complex logic over expressions or statements to tell if a user is allowed to do it or not. As an example, let's think of a DSL for which you want to make sure that users only call methods for which the name is in lowercase:

Code Block
scz.addExpressionChecker { expr ->
   if (expr instanceof MethodCallExpression) {
      return expr.methodAsString?.toLowerCase() == expr.methodAsString
   }
   true
} as ExpressionChecker

Here, we say that if the expression is a method call expression, then we can check the name and return true only if it's all lowercase. Otherwise, the expression is allowed.

The SourceAwareCustomizer

This customizer, available since Groovy 2.1.0 only, is a bit special in the sense that it may be used as a filter for other customizers. The filter, in that case, is the org.codehaus.groovy.control.SourceUnit. For this, the source aware customizer takes another customizer as a delegate, and it will apply customization of that delegate only and only if predicates on the source unit match.

SourceUnit gives you access to interesting things, in particular the file being compiled (if compiling from a file, of course), which gives you the potential to perform operation based on the file name, for example. Here is how you would create a source aware customizer:

Code Block
import org.codehaus.groovy.control.customizers.SourceAwareCustomizer
import org.codehaus.groovy.control.customizers.ImportCustomizer

def delegate = new ImportCustomizer()
def sac = new SourceAwareCustomizer(delegate)

Then you can use predicates on the source aware customizer:

Code Block
// the customizer will only be applied to classes ending with 'Bean'
sac.baseNameValidator = { baseName ->
   baseName.endsWith 'Bean'
}

// the customizer will only be applied to files which extension is '.spec'
sac.extensionValidator = { ext -> ext == 'spec' }

// source unit validation
sac.sourceUnitValidator = { sourceUnit -> ... }

The customization builder

If you are using compilation customizers in Groovy code (like the examples above) and you are using Groovy 2.1+, then you can use an alternative syntax to customize the compilation. A builder org.codehaus.groovy.control.customizers.builder.CompilerCustomizationBuilder is available. Creating a customizer has never been so easy!

Code Block
import org.codehaus.groovy.control.CompilerConfiguration
import static org.codehaus.groovy.control.customizers.builder.CompilerCustomizationBuilder.withConfig

def conf = new CompilerConfiguration()
withConfig(conf) {
   ...
}

The code sample above shows how to use the builder. A static method, withConfig, takes a closure corresponding to the builder code, and automatically registers compilation customizers to the configuration. You can use:

Imports

Code Block
withConfig(configuration) {
   imports { // imports customizer
	  normal 'my.package.MyClass' // a normal import
      alias 'AI', 'java.util.concurrent.atomic.AtomicInteger' // an aliased import
      star 'java.util.concurrent' // star imports
      staticMember 'java.lang.Math', 'PI' // static import
      staticMember 'pi', 'java.lang.Math', 'PI' // aliased static import
   }
}

For the AST customizer

Code Block
// apply @Log transparently
withConfig(conf) {
   ast(Log)
}

// apply @Log, with a different logger name
withConfig(conf) {
   ast(Log, value: 'LOGGER')
}

Secure ast

Code Block
withConfig(conf) {
   secureAst {
       closuresAllowed = false
       methodDefinitionAllowed = false
   }
}

Source based

Code Block
// apply CompileStatic AST annotation on .sgroovy files
withConfig(configuration){
    source(extension: 'sgroovy') {
        ast(CompileStatic)
    }
}

// apply CompileStatic AST annotation on .sgroovy or .sg files
withConfig(configuration){
    source(extensions: ['sgroovy','sg']) {
        ast(CompileStatic)
    }
}

// apply CompileStatic AST annotation on .sgroovy or .sg files
withConfig(configuration) {
    source(extensionValidator: { it.name in ['sgroovy','sg']}) {
        ast(CompileStatic)
    }
}

// apply CompileStatic AST annotation on files whose name is 'foo'
withConfig(configuration) {
    source(basename: 'foo') {
        ast(CompileStatic)
    }
}

// apply CompileStatic AST annotation on files whose name is 'foo' or 'bar'
withConfig(configuration) {
    source(basenames: ['foo', 'bar']) {
        ast(CompileStatic)
    }
}

// apply CompileStatic AST annotation on files whose name is 'foo' or 'bar'
withConfig(configuration) {
    source(basenameValidator: { it in ['foo', 'bar'] }) {
        ast(CompileStatic)
    }
}

// apply CompileStatic AST annotation on files that do not contain a class named 'Baz'
withConfig(configuration) {
    source(unitValidator: { unit -> !unit.AST.classes.any { it.name == 'Baz' } }) {
        ast(CompileStatic)
    }
}

Inlining a customizer

Inlined customizer allows you to write a compilation customizer directly, without having to create a dedicated class for it.

Code Block
withConfig(configuration) {
    inline(phase:'CONVERSION') { source, context, classNode ->
        println "visiting $classNode"
    }
}

Multiple customizers

Of course, the builder allows you to define multiple customizers at once:

Code Block
withConfig(configuration) {
   ast(ToString)
   ast(EqualsAndHashCode)
}

Configuring compilation using groovyc or the ant task

For now, we've shown you how to customize compilation using a CompilationConfiguration access, but this is only possible if you embed Groovy and that you create your own instances of CompilerConfiguration (be it with GroovyShellGroovyScriptEngine, ...). But if you want it to be applied on the classes you compile (with groovycant or gradle, for example), until Groovy 2.1.0, there was no way to do that.

Since Groovy 2.1.0 (and Groovy 2.1.1 for the groovy Ant task), it is possible to use a compilation flag named configscript that takes a groovy configration script as a parameter. This script gives you access to the CompilerConfiguration instance before the files are compiled (exposed as a variable named configuration), so that you can tweak it. It also transparently integrates the compiler configuration builder above.

Static compilation by default

Since static compilation has been released, many people asked for it to be enabled by default. For various reasons, including the fact that we think you should only limit static compilation to pieces of code where you have performance problems, we never included such a feature. Other people asked for default imports too. Since we didn't want to add lots of flags for each and every magic that Groovy can do, we decided to go for a configuration script. This means that having static compilation by default is just a matter of compiling classes using this configuration file. And the content is very easy:

Code Block
withConfig(configuration) {
   ast(groovy.transform.CompileStatic)
}

You don't need to add an import for the builder, it's automatically added. Then, compile your files using the following command line:

Code Block
groovyc -configscript src/conf/config.groovy src/main/groovy/MyClass.groovy

We strongly recommand you to separate configuration files from classes, hence the src/main and src/conf directories above.