AspectWerkz performance

The need for performance metrics and comparison

Since we released AW 1.0, and more generally for every release of any AOP / interceptor framework (AspectJ, JBoss AOP, Spring AOP, cglib etc), a question is always raised: "what is the performance cost of such an approach ?", "how much do I loose per method invocation when an advice / interceptor is applied ?".

This is indeed an issue that needs to be carefully addressed, and that in fact has affected the design of every mature enough framework.

We are probably all scared by the cost of the java.lang.reflect despite its relative power, and usually, even before starting to evaluate semantics robustness and ease of use in general - we start doing some hello world bench.

We have started AWbench for that purpose. Offering a single place to measure the relative performance of AOP / interceptor framework, and even measure it by your own.

More than providing performance comparison, AWbench is a good place to figure out the semantics differences and ease of use of each framework. A simple "line of count" metrics could be provided as well (source + external file like xml + Ant glue).

Current results

This table provides the figures from a bench in "nanosecond per advised method invocation". A non advised method invocation is roughly about 5 ns/iteration.
Results were obtained with 2 millions iterations.

In this table, the two lines in bold are very important. In a real world application, it is likely that the before or around advice will really do something and for such needs to access runtime information like method parameter and target instance. It is likeley as well that the join point is affected by more than one advice.

On the opposite it is very unlikely to have just a before advice that does nothing, but it gives us a good evaluation on the most minimal overhead we can expect.

• before, args() target()

  10

  576

  10

  215

  591

  2463

  135

  -

  210

  -

  *

• around x 2, args() target()	85	641	45	290	681	4647	150	461	461	-	*
  before	10	476	15	130	526	320	75	-	40	10
  before, static info access	30	480	20	130	520	305	75	-	40	-
  before, rtti info access	50	506	50	135	526	2373	70	-	35	-
  after returning	10	475	10	165	535	325	80	-	40	15
  after throwing	3360	5468	2984	5187	-	6604	7991	-	-	3440
  before + after	20	480	15	245	671	340	85	-	40	20
  before, args() primitives	10	546	10	260	580	370	135	-	200	-
  before, args() objects	5	540	15	185	561	350	105	-	195	-
  around	60	470	10	125	460	310	75	70	70	85
  around, static info access	90	480	25	125	481	330	75	70	70	-
  around, rtti info access	85	506	50	140	495	2433	85	70	75	-

This table provides the figures from the same bench where for each category AspectWerkz 2.0.RC1 is the reference.
The first line illustrates that for the most simple before advice, AspectWerkz is 13 times faster than JBoss AOP 1.0.

• before, args() target()

  1 x

  57.6 x

  1 x

  21.5 x

  59.1 x

  246.3 x

  13.5 x

  -

  21 x

  -

  *

• around x 2, args() target()	1 x	7.5 x	0.5 x	3.4 x	8 x	54.6 x	1.7 x	5.4 x	5.4 x	-	*
  before	1 x	47.6 x	1.5 x	13 x	52.6 x	32 x	7.5 x	-	4 x	1 x
  before, static info access	1 x	16 x	0.6 x	4.3 x	17.3 x	10.1 x	2.5 x	-	1.3 x	-
  before, rtti info access	1 x	10.1 x	1 x	2.7 x	10.5 x	47.4 x	1.4 x	-	0.7 x	-
  after returning	1 x	47.5 x	1 x	16.5 x	53.5 x	32.5 x	8 x	-	4 x	1.5 x
  after throwing	1 x	1.6 x	0.8 x	1.5 x	-	1.9 x	2.3 x	-	-	1 x
  before + after	1 x	24 x	0.7 x	12.2 x	33.5 x	17 x	4.2 x	-	2 x	1 x
  before, args() primitives	1 x	54.6 x	1 x	26 x	58 x	37 x	13.5 x	-	20 x	-
  before, args() objects	1 x	108 x	3 x	37 x	112.2 x	70 x	21 x	-	39 x	-
  around	1 x	7.8 x	0.1 x	2 x	7.6 x	5.1 x	1.2 x	1.2 x	1.2 x	1.4 x
  around, static info access	1 x	5.3 x	0.2 x	1.3 x	5.3 x	3.6 x	0.8 x	0.7 x	0.8 x	-
  around, rtti info access	1 x	5.9 x	0.5 x	1.6 x	5.8 x	28.6 x	1 x	0.8 x	0.8 x	-

Bench were run on a Java HotSpot 1.4.2, Windows 2000 SP4, Pentium M 1.6 GHz, 1 Go RAM.

Introducing AWbench

Summary

AWbench is a micro benchmark suite, which aims at staying simple. The test application is very simple, and AWbench is mainly the glue around the test application that applies one or more very simple advice / interceptor of the framework of your choice.

AWbench comes with an Ant script that allows you to run it on you own box, and provide some improvement if you know some for a particular framework.

What is the scope for the benchmark?

So far, AWbench includes method execution pointcuts, since call side pointcuts are not supported by proxy based framework (Spring AOP, cglib, dynaop etc).

The awbench.method.Execution class is the test application, and contains one method per construct to bench. An important fact is that bytecode based AOP may provide much better performance for before advice and after advice, as well as much better performance when it comes to accessing contextual information.
Indeed, proxy based frameworks are very likely to use reflection to give the user access to intercepted method parameters at runtime from within an advice, while bytecode based AOP may use more advanced constructs to provide access at the speed of a statically compiled access.

The current scope is thus:

For method execution pointcut

By accessing contextual information we means:

• accessing a method parameter using its real type (i.e. boxing and unboxing might be needed)
• accessing a the advised instance using its real type (i.e. casting might be needed)

A pseudo code block is thus likely to be:

Which AOP and Proxy frameworks are benched?

The following are included in AWbench:

Bytecode based frameworks

Proxy based frameworks

Moreover, AWbench includes AspectWerkz Extensible Aspect Container that allow to run any Aspect / Interceptor framework within the AspectWerkz 2.x runtime:

AWbench is extensible. Refer to Contributing to add a framework.

How to contribute? (general rules)

Full blown AOP

AspectWerkz

AspectJ

JBoss AOP

Proxy based AOP, interceptor frameworks

Spring AOP

dynaop

cglib proxy

AspectWerkz extensible AOP container

Spring AOP within the AspectWerkz extensible AOP container

What's next ?

Running awbench by your own

Contributing