Remoting for GPars

This is quick overview of Remoting for GPars realized during Google Summer of Code 2014 by Rafal Slawik. More details can be found in the official documentation in section on Remoting. The implementation has already been made part of GPars 1.3-SNAPSHOT is is available for immediate use.

Behind the scenes, the Netty library and the standard serialization mechanism were used as the transportation layer. Basically, you can use Dataflows with any data type, send custom messages to Actors and store custom states in Agents as long as these objects are seralizable.

The Dataflow structures that support remoting: DataflowVariable, DataflowBroadcast, DataflowQueue.

General use of our remoting implementation requires:

  • at host A: creating a context and publishing a structure (variable, queue, actor, etc.) under some name
  • at host B: creating a context and retrieval of a structure with that name

The context concept is useful for testing (one can have original instance and a remote proxy withing that same VM) or other purposes (eg. each thread has its own remote proxy). What's important, such a remote proxy has the same interface and therefore can be used like if it was a regular intstance. Let's see an example on how to use remoting for DataflowVariables:

  1. At host A:

    def remoteDataflows = RemoteDataflows.create() // creates context 
    remoteDataflows.startServer HOST PORT // starts server that waits for requests at HOST:PORT
    def variable = new DataflowVariable() // creates variable instance
    remoteDataflows.publish variable "my-first-variable" // registers it within the context under given name
  2. At host B:

    def remoteDataflows = RemoteDataflows.create() // creates context 
    def remoteVariablePromise = remoteDataflows.getVariable HOST, PORT, "my-first-variable" // retrieves promise of variable with given name
    def remoteVariable = remoteVariablePromise.get() // extracts remote proxy from promise

You can find more examples in samples package: groovyx.gpars.samples.remote.dataflow.*

Now, let's take a look at remoting for Actors and consider Ping-Pong example (groovyx.gpars.samples.remote.actor.pingpong). Let's start with creating an actor that responds to every message with "PONG". Such actor can look as follows:

def pongActor = Actors.actor { loop { react { println it reply "PONG" } } }

It waits in loop from messages and when one arrives it prints it and replies with "PONG". To be able to access this actor from remote host it has to be published:

def remoteActors = RemoteActors.create() // creates context
remoteActors.startServer HOST, PORT // starts server that waits for requests at HOST:PORT
remoteActors.publish pongActor, "pong" // registers pongActor within context under name "pong"

What's left is to retrieve the proxy object to that actor at remote host. It can be done as follows:

def remoteActors = RemoteActors.create() // creates context 
def pingActor = Actors.actor {
def remotePongActor = remoteActors.get HOST, PORT, "pong" get() // gets remote proxy to actor name "pong" at HOST:PORT
remotePongActor << "PING" // sends message to it
react {
println it // prints reply from remote actor
}
}

An extended example can be found in groovyx.gpars.samples.remote.actor.pingpong. More examples of remoting for Actors are available in groovyx.gpars.samples.remote.actor.*. An example of remoting for Agents is available in groovyx.gpars.samples.remote.agent.

In the future we can introduce multiplexing of connections between hosts (currenly each get opens new connection) and some discovery mechanism (to avoid using explicit HOST:PORT).


Here comes GPars 1.1

The GA release of GPars 1.1.0 has just been published and is ready for you to grab. It brings gradual improvements into dataflow as well as a few other domains. Some highlights:

  • LazyDataflowVariable added to allow for lazy asynchronous values
  • Timeout for Selects
  • Added a Promise-based API for value selection through the Select class
  • Enabled listening for bind errors on DataflowVariables
  • Minor API improvement affecting Promise and DataflowReadChannel
  • Protecting agent's blocking methods from being called from within commands
  • Updated to the latest 0.7.0 GA version of Multiverse
  • Migrated to Groovy 2.0
  • Used @CompileStatic where appropriate
  • A few bug fixes

You can download GPars 1.1.0 directly or grab it from the maven repo.

Have a lot of fun trying out GPars 1.1.0!

Arriving at 1.1.0

A first release candidate for GPars 1.1.0 has been made available. The final 1.1.0 GA should be expected in a few days. The 1.1.0 release is a gradual improvement of 1.0.0 with additions mostly in the Dataflow domain. Starting with 1.1 GPars requires Groovy 2.0 or higher. Check out the most noteworthy new capabilities:

Dataflow

  • LazyDataflowVariable added to allow for lazy asynchronous values
  • Timeout for Selects
  • Added a Promise-based API for value selection through the Select class
  • Enabled listening for bind errors on DataflowVariables
  • Minor API improvement affecting Promise and DataflowReadChannel

Agent

  • Protecting agent's blocking methods from being called from within commands

Stm

  • Updated to the latest 0.7.0 GA version of Multiverse

Other

  • Migrated to Groovy 2.0
  • Used @CompileStatic where appropriate

 

Get GPars 1.1.0-rc1, take it for a spin and please report all issues so we can fix them before GA.

 

GPars 1.0 arrived

I'm happy to announce that after four years of development GPars, the Groovy concurrency library, has just reached its 1.0 mark. A fresh and crispy GPars 1.0.0 is now ready for you to grab or download and use on your projects. Also, the up-coming Groovy releases will bundle GPars 1.0.

Compared to the previous release, 1.0 brings several performance enhancements, considerable API updates, polished documentation and numerous functionality improvements, mostly in the dataflow area. Please, check out the What's new section of the user guide for the details. Full release notes are also available.

I would like to use this opportunity to thank all the Groovy people, who have over time contributed in one way or another to the success of GPars. It is my honour to be part of such a helpful and encouraging community. In particular, I would like to thank my colleague GPars commiters, namely Paul King, Dierk Koenig, Alex Tkatchman and Russel Winder, who we've been consistently pushing the project forward and without whom it would hardly ever get this far. I also greatly appreciate the support we received from Guillaume Laforge, the Groovy supreme commander. Thank you all gentlemen!

Groovy concurrency times ahead!

 

Václav

 

We are almost there. The 1.0 release is just round the corner. To ensure that 1.0 meets your quality expectations we first prepared a release candidate to test the waters. To take GPars for a test ride, please download or grab it at the usual places, check out the release notes and let us know if something is missing.

 

Václav

Beta 3 is out

GPars-1.0-beta-3 has been made available for you to try out.
Apart from the usual doze of features and fixes, including speed-up for some operations on parallel collections or lifecycle events for dataflow operators, there is one major change compared to beta-2 worth pointing out explicitly:

  • GPars no longer depends on the extra166y artifact. The parallel array library by Doug Lea has been integrated into GPars. The jsr166y (Fork/Join) jar still remains in the dependency list until we migrate GPars to jdk7.

Grab gpars-1.0-beta-3 and have a lot of fun with the new release.

 

 

 Our first step towards the 1.0 release has been achieved. The beta-1 release is now available for you to grab or download. Have fun and if you feel somethings needs our attention, please let us know.

 The GPars team

 

 

Parallel Game of Life

I'd like to direct you to my recent blog post detailing the use of Dataflow operators. It uses the popular Game of Life coding excercise to illustrate the principles of the dataflow concept. Check it out at my personal blog.

GPars turns 0.12 today

We have some great news to all the parallel souls out there - GPars 0.12 has just hit the shelves. The new version comes with lots of big and small improvements, out of which these are the most notable ones:

  • Composable asynchronous functions
  • The newest version of Doug Lea's Fork/Join framework (aka jsr-166y)
  • Active Objects
  • Initial stub at Software Transactional Memory support using Multiverse

Check out the full release notes for more details.
To quickly get up-to-speed with GPars, check out our updated User Guide, which is now also available in pdf format.

Your GPars team

Dr.Dobb's has just published my overview article on actors in GPars. You may check it out at http://drdobbs.com/high-performance-computing/229402193

Vaclav

GSoC assignments

It is just about the time for students to submit their GSoC applications. If you are a student and concurrency is your passion, the GPars team have prepared a list of GSoC assignments. Have a look and apply quickly. The deadline is only four days from now!

Vaclav

User Guide in Pdf

For the upcoming 0.12 release gpars now provides a Pdf version of the user guide. Check it out at http://gpars.org/SNAPSHOT/guide/gpars-guide-0.12-beta-1.pdf

Great thanks to the CZJUG hackathon team in Prague, who did the implementation, as well as to Andres Almiray of the Griffon project, who shared his valuable insight into the matter.

Getting ready for Java 7

In short - GPars is now using the Java 7 concurrency artifacts. Russel Winder has taken initiative and with an official blessing from the jsr-166 team he mirrored the most recent jsr-166 artifacts into the maven repository. This allowed us to remove the dependency on the ancient coconut clone of the fork/join framework and use the "real" live matter. With all the improvements and fixes that jsr-166y has gone through during the last couple of years this has immediate effect on GPars plus once Java 7 ships with jsr-166y bundled, we will be ready.

Nothing changes on the user side of things, provided you're using an automated dependency management tool such as Gradle or Maven. The recent 0.12-beta-1 snapshot correctly declares the new dependencies. You may check out more details in the snapshot user guide.

Enjoy concurrency in Groovy!

Yours GPars Team

GPars 0.11 is ready

Here's some good news for all the developers crafting concurrent code - GPars 0.11 has just been released. Get it fresh at your local Groovy store or go directly to the GPars download site. A lot has changed since the previous release. The most notable additions and changes in 0.11 include:

  • Great actor speed-up, so now you don't have to shop elsewhere for fast actors
  • Full-blown dataflow implementation with selectors, prioritySelectors, composable tasks, point-to-point and publish-subscribe channels and more
  • Java API allowing you to take GPars with you wherever you go

You can check out more details at at the What's new section of the User Guide or read full release notes.

Concurrency has never been easier. Get GPars now, have fun and let us know what you think.

The GPars Team

0.11 arrives at beta-1

Hi GPars enthusiasts,

on our path towards the next GPars release we've reached an important milestone - the beta-1 has just been made available. To get a feel of what's coming, experiment with the new dataflow constructs, try agent validators, test GPars from pure Java applications using the new Java API or just get your hair blown back by our lightning fast actors, grab it now at http://bamboo.ci.codehaus.org/browse/GPARS-REL/latest/artifact or use the usual integration options described at http://gpars.codehaus.org/Integration

As is our good tradition, an updated User Guide is ready at http://gpars.org/SNAPSHOT/guide/index.html
You might also like to check out what's new - http://gpars.org/SNAPSHOT/guide/guide/2.%20Getting%20Started.html#2.6%20What%27s%20new

Have groovy times and let us know your opinion.

The GPars team