I have stuff in my sandbox to handle the distributed locking of session access...
There are three roles involved :
1) Process owner (PO) - the node which requires the state's presence in order to run some process over it...
2) Bucket Owner (BO) - the node managing the bucket into which the session's key hashes. The key is associated with the location of the:
3) State Owner (SO) - the node which actually owns the only copy (unless replication enabled - NYI) of this piece of state.
WADI will try to use the load-balancers session affinity to maintain the colocation of PO, BO and SO within the same JVM - but, if this fails, should be equally happy, though not as fast.
Only a few functions are supported (initially - READ/WRITE/REMOVE - REMOVE is treated as a WRITE key=null).
Two cases present themselves :
a). PO wants to READ or WRITE using a key that is not known by BO (know as the Absent.pdf case)
- PO acquires local lock and repeats message to BO until it gets a response/gives up
- BO inserts new association Key:PO int bucket and replies
- PO inserts new association Key:State and releases local lock
b). PO wants to READ or WRITE using a key that is known by BO - thus an SO must be involved in the interaction (known as the Present.pdf case).
- PO acquires local lock and repeats message to BO until it gets a response/gives up
- BO acquires local lock for Key and repeats message to SO until receives response or gives up and releases lock...
- SO takes acquires local lock and send state across to PO (including PO's correlation ID) (repeated)
- PO receives response to original message, containing state - inserts locally, releases lock and acks to SO
- SO releases lock and acks to BO
- BO release lock
I'm still thinking about these... I thought I had a solution to deadlocking - always take the lock in BO first, then SO (if required) then PO - but I've just realised that I want to have taken the lock in PO at the beginning of the request - 
If I was going to use the above deadlock solution, I would have required an extra final message between PO and SO, in the Absent case, to ensure that BO unlocks correctly....
I thought about using leases, but decided that it was better to allow nodes holding locks to be responsible for ensuring a reply (either success or failure) from the downstream transaction, by repeating their original message until they receive one or decide that they are not going to....