Recommended Readings
Must read
If you are seriously considering using an XA transaction manager in your application, it is highly recommended to read Mike Spille's XA Exposed I, II and III.
Those blog posts are summarizing the most important things one should know about XA and they are quite pleasant to read.
Specifications
The JTA specification is a fair starting point to learn what all that is about. The XA Specification should be read in parallel as the JTA spec implicitly depends on it.
Misc
A JavaWorld article has been written about XA transactions and Spring. It uses and briefly discusses BTM as well as other transaction managers. Examples with source code are provided.