RULE_CLASS being the name of the .net rule class.
ASSEMBLY.DLL being the name of the dll assembly file containing the rule class.
The XML fragment above assumes that the dll is located in the same directory as the standard fxcop assembly rules files.
Note: to help writing this XML files, "vladonemo" has written a C# program that generates this file based on your custom rules DLL. You can check this out on GitHub, but be aware that this comes with no garanty.
Let's assume you have implemented a rule that checks if variable name follows a naming convention. You have implemented it in a class called "VariableConventionNameCheck", and you have compiled it into a "MyNamingConvetions.dll" assembly file.