For those who already had a look at my first draft of environment management, here's a more mature example of the Gradle scripts written for a decent sized J2EE project. Our needs are specific, but representative of a software development and release process. We still have some work to do to have our tasks fully automated, but we are getting there.
The main idea is to have Gradle take care of most of the things. We are using Bamboo to run not only unit tests, but also Tellurium (read Sellenium) tests. I'm not gonna talk about the process itself, have a look here for more information.
What tasks do we want to achieve?
- compile, run unit tests and build WAR (gradle's out of the box features)
- generate a WAR targeted for a given environment
- deploy WAR (copy by FTP) on a Weblogic server
- restart the Weblogic server
- release WAR, JAR and sources on Maven repo
First of all, here's the layout of the project:
RASFF/
RASFF-configurator/
RASFF-core/
RASFF-web/
RASFF-WAR/
There are 3 project:
- RAS-core: main services project
- RAS-configurator: depends on RAS-core, only use is to run DB configuration tasks on blank environment.
- RAF-web:RAS-WAR: depends on RAS-core
As you will see, I'm far from speaking Groovy fluently. To be honest, I'm not a big fan. So please don't blame me for using long inelegant detours.
enough talking, let's skip to the scripts:
This is the main Gradle file. It contains of course all the common dependencies.
The interesting part here is the Maven deploying part. For all the projects, I wish to define a sources archive task. Since there is a bug (704 , fixed in 0.9) classifiers wont work and prevent from deploying. Nothing too bad for now.
The other feature I introduced is checking that no project depends on a SNAPSHOT library, excepts if the project itself is SNAPSHOT. That is done by parsing version name. Hopefully issue 752 will provide a more elegant way to do so.
TODO: to be closer to Maven's behaviour, I still need to add some features.
- handling of the version number. We always develop over a SNAPSHOT. When release task is called, it will change the version to non-snapshot. After processing, it could commit to SVN the new version number increased, with SNAPSHOT appended. This feature is quite low in my priorities.
- disable test skipping. You wanna release? you'll have to wait for tests to run...
- SVN tag the trunk. before or after Maven upload, it could be nice to put a tag in SVN. Right now, we are doing so with Bamboo.
- handle the version number with a parameter. Maybe it is already possible to change a Gradle project version with a command line parameter. That would be nice, since according to our process, the task 'trigerrer' will want to provide a different version number than the one from the Gradle file. High priority. This is actually not linked to Maven deploy, but to a more general scope. You will see why later on.
Nothing fancy here. Just the creation of an ANT task to process JIBX binding.
And now, the hardcore part.
This is where all the environment configuration comes in. The need is to generate a WAR containing different properties file depending on the target environment. Basically, the RASFF-WAR folder contains a none conventional folder called 'environment' containing the environment specific files. That will be 'environment.properties.dev', 'environment.properties.test' etc. It also contains a log4j conf file that is used by all the environment (though different from the developer's log4j). There is also a set of folders called WEB-INF.$ENV.
In the root of the project, you will find the 'environment.groovy' conf file (see below). This file defines all the environment deployment and build properties.
What is the need here?
- generate a WAR file containing specific 'environment.properties' file.
- add the content of the WEB-INF.$ENV to the WAR.
- if defined, deploy the generated WAR file to the given FTP folder, and restart the server.
- add the classifier $ENV to the war file so when deployed to Maven, user knows which server file can be deployed to.
+0) Initializing the environment variables.+Look at 'initConfiguration' task. If a property is defined in command line, then the task will load the ConfigSlurper associated to the given environment. In the following steps, the presence of variable "deployConfig" will be the condition for specific processing.
+1 & 2) customize the WAR file.+Look at war.doFirst and processEnvFiles(). This last method is the key. It will copy all the environment specific files (e.g. 'environment.properties.ENV' to the given folder ('build/classes'), overriding existing files. It will also copy the cross-environment files, overriding as well. Last part is to include the specific WEB-INF folder. Task is now fully configured, packaging can go on normally. After all of this, we make sure to add the right version number to a specific property file (surprisingly the same 'environment.properties' file) by appending the property to it.
+2) deploy to application server.+If deployConfig is defined, then copy the WAR file to the given FTP address/folder. User/pass must be passed in the command line. After copy is done, then the task polls every X seconds the given URL until HTTP answer is OK. Of course poll stops after a number of failed attempts.
+3) restart server.+Since Weblogic is full of bugs, and needs to be regularly restarted, our server management team provided a 'convenient' way to restart the server: copy a empty 'restart' file in the root FTP folder, and wait until cron job is launched. Until you have the 'restart' or a 'processingJob*_' file i the root, your server is restarting. This task is totally custom, and could be replaced by anything as long as you have a administrative console over your server. Of course a stable server wont need a restart for every deploy.
As you can see, all these tasks are not 100% neat. I could add a lot more of safety checks. Some parameters could be brought as well.
TODO:
- I should secure the 1 & 2 task. Right now we must run a clean as first task, or else the WAR file could be funny looking. The most problematic part is for the inclusion of the custom WEB-INF folder. If ever it contains files already present in the original WEB-INF, then they are not replaced. Instead the target WAR will contain twice the same file...
- deploy and restart should be tuned by the 'autodeploy' parameter. For the ACCEPTANCE environment, we don't have any rights over the server. The only thing we need to do is drop the WAR in a FTP folder and manually open a ticket for the Weblogic teams to deploy it.
- include some version name parsing to detect which environment deploy to. If no 'targetEnvironment' is specified, then the name of the version should determine this parameter. E.g. X.X.X.X-SNAPSHOT -> DEV, X.X.X.DEVX (release) -> TEST, X.X.X.RCX -> ACC
How all of this is used by our process?
As I stated before, we use Bamboo for CI, and we most likely want to use it as a manual deploy and test trigger, using the Jira plugin.
Our plans would look like:
CI-mainline: (on commit)
$ gradlew clean test war deploy -TargetEnvironment=DEV {make sure that the version is SNAPSHOT}
CI-mainline-unit-tellurium-test: (separate plan triggered by the first one on success)
$ launch gradle over an other project containing our Tellurium tests.
functional-test: (manual)
$ launch gradle over another project containing Tellurium functional tests. Developers shouldn't see that, the purpose is for the project lead/release manager to keep an eye on functionalities.
manual-deploy: (manual) linked to the Jira plugin. This plugin allows to pass arguments to Bamboo command lie according to the state of Jira versions and the version choosed.
$ gradlew clean release deploy -PtargetVersion={jiraPlugin.version}
as stated earlier, targetVersion parameter should be used to define on which environment to deploy.