Discovering a web application's security requirements

Author: *Mark S. Petrovic, mspetrovic at gmail dot com.

Probably the best reference, and a bit more readable: http://www.onjava.com/pub/a/onjava/2007/01/03/discovering-java-security-requirements.html

Introduction

The Java Runtime Environment provides well-documented means by which an application can run subject to a defined security policy. The purpose of this managed security is to selectively grant or deny the application access to specific system resources. For our purposes, resources are things such as system- and application-level properties, disk files, network connections, and class loading privileges, to name a few. Resources represent things the application either wants to know or do. It is the Java java.lang.SecurityManager class that enforces this resource access, with the rules of enforcement based on simple rules in a text-based human readable security policy file.

In this article we introduce and discuss a tool, a profiling security manager, that programmatically produces a starting point in crafting an effective security policy. This draft policy describes to what resources the application wants access and in what capacity (e.g., read, write). Our motivation is for both educational purposes and to develop a tool to learn what resources complex applications require access, and to what degree, to run according to their specification. The rules produced by the tool may be used as a point of departure in producing a policy file for these complex applications, presumably sufficiently complex that producing a fine-grained policy file without runtime information may be too difficult, or simply as a subject of study and examination.

The version of the profiling security manager discussed here requires a late-model Sun JVM, as it relies on reflection to acquire information on a particular system class's private member data. I used JSE 5, and the tool requires java.security.AccessControlContext.context of type ProtectionDomain[] to exist. Of course, once the policy is generated, one can revert to using it under whatever JVM one prefers.

The default Security Manager

By default a Java application runs with no runtime security manager. The application has full access to the machine resources, including disk, network, and application shutdown. Such broad access is easily restricted, however. An application can be forced to run under a "default" Java security policy by setting the -Djava.security.manager option on the JVM command line.

Consider this simple application, designed to print the user's home directory represented by the System property user.home

public class PrintHome {
   public static void main(String\[\] argv) {
      System.out.println(System.getProperty("user.home"));
   }
}

and with this runtime invocation

$ java -Djava.security.manager PrintHome
Exception in thread "main" java.security.AccessControlException: access denied
(java.util.PropertyPermission user.home read)
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:264)
at java.security.AccessController.checkPermission(AccessController.java:427)
at java.lang.SecurityManager.checkPermission(SecurityManager.java:532)
at java.lang.SecurityManager.checkPropertyAccess(SecurityManager.java:1285)
at java.lang.System.getProperty(System.java:627)
at PrintHome.main(PrintHome.java:5)

The application fails to execute because the default security manager running with the default security policy forbids accessing the property user.home. Without the ability to read property user.home, the application cannot fulfill its intended purpose. This privilege must be granted explicitly in a runtime policy file.

Creating a policy file policy.txt, containing the single rule

grant codeBase "file:/home/cid/Projects/CustomSecurityManager/"
{permission java.util.PropertyPermission "user.home", "read";};

and rerunning the application with a reference to the policy file solves the problem of access to user.home:

$ java -Djava.security.policy=policy.txt -Djava.security.manager PrintHome
/home/u

The class PrintHome resides in the directory /home/cid/Projects/CustomSecurityManager/, and the rule permits any code originating in that directory to read system property user.home. As a result, the rule allows PrintHome to run as intended.

Introducing ProfilingSecurityManager

For simple applications, creating a policy file by hand is relatively straightforward and uninvolved. And even more so when one takes advantage of powerful URL specification rules allowed in the policy file. Using advanced rules notation, one can specify codebase URLs that refer recursively to entire directory trees. While such recursive URL specification is powerful, it can also mask to the human reader the true depth of the resource needs of the application. It is precisely that fine-grained depth that we seek.

We are interested in a programmatic method of determining precisely which resources to which an application requires access for it to successfully execute over its runtime trajectory. Toward that end, we introduce the custom security manager ProfilingSecurityManager (source code appears as an appendix). ProfilingSecurityManager extends java.lang.SecurityManager, but does not implement a security policy in the traditional sense. It instead determines what that security policy would be if the application were granted access to everything it requests at runtime – under the application's intended conditions of use, of course.

To use ProfilingSecurityManager, specify it as the security manager when the application is invoked, noting that no policy file specification is necessary

$ java -classpath .:classes \
-Djava.security.manager=com.homosuperiorus.security.ProfilingSecurityManager \
PrintHome

The custom security manager will write to System.out the rules needed in a policy file that will allow the application to run without throwing security violation exceptions:

grant codeBase "file:/home/cid/Projects/CustomSecurityManager/"
{permission java.util.PropertyPermission "user.home", "read";};

Because the raw output from ProfilingSecurityManager can legitimately contain duplicate grant statements, and in the order the running application requires them, a Perl script parsecodebase.pl is provided to aggregate, format, and output only unique rules, grouped by codebase. Rules must be processed after runtime, because only then is it known that the ProfilingSecurityManager is finished outputting new rules.

Rerunning the application with output processed by parsecodebase.pl

$ java -classpath .:classes \
-Djava.security.manager=com.homosuperiorus.security.ProfilingSecurityManager \
PrintHome | parsecodebase.pl
grant codeBase "file:/home/cid/Projects/CustomSecurityManager/"
{permission java.util.PropertyPermission "user.home", "read";};

Here is parsecodebase.pl

#!/usr/bin/perl -w

use strict 'vars';
use strict 'subs';
use Uniq;   # See http://search.cpan.org/~syamal/Uniq-0.01/Uniq.pm

# Usage:  $0 < logs/catalina.out

# Extract all rules from stdin
my @grantList= uniq sort grep(/^grant/,<STDIN>);

# Extract all codeBase instances
my @codeBaseList;
foreach my $j (@grantList) {
   $j =~ m/"(.*?)"/;
   push @codeBaseList, $1;
}

# Keep only unique codebase instances
@codeBaseList = uniq sort @codeBaseList;

# For each unique codebase, extract all rules containing it and print aggregated rules
foreach my $k (@codeBaseList) {
   my @tlist = uniq sort grep(/^grant codeBase \"$k/,@grantList);
   print "grant codeBase \"$k\" {\n";
   foreach my $n (@tlist) {
      $n =~ m/\{(.*)\}/;
      print "   $1\n";
   }
   print "};\n\n"
}

Significance to webapp development

We put forth that it is good, albeit potentially difficult, practice to use a runtime security policy for webapps, and push toward using ProfilingSecurityManager to profile the security needs of those applications. With that in mind, we discuss how to profile a Tomcat webapp using ProfilingSecurityManager. In the course of that discussion we will develop a temporary custom startup script for Tomcat that allows that profiling. Other servlet containers should have similar startup procedures that can be modified to use ProfilingSecurityManager as the security manager for profiling purposes.

Tomcat can be made to run under the default Java security manager in the following manner

$ $CATALINA_HOME/bin/startup.sh -security

which in turn leads to the calling of $CATALINA_HOME/bin/catalina.sh with the -security option. We examine the startup process in more detail by inserting a set -x in $CATALINA_HOME/bin/catalina.sh.

To profile the webapp using ProfilingSecurityManager, we must develop a new Tomcat startup script. The new startup script is a temporary device, and will be used only for profiling, then discarded.

Make a backup copy of $CATALINA_HOME/bin/catalina.sh. Insert the shell command set -x in $CATALINA_HOME/bin/catalina.sh near the top of the script, and start Tomcat. Save the displayed shell executed command to a file that will hold the temporary startup script. Stop Tomcat, and edit the temporary script, specifying ProfilingSecurityManager as the security manager and modifying the classpath to locate ProfilingSecurityManager.

Under Tomcat 5.5.17, here is what the temporary startup script looks like before we edit it for purposes of using ProfilingSecurityManager, with a bit of tweaking and formatting

#!/bin/sh

log=$CATALINA_HOME/logs/catalina.out

/java/jdk/jdk1.5.0_06/bin/java \
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager \
-Djava.util.logging.config.file=/home/tomcat/tomcat/conf/logging.properties \
-Djava.endorsed.dirs=/home/tomcat/tomcat/common/endorsed \
-classpath :/home/tomcat/tomcat/bin/bootstrap.jar:/home/tomcat/tomcat/bin/commons
-logging-api.jar \
-Djava.security.manager \
-Djava.security.policy==/home/tomcat/tomcat/conf/catalina.policy \
-Dcatalina.base=/home/tomcat/tomcat \
-Dcatalina.home=/home/tomcat/tomcat \
-Djava.io.tmpdir=/home/tomcat/tomcat/temp org.apache.catalina.startup.Bootstrap \
start >> $log 2>&1 &

and here is what the temporary startup script looks like after editing for purposes of using ProfilingSecurityManager:

#!/bin/sh

log=$CATALINA_HOME/logs/catalina.out
PATHTOPSM=classes

/java/jdk/jdk1.5.0_06/bin/java \
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager \
-Djava.util.logging.config.file=/home/tomcat/tomcat/conf/logging.properties \
-Djava.endorsed.dirs=/home/tomcat/tomcat/common/endorsed \
-classpath $PATHTOPSM:/home/tomcat/tomcat/bin/bootstrap.jar:/home/tomcat/tomcat/b
in/commons-logging-api.jar \
-Djava.security.manager=com.homosuperiorus.security.ProfilingSecurityManager \
-Djava.security.policy==/home/tomcat/tomcat/conf/catalina.policy \
-Dcatalina.base=/home/tomcat/tomcat \
-Dcatalina.home=/home/tomcat/tomcat \
-Djava.io.tmpdir=/home/tomcat/tomcat/temp org.apache.catalina.startup.Bootstrap \
start >> $log 2>&1 &

The only difference between the two scripts is a) The new version has an augmented classpath that points to where ProfilingSecurityManager resides. Note that for this example, we assume ProfilingSecurityManager can be found in the directory classes located in the subdirectory in which the script executed, and b) The new version uses ProfilingSecurityManager as the security manager in the -Djava.security.manager parameter.

Start Tomcat with the temporary startup script and look for grant statements in $CATALINA_HOME/logs/catalina.out, and code-cover (easier said than done) the running webapp. Stop the webapp, and run $CATALINA_HOME/logs/catalina.out through parsecodebase.pl above.

$ parsecodebase.pl < $CATALINA_HOME/logs/catalina.out > yourpolicy.txt

with the output saved to yourpolicy.txt. The rules output to yourpolicy.txt are the starting point for our production security policy. We should examine every rule carefully to understand what it does and to confirm that it is consistent with our application's goals. When we are confident we have a good draft policy, make a backup copy of $CATALINA_HOME/conf/catalina.policy, and incorporate the new draft rules into it. Then revert back to the original Tomcat startup script with the -security option set, and continue testing.

Here is a typical set of rules captured with ProfilingSecurityManager under a stock version of Tomcat 5.5.17 during startup. When we examine the policy file $CATALINA_HOME/conf/catalina.policy that ships with Tomcat, we notice rules that grant all permissions to Tomcat ("Catalina") system codebases. ProfilingSecurityManager discovers these same rules for certain Tomcat system classes, but specifies them in a fine-grain fashion. We recommend the rules ProfilingSecurityManager discovers about Tomcat system classes be pruned and ignored in its output, and let the existing Tomcat system rules in $CATALINA_HOME/conf/catalina.policy stand. ProfilingSecurityManager has as its primary use discovering our application policy requirements, not Tomcat's. Tomcat's security requirements are already expertly addressed in the shipped policy file.

ProfilingSecurityManager works by inspecting the current executing context in its essentially "called-back" checkPermission() method. For each java.security.ProtectionDomain checkPermission() finds in the context, it attempts to determine the associated CodeSource through standard class methods. When the CodeSource is determined, a grant rule can be constructed.

Troubleshooting

What if the application will not run with the rules thus-produced? By "not run" we mean serious indications in $CATALINA_HOME/logs/catalina.out relating to security policy issues. Set the environment variable

$ export CATALINA_OPTS=-Djava.security.debug=access,failure

then start Tomcat. If ProfilingSecurityManager missed some rules, for example and for whatever reason, one will see output in $CATALINA_HOME/logs/catalina.out bracketed by two key records of the form:

access: access denied (java.util.PropertyPermission log4j.defaultInitOverride write)

access: domain that failed ProtectionDomain
 (file:/u/homer/apache-tomcat-5.5.12/webapps/crmrpc/WEB-INF/classes/org/binky/crm/CRMContextInit.class

These two records (ignoring all records in between) provide sufficient information to build a grant rule that should be placed in the policy file, by hand presumably at this point.

Simple command line experimentation

It is straightforward and informative to experiment with ProfilingSecurityManager from the command line.

Consider these two simple application classes, App.java and JarCode.java:

// This is App.java
package com.homosuperiorus;

import static java.lang.System.out;
import java.net.*;

public class App {

   private void doSomething() {
      // Get a system property
      String userHome = System.getProperty("user.home");
      out.println("user home: " + userHome);

      // Make a network call
      try {
         Socket sock = new Socket("localhost",80);
         sock.close();
      }
      catch (Exception e) {
         ;
      }

      // Call code we know to exist in a jar file
      JarCode jc = new JarCode();
      jc.doSomething();
   }

   public static void main(String[] argv) {
      App app = new App();
      app.doSomething();
   }
}
// This is JarCode.java
package com.homosuperiorus;

import static java.lang.System.out;

public class JarCode {
   public void doSomething() {
      String s = System.getProperty("java.version");
      out.println("I am JarCode/ Java version: " + s);
   }
}

built with make.sh given by

$ cat make.sh
#!/bin/sh

classes=classes
mkdir -p $classes
rm -rf $classes/*

javac -d ${classes} App.java JarCode.java ProfilingSecurityManager.java

# Force some of the runtime code to come from a jar file, as this generates
# unique CodeSources for our policy file rules.

(cd $classes;
jar cf ../externaljar.jar com/homosuperiorus/JarCode.class
rm com/homosuperiorus/JarCode.class
)

We run the simple application using this run.sh script, saving the rules to apolicy.txt:

$ cat run.sh
#!/bin/sh

pkg=com.homosuperiorus
debug=-Djava.security.debug=access,failure
sm=-Djava.security.manager=${pkg}.security.ProfilingSecurityManager

java ${sm} ${debug} -classpath classes:externaljar.jar ${pkg}.App
$ sh run.sh |parsecodebase.pl |tee apolicy.txt
grant codeBase "file:/home/cid/Projects/CustomSecurityManager/classes/" {
   permission java.io.FilePermission "/dev/random", "read";
   permission java.io.FilePermission "/dev/urandom", "read";
   permission java.io.FilePermission
"/home/cid/Projects/CustomSecurityManager/classes/com/homosuperiorus/JarCode.class",
"read";
   permission java.io.FilePermission
"/home/cid/Projects/CustomSecurityManager/externaljar.jar", "read";
   permission java.io.FilePermission
"/home/cid/java/jdk/jdk1.5.0_06/jre/lib/i386/libnet.so", "read";
   permission java.io.FilePermission
"/home/cid/java/jdk/jdk1.5.0_06/jre/lib/net.properties", "read";
   permission java.lang.RuntimePermission "loadLibrary.net", "";
   permission java.lang.reflect.ReflectPermission "suppressAccessChecks", "";
   permission java.net.NetPermission "getProxySelector", "";
   permission java.net.NetPermission "specifyStreamHandler", "";
   permission java.net.SocketPermission "127.0.0.1:80", "connect,resolve";
   permission java.net.SocketPermission "localhost", "resolve";
   permission java.security.SecurityPermission "getProperty.securerandom.source", "";
   permission java.security.SecurityPermission "putProviderProperty.SUN", "";
   permission java.util.PropertyPermission "impl.prefix", "read";
   permission java.util.PropertyPermission "java.home", "read";
   permission java.util.PropertyPermission "java.net.preferIPv4Stack", "read";
   permission java.util.PropertyPermission "java.net.preferIPv6Addresses", "read";
   permission java.util.PropertyPermission "java.net.useSystemProxies", "read";
   permission java.util.PropertyPermission "java.security.egd", "read";
   permission java.util.PropertyPermission "java.version", "read";
   permission java.util.PropertyPermission "sun.net.spi.nameservice.provider.1","read";
   permission java.util.PropertyPermission "user.home", "read";
};

grant codeBase "file:/home/cid/Projects/CustomSecurityManager/externaljar.jar" {
   permission java.util.PropertyPermission "java.version", "read";
};

by which we see a simple application requiring a rich set of permissions to get the most common of things done.

We now run the application with the apolicy.txt policy file

$ java -classpath .:classes:externaljar.jar \
-Djava.security.manager -Djava.security.policy=apolicy.txt com.homosuperiorus.App
user home: /home/cid
I am JarCode/ Java version: 1.5.0_06

Source code for PrintHome.java, ProfilingSecurityManager.java, App.java, JarCode.java, and the various shell and Perl scripts discussed here can be found at http://www.petrovic.org/content/SecMgrTutorial/tutorial.tar.

Acknowledgments

Thanks to Jason Brittain for a discussion that started me back on this track, and to one of my blog readers who suggested the method was possible.

Appendix

Here is ProfilingSecurityManager

/*

* Copyright (c) 2006 Mark Petrovic <mspetrovic@gmail.com>
*
* Permission is hereby granted, free of charge, to any person obtaining
* a copy of this software and associated documentation files (the
* "Software"), to deal in the Software without restriction, including
* without limitation the rights to use, copy, modify, merge, publish,
* distribute, sublicense, and/or sell copies of the Software, and to
* permit persons to whom the Software is furnished to do so, subject to
* the following conditions:
*
* The above copyright notice and this permission notice shall be
* included in all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
* LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
* OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*
* Original Author: Mark Petrovic <mspetrovic@gmail.com>
* */

package com.homosuperiorus.security;

import static java.lang.System.out;

import java.lang.reflect.Field;

import java.net.URL;
import java.security.AccessController;
import java.security.AccessControlContext;
import java.security.CodeSource;
import java.security.Permission;
import java.security.ProtectionDomain;
import java.util.ArrayList;

/**
 * <code>ProfilingSecurityManager</code> is a Java security manager that profiles
 * what resources an application accesses, and in what manner --- e.g., read,
write, etc.  It does not
enforce a
 * security policy, but rather produces a starting point for crafting one.
 * <p>
 * It extends <code>java.lang.SecurityManager</code> and overrides the two forms
of the <code>checkPerm
ission()</code> method.
 * For each call to <code>checkPermission()</code>,
<code>ProfilingSecurityManager</code> first guards
against the
 * condition that it itself induced the call to <code>checkPermission()</code>,
which would result in
 * unterminated recursion.  If a call to <code>checkPermission()</code> resulted
from a call outside
 * <code>ProfilingSecurityManager</code>, the current context is examined and each
class found therein
is
 * profiled as needing access to the
<code>java.security.Permission</code> in question.
 *
 * Profiling is manifested as a writing to <code>System.out</code> a "grant"
rule for each <code>java.s
ecurity.Permission</code> requested
 * on a per <code>CodeBase</code> basis.
 *
 * @author Mark S. Petrovic
 */

public class ProfilingSecurityManager extends SecurityManager {

   /* Variables of pure convenience */
   private String thisClassName;
   private Class thisClass;
   private ClassLoader cl;

   public ProfilingSecurityManager() {
      cl = this.getClass().getClassLoader();
      thisClassName=this.getClass().getName();
      thisClass = this.getClass();
   }

   // -----------------

   // With a Permssion and AccessControlContext, we can build and print a rule
   private void buildRules(Permission permission, AccessControlContext ctx) {
      ProtectionDomain[] protectionDomain = getProtectionDomains(ctx);
      for(int i=0;i<protectionDomain.length;++i) {
         printRule(permission, protectionDomain[i]);
      }
   }

   // -----------------

   /*
      Traverse the stack, returning true if the stack indicates we called ourselves.
      Avoid recursion that can occur in the course of getting ProtectionDomain's and such.
   */
   private boolean isRecur(StackTraceElement[] st) {
      boolean v = false;
      for(int i=st.length-1;i>=1;--i) {
         boolean c = st[i].getClassName().equals(thisClassName);
         boolean m = st[i].getMethodName().equals("buildRules");
         if (c && m)  {
            v = true;
            break;
         }
      }
      return v;
   }

   // -----------------

   @Override
   public void checkPermission(Permission permission) {
      Throwable t = new Throwable("Profiler stack probe");
      StackTraceElement[] stack = t.getStackTrace();
      // Avoid recursion owing to actions in this class itself inducing callbacks
      if( !isRecur(stack) ) {
         buildRules(permission, AccessController.getContext());
      }
   }

   @Override
   public void checkPermission(Permission permission, Object context) {
      buildRules(permission, (AccessControlContext)context);
   }

   // -----------------

   /* Get the protection domains by Java reflection.  There is no public API for this info.
    * Making this code JVM implementation dependent!!!
    */
   private ProtectionDomain[] getProtectionDomains(AccessControlContext context) {
      ProtectionDomain[] pda = null;
      try {
         Field[] fields = AccessControlContext.class.getDeclaredFields();
         for(int i=0; i<fields.length; ++i ) {
            if( fields[i].getName().equals("context") ) {  // Warning:  JVM-dependent
               fields[i].setAccessible(true);
               Object o = fields[i].get(context);
               pda = (ProtectionDomain[] )o;
               break;
            }
         }
      }
      catch (IllegalAccessException e) {
         e.printStackTrace();
      }
      finally {
         return pda;
      }
   }

   // -----------------

   /* Now that we have a Permission and a ProtectionDomain, we can construct a grant rule
   */
   private void printRule(Permission permission, ProtectionDomain pd) {
         CodeSource cs = pd.getCodeSource();
         if( null == cs ) {
            out.println("csnull: " + pd.toString());
            return;
         }
         URL url = cs.getLocation();

         // don't print rules for the security manager codebase itself
         CodeSource thisCodeSource = thisClass.getProtectionDomain().getCodeSource();
         URL thisCodeSourceURL = thisCodeSource.getLocation();
         if( url.toString().equals(thisCodeSourceURL.toString()) ) {
            return;
         }

         StringBuilder sb = new StringBuilder("grant codeBase \"");
         sb.append(url.toString());
         sb.append("\" {");
         sb.append("permission ");
         sb.append(" ");
         sb.append(permission.getClass().getName());
         sb.append(" ");
         sb.append("\"");
         sb.append( permission.getName().replace("\"","\\\"") );
         sb.append("\", ");
         sb.append("\"");
         sb.append(permission.getActions());
         sb.append("\";");
         sb.append("};");

         out.println(sb);
   }

 // -----------------

   @Override
   public String toString() {
      return "SecurityManager:  ProfilingSecurityManagerr";
   }

}

Labels

 
(None)