ArcSDE is becoming one of the main supported GeoServer DataStores, although its quite hard to maintain without a steady instance to reach, mostly due to the expense and license limitations of setting up ArcSDE instances. We have also found the underlying ESRI ArcSDE connection drivers a bit buggy (and the source is closed so we can't just fix them ourselves), and have also encountered problems that only seem to manifest themselves in certain instances, leading us to do some complicated work-arounds. We suspect the differing problems have to do with the backend data format, but could be different versions of ArcSDE. The DataStore was contributed by Gabriel Roldan and is maintained by Gabriel Roldan and Saul Farber, who are testing against ArcSDE 8.3, 9.0 and 9.2.
All of the above qualifications aside, multiple GeoServer users are running production web-GIS systems using GeoServer-on-ArcSDE, and the ArcSDE datastore team has done much work in the last year both in the areas of datastore maintenance/bug-fixing, and adding much new functionality to the datastore itself. As of GeoServer 1.5.1, the ArcSDE extension download supports serving certain types of ArcSDE stored raster datasets and work to support more ArcSDE stored raster formats is ongoing.
Features
The ArcSDE GeoTools plugin features vector and raster "featureclass" support.
A lot of improvements were done for the ArcSDE DataStore implementation that were introduced at the GeoTools 2.4.2 stable version (followed by GeoServer 1.6.3) and 2.5 development branch (followed by uDig 1.2.x).
The ArcSDE DataStore features comprise:
- Read/Write support of ArcSDE registered "Layers"
- Read only support for ArcSDE registered spatial views (new)
- Ability to edit versioned layers by only editing the default version (new)
- Native transaction support (new)
- Concurrent access during transaction (new)
Wish List
At the time of writing the following is a wish list of improvements and new features that we'd like to implement
- Support for non spatial, registered tables
- Ability to define a run-time, read-only FeatureType based on a SQL query (this is implemented but lacks further testing)
- Improve concurrency handling by using a per transaction work queue (this is work in progress)
Getting the JSDE and JPE jar files
GeoServer supports ArcSDE 8 and 9, users have reported success on 8.3, 9.0, 9.1 and 9.2. In the past, the GeoServer downloads included the ESRI-distributed ArcSDE client jars (jsde_sdkXX.jar and others). However, due to unclear licensing terms, we decided to discontinue this practice as anyone who has a copy of the ArcSDE install media already has these jars on the installation media itself. In addition, end-users can download the latest ArcSDE client jars for any given ArcSDE version from the ERSI website without clicking through any EULA pages (see the table below for URLs). So the first step in getting GeoServer working is installing the proper arcsde java libraries.
For connecting both to ArcSDE 8.x and 9.x you will need the jsde and jpe jars for version 9.x, since the geotools arcsde plugin that GeoServer depends on is built against them. These jars can be found in the following locations:
| ArcSDE Version | Jars needed | Location on install media | Location for ArcSDE SDK Download from ESRI Website |
|---|---|---|---|
| 9.0 | jsde90_sdk.jar and jpe90_sdk.jar | [GEOSDOC:arcsde install dir]/arcsdesdk/sdeexe90/lib/ | ArcSDE 9.0 Service Pack 3 SDK Download |
| 9.1 | jsde91_sdk.jar and jpe91_sdk.jar | [GEOSDOC:arcsde install dir]/arcsdesdk/sdeexe91/lib/ | ArcSDE 9.1 Update 3 SDK Download |
| 9.2 | jsde92_sdk.jar jpe92_sdk.jar and also icu4j_3_2.jar | [GEOSDOC:arcsde install dir]/arcsdesdk/sdeexe92/lib/ | ArcSDE 9.2 Service Pack 2 SDK Download also get icu4j_3_2.jar |
Note that these jars are backwards compatible, so if you need to access an 8 and a 9 instance you can just use the newer jars. If you've used the CD to install the ArcSDE client on your computer than it likely copied the jars in to your JRE and they should be available. Check [GEOSDOC:JAVA_HOME]/lib/ext for the jars. If not you can copy the appropriate jars to your GeoServer library directory ([GEOSDOC:HOME]/geoserver/WEB-INF/lib).
Installing the ArcSDE GeoServer Extension
Next you'll want to download the appropriate ArcSDE jar from GeoServer Extensions.
To install the ArcSDE GeoServer Extension with GeoServer version 1.5.1+, simply copy the downloaded zip file into your GeoServer WEB-INF directory ([GEOSDOC:HOME]/webapps/geoserver/WEB-INF ). Then unzip it in-place in that directory.
To install the ArcSDE GeoServer Extension with GeoServer versions 1.5.0 or earlier, copy the downloaded zip file into your GeoServer library directory (webapps/geoserver/WEB-INF/lib on a war install, [GEOSDOC:GEOSERVER_HOME]/server/geoserver/WEB-INF/lib on a binary install). Then unzip it in-place in that directory.
Using the ArcSDE GeoServer Extension
After you've done that restart GeoServer and you should now have ArcSDE as an option in the drop down menu from the web admin tool at Config -> Data -> Stores -> New and also at Config -> Data -> CoverageStores -> New
Creating an ArcSDE DataStore (for ArcSDE Vector-data layers)
To connect to an ArcSDE instance you must have the following parameters:
| Option Name | Required/Optional | Description |
|---|---|---|
| server | Required | The machine which the ArcSDE instance is running on. |
| port | Required | The port the ArcSDE instance is running on. The default is 5151 |
| instance | Optional | The ArcSDE instance name (generally "sde", or whatever you called the database). |
| user | Required | The name of the user to connect with. |
| passwd | Required | The password of the user you are connecting with. |
As with PostGIS and Oracle configuration of the files directly also needs a 'dbtype' parameter equal to 'arcsde'.
There are also a number of optional parameters to configure the ArcSDE connection 'pool'. GeoTools makes use of a number of connections, but does a decent job of managing them, so that new connections need not be made for each request. The big reason for this is that some ArcSDE licenses only allow a limited number of connections, so these values can be adjusted to minimize the number of simultaneous connections. GeoTools will share the active connections instead of making new ones. Right now it requires at least two connections, as we were having some nasty bugs with ArcSDE connections going stale (only with certain instances, and only with spatial queries, which made debugging a big hassle). With some funded development work this could easily be improved, ArcSDE could definitely benefit from some more effort. If you have more connections available we do recommend upping the pool.maxConnections parameter. The meaning of these optional parameters is as follows:
| Option Name | Description |
|---|---|
| pool.minConnections | The number of connections the pool makes on start up. If needed these will be incremented. |
| pool.maxConnections | The maximum number of connections that a pool is allowed to make. This should be as high as possible, but there may be license limitations. |
| pool.timeOut | The amount of time that a connection request should wait for an unused connection before failing. |
One more thing to note is that in the info.xml file the <name> parameter of the featureType may need to be the full qualified type name. It should be <instance>.<table_owner>.<table_name>, so in the example above a table called roads should be named sde.testsde.roads. But some users have also reported success without the instance prepended.
*GeoServer 1.5.1 or above ONLY* Creating an ArcSDE CoverageStore (for ArcSDE Raster-data layers)
Starting with the GeoServer ArcSDE Extension version 1.5.1, there is preliminary support for serving Rasters stored inside ArcSDE, either as WMS layers or as WCS layers.
Connecting to an ArcSDE Raster Layer is done by creating a new CoverageStore (Config -> Data -> CoverageStores -> New) and choosing "ArcSDE Raster Format" from the pull-down list.
To connect to a particular ArcSDE raster layer, you must supply an ArcSDE connection URL in the CoverageStore Editor page. These ArcSDE connection URLs use the following syntax:
sde://user:pass@host[GEOSDOC::port]/[GEOSDOC:instance]#raster_table_name[GEOSDOC:;extra_param=extra_value]
Here's an example:
sde://scott:tiger@localhost:5151/gis#SDE.MY_RASTER_TABLE;LZERO_ORIGIN_TILE=0,1438
NOTE: Since CoverageStores are linked to a URL, you must create a new CoverageStore for each raster layer you wish to serve.
Raster layers will try to find an already-existing ArcSDE Connection Pool to connect to the ArcSDE server. If there are other ArcSDE feature layers being served from your GeoServer instance then the ArcSDE raster code will simply use those connections. Otherwise a new ConnectionPool will be created with default settings.
Supported Extra Parameters
| param name | param description |
|---|---|
| LZERO_ORIGIN_TILE | I've run into raster datasets where the 0,0 tile at level zero isn't actually located in the upper-left of the raster dataset. This specifies the actual proper location for the 0,0 tile, in level-zero tilespace. You'll know that you need to set this parameter if everything works great, but when you zoom WAY in and get to level zero of the pyramid, your rasters either disappear or are shifted by a huge amount |
Supported Raster Types
Currently ArcSDE Raster support is limited to 3-band or 4-band, one-byte-per-band, non-colormapped rasters. Support for further ArcSDE raster types is in progress. To check out what's on the roadmap, look at the GeoTools ArcSDE Raster Format information page.
| Support Status | Raster Type |
|---|---|
| |
3 or 4 Band unsigned-byte RGBX Rasters |
| |
Colormapped rasters |
| |
Non unsigned-byte Rasters |
General configuration notes
Also make sure that the port number specified is the same as in the services file (%(%(%systemRoot%\system32\drivers\etc\services in windows), it should contain a line like (if yours is different then change the port param of your datastore definition):
esri_sde 5151/tcp
For ArcSDE/ArcGIS 9.2, there is an issue with GeoServer (tested with 1.4.1) and the way ArcCatalog 9.2 creates feature classes. ArcCatalog 9.2 creates text fields as NVARCHAR data type and it seems GeoServer does't like this. Trying to setup a new FeatureType results in a java.lang.NullPointerException error message.
The work around is to add
UNICODE_STRING "FALSE"
to the dbtune.sde file and then import the contents of this file into the DBTUNE table using the sdedbtune command. This forces ArcSDE 9.2 to create text field as VARCHAR data type.
Is it possible to have future versions of GeoServer support the NVARCHAR data type?
For *nix installs, you may need to check the permissions on the jar files that are added to the GeoServer library directory. Be sure that they correspond to a user and group that can be read by Geoserver.
If you need to use ArcSDE jars you need to do the following in additon to what's written on this page:
- Download the src of Geoviewer (bin version won't work)
- Modify ArcSDEDataStoreFactory.java file. You will need to comment and uncomment a few lines there
- Recompile ArcSDEDataStoreFactory.
You are all set. I will highly recommend that you use Maven if you are just starting. I was a skeptic but it really helped me going forward.
The ArcSDE 9 jars are included with the default GeoServer 1.3.0 download.