A Google Summer of Code 2007 project.
The 3D Renderer provides a three dimensional view of GeoTools geographical data.
It will uses the normal 2D renderer for rendering the surface texture.
It implements a level of detail based loading and caching system for the geographical data to speed up rendering, and allow perspective views showing both nearby and far away features at the same time.
Possible future improvements are rendering elevation data based on height coverage data. In addition, there could be support for some common 3D rendering styles that can be used for features consisting of points, lines, and areas.
I created a powerpoint presentation for handouts and similar for the FOSS4G conference. It gives a short overview of the 3D Map Renderer, it's current status, and future plans.
- Student: Hans Häggström
- Handle: zzorn
- Email: zzorn at iki.fi
- Mentor 1: Jody Garnett
- Mentor 2: Jesse Eichar
- Daily IRC: 8pm Helsinki / 10 am Victoria
For ongoing project status see this page.
NOTE: These build instructions assume that you have the 2.4-SNAPHSOT version of GeoTools installed - the build file needs to be updated to work with the latest stable release instead. I will do that as soon as I have time, but it may take until after FOSS4G (so start of October 2007).
- Check out the code
- Install third party libraries in the local repository
- On Windows:
- On Linux:
- Build using Maven
- You can run the example with:
A tiled map using multi-detail level tiles (thanks Petr!) that I hacked together in two days during the FOSS4G 2007 conference.
Screenshots from the final version at the end of the summer. The left picture shows a normal view, and the right picture shows how a low resolution picture is used as a preview while a tile is being rendered.
The left picture shows the quad tree structure of the terrain, and the right one shows the first hand generated test map.
Work Plan / Task List
- Contributor setup (mailing lists, access rights, etc)
- 2D Spike - Loading and rendering a shapefile
- 3D Spike - Show 3D map in a swing window
- Test Data
- Getting Test Data (I need a shapefile and a matching SLD file so that I can render a map with StreamingRenderer) - GeoServer has some shapefiles with correct SLD files, use them
- Navigation on the 3D map
- Pan with left mouse button drag use side arrow keys and pgup & pgdown instead, or maybe middle button drag (will move camera along the left/right, up/down axes of the camera). (A single left mouse button click will select / click on an item on map). The drag has some inertia, so a quick drag will move the camera faster than a slow one. The camera is kept above the ground level at all times though.
- Move across map with left mouse button drag. The camera moves along the ground plane (at a constant altitude?), relative to the direction the camera is facing. The drag has some inertia, so a quick drag will move the camera faster than a slow one.
- Turn camera using right mouse button drag (change yaw and pitch, pitch locked to avoid rolling to back, there could be a maximum angle to avoid getting lost looking at the sky) (cursor disappears while mouse pressed, relative movement is measured. Similar to first person view games). (A right mouse button click without drag will typically show a context menu for the feature under the mouse).
- Move camera along forward axis with W,D or up/down arrow keys or scroll wheel (acceleration is enabled (and a bit of inertia too for effect)). With scroll wheel, a few scrolls gives a small thrust, many scrolls leaves the thrust on. Scroll back to turn the thrust off again (and to thrust backwards).
- Use middle click to reset view to standard view that the camera currently is closest to - N,S,W,E or straight down, with up on screen being north (maybe also NW,NE,SW, and SE directions, and 45 degree views). Also stops movement.
- Build setup
- Include the required dependencies (JME & LWJGL) and provide an install script for them
- Change the maven project file to a Java 1.5 one
- 3D Terrain
- Quadtree Code
- Generate 3D Mesh for a block of terrain
- Make 3D Mesh have a down turned 'skirt' around its edges to avoid 1 pixel cracks in the terrain caused by rounding errors.
- Assign a texture to a terrain mesh from a BufferedImage
- Define low level interfaces for how the renderer gets height and texture information, and implement them.
- Render 2D Map to an Image and from there to a terrain mesh
- Map Block Management - subdivide and merge as camera moves
- Fix growing of the quad tree so that the map blocks always extend to the horizon
- Set up textures to use smooth rendering, to avoid the pixelated effect visible in the second screenshot.
- Implement pooling of BufferedImages used for generating the ground textures
- Implement pooling of TerrainBlocks.
- Use a low resolution preview taken from a parent block for textures that are waiting to be rendered.
- Fix y-flipping bug (maybe turn around whole terrain?)
- Split map blocks so that neighboring ones differ at most by one subdivision
- Reposition the 3D scene to origo when it gets too far away - floats that the geometry is represented in have accuracy problems with too large values.
- Investigate Issues
- Check if there is a memory leak in reserving texture memory on the 3D card, and how to free it when it is no longer used.
- Investigate projections - what coordinate system to use in the 3D view, and how to translate coordinates
- Find out if there is a way to force Swing menus and tooltips to use heavyweight awt rendering, so that they are visible on top of the 3D canvas.
- Elevation Support elevation data, use coverages or other data sources to get it.
- 3D Markers (3D models on the map) and Labels
- Point and click selection of a marker or point on the ground
- Fancy features - buildings, forest, 3D lines, globe view
The java.library.path needs to point to the LWJGL native code libraries for the correct platform. It can be set on the java command line like this: java -Djava.library.path=C:/Libs/LWJGL/Win32 foo.class
The mvn exec:exec goal does this when running the example program.
Using the 3D Renderer in your own projects is easy. Just make sure you have the necessary dependencies available (and remember to set the java library path to point to the LWJGL native lib directory), then just:
Create a map context that should be rendered:
Create a 3D renderer:
Get a 3D view UI component from the renderer:
The component is a normal AWT UI component, and can be added to any Swing or AWT UI:
See Show3DMapExample.java for a working example program.
Lack of multithreaded UI support
The Swing support in JME version 0.10 does not manage multithreaded UI:s very well - JME 0.11 has better support for multithreading, but requires Java version 1.5.
For now I decided to just use the Java 1.4 compatible version. This may limit the kind of UI:s that can be built around it, but basic Swing widgets seem to work. If we later start to use Java 1.5 in GeoTools it should be easy to port the code to the newer version of JME.
Java 1.5 modules seem to be acceptable, so I'll be using Java 1.5 and the latest version of JME.
Lightweight Swing components do not render on top of the 3D Canvas.
The 3D canvas is an AWT component, and is updated by the 3D card/library, so it is not possible to draw lightweight swing components on top of it. This causes problems especially with menus, context menus, and tooltips. Maybe there is some way to force them to be heavy weight (=own windows at the window system level)? I seem to recall that something like that was possible to do.
A workaround is to use AWT when implementing the UI, it should work (to be checked). But that is a major disadvantage.
UPDATE: It's possible to use JPopupMenu.setDefaultLightWeightPopupEnabled( false ); to disable lightweight rendering for popups, this should help make them overlap the 3D window (thanks Eclesia!).
The map could be based around a multi-resolution quad tree - each leaf in the quad tree represents a square area of some size - a MapBlock.
The quad tree is subdivided into smaller grids near the camera, to provide higher resolution for nearby terrain.
Each MapBlock may have a texture associated with it, to be used for texturing the terrain. It may also have an array of elevation data, and a set of geometrical features.
The interface towards data sources should be such that features can be queried for (rectangular) areas using a significance threshold (where the significance is calculated from feature size and/or how important it is to show - small roads are not important in a large overview of a country, while individual Google SoC developer locations should be visible even in a global view when showing the developer distribution).
3D styles may be implemented by extending the current style system in some way if possible, or otherwise by specifying datasources and the 3D style to use for them separately.
In addition, the 3D renderer provides an interface with methods to get a Swing component containing the 3D view, methods for moving the camera, and constructor parameters / hints for specifying the resolutions to use for the textures and elevation density on MapBlocks.
Features will normally be rendered directly to the ground texture, but optionally 3D styles can be used for features, creating 3D geometry for them instead. Examples of 3D styles could be abstract shapes used for markers (pyramids, spheres), slightly elevated roads, buildings, and imported custom 3D landmarks (Eiffel tower in a map of Paris, etc).
The ground textures will be rendered using the normal 2D rendering pipeline (although the 3D renderer could offer an extension point for custom texture renderers).
The 2D ground rendering should take place in a background thread. While it is ongoing, a rougher version can be displayed by using previously rendered ground textures from a higher quadtree level, or a placeholder image can be used.
Soft transitions in the texture may be needed at the edges between MapBlocks of different resolutions. This could be done in the graphics card, using shaders.
Different features may have labels that should be shown. They are best rendered to a texture, and shown as a billboard (3D polygon always turned towards the camera)
Markers can be either 2D or 3D.
2D markers are simply icons / pictures rendered to a texture and shown on a billboard, like labels.
3D markers use some built in shape (sphere, pyramid, cube) in a solid color or textured with some texture. Alternatively they can use a custom 3D model (e.g. Eiffel tower, map pin, etc ).
Building Rendering (optional)
Buildings or city blocks may be represented by arbitrary polygons on some maps. They can be rendered as 3D by simply extending the polygon upwards, giving it a height. It might also be textured, and depending on the style of building, the roof could be sloped. The style how to render buildings would probably be specified as some kind of metadata for a larger area (e.g. downtown has taller buildings with flat roofs, while suburbs have lower buildings with slanted roofs).
Initially just a simple fixed height, flat roof, untextured building style could be implemented.
See Instant Architecture (pdf) for one approach on building texture generation.
See Automatically Generating Roof Models from Building Footprints for a paper on generating building roofs.
There is also a lot of research on automatically creating 3D buildings from satellite photos (identifying building shape and height from the shadows that they cast), but that is out of scope for this project.
Forest Rendering (optional)
Optionally, forest and vegetation could be rendered using multiple textured layers for each MapBlock, as described in Forest Scenes in Real-Time . The extent and type of a forest could be specified by polygonal features or coverage (which one is normally used?).
Globe View (optional)
It would be nice to be able to support a globe view, where the earth is shown as a sphere, in addition to a detailed low altitude view, and provide seamless transitions between them. However, for now it remains a lower priority.
To optimize rendering, features could be classified by size and/or importance (significance). Only features with a significance over a certain threshold would be included on a MapBlock. The significance threshold would be lower for small, detailed MapBlocks close to the camera, and higher for large MapBlocks far away from the camera.
Optionally the rendering speed of 3D markers, buildings, landmarks, and such can be sped up by rendering them to impostor textures, which are always turned towards the camera, and only updated when the camera moves significantly in relation to them.
Point And Click
Another problem that needs to be solved is picking - determining which feature or map coordinate has been clicked when the user presses a mouse button over the 3D view. At least for 3D features, picking could be solved by using an existing 3D scenegraph, such as JME.
The 3D view should include at least some rudimentary navigation support based on mouse and keyboard input, but allow clients of the library to override those.
I'm leaning towards using the Java Monkey Engine (JME). It is a scenegraph library built on top of LWJGL, and is actively developed. It would provide a number of useful features, such as visibility culling, picking, impostor support, and Swing integration (it's terrain support is not very advanced though, but that is what I'm working on here).
Shaders may be utilized, although the 3D renderer should also work without them.
The 3D renderer might be using a lot of texture memory too, depending on the resolution settings specified by clients.