Asynchronous Servlets: Suspendable Requests.
Most non trivial web applications need to wait at some stage during the processing of a HTTP request, examples of waiting request handling include:
- waiting for a resource (eg. thread, JDBC Connection) to be available before processing the request.
- waiting for an application event in an Ajax Comet application (eg Chat message, price change, etc.)
- waiting for a response from a remote service (eg RESTful or SOAP call to a web service).
The current servlet API (<=2.5) supports only a synchronous call style, so that any waiting that a servlet needs to do must be with blocking. Unfortunately this means that the thread allocated to the request must be held during that wait with all its resources including kernal thread, stack memory and often pooled buffers, character converters, EE authentication context, etc. It is wasteful of system resources to be held while waiting.
blocking servlet example
Consider a modest web application with requests taking 10ms of CPU to handle and generate the response.If run a 2 CPU machine, then you could expect close to 2*1000/10 = 200 requests per second to be handled with only 2 threads.
However, if the handling of the request needs to wait for resources, more threads will be required. If for example, the average handling spent 5ms waiting for synchronized locks and 15ms waiting for IO to flush write the response, then total time per request will be 35ms and the thread pool will need to contain 200*30/1000 = 6 threads.
Now typically a web application will interact with a database of some kind, often on a remote machine, so database operations can often take 50ms or more, during which time the servlet has to wait for the response. Thus total request time is increased to 80ms and 200*(30+50)/1000 = 16 threads are needed.
If that web application is modified so that it requires a remote web service (eg a SOAP or RESTful call), then it is not unusual for a web service to take many hundred milliseconds to produce a response. Assuming only 100ms for a web service request, the total request time is increased to 180ms and now 200*(30+50+100)= 36 threads are needed.
If the database is needed both before an after the web service request, that means that the JDBC connection will be held by the application for 50+100 ms,
so 200*(50+100) = 30 database connections are needed. Frequently database connections are limited to protect the DB from bursts of activity, so it not uncommon for a JDBC connection pool to be used. If the pool contained 20 JDBC connections,