Message-ID: <2107720164.9361.1413864938951.JavaMail.firstname.lastname@example.org> Subject: Exported From Confluence MIME-Version: 1.0 Content-Type: multipart/related; boundary="----=_Part_9360_1421627099.1413864938951" ------=_Part_9360_1421627099.1413864938951 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Content-Location: file:///C:/exported.html
The blocking nature of the standard servlet API makes it impossible to i= mplement web applications that can guarantee some level of Quality of Service (QoS). Threads and memory are limited reso= urces within a servlet container, yet with the standard servlet API, the on= ly way to handle a HTTP request is with a thread allocated for the entire d= uration of the request. If a request is of low priority, or if other resour= ces needed by the request are not available, then it is not possible to reu= se the thread allocated to the request for high priority requests or reques= ts that can proceed.=20
Jetty susports Suspendab= le Requests, which allows non-blocking handling of HTTP requests, so th= at that threads may be allocated in a managed way to provide application sp= ecific QoS. The QoSFilter is a utility servlet fi= lter that uses Suspendable = Requests to implement some QoS features.=20
Web application frequently uses JDBC Connection pool to limit the simult= aneous load on the database. This protects the database from peak loads, bu= t makes the web application vulnerable to thread starvation.=20
Consider a thread pool with 20 connections, being used by a web applicat= ion that that typically receives 200 request per second and each request ho= lds a JDBC connection for 50ms. Such a pool can service on average 200*20*1= 000/50 =3D 400 requests per second.=20
However, if the request rate raises above 400 per second, or if the data= base slows down (due to a large query) or becomes momentarily unavailable, = the thread pool can very quickly accumulate many waiting requests. If (for = example) the website is slashdotted or experiences some other temporary bur= st of traffic and the request rate rises from 400 to 500 requests per secon= d, then 100 requests per second join those waiting for a JDBC connection. T= ypically, a web servers thread pool will contain only a few hundred threads= , so a burst or slow DB need only persist for a few seconds to consume the = entire web servers thread pool. This is called thread starvation.=20
The key issue with thread starvation, is that it effects the entire web = application (and potentially the entire web server). So that even if the re= quests use the database are only a small proportion of the total request on= the web server, all requests are blocked because all the available threads= are waiting on the JDBC connection pool. This represents non graceful degr= adation under load and provides a very poor quality of service.=20
Consider a web application that is under extreme load. This load may be = due to a popularity spike (slash dot), usage burst (christmas or close of b= usiness), or even a denial of service attack. During such periods of load, = it is often desirable to not treat all requests as equals and to give prior= ity to high value customers or administration users.=20
The typical behaviour of a web server under extreme load is to use all i= t's threads servicing requests and to build up a back log of unserviced req= uests. If the backlog grows deep enough, then requests will start to timeou= t and the users will experience failures as well as delays.=20
Ideally, the web application should be able to examine the requests in t= he backlog, and give priority to high value customers and administration us= ers. But with the standard blocking servlet API, it is not possible to exam= ine a request without allocating a thread to that request for the duration = of it's handling. There is no way to delay the handling of low priority req= uests and if the resources are to be reallocated, then the low priority req= uests must all be failed.=20
The Quality of Service Filter (QoSFilter) uses Suspendable Requests to avoid thread starvation,= prioritize requests and give graceful degredation under load, so that a hi= gh quality of service can be provided. The filter can be applied to specifi= c URLs within a web application and will limit the number of active request= s being handled for those URLs. Any requests in excess of the limit are sus= pended.=20
When a request completes handling the limited, URL, one of the waiting r= equests is resumed, so that it may be handled. Priorities may be given to e= ach suspended request, so that high priority requests are resumed before lo= w prority requests.=20
These JAR files must be available within WEB-INF/lib:=20
Note that the QoSFilter requires at least Java 5, unlike the rest of Jet= ty-6 which only requires Java 1.4.=20
This configuration will process only five requests at a time, servicing = more important requests first, and queuing up the rest:=20 =20
You can use the
<filter-mapping> syntax to map the Qo=
SFilter to a servlet, either by using the servlet name, or by using a URL p=
attern. In this example, a URL pattern is used to apply the QoSFilter to ev=
ery request within the web application context:
The following init parameters are supported:=20
the maximum number of requests to be service= d at a time.
the maximum valid priority that can be assig= ned to a request. A request with a high priority value is more important th= an a request with a low priority value.
length of time, in milliseconds, to wait whi= le trying to accept a new request. Used when the maxRequests limit is reach= ed.
length of time, in milliseconds, that the re= quest will be suspended if it is not accepted immediately. If not set, the = container's default suspend period will be used.
The default request priorities assigned by the QoSFilter are:=20
To customize the priority, subclass QoSFilter and then override the getP= riority(ServletRequest request) method to return an appropriate priority fo= r the request. Higher values have a higher priority. You can then use this = subclass as your QoS filter. Here's a trivial example:=20