Recently, I came across a requirement of serving 50,000 concurrent users within an OAM deployment. To hold this requirement, the first thing we need to make sure that the web tier scales well and can support such large number of concurrent requests at the same time.
Tuning of OHS configuration parameters is always a challenge, especially when it belongs to an OAM stack where the requirement is to support such high number of concurrent sessions. One has to be careful as incorrect configuration parameters may cause swapping in memory and instability of the server.
For OAM deployment, we always recommend Apache MPM worker module. It uses threads to serve requests and therefore is able to serve a large number of requests with fewer system resources than a process-based server module.
The following parameters mainly determine the performance of an OHS server for a MPM worker model and in this post we will recommend values for them so that it can be tuned properly.
We will consider an example to show how to find the values of these parameters. Firstly, one should start with ServerLimit, ThreadsPerChild and MaxClients. Let us assume that the machine where the OHS server will run has a memory of 6GB. After leaving some space to other system related process and space for MaxSpareThreads, a safe estimate of available memory for httpd.worker thread process will be 4.5 GB. Normally the process size of an httpd.worker process will be around 25 MB (This can vary, so you need to determine the size of the average OHS process in your machine by starting the OHS server with the default settings). Thus in our example, without having a swapping of memory the maximum value of ServerLimit can be 4500/25 = 180. A reasonable limit for ThreadsPerChild in an OHS server can be set to 64. So MaxClients can be set to = 180 * 64 = 11520. This is maximum concurrent clients that the OHS can handle in our example.
However, OHS has a maximum allowed value (a hard limit) of MaxClients of 8192 (8K). http://docs.oracle.com/cd/E23943_01/core.1111/e10108/http.htm.
So in our example, we cannot set MaxClients to 11520, instead of we will set the value to 8000. Now we will back calculate the ServerLimit and that will be 8000/64=125. After determining MaxClients, ServerLimit and ThreadsPerChild, we can estimate the values for other parameters. The following table provides a guideline how to determine those.
|KeepAliveTimeout||31||In heavily loaded server it is not advised to keep this value very high. Keeping this value very high will make number of threads in ideal condition and waiting for connections. For OHS server configured with OAM server it is advised to keep this value to 31 seconds.|
|StartServers||20||Specifies the number of child server processes created on startup. Note that the following parameters are inter-related |
|ServerLimit||125||Number of maximum httpd worker process can run|
|MaxClients||8000||It should be multiply of ThreadsPerChild.It should not exceed (ServerLimit*ThreadsPerChild)Remember, OHS has a hard limit of 8192 for this.|
|ThreadLimit||72||Specifies the upper limit on the number of threads that can be created under a server (child) process. This value overrides the ThreadsPerChild value if that value is greater than the ThreadLimit value. This is used to control the maximum number of threads created per process to avoid conflicts/issues|
|ThreadsPerChild||64||This is the suggested value to support high loads for OAM.|
|MinSpareThreads||64||Advised to be equal to ThreadsPerChild|
|MaxSpareThreads||96||Empirically, it should be around 1.5 times of ThreadsPerChild so it can spawn 1 process per second|
|MaxRequestsPerChild||10240||To avoid memory leak in server it is advised to put this value from 8000 to 12000.|
It should be noted that the values for ServerLimit and MaxClients are always need to be calculated with respect to the requirements of concurrent sessions that the OHS need to handle. The above settings for ServerLimit and MaxClients are for maximum 8K concurrent sessions OHS can handle. If the requirement is less, one should re-calculate these values as required.
Another thing to be remembered, that some parameters like ServerLimit, MaxClients etc. are only available for OHS running on Unix based OS and not for WinNT as the later uses a single process multithreaded model.
So now let us come back to the original requirements to serve 50K concurrent connections. How many OHS servers do we need for this? You are right, the arithmetic is simple here, and one would need at least 7 OHS servers in this case.