Performance tuning for web servers

The Apache HTTP server is the most popular web server in use on the Internet [1]. The first version appeared on the market two decades ago, in April 1995. Due to constant development efforts by the Apache Software Foundation, the server is still in use today. It is known for its modular architecture as well as its rich functionality.

However, the competition has not been asleep. For example, the market share of Nginx has been growing quite a bit in the past few years. As a result, one goal of the Apache Software Foundation is to improve performance of the Apache HTTP server to ensure that its performance compares well with Nginx. And, in fact, version 2.4 has caught up somewhat in terms of performance.

This becomes apparent when comparing the performance of consecutive versions of the Apache web server running on Debian Wheezy (2.2.22) and Debian Jessie (2.4.10). Figure 1 shows that version 2.4 processes one million requests at a rate that is about 20 seconds faster than the previous 2.2 version.

Figure 1: The Apache HTTP server in its standard configuration has faster performance on Debian Jessie than on Debian Wheezy.

On Jessie, this performance increase was achieved by activating the Multi-Processing Module (MPM) Worker in place of the MPM Event in order to use the same module. I tested direct access via localhost in a virtual box (VM) with 512MB each of RAM. I used the Apache benchmarking tool ab each time to access a text file containing a simple string: hello world .

The performance of a server when delivering web pages depends on many factors. For example, actual web applications mostly use script languages like PHP and Perl. Additionally, there are database queries via MySQL and the like. All these aspects are the real reason for slow page delivery by a web server. The Apache web server by itself usually does not cause these slowdowns.

It is a good idea to maintain different versions of your configuration by using etckeeper [3]. This will keep you from accidentally destroying your working configuration by repeated tuning activities. You should check completed configuration changes for syntactic accuracy with apache2ctl -t before executing an Apache HTTP reload/restart.

RAM, RAM, and More RAM

Every Apache HTTP process needs several megabytes of working storage. Therefore, it is essential that the server has enough working storage available to deal with a large number of requests. If there is not enough RAM, the server will begin to swap and transfer the working memory onto the hard disk. Even in these times of SSDs, the throughput of hard drives is still orders of magnitude slower than that of RAM. The web server administrator should definitely take steps to avoid swapping.

With Linux, more RAM also means that the kernel can hold a larger page cache. This makes for an incredible increase in the speed of I/O requests. The web server admin should also consider this when allocating RAM. The Apache configuration also has to be coordinated with available RAM. If there are too many processes, then available RAM will quickly be consumed. The formula in Figure 2 can be used for specifying the MPM Prefork.

Figure 2: The number of processes on an Apache server should correspond to the size of the available RAM.

Web server administrators can determine component values via the top tool (Figure 3). If you press Shift+M, the tool sorts the processes according to memory usage. The actual main memory requirements of the processes appear in the RES column. There are 16GB of RAM available altogether in the example provided here. MySQL needs 39MB. The largest Apache HTTP process needs 22MB. The web server admin can ignore the DNS software BIND; it barely uses 1MB of RAM.

Figure 3: The preinstalled Linux tool top indicates the processes running on the HTTP server.

Therefore, if the formula presented above is applied, allocating 100MB for the operating system, 50MB for MySQL, and 2GB for the page cache, then this yields a value of 644 for MaxRequestWorkers . This variable determines how many concurrent requests the server can handle. It is a good idea to build in a bit of reserve and therefore choose the value 400 for the number of workers.

As an alternative, it is possible to use assistants to calculate this for you; see the "apachebuddy.pl" box for more information.

01 root@testserver:~# wget apachebuddy.pl -O apachebuddy.pl
02 root@testserver:~# perl apachebuddy.pl
03 ########################################################################
04 # Apache Buddy v 0.3 ###################################################
05 ########################################################################
06 Gathering information...
07 We are checking the service running on port 80
08 The process listening on port 80 is /usr/sbin/apache2
09 The process running on port 80 is Apache/2.2.16 (Debian)
10 Apache has been running 7d 01h 39m 11s
11 The full path to the Apache config file is: /etc/apache2/apache2.conf
12 Apache is using prefork model
13
14 Examining your Apache configuration...
15 Apache runs as apache
16 Your max clients setting is 150
17
18 Analyzing memory use...
19 Your server has 16024 MB of memory
20 The largest apache process is using 30.59 MB of memory
21 The smallest apache process is using 15.01 MB of memory
22 The average apache process is using 20.53 MB of memory
23 Going by the average Apache process, \
   Apache can potentially use 3079.51 MB RAM (19.22 % of available RAM)
24 Going by the largest Apache process, \
   Apache can potentially use 4588.51 MB RAM (28.64 % of available RAM)
25
26 Generating reports...
27 ### GENERAL REPORT ###
28
29 Settings considered for this report:
30
31         Your server's physical RAM:             16024MB
32         Apache's MaxClients directive:          150
33         Apache MPM Model:                       prefork
34         Largest Apache process (by memory):     30.59MB
35 [ OK ]  Your MaxClients setting is within an acceptable range.
36         Max potential memory usage:             4588.5 MB
37
38         Percentage of RAM allocated to Apache   28.64 %
39
40 -----------------------------------------------------------------------
41 -----------------------------------------------------------------------

Multi-Processing Modules

Apache HTTP Version 2.4 supports three different Multi-Processing Modules on Linux. They are:

• Prefork
• Worker
• Event

Additionally, version 2.4 makes it possible for the server to load MPMs during runtime.

The MPM Prefork module, mod_mpm_prefork.so , does not make use of threads. This means that each server request will be handled by its own process. When using PHP as an Apache module, this is usually the only option for running the server because, in PHP, many third-party provider libraries are not thread safe [4].

Fast CGI and PHP-FPM make it possible to use a threaded MPM together with PHP. However, it is not clear from a performance perspective that this is a good idea. Due to the loss of performance caused by the PHP integration via Fast CGI, the advantages of using threaded MPM are partially lost [5]. The following options are relevant to MPM Prefork:

• StartServers
• MinSpareServers/MaxSpareServers
• MaxRequestWorkers
• ServerLimit
• MaxConnectionsPerChild

The StartServers option specifies the number of processes that will initially launch when the server starts. MinSpareServer and MaxSpareServers maintain an appropriate number of spare server processes based on the number of incoming requests. MaxRequestWorkers , referred to as MaxClients up to version 2.3.13, limits the maximum number of processes and as a result also the number of concurrent requests. You should also pay attention to the directive ServerLimit . This sets the upper limit for MaxRequestWorkers with the standard value being 256 .

The ServerLimit directive will have to be raised in parallel with an increase in the value for MaxRequestWorkers . MaxConnectionsPerChild sets the number of connections that may be handled by a single process. In a perfect world, this value would be 0, indicating that the server processes would handle an unlimited number of requests. However, when there are complex applications it makes sense to restart the processes after every few hundred connections. This releases the RAM area that is used by the process and prevents memory leaks.

If there are not enough configured MaxRequestWorkers , the message from Listing 2 will appear in the log file.

[Fri Jun 05 13:15:24.760818 2015] [mpm_prefork:error] \
  [pid 1649] AH00161: server reached MaxRequestWorkers setting, \
  consider raising the MaxRequestWorkers setting.

The web server administrator doesn't need to worry about using threaded MPMs when no PHP module is present. By using threaded, the server handles a request via a thread instead of a process. Because a thread uses less overhead than a process, this approach enables the server to achieve a better performance.

Two threaded MPMs are available on Apache. These are MPM Worker, or mod_mpm_worker.so , which was introduced in version 2.0, and Event mod_mpm_event.so , which was added with version 2.2 and which has been considered stable since version 2.4.

The most important configuration options for MPM Worker and Event are identical:

• ThreadsPerChild
• MinSpareThreads/MaxSpareThreads
• MaxRequestWorkers
• ServerLimit

ThreadsPerChild specifies how many threads may be created by a single process. MinSpareThreads/MaxSpareThreads function analogously to the MinSpareServers/MaxSpareServers directives referred to above.

MaxRequestWorkers is used to limit the total number of threads. The default value for ServerLimit is 16 with threaded MPMs. Multiplying ThreadsPerChild with a standard value of 25, by the ServerLimit , with a standard value of 16, gives the upper limit for the number of threads. As a result, 400 is the default setting of the threads value for MaxRequestWorkers .