Weight Lifting
Long-running and resource-hungry programs are best deployed when the computer has nothing much to do. The loadwatch tool can make this happen.
Start and stop programs depending on the system load
HONGQI ZHANG, 123RF.com
The following is a very simple idea. During times when the load on the system is low, a daemon starts programs that have long computing times and therefore could otherwise interfere with user interaction.
In systems built along the lines of Unix, these helpful spirits are referred to as daemons. They putter along as processes in the background and spring into action as soon as a particular event or condition occurs.
The concept of taking advantage of low system load is easy to implement. Thanks to the /proc/ filesystem, important system information is made available to all processes. Even the average user can enter cat /proc/loadavg on a terminal and find out what the current system is or what it was. Listing 1 shows the output the user will receive.
Listing 1
Checking Load Average
§§nonmuber $ cat /proc/loadavg 2.83 3.02 2.95 2/580 1462 $
Entering man 5 proc takes you to an extensive man page where you can find out what the numbers mean by searching for loadavg . The first three entries refer to the load on the system produced by currently executing processes (running : designated with the letter "R") in the last 1, 5, and 15 minutes.
How Does It Work?
The idea behind loadwatch is very simple. The system load gets regularly checked. If something like a threshold event has occurred, then the selected process will be stopped and then reactivated once the load has fallen below a lower threshold (see the "Installation" box for more details).
Installation
For now, loadwatch can only been found in the Debian and Ubuntu repositories. For all other distributions, you need to perform the relatively straightforward task of compiling the program yourself from the source code.
You will find two relevant files online [1]. These include loadwatch_1.0+1.1alpha1.orig.tar.gz with the original source code and loadwatch_1.0+1.1alpha1-5.diff.gz , which has an additional patch. This patch modifies the source code yielding the most up-to-date version. For purposes of this project, you should save both files under /tmp/ .
First, decompress the original source code via tar xf loadwatch_1.0+1.1alpha1.orig.tar.gz . This generates the directory loadwatch-1.1/ to which you should then switch.
Then, you should load the patch via the original source code by typing
patch -p1 -i ../loadwatch_1.0+1.1alpha1-5.diff
See Listing 2.
This patch modifies all of the designated files including the man pages, makefile, etc. which is why you should control the output. Because the patch will be located in a higher level directory – in this case /tmp/ – the patch will be appropriately marked via -p1 when you use it in the working directory.
Now you can type ./configure && make && sudo make install to install the program in /usr/local/ in the usual way. It should execute without any difficulties.
Listing 2
Patching the Source Code
$ patch -p1 -i ../loadwatch_1.0+1.1alpha1-5.diff patching file Makefile.in patching file loadwatch.c patching file lw-ctl.c patching file debian/loadwatch.1 patching file debian/dirs patching file debian/README.debian patching file debian/changelog patching file debian/control patching file debian/copyright patching file debian/docs patching file debian/lw-ctl.1 patching file debian/rules patching file debian/watch
A second program lw-ctl , makes it possible to externally govern processes controlled by loadwatch . The lw-ctl tool communicates with loadwatch , thereby making it possible to do things like restart programs that have been stopped or stop those that are running.
A Practical Example
The following example will show how loadwatch and lw-ctr work. First, digiKam should be started, and its databases should be updated. Updating is especially important for the pHash fingerprints database, which is used to search for out of focus images and compare them. Updating the databases should not eat up all of the machine resources.
In most cases, it is not enough to do a simple start via loadwatch for programs like digiKam. Listing 3 shows why. The parameters selected here are actually fairly good for this computer. Generating and updating database is done quietly in the background without taking endless amounts of time. The changing states are documented on the terminal (Listing 4).
Listing 3
Starting digiKam
§§nonmuber $loadwatch -- digikam Wed Aug 26 16:00:56 2015: load too high, stopping. $
Listing 4
Changing States
§§nonmuber Wed Aug 26 17:00:55 2015: load low, continuing. Wed Aug 26 17:01:15 2015: load too high, stopping. Wed Aug 26 17:03:45 2015: load low, continuing. Wed Aug 26 17:03:55 2015: load too high, stopping. Wed Aug 26 17:04:55 2015: load low, continuing. digikam(8477)/digikam (core) digikam::PreviewLoadingTask::execute: Cannot extract preview for "32bit_8040963.tif" digikam(8477)/digikam (core) digikam::PreviewLoadingTask::execute: Cannot extract preview for "32bit_8040963.tif" Wed Aug 26 17:05:05 2015: load too high, stopping. ...
Just the initialization of digiKam takes up so much compute time that loadwatch has to switch into wait mode almost immediately. Therefore, it is a good idea to give an upper limit so that digiKam can at least start without immediately being forced to stop. The alternative would be to first execute the start in "RUN" mode via lw-ctr and then switch to "WATCH" mode. See below for details.
Experience teaches that a computer runs fairly well up to a load of 2. Thus, a good upper load limit should be about 1.8, or 90% of 2.0. Additionally, the lower limit should not be set too low, otherwise program execution will be prevented for long periods of time. This is because it takes a certain amount of time before the system load goes down sufficiently. The value for the lower limit is estimated to be 1.6. Controlling this value via lw-ctr is ensured by means of a socket in the /tmp/ directory.
Now the command to be used looks like this:
loadwatch -l 1.6 -h 1.8 -u /tmp/tst.loadwatch -- digikam
As expected, the program will now start easily. At first, the system load will not reach the critical limit because the threshold has been set high enough for the start to proceed without interruption.
The entry for updating the specified databases separately can be found under Maintenance in the Extras menu option (Figure 1). See the "Loadwatch Options" box for additional information.
Loadwatch Options
loadwatch is controlled with a small number of options. The syntax for loadwatch looks as follows:
loadwatch Options -- Program with Options
- The pseudo-option – can usually be elided when there is no danger that the options for the Program will collide with the options for loadwatch .
- -h <upper threshold> defines the level of the system load at which the Program will be stopped. The upper threshold is given as a decimal number.
- -l <lower threshold> defines the level of the system load at which the Program will restart. The lower threshold is given as a decimal number.
- -d <Delay> determines the loadwatch wait interval between two scans of the system load. Delay is given in seconds.
- -n <Copies> define how many copies are started by the programs controlled by loadwatch . The default setting is one.
- -u <Socket> defines the socket file for communicating with lw-ctr .
- -p <PID> makes it possible using the specified PID to retroactively control a program that is currently executing via loadwatch . You should be careful here. This does not always work properly and can at times lead to problems with the system such as blocked input devices. Fortunately, these problems seldom appear and then only with complex programs.
Figure 1: Digikam lets the user specify individual databases to be updated.
Manual Override
Sometimes, it's also necessary to manually access the mechanism triggered by loadwatch . For example, a program might not stop when loadwatch has been put it in sleep mode. In a situation like this, lw-ctr will help.
In order for loadwatch and lw-ctr to communicate, loadwatch will need to be invoked with the -u <Socket> option . Then, lw-ctr will send commands over this connection to loadwatch .
The simple lw-ctr program merely recognizes three commands and sends them on to loadwatch .
- RUN : Starts the program controlled by loadwatch
- STOP : Stops the program controlled by loadwatch
- WATCH : Activates the standard mode in loadwatch , where the thresholds set for loadwatch are observed
The syntax for lw-ctr looks like the following:
lw-ctr Socket Command
For example, you can say: lw-ctr /tmp/tst.loadwatch RUN . You will always need to provide a socket.
A second application should briefly be described here. Programs like MediathekView are not easy to control remotely via the command line. Thus, if you want to do something with this program, such as record a live stream during an inconvenient broadcast time, you can proceed as follows.
Start the program via loadwatch with a threshold high enough to prevent interruptions. Now MediathekView will perform all of the required initializations. This will often take several minutes. After this step, you can select the Live Stream and start recording.
You can stop the recording process with lw-ctr … STOP . Next, you can use sleep … or better yet the at command to send a START command to the program at the desired time. The program will quickly execute this command.
Conclusion
Dale E Martin and later Nicolas Boullis thought up a mechanism that is quite similar and that works well in many cases. It is not always easy to find suitable parameters for the thresholds, so a graphical interface with a slider would be a good solution. Also, one feature is still lacking: lw-ctr cannot adapt the thresholds for loadwatch . This is too bad.
Of course, it is possible to manually do everything that loadwatch does using the signals STOP and CONT. This can be done directly via kill on the command line or by means of qps or similar tools invoked from a graphical interface. This manual method, however, requires a lot more work for the user. Altogether, the effort-to-benefit ratio for loadwatch is extremely good. You should take the time to experiment with this tool.
Infos