moreutils completes the shell's toolbox

isutf8

Text files on current Linux systems normally exist as UTF-8 coded data. If this is not the case, when you process them, unexpected stuff happens. It is a good idea to first check to see whether the coding is correct. You can test files using isutf8 from the moreutils package to determine whether the file is indeed coded as UTF-8. If the coding is OK, then there will not be any output. If not, the software will indicate text that does not conform (Listing 8).

$ isutf8 1bicec.tmp
1bicec.tmp: line 1, char 15, byte 14: Expecting bytes in the following ranges: 00..7F C2..F4.

The actual evaluation is performed as usual with the return values of the command. The tool outputs zero for UTF-8 files. There are a few options for modifying the behavior of isutf8 . For example, -q , or --quiet , suppresses the message that a file does not conform to UTF-8. Using -l , or --list , you can specify that the software outputs only the names of nonconforming files. Using -i , or --invert , only those files that conform to UTF-8 are shown.

Timestamps

Timestamps are made up of short strings containing the current time. These are used for things like dating entries in the log files. The moreutils package comes with the ts tool. It has a command that adds a timestamp to the beginning of each line of STDOUT output. These timestamps let you monitor things like how long a command is taking to complete before passing control to the next command.

Take the script, tstest.sh , in Listing 9, for example. Listing 10 shows what the output looks like using ts combined with different options.

echo "Here we go."
sleep 3
echo "three seconds later."
sleep 2
echo "five Seconds have passed."

$ bash tstest.sh | ts
Nov 28 14:23:15 Here we go.
Nov 28 14:23:18 Three seconds later.
Nov 28 14:23:20 five seconds have passed.
$ bash tstest.sh | ts "%c"
Mo 28 Nov 2016 14:30:35 CET Here we go.
Mo 28 Nov 2016 14:30:38 CET three seconds later.
Mo 28 Nov 2016 14:30:40 CET five seconds have passed.
$ bash tstest.sh | ts -s
00:00:00 Here we go.
00:00:03 three seconds later.
00:00:05 five seconds have passed.
$ bash tstest.sh | ts -i
00:00:00 Here we go.
00:00:03 Three seconds later.
00:00:02 five seconds have passed.

The tool constructs these timestamps with the strftime function. This function is like the one used in the date command. It is possible to adapt the output using placeholders in the form of short character strings, such as %s and %D . Each of these strings represents a particular aspect. You can learn more about the possibilities with man 3 strftime . Table 2 contains a summary of the most important character strings.

By default, ts uses the format "%b %d %H:%M:%S" for output. You can specify another format with quotation marks for the last argument or as the argument for the -s option.

There are some additional options for controlling the tool. The software converts given timestamps into relative amounts with the -r option. In testing however, this option only works for stamps that have been generated in the default format. With -i , the software computes the difference between the current and the previous timestamp.