Innovation in the Linux environment

The Linux operating system kernel will celebrate its 25th birthday this coming summer. During its lifetime, a fantastic variety of digital ecosystems have developed around both the kernel and the GNU software suite. In the meantime, developers continue to work carefully and in well-defined increments on the kernel itself. There are no huge leaps forward. Instead, steady improvement done in manageable steps is preferred.

The term "Linux" has taken on various meanings in popular usage. In this article, however, the term refers to the Linux kernel and the surrounding distributions. Here, I will look at the question of who has set the technology and idealogical standards for development in the past, and who will continue to do so in the future.

What Worked Before

The development occurring over the past few years reflects the adage that it's not always necessary to reinvent the wheel. Developers are using and extending older technologies in new ways. An example of this is systemd, a tool that has become established in the kernel, using functions like namespaces and Cgroups as its essential components (Figure 1).

Figure 1: Systemd Nspawn can be used to isolate applications in a container.

Wayland [1] is a new display protocol that also exemplifies modernization by evolution. Like many core functions, Docker and CoreOS rely on time-proven ingredients like BSD jails [2] and Solaris zones [3]. The filesystem Btrfs picks up the torch from SunZFS [4], making use of RAID and integrating Snapshots [5], as well as the Solaris zones model.

Even /usr merge [6], a scheme already executed in some distributions (Figure 2) and on the cusp of being executed in others, takes it cues from the direction Solaris took some 15 years ago. The underlying problem and the contemporary directory tree that the Filesystem Hierarchy Standard (FHS) [7] is built on date back to the mundane storage problems that Unix creators Ken Thompson and Dennis Ritchie encountered back in 1970 [8].

Figure 2: The /usr merge scheme simplifies the FHS directory tree.

Therefore, it's a good idea to investigate the roots of ideas that appear to be innovative. Sometimes, all that is actually involved is a long-delayed correction to a past shortcoming.

Many genuine innovations have their start inside a corporation. At the very least, corporate settings have been responsible for adding improvements. Thanks to this pattern of innovation, the large distributions from the 1990s continue to survive. SUSE originated in 1996 with roots in Softlanding Linux System (SLS) [9] (Figure 3) and Slackware [10]. Debian had already been founded by 1993. In that same year, Red Hat was launched by developers in the United States. Later, Red Hat conceived an industrial strength distribution – Red Hat Enterprise Linux (RHEL) – and drove it forward in parallel with development of the community distribution Fedora.

Figure 3: SLS counts as the first complete Linux distribution.

Ubuntu appeared for the first time in 2004, along with the promise that it would revolutionize the Linux desktop. Since then, people have been waiting not only for the proverbial Godot but also for the "Year of the Desktop." And, they have been waiting in vain. Linux has developed in line with its usage. It is still not a unified product like Microsoft in spite of the many innovations that have been made.

Impulses Emanating from Debian

Aside from Debian, the distributions referred to belong to companies that earn money with Linux. This explains the ongoing 23 years of performance by Debian. Even today, there exists a flock of countless Debian developers who have no financial interest in the project; they maintain it as a Do-o-cracy (in which you have a say if you participate – and the more you participate, the greater your say).

The developers work together according to a set of rules and guidelines. The most important among these are the Debian Manifesto [11] and the Social Contract [12]. The latter is a type of contract, which, among other provisions, incorporates the Debian Free Software Guidelines (DSFG) [13]. The developers adopted these guidelines in 1997 and revised them in 2004.

The rules and guidelines for the project have taken on importance beyond the scope of their original use. Many derivative projects use them as a standard for working on the development of open source software and also for getting along in a developer community. In this way, Debian has contributed a great deal in terms of ideology for working on open source software.

The approach on the technology side of the project is by contrast leisurely. Stability is the primary focus – not a rush for innovation's sake found elsewhere. Even so, Debian has recently shown itself more inclined to adopt innovation from the outside. The change to systemd (Figure 4), however painful, is an example. Currently, developers are laying the groundwork for /usr merge . As usual, a lot of discussion is taking place, yet things are moving forward.

Figure 4: Systemd service files are easier to maintain than SysV init scripts.