A Short History of Unix

Stephen W. NuchiaCenter for Information Security

The University of Tulsa

Stephen W. NuchiaCenter for Information Security

The University of Tulsa

Chapter 1: Prehistory

● General Electric, the people who brought you the electric power industry, manufactured large computers in the '60s.

● AT&T, the people who brought you the telephone system, had a network.

● Bell Labs, the people who brought you the transistor, Information Theory, optical communications, and military radar.

● Were collaborating on a plan ...

Multics

● The concept was that computing would become a utility in the future. Much like cable TV has.

● Ordinary people would subscribe and gain access (over AT&T's network) to a user-friendly environment running on the next generation of GE mainframes.

● Secure cooperative use of the machine was an explicit design goal of Multics.

The Bell Labs Group● Bell labs was, for most of the 20th century, a very

special place.● Ritchie, in his 1983 Turing award lecture,

presciently foreshadowed its downfall:– On the other hand, in smaller companies, even the

most vigorous research support is highly dependent on market conditions. The New York Times, in an article describing Alan Kay's passage from Atari to Apple, notes the problem [...] ``When I left last month it was clear that they would be putting their efforts into the short term. [...] I guess the tree of research must from time to time be watered with the blood of bean counters.''

Digital Equipment Company● DEC made a line of small(ish) computers as well

as supplying sub assemblies to IBM. When IBM made them an offer they couldn't refuse, they responded by ceasing to call their machines computers. Thus was born the PDP-N series of Peripheral Data Processors.

● Most scientific and technical organizations had them laying around. They made the machines of choice for most early Unix sites.

● Much later, DEC became an important direct force in the Unix marketplace.

Dennis Ritchie

Quite possibly the nicest person I've ever met, and a truly great programmer. He was the scribe throughout the AT&T phase of Unix history as well as one of its most prolific visionaries.

Ch. 2: From The Evolution of the Unix Time-sharing System (1979)

For computer science at Bell Laboratories, the period 1968-1969 was somewhat unsettled. The main reason for this was the slow, though clearly inevitable, withdrawal of the Labs from the Multics project. To the Labs computing community as a whole, the problem was the increasing obviousness of the failure of Multics to deliver promptly any sort of usable system, let alone the panacea envisioned earlier. For much of this time, the Murray Hill Computer Center was also running a costly GE 645 machine that inadequately simulated the GE 635. Another shake-up that occurred during this period was the organizational separation of computing services and computing research.

Ritchie, contd.

From the point of view of the group that was to be most involved in the beginnings of Unix (K. Thompson, Ritchie, M. D. McIlroy, J. F. Ossanna), the decline and fall of Multics had a directly felt effect. We were among the last Bell Laboratories holdouts actually working on Multics, so we still felt some sort of stake in its success. More important, the convenient interactive computing service that Multics had promised to the entire community was in fact available to our limited group, at first under the CTSS system used to develop Multics, and later under Multics itself.

Ritchie, contd.Even though Multics could not then support many users, it could support us, albeit at exorbitant cost. We didn't want to lose the pleasant niche we occupied, because no similar ones were available; even the time-sharing service that would later be offered under GE's operating system did not exist. What we wanted to preserve was not just a good environment in which to do programming, but a system around which a fellowship could form. We knew from experience that the essence of communal computing, as supplied by remote-access, time-shared machines, is not just to type programs into a terminal instead of a keypunch, but to encourage close communication.

Ritchie, contd.

Thus, during 1969, we began trying to find an alternative to Multics. The search took several forms. Throughout 1969 we (mainly Ossanna, Thompson, Ritchie) lobbied intensively for the purchase of a medium-scale machine for which we promised to write an operating system; the machines we suggested were the DEC PDP-10 and the SDS (later Xerox) Sigma 7. The effort was frustrating, because our proposals were never clearly and finally turned down, but yet were certainly never accepted. Several times it seemed we were very near success.

Ritchie, contd.

The final blow to this effort came when we presented an exquisitely complicated proposal, designed to minimize financial outlay, that involved some outright purchase, some third-party lease, and a plan to turn in a DEC KA-10 processor on the soon-to-be-announced and more capable KI-10. The proposal was rejected, and rumor soon had it that W. O. Baker (then vice-president of Research) had reacted to it with the comment `Bell Laboratories just doesn't do business this way!'

Ritchie, contd.Actually, it is perfectly obvious in retrospect (and should have been at the time) that we were asking the Labs to spend too much money on too few people with too vague a plan. Moreover, I am quite sure that at that time operating systems were not, for our management, an attractive area in which to support work. They were in the process of extricating themselves not only from an operating system development effort that had failed, but from running the local Computation Center. Thus it may have seemed that buying a machine such as we suggested might lead on the one hand to yet another Multics, or on the other, if we produced something useful, to yet another Comp Center for them to be responsible for.

Ritchie, contd.

Besides the financial agitations that took place in 1969, there was technical work also. Thompson, R. H. Canaday, and Ritchie developed, on blackboards and scribbled notes, the basic design of a file system that was later to become the heart of Unix. Most of the design was Thompson's, as was the impulse to think about file systems at all, but I believe I contributed the idea of device files. Thompson's itch for creation of an operating system took several forms during this period; he also wrote (on Multics) a fairly detailed simulation of the performance of the proposed file system design and of paging behavior of programs.

Ritchie, contd.

In addition, he started work on a new operating system for the GE-645, going as far as writing an assembler for the machine and a rudimentary operating system kernel whose greatest achievement, so far as I remember, was to type a greeting message. The complexity of the machine was such that a mere message was already a fairly notable accomplishment, but when it became clear that the lifetime of the 645 at the Labs was measured in months, the work was dropped.

Ritchie, contd.

Also during 1969, Thompson developed the game of `Space Travel.' First written on Multics, then transliterated into Fortran for GECOS (the operating system for the GE, later Honeywell, 635), it was nothing less than a simulation of the movement of the major bodies of the Solar System, with the player guiding a ship here and there, observing the scenery, and attempting to land on the various planets and moons. The GECOS version was unsatisfactory in two important respects: first, the display of the state of the game was jerky and hard to control because one had to type commands at it, and second, a game cost about $75 for CPU time on the big computer.

Ritchie, contd.It did not take long, therefore, for Thompson to find a little-used PDP-7 computer with an excellent display processor; the whole system was used as a Graphic-II terminal. He and I rewrote Space Travel to run on this machine. The undertaking was more ambitious than it might seem; because we disdained all existing software, we had to write a floating-point arithmetic package, the pointwise specification of the graphic characters for the display, and a debugging subsystem that continuously displayed the contents of typed-in locations in a corner of the screen. All this was written in assembly language for a cross-assembler that ran under GECOS and produced paper tapes to be carried to the PDP-7.

Ritchie, contd.

Space Travel, though it made a very attractive game, served mainly as an introduction to the clumsy technology of preparing programs for the PDP-7. Soon Thompson began implementing the paper file system (perhaps `chalk file system' would be more accurate) that had been designed earlier. A file system without a way to exercise it is a sterile proposition, so he proceeded to flesh it out with the other requirements for a working operating system, in particular the notion of processes.

Ritchie, contd.

Then came a small set of user-level utilities: the means to copy, print, delete, and edit files, and of course a simple command interpreter (shell). Up to this time all the programs were written using GECOS and files were transferred to the PDP-7 on paper tape; but once an assembler was completed the system was able to support itself. Although it was not until well into 1970 that Brian Kernighan suggested the name `Unix,' in a somewhat treacherous pun on `Multics,' the operating system we know today was born.

Chapter 2a: Berkeley● It has been reported that Ken Thompson hacked

out the original Unix file system in one weekend, while his wife was visiting relatives at Berkeley.

● Thompson later took a sabbatical at UCB, and introduced Unix to their CS people.

● From Leffler et al: ``The second important distinction of Unix was its early release from Bell Laboratories to other research environments in source form. By providing source, the system's founders ensured that other organizations would be able not only to use the system, but also to tinker with its inner workings. …

Leffler, cont'd.

... The ease with which new ideas could be adopted into the system always has been the key to changes that have been made to it. Whenever a new system that tried to upstage Unix came along, somebody would dissect the newcomer and clone its central ideas into Unix. The unique ability to use a small, comprehensible system, written in a high-level language, in an environment swimming in new ideas led to a Unix system that evolved far beyond its humble beginnings.’’

Chapter 3: Commercial Evolution

● Companies seeking to exploit the demand for Unix-based systems chose either AT&T or BSD as their starting point. Since the Unix Support Group at AT&T did not incorporate ``Berkeleyisms'' into their source tree, this led to The Great Schism

● Trivia Question: What does SUN (as in Sun Microsystems) stand for?

Interlude: Features & Characteristics or, What Makes Unix so Great?

● #1: Great Documentation● Written by top-shelf research programmers, for

programmers.● Contrast with Microsoft fighting disclosure of the

Windows API.● It's fun to read.● Table of contents of v7 manual follows point 4.

What Makes Unix Great

● #2: Profoundly simple API● Everything is a file (except processes)● A very manageable set of system calls● An easy-to-live-with process model

Seventh Edition API

● File System– Access– Chdir– Chmod– Chown– Chroot– Close– Dup– Ioctl– Link– Lseek

– Mknod– Mount– Open– Pipe– Read– Stat– Sync– Umask– Unlink– Utimes– Write

Seventh Edition API

● Process Control– Brk– Execve– Exit– Fork– Kill– Profil– Ptrace– Signal– Wait

● Security/Misc– Access– Acct– Getgid/getegid– Getuid/geteuid– Setgid– Setuid– Time

What Makes Unix Great

● #3: Portability● Written in C, arguably a high-level language.● Licensable for finite money in source form.

– Consent decree to MFJ, AT&T not competing with licencees. Unique in this respect = startup appeal.

● Led to wide commercialization– Bill Gates' Mom and Xenix– Amdahl and Unix

What Makes Unix Great

● #4: Accretion● Attractive environment for programmers => lots

of good programmers use it => many good programs (at least from a programmer's perspective!) written (portably!) for Unix.

● Many of these programs became part of the base Unix system.– Trivia question: what criteria were used to partition

commands between /bin and /usr/bin?

What Makes Unix Great

● #5: The Shell● Steve Bourne's experiment in functional

programming leads to the pipes-and-filters paradigm, the surround-and-tinker programming methodology, and a very programmer-friendly user environment.

● The idea of a command line processor that is a full-featured and conceptually well-founded programming language interpreter was novel and terribly important. Most people still don't get it.

Chapter 4: Opening the Sourceand The Lesser Schism

● ``Big'' sites had source. Mystique creates a desire for source in mere mortals.

● The advent of PC machine with 80286 processors put virtual memory-capable machines (comparable to PDP-11 architecture) in their hands for a tiny fraction of the cost of a source license. => Demand not economically satisfiable.

● Xenix available, but: it is AT&T flavored, not source, and not cool.

Use the Source, Luke

● Most of the good stuff was contributed software, much of it more-or-less open source already.

● Stallman began the Open Source movement (Software should be free. As in Free Speech, not as in Free Beer. -approximate quote). His articulation of the philosophy and economics of free software gave the demand form and voice.

● UCB's mission & commercial interests led to schizophrenic paralysis, but the CSRG began a review process.

Let My Bits Go!● They concluded that only a tiny fraction of BSD

Unix was ``contaminated'' with AT&T-licensed code and released the rest.

● Unfortunately, some of the contaminated source was in the kernel's hindbrain. It was not possible to build a runnable system from the freed source.

● A widespread effort was begun to replace the missing modules.

● The effort focused on making a kernel that was bootable on off-the-shelf PC hardware.

The Jolix War

● This was a seminal event in the cultural evolution of the free software movement, but it was crippled by– The stupidity of the PC architecture and– The personalities involved.

● More time was spent arguing about which camp had the true Unix nature than on writing code.

A Bolt from the Blue• Meanwhile, writing operating

systems had gotten easy. At many schools, a basic kernel is written as a standard undergraduate exercise.

So, Linus Torvald (Helsinki University) writes a kernel in his spare time. Linux becomes usable and unambiguously uncontaminated before FreeBSD happens.

The rest is history.

Linux Takes Over. Long Live Unix

● The army of salivating developers adopts Linux and begin ``accreting'' good stuff for it. Soon they had essentially all the classic Unix stuff ported or replaced, and Linux became the feature leader of the Unix family.

● Linux is Unix, mostly of the AT&T flavor. Same philosophy (except on documentation), same herd-of-cats development culture. Different kernel architecture, so no common drivers.

Linux Innovations

● Linux has innovated in administrative as well as user-oriented features, but its culture lacks the heavy industrial component that the mainstream (or old-school) Unix community had (remember Multics). Thus, security is not always a well-thought-out part of their innovations.

● Questions?