Apollo Computing

On 20 July, it will be the 50 years anniversary of the first landing of humans on the moon. 1969. Apollo 11. I asked myself, What computer and software technologies did NASA employ back then? It was the time when computers filled entire rooms, and for sure NASA used these behemoths for the operations on the ground. But what was flying in the two spacecrafts, namely the Command Module (CM) and the Lunar Module (LM), where each kilogram of weight, and Watt of power consumption, counts?

Given the technology limitations of the time, I expected to find Good Things™ in the sense of smart engineering, simply as bloat of any kind was not even possible. Reading up on the topic, I was not disappointed. NASA’s computers on the ground were IBM machines – what else? –, but MIT was tasked with the design and implementation of the computers and their software embedded into the spacecrafts.

Hardware

Both the Apollo Command Module (CM) and the Lunar Module (LM¹) had an Apollo Guidance Computer (AGC) installed.² Each weighed 32 kilograms, and consumed 50 Watts.

The computers had to be hardened against all kinds of radiation, so I guess some weight has to be attributed to the corresponding shielding. The big but, though, is that the AGC’s processor was built using discrete logic components. In the sixties, no microprocessors were available yet. They appeared in the early seventies. Integrated circuits, ie. the integration of several discrete components, such as transistors and resistors, into one package, were in their earliest days. The AGC used several thousand three-input NOR gates, which came in sets of two packed into an integrated circuits – six transistors (and some resistors) per package. That was the state of the art of integration on a single chip.³ It was leading edge. No-one had created this kind of compact computer before.

In a spacecraft, where each and every kilogram counts, 32 kg is a lot. Especially in the Lunar Module, which had to take off again from the moon. Each kilogram of computer meant one kilogram fewer of rock samples from the lunar surface. The 50 Watts had to be produced by fuel cells, additionally adding to the payload.

Reading up on all this reminded me of my first programming work I did for Ericsson’s AXE10 digital telephone exchange back in the day. The specific processor I wrote my program for, the so called Regional Processor (RP), was also built using discrete components. The RPs were the front-end real-time controllers for the actual phone switching hardware, and they communicated with the Central Processor.⁴ There were also newer versions of the RP, the so called ERPs, which were actually using microprocessors, and also the CP was technologically way more advanced already. I assume the older RPs were remnants of the first version of AXE10 that was introduced in 1976, which were then replaced in a step-wise fashion by ERPs with each product iteration. Luckily, I didn’t have to care too much for the hardware implementation specifics, I just needed to understand the conceptual architecture, and to master the RP’s assembly language, and was off to the races. But I digress, back to the AGC.

The Apollo 11’s AGC had 36 kilo-words of read-only memory, and two kilo-words of erasable memory. A word was 16 bits, with 15 bits of actual data and one parity bit for error correction. Hence, the AGC had 72 kBytes of fixed memory plus 4 kBytes of erasable memory. Read-only memory was implemented as rope memory, read-write memory as core memory, technologies that were the most space-efficient at the time. Check out how “the girls” are weaving the program code in this video. During development, MIT used a rack-size in-circuit debugger, which also provided the program code via a rope memory emulator, hence allowing to make quick changes without the need to physically rewire a whole block of memory, which, as shown in the video, is a quite laborious and time-consuming task. This instrumentation of the computer must have been a world-first as well.⁵

If this seems laughably little memory capacity in the light of today’s computers and mobile phones, consider that the world of (micro-) controllers is different, still to this day. My experimental microprocessor board, depicted to the right, has 512 kBytes of fixed memory (flash), and 128 kBytes of volatile memory, aka RAM. And that’s using a relatively modern and powerful ARM Cortex-M3 32-bit microprocessor. Yes, more than the AGC, but not that much more.⁶

The Cortex-M3 weighs less for sure, but would probably not survive the rough operational conditions in space (think: radiation). Also, it does not have error-correcting memory. So, ok, it might survive, but not compute reliable results.

And reliable results you really want – you know, the computer controls your spacecraft’s flight. A run-time problem might even occur, but the computer has to cope with it, and recover autonomously. A program failure was not an option if the LM was to land – a faulting essential process meant to abort the mission, maybe even losing the LM’s crew up there.

Software

The AGC’s programs were written by MIT, one for the Command Module (called GNC), and one for the Lunar Module (called PGNCS). Considering the state of knowledge about programming in general at that time – the term software engineering wasn’t coined yet, and the craft in its infancy – they did a remarkable job. In fact, many of the employed principles and techniques are valid today in the controller realm. Spacecrafts and similar contraptions now use full-blown real-time operating systems, such as VxWorks. I am not an aerospace engineer, and thus cannot speak to the actual necessity, benefits, and drawbacks of using this kind of complex platform software.

The AGC did not make use of an operating system in today’s terms, but a well-designed real-time kernel, which provided the abstraction of a process (or job), hence allowing to split the tasks of the flight controller into separate modules, or threads of control. In general, a controller is basically reading the state of the world via sensors, and issuing commands and signals via actuators to change said state to conform with its objectives, in an endless loop. The naive approach is to actually program exactly that, one endless loop that reads and commands as needed. While OK-ish for trivial problems, this approach quickly breaks down if your controller has several realms of control, each with its own possible states. Your single endless loop becomes a mess. The Right Thing™ to do is to handle each realm of control by a separate process (thread of control).

That’s what the MIT team realised and implemented for the AGC, within the tight limits of their hardware. I would venture to say that this laid the groundwork for today’s real-time operating systems.

I had learned the very same principles when writing the software for the power station. People more knowledgeable than me at the time provided us with a real-time kernel based on AGC principles. My software wasn’t flying a spacecraft, but program failure wasn’t an option either. To this day, when I want to create a program for a microcontroller, such as the Cortex-M3 mentioned and depicted above, the first thing I write is a process kernel. For reliability reasons, I prefer non-preemptive, ie. cooperative, schedulers, with a layer of interrupt handlers on a lower architectural level, which operate independently of the scheduler. The processes cooperate with the interrupt handlers via simple data-structures.⁷ This set-up is very similar to the AGC’s run-time kernel, a fact I was not aware of until recently when I read up about the Apollo computing hardware and software.

The Landing

Check out the video below of Armstrong and Aldrin approaching the moon with the Lunar Module, and landing the thing, along the trajectory depicted to the right.⁸ Put yourself into their shoes: they have trained this procedure dozens of times in the simulator, so the flight software has been tested there, and I assume NASA had a very realistic simulator.⁹ But the map is not the terrain, they could never actually fly the LM before in reality, which was designed to only fly in space, ie. in a vacuum and without earth’s gravity. Until now, a few kilometres above the moon.

Now put yourself into the shoes of the team that designed and constructed the LM, and in particular the AGC, watching the approach and landing. I can only try to imagine what was racing through their minds, experiencing this first and only test in the real world. The astronauts even had two computer errors right before the landing (the famous 1202, 1201), and the AGC team lead needed to decide within a few seconds if it was a “go” anyway: the LM was running low on fuel, so there was no time to evaluate the situation deeply. The LM had just 17 seconds of propellant left after the landing.¹⁰ 1202 and 1201 meant that the AGC had done the right thing, and shut down non-essential processes not required for the landing, but which had overloaded the processor due to a problem in the radar system. The AGC’s software kernel-design saved the mission.¹¹