Toggle Menu
November 3, 2023

ILLIAC IV and the Connection Machine: DARPA’s varied approaches to developing early parallel computers

By Eric Gilliam

This piece is a part of a FreakTakes series. The goal is to put together a series of administrative histories on specific DARPA projects just as I have done for many industrial R&D labs and other research orgs on FreakTakes. The goal — once I have covered ~20-30 projects — is to put together a larger ‘ARPA Playbook’ which helps individuals such as PMs in ARPA-like orgs navigate the growing catalog of pieces in a way that helps them find what they need to make the best decisions possible. In service of that, I am including in each post a bulleted list of ‘pattern language tags’ that encompass some categories of DARPA project strategies that describe the approaches contained in the piece — which will later be used to organize the ARPA Playbook document. These tags and the piece itself should all be considered in draft form until around the Spring of 2024. In the meantime, please feel free to reach out to me on Twitter or email (egillia3 | at | alumni | dot | stanford | dot | edu) to recommend additions/changes to the tags or the pieces. Also, if you have any ideas for projects from ARPA history — good, bad, or complicated — that would be interesting for me to dive into, please feel free to share them!

Pattern Language Tags:

  • Pushing forward the technological base via speculative machine building contracts
  • Gray coding
  • Incentivizing a group of related goals by organizing them all into a single contract
  • Selecting multiple industry contractors for a trial period before choosing the final contractor
  • Funding a conservative project to use as a measuring stick for more ambitious projects in the area

Introduction

DARPA’s multi-decade investment in developing early versions of parallel processing computers provides a lot to learn from for ARPA-style PMs. Two of DARPA’s most prominent, long-standing investments in this area took two completely different approaches to push the technology base forward in a coordinated fashion. Both are viewed as successes to some and abject failures to others. In this piece, I will dive into the operational structures that informed DARPA’s investments both in the ILLIAC IV parallel processing computer in the late 1960s — the project was based out of the University of Illinois under a researcher named Daniel Slotnick — and in the young Thinking Machines Company in the 1980s and their Connection Machine computer — which brought them fame and infamy in the computing world in the years in which DARPA was most heavily involved with the company. 

To start, I’ll dive into DARPA’s investment in the ILIAC IV. The cost overruns and time delays for which this project became internally famous very heavily informed the approach that Stephen Squires took to managing DARPA’s investments in Thinking Machines and his computer architectures portfolio as a whole in the 1980s.

ILLIAC IV’s Context

When DARPA Director Charles Herzfeld took over in the mid-1960s, computing research was not nearly as pressing of a defense research issue as others like ballistic missile defense or nuclear test detection. The previous director, Jack Ruina, often only met with the then-IPTO Director J.C.R. Licklider once or twice a month. Now, that is in large part because he both trusted Licklider and attempted to have a hands-off management style. However, it also speaks to the lack of priority that the Information Processing Techniques Office (IPTO) had within the DARPA portfolio at the time. PMs in hotter areas of the portfolio were said to have quickly met with Ruina as many as five times a day to discuss small issues. 

As the reader surely knows, great work was done by Licklider and his portfolio of performers in the early 1960s with this approach. As Herzfeld took over, the progression of the field was apparent to him and he thought it had the potential to help improve programs all across DARPA and the DoD. He recounted his approach to managing the IPTO as he took over, saying:

I thought that my job vis-a-vis the computing program, when I was Deputy Director and Director, was first of all to make sure to get the best office directors we could get, and second, to help them do what they thought they needed to do, and third, to look for applications across the projects for computing, when you apply the new computing ideas, technology and its capabilities as widely as possible.

Ivan Sutherland — Licklider’s young, hand-picked replacement who had gotten his Ph.D. at MIT and had been at CMU for several years since graduating — became IPTO’s Director under Herzfeld. Sutherland, while young, had been making a name for himself in his field. His Sketchpad program, written for his MIT thesis, would later help win him a Turing Award.

Herzfeld also lived up to his word of helping good ideas from this office find funding. One example of this was when, in 1965, Robert Taylor brought forward the network proposal to Herzfeld that would become the ARPANET. Herzfeld, thinking the idea had promise, quickly determined that ARPA should fund the idea and took some money that had been set aside from a lower-priority program — almost $500,000 (~$5 million today) — to get started immediately. They would get funds from Congress to continue the program in the following fiscal year. This kind of administrative move, while not rare in managing the DARPA budget at the time, was more likely to be done for higher-priority projects.

As IPTO’s technology base was becoming more mature, its budget began to increase and PMs were looking to fund projects that were further along in the development process — IPTO’s early-1960s budget had been almost entirely allocated to early-stage research.1 One development area of interest was the idea of building a more efficient computer leveraging hardware that was more parallel in nature. If implemented successfully, it was thought that the machine could help the DoD solve many pressing problems that required more computing power — such as calculating nuclear weapons effects or real-time processing of radar data.

ILLIAC IV’s Beginnings

In 1965, Sutherland reached out to Daniel Slotnick a month after Slotnick had arrived at the University of Illinois to set up a visit at Slotnick’s new office in Urbana-Champaign. Slotnick had become acquainted with Sutherland while he was working on his SOLOMON parallel computing machine at Westinghouse; this area of work at Westinghouse had many DoD ties. In their meeting, Sutherland would ask Slotnick if he was interested in developing a large parallel computer at Illinois. 

At least, that is how the events unfolded according to Daniel Slotnick’s IEEE memoir. Sutherland, in an oral history conducted by William Aspray, seems to remember things differently. According to his memory, Slotnick came into his office while he was still at Westinghouse and funding for his project was drying up, and an interaction resembling the following ensued:

He came into the office and said,” Basically, I want to do these things and I cannot do it at Westinghouse, because they will not let me do it. Where should I go?” I introduced him to some of the university folk. So that one was fairly a long time in the doing.

This, also, is a quite believable ordering of events given my understanding of how things worked at IPTO in the 1960s. Whether Slotnick’s account or Sutherland’s is more accurate does not matter much. Regardless, in 1965 Slotnick and Sutherland had a conversation about building a parallel computing machine — which they had clearly discussed before — and Sutherland let Slotnick know that this might be a good time to submit a proposal now that he was at the University of Illinois. When the proposal was submitted, other IPTO staff as well as Herzfeld supported funding the proposal.2 In the next few paragraphs describing the details of how the finer points of the contract materialized, I’ll rely on Slotnick’s more clear account of the events. Certain small details might differ between the two accounts, but none that should matter much for the takeaways of this piece.

The project would be quite similar to Slotnick’s work at Westinghouse, and it was an area of obvious interest to IPTO’s mission. After Slotnick thought it over, he agreed. Sutherland initially proposed starting the project with a small study phase. But this tentative approach was not agreeable to Slotnick. Arriving at Illinois, the disappointment of the SOLOMON project was fresh on Slotnick’s mind. Slotnick’s intensive work on the SOLOMON project jointly funded by the DoD and Westinghouse for two years, Slotnick’s Westinghouse team steadily grew to around 100 people. In spite of the team’s promising early work on the technology, the project’s steady progress came to an abrupt halt when the project’s main sponsor in the DoD suffered a tragic drowning accident. The man’s replacement did not back the project in the same way. Quickly, the project went from pushing for the funds to build a full-scale version of the machine to having no funding at all. Not long after, Slotnick found himself at Illinois.

With this disappointment fresh on his mind, Slotnick said he was “anxious to do something else.” He only wished to pursue the project with DARPA sponsorship if Sutherland meant business.  Slotnick described his reaction to Sutherland’s proposal of a modest seed funding stage as insistent:

I absolutely refused. I wanted a $2 million payment at the outset and a contract for a total of $10 million. I did this to make sure that the ARPA commitment was real and had passed the highest levels of review.” Sutherland must have understood. Slotnick continued,  “Ivan agreed. I wrote the proposal and a few weeks later we had our contract.”

The proposed machine was heavily inspired by Slotnik’s time spent developing the SOLOMON prototype machines. The newly proposed machine — the ILLIAC IV, named after three earlier computers built at Illinois — sought to achieve over one billion operations per second. Beyond the project’s aim to produce a machine with a substantial improvement in computing power, its management sought to roll out improvements to much of the component technology in the machine. When all was said and done, Slotnick and the DARPA team involved believed the machine would have several useful areas for military application. DARPA and the armed services research organizations planned to deploy the resultant machine to help advance research problems in climate dynamics, anti-submarine warfare, ballistic missile defense, and the army’s multiple target problem. Additionally, a nuclear research office was interested in the capability of the computer to help solve the climate modification problem. Lastly, IPTO itself was particularly interested in the computer’s applications to researchers in cryptanalysis, simulation of fluid flow, seismic array processing, economics, and other problems ideally structured to utilize the computer’s parallel processing capabilities.

Sutherland — as the IPTO Director — was very explicitly attempting to use the ILLIAC IV project to push the boundaries of several component fields of computing simultaneously. As later-DARPA Director Stephen Lukasik described the ILLIAC IV team’s full frontal assault approach, IPTO and DARPA were “pushing machine architecture, software for parallel machines…medium scale integrated circuits — and then the application of that to a variety of important defense problems.”

ILLIAC IV’s Operations

In addition to this project helping push forward the technology base in the area of parallel computing, Sutherland hoped that it could help spark some efforts from industry in building new computer components relevant to this general area of technology. So, while the overarching contract for the project was issued to the University of Illinois team under Slotnick, the work of an industry subcontractor to build the machines the Illinois team designed would also be vital to the project’s success. In addition, contracts for the research and engineering development of various components of the machine would be given out to other parties from the IPTO performer pool. However, the work of these additional parties would generally be less important to the project’s success than the work of the Illinois team and the computer manufacturer chosen to produce the final machine.

The Illinois team’s initial design specified a machine with four modules of 64 processors under the control of a single construction unit which would allow them to simultaneously work on the same problem. As Slotnick began to assemble a team at Illinois, the team also began to figure out which computer manufacturer to partner with. Slotnick was not upset by the project’s partnership structure. From the beginning, he felt this was a natural approach to doing a project like the ILLIAC IV. Slotnick had come from industry and understood the comparative advantages of industry and academia when it came to designing and building new computers at that time. As Slotnick put it, “The days when a university could design and fabricate a big machine by itself were over, and we decided that while we would do the architectural design and most of the software and applications work, we would rely on industry for detail design and fabrication.”

Since selecting the right contractor to build the machine was pivotal to the success of the contract, Slotnick and the DARPA team held a competition. Slotnick and the DARPA team outlined the major approaches and incorporated them into a bid set. In August 1966, after “many months of intensive contacts with industry,” RCA, the Burroughs Corporation, and Univac were all awarded 8-month contracts to continue working on the early stages of the contract and collaborating with the Illinois team. After this period, in 1967, the Burroughs Corporation was selected as the best fit to continue on as a partner in building the ILLIAC IV. Burroughs had partnered with Texas Instruments in its bid. The plan was for TI to develop the integrated circuits for the machine’s all-important process element (PE) array. With that trial period over and the DARPA and the Illinois teams having gotten the chance to work with all three companies for a time, the industry partner for the project was decided upon.

Either Larry Roberts or Robert Taylor — Taylor had become the new IPTO Director in the intervening period and Roberts was the PM who focused on the ILLIAC IV project — and the Illinois team awarded the related contracts to work on other component parts of the ILLIAC IV to the following non-U of I teams: 

  • The design and implementation of the FORTRAN compiler for the ILIAC IV to the Applied Data Research Corporation
  • Various hardware and software to help link the ILLIAC IV and a trillion bit memory with the ARPANET to the Computer Corporation of America
  • And work on the development of the interactive front end processor to BBN.

As the project progressed, the design and construction phase was riddled with research problems and setbacks that forced design changes on the research team, delays, and cost overruns. Two main issues contributed to the early cost overruns and time delays in the first couple of years of the project. The first was TI’s inability to produce the 64-pin emitter-coupled logic (ECL) packages. This caused a delay of over a year and a $4 million cost overrun. Slotnick documented that TI was not exactly forthright in taking the blame, writing, “A great deal of inconclusive finger-pointing went on between TI and its suppliers.” Regardless, the problem — and loss of cash and time — remained. The second major problem was Burroughs’ inability to make workable magnetic thin-film PE memories for the evolving design of the machine. This problem caused an additional year of delay and another $2 million cost overrun. Additional problems existed with packaging, circuit design, and interconnections for the large machine. But these two problems, in the early years of the project, caused the most headaches.

Slotnick, in his writings, explains how these very early setbacks began to affect project planning. Certain goals became more modest and the team had to substitute in and out certain component technologies to fit in better with the evolving design and changing circumstances. Slotnick writes: 

We retreated from the 64-pin packages to standard 16-pin dual in-line packages. In doing so, however, everything got bigger and more expensive. A lot of logic design was salvageable as a consequence of making the 16-pin packages logically derivative of the abandoned 64-pin packages, but board layout, back-panel wiring, and all system level hardware, of course, had to be restarted from scratch. The memory situation was even messier. Burroughs had a large investment in its thin films and didn’t want to give up on them even after my own independent review had concluded they still represented an intolerable development risk. Semiconductor manufacturers were just beginning to gear up for memory chip manufacturing; the manufacturing means were clearly at hand, or at least so it seemed to me, but the chips were not. I made the painful decision to drop films and go with semiconductors.

In making this semiconductor pivot, Slotnick believed the young Fairchild Semiconductor was easily the most qualified supplier. While Burroughs railed against the selection of the young company, a panel of independent ARPA experts — assembled due to Burroughs’ ‘furor’ at Slotnick’s selection of Fairchild — confirmed the decision. Fairchild seemed to be one of the only — if not the only — suppliers delivering high-speed semiconductor memory at the time. 

As the project wore on, the projected cost of the project roughly tripled and Slotnick was now only promising to deliver a system a fourth the size of the one originally promised. In spite of these shifts, Larry Roberts — who would become IPTO Director himself and who was already known as the “father of computer networks” — remained behind the project. Not only did Roberts, as Slotnick jokingly put it, share in his “sick attachment to really big pieces of hardware,” he was very interested in the list of applications which the machine was set to work on — including numerical weather prediction, sonar, radar, seismic signal processing, and an assortment of other computations array computers tended to do well. But, as Slotnick saw it, to justify the escalating costs, making the machine available to as large a community of users as possible became even more essential. The larger the scale of users the machine achieved, the more justified the escalating costs would be. Slotnick believed that this was a major factor that drove making the ILLIAC IV easily available over the ARPANET to be a much stronger point of emphasis for the project as the years wore on.

In 1970, five years into the constantly delayed project, the Illinois team was finally beginning to hone in on a workable machine. But, even then, there was still an element of instability in the project’s operations. From a technical standpoint, things were beginning to get on more sure footing. The PE had been successfully redone with the new TI integrated circuits, PE memory chips were being delivered by Fairchild, and other pieces of equipment were getting produced on schedule. However, cultural conflicts were now coming to a head on the Illinois campus. Not only were stresses between Slotnick and those faculty not involved with the million-dollar-a-month project being run out of an academic department coming to a head, but Vietnam War-era turmoil from the students was making it more difficult to peacefully continue the defense-funded project on campus. In the midst of all of these non-scientific issues, the project was first moved from within the academic department to become its own free-standing center at the university. Then, in January 1971, ARPA decided to move the system to the NASA Ames Research Center when it became feasible. The move was carried out by Burroughs in 1972 and may have happened earlier in the technological development process than DARPA would have preferred.

Hardware problems plagued the machine in its early years at Ames. Early circuit batches failed at high rates and in ways that were often difficult to detect. Additionally, many of the back-panel connections and terminating transistors functioned poorly. While successful runs were made as early as 1973, the team at Ames did not have the machine working in a way that could be considered reliable until 1975. However, even after the machine was completed, one problem continued to cause severe headaches: producing software to run on the machine. This would be an area of lasting difficulty that plagued the machine for its entire lifetime — which was admittedly not so long. The difficulty in writing software for the machine would be remembered when the next generation of IPTO leaders sought to push the area of parallel computing to the next level.

The ILLIAC IV would only be used until about 1981 when NASA replaced the machine with a more reliable and easy-to-use successor

Three men installing a chassis or module onto the ILLIAC IV. Photo provided to the Computer History Museum by the Burroughs Corporation. https://www.computerhistory.org/collections/catalog/102651989

ILLIAC IV’s Results

By the time ILLIAC IV was finally constructed, the total cost was $31 million (over $200 million today) — substantially higher than the original, expected price tag of $8 million. Beyond the cost problem, the final machine only had one quadrant of 64 processors rather than the originally planned four. Several of the many experimental components of the machine could not be made to work and existing components had to be used in their place. The resulting machine only achieved one-twentieth of the computing power that the researchers originally sought to achieve — the initial goal was one billion operations per second. Also, even though the ILLIAC IV was considered finished in 1972, much of the 1972 to 1975 period was spent in an intensive effort to correct problems and improve the reliability of the machine as researchers affiliated with the armed services were attempting to use the machine for practical applications. 

Notwithstanding the setbacks, once the ILLIAC IV became operational in 1975, it was thought to be the fastest machine in the world for certain types of problems — calculations of fluid flow being one example. In the late 1970s, the computer was used in service of many of the armed services and DARPA applications on which it was intended to be used. This includes the army’s multiple target problem, research in cryptanalysis, seismic array processing, economics, and more. For the few years in which the machine was in operation, it may have been the best computer in the world for solving problems whose calculations could be performed in an “all at once” manner and were, thus, ideally suited to being solved by a parallel processing computer. 

From the standpoint of developing the technology base, the component technology of the computer helped improve many pieces of the technology base. Some of the prominent examples:

  • The first large-scale computer to employ emitter-coupled logic in its processors — taking the place of transistor-transistor logic. This technology would go on to be used in many high-speed computers in subsequent years. 
  • Its circuit boards were the first to be designed with the aid of a computer. Computer design automation is now widely used in industry.
  • The design of its storage technology — which consisted of 64 disks that could be read from and written to concurrently — led to higher speed input/output.
  • The machine’s laser memory system as an additional, tertiary memory had a capacity in the trillion-bit range and read-in and read-out rates in the million bits per second range.
  • ILLIAC IV’s architecture employed a single instruction stream to control the multiple data streams involved in interprocessor communication.
  • ILLIAC IV was the first large system to employ a semiconductor primary store.

On top of all of the new functionality, the program also succeeded in (eventually) producing a machine built by a joint industry-university collaboration. Slotnick and the Illinois team did not believe they’d have been better off working on the project with no help from industry. In fact, Slotnick later reflected that he believed running such an operation with him and the university as primary contractor could have been a mistake — or something to learn from at the very least. He referred to trying to administratively manage such a technically complex operation with that sort of headcount, scale, budget, and operational needs as having “all the sense of trying to build  a battleship in a bathtub.”

The battleship in the bathtub. Engineer Jay Patton is at the oscillosope. The two men kneeling are testing one of the boards on a test machine. Photo provided to the Computer History Museum by UIUC. https://www.computerhistory.org/collections/catalog/102651994

ILLIAC IV’s Lessons Learned (and Caveats)

Joining all of these non-obvious developments into a project meant to produce a single, useful machine caused substantial time delays and cost escalation. But the field of computing got more than nothing in return for the tradeoff. The ambitious design of the machine created a technology “pull” and stimulated the development of new component technologies that had the potential to find broad use. 

For example, when the project ran into issues incorporating the originally planned film memories into the design, Slotnick turned to the nascent Fairchild Semiconductors to design the project’s all-important memories to fit with its new design. The new design was seen to be quite risky since it required advances in semiconductor art, engineering design, and production. But Slotnick was convinced the Fairchild team could do the work. This confidence may have come from the fact that Slotnick’s old mentor, Rex Rice, headed memory system operations at Fairchild. It also may have been due to Fairchild’s rapidly growing reputation as the new provider of semiconductors to the NASA program and certain Air Force programs. Regardless, in the end, Slotnick was right to put his faith in the Fairchild team because they successfully produced the required memory chips and the ILLIAC IV was the first to use chips of this design. Some believe that this project helped speed up the pace with which semiconductor memories became commercially available — although given the field was growing so quickly, it is hard to tell if such a claim is true. 

Taking a step back, it is reasonable to question whether or not the ILLIAC IV’s approach is the ideal strategy to orchestrate new technological developments. As always, the suitability of the approach depends on the situation and what the PM thinks is appropriate given the context. Developing so many new pieces of component technology to use, for the first time, in one machine seems to leave one obviously more prone to cost overruns and time delays than most projects. The upside is that the approach provides new funding and a focused direction to help bring what could become very useful component technology into existence. In the best case, the approach can push a field forward much faster than if one had taken a more measured approach. But the approach does also run the substantial risk of generating demand for and funding pieces of component technology that might be useful to the current project, but will not be useful to the field as a whole in later years. For example, if the components required by the design of the machine are not reflective of the needs of the field as it progressed outside of that individual project.

In essence, the  approach of the ILLIAC IV team should be considered a high-risk, high-reward strategy. It probably won’t achieve outcomes as stable as those of the gray coding approach covered in the following Connection Machine sections, but there are still situations where the full frontal assault approach of the ILLIAC IV has a high enough expected value to justify the risks. With this approach, one may run the risk of cost overruns, delays in producing a working machine, and developing components that the broader field turns out to not need. But the approach also might help a technological area make discontinuous leaps in component technology and overall system performance that would not happen by changing one component at a time. Particularly in the early days of a technological field, changing multiple design components at once is often an attractive approach given that existing approaches might be extremely far from what researchers know to be theoretically possible.3 (See this earlier FreakTakes piece on the field of early molecular biology to read more about this strategy.)

Depending on the political and technical context of the project, the full frontal assault approach might be just what is required. But, in other situations, it might be considered far too risky. Stephen Squires, with his computer architectures program in the more mature DARPA computing portfolio of the 1980s, considered the approach far too risky.

Connection Machine’s Beginnings

While the final ILLIAC IV machine did work, Squires took the management of the program as a cautionary tale of what not to do when developing machines within IPTO. In his eyes, the cost overruns and delays in producing the machine were entirely foreseeable given the structure of the project. Squires believed the “full frontal assault” approach of the ILLIAC IV project — with novel approaches to the machine’s architecture, processors, memory, storage, and other components — pushed too many untried technologies at once. Problems in developing or implementing any one of the components could negatively impact the entire project. In the uncertain enterprise that is research, Squires believed this was an untenable risk profile. 

Squires wanted to encourage projects funded through his architectures program to limit the number of new technologies a project worked on at a time — usually to one. He called the approach “gray coding.” Roland and Shiman — who wrote the most extensive history on DARPA’s Strategic Computing Initiative of the 1980s and extensively interviewed Squires — expanded on the thinking behind the approach as follows:

That way, problems could be easily traced, isolated, and corrected, and the chances of catastrophic failure of the system would be minimized if not eliminated. Advance of the field would not occur in one fell swoop by ‘frontal assault’ of some ‘point solution,’ but through the selected development and installation of key technologies, all well-timed and well-coordinated. Advance would occur over a series of generations. Once a technology or key concept had proven reliable, a new component or design would be implemented and tested.

Squires portion of DARPA would still pursue several technological trajectories at once — which Squires called the ensemble model —  to allow the research ecosystem to prove out which one held the most promise. 

Additionally, learning from the ILLIAC IV example and his own experience working on systems software at the NSA, Squires insisted that software development be emphasized as much as hardware work for the architectures being developed. In the case of the ILLIAC IV, the development largely focused on getting the hardware working and later shifted efforts towards generating useful applications via software development. Many believe the ILLIAC IV team emphasizing the hardware first and software later was a large part of what took the machines so long to finally get working at the back half of the project. Squires was going to do things differently.

Squires planned for the architectures program to progress in three phases — each lasting roughly three years. The projects funded in the first phase would generally seek to refine promising architecture ideas and develop microelectronics components that would later be used in building parallel machines. Projects funded in the second phase would generally seek to build full prototypes of the most promising architectures. The projects funded in the third phase would integrate technology that now (hopefully) existed into composite systems that could support DARPA–relevant applications — such as military applications in computer vision, natural language understanding, simulations, and more. Squires hoped that the machines being developed would achieve 100 million floating point operations per second by 1987, a billion by 1989, and a trillion by 1992. In service of this goal, all of the early work on smaller-scale prototypes had to be easily scalable — for example, being designed in such a way that they could be easily expanded by simply increasing the number or power of processors in the prototype. As all of this work progressed, work on new programming languages, compilers, tools, and concepts that better served parallel programming would be also funded.

With the high level plan in place, Squires began by traveling around and further familiarizing himself with the pool of potential performers that might be funded and meeting with those in industry — such as computer manufacturers. In addition, this familiarizing process also entailed a February 1984 call for qualified sources asking for potential contractors to explain what they could do in the areas of signal, symbolic, or multifunction processing if they were funded by Squires’ architectures program.

Squires, at the time, primarily issued the formal solicitation for informational purposes — as a way to see what was out there, not necessarily planning to immediately award any contracts to the proposals submitted. But one proposal jumped off the page from the pool of submissions — two-thirds of which from industry and one-third from academia. That proposal was a big black notebook sent in by a newly formed company called Thinking Machines Corporation (TMC) outlining their technical and business plans for a computer called the Connection Machine.

Connection Machine’s Operations

The proposed machine was exactly what Squires was looking for. The machine the company proposed was an ambitious, truly parallel hardware architecture that seemed like it could scale up by simply adding more or better processors. In spite of the ambitious design, Squires still saw the approach as conservative — in a positive way. Not only had the Thinking Machines Corporation (TMC) already developed a workable VLSI design for the processor chips — which was one notable source of technological risk already mitigated — but the company also set out to construct the architecture using as many commonly used components as possible. In fact, in several areas they went as far as to use components that were far simpler than the ones commonly used in other machines at the time. One example is that, for its first machine, TMC planned to use the simplest 1-bit processors instead of the more powerful 4, 8, or 16-bit processors commonly in use at the time. In addition, the Connection Machine would not even be a stand-alone machine in the early iterations. The front end used to access the machine would be a more commonly used computer — a LISP machine in this case. In this early stage, SC’s main goal was to demonstrate the feasibility of basic parallel processing concepts while not adding needless complications to projects that did not serve the main goal. TMC — maybe not surprisingly since its founder came from a lab that was quite used to working with DARPA PMs — clearly got the memo and submitted a proposal that perfectly complied with the gray coding approach while maintaining an ambitious vision.

After some meetings between DARPA staff and TMC executives, it seemed clear to DARPA that TMC as a venture was well thought out from both the business and technical perspectives. In early 1984, $3 million (~$9 million today) was approved to fund TMC’s work on a small prototype of the machine containing 16k processors — with the option for an additional $1.65 million to scale up the prototype to 64k processors. Raising an additional $16 million in private investments around this time from private investors, TMC was well-capitalized and ready to begin work on its first machine.

Even having laid out a plan of action that complied with Squires’ gray coding framework, the work was daunting for the company. Stephen Hillis, the CEO of TMC who came up with the idea for the Connection Machine during his Ph.D. under Marvin Minksy at MIT, said the early work building the simplified version of the Connection Machine underway in 1984-1985 was “overwhelming.” He continued, describing the early work of the company in a 1989 Physics Today article:

We had to design our own silicon integrated circuits, with processors and a router. We also had to invent packaging and cooling mechanisms, write compilers and assemblers, devise ways of testing processors simultaneously, and so on. Even simple problems like wiring the boards together took on a whole new meaning when working with tens of thousands of processors. In retrospect, if we had had any understanding of how complicated the project was going to be, we never would have started.

This reflection from Hillis probably lends some credibility to Squires’ gray coding approach having been a good fit for this particular project. The design and need to develop new software approaches for making use of the novel design would also bring with it more difficulties. There was no need to compound on these difficulties by ensuring the initial prototype had the most cutting-edge component tech possible. Novel engineering development projects like this were sure to run into plenty of issues that required novel solutions to overcome. As Hillis stated, they ran into so many of these in the early days that they might not have even undertaken the project had they understood how painful it would be. As the project grew more complicated, Hillis seemed thankful to have Manhattan Project alum Richard Feynman around as an intern for some summers — Feynman’s son Carl had worked with Hillis when Hillis was a grad student and Carl was an undergrad at MIT. Feynman seemed relatively influential in pushing the company to take a page out of the Manhattan Project’s management playbook and designate a group lead in each area of technology — such as software, packaging, electronics, etc. Feynman — who was initially brought on to help brainstorm applications for the computer — also pushed the company to hold regular seminars of invited speakers, often scientists, who the company believed might have interesting use cases for the machine.

While the project did have its difficulties in the early years, in spite of Hillis’s statement about the project being “overwhelming,” the project was considered the gem of the architectures portfolio in the early years of SC. Squires’ program had also invested in other prototype-stage projects for at least a two year trial stage — which all had some positive developments as well as some downsides. Each of the general-purpose, prototype-stage projects was meant to help in the architectures program’s larger goals of identifying promising approaches to parallel computing and providing experience working with the technology. Simultaneously, smaller projects called “accelerators” were funded with the goal of doing development work to produce components that could later improve the performance of the general-purpose machines. Besides the Connection Machine, some of the architectures program’s other general-purpose prototype projects included:

  • BBN’s Butterfly computer series
    • The Butterfly computer had been in the DARPA funding pipeline since 1977, with funding going to one of DARPA’s workhouse computing research contractors, BBN. At the start of the architectures program, a model of the Butterfly with 10 processors already existed. BBN received funding to continue to scale the prototype to 128 processors. The butterfly computer was a coarse-grained shared-memory machine — coarse-grained machines have fewer but more powerful processors capable of running entire programs in a moderately parallel but not truly parallel fashion. This technology was both further along in development than smooth-grained parallel computers — like the Connection Machine — were. Additionally, machines like the Butterfly could easily make use of traditional programming methodologies rather than developing new ones. For those reasons, the Butterfly was meant to serve as the benchmark against which progress on other machines in the portfolio would be measured. 
  • CMU’s Warp
    • The architectures program also contracted CMU to build Warp. Warp was a systolic array machine that sought to achieve more efficient performance using an approach in which processors are connected sequentially and data flows from one processor to the next with each performing a different operation — analogous to an assembly line. CMU had received this contract based on its prior demonstration of the systolic approach having built a programmable systolic chip.
  • Columbia’s DADO
    • DADO was designed as a coarse-grained parallel machine. DADO was one of two tree-structured machines — in which a central processing unit was connected to two others, and each of those to two others, etc. — started at Columbia University in 1981. DARPA discontinued funding for this project after the early prototype stage.
  • Columbia’s Non-Von
    • The second of Columbia’s two tree-structured machines, the Non-Von, was somewhere in between the DADO and the Connection Machine in terms of how fine-grained it was. A machine like this one would likely prove simpler to program than a Connection Machine, but would also not achieve quite the level of parallelization. The Non-Von sought to prove useful in managing and manipulating large databases. DARPA also discontinued funding for this project after the early prototype stage.
  • TI’s compact LISP machine
    • TI sought DARPA funding to fund its development of a miniature version of a LISP machine. To do this, TI planned to implement LISP processors onto individual chips. TI hoped that the entire machine would fit on nine small printed circuit cards and, in the end, would be small enough to be embedded in computers of military systems such as within a pilot’s cockpit to help in expert decision making. DARPA agreed to the $6 million in hardware costs and TI, sharing the costs, funded the $6 million in software development.

But none of these projects seemed to carry as much promise as the Connection Machine and its model of performing identical operations simultaneously across its many, many processors. However, each of the other machines did often have at least one piece of component technology in development that DARPA was intrigued to pursue further. For example, the Non-Von project developed a way of using multiple disk heads in operating its storage system. TMC later incorporated this idea into one of its later prototypes — all SC-funded developments were eligible to be used by other SC contractors. This is just one case of discontinued portfolio projects still finding help push the technology base forward in Squires’ portfolio.

As the Connection Machine project got underway, Hillis had arranged for DARPA funding to install a LISP machine at the chip foundry TMC worked with in California. With that machine installed, TMC could test chips at the foundry rather than having them shipped to Massachusetts to test them there. By mid-May 1985, a little over a year after TMC had received its first DARPA grant, the 16k processor prototype had been finished — more than a month ahead of schedule. After DARPA checked out the machine in the TMC offices, DARPA immediately invoked the option for the 64k processor machine. This scaled-up prototype of the machine, too, would be finished ahead of schedule — by the end of 1985. Hillis’ design seemed to be scaling the way he and DARPA had hoped. And for all of the sub-problems that the initial design could not account for, the very talented (and growing) TMC research team — largely drawn from the staff and students of MIT, CMU, Yale, Stanford, and other elite pools of computer engineering development talent — was proving more than up to the task of solving. 

By the spring of 1986, the first Connection Machine — the CM1 — was being manufactured and was available for sale. The machine’s chips each contained 16 customized 1-bit VLSI processors. Thirty-two of these chips would be printed on a circuit board and mounted next to each other — eventually making up four separate quadrants of 16k processors. Each quadrant could stand alone as a cheaper, scaled-down model of the 64k processor machine. In spite of the novelty of the truly parallel design of the architecture, each of the components remained quite safe and reliable to use — as the gray coding approach intended. The 1-bit processors were basic and slow compared to the state-of-the-art processors at the time, but they also produced so little heat that the machine did not require much cooling. Installation of the machine on a buyer’s site was easy and could be readily set up with a LISP machine on-site — which customers knew how to use. However, customers did have to learn new, Connection Machines-workable, versions of C and LISP to make full use of the machine’s capabilities while programming. The user did not need to fully understand how memory assignment and other aspects of the machines worked with these languages, but learning the languages was a necessity.

With the project going well, in the beginning of 1986 DARPA funded TMC for an additional $3.8 million over two years to find ways to exploit the CM1 and CM2 on various scientific and military applications. While the Connection Machines were still in the process of becoming truly useful machines, this particular generation of DARPA’s computing programs carried a strong political emphasis on pushing successful research projects into application mode as early as possible. This iteration of DARPA funding for TMC included covering work on projects to further improve software development practices for the machine, develop the machine so it could be used as a network’s server, increase storage capabilities to aid in work on data-intensive problems, and developing a training program to help others learn to use the machine. Beyond this $3.8 million, DARPA would also serve as TMC’s biggest buyer in the coming years. 

With this DARPA funding, the work continued on into more mature prototyping and scaling stages. As of 1987, around twelve CM1s had been purchased from TMC — about half of them purchased by DARPA for its contractor community. The machines were typically purchased for around $5 million a piece in 1987 (~$14 million in 2023). As the TMC team sought new applications for its machines, under Hillis’ leadership, they eagerly sought out projects with many in the research community. This included partnering with physicists, astronomers, geologists, biologists, and chemists to understand the details of their applications. Someone like Feynman worked in application areas at TMC such as problems in database searches, geophysical modeling, protein folding, analyzing images, and simulated evolution. Establishing partnerships with researchers proved, unsurprisingly, quite natural to the organization. It should be remembered that, during his graduate work, Hillis designed the machine to be something like an optimal tool for researchers studying artificial intelligence — as well as other complex research problems. People like himself, his former lab mates in Minsky’s lab, and other researchers from similar environments were the kinds of people who staffed the company as well as its initial, intended customers.

The company did not prove nearly as eager to establish profitable partnerships with industry as it did partnerships with researchers. Of course, not all of the problems the company explored applying the machine to were research related. For example, Feynman also worked on one application area related to reading insurance forms. However, this sort of application work did seem to be a less natural fit for Hillis and the company.  Hillis, after all, saw the Connection Machine as a machine for science. In practice, TMC very clearly preferred not to make compromises to the goal of pushing science forward in search of increased revenue. For now, DARPA was TMC’s guardian angel when it came to buying machines, helping broker sales with research consumers of the machines that DARPA didn’t buy itself, and funding the company’s R&D work.

The CM1 — which TMC largely relied on DARPA to facilitate purchases for — did not only have limited market potential outside of the DARPA universe because of the price tag of its machines, but also, simply, because of what they just couldn’t do. From a consumer standpoint, the first major deficiency was that the machines could not yet run the standard computer language of the scientific community: FORTRAN. The second major deficiency was even though the CM1 could perform ten times as many computations per second as many contemporary, commercial supercomputers — 2.5 billion operations per second vs. a few hundred million — the 2.5 billion operations it could do per second were not floating-point operations. And floating-point operations were vital to most computationally intensive research problems at the time. Nevertheless, DARPA likely felt comfortable facilitating these purchases because much of the SC program emphasized folding other SC-developed component technologies into projects sooner rather than later. At the beginning of SC, DARPA had also facilitated the purchases of the young BBN Butterfly computer to some of its contractors in order to ensure they had the most up-to-date technology available to them and their development work. So, TMC was not the only contractor in the architectures portfolio to have received this sort of favorable treatment from DARPA.

With the deficiencies of the CM1 in mind, TMC got to work incorporating the clearly needed functionality into its CM2. Surely this lack of functionality, at this stage in the development process, was not considered a failure by any means. The first successful prototype — the CM1 — lacking much needed functionality from the user’s perspective was likely just a symptom of the gray coding approach. The project was likely unfolding as Squires hoped successful architectures projects would — in successive generations. By this time, the architectures program, headlined by TMC, was clearly proving that parallel computing could be very workable. From a technology development standpoint, that was no small thing, even if the tech was not the most powerful or user friendly yet. 

The CM2 was brought to market in April 1986 and could run FORTRAN as well as do floating-point operations. This version of the machine was much more usable to the scientific community and the machine’s very novel, massively parallel architecture still required its customers to learn to use special software and new programming techniques. Using these programming languages proved difficult for even the machine’s quite savvy researcher-customers and far more trouble than it was worth for almost all commercial customers. An additional factor that made the machine more trouble than it was worth for many corporate customers was that the machine’s rate of breakdowns and subsequent downtime, while not an issue to academic work schedules, was far more than corporate customers were comfortable with.

Some in the research community found useful applications for the new machines in areas such as fluid flow dynamics and large database searches. Yet, in spite of these successes, the machines were not yet considered very usable to their potential customers. Years later, Dave Waltz — the head of TMC’s AI group — recounted that most of the CM2’s early customers were not using the machine correctly. In his eyes, TMC had built a machine that worked far more like a human brain than a sequential computer like the Cray — the most successful commercial supercomputer company at the time — but, he noted, that people actually knew how to write programs for a Cray! Many of the customers were running  their computations in such a way that they used the floating point processors in the CM2s in a standard way and largely ignored the machine’s novel, 64,000 single-bit processors. But, with the technology continually progressing, DARPA continued to help TMC facilitate sales. As one of TMC’s research directors later recounted,  “Our charter wasn’t to look at a machine and figure out the commercial profit. Our charter was to build an interesting machine.”

As the Cold War waned and the acute need for military supercomputing and near-term AI applications lessened, the Bush administration and the president’s main science advisor — nuclear physicist Allan Bromley — turned its eye towards helping solve “grand science challenges.4” These challenges included problems related to climate modeling, analyzing protein folding patterns, mapping the human genome, earthquake prediction, and learning more about the underpinnings of quantum mechanics. Bromley seemed to believe that these problems required enormous computing power rather than artificial intelligence. From the modern perspective, smart people could argue whether or not Bromley was right in that idea. But, surely, massive increases in computing power paved the way for specific solutions in those problem domains — even if novel AI methods were what pushed some of those things across the finish line. Regardless, that general ethos was what drove the government’s new High Performance Computing and Communications (HPCC) program. DARPA was the lead agency in this work and received an additional budget of several billion dollars through 1996 for the program. One of the primary goals of the new program was a computer that could achieve one trillion computations per second.

Naturally, DARPA turned to TMC and gave the company some of this funding to help the company expand on its work that was so successful thus far. DARPA granted the company a $12 million initial contract to produce a scaled-down machine that could theoretically hit the trillion computations per second benchmark by 1992. It was in this period when TMC was working on the CM5 — TMC CEO Sheryl Handler skipped “CM3” and “CM4” in the hopes that the naming curveball would make espionage harder for would-be infiltrators — that the music stopped for the young technology darling.

In this period, as TMC was hitting its objectives and in a great place from a technology development standpoint, they began to spend as if they were having equivalent levels of commercial success — which they were not. Company executive Sheryl Handler had been brought in to co-found the company to bring a level of business maturity to the operation, but that proved to be a mistake. Her corporate spending habits were, putting it charitably, a bit decadent. This habit showed itself early on in the company’s lifespan and continued to worsen as the 1980s wore on. As early as 1984, the very young TMC moved into the top two floors of the expensive Carter Ink building and began spending money on modern, luxurious tech company-like amenities. A couple of these expenses included shelling out for a custom design plan and a gourmet chef. In 1989, as TMC’s work on the CM5 was getting underway, Handler signed a whopping 10-year lease with the Carter Ink building which cost the company $6 million a year — the $37 per square foot for the lease was reportedly 4.5 times higher than Lotus Development Corp. was paying just down the road. Additionally, management rapidly increased headcount by around 40% around this time — resulting in a staff of around 400. The company’s spending was more in line with the company’s growing status in the computing world than its actual financial standing. As Stephen Wolfram described the company’s status at the time, the company was “the place that foreign trade delegations would come to visit to see where American business was at these days.” An IBM computer scientist put it a little differently when he described the company as having cornered the market “on sex appeal in high-performance computing.”

While TMC was in the process of developing machines that were on their way to doing things that no other computer could seemingly do at the time, they were not producing machines that the market wanted as measured in total dollars of actual market demand. With the launch of the government’s HPCC initiative, the company’s research emphasis shifted even more strongly into building the most powerful computer possible by 1992. As it did this, the company continued to put off making the developments it needed to make to find some level of product market fit. The company had produced a machine that cost millions, but was almost exclusively sourcing its customers from the research community. The number of customers and the budget per customer in this community was far more limited than it was in the corporate sphere.

As work on the CM2 and later CM5 progressed, it does seem that the company had at least some opportunity to slowly ramp up efforts in the commercial sector. For example, in the late 1980s TMC sold two Connection Machines to American Express as database mining for improved customer analytics was a growing trend at the time. There was some internal debate about whether or not to start a business supercomputing group at the company — which would be an obvious “yes” for most young and growing companies with shareholders — but nothing much came of it. The company, in practice, clearly preferred to serve the research vision that drove its founder and employees to pursue computing over maximizing shareholder value.

As the work got underway on the CM5, progress was not as smooth as it had been on the previous two generations of machines. The first CM5 would not be completed until October 1991 — and even then it was not quite as powerful as promised because there was a delay in producing chips that fit the machine’s design. However, throughout this 1988 to 1991 period, TMC was peaking from a revenue standpoint. In 1989, thanks to the company’s good reputation within DARPA’s walls, the company achieved its first profit — $700,000 profit on revenues of $45 million. Then, in 1990 the company saw profits of $1 million on revenues of $65 million. This would be the peak of TMC’s profits.

Sadly, for TMC and the world of early 1990s parallel computing, that was as good as things would ever get for the firm. The company’s introduction of an early version of the CM5 in late 1991 was lauded by Hillis as possessing the highest theoretical peak performance if you added enough processors to it — but that particular version of the machine was less powerful than the CM2. It’s very possible that with more DARPA funding and further work upgrading the prototype, this branch of the technology base would have continued to progress from within the expensive walls of the Carter Ink building, but that never came to be.

In August 1991, in the midst of the HPCC initiative’s leaders beginning to make plans for how to deploy most of its sizable budget moving forward, the Wall Street Journal released a piece diving into DARPA’s subsidies of at least two dozen Connection Machines in addition to the machines it had bought. While this in and of itself was not necessarily illegal or even nefarious, this particular political era was one in which anything that looked like “industrial policy” or government interference in markets was extremely scrutinized. DARPA buying its contractor pool BBN Butterfly machines and some machines that resulted from CMU’s Warp project does not seem to have received negative press in the same way. This disparity in PR outcomes might be because buying machines that might soon become obsolete due to technology in development within your portfolio did not come off as bad as subsidizing machines that were not yet considered to be truly finished, market-competitive products. 

Regardless of why the negative press came to be, the Bush administration made a point of ensuring that “industrial policy”-like help to TMC did not continue, and the company did not fare so well when it was abruptly thrust out into the competition of the commercial market. In 1992, TMC reported a $17 million loss. Massive layoffs and spending cuts began around this time. A new CEO was brought on who attempted to broker some kind of merger or partnership deal with Sun Microsystems and IBM, but TMC’s balance sheet was more toxic than its tech was impressive. To give the reader an idea: TMC owed $36 million in rent to the Carter Ink Building over the next six years for its ill-advised ten-year lease. And that was just one of the apparently many dark spots on its balance sheet. A deal like the one the new CEO sought could surely have been brokered around 1989 or 1990, when the company still seemed several years ahead of the field in the rapidly developing field of parallel computation and was beginning to show profits. However, the company had rejected offers for mergers and acquisitions at that time.

TMC filed for Chapter 11 bankruptcy in late 1993 and re-emerged from this as a small software company attempting to sell programs to run on their former competitors’ parallel computers.

Photo of the CM-1 in 1985. I usually share all the photos on this Substack in black and white, but I shared this one in color so maybe the reader could recognize the computer from the CM series’ cameo in the Jurassic Park movie. Photo via the Computer History Museum. https://www.computerhistory.org/revolution/supercomputers/10/73/284

Connection Machine’s Results

Thinking Machines had been capable of hanging onto the dream of building a machine that helped raise all boats in the research community when it was fueled by DARPA funding. With DARPA funding, it was able to largely ignore the commercial market and commit itself to pushing the technology base forward with DARPA’s relatively rare form of risk-tolerant, farther-sighted capital. And they were doing a good job of meeting DARPA’s needs. However, TMC was spending its privately raised capital as if they were doing an equally good job courting commercial customers as they were hitting computing benchmarks. And that was absolutely not the case. Of the 125 machines the company sold in its entire history, only ten percent were to commercial customers. So, when the DARPA relationship took a turn, the company inevitably had to file for Chapter 11 bankruptcy after some rushed attempts at solvency.

The company surely succeeded in its primary goal: producing a machine that proved truly parallel machines could work. Also, the firm did this while developing its technology, step-by-step, in a way that fit with DARPA’s preferred approach to moving the architectures program’s technology base along at the time: gray coding. From a technological standpoint, for the first five or six years it seems the company did an effective job of moving technological progress forward — in the form of a machine with steadily increasing functionality — on a consistent timeline that DARPA was very happy about. From a commercial perspective, however, the company was clearly a failure. Hillis himself admitted as much. At the time of the company’s bankruptcy, Hillis was asked how he felt. He replied, “Sad, that the people who put so much in — the employees and the investors — won’t necessarily be the ones who benefit. And sad that I haven’t been able to build a technology success into a business success.” DARPA surely got more out of the program for what it spent than the company’s investors did. 

In retrospect, DARPA’s general thinking on the problem area seems to have proven quite prescient. A major application area like image processing was exactly what they believed building a truly parallel processing machine like the Connection Machine could do. In the seminar series that Feynman encouraged TMC to put on, they even had a speaker from a not-so-popular field called neural networks come and talk. This story is surely proof that the DARPA ecosystem was, in many ways, functioning quite well — even if no commercial success came from TMC. Hillis and individuals at DARPA were in contact, talking about securing money for the idea contained in his Ph.D. thesis, at least as early as 1983. DARPA’s awareness of his idea and the young company almost from the moment of inception — as well as its recognition of Hillis’ ability to lead such an operation from a technical perspective — should surely be seen as a feather in DARPA’s cap. This awareness, of course, was possibly facilitated by Hillis’ prior connection with Minsky’s DARPA-funded lab at the heavily DARPA-funded MIT. But these kinds of interpersonal networks at key universities that were very good at certain sub-areas of research — such as MIT and CMU in AI research — were what DARPA PMs made it their business to maintain. That sort of relationship management, first and foremost, will always be one of the primary pieces of the puzzle that determines whether or not ARPA-like organizations succeed or fail.

A final note on TMC’s results: it is very possible that even a change as small as Hillis partnering with a much more responsible, business side co-founder than Handler — who proved adept at fundraising, but deficient in many key areas that TMC required — could have been enough to shepherd the company to a longer lifespan. From a commercial strategy standpoint, an individual like this may have helped TMC find commercial uses that fit in with the company’s goals and culture. From a financial standpoint, a better CEO might have ensured that the company either did not raise money from misaligned capital pools or did not spend the money it did raise as poorly as it did. 

Connection Machine’s Lessons Learned (and Caveats)

One quote that encapsulates several of the PM-relevant lessons from the Thinking Machines Corporation’s work and death was said by Hillis as the company was filing for bankruptcy, “The real money is in handling Wal-Mart’s inventory rather than searching for the origins of the universe.” From the perspective of a pure venture capitalist, this quote might read as the complaint of an academic wishing that the financial markets rewarded what researchers found important. Of course, that is not the ideal perspective most venture capitalists would not want in their portfolio founders. But the quote can also be read as Hillis calling attention to what might be a very major market inefficiency. Research companies like Thinking Machines are great vehicles for building new, exploratory technology, but there is often a very salient tradeoff between a company like this moving the technology base forward and profit-maximizing for shareholders in the near term.

Let’s not forget, the ILLIAC IV project did not only run into difficulties because its team was attempting to build a machine with too many novel components at once. Slotnick’s “battleship in a bathtub” remark was explicitly describing how ill-suited modern universities had become to building big, new machines. Hillis did not make the identical mistake of trying to build the Connection Machine from within a university, but he still did make a mistake in choosing his organizational setup. TMC ran into the conundrum of raising money from private investors — who would naturally prefer the company do things like build a computer to handle Wal-Mart’s inventory — as well as a funder (and initial customer) in DARPA that primarily cared about building a computer to move technology forward. In most cases, a company is going to make one of its two (rather different) funders unhappy in a situation where their incentives are only partially aligned.

Had Thinking Machines only pursued DARPA funding and spent more frugally — as an entity like ISI, which ran DARPA’s MOSIS program, would have done — it is very possible the company would have continued to progress as the darling of DARPA’s architectures program for the foreseeable future. If the company would have taken this approach, it could still have raised money from private markets at some point, just later on in its existence when it had a product that was closer to market-ready and had a more natural customer. Additionally, the company could have attempted to find a variety of customers so it would not have all its eggs in the DARPA basket the way BBN (covered at length in a coming piece) was known to have done in the 1960s and 1970s. BBN, in its early existence, did not seem to be purely profit-maximizing. Rather, it sought a variety of contracts — from DARPA, private philanthropies, engineering industry, and more — that its researchers found new and useful, but that still paid enough to facilitate a rather profitable operation.

Of course, none of this alternative history brings any assurance of success either. The political winds may have blown against TMC one way or another regardless. It should be noted, though, that Hillis’ Wal-Mart quote was predictive in some sense. The company that did make possibly the biggest parallel computing breakthrough in the late 1990s — NVIDIA with its new GPU — found a way to fund its breakthrough research by servicing a market that required high-tech solutions and was surprisingly large given how unimportant it was in the grand scheme of things: gaming graphics. 

It is also important to note that, in the first five or six years of TMC’s existence, it seems that the gray coding approach was working just as Squires had hoped. TMC was often putting out versions of Connection Machines that did not have the functionality or usability that many users would have wanted. But that could be considered a marketing issue as much as anything else. In bringing new functionality to the machines in this steady way, this gradual decrease in user dissatisfaction seems par for the course. The rate of progress of the Connection Machine seems, generally, far preferable to the messy process that was the ILLIAC IV development. Of course, individual PMs will know best when either approach is most suitable to a given project or portfolio. A variety of factors can impact the decision and there is no one-size-fits-all solution. One factor that could impact a decision like this, for example, is the higher the probability of failures or surprises in developing components in a given project, the more gray coding might be the clearly responsible approach. However, there are obviously also cases in which multiple components need to be changed between iterations of a machine for anything truly new to come of the process. When to apply either approach is, as always, best left up to a smart PM.

While the legacy of the Connection Machine is a complicated one, it was still the gem in an architectures portfolio that accomplished what it sought to do. As Roland and Shiman put it:

The first stage of the SC architectures program proved unquestionably that parallelism was viable and did have promise in solving many real-world problems…According to Professor John Hennessy of Stanford University, a respected authority on microelectronics and computer architectures, computer designers before SC, in their quest for greater speed and performance, kept running into the wall posed by the fundamental physical limits to the improvement of conventional systems and components. Each architect would try, and each would join the pile at the base of the wall. It was Squires, he said, who showed the way, and, through parallelism, led designers over the wall.

Hopefully the collection of history and lessons in this post prove of some use to PMs thinking through decisions like those that arose in the ILLIAC IV and CM1-CM5 projects.

ILLIAC IV

Connection Machine

  • Chapter 5, pages 158-184 of Strategic Computing: How DARPA Built the Computer Age 
    • Provides more detail on the history of Squires’ approach to the architectures program and other projects that made up the portfolio. Some details about how the projects were run can also be found there. 
  • The Rise and Fall of Thinking Machines
    • A thorough article diving into the lifespan of the company from a business and cultural perspective. The piece covers Thinking Machine’s corporate decision-making and relationships from its early years until its bankruptcy.
  • Richard Feynman and the Connection Machine
    • This Physics Today article, written by Hillis himself, not only discusses Feynman and his work with the company, but also paints a picture of what went on in the early days of the company when it was a bunch of technical researchers working out of an old mansion outside of Boston.
  • Thinking Machines Rather Than Markets
    • This Washington Post article, written in the midst of TMC’s demise, provides additional facts about the situation, but will be primarily useful to readers of this piece in helping them understand how reasonable-seeming help provided by an actor like DARPA can be portrayed depending on the given political climate.

1 In 1962, IPTO’s $9 million budget was entirely earmarked for category 6.1 research.

2 For the rest of the interpersonal aspects of the accounts in the next few paragraphs, I’ll rely on Slotnick’s memoir since he seemed to have a more clear recollection of the order in which things happened than Sutherland.

3 Another FreakTakes piece dives into the analogous case of early molecular biology researchers — many of whom would eventually win Nobel Prizes — taking risks and changing multiple parameters between experiments. The piece explores what they did and why they thought that was an optimal strategy for a field as young as molecular biology.

4 For those curious about what “AI applications” looked like in the mid-1980s, check out next week’s post on the Autonomous Land Vehicle, Pilot’s Associate, and Battle Management

Contact

© 2024 Good Science Project | Privacy Policy | All Rights Reserved.