One awaits a macroinvention as the evangelical does a miracle. Truly original breakthroughs in technology are rare and random events, impossible to induce and difficult to inspire. To Joel Mokyr—the economic historian who famously coined the term—macroinventions “without clear precedent… emerge more or less ab nihilo” and “do not seem to obey obvious laws, do not necessarily respond to incentives, and defy most attempts to relate them to exogenous economic variables.” Indeed, many can be traced to little more than “genius, luck, or serendipity.” And while most advances in productivity result from the more numerous microinventions—the iterative and incremental refinements that turn curiosities into economic realities—diminishing returns inevitably set in without periodic infusions of genuine novelty. One could increase communication speeds by many multiples by improving roads and mail carriages; but eventually reductions in information delays would plateau without a paradigm shift—the invention of the telegraph, say. Sometimes, we need a new kind of technology to do something entirely new altogether. You might plausibly argue that, with our present demands for carbon capture, batteries, and clean energy, this need is more pressing than ever before.
The steam engine—the flagship technology of the British Industrial Revolution—is the canonical macroinvention. While it’s difficult to pinpoint a single moment in which it came into being, the steam engine was, in aggregate, a radical new solution to a very old problem: that of converting stored energy into power. Prior to 1700, the limits to human productivity were fixed by the potential of water, wind, and old-fashioned muscle. The use of power technology was constrained geographically by the location of rivers and temporally by the seasons (which iced over and dried up streams) and the weather (which governed the utility of the windmill). It could not be moved, had to be operated inconsistently, and was strictly limited in the possible application of horsepower. The steam engine, however, unlocked the tremendous potential of the latent heat energy stored over millennia underground as coal. Power technology could be spread across the land, automating and augmenting human labor to an extent impossible even with the most efficient windmills and waterwheels of the day; it could be used to move ships and trains; and it could operate in all seasons and at untold intensities. Though the finest steam engines have long since been superseded, the basic principle—conversion of fossil fuel into mechanical energy—remains the foundation for the economic life of our civilization. It’s a legacy that’ll be difficult to escape.
While Europeans had dabbled with prototype steam machines for centuries, beginning with the fabulous aeolipile suggested by Hero of Alexandria, the scientific basis for the modern technology awaited the Scientific Revolution of the seventeenth century. Two key developments occurred during the 1640s. The Grand Duke of Tuscany had ordered his workmen to dig a well to supply his palace with water, but the suction pump used could not bring the fluid above seven feet from the top of the pit. Galileo was called in as a consultant. Though influenced by the Aristotelian dictum that “nature abhors a vacuum,” he nevertheless observed that there were limits to this abhorrence, which seemed to end at 33 feet.1 While under house arrest for his espousal of Copernicanism, Galileo suggested to his friend and secretary, Evangelista Torricelli, that he take up the problem. In 1644, Torricelli inverted a glass tube of mercury in a bowl containing the same substance. The mercury stabilized in a column 76 centimeters high with an atmosphere above it—the world’s first barometer. The fluid was suspended, he realized, not by “nature's intolerance of a vacuum” but by the weight of the air itself. “The horror vacui doctrine, lasting for two millennia,” wrote one historian, “was finally thrown into the dustbin of history.” Torricelli’s friends sent word to Blaise Pascal in France, who showed that the deeper one submerged a barometer in water, the higher the mercury was forced upward by water pressure.
During the following decade, Otto von Guericke, the mayor of Magdeburg, performed a series of experiments that proved of equal importance. In 1654, he pumped air out of a cylinder fitted with a piston, which slowly descended under the weight of the atmosphere, despite the efforts of twenty rope-pulling men to restrain it. The following year, he evacuated air from between two hemispheres, and it took sixteen horses to rip them apart. Finally, in 1661, he repeated the piston exercise by using the mechanism to pull up a scale with weights. In 1666, Christiaan Huygens took a dramatic next step: he exploded gunpowder beneath a piston to drive it up a cylinder, and then released the gases to create a vacuum below it. Air pressure forced the piston downward, raising an attached load. And while gunpowder was obviously unsuited for a power technology, Huygens’s assistant, Denis Papin, substituted steam: filling the cylinder to push up the piston, and then condensing it to create the vacuum. This was the first prototype steam engine. He wrote of this result in a 1690 paper published in England, to which he had fled in fear of religious persecution. Eight years later, Captain Thomas Savery, a fellow of the Royal Society—which had been coolly indifferent to Papin’s achievement—had acquired a patent for the first working steam engine.
Savery had been interested in the problem of pumping water out of mines, something that Papin had himself considered. This issue that motivated Thomas Newcomen to conduct the research that led to the original successful steam engine. Indeed, Robert Allen goes so far as to call the steam engine itself “Newcomen’s macro-invention” and suggests that Savery’s design was not for an engine at all, but rather a flawed sort of vacuum pump. Fair or not, the units installed following the establishment of the Dudley Castle Machine of 1712 bore his name and operated on his, not Savery’s, principles. The Newcomen engine worked as follows: a balance beam was set atop a column with one end attached to a cylinder-piston and the other to the rod used to pump out the mine. Water was heated in a boiler and the resultant steam admitted to the cylinder by using a valve. Cold water was injected into the cylinder, condensing the steam and causing the atmospheric weight to drive down the piston, which lowered the near end of the beam and lifted the end with the rod. The result was a machine that was powerful, safe, and operable by contemporary craftsmen.
Though the Newcomen engine was the end product of ten years of development, it still had major flaws. First, it consumed fuel voraciously, thanks to the need to cool and heat the cylinder with each stroke. And second, the irregularity of the drive produced meant that rotational motion—needed to power mills—was not directly possible. Both factors conspired to restrict the spread of the Newcomen engine to the vicinity of the coal fields, where fuel was essentially free. As a result, the initial diffusion of the machines was frustratingly slow. In 1733, when Newcomen was finally freed from his joint patent with Savery, there were only about 100 engines of any kind in all of Britain. Fuel-efficiency increased gradually until 1769, from 45 to 30 pounds per horsepower-hour, at which point the engineer John Smeaton systematically improved the design through experimentation to reach 17.6. This, however, proved to be the hard ceiling of the Newcomen design.
By this stage, however, the Newcomen engine was being made obsolete by the better-known James Watt. While employed at the University of Glasgow to repair a model, he realized that the machine’s fuel issues could be solved by chilling the steam in the famous “separate condenser,” removing the energy loss from repeatedly cooling and reheating the cylinder. The Watt design’s 8.8 pounds per hp/hr were a 50 percent saving over Smeaton’s furthest refinement of the Newcomen paradigm. In 1800, Richard Trevithick designed the high-pressure engine, which used the force of injected steam to drive up the piston and, since it eschewed the separate condenser, air pump, and beam, was cheaper to install and more compact. By saving on fuel and capital, the new method set off a train of microinventions that lowered consumption to 2 pounds by 1830 and was eventually applied to trains and ships. At the prompting of his partner Matthew Boulton, Watt also developed the “sun and planets gears,” which translated the engine’s reciprocating into rotatory motion. Combined with the double-acting technique of inserting steam both above and below the piston—ensuring a smoother drive—the Watt engine could be applied to mills and textile machinery.
As the Watt and Trevithick designs were refined and recombined, steam power began to diffuse throughout the British economy. In 1800, there were 2500 steam engines operating in Britain, of which two-thirds were Newcomen- and the remainder Watt-types. But the critical fact is that this process was staggeringly slow. In 1760, a half-century after Newcomen’s macroinvention, only 5,000 out of the 85,000 horsepower installed in Britain was provided by steam. In 1800, with Watt’s efficiency increases and applicability expansions now best-practice, the corresponding figures were still just 35,000 and 170,000. Even in 1830, steam power had only just reached parity with water power, when both sources comprised 160,000 horsepower each out of 340,000 total. This was thirty years after Trevithick, fifty years after the “sun and planets” gears, a century after the Savery-Newcomen patent expired, and 150 years after Papin had built his prototype. And it was only in the four decades between 1830 and 1870 that the high-pressure compounding steam engine became the dominant power provider in Britain. Steam horsepower per worker was no higher in 1850 than in 1760, as the technology diffused barely at the rate of labor force growth.
Why? First, best-practice technologies were not always cost-effective. Robert Allen discusses at length the extent to which diffusion was mediated by fuel costs, which were lowest in Britain—particularly in the Northeast coal regions—and higher in many continental industrial centers. The areas of effectively free energy were a British luxury, and equally fortunate was the local demand for mine pumping fostered by the metallurgical industry. As late as 1840, Britain commanded double the steam power of France, Prussia, Belgium, and the United States combined. Second, average practice was slow to reach the productivity frontier regardless of cost-effectiveness. The Lancashire cotton industry used low-pressure engines until the late 1840s, decades after the technology had become obsolete. This may have had something to do with initial capital costs, which actually increased until the 1830s and offset many of the fuel savings gains made possible by Smeaton, Watt, and Trevithick. A skilled workforce of engineers able to run and repair the machines also had to be expanded sufficiently to ensure that firms could continue to operate them at less-than-prohibitive expense. This point, along with the primacy of coal costs, is confirmed by Nuvolari et al (2011), who find that a one-shilling increase in price decreased the expected number of Newcomen engines in a county by 0.7. But we should also not underestimate the nightmarish weight of history upon the living. Inertia and path dependence continue to slow adoption and diffusion of novelties long after the economic tide appears to have turned.
We have many macroinventions we might desire today, from carbon capture to nuclear fusion, but the history of such events tells us they come unexpectedly, stochastically, and, above all, rarely. There were rational economic and scientific reasons underlying the invention of the steam engine. But no natural law governed when or whether the Newcomen engine was invented, and still less in the case of Papin’s prototype. Economic incentives, as Mokyr points out, may decide the direction of technological change, but they have no deterministic grasp of the pace. The Scientific Revolution made the steam engine possible, but not immediately inevitable. And invention is only the first step. Legacy economic systems change only slowly—it’s one thing to invent the technology, and quite another to diffuse it through the entire economy. Steam’s conquest of British power production came over a century after Watt’s separate condenser, and lagged two centuries behind Papin. And that was just one tiny island corner of a planet that even today is still in the throes of its first industrial revolution.
There is some disagreement as to the exact height. Mokyr says 31 feet, and Allen 28.
An amazing research, thanks for summarizong it! So many parallels with today's tech waves and Cloud adoption by enterprises.