Discontinuous progress investigation

This page is in the process of being revised to account for new research. Its present contents may be out of date.

We are collecting cases of discontinuous technological progress, to inform our understanding of whether artificial intelligence research is likely to undergo such a discontinuity. This page details our investigation.

We found that 17 technological trends that underwent at least one discontinuous jump in improvement equivalent to more than a decade at previous rates, 13 of which underwent more than a century of progress. All together we found 68 discontinuities of over ten years, 37 of them of over 100 years.

Details

Motivations

We are interested in learning whether artificial intelligence is likely to see discontinuous progress in the lead-up to human-level capabilities, or to produce discontinuous change in any other socially important metrics (e.g. percent of global wealth possessed by a single entity, economic value of hardware). We are interested because we think this tells us a lot about the plausibility of different future scenarios and about which research and other interventions are best now, and also because it is a source of disagreement, and so perhaps fruitful for resolution.1

We seek to answer this question by investigating the prevalence and nature of discontinuities in other technological progress trends. The prevalence can then act as a baseline for our expectations about AI, which can be updated with any further AI-specific evidence,2 including that which comes from looking at the nature of other discontinuities (for instance, whether they arise in circumstances that are predicted by the arguments that are made for predicting discontinuous progress in AI).

In particular, we want to know:

  • How common are large discontinuities in metrics related to technological progress?
  • Do any factors predict where such discontinuities will arise? (For instance, is it true that progress in a conceptual endeavor is more likely to proceed discontinuously? If there have been discontinuities in progress on a metric in the past, are further discontinuities more likely?) 

As a secondary goal, we are interested in learning about the circumstances that have surrounded discontinuous technological change in the past, insofar as it may inform our expectations about the social consequences of discontinuous progress in AI, should it happen.

Methods

Main article: methodology for discontinuous progress investigation.

To learn about the prevalence and nature of discontinuities in technological progress, we:

  1. Searched for potential examples of discontinuous progress (e.g. ‘Eli Whitney’s cotton gin’)
  2. Chose specific metrics related to these potential examples (e.g. ‘cotton ginned per person per day’, ‘value of cotton ginned per cost’) and found historic data on progress on those metrics (usually in conjunction with choosing metrics, since metrics for which we can find data are much preferred). Some datasets we found already formed in one place, while others we collected ourselves from secondary sources.
  3. Defined a ‘rate of past progress’ throughout each historic dataset (e.g. if the trend is broadly flat then gets steeper, we decide whether to call this exponential progress, or two periods of linear growth.)
  4. Measured the discontinuity at each datapoint in each trend by comparing the progress at the point to the expected progress at that point based on the last datapoint and the rate of past progress (e.g. if the last datapoint five years ago was 600 units, and progress had been going at two units per year, and now a development took it to 800 units, we would calculate 800 units – 600 units = 200 units of progress = 100 years of progress in 5 years, for a 95 year discontinuity.)
  5. Noted any discontinuities of more than ten years (‘moderate discontinuities’), and more than one hundred years (‘large discontinuities’)
  6. Noted anything interesting about the circumstances of each discontinuity (e.g. the type of metric it was in, the events that appeared to lead to the discontinuity, the patterns of progress around it.)

Note that this is not an attempt to rigorously estimate the frequency of discontinuities in arbitrary trends, since we have not attempted to select arbitrary trends. We have instead selected trends we think might contain large discontinuities. Given this, it may be used as a loose upper bound on the frequency of discontinuities in similar technological trends.

It is likely that there are a large number of errors in this collection of data and analysis, based on the rate at which we have found and corrected them, and the unreliability of sources used.

Definitions

Throughout, we use:

  • Moderate discontinuity: 10-100 years of progress at previous rates occurred on one occasion
  • Large discontinuity: at least 100 years of progress at previous rates occurred on one occasion
  • Substantial discontinuity: a moderate or large discontinuity

Summary observations

Notable findings3

  • We collected 21 case studies of potentially discontinuous technological progress (see Case studies below) and investigated 37 trends associated with them.
  • 17 trends had a substantial discontinuity, and 13 had a large discontinuity.4
  • Trends with any discontinuities usually had more than one. We found 31 moderate discontinuities and 37 large discontinuities in total (a rough expectation of 1.8 discontinuities, 1 of them large)5

Case studies

This is a list of areas of technological progress which we have tentatively determined to either involve discontinuous technological progress, or not. Note that we largely investigate cases that looked likely to be discontinuous.

Key:

LD: Large discontinuity (>100 years of progress at once)

MD: Moderate discontinuity (>10 years of progress at once)

NSD: No sign of substantial discontinuities

IIP: Investigation in progress

Ship size

Main article: Historic trends in ship size

Trends for ship tonnage (builder’s old measurement) and ship displacement for Royal Navy first rate line-of-battle ships saw eleven and six discontinuities of between ten and one hundred years respectively during the period 1637-1876, if progress is treated as linear or exponential as usual. There is a hyperbolic extrapolation of progress such that neither measurement sees any discontinuities of more than ten years.

We do not have long term data for ship size in general, however the SS Great Eastern seems to have produced around 400 years of discontinuity in both tonnage (BOM) and displacement if we use Royal Navy ship of the line size as a proxy, and exponential progress is expected, or 11 or 13 in the hyperbolic trend.

Figure 1a: Record tonnages for Royal Navy ships of the line
Figure 1b: Ship weight (displacement) over time, Royal Navy ships of the line and the Great Eastern, a discontinuously large civilian ship. The largest ship in the world three years prior to the Great Eastern was around 4% larger than the Ship of the Line of that time in this figure, so we know that the overall largest ship trend cannot have been much steeper than the Royal Navy ship of the line trend shown.

Image recognition

Main article: Effect of AlexNet on historic trends in image recognition

AlexNet did not represent a greater than 10-year discontinuity in fraction of images labeled incorrectly, or log or inverse of this error rate, relative to progress in the past two years of competition data.

Figure 2: Error rate (%) of ImageNet competitors from 2010 – 2012

Transatlantic passenger travel

Main article: Historic trends in transatlantic passenger travel

The speed of human travel across the Atlantic Ocean has seen at least seven discontinuities of more than ten years’ progress at past rates, two of which represented more than one hundred years’ progress at past rates: Columbus’ second journey, and the first non-stop transatlantic flight.

Figure 3a: Historical fastest passenger travel across the Atlantic (speeds averaged over each transatlantic voyage)
Figure 3b: Previous figure, shown since 1730

Transatlantic message speed

Main article: Historic trends in transatlantic message speed

The speed of delivering a short message across the Atlantic Ocean saw at least three discontinuities of more than ten years before 1929, all of which also were more than one thousand years: a 1465-year discontinuity from Columbus’ second voyage in 1493, a 2085-year discontinuity from the first telegraph cable in 1858, and then a 1335-year discontinuity from the second telegraph cable in 1866.

Figure 4: Average speed for message transmission across the Atlantic.

Long range military payload delivery

Main article: Historic trends in long range military payload delivery

The speed at which a military payload could cross the Atlantic ocean contained six greater than 10-year discontinuities in 1493 and between 1841 and 1957:

DateMode of transportKnotsDiscontinuity size
(years of progress
at past rate)
1493Columbus’ second voyage5.81465
1884Oregon18.610
1919WWI Bomber
(first non-stop
transatlantic flight)
106351
1938Focke-Wulf Fw 200 Condor17419
1945Lockheed Constellation28825
1957R-7 (ICBM)~10,000~500
Figure 5: Historic speeds of sending hypothetical military payloads across the Atlantic Ocean

Bridge spans

Main article: Historic trends in bridge span length

We measure eight discontinuities of over ten years in the history of longest bridge spans, four of them of over one hundred years, five of them robust as to slight changes in trend extrapolation.

Figure 6: Record bridge span lengths for five bridge types since 1800

Light intensity

Main article: Historic trends in light intensity

Maximum light intensity of artificial light sources has discontinuously increased once that we know of: argon flashes represented roughly 1000 years of progress at past rates.

Figure 7: Light intensity trend since 1800 (longer trend available)

Book production

Main article: Historic trends in book production

The number of books produced in the previous hundred years, sampled every hundred or fifty years between 600AD to 1800AD contains five greater than 10-year discontinuities, four of them greater than 100 years. The last two follow the invention of the printing press in 1492.

The real price of books dropped precipitously following the invention of the printing press, but the longer term trend is sufficiently ambiguous that this may not represent a substantial discontinuity.

The rate of progress of book production changed shortly after the invention of the printing press, from a doubling time of 104 years to 43 years.

Figure 8a: Total book production in Western Europe
Figure 8b: Real price of books in England

Telecommunications performance

Main article: Historic trends in telecommunications performance

There do not appear to have been any greater than 10-year discontinuities in telecommunications performance, measured as:

  • bandwidth-distance product for all technologies 1840-2015
  • bandwidth-distance product for optical fiber 1975-2000
  • total bandwidth across the Atlantic 1956-2018

Radio does not seem likely to have represented a discontinuity in message speed.

Figure 9a: Growth in bandwidth-distance product across all telecommunications during 1840-2015 from Agrawal, 20166
Fig. 8.8
Figure 9b:
Bandwidth-distance product in fiber optics alone, from Agrawal, 20167 (Note: 1 Gb = 10^9 bits)
Figure 9c: Transatlantic cable bandwidth of all types. Pre-1980 cables were copper, post-1980 cables were optical fiber.

Cotton gins

Main article: Effect of Eli Whitney’s cotton gin on historic trends in cotton ginning

We estimate that Eli Whitney’s cotton gin represented a 10 to 25 year discontinuity in pounds of cotton ginned per person per day, in 1793. Two innovations in 1747 and 1788 look like discontinuities of over a thousand years each on this metric, but these could easily stem from our ignorance of such early developments. We tentatively doubt that Whitney’s gin represented a large discontinuity in the cost per value of cotton ginned, though it may have represented a moderate one.

Figure 10: Claimed cotton gin productivity figures, 1720 to modern day, coded by credibility and being records. The last credible best point before the modern day is an improved version of Whitney’s gin, two years after the original (the original features in the two high non-credible claims slightly earlier).

Altitude

Main article: Historic trends in altitude

Altitude of objects attained by man-made means has seen six discontinuities of more than ten years of progress at previous rates since 1783, shown below.

YearHeight (m)Discontinuity (years)Entity
178440001032Balloon
180372801693Balloon
191842,300227Paris gun
194285,000120V-2 Rocket
1944174,60011V-2 Rocket
1957864,000,00035Pellets (after one day)
Figure 11: Post-1750 altitudes of various objects, including many non-records. Whether we collected data for non-records is inconsistent, so this is not a complete picture of progress within object types. See image in detail here.

Slow light

Main article: Historic trends in slow light technology

Group index of light appears to have seen discontinuities of 22 years in 1995 from Coherent Population Trapping (CPT) and 37 years in 1999 from EIT (condensate). Pulse delay of light over a short distance may have had a large discontinuity in 1994 but our data is not good enough to judge. After 1994, pulse delay does not appear to have seen discontinuities of more than ten years.

Figure 12: Progress in pulse delay and group index. “Human speed” shows the rough scale of motion familiar to humans.

Particle accelerators

Main article: Historic trends in particle accelerator performance

None of particle energy, center-of-mass energy nor Lorentz factor achievable by particle accelerators appears to have undergone a discontinuity of more than ten years of progress at previous rates.

Figure 13a: Particle energy in eV over time
Figure 13b: Center-of-mass energy in eV over time
Figure 13c: Lorentz factor (gamma) over time.

Penicillin on syphilis

Main article: Penicillin and historic syphilis trends

Penicillin did not precipitate a discontinuity of more than ten years in deaths from syphilis in the US. Nor were there other discontinuities in that trend between 1916 and 2015.

The number of syphilis cases in the US also saw steep decline but no substantial discontinuity between 1941 and 2008.

On brief investigation, the effectiveness of syphilis treatment and inclusive costs of syphilis treatment do not appear to have seen large discontinuities with penicillin, but we have not investigated either thoroughly enough to be confident.

Syphilis infection in the US
Figure 14a: Syphilis—Reported Cases by Stage of Infection, United States, 1941–2009, according to the CDC8
syphilis
Figure 14b: Syphilis and AIDS mortality rates in the US during the 20th century.9

Nuclear weapons

Main article: Effect of nuclear weapons on historic trends in explosives

Nuclear weapons constituted a ~7 thousand year discontinuity in energy released per weight of explosive (relative effectiveness).

Nuclear weapons do not appear to have clearly represented progress in the cost-effectiveness of explosives, though the evidence there is weak.

Figure 15: Relative effectiveness of explosives, up to early nuclear bomb (note change to log scale)

High temperature superconductors

Main article: Historic trends in the maximum superconducting temperature

The maximum superconducting temperature of any material up to 1993 contained four greater than 10-year discontinuities: A 14-year discontinuity with NbN in 1941, a 26-year discontinuity with LaBaCuO4 in 1986, a 140-year discontinuity with YBa2Cu3O7 in 1987, and a 10-year discontinuity with BiCaSrCu2O9 in 1987.

YBa2Cu3O7 superconductors seem to correspond to a marked change in the rate of progress of maximum superconducting temperature, from a rate of progress of .41 Kelvin per year to a rate of 5.7 Kelvin per year.

Figure 16: Maximum superconducting temperate by material over time through 2015

Land speed records

Main article: historic trends in land speed records

Land speed records did not see any greater-than-10-year discontinuities relative to linear progress across all records. Considered as several distinct linear trends it saw discontinuities of 12, 13, 25, and 13 years, the first two corresponding to early (but not first) jet-propelled vehicles.

The first jet-propelled vehicle just predated a marked change in the rate of progress of land speed records, from a recent 1.8 mph / year to 164 mph / year.

Figure 17: Historic land speed records in mph over time. Speeds on the left are an average of the record set in mph over 1 km and over 1 mile. The red dot represents the first record in a cluster that was from a jet propelled vehicle. The discontinuities of more than ten years are the third and fourth turbojet points, and the last two points.

Chess AI

Main article: Historic trends in chess AI

The Elo rating of the best chess program measured by the Swedish Chess Computer Association did not contain any greater than 10-year discontinuities between 1984 and 2018. 

Figure 18: Elo ratings of the best program on SSDF at the end of each year.

Flight airspeed

Main article: Historic trends in flight airspeed records

Flight airspeed records between 1903 and 1976 contained one greater than 10-year discontinuity: a 19-year discontinuity corresponding to the Fairey Delta 2 flight in 1956.

The average annual growth in flight airspeed markedly increased with the Fairey Delta 2, from 16mph/year to 129mph/year.

Figure 19: Flight airspeed records over time

Structure heights

Main article: Historic trends in structure heights

Trends for tallest ever structure heights, tallest ever freestanding structure heights, tallest existing freestanding structure heights, and tallest ever building heights have each seen 5-8 discontinuities of more than ten years. These are:

  • Djoser and Meidum pyramids (~2600BC, >1000 year discontinuities in all structure trends)
  • Three cathedrals that were shorter than the all-time record (Beauvais Cathedral in 1569, St Nikolai in 1874, and Rouen Cathedral in 1876, all >100 year discontinuities in current freestanding structure trend)
  • Washington Monument (1884, >100 year discontinuity in both tallest ever structure trends, but not a notable discontinuity in existing structure trend)
  • Eiffel Tower (1889, ~10,000 year discontinuity in both tallest ever structure trends, 54 year discontinuity in existing structure trend)
  • Two early skyscrapers: the Singer Building and the Metropolitan Life Tower (1908 and 1909, each >300 year discontinuities in building height only)
  • Empire State Building (1931, 19 years in all structure trends, 10 years in buildings trend)
  • KVLY-TV mast (1963, 20 year discontinuity in tallest ever structure trend)
  • Taipei 101 (2004, 13 year discontinuity in building height only)
  • Burj Khalifa (2009, ~30 year discontinuity in both freestanding structure trends, 90 year discontinuity in building height trend)
Figure 20a: All-time record structure heights, long term history
Figure 20b: All-time record structure heights, recent history
Figure 20c: All-time record freestanding structure heights, long term history
Figure 20d: All-time record freestanding structure heights, recent history
Figure 20e: At-the-time record freestanding structure heights, long term history
Figure 20f: At-the-time record freestanding structure heights, recent history
Figure 20g: All-time record building heights, longer term history
Figure 20h: All-time record building heights, longer term history

Breech loading rifles

Main article: Effects of breech loading rifles on historic trends in firearm progress

Breech loading rifles do not appear to have represented a discontinuity in firing rate of guns, since it appears that other guns had a similar firing rate already. It remains possible that breech loading rifles represent a discontinuity in another related metric.

Incomplete case studies

This is a list of cases we have partially investigated, but insufficiently to include in this page.

Extended observations

This spreadsheet contains summary data and statistics about the entire set of case studies, including all calculations for findings that follow.

Prevalence of discontinuities

  • We investigated 37 trends in around 21 broad areas10
  • Of the 37 trends that we investigated, we found 17 to contain at least one substantial discontinuity, and 13 to contain at least one large discontinuity. (Note that our trends were selected for being especially likely to contain discontinuities, so this is something like an upper bound on their frequency in trends in general. However some trends we investigated for fairly limited periods, so these may have contained more discontinuities than we found.)
  • Trends with any discontinuities usually had more than one.
  • In total, we found 37 large discontinuities and 31 moderate discontinuities.
  • Thus among our case studies, trends had in expectation at least 1.8 discontinuities each, including 1 large discontinuity each (this should underestimate, since we did not necessarily investigate trends for the entirety of their history)

Metrics which saw more discontinuities

Among our case studies, metrics saw more discontinuities if:

  • They had already seen one
    Among our case studies, a trend with a discontinuity was much more likely to have a second one than a trend was to have a first discontinuity. P(discontinuity|trend in our set) = 0.46, whereas P(>1 discontinuity| 1 discontinuity & trend in our set) = 0.75. For large discontinuities, the situation was similar. P(large discontinuity|trend in our set) = 0.35, whereas P(>1 discontinuity| 1 discontinuity & trend in our set) = 0.77.
  • The overall history appeared to be a mixture of exponential and linear sections
    The 8 metrics whose trends we modeled as ‘mixed’ saw an average of 5.9 discontinuities (3.4 large) each, compared to 1.4 each (0.6 large) for seven linear trends, 0.9 each (0.5 large) for eleven exponential trends, and 0.09 each (0 large) for eleven other metrics (including those with hyperbolic trends and metrics where we only investigated one development and did not determine the shape of the long-run trend). 
  • They were to do with ‘product features’
    See ‘nature of metrics’ section below. Other common categories with fewer discontinuities were ‘technical features’ and ‘product performance proxies’.
Nature of metrics

We categorized each metric as one of:

  1. ‘technical’: to do with basic physical parameters (e.g. light intensity, particle energy in particle accelerators)
  2. ‘product’: to do with usable goods or services (e.g. cotton ginned per person per day, size of largest ships, height of tallest structures)
  3. ‘industry’: to do with an entire industry rather than individual items (e.g. total production of books)
  4. ‘societal’: to do with society at large (e.g. syphilis mortality)

We also categorized each metric as one of:

  1. ‘feature’: a characteristic that is good, but not close to encompassing the purpose of most related efforts (e.g. ship size, light intensity)
  2. ‘performance proxy’: approximates the purpose of the endeavor (e.g. cotton ginned per person per day, effectiveness of syphilis treatment)
  3. ‘value proxy’: approximates the all-things-considered value of the endeavor (e.g. real price of books, cost-effectiveness of explosives)

Most metrics fell into ‘product feature’ (15) ‘technical feature’ (8) or ‘product performance proxy’ (6), with the rest (8) spread across the categories.

Here is what these trends are like (or see table below): 

  1. Product feature (15 metrics) 
    P(discontinuity) = 67%, P(large discontinuity) = 60%. On average, a trend sees 3.5 discontinuities, 2 of them large. 
  2. Technical feature (8 metrics)
    P(discontinuity) = 50%, P(large discontinuity) = 38%. On average, a trend sees 1 discontinuity, 0.4 of them large.
  3. Product performance proxy (6 metrics)
    P(discontinuity) = 30%, P(large discontinuity) = 0%. On average, a trend sees 0.3 discontinuities, none of them large.
  4. Everything else (8 metrics)
    P(discontinuity) = 13%, P(large discontinuity) = 13%. On average, a trend sees 0.6 discontinuities, 0.5 of them large.

These differences could arise from factors other than the true propensities of these different kinds of metric to undergo discontinuous change, such as different categories of subject matter lending themselves to different specific types of metric that are more or less likely to capture discontinuities, or differences in ease of gathering enough data to explore long time periods rather than short ones. We have not investigated these factors.

  product feature technical feature product performance proxy other categories
10 year + discontinuities
       
discontinuities 53 8 2 5
trends 15 8 6 8
trends with discontinuities 10 4 2 1
discontinuities/ trend 3.5 1.0 0.3 0.6
fraction of trends with discontinuity 0.67 0.50 0.33 0.13
100 year + discontinuities
       
large discontinuities 30 3 0 4
trends 15 8 6 8
trends with large discontinuities 9 3 0 1
large discontinuities / trend 2.0 0.4 0.0 0.5
fraction of trends with large discontinuity 0.60 0.38 0.00 0.13
  1. For instance, if the development of advanced AI takes place in the context of a large discontinuity, then it is arguably more likely to involve large shifts in power, to take place sooner than predicted, to be surprising, to be disruptive, and to be dangerous. Also, our research should investigate questions such as how to prepare or be warned, rather than questions like when the present trajectories of AI progress will reach human-level capabilities. See likelihood of discontinuous progress around the development of AGI for more discussion.
  2. A large collection of considerations that may make AI unusual is available here.
  3. Extended observations below contains further observations, as well as more detail on these.
  4. Recall that our trends were selected for being especially likely to contain discontinuities, so this is something like an upper bound on their frequency in trends in general. However some trends we investigated for fairly limited periods, so these may have contained more discontinuities than we found.
  5. This underestimates, since we did not necessarily investigate trends for the entirety of their history
  6. Agrawal, Govind P. 2016. “Optical Communication: Its History And Recent Progress”. Optics In Our Time, 177-199. Springer International Publishing. doi:10.1007/978-3-319-31903-2_8., https://link.springer.com/chapter/10.1007/978-3-319-31903-2_8
  7. Agrawal, Govind P. 2016. “Optical Communication: Its History And Recent Progress”. Optics In Our Time, 177-199. Springer International Publishing. doi:10.1007/978-3-319-31903-2_8., https://link.springer.com/chapter/10.1007/978-3-319-31903-2_8
  8. From Figure 33 in Division of STD Prevention, “Sexually Transmitted Disease Surveillance 2009,” November 2010, https://web.archive.org/web/20170120091355/https://www.cdc.gov/std/stats09/surv2009-Complete.pdf.
  9. See table 4D in Gregory L. Armstrong, Laura A. Conn, and Robert W. Pinner, “Trends in Infectious Disease Mortality in the United States During the 20th Century,” JAMA 281, no. 1 (January 6, 1999): 61–66, https://doi.org/10.1001/jama.281.1.61.
  10. e.g. within the area of ‘structure height’ we investigated ‘all time tallest buildings, measured by architectural height’ and also ‘tallest at the time freestanding structures, measured by pinnacle height’