Hardware and AI Timelines

2019 recent trends in Geekbench score per CPU price

From 2006 – 2020, Geekbench score per CPU price has grown by around 16% a year, for rates that would yield an order of magnitude over roughly 16 years.

Trends in DRAM price per gigabyte

The price of a gigabyte of DRAM has fallen by about a factor of ten every 5 years from 1957 to 2020. Since 2010, the price has fallen much more slowly, at a rate that

2019 recent trends in GPU price per FLOPS

Published 25 March, 2020 We estimate that in recent years, GPU prices have fallen at rates that would yield an order of magnitude over roughly: Details GPUs (graphics processing units) are specialized electronic circuits originally

Trend in compute used in training for headline AI results

Compute used in the largest AI training runs appears to have roughly doubled every 3.5 months between 2012 and 2018. Details According to Amodei and Hernandez, on the OpenAI Blog: …since 2012, the amount of compute

Transmitting fibers in the brain: Total length and distribution of lengths

The human brain’s approximately 86 billion neurons are probably connected by something like 850,000 km of axons and dendrites. Of this total, roughly 80% is short-range, local connections (averaging 680 microns in length), and approximately

Electrical efficiency of computing

Computer performance per watt has probably doubled every 1.5 years between 1945 and 2000. Since then the trend slowed. By 2015, performance per watt appeared to be doubling every 2.5 years. Details In 2011 Jon

Nordhaus hardware price performance dataset

This page contains the data from Appendix 2 of William Nordhaus’ The progress of computing in usable formats. Notes This data was collected from Appendix 2 of The progress of computing, using Tabula (a program for turning

2018 price of performance by Tensor Processing Units

Tensor Processing Units (TPUs) perform around 1 GFLOPS/$, when purchased as cloud computing. Details In February 2018, Google Cloud Platform blog says their TPUs can perform up to 180 TFLOPS, and currently cost $6.50/hour. This

2015 FLOPS prices

In April 2015, the lowest GFLOPS prices we could find were approximately $3/GFLOPS. However recent records of hardware performance from 2015 and earlier imply substantially lower prices, suggesting that something confusing has happened with these

Effect of marginal hardware on artificial general intelligence

We do not know how AGI will scale with marginal hardware. Several sources of evidence may shed light on this question. Details Background Suppose that at some point in the future, general artificial intelligence can be