Promising research projects

This is an incomplete list of concrete projects that we think are tractable and important. We may do any of them ourselves, but many also seem feasible to work on independently. Those we consider especially well suited to this are marked Ψ. More potential projects are listed here.


Determine the relationship between brain size and intelligence within the human population (in progress)

Critically review the existing literature, to learn how much increasing brain size increases intelligence, in humans. Investigate whether apparent correlation is causal. Review known details of this relationship, e.g. does size matter more for some parts of the brain than others? Is number of neurons the crucial factor?

The relationship between brain size and intelligence is important because if modest increases in human ‘hardware’ account for large increases in success at some human endeavors (e.g. contribution to physics progress), this suggests that increases in computing hardware in artificial intelligence may also produce radical increases in capability.

Run a new version of our 2016 survey (under consideration)

Use the questions and methods from last time, with minimal modification. Send them to the 2017 participants at the same conferences as last time.

This would allow us to assess how opinions in machine learning have changed over the past two years, as well as providing updated answers to many questions. It may also get huge amounts of media attention, based on past experience. We are not sure about the value of these benefits, but the project could be quite cheap, given that we already have the survey.

Check for discontinuities in historical technological trends Ψ

Look for technologies that may have caused discontinuous progress on any metric. Find data for that metric over the relevant time, and measure the size of any discontinuity in terms of how many years of progress at usual rates happened at once. We have a list of technologies which others purport were discontinuous, to check. A particularly important one is recent Go AI as a discontinuity in Elo rating achievable, adjusted for hardware.

This is an input to our ongoing investigation into how frequently, and when, technological trends undergo discontinuous progress. This should inform our guesses about how likely AI development is to see discontinuous progress, both by providing a base rate, and telling us whether AI technologies seem especially susceptible. We take discontinuous progress in AI to be related to risk of fast takeoff.

Quantitatively model an intelligence explosion Ψ

An intelligence explosion (or ‘recursive self-improvement’) consists of a feedback loop where researcher efforts produce scientific progress, which produces improved AI performance, which produces more efficient researcher efforts. This forms a loop, because the researchers involved are artificial themselves.

Though this loop does not yet exist, relatively close analogues to all of the parts of it already occur: for instance, researcher efforts do lead to scientific progress; scientific progress does lead to better AI; better AI does lead to more capacity at the kinds of tasks that AI can do.

Collect empirical measurements of proxies like these, for different parts of the hypothesized loop (each part of this could be a stand-alone project). Model the speed of the resulting loop if they were put together, under different background conditions.

This would give us a very rough estimate of the contribution of intelligence explosion dynamics to the speed of intelligence growth in a transition to an AI-based economy. Also, a more detailed model may inform our understanding of available strategies to improve outcomes.

Review the literature on forecasting (in progress) Ψ

Summarise what is known about procedures that produce good forecasts, and measures that are relatively easier to forecast. This may involve reading secondary sources, or collecting past forecasts and investigating what made some of them successful.

This would be an input to improving our own forecasting practices, and to knowing which other forecasting efforts to trust.

Review considerations regarding the chance of local, fast takeoff Ψ

We have a list of considerations here. If you find local, fast take-off likely, check if the considerations that lead you to this view are represented. Alternately, interview someone else with a strong position about the considerations they find important. If there are any arguments or counterarguments that you think are missing, write a short page explaining the case.

Collecting arguments on this topic is helpful because opinion among well-informed thinkers on the topic seems to diverge from what would be expected given the arguments that we know about. This suggests that we are missing some important considerations, that we would need to well assess the chance of local, fast takeoff.

Interview AI researchers on topics of interest Ψ

Find an AI researcher with views on matters of interest (e.g. AI risk, timelines, the relevance of neuroscience to AI progress) and interview them. Write a summary, or transcript (with their permission). Some examples here, here, here. (If you do not expect to run an interview well enough to make a good impression on the interviewee, consider practicing elsewhere first, so as not to discourage interacting with similar researchers in the future.)

Talking to AI researchers about their views can be informative about the nature of AI research (e.g. What problems are people trying to solve? How much does it seem like hardware matters?), and provide an empirically informed take on questions and considerations of interest to us (e.g. Current techniques seem really far from general). They also tell us about state of opinion within the AI research community, which may be relevant in itself.

Collect considerations regarding whether to expect current AI techniques to get us to human-level AGI Ψ

Search for writing or people to talk to on this topic, and review and summarize the important considerations.

This would allow further investigation of the validity and strength of the considerations, hopefully leading to an informed view on the topic. This might influence views on the expected timing and nature of an AI transition, as well as the relevance of different safety efforts.

Review what is known about the relative intelligence of humans, chimps, and other animals (in progress)

Review efforts to measure animal and human intelligence on a single scale, and efforts to quantify narrower cognitive skills across a range of animals.

Humans are radically more successful than other animals, in some sense. This is taken as reason to expect that small modifications to brain design (for instance whatever evolution did between the similar brains of chimps and humans) can produce outsized gains in some form of mental performance, and thus that AI researchers may see similar astonishing progress near human-level AI.

However without defining or quantifying the mental skills of any relevant animals, it is unclear a) whether individual intelligence in particular accounts for humans’ success (rather than e.g. ability to accrue culture and technology), b) whether the gap in capabilities between chimps and humans is larger than expected (maybe chimps are also astonishingly smarter than smaller mammals), or c) whether the success stems from something that evolution was ‘intentionally’ progressing on. These things are all relevant to the strength of an argument for AI ‘fast take-off’ based on human success over chimps (see here).

Review explanations for humans’ radical success over apes

Investigate what is known about the likely causes of human success, relative to that of other similar animals. In particular, we are interested in how likely improvement in individual cognitive ability is to account for this (as opposed to say communication and group memory abilities).

This would help resolve the same issues described in the last section (‘Review what is known about the relative intelligence of humans, chimps, and other animals’).

Collect data on time to cross the human range on intellectual skills where machines have surpassed us (in progress) Ψ

For intellectual skills where machines have surpassed humans, find out how long it took to go from the worst performance to average human skill, and from average human skill to superhuman skill.

This would contribute to this project.

Measure the importance of hardware progress in a specific narrow AI trajectory Ψ

Take an area of AI progress, and assess how much of annual improvement can be attributed to hardware improvements vs. software improvements, or what the more detailed relationship between the two is.

Understanding the overall importance of hardware progress and software progress (and other factors) in overall AI progress lets us know to what extent our future expectations should be a function of expected hardware developments, versus software developments. This both alters what our timelines look like (e.g. see here), and tells us what we should be researching to better understand AI timelines.

Review and summarise considerations around AI agency (in progress)

Collect, summarize and evaluate arguments for expecting future AI intelligence to be concentrated in autonomous ‘agents’ rather than ‘tools’.

Arguments for AI risk often depend on AI agency, and we might generally expect the world to look quite different with a proliferation of inhuman agents, relative to smart tools. There is disagreement about the likelihood or inevitability of ubiquitous agents. Documenting this may help us resolve the issue.

We welcome suggestions for this page or anything on the site via our feedback box, though will not address all of them.


  1. Research on consciousness is important such as the relationship between qualia and intelligence

  2. Can I get some advice? I apologize for my English, this is not my native language 🙂

    it is probably worth starting to investigate the performance of algorithms from here- There is also the opinion of David May.
    from some papers, we can conclude that when the hardware improves, the performance increases linearly. For example:

    What about broader interviews? It would be interesting to know the opinion of experts, who were interviewed by Alexander Kruel. An extensive interview of the most prominent experts would be more interesting than the opinion of hundreds of nameless experts, im think.

    It would also be interesting to know the opinion of the spoilers on neurology about the complexity of the human brain. it seems to me that the estimates on which the human brain uses all 100-1000 trillion synapses 10-100 times per second are completely untrue.

    In any case, many thanks to you for your work 🙂

3 Trackbacks / Pingbacks

  1. May 2018 Newsletter « Chillycon
  2. May 2018 Newsletter | Tech News | Stay Connected with Tech
  3. The most important questions and problems | Pablo's miscellany

Comments are closed.