Skip to content

Superintellignce

Contents

  1. Past developments and present capabilities Growth modes and big history Great expectations Seasons of hope and despair State of the art Opinions about the future of machine intelligence

  2. Paths to superintelligence Artificial intelligence Whole brain emulation Biological cognition Brain–computer interfaces Networks and organizations Summary

  3. Forms of superintelligence Speed superintelligence Collective superintelligence Quality superintelligence Direct and indirect reach Sources of advantage for digital intelligence

  4. The kinetics of an intelligence explosion Timing and speed of the takeoff Recalcitrance Non-machine intelligence paths Emulation and AI paths Optimization power and explosivity

  5. Decisive strategic advantage Will the frontrunner get a decisive strategic advantage? How large will the successful project be? Monitoring International collaboration From decisive strategic advantage to singleton

  6. Cognitive superpowers Functionalities and superpowers An AI takeover scenario Power over nature and agents

  7. The superintelligent will The relation between intelligence and motivation Instrumental convergence Self-preservation Goal-content integrity Cognitive enhancement Technological perfection Resource acquisition

  8. Is the default outcome doom? Existential catastrophe as the default outcome of an intelligence explosion? The treacherous turn Malignant failure modes Perverse instantiation Infrastructure profusion Mind crime

  9. The control problem Two agency problems Capability control methods Boxing methods Incentive methods Stunting Tripwires Motivation selection methods Direct specification Domesticity Indirect normativity Augmentation Synopsis

  10. Oracles, genies, sovereigns, tools Oracles Genies and sovereigns Tool-AIs Comparison

  11. Multipolar scenarios Of horses and men Wages and unemployment Capital and welfare The Malthusian principle in a historical perspective Population growth and investment Life in an algorithmic economy Voluntary slavery, casual death Would maximally efficient work be fun? Unconscious outsourcers? Evolution is not necessarily up Post-transition formation of a singleton? A second transition Superorganisms and scale economies Unification by treaty

  12. Acquiring values The value-loading problem Evolutionary selection Reinforcement learning Associative value accretion Motivational scaffolding Value learning Emulation modulation Institution design Synopsis

  13. Choosing the criteria for choosing The need for indirect normativity Coherent extrapolated volition Some explications Rationales for CEV Further remarks Morality models Do What I Mean Component list Goal content Decision theory Epistemology Ratification Getting close enough

  14. The strategic picture Science and technology strategy Differential technological development Preferred order of arrival Rates of change and cognitive enhancement Technology couplings Second-guessing Pathways and enablers Effects of hardware progress Should whole brain emulation research be promoted? The person-affecting perspective favors speed Collaboration The race dynamic and its perils On the benefits of collaboration Working together 15. Crunch time Philosophy with a deadline What is to be done? Seeking the strategic light Building good capacity Particular measures Will the best in human nature please stand up

1.Past developments and current capabilities

  • ai winter
  • complex tasks does not need general intelligence. e.g. chess
  • automation all-around us
  • can have huge impacts when things go wrong (flash crash 2010)

2. Paths to superintelligence

  • whole brain evolution is slow, takes times.
  • limited by either scanning or computing power
  • lower limit