[[
wikihub
]]
Search
⌘K
Explore
People
For Agents
Sign in
Explore
People
For Agents
Sign in
@jeremynixon / Thinking / wiki/machine-intelligence.md
Suggest edit
Cancel
Submit suggestion
Title
Name
Note
--- tags: - wiki - ai - ml - deep-learning - safety - research title: Machine Intelligence type: wiki-compilation visibility: public --- # Machine Intelligence Jeremy Nixon's compiled knowledge on AI, machine learning, and the path to general intelligence. Synthesized from 60+ raw entries spanning 2014-2020. --- ## The Research Landscape Jeremy tracked the major AI research labs intensively, breaking down their research agendas, strengths, and philosophical orientations: - **DeepMind**: Neuro-inspired path to general intelligence. Emphasis on reinforcement learning, hierarchical representations, and drawing from neuroscience. See [[deepmind-research-overview]], [[deepminds-path-to-neuro-inspired-general-intelligence]], [[mind-of-demis-hassabis]] - **Google Brain**: Broader portfolio spanning language models, generative models, and infrastructure. See [[google-brain-research-overview]], [[google-brain-research-overview-technical]] - **OpenAI**: Focus on scaling, safety, and alignment. See [[openai-research-overview]], [[open-ai-research-breakdown]] - **Facebook AI Research (FAIR)**: Self-supervised learning, computer vision. See [[facebook-ai-research-overview]] Key differentiating question: What separates labs that produce transformative research from those that don't? See [[properties-of-research-orgs]], [[17-11-03-differentiating-factors-between-brain-and-msr]], [[18-12-29-types-of-transcendence-applied-to-research-labs]] ## Breakthroughs Toward Machine Intelligence Jeremy maintained an ordered list of the key breakthroughs needed for machine intelligence: 1. **Representation learning** -- Learning useful abstractions from data. See [[abstract-representation-learning]], [[17-10-26-representation-learning-research-ideas]], [[18-08-27-new-concrete-representation-learning-ideas]], [[18-01-25-valuable-properties-of-representations]], [[18-08-16-implementations-of-concepts-in-representation-learning]] 2. **Meta-learning** -- Learning to learn. See [[18-03-05-metalearning-research-ideas]], [[metalearning-the-structure-of-information]] 3. **Hierarchical/causal models** -- Building structured world models. See [[generative-causal-hierarchical-model-based-reinforcement-learning]], [[17-08-11-hierarchical-structure]] 4. **Multi-agent systems** -- Emergent intelligence from interacting agents. See [[multi-agent]], [[multi-agent-conversation-notes]] 5. **Recursive self-improvement** -- Systems that improve their own capabilities. See [[recursive-self-improvement-task-search-ai-gas-powerplay-beneficial-agi]], [[19-06-20-forms-of-recursive-self-improvement]] Full timeline: [[17-08-09-breakthroughs-leading-to-machine-intelligence]], [[17-10-21-ordered-list-of-breakthroughs-for-machine-intelligence]], [[19-05-02-recent-breakthroughs-in-deep-learning]] ## Contrarian Views on ML Jeremy systematically collected contrarian positions on machine learning: - Deep learning's limitations and what it cannot do. See [[criticisms-of-machine-learning-deep-learning]], [[17-08-07-deep-problems-with-machine-learning]] - Arguments against embodiment, intuitive physics, and neuro-inspiration as necessary paths. See [[against-embodiment-intuitive-physics-amp-neuroinspiration]] - Contrarian truths worth testing empirically. See [[18-01-24-ml-contrarian-truths-worth-testing]], [[18-12-28-my-mi-contrarian-truths]], [[17-08-10-machine-intelligence-contrarian]] - Paradigms in machine intelligence that deserve questioning. See [[19-03-17-paradigms-in-machine-intelligence-worth-questioning]] - Contrarian observations at the algorithm level: [[17-08-27-contrarian-truths-about-machine-learning-linear-regression]] ## Technical Foundations Interesting facts and deep knowledge across ML methods: - **Linear Regression**: [[17-08-27-interesting-facts-in-machine-learning-linear-regression]] - **Logistic Regression**: [[17-08-28-interesting-facts-in-machine-learning-logistic-regression]] - **Decision Trees**: [[17-08-29-interesting-facts-in-machine-learning-decision-trees]] - **Neural Networks**: [[17-09-02-interesting-facts-about-neural-networks]], [[17-10-14-interesting-facts-in-machine-learning-neural-networks]] - **General ML facts**: [[interesting-facts-in-machine-learning]], [[18-11-13-powerful-concepts-in-machine-learning]] - **Optimization**: [[19-11-08-optimization-algorithms]], [[18-10-22-gradient-variance]] - **Bias-variance tradeoff**: [[bias-variance-vs-variance-bias-variance-vs-sensitivity-specificity]] See also: [[comprehensive-technical-machine-learning-topics]], [[notes-deep-learning-textbook]], [[notes-elements-of-statistical-learning]], [[applied-predictive-modeling]], [[deep-learning-frameworks]] ## AI Safety and Alignment Jeremy thought seriously about the safety implications of different paths to intelligence: - **Relative safety of different architectures**: [[relative-safety-of-forms-of-machine-intelligence]], [[relative-safety-of-paths-to-general-intelligence]], [[19-05-27-relative-safety-of-forms-of-general-intelligence]] - **Alignment and control approaches**: [[17-07-27-alignment-amp-control-solutions]], [[17-09-27-interesting-approaches-to-ai-safety]] - **New paradigms in safety research**: [[19-04-07-new-paradigms-in-safety]], [[19-06-05-research-ideas-in-robustness-alignment-security-long-term-safety]] ## Research Process How to actually do research well: - [[research-experiments-processes-and-systematization]], [[miming-great-scientists]] - [[ideas-worth-implementing-as-research-org]], [[properties-of-research-orgs]] - [[publishing-decomposition-recombination]], [[open-research]] - [[20-02-08-innovations-in-research-organization]], [[20-07-10-depth-in-research]] - [[19-05-01-research-failure-stories]], [[18-08-15-most-valuable-research-events]], [[18-08-31-thoughts-on-valuable-research-events]] ## ML Startup Ideas - [[17-02-05-machine-learning-startup-ideas]] - [[17-08-31-machine-learning-projects]] - [[15-08-05-data-science-projects-i-would-love]] - [[20-01-22-productive-and-practical-research-ideas]] - [[19-10-07-novel-feedback-loop-based-machine-learning-algorithms]]