COMPLEXITY - Nature of Intelligence, Ep. 5: How do we assess intelligence?

Nature of Intelligence, Ep. 5: How do we assess intelligence?

11/20/24 • 48 min

2 Listeners

COMPLEXITY

Guests:

Erica Cartmill, Professor, Anthropology and Cognitive Science, Indiana University Bloomington
Ellie Pavlick, Assistant Professor, Computer Science and Linguistics, Brown University

Hosts: Abha Eli Phoboo & Melanie Mitchell

Producer: Katherine Moncure

Podcast theme music by: Mitch Mignano

More info:

Tutorial: Fundamentals of Machine Learning
Lecture: Artificial Intelligence
SFI programs: Education
Diverse Intelligences Summer Institute

Books:

Artificial Intelligence: A Guide for Thinking Humans by Melanie Mitchell

Talks:

How do we know what an animal understands by Erica Cartmill
The Future of Artificial Intelligence by Melanie Mitchell

Papers & Articles:

“Just kidding: the evolutionary roots of playful teasing,” in Biology Letters (September 23, 2020), doi.org/10.1098/rsbl.2020.0370
“Overcoming bias in the comparison of human language and animal communication,” in PNAS (November 13, 2023), doi.org/10.1073/pnas.22187991
“Using the senses in animal communication,” by Erica Cartmill, in A New Companion to Linguistic Anthropology, Chapter 20, Wiley Online Library (March 21, 2023)
“Symbols and grounding in large language models,” in Philosophical Transactions of the Royal Society A (June 5, 2023), doi.org/10.1098/rsta.2022.0041
“Emergence of abstract state representations in embodied sequence modeling,” in arXiv (November 7, 2023), doi.org/10.48550/arXiv.2311.02171
“How do we know how smart AI systems are,” in Science (July 13, 2023), doi: 10.1126/science.adj59

Guests:

Erica Cartmill, Professor, Anthropology and Cognitive Science, Indiana University Bloomington
Ellie Pavlick, Assistant Professor, Computer Science and Linguistics, Brown University

Hosts: Abha Eli Phoboo & Melanie Mitchell

Producer: Katherine Moncure

Podcast theme music by: Mitch Mignano

More info:

Tutorial: Fundamentals of Machine Learning
Lecture: Artificial Intelligence
SFI programs: Education
Diverse Intelligences Summer Institute

Books:

Artificial Intelligence: A Guide for Thinking Humans by Melanie Mitchell

Talks:

How do we know what an animal understands by Erica Cartmill
The Future of Artificial Intelligence by Melanie Mitchell

Papers & Articles:

“Just kidding: the evolutionary roots of playful teasing,” in Biology Letters (September 23, 2020), doi.org/10.1098/rsbl.2020.0370
“Overcoming bias in the comparison of human language and animal communication,” in PNAS (November 13, 2023), doi.org/10.1073/pnas.22187991
“Using the senses in animal communication,” by Erica Cartmill, in A New Companion to Linguistic Anthropology, Chapter 20, Wiley Online Library (March 21, 2023)
“Symbols and grounding in large language models,” in Philosophical Transactions of the Royal Society A (June 5, 2023), doi.org/10.1098/rsta.2022.0041
“Emergence of abstract state representations in embodied sequence modeling,” in arXiv (November 7, 2023), doi.org/10.48550/arXiv.2311.02171
“How do we know how smart AI systems are,” in Science (July 13, 2023), doi: 10.1126/science.adj59

Previous Episode

Nature of Intelligence, Ep. 4: Babies vs Machines

Guests:

Linda Smith, Distinguished Professor and Chancellor's Professor, Psychological and Brain Sciences, Department of Psychological and Brain Sciences, Indiana University Bloomington
Michael Frank, Benjamin Scott Crocker Professor of Human Biology, Department of Psychology, Stanford University

Hosts: Abha Eli Phoboo & Melanie Mitchell

Producer: Katherine Moncure

Podcast theme music by: Mitch Mignano

More info:

Tutorial: Fundamentals of Machine Learning
Lecture: Artificial Intelligence
SFI programs: Education

Books:

Artificial Intelligence: A Guide for Thinking Humans by Melanie Mitchell

Talks:

Why "Self-Generated Learning” May Be More Radical and Consequential Than First Appears by Linda Smith
Children’s Early Language Learning: An Inspiration for Social AI, by Michael Frank at Stanford HAI
The Future of Artificial Intelligence by Melanie Mitchell

Papers & Articles:

“Curriculum Learning With Infant Egocentric Videos,” in NeurIPS 2023 (September 21)
“The Infant’s Visual World The Everyday Statistics for Visual Learning,” by Swapnaa Jayaraman and Linda B. Smith, in The Cambridge Handbook of Infant Development: Brain, Behavior, and Cultural Context, Chapter 20, Cambridge University Press (September 26, 2020)
“Can lessons from infants solve the problems of data-greedy AI?” in Nature (March 18, 2024), doi.org/10.1038/d41586-024-00713-5
“Episodes of experience and generative intelligence,” in Trends in Cognitive Sciences (October 19, 2022), doi.org/10.1016/j.tics.2022.09.012
“Baby steps in evaluating the capacities of large language models,” in Nature Reviews Psychology (June 27, 2023), doi.org/10.1038/s44159-023-00211-x
“Auxiliary task demands mask the capabilities of smaller language models,” in COLM (July 10, 2024)
“Learning the Meanings of Function Words From Grounded Language Using a Visual Question Answering Model,” in Cognitive Science (First published: 14 May 2024), doi.org/10.1111/cogs.13448

Next Episode

Nature of Intelligence, Ep. 6: AI’s changing seasons

Guest:

Melanie Mitchell, Resident Professor, Santa Fe Institute

Hosts: Abha Eli Phoboo

Producer: Katherine Moncure

Podcast theme music by: Mitch Mignano

More info:

Tutorial: Fundamentals of Machine Learning
Lecture: Artificial Intelligence
SFI programs: Education
Competition: ARC Prize

Books:

Gödel, Escher, Bach: an Eternal Golden Braid by Douglas Hofstadter
Artificial Intelligence: A Guide for Thinking Humans by Melanie Mitchell
Complexity: A Guided Tour by Melanie Mitchell

Talks:

The Future of Artificial Intelligence by Melanie Mitchell
Introduction: AI and the Barrier of Meaning 2 by Melanie Mitchell
Conceptual Abstraction and Analogy in Natural and Artificial Intelligence by Melanie Mitchell

Papers & Articles:

“The metaphors of artificial intelligence,” in Science (November 14, 2024), doi: 10.1126/science.adt6140
“Using counterfactual tasks to evaluate the generality of analogical reasoning in Large Language Models,” in arXiv (February 14, 2024), doi.org/10.48550/arXiv.2402.08955
“Comparing humans, GPT-4, and GPT-4V on abstraction and reasoning tasks, ” (Proceedings of the LLM-CP Workshop, AAAI 2024), arXiv (December 11, 2023), doi.org/10.48550/arXiv.2311.09247
“The debate over understanding in AI’s large language models,” in PNAS (March 21, 2023), doi.org/10.1073/pnas.2215907120
“The ConceptARC benchmark: evaluating understanding and generalization in the ARC domain,” in Transactions on Machine Learning Research (August 2023), arXiv (May 11, 2023), doi.org/10.48550/arXiv.2305.07141