Eye On A.I.

Welcome to episode #126 of Eye on AI with Craig Smith and Noam Chomsky. Are neural nets the key to understanding the human brain and language acquisition? In this conversation with renowned linguist and cognitive scientist Noam Chomsky, we delve into the limitations of large language models and the ongoing quest to uncover the mysteries of the human mind. Together, we explore the historical development of research in this field, from Minsky’s thesis to Jeff Hinton’s goals for understanding the brain. We also discuss the potential harms and benefits of large language models, comparing them to the internal combustion engine and its differences from a gazelle running. We tackle the difficult task of studying the neurophysiology of human cognition and the ethical implications of invasive experiments. As we consider language as a natural object, we discuss the works of notable figures such as Albert Einstein, Galileo, Leibniz, and Turing, and the similarities between language and biology. We even entertain the possibility of extraterrestrial language and communication. Join us on this thought-provoking journey as we explore the intricacies of language, the brain, and our place in the cosmos. (00:00) Preview

(00:43) Introduction

(01:54) Noam Chomsky’s neural net ideology & criticisms

(6:58) Jeff Hinton & Noam Chomsky’s: How the brain works

(10:05) Correlation between neural nets and the brain

(11:11) Noam Chomsky’s reaction to Chat-GPT & LLMs

(15:21) Exploring the mechanisms of the brain

(19:00) What do we learn from chatbots?

(22:30) What are impossible languages?

(26:45) Generative AI doesn’t show true intelligence?

(28:40) Is there a danger of AI becoming too intelligent?

(31:30) Can AI language models become sentient?

(36:40) Turing machine and neural nets experimentations

(42:40) Non-evasive procedures for understanding the brain

(45:54) Does Noam Chomsky still work on understanding the brain?

(49:33) Is Noam Chomsky excited about the future of neural nets?

(55:30) Albert Einstein and Galileo’s principles

(55:40) Is there an extraterrestrial language model?

Craig Smith Twitter: https://twitter.com/craigss

Eye on A.I. Twitter: https://twitter.com/EyeOn_AI

In this podcast episode, Ilya Sutskever, the co-founder and chief scientist at OpenAI, discusses his vision for the future of artificial intelligence (AI), including large language models like GPT-4. Sutskever starts by explaining the importance of AI research and how OpenAI is working to advance the field. He shares his views on the ethical considerations of AI development and the potential impact of AI on society. The conversation then moves on to large language models and their capabilities. Sutskever talks about the challenges of developing GPT-4 and the limitations of current models. He discusses the potential for large language models to generate a text that is indistinguishable from human writing and how this technology could be used in the future. Sutskever also shares his views on AI-aided democracy and how AI could help solve global problems such as climate change and poverty. He emphasises the importance of building AI systems that are transparent, ethical, and aligned with human values. Throughout the conversation, Sutskever provides insights into the current state of AI research, the challenges facing the field, and his vision for the future of AI. This podcast episode is a must-listen for anyone interested in the intersection of AI, language, and society. Timestamps: (00:04) Introduction of Craig Smith and Ilya Sutskever. (01:00) Sutskever's AI and consciousness interests. (02:30) Sutskever's start in machine learning with Hinton. (03:45) Realization about training large neural networks. (06:33) Convolutional neural network breakthroughs and imagenet. (08:36) Predicting the next thing for unsupervised learning. (10:24) Development of GPT-3 and scaling in deep learning. (11:42) Specific scaling in deep learning and potential discovery. (13:01) Small changes can have big impact. (13:46) Limits of large language models and lack of understanding. (14:32) Difficulty in discussing limits of language models. (15:13) Statistical regularities lead to better understanding of world. (16:33) Limitations of language models and hope for reinforcement learning. (17:52) Teaching neural nets through interaction with humans. (21:44) Multimodal understanding not necessary for language models. (25:28) Autoregressive transformers and high-dimensional distributions. (26:02) Autoregressive transformers work well on images. (27:09) Pixels represented like a string of text. (29:40) Large generative models learn compressed representations of real-world processes. (31:31) Human teachers needed to guide reinforcement learning process. (35:10) Opportunity to teach AI models more skills with less data. (39:57) Desirable to have democratic process for providing information. (41:15) Impossible to understand everything in complicated situations. Craig Smith Twitter: https://twitter.com/craigssEye on A.I. Twitter: https://twitter.com/EyeOn_AI

Yoshua Bengio, the legendary AI expert, will join us for Episode 128 of Eye on AI podcast. In this episode, we delve into the unnerving question: Could the rise of a superhuman AI signal the downfall of humanity as we know it? Join us as we embark on an exploration of the existential threat posed by superhuman AI, leaving no stone unturned. We dissect the Future of Life Institute’s role in overseeing large language model development. As well as the sobering warnings issued by the Centre for AI Safety regarding artificial general intelligence. The stakes have never been higher, and we uncover the pressing need for action. Prepare to confront the disconcerting notion of society’s gradual disempowerment and an ever-increasing dependency on AI. We shed light on the challenges of extricating ourselves from this intricate web, where pulling the plug on AI seems almost impossible. Brace yourself for a thought-provoking discussion on the potential psychological effects of realizing that our relentless pursuit of AI advancement may inadvertently jeopardize humanity itself. In this episode, we dare to imagine a future where deep learning amplifies system-2 capabilities, forcing us to develop countermeasures and regulations to mitigate associated risks. We grapple with the possibility of leveraging AI to combat climate change, while treading carefully to prevent catastrophic outcomes. But that’s not all. We confront the notion of AI systems acting autonomously, highlighting the critical importance of stringent regulation surrounding their access and usage.

(00:00) Preview

(00:42) Introduction

(03:30) Yoshua Bengio's essay on AI extinction

(09:45) Use cases for dangerous uses of AI

(12:00) Why are AI risks only happening now?

(17:50) Extinction threat and fear with AI & climate change

(21:10) Super intelligence and the concerns for humanity

(15:02) Yoshua Bengio research in AI safety

(29:50) Are corporations a form of artificial intelligence?

(31:15) Extinction scenarios by Yoshua Bengio

(37:00) AI agency and AI regulation

(40:15) Who controls AI for the general public?

(45:11) The AI debate in the world

Craig Smith Twitter: https://twitter.com/craigss Eye on A.I. Twitter: https://twitter.com/EyeOn_AI

This episode is sponsored by ISS, a leading global provider of video intelligence and data awareness solutions. ISS offers a robust portfolio of AI-powered, high-trust video analytics for streamlining security, safety and business operations within a wide range of vertical markets. So, what do you want to know about your environment?

To learn more about our video intelligence solutions, visit https://issivs.com/

On episode #155 of Eye on AI, Craig Smith sits down with Karen Hao, who is currently a contributing writer for The Atlantic with an impressive background as a foreign correspondent for The Wall Street Journal in Hong Kong and as a senior artificial intelligence editor at MIT Technology Review.

Known for her incisive coverage of AI, including its ethical and societal implications, Hao brings a wealth of knowledge from her experiences in journalism and data science.

In this episode, Karen delves into the recent controversies surrounding OpenAI, shedding light on the power struggles, ethical dilemmas, and corporate alliances shaping the future of artificial intelligence. Her unique perspective, gained from her experience as a foreign correspondent and senior AI editor, offers a deep understanding of the complexities that exist in the AI world.

We explore the intricate narrative constructed by OpenAI, its relationship with giants like Microsoft, and the broader implications of these partnerships on AI development and ethics. Karen's critical analysis provides an insightful look into the often opaque world of AI and its global impact.

If you find this discussion as enlightening as we did, please consider leaving a 5-star rating on Spotify and a review on Apple Podcasts.

Stay Updated:

Craig Smith Twitter: https://twitter.com/craigss

Eye on A.I. Twitter: https://twitter.com/EyeOn_AI

(00:00) Preview and Introduction

(02:28) Karen Hao's Background and Entry into Tech Journalism

(09:14) OpenAI's GPT-2 Controversy and Company Evolution

(11:35) Nonprofit and For-Profit Structure of OpenAI

(15:26) OpenAI Board Dynamics and Power Struggles

(18:07) Transparency and Open Source in AI Development

(21:27) Future of OpenAI and Tech Industry Speculations

(26:22) Microsoft's Investment and Partnership with OpenAI

(31:28) Sam Altman's Potential Chip Company Endeavor

(33:20) AGI Speculations and Existential Risks

(34:38) AGI Definitions and Real-World AI Limitations

This episode is sponsored by Oracle. AI is revolutionizing industries, but needs power without breaking the bank. Enter Oracle Cloud Infrastructure (OCI): the one-stop platform for all your AI needs, with 4-8x the bandwidth of other clouds. Train AI models faster and at half the cost. Be ahead like Uber and Cohere.

If you want to do more and spend less like Uber, 8x8, and Databricks Mosaic - take a free test drive of OCI at https://oracle.com/eyeonai

Welcome to episode 150 of the ‘Eye on AI’ podcast. In this episode, host Craig Smith sits down with Yann LeCun, a Turing Award winner who has been instrumental in advancing convolutional neural networks and whose work spans machine learning, computer vision, and more.

Tune is as Craig and Yann explore the intricacies of AI, world models, and the challenges of continuous learning.

In this episode, Yann delves deep into the concept of a "world model" - systems that can predict the world's future states, allowing agents to make informed decisions. The discussion transitions to the challenges of training these models, particularly when dealing with diverse data like text and images. We then discuss the computational demands of modern AI models, with Yann highlighting the nuances between generative models for videos and language.

He also touches upon the idea of the "Embodied Turing Tests" and how augmented language models can bridge the gap between human-like behavior and computational efficiency.The spotlight then shifts to pressing concerns surrounding the open-source nature of AI models, with Yann articulating the legal ramifications and the future of open-source AI. Drawing from global perspectives, including China's stance on open-source, Yann underscores the imperative for a collaborative approach in the AI space, ensuring it's reflective of diverse global needs.

Craig Smith Twitter: https://twitter.com/craigss

Eye on A.I. Twitter: https://twitter.com/EyeOn_AI

(00:00) Preview, Oracle and Introduction

(02:42) Decoding The World Model and Gaia 1

(07:43) Energy and Computational Demands of AI

(08:06) Video vs. Text Processing & True AI Capabilities

(11:17) Embodied Turing Test & Augmented LLMs

(15:38) Is AI a Threat To Society?

(25:04) Where is AI Development Headed?

(31:06) Interplay of Neuroscience and AI**

(33:33) Yann's Vision, JEPA, and Learning Challenges

(39:05) Yann's Career, AI Progress, and Challenges

(44:47) The Open Source Debate in AI

(55:30) Oracle Cloud Infrastructure