054. Jay McClelland: Networks That Learn

05/18/23 • 80 min

TODAY'S GUEST

Jay McClelland is a Computational Cognitive Neuroscientist and one of the founding fathers of the field of neural networks and deep learning in the 1980s, which led directly to today's explosion in AI and machine learning algorithms that are transforming our lives. He is the Lucie Stern Professor at Stanford University, where he was formerly the chair of the psychology department, and is currently a Consulting Research Scientist at DeepMind, perhaps the leader in machine learning technologies today.

Jay is best known for his work on statistical learning and parallel distributed processing, applying connectionist models (or neural networks) to explain cognitive phenomena such as spoken word recognition and visual word recognition. Today, he works on integrating language, memory, and visuospatial cognition in an integrated understanding system to capture human intelligence and enhance artificial intelligence, exploring how education and human-invented tools of thought can enhance human and machine intelligence.

EPISODE SUMMARY

In this conversation we talk about:

Lessons from his youth, where he moved around the world as a child and interacted with different religions and backgrounds, which helped him understand that we are shaped by our contexts and experiences.
His entry into cognitive psychology, and going beyond the laws of behavior into: Why do people behave the way they do?
Building neural networks to model cognition.
His world-changing PDP paper (Parallel Distributed Processing: Explorations in the Microstructure of Cognition), a paper that was published in 1986 and transformed this whole field, and directly led to more and more people embracing the connectionist model and neural networks.
The fact and meaning of bi-directionality in neural networks. What does it mean that information can flow both ways in the same network structure?
Generative models, and in this context, OpenAI's DALL-E 2 algorithm, which can create amazing illustrations and artworks — and should we credit generative or creative algorithms with artistry and give them credit for their art?
Consciousness — does it extend beyond humans and is it something that we may be able to find someday in algorithms?

Talking to Jay really reminded me of the best in mankind, that through curiosity, asking interesting questions, and constructing thought models and experiments, we can unlock such a subtle and fundamental thing like cognition and the connectionist model, which then unlocks all of this power for society at large. We now have this responsibility to reign in the worst of mankind in how we exploit, curate, and share in the benefits of this incredible power. This will be a running topic for us, AI in the future. We explore the power of design and human-centered thinking to create a better future for everyone.

This conversation with Jay is one of many weekly conversations we already have lined up for you with leading authors, thinkers, designers, makers, scientists, and social entrepreneurs who are working to change our world for the better. So follow this podcast on your favorite podcast app, or head over to remakepod.org to subscribe.

And now, let's jump right in with Jay McClelland.

TIMESTAMP CHAPTERS

[7:28] Life in the Present

[9:08] Early Childhood Perspectives

[12:33] A Path to Psychology

[22:16] Modeling Cognition

[27:37] Neural Networks

[35:16] The Significance of Bi-Directionality

[40:21] Bistable Perception

[43:55] The Truth of Mathematics

[49:24] An Emergentist

[55:17] Technology and AI

[1:01:17] An Accumulation of Experience

[1:07:20] On Consciousness

[1:15:47] A Short Sermon

EPISODE LINKS

Jay's Links
- 🏫 Stanford University
- 🏫 Columbia University
- 🏫 University of Pennsylvania
- 🤖 DeepMind
- 📘 Parallel Distributed Processing, Vol. 1: Foundations by James L. Mcclelland and David E. Rumelhart
- 📘

TODAY'S GUEST

EPISODE SUMMARY

In this conversation we talk about:

Lessons from his youth, where he moved around the world as a child and interacted with different religions and backgrounds, which helped him understand that we are shaped by our contexts and experiences.
His entry into cognitive psychology, and going beyond the laws of behavior into: Why do people behave the way they do?
Building neural networks to model cognition.
His world-changing PDP paper (Parallel Distributed Processing: Explorations in the Microstructure of Cognition), a paper that was published in 1986 and transformed this whole field, and directly led to more and more people embracing the connectionist model and neural networks.
The fact and meaning of bi-directionality in neural networks. What does it mean that information can flow both ways in the same network structure?
Generative models, and in this context, OpenAI's DALL-E 2 algorithm, which can create amazing illustrations and artworks — and should we credit generative or creative algorithms with artistry and give them credit for their art?
Consciousness — does it extend beyond humans and is it something that we may be able to find someday in algorithms?

And now, let's jump right in with Jay McClelland.

TIMESTAMP CHAPTERS

[7:28] Life in the Present

[9:08] Early Childhood Perspectives

[12:33] A Path to Psychology

[22:16] Modeling Cognition

[27:37] Neural Networks

[35:16] The Significance of Bi-Directionality

[40:21] Bistable Perception

[43:55] The Truth of Mathematics

[49:24] An Emergentist

[55:17] Technology and AI

[1:01:17] An Accumulation of Experience

[1:07:20] On Consciousness

[1:15:47] A Short Sermon

EPISODE LINKS

Jay's Links
- 🏫 Stanford University
- 🏫 Columbia University
- 🏫 University of Pennsylvania
- 🤖 DeepMind
- 📘 Parallel Distributed Processing, Vol. 1: Foundations by James L. Mcclelland and David E. Rumelhart
- 📘

Previous Episode

073. Eli Green: Gender Beyond the Binary

TODAY'S GUEST

Today we're talking about transgender issues.

Specifically, what should the rest of us know about transgender people? This is a topic that I admit I know very little about, but one that feels important at the very least, if one wants to avoid causing unnecessary pain. Language itself seems to be changing when it comes to gender, and while some resistance is natural and no generally agreed upon set of rules has been widely accepted, it's important to understand why these changes are happening and what's behind them.

I can't imagine someone more perfect to talk to about this subject than Dr. Eli Green. Eli is the founder and CEO of the Transgender Training Institute. They are an award winning educator and author with over 20 years facilitating transgender related education and supporting other gender educators. Dr. Green has helped thousands of people have a better understanding of what it means to be transgender and nonbinary and how to support and affirm the transgender and nonbinary people in their lives, workplaces, and communities. Eli brings kindness, patience, and authentic connection to the topic, making the gender conversation one that seems inviting, a place you'd like to be rather than a place to avoid.

We spoke in mid January 2022, and it was really the first time I felt like I had a grasp of what we're talking about. When we talk about gender and transgender rights and challenges.

EPISODE SUMMARY

In this conversation we talk about:

Eli's childhood and early realization on how people are treated differently based on gender.
How Eli recognized they were nonbinary at a young age, but not having the language for it.
Eli's process of coming out as nonbinary is transgender transition, facing discrimination as a nonbinary person, including transphobia and barriers to accessing resources.
The differences between gender and sex and how they are not the same.
The importance of kindness and affirmation for the LGBTQ plus community.
How Cisgendered people can support and advocate for trans rights.
Eli's work with the Transgender Training Institute, which provides education and training on trans issues to businesses, organizations, and government agencies changing hearts and minds, and the need for systemic change to support nonbinary people.
How does religion enter into the picture?
What do most people still need to understand about transgender and non-binary individuals?
What is the significance of the dynamism of language in this area?
Understanding Gender Prejudice

What stayed with me most of all is the ever present need for kindness, the need to make an effort to see things from a different person's perspective. This applies equally to cisgendered folks learning about gender dysphoria, as it does to activists understanding that ignorance does not equal malice. In both cases, the smallest gesture of kindness can open the door to a whole new way of relating and moving forward, in which specific details can be discussed from a place of trust.

We already have lined up for you with thinkers, designers, makers, authors, entrepreneurs, and activists who are working to change our world for the better. We have some amazing episodes lined up for you, answering questions like:

Why is prototyping essential to making truly new things?

What's the value of knowing what you're about and crafting a personal manifesto? How can we find the freedom to think in an increasingly connected world?

So follow this podcast on your favorite podcast app, or head over to RemakePod.org to subscribe. And if you're a subscriber already and enjoy our show, you can go to RemakePod.org/support and join our supporter community.

And now, without further ado, let's jump right in with Dr. Eli Green.

TIMESTAMP CHAPTERS

[5:31] Life in the Present

[6:53] Early Childhood Realizations

[15:56] Coming Out as a Trans

[20:42] Significance, Evolution and Expansion of Labels and Languages

[28:30] Learning Human Sexuality in Academia

[32:07] From Academia to Establishing the Transgender Training Institute

[35:50] The TTI Teaching Pedagogy

[38:37] Success Stories

[42:29] LGBTQ+ Present Issues and Obstacles

[47:33] Alliance vs Friendship

[51:19] Elements of being a Good Ally

[53:04] Gender Pronouns

[57:15] Importance of Being Kind

[01:02:51] Company Design and Structure

[01:09:33] Short Sermon

EPISODE LINKS

Eli's Links
- Dr. Eli Green

Next Episode

053. Irene Au: Bridging Design and Technology

TODAY'S GUEST

Irene Au is Design Partner at Khosla Ventures, where she works with early-, mid-, and late-stage startup CEOs. She is dedicated to raising the strategic value of design and user research within software companies through better methods, practices, processes, leadership, talent, and quality. Irene has unprecedented experience elevating the strategic importance of design within technology companies, having built and led the entire User Experience and Design teams at Google, Yahoo!, and Udacity. She began her career as an interaction designer at Netscape Communications, where she worked on the design of the internet’s first commercial web browser.

Irene also teaches yoga at Avalon Yoga Center in Palo Alto where she is among the teacher training program faculty and is a frequent author and speaker on mindfulness practices, design, and creativity. An adjunct lecturer at Stanford University, she teaches product design in the mechanical engineering department. Irene also serves as a trustee for the Smithsonian Cooper Hewitt Museum of Design.

Irene authored the definitive O’Reilly book, Design in Venture Capital, and her popular essays can be found on Medium. She has been featured in WIRED magazine, Fast Company magazine, CommArts magazine, and on the cover of Mindful magazine.

EPISODE SUMMARY

In this conversation we talk about:

Developing listening skills as an introspective child, and how feeling like an outsider helped her develop those skills.
Her electrical engineering studies, and her transition into looking at how technology influences society and people and how we live.
Her time at Netscape, and tying together the products for a consistent look and feel across a suite of products that came out at the time called Netscape Communicator.
Her move from Netscape to Yahoo!, and what went wrong for Yahoo! as a company trying to find its way.
Her time at Google as we look at it from all angles. What was the state of design at Google before she joined and what were the changes she tried to implement as she brought human-centered design and practices to Google?
Hiring strategies, staff training, and how design workshops ultimately became the Design Sprint at Google.
What is design and what is a designer?
And the role of the designer in venture capital.

I think my greatest takeaway from this interview is this sense of hope that someone like Irene is able to walk into these very "techy" cultures and produce real change. And all it takes is really showing the value of the work and being willing to engage and promote better practices. I think Irene will be an inspiration to many non-engineers who find themselves in heavy engineering cultures and want to make a contribution.

This conversation with Irene is one of many weekly conversations we already have lined up for you with thinkers, best-selling authors, designers, makers, scientists, impact entrepreneurs, and others who are working to change our world for the better. So please follow this podcast on your favorite podcast app, or head over to remakepod.org to subscribe.

And now, let's jump right in with Irene Au.

TIMESTAMP CHAPTERS

[5:54] Life in the Present

[7:08] Early Childhood Driving Forces

[9:40] A Journey to Design

[13:20] Entering Netscape

[16:00] The Challenges of the Early Internet

[19:23] A Transition From Netscape to Yahoo!

[22:58] The Infrastructure of Yahoo!

[30:14] Good Design Versus Bad Design

[34:04] The Winners and the Failures

[39:48] Infusing Design With Google