Mystery AI Hype Theater 3000

Emily M. Bender and Alex Hanna

Artificial Intelligence has too much hype. In this podcast, linguist Emily M. Bender and sociologist Alex Hanna break down the AI hype, separate fact from fiction, and science from bloviation. They're joined by special guests and talk about everything, from machine consciousness to science fiction, to political economy to art made by machines.

#23 in the Top 100 artificial intelligence All time chart

2 Listeners

All episodes

Best episodes

Top 10 Mystery AI Hype Theater 3000 Episodes

Goodpods has curated a list of the 10 best Mystery AI Hype Theater 3000 episodes, ranked by the number of listens and likes each episode have garnered from our listeners. If you are listening to Mystery AI Hype Theater 3000 for the first time, there's no better place to start than with one of these standout episodes. If you are a fan of the show, vote for your favorite Mystery AI Hype Theater 3000 episode by adding your comments to the episode page.

Episode 25: An LLM Says LLMs Can Do Your Job, January 22 2024

Mystery AI Hype Theater 3000

02/01/24 • 56 min

Is ChatGPT really going to take your job? Emily and Alex unpack two hype-tastic papers that make implausible claims about the number of workforce tasks LLMs might make cheaper, faster or easier. And why bad methodology may still trick companies into trying to replace human workers with mathy-math.
Visit us on PeerTube for the video of this conversation.

References:
OpenAI: GPTs are GPTs
Goldman Sachs: The Potentially Large Effects of Artificial Intelligence on Economic Growth

FYI: Over the last 60 years, automation has totally eliminated just one US occupation.

Fresh AI Hell:
Microsoft adding a dedicated "AI" key to PC keyboards.

Dr. Damien P Williams: "Yikes."

The AI-led enshittification at Duolingo

University of Washington Provost highlighting “AI”
“Using ChatGPT, My AI eBook Creation Pro helps you write an entire e-book with just three clicks -- no writing or technical experience required.”
"Can you add artificial intelligence to the hydraulics?"

Check out future streams at on Twitch, Meanwhile, send us any AI Hell you see.
Our book, 'The AI Con,' comes out in May! Pre-order now.
Subscribe to our newsletter via Buttondown.

Bluesky: emilymbender.bsky.social
Mastodon: dair-community.social/@EmilyMBender

Alex

Music by Toby Menon.
Artwork by Naomi Pleasure-Park.
Production by Christie Taylor.

1 Listener

Episode 6: Stochastic Parrot Galactica, November 23, 2022

Mystery AI Hype Theater 3000

07/17/23 • 63 min

Emily and Alex discuss MetaAI's bullshit science paper generator, Galactica, along with its defenders. Plus, where could AI actually help scientific research? And more Fresh AI Hell.
Watch the video of this episode on PeerTube.
References:
Imre Lakatos on research programs
Shah, Chirag and Emily M. Bender. 2022. Situating Search. Proceedings of the 2022 ACM SIGIR Conference on Human Information Interaction and Retrieval (CHIIR ’22).
UW RAISE (Responsibility in AI Systems and Experiences)
Stochastic Parrots:
Bender, Emily M., Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜. In Proceedings of FAccT 2021, pp.610-623.
The Octopus Paper:
Bender Emily M. and Alexander Koller. 2020. Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data. ACL 2020
Palestinian man arrested because of bad machine translation.
Katherine McKittrick, Dear Science and Other StoriesThe Sokal Hoax
Safiya Noble, Algorithms of OppressionLatanya Sweeney, "Discrimination in Online Ad Delivery"
Mehtab Khan and Alex Hanna, The Subjects and Stages of AI Dataset Development: A Framework for Dataset Accountability
(What is 'sealioning'?)
http://wondermark.com/1k62/

Grover:
Raji, Inioluwa Deborah, Emily M. Bender, Amandalynne Paullada, Emily Denton and Alex Hanna. 2021. AI and the Everything in the Whole Wide World Benchmark. Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS 2021) Track on Datasets and Benchmarks.

Ben Dickson's coverage of Grover:
Why we must rethink AI benchmarks, Ben Dickson, Tec

Check out future streams at on Twitch, Meanwhile, send us any AI Hell you see.
Our book, 'The AI Con,' comes out in May! Pre-order now.
Subscribe to our newsletter via Buttondown.

Bluesky: emilymbender.bsky.social
Mastodon: dair-community.social/@EmilyMBender

Alex

Music by Toby Menon.
Artwork by Naomi Pleasure-Park.
Production by Christie Taylor.

1 Listener

Episode 1: "Can Machines Learn To Behave?" Part 1, August 31, 2022

Mystery AI Hype Theater 3000

05/31/23 • 44 min

Technology researchers Emily M. Bender and Alex Hanna kick off the Mystery AI Hype Theater 3000 series by reading through, "Can machines learn how to behave?" by Blaise Aguera y Arcas, a Google VP who works on artificial intelligence.
This episode was recorded in August of 2022, and is the first of three about Aguera y Arcas' post.
Watch the video stream on PeerTube.

Check out future streams at on Twitch, Meanwhile, send us any AI Hell you see.
Our book, 'The AI Con,' comes out in May! Pre-order now.
Subscribe to our newsletter via Buttondown.

Bluesky: emilymbender.bsky.social
Mastodon: dair-community.social/@EmilyMBender

Alex

Music by Toby Menon.
Artwork by Naomi Pleasure-Park.
Production by Christie Taylor.

Episode 9: Call the AI Quack Doctor, February 17, 2023

Mystery AI Hype Theater 3000

08/08/23 • 62 min

Should the mathy-maths be telling doctors what might be wrong with you? And can they actually help train medical professionals to treat human patients? Alex and Emily discuss the not-so-real medical and healthcare applications of ChatGPT and other large language models.
Plus another round of fresh AI hell, featuring "charisma as a service," and other assorted reasons to tear your hair out.
This episode was first recorded on February 17th of 2023.
Watch the video of this episode on PeerTube.

References:
Glass.ai makes “diagnosis machine”:
https://twitter.com/AiBreakfast/status/1620128621821317125?t=Q6tTAOcGAoFJ3Ko9m4EC9g&s=19

Percy Liang claims 'PubMedGPT' can pass medical exams:
https://crfm.stanford.edu/2022/12/15/pubmedgpt.html
https://twitter.com/percyliang/status/1603469265583353856?s=20&t=SdWeINzUw92pbkTO8OAVqQ

Emily's reaction to the above:
https://twitter.com/emilymbender/status/1603766381807570944?s=20&t=SdWeINzUw92pbkTO8OAVqQ

ChatGPT gets 60 percent of questions right in US Medical Licensing Exam:
https://healthitanalytics.com/news/chatgpt-passes-us-medical-licensing-exam-without-clinician-input

An Apple Watch error is clogging up 911 lines:
https://www.nytimes.com/2023/02/03/health/apple-watch-911-emergency-call.html

ChatGPT-assisted diagnosis: Is the future suddenly here?
https://www.statnews.com/2023/02/13/chatgpt-assisted-diagnosis/

NVIDIA “eye contact” demo:
https://twitter.com/Jousefm2/status/1616878021280993284

“Theory of the mind":
https://twitter.com/LChoshen/status/1623575423652139015?t=Ohc9tzB09pAEddAReLc6mA&s=09

Check out future streams at on Twitch, Meanwhile, send us any AI Hell you see.
Our book, 'The AI Con,' comes out in May! Pre-order now.
Subscribe to our newsletter via Buttondown.

Bluesky: emilymbender.bsky.social
Mastodon: dair-community.social/@EmilyMBender

Alex

Music by Toby Menon.
Artwork by Naomi Pleasure-Park.
Production by Christie Taylor.

Episode 10: Don't Be A Lawyer, ChatGPT. March 3, 2023

Mystery AI Hype Theater 3000

08/16/23 • 66 min

Alex and Emily are taking AI to court! Amid big claims about LLMs, a look at the facts about ChatGPT, legal expertise, and what the bar exam actually tells you about someone's ability to practice law--with help from Harvard legal and technology scholar Kendra Albert.
This episode was first recorded on March 3, 2023.
Watch the video of this episode on PeerTube.

References:
Social Science Research Network paper “written” by ChatGPT

Joe Wanzala, “ChatGPT is ideal for eDiscovery”

Legal applications for ChatGPT:
Shot: GPT-4 'could pass the. bar exam'
Chaser: ChatGPT had bigger dreams.

"AI for law"

“AI can legally run a company”

Wired: Generative AI Is Coming for the Lawyers

“This is a decision by a Colombian court in Cartagena (dated January, 30, 2023).
As far as we know, it is the first time that a judicial decision has been taken by explicitly resorting to #ChatGPT @sama @OpenAI. The Court poses a series of specific questions to #ChatGPT"

"Don't Be A Lawyer" song from "Crazy Ex-Girlfriend"

Rep. Ted Lieu introduces legislation written by an LLM

DoNotPay offers money to anyone willing to use their AI to argue in court

Fresh AI Hell:
Vanderbilt University responds to MSU shooting with e-mail written using ChatGPT

Science fiction magazine closes submissions due to LLM spam

The 1st International Workshop on Implicit Author Characterization from Texts for Search and Retrieval (IACT’23)

Check out future streams at on Twitch, Meanwhile, send us any AI Hell you see.
Our book, 'The AI Con,' comes out in May! Pre-order now.
Subscribe to our newsletter via Buttondown.

Bluesky: emilymbender.bsky.social
Mastodon: dair-community.social/@EmilyMBender

Alex

Music by Toby Menon.
Artwork by Naomi Pleasure-Park.
Production by Christie Taylor.

Episode 11: A GPT-4 Fanfiction Novella, April 7 2023

Mystery AI Hype Theater 3000

08/24/23 • 63 min

After a hype-y few weeks of AI happenings, Alex and Emily shovel the BS on GPT-4’s “system card,” its alleged “sparks of Artificial General Intelligence,” and a criti-hype heavy "AI pause" letter. Hint: for a good time, check the citations.

This episode originally aired on Friday, April 7, 2023.
You can also watch the video of this episode on PeerTube.

References:

GPT-4 system card: https://cdn.openai.com/papers/gpt-4-system-card.pdf
“Sparks of AGI” hype: https://twitter.com/SebastienBubeck/status/1638704164770332674
And the preprint from Bubeck et al.: https://arxiv.org/abs/2303.12712

“Pause AI” letter: https://futureoflife.org/open-letter/pause-giant-ai-experiments/

The “Sparks” paper points to this 1997 editorial in their definition of “intelligence”:
https://www1.udel.edu/educ/gottfredson/reprints/1997mainstream.pdf

Radiolab's miniseries, 'G': https://radiolab.org/series/radiolab-presents-g

Baria and Cross, "The brain is a computer is a brain.": https://arxiv.org/abs/2107.14042

Senator Chris Murphy buys the hype:
https://twitter.com/ChrisMurphyCT/status/1640186536825061376

Generative “AI” is making “police sketches”:
https://twitter.com/Wolven/status/1624299508371804161?t=DXyucCPYPAKNn8TtAo0xeg&s=19

More mathy math in policing:
https://www.cbsnews.com/colorado/news/aurora-police-new-ai-system-bodycam-footage/?utm_source=dlvr.it&utm_medium=twitter

User Research without the Users:
https://twitter.com/schock/status/1643392611560878086

DoNotPay is here to cancel your gym membership:
https://twitter.com/BrianBrackeen/status/1644193519496511488?s=20

Check out future streams at on Twitch, Meanwhile, send us any AI Hell you see.
Our book, 'The AI Con,' comes out in May! Pre-order now.
Subscribe to our newsletter via Buttondown.

Bluesky: emilymbender.bsky.social
Mastodon: dair-community.social/@EmilyMBender

Alex

Music by Toby Menon.
Artwork by Naomi Pleasure-Park.
Production by Christie Taylor.

Episode 15: The White House And Big Tech Dance The Self-Regulation Tango, August 11 2023

Mystery AI Hype Theater 3000

09/20/23 • 64 min

Emily and Alex tackle the White House hype about the 'voluntary commitments' of companies to limit the harms of their large language models: but only some large language models, and only some, over-hyped kinds of harms.

Plus a full portion of Fresh Hell...and a little bit of good news.

References:

White House press release on voluntary commitments
Emily’s blog post critiquing the “voluntary commitments”
An “AI safety” infused take on regulation
AI Causes Real Harm. Let’s Focus on That over the End-of-Humanity Hype
“AI” Hurts Consumers and Workers — and Isn’t Intelligent
Fresh AI Hell:

Future of Life Institute hijacks SEO for EU's AI Act

LLMs for denying health insurance claims

NHS using “AI” as receptionist

Automated robots in reception

Can AI language models replace human research participants?

A recipe chatbot taught users how to make chlorine gas

Using a chatbot to pretend to interview Harriet Tubman

Worldcoin Orbs & iris scans

Martin Shkreli’s AI for health start up

Authors impersonated with fraudulent books on Amazon/Goodreads

Good News:

Zoom rest

Check out future streams at on Twitch, Meanwhile, send us any AI Hell you see.
Our book, 'The AI Con,' comes out in May! Pre-order now.
Subscribe to our newsletter via Buttondown.

Bluesky: emilymbender.bsky.social
Mastodon: dair-community.social/@EmilyMBender

Alex

Music by Toby Menon.
Artwork by Naomi Pleasure-Park.
Production by Christie Taylor.

Episode 13: Beware The Robo-Therapist (feat. Hannah Zeavin), June 8 2023

Mystery AI Hype Theater 3000

09/07/23 • 60 min

Emily and Alex talk to UC Berkeley scholar Hannah Zeavin about the case of the National Eating Disorders Association helpline, which tried to replace human volunteers with a chatbot--and why the datafication and automation of mental health services are an injustice that will disproportionately affect the already vulnerable.

Content note: This is a conversation that touches on mental health, people in crisis, and exploitation.

This episode was originally recorded on June 8, 2023. Watch the video version on PeerTube.

Hannah Zeavin is a scholar, writer, and editor whose work centers on the history of human sciences (psychoanalysis, psychology, and psychiatry), the history of technology and media, feminist science and technology studies, and media theory. Zeavin is an Assistant Professor of the History of Science in the Department of History and The Berkeley Center for New Media at UC Berkeley. She is the author of, "The Distance Cure: A History of Teletherapy."

References:

VICE: Eating Disorder Helpline Fires Staff, Transitions to Chatbot After Unionization

... and then pulls the chatbot.

NPR: Can an AI chatbot help people with eating disorders as well as another human?

Psychiatrist.com: NEDA suspends AI chatbot for giving harmful eating disorder advice

Politico: Suicide hotline shares data with for-profit spinoff, raising ethical questions

Danah Boyd: Crisis Text Line from my perspective.

Tech Workers Coalition: Chatbots can't care like we do.

Slate: Who's listening when you call a crisis hotline? Helplines and the carceral system.

Hannah Zeavin:

Check out future streams at on Twitch, Meanwhile, send us any AI Hell you see.
Our book, 'The AI Con,' comes out in May! Pre-order now.
Subscribe to our newsletter via Buttondown.

Bluesky: emilymbender.bsky.social
Mastodon: dair-community.social/@EmilyMBender

Alex

Music by Toby Menon.
Artwork by Naomi Pleasure-Park.
Production by Christie Taylor.

Episode 18: Rumors of Artificial General Intelligence Have Been Greatly Exaggerated, October 23 2023

Mystery AI Hype Theater 3000

10/31/23 • 60 min

Emily and Alex read through Google vice president Blaise Aguera y Arcas' recent proclamation that "artificial general intelligence is already here." Why this claim is a maze of hype and moving goalposts.

References:

Noema Magazine: "Artificial General Intelligence Is Already Here."

"AI and the Everything in the Whole Wide World Benchmark"

"Targeting the Benchmark: On Methodology and Current Natural Language Processing Research"

"Recoding Gender: Women's Changing Participation in Computing"

"The Computer Boys Take Over: Computers, Programmers, and the Politics of Technical Expertise"

"Is chess the drosophila of artificial intelligence? A social history of an algorithm"

"The logic of domains"

"Reckoning and Judgment"

Fresh AI Hell:

Using AI to meet "diversity goals" in modeling
AI ushering in a "post-plagiarism" era in writing

"Wildly effective and dirt cheap AI therapy."

Applying AI to "improve diagnosis for patients with rare diseases."

Using LLMs in scientific research

Health insurance company Cigna using AI to deny medical claims.

AI for your wearable-based workout

Check out future streams at on Twitch, Meanwhile, send us any AI Hell you see.
Our book, 'The AI Con,' comes out in May! Pre-order now.
Subscribe to our newsletter via Buttondown.

Bluesky: emilymbender.bsky.social
Mastodon: dair-community.social/@EmilyMBender

Alex

Music by Toby Menon.
Artwork by Naomi Pleasure-Park.
Production by Christie Taylor.

Episode 7: There Are Now 15 Competing Evaluation Metrics (ft. Dr. Jeremy Kahn). December 12, 2022

Mystery AI Hype Theater 3000

07/26/23 • 63 min

Emily and Alex are joined by Dr. Jeremy G. Kahn to discuss the distressingly large number of evaluation metrics for artificial intelligence, and some new AI hell.

Jeremy G. Kahn has a PhD in computational linguistics, with a focus on information-theoretic and empirical engineering approaches to dealing with natural language (in text and speech). He’s gregarious, polyglot, a semi-auto-didact, and occasionally prolix. He also likes comic books, coffee, progressive politics, information theory, lateral thinking, science fiction, science fact, linear thinking, bicycles, beer, meditation, love, play, and inquiry. He lives in Seattle with his wife Dorothy and son Elliott.
This episode was recorded on December 12, 2022.
Watch the video of this episode on PeerTube.

References:
XKCD: Standards

WikidataCon

Gish Gallop

The Bender Rule
DJ Khaled - You Played Yourself
Jeff Kao's interrogation of public comment periods.

Emily's blog post response to NYT piece

Check out future streams at on Twitch, Meanwhile, send us any AI Hell you see.
Our book, 'The AI Con,' comes out in May! Pre-order now.
Subscribe to our newsletter via Buttondown.

Bluesky: emilymbender.bsky.social
Mastodon: dair-community.social/@EmilyMBender

Alex

Music by Toby Menon.
Artwork by Naomi Pleasure-Park.
Production by Christie Taylor.

Show more best episodes

FAQ

How many episodes does Mystery AI Hype Theater 3000 have?

Mystery AI Hype Theater 3000 currently has 55 episodes available.

What topics does Mystery AI Hype Theater 3000 cover?

The podcast is about News, Ai, Tech News, Podcasts, Technology and Artificial Intelligence.

What is the most popular episode on Mystery AI Hype Theater 3000?

The episode title 'Episode 6: Stochastic Parrot Galactica, November 23, 2022' is the most popular.

What is the average episode length on Mystery AI Hype Theater 3000?

The average episode length on Mystery AI Hype Theater 3000 is 61 minutes.

How often are episodes of Mystery AI Hype Theater 3000 released?

Episodes of Mystery AI Hype Theater 3000 are typically released every 13 days, 2 hours.

When was the first episode of Mystery AI Hype Theater 3000?

The first episode of Mystery AI Hype Theater 3000 was released on May 31, 2023.

Show more FAQ

Mystery AI Hype Theater 3000

Emily M. Bender and Alex Hanna

Top 10 Mystery AI Hype Theater 3000 Episodes

FAQ

How many episodes does Mystery AI Hype Theater 3000 have?

What topics does Mystery AI Hype Theater 3000 cover?

What is the most popular episode on Mystery AI Hype Theater 3000?

What is the average episode length on Mystery AI Hype Theater 3000?

How often are episodes of Mystery AI Hype Theater 3000 released?

When was the first episode of Mystery AI Hype Theater 3000?

Comments