Episode 6: Stochastic Parrot Galactica, November 23, 2022

07/17/23 • 63 min

Emily and Alex discuss MetaAI's bullshit science paper generator, Galactica, along with its defenders. Plus, where could AI actually help scientific research? And more Fresh AI Hell.
Watch the video of this episode on PeerTube.
References:
Imre Lakatos on research programs
Shah, Chirag and Emily M. Bender. 2022. Situating Search. Proceedings of the 2022 ACM SIGIR Conference on Human Information Interaction and Retrieval (CHIIR ’22).
UW RAISE (Responsibility in AI Systems and Experiences)
Stochastic Parrots:
Bender, Emily M., Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜. In Proceedings of FAccT 2021, pp.610-623.
The Octopus Paper:
Bender Emily M. and Alexander Koller. 2020. Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data. ACL 2020
Palestinian man arrested because of bad machine translation.
Katherine McKittrick, Dear Science and Other StoriesThe Sokal Hoax
Safiya Noble, Algorithms of OppressionLatanya Sweeney, "Discrimination in Online Ad Delivery"
Mehtab Khan and Alex Hanna, The Subjects and Stages of AI Dataset Development: A Framework for Dataset Accountability
(What is 'sealioning'?)
http://wondermark.com/1k62/

Grover:
Raji, Inioluwa Deborah, Emily M. Bender, Amandalynne Paullada, Emily Denton and Alex Hanna. 2021. AI and the Everything in the Whole Wide World Benchmark. Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS 2021) Track on Datasets and Benchmarks.

Ben Dickson's coverage of Grover:
Why we must rethink AI benchmarks, Ben Dick

You can check out future livestreams at https://twitch.tv/DAIR_Institute.
Subscribe to our newsletter via Buttondown.

Twitter: https://twitter.com/EmilyMBender
Mastodon: https://dair-community.social/@EmilyMBender
Bluesky: https://bsky.app/profile/emilymbender.bsky.social

Alex

Twitter: https://twitter.com/@alexhanna
Mastodon: https://dair-community.social/@alex
Bluesky: https://bsky.app/profile/alexhanna.bsky.social

Music by Toby Menon.
Artwork by Naomi Pleasure-Park.
Production by Christie Taylor.

Explicit content warning

07/17/23 • 63 min

1 Listener

Mystery AI Hype Theater 3000 - Episode 6: Stochastic Parrot Galactica, November 23, 2022

Transcript

ALEX: Welcome everyone!...to Mystery AI Hype Theater 3000, where we seek catharsis in this age of AI hype! We find the worst of it and pop it with the sharpest needles we can find.

EMILY: Along the way, we learn to always read the footnotes. And each time we think we’ve reached peak AI hype -- the summit of bullshit mountain -- we discover there’s worse to come.

I’m Emily M. Bender, a professor of linguistics at the University of Washington.

ALEX: And I’m Alex Hanna,

Generate a badge

Get a badge for your website that links back to this episode

Select type & size

Copy