Episode 11: A GPT-4 Fanfiction Novella, April 7 2023

Explicit content warning

08/24/23 • 63 min

After a hype-y few weeks of AI happenings, Alex and Emily shovel the BS on GPT-4’s “system card,” its alleged “sparks of Artificial General Intelligence,” and a criti-hype heavy "AI pause" letter. Hint: for a good time, check the citations.

This episode originally aired on Friday, April 7, 2023.
You can also watch the video of this episode on PeerTube.

References:

GPT-4 system card: https://cdn.openai.com/papers/gpt-4-system-card.pdf
“Sparks of AGI” hype: https://twitter.com/SebastienBubeck/status/1638704164770332674
And the preprint from Bubeck et al.: https://arxiv.org/abs/2303.12712

“Pause AI” letter: https://futureoflife.org/open-letter/pause-giant-ai-experiments/

The “Sparks” paper points to this 1997 editorial in their definition of “intelligence”:
https://www1.udel.edu/educ/gottfredson/reprints/1997mainstream.pdf

Radiolab's miniseries, 'G': https://radiolab.org/series/radiolab-presents-g

Baria and Cross, "The brain is a computer is a brain.": https://arxiv.org/abs/2107.14042

Senator Chris Murphy buys the hype:
https://twitter.com/ChrisMurphyCT/status/1640186536825061376

Generative “AI” is making “police sketches”:
https://twitter.com/Wolven/status/1624299508371804161?t=DXyucCPYPAKNn8TtAo0xeg&s=19

More mathy math in policing:
https://www.cbsnews.com/colorado/news/aurora-police-new-ai-system-bodycam-footage/?utm_source=dlvr.it&utm_medium=twitter

User Research without the Users:
https://twitter.com/schock/status/1643392611560878086

DoNotPay is here to cancel your gym membership:
https://twitter.com/BrianBrackeen/status/1644193519496511488?s=20

Check out future streams at on Twitch, Meanwhile, send us any AI Hell you see.
Our book, 'The AI Con,' comes out in May! Pre-order now.
Subscribe to our newsletter via Buttondown.

Bluesky: emilymbender.bsky.social
Mastodon: dair-community.social/@EmilyMBender

Alex

Music by Toby Menon.
Artwork by Naomi Pleasure-Park.
Production by Christie Taylor.

This episode originally aired on Friday, April 7, 2023.
You can also watch the video of this episode on PeerTube.

References:

“Pause AI” letter: https://futureoflife.org/open-letter/pause-giant-ai-experiments/

The “Sparks” paper points to this 1997 editorial in their definition of “intelligence”:
https://www1.udel.edu/educ/gottfredson/reprints/1997mainstream.pdf

Radiolab's miniseries, 'G': https://radiolab.org/series/radiolab-presents-g

Baria and Cross, "The brain is a computer is a brain.": https://arxiv.org/abs/2107.14042

Senator Chris Murphy buys the hype:
https://twitter.com/ChrisMurphyCT/status/1640186536825061376

Generative “AI” is making “police sketches”:
https://twitter.com/Wolven/status/1624299508371804161?t=DXyucCPYPAKNn8TtAo0xeg&s=19

More mathy math in policing:
https://www.cbsnews.com/colorado/news/aurora-police-new-ai-system-bodycam-footage/?utm_source=dlvr.it&utm_medium=twitter

User Research without the Users:
https://twitter.com/schock/status/1643392611560878086

DoNotPay is here to cancel your gym membership:
https://twitter.com/BrianBrackeen/status/1644193519496511488?s=20

Check out future streams at on Twitch, Meanwhile, send us any AI Hell you see.
Our book, 'The AI Con,' comes out in May! Pre-order now.
Subscribe to our newsletter via Buttondown.

Bluesky: emilymbender.bsky.social
Mastodon: dair-community.social/@EmilyMBender

Alex

Music by Toby Menon.
Artwork by Naomi Pleasure-Park.
Production by Christie Taylor.

Previous Episode

Episode 10: Don't Be A Lawyer, ChatGPT. March 3, 2023

Alex and Emily are taking AI to court! Amid big claims about LLMs, a look at the facts about ChatGPT, legal expertise, and what the bar exam actually tells you about someone's ability to practice law--with help from Harvard legal and technology scholar Kendra Albert.
This episode was first recorded on March 3, 2023.
Watch the video of this episode on PeerTube.

References:
Social Science Research Network paper “written” by ChatGPT

Joe Wanzala, “ChatGPT is ideal for eDiscovery”

Legal applications for ChatGPT:
Shot: GPT-4 'could pass the. bar exam'
Chaser: ChatGPT had bigger dreams.

"AI for law"

“AI can legally run a company”

Wired: Generative AI Is Coming for the Lawyers

“This is a decision by a Colombian court in Cartagena (dated January, 30, 2023).
As far as we know, it is the first time that a judicial decision has been taken by explicitly resorting to #ChatGPT @sama @OpenAI. The Court poses a series of specific questions to #ChatGPT"

"Don't Be A Lawyer" song from "Crazy Ex-Girlfriend"

Rep. Ted Lieu introduces legislation written by an LLM

DoNotPay offers money to anyone willing to use their AI to argue in court

Fresh AI Hell:
Vanderbilt University responds to MSU shooting with e-mail written using ChatGPT

Science fiction magazine closes submissions due to LLM spam

The 1st International Workshop on Implicit Author Characterization from Texts for Search and Retrieval (IACT’23)

Check out future streams at on Twitch, Meanwhile, send us any AI Hell you see.
Our book, 'The AI Con,' comes out in May! Pre-order now.
Subscribe to our newsletter via Buttondown.

Bluesky: emilymbender.bsky.social
Mastodon: dair-community.social/@EmilyMBender

Alex

Music by Toby Menon.
Artwork by Naomi Pleasure-Park.
Production by Christie Taylor.

Next Episode

Episode 12: It's All Hell, May 5 2023

Take a deep breath and join Alex and Emily in AI Hell itself, as they take down a month's worth of hype in a mere 60 minutes.

This episode aired on Friday, May 5, 2023.
Watch the video of this episode on PeerTube.

References:

Terrifying NEJM article on GPT-4 in medicine

“Healthcare professionals preferred ChatGPT 79% of the time”

Good thoughts from various experts in response

ChatGPT supposedly reading dental x-rays

Chatbots “need” therapists

CEO proposes AI therapist, removes proposal upon realizing there’s regulation:
https://twitter.com/BEASTMODE/status/1650013819693944833 (deleted)

ChatGPT is more carbon efficient than human writers

Asking disinformation machine for confirmation bias

GPT-4 glasses to tell you what to say on dates, "Charisma as a Service"

Context-aware fill for missing data

“Overemployed” with help from ChatGPT

Pakistani court uses GPT-4 in bail decision
ChatGPT in Peruvian and Mexican courts

Elon Musk’s deepfake defense

Elon Musk's TruthGPT

Fake interview in German publication revealed as “AI” at the end of the article

Check out future streams at on Twitch, Meanwhile, send us any AI Hell you see.
Our book, 'The AI Con,' comes out in May! Pre-order now.
Subscribe to our newsletter via Buttondown.

Bluesky: emilymbender.bsky.social
Mastodon: dair-community.social/@EmilyMBender

Alex

Music by Toby Menon.
Artwork by Naomi Pleasure-Park.
Production by Christie Taylor.

Mystery AI Hype Theater 3000 - Episode 11: A GPT-4 Fanfiction Novella, April 7 2023

Transcript

ALEX: Welcome everyone!...to Mystery AI Hype Theater 3000, where we seek catharsis in this age of AI hype! We find the worst of it and pop it with the sharpest needles we can find.

EMILY: Along the way, we learn to always read the footnotes. And each time we think we’ve reached peak AI hype -- the summit of bullshit mountain -- we discover there’s worse to come.

I’m Emily M. Bender, a professor of linguistics at the University of Washington.

ALEX: And I’m Alex Hanna,