Emulating Humans with NSFW Chatbots - with Jesse Silver
05/16/24 • 54 min
Disclaimer: today’s episode touches on NSFW topics. There’s no graphic content or explicit language, but we wouldn’t recommend blasting this in work environments.
Product website: https://usewhisper.me/
For over 20 years it’s been an open secret that porn drives many new consumer technology innovations, from VHS and Pay-per-view to VR and the Internet. It’s been no different in AI - many of the most elite Stable Diffusion and Llama enjoyers and merging/prompting/PEFT techniques were born in the depths of subreddits and 4chan boards affectionately descibed by friend of the pod as The Waifu Research Department. However this topic is very under-covered in mainstream AI media because of its taboo nature.
That changes today, thanks to our new guest Jesse Silver.
The AI Waifu Explosion
In 2023, the Valley’s worst kept secret was how much the growth and incredible retention of products like Character.ai & co was being boosted by “ai waifus” (not sure what the “husband” equivalent is, but those too!).
And we can look at subreddit growth as a proxy for the general category explosion (10x’ed in the last 8 months of 2023):
While all the B2B founders were trying to get models to return JSON, the consumer applications made these chatbots extremely engaging and figured out how to make them follow their instructions and “personas” very well, with the greatest level of scrutiny and most demanding long context requirements. Some of them, like Replika, make over $50M/year in revenue, and this is -after- their controversial update deprecating Erotic Roleplay (ERP).
A couple of days ago, OpenAI announced GPT-4o (see our AI News recap) and the live voice demos were clearly inspired by the movie Her.
The Latent Space Discord did a watch party and both there and on X a ton of folks were joking at how flirtatious the model was, which to be fair was disturbing to many:
From Waifus to Fan Platforms
Where Waifus are known by human users to be explicitly AI chatbots, the other, much more challenging end of the NSFW AI market is run by AIs successfully (plausibly) emulating a specific human personality for chat and ecommerce.
You might have heard of fan platforms like OnlyFans. Users can pay for a subscription to a creator to get access to private content, similarly to Patreon and the likes, but without any NSFW restrictions or any other content policies. In 2023, OnlyFans had over $1.1B of revenue (on $5.6b of GMV).
The status quo today is that a lot of the creators outsource their chatting with fans to teams in the Philippines and other lower cost countries for ~$3/hr + 5% commission, but with very poor quality - most creators have fired multiple teams for poor service.
Today’s episode is with Jesse Silver; along with his co-founder Adam Scrivener, they run a SaaS platform that helps creators from fan platforms build AI chatbots for their fans to chat with, including selling from an inventory of digital content. Some users generate over $200,000/mo in revenue.
We talked a lot about their tech stack, why you need a state machine to successfully run multi-thousand-turn conversations, how they develop prompts and fine-tune models with DSPy, the NSFW limitations of commercial models, but one of the most interesting points is that often users know that they ...
Disclaimer: today’s episode touches on NSFW topics. There’s no graphic content or explicit language, but we wouldn’t recommend blasting this in work environments.
Product website: https://usewhisper.me/
For over 20 years it’s been an open secret that porn drives many new consumer technology innovations, from VHS and Pay-per-view to VR and the Internet. It’s been no different in AI - many of the most elite Stable Diffusion and Llama enjoyers and merging/prompting/PEFT techniques were born in the depths of subreddits and 4chan boards affectionately descibed by friend of the pod as The Waifu Research Department. However this topic is very under-covered in mainstream AI media because of its taboo nature.
That changes today, thanks to our new guest Jesse Silver.
The AI Waifu Explosion
In 2023, the Valley’s worst kept secret was how much the growth and incredible retention of products like Character.ai & co was being boosted by “ai waifus” (not sure what the “husband” equivalent is, but those too!).
And we can look at subreddit growth as a proxy for the general category explosion (10x’ed in the last 8 months of 2023):
While all the B2B founders were trying to get models to return JSON, the consumer applications made these chatbots extremely engaging and figured out how to make them follow their instructions and “personas” very well, with the greatest level of scrutiny and most demanding long context requirements. Some of them, like Replika, make over $50M/year in revenue, and this is -after- their controversial update deprecating Erotic Roleplay (ERP).
A couple of days ago, OpenAI announced GPT-4o (see our AI News recap) and the live voice demos were clearly inspired by the movie Her.
The Latent Space Discord did a watch party and both there and on X a ton of folks were joking at how flirtatious the model was, which to be fair was disturbing to many:
From Waifus to Fan Platforms
Where Waifus are known by human users to be explicitly AI chatbots, the other, much more challenging end of the NSFW AI market is run by AIs successfully (plausibly) emulating a specific human personality for chat and ecommerce.
You might have heard of fan platforms like OnlyFans. Users can pay for a subscription to a creator to get access to private content, similarly to Patreon and the likes, but without any NSFW restrictions or any other content policies. In 2023, OnlyFans had over $1.1B of revenue (on $5.6b of GMV).
The status quo today is that a lot of the creators outsource their chatting with fans to teams in the Philippines and other lower cost countries for ~$3/hr + 5% commission, but with very poor quality - most creators have fired multiple teams for poor service.
Today’s episode is with Jesse Silver; along with his co-founder Adam Scrivener, they run a SaaS platform that helps creators from fan platforms build AI chatbots for their fans to chat with, including selling from an inventory of digital content. Some users generate over $200,000/mo in revenue.
We talked a lot about their tech stack, why you need a state machine to successfully run multi-thousand-turn conversations, how they develop prompts and fine-tune models with DSPy, the NSFW limitations of commercial models, but one of the most interesting points is that often users know that they ...
Previous Episode
WebSim, WorldSim, and The Summer of Simulative AI — with Joscha Bach of Liquid AI, Karan Malhotra of Nous Research, Rob Haisfield of WebSim.ai
We are 200 people over our 300-person venue capacity for AI UX 2024, but you can subscribe to our YouTube for the video recaps.
Our next event, and largest EVER, is the AI Engineer World’s Fair. See you there!
Parental advisory: Adult language used in the first 10 mins of this podcast.
Any accounting of Generative AI that ends with RAG as its “final form” is seriously lacking in imagination and missing out on its full potential. While AI generation is very good for “spicy autocomplete” and “reasoning and retrieval with in context learning”, there’s a lot of untapped potential for simulative AI in exploring the latent space of multiverses adjacent to ours.
GANs
Many research scientists credit the 2017 Transformer for the modern foundation model revolution, but for many artists the origin of “generative AI” traces a little further back to the Generative Adversarial Networks proposed by Ian Goodfellow in 2014, spawning an army of variants and Cats and People that do not exist:
We can directly visualize the quality improvement in the decade since:
GPT-2
Of course, more recently, text generative AI started being too dangerous to release in 2019 and claiming headlines. AI Dungeon was the first to put GPT2 to a purely creative use, replacing human dungeon masters and DnD/MUD games of yore.
More recent gamelike work like the Generative Agents (aka Smallville) paper keep exploring the potential of simulative AI for game experiences.
ChatGPT
Not long after ChatGPT broke the Internet, one of the most fascinating generative AI finds was Jonas Degrave (of Deepmind!)’s Building A Virtual Machine Inside ChatGPT:
The open-ended interactivity of ChatGPT and all its successors enabled an “open world” type simulation where “hallucination” is a feature and a gift to dance with, rather than a nasty bug to be stamped out. However, further updates to ChatGPT seemed to “nerf” the model’s ability to perform creative simulations, particularly with the deprecation of the `completion` mode of APIs in favor of `chatCompletion`.
WorldSim (https://worldsim.nousresearch.com/)
It is with this context we explain WorldSim and WebSim. We recommend you watch the WorldSim demo video on our YouTube for the best context, but basically if you are a developer it is a Claude prompt that is a portal into another world of your own choosing, that you can navigate with bash commands that you make up.
The live video demo was highly enjoyable:
Why Claude? Hints from Amanda Askell on the Claude 3 system prompt gave some inspiration, and subsequent discoveries that Claude 3 is "less nerfed” than GPT 4 Turbo turned the growing Simulative AI community into Anthropic stans.
WebSim (https://websim.ai/)
This was a one day hackathon project inspired by WorldSim that should have won:
In short, you type in a URL that you made up, and Claude 3 does its level best to generate a webpage that doesn’t exist, that would fit your URL. All form POST requests are intercepted and responded to, and all links lead to even more webpages, that don’t exist, that are generated when you make them. All pages are cachable, modifiable and regeneratable -...
Next Episode
ICLR 2024 — Best Papers & Talks (ImageGen, Vision, Transformers, State Space Models) ft. Durk Kingma, Christian Szegedy, Ilya Sutskever
Speakers for AI Engineer World’s Fair have been announced! See our Microsoft episode for more info and buy now with code LATENTSPACE — we’ve been studying the best ML research conferences so we can make the best AI industry conf!
Note that this year there are 4 main tracks per day and dozens of workshops/expo sessions; the free livestream will air much less than half of the content this time.
Apply for free/discounted Diversity Program and Scholarship tickets here. We hope to make this the definitive technical conference for ALL AI engineers.
UPDATE: This is a 2 part episode - see Part 2 here.
ICLR 2024 took place from May 6-11 in Vienna, Austria.
Just like we did for our extremely popular NeurIPS 2023 coverage, we decided to pay the $900 ticket (thanks to all of you paying supporters!) and brave the 18 hour flight and 5 day grind to go on behalf of all of you. We now present the results of that work!
This ICLR was the biggest one by far, with a marked change in the excitement trajectory for the conference:
Of the 2260 accepted papers (31% acceptance rate), of the subset of those relevant to our shortlist of AI Engineering Topics, we found many, many LLM reasoning and agent related papers, which we will cover in the next episode. We will spend this episode with 14 papers covering other relevant ICLR topics, as below.
As we did last year, we’ll start with the Best Paper Awards. Unlike last year, we now group our paper selections by subjective topic area, and mix in both Outstanding Paper talks as well as editorially selected poster sessions. Where we were able to do a poster session interview, please scroll to the relevant show notes for images of their poster for discussion. To cap things off, Chris Ré’s spot from last year now goes to Sasha Rush for the obligatory last word on the development and applications of State Space Models.
We had a blast at ICLR 2024 and you can bet that we’ll be back in 2025 🇸🇬.
Timestamps and Overview of Papers
[00:02:49] Section A: ImageGen, Compression, Adversarial Attacks
[00:02:49] VAEs
[00:32:36] Würstchen: An Efficient Architecture for Large-Scale Text-to-Image Diffusion Models
[00:37:25] The Hidden Language Of Diffusion Models
[00:48:40] Ilya on Compression
[01:01:45] Christian Szegedy on Compression
[01:07:34] Intriguing properties of neural networks
[01:26:07] Section B: Vision Learning and Weak Supervision
[01:26:45] Vision Transformers Need Registers
[01:38:27] Think before you speak: Training Language Models With Pause Tokens
[01:47:06] Towards a statistical theory of data selection under weak supervision
[02:00:32] Is ImageNet worth 1 video?
[02:06:32] Section C: Extending Transformers and Attention
[02:06:49] LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models
[02:15:12] YaRN: Efficient Context Window Extension of Large Language Models
[02:32:02] Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs
[02:44:57] ZeRO++: Extremely Efficient Collective Communication for Giant Model Training
[02:54:26] Section D: State Space Models vs Transformers
[03:31:15] Never Train from Scratch: Fair Comparison of Long-Sequence Models Requires Data-Driven Priors
[03:37:08] End of Part 1
A: ImageGen, Compression, Adversarial Attacks
Durk Kingma (OpenAI/Google DeepMind) & Max Welling:
If you like this episode you’ll love
Episode Comments
Generate a badge
Get a badge for your website that links back to this episode
<a href="https://goodpods.com/podcasts/latent-space-the-ai-engineer-podcast-262472/emulating-humans-with-nsfw-chatbots-with-jesse-silver-51956426"> <img src="https://storage.googleapis.com/goodpods-images-bucket/badges/generic-badge-1.svg" alt="listen to emulating humans with nsfw chatbots - with jesse silver on goodpods" style="width: 225px" /> </a>
Copy