Building Snipd: The AI Podcast App for Learning

03/14/25 • 77 min

We are working with Amplify on the 2025 State of AI Engineering Survey to be presented at the AIE World’s Fair in SF! Join the survey to shape the future of AI Eng!

We first met Snipd (affiliate link! we get a free month, you get a free month. but this is not a sponsored pod, we’ve never done one) over a year ago, and were immediately impressed by the design, but were doubtful about the behavior of snipping as the title behavior:

Podcast apps are enormously sticky - Spotify spent almost $1b in podcast acquisitions and exclusive content just to get an 8% bump in market share among normies.

However, after a disappointing Overcast 2.0 rewrite with no AI features in the last 3 years, I finally bit the bullet and switched to Snipd.

It’s 2025, your podcast app should be able to let you search transcripts of your podcasts. Snipd is the best implementation of this so far.

And yet they keep shipping:

What impressed us wasn’t just how this tiny team of 4 was able to bootstrap a consumer AI app against massive titans and do so well; but also how seriously they think about learning through podcasts and improving retention of knowledge over time, aka “Duolingo for podcasts”.

As an educational AI podcast, that’s a mission we can get behind.

Full Video Pod

Find us on YouTube! This was the first pod we’ve ever shot outdoors!

Show Notes

How does Shazam work?

Flutter/FlutterFlow

wav2vec paper

Perplexity Online LLM

Google Search Grounding

Comparing Snipd transcription with our Bee episode

NIPS 2017 Flo Rida

Gustav Söderström - Background Audio

Timestamps

[00:00:03] Takeaways from AI Engineer NYC

[00:00:17] Weather in New York.

[00:00:26] Swyx and Snipd.

[00:01:01] Kevin's AI summit experience.

[00:01:31] Zurich and AI.

[00:03:25] SigLIP authors join OpenAI.

[00:03:39] Zurich is very costly.

[00:04:06] The Snipd origin story.

[00:05:24] Introduction to machine learning.

[00:09:28] Snipd and user knowledge extraction.

[00:13:48] App's tech stack, Flutter, Python.

[00:15:11] How speakers are identified.

[00:18:29] The concept of "backgroundable" video.

[00:29:05] Voice cloning technology.

[00:31:03] Using AI agents.

[00:34:32] Snipd's future is multi-modal AI.

[00:36:37] Snipd and existing user behaviour.

[00:42:10] The app, summary, and timestamps.

[00:55:25] The future of AI and podcasting.

[1:14:55] Voice AI

Transcript

swyx [00:00:03]: Hey, I'm here in New York with Kevin Ben-Smith of Snipd. Welcome.

Kevin [00:00:07]: Hi. Hi. Amazing to be here.

swyx [00:00:09]: Yeah. This is our first ever, I think, outdoors podcast recording.

Kevin [00:00:14]: It's quite a location for the first time, I have to say.

swyx [00:00:18]: I was actually unsure because, you know, it's cold. It's like, I checked the temperature. It's like kind of one degree Celsius, but it's not that bad with the sun. No, it's quite nice. Yeah. Especially with our beautiful tea. With the tea. Yeah. Perfect. We're going to talk about Snips. I'm a Snips user. I'm a Snips user. I had to basically, you know, apar...

We are working with Amplify on the 2025 State of AI Engineering Survey to be presented at the AIE World’s Fair in SF! Join the survey to shape the future of AI Eng!

Podcast apps are enormously sticky - Spotify spent almost $1b in podcast acquisitions and exclusive content just to get an 8% bump in market share among normies.

However, after a disappointing Overcast 2.0 rewrite with no AI features in the last 3 years, I finally bit the bullet and switched to Snipd.

It’s 2025, your podcast app should be able to let you search transcripts of your podcasts. Snipd is the best implementation of this so far.

And yet they keep shipping:

As an educational AI podcast, that’s a mission we can get behind.

Full Video Pod

Find us on YouTube! This was the first pod we’ve ever shot outdoors!

Show Notes

How does Shazam work?

Flutter/FlutterFlow

wav2vec paper

Perplexity Online LLM

Google Search Grounding

Comparing Snipd transcription with our Bee episode

NIPS 2017 Flo Rida

Gustav Söderström - Background Audio

Timestamps

[00:00:03] Takeaways from AI Engineer NYC

[00:00:17] Weather in New York.

[00:00:26] Swyx and Snipd.

[00:01:01] Kevin's AI summit experience.

[00:01:31] Zurich and AI.

[00:03:25] SigLIP authors join OpenAI.

[00:03:39] Zurich is very costly.

[00:04:06] The Snipd origin story.

[00:05:24] Introduction to machine learning.

[00:09:28] Snipd and user knowledge extraction.

[00:13:48] App's tech stack, Flutter, Python.

[00:15:11] How speakers are identified.

[00:18:29] The concept of "backgroundable" video.

[00:29:05] Voice cloning technology.

[00:31:03] Using AI agents.

[00:34:32] Snipd's future is multi-modal AI.

[00:36:37] Snipd and existing user behaviour.

[00:42:10] The app, summary, and timestamps.

[00:55:25] The future of AI and podcasting.

[1:14:55] Voice AI

Transcript

swyx [00:00:03]: Hey, I'm here in New York with Kevin Ben-Smith of Snipd. Welcome.

Kevin [00:00:07]: Hi. Hi. Amazing to be here.

swyx [00:00:09]: Yeah. This is our first ever, I think, outdoors podcast recording.

Kevin [00:00:14]: It's quite a location for the first time, I have to say.

Previous Episode

⚡️The new OpenAI Agents Platform

While everyone is now repeating that 2025 is the “Year of the Agent”, OpenAI is heads down building towards it. In the first 2 months of the year they released Operator and Deep Research (arguably the most successful agent archetype so far), and today they are bringing a lot of those capabilities to the API:

A new open source Agents SDK with integrated Observability Tools

We cover all this and more in today’s lightning pod on YouTube!

More details here:

Responses API

In our Michelle Pokrass episode we talked about the Assistants API needing a redesign. Today OpenAI is launching the Responses API, “a more flexible foundation for developers building agentic applications”. It’s a superset of the chat completion API, and the suggested starting point for developers working with OpenAI models.

One of the big upgrades is the new set of built-in tools for the responses API: Web Search, Computer Use, and Files.

Web Search Tool

We previously had Exa AI on the podcast to talk about web search for AI. OpenAI is also now joining the race; the Web Search API is actually a new “model” that exposes two 4o fine-tunes: gpt-4o-search-preview and gpt-4o-mini-search-preview. These are the same models that power ChatGPT Search, and are priced at $30/1000 queries and $25/1000 queries respectively.

The killer feature is inline citations: you do not only get a link to a page, but also a deep link to exactly where your query was answered in the result page.

Computer Use Tool

The model that powers Operator, called Computer-Using-Agent (CUA), is also now available in the API. The computer-use-preview model is SOTA on most benchmarks, achieving 38.1% success on OSWorld for full computer use tasks, 58.1% on WebArena, and 87% on WebVoyager for web-based interactions.

As you will notice in the docs, `computer-use-preview` is both a model and a tool through which you can specify the environment.

Usage is priced at $3/1M input tokens and $12/1M output tokens, and it’s currently only available to users in tiers 3-5.

File Search Tool

File Search was also available in the Assistants API, and it’s now coming to Responses too. OpenAI is bringing search + RAG all under one umbrella, and we’ll definitely see more people trying to find new ways to build all-in-one apps on OpenAI.

Usage is priced at $2.50 per thousand queries and file storage at $0.10/GB/day, with the first GB free.

Agent SDK: Swarms++!

https://github.com/openai/openai-agents-python

To bring it all together, after the viral reception to Swarm, OpenAI is releasing an officially supported agents framework (which was previewed at our AI Engineer Summit) with 4 core pieces:

Agents: Easily configurable LLMs with clear instructions and built-in tools.

Handoffs: Intelligently transfer control between agents.

Guardrails: Configurable safety checks for input and output validation.

Tracing & Observability: Visualize agent execution traces to debug and optimize performance.

Multi-agent workflows are here to stay!

OpenAI is now explicitly designs for a set of common agentic patterns: Workflows, Handoffs, Agents-as-Tools, LLM-as-a-Judge, Parallelization, and Guardrails. OpenAI previewed this in part 2 of their talk at NYC:

Further coverage of the launch from

Next Episode

The Agent Network — Dharmesh Shah

If you’re in SF: Join us for the Claude Plays Pokemon hackathon this Sunday!

If you’re not: Fill out the 2025 State of AI Eng survey for $250 in Amazon cards!

We are SO excited to share our conversation with Dharmesh Shah, co-founder of HubSpot and creator of Agent.ai.

A particularly compelling concept we discussed is the idea of "hybrid teams" - the next evolution in workplace organization where human workers collaborate with AI agents as team members. Just as we previously saw hybrid teams emerge in terms of full-time vs. contract workers, or in-office vs. remote workers, Dharmesh predicts that the next frontier will be teams composed of both human and AI members. This raises interesting questions about team dynamics, trust, and how to effectively delegate tasks between human and AI team members.

The discussion of business models in AI reveals an important distinction between Work as a Service (WaaS) and Results as a Service (RaaS), something Dharmesh has written extensively about. While RaaS has gained popularity, particularly in customer support applications where outcomes are easily measurable, Dharmesh argues that this model may be over-indexed. Not all AI applications have clearly definable outcomes or consistent economic value per transaction, making WaaS more appropriate in many cases. This insight is particularly relevant for businesses considering how to monetize AI capabilities.

The technical challenges of implementing effective agent systems are also explored, particularly around memory and authentication. Shah emphasizes the importance of cross-agent memory sharing and the need for more granular control over data access. He envisions a future where users can selectively share parts of their data with different agents, similar to how OAuth works but with much finer control. This points to significant opportunities in developing infrastructure for secure and efficient agent-to-agent communication and data sharing.

Other highlights from our conversation

The Evolution of AI-Powered Agents – Exploring how AI agents have evolved from simple chatbots to sophisticated multi-agent systems, and the role of MCPs in enabling that.

Hybrid Digital Teams and the Future of Work – How AI agents are becoming teammates rather than just tools, and what this means for business operations and knowledge work.

Memory in AI Agents – The importance of persistent memory in AI systems and how shared memory across agents could enhance collaboration and efficiency.

Business Models for AI Agents – Exploring the shift from software as a service (SaaS) to work as a service (WaaS) and results as a service (RaaS), and what this means for monetization.

The Role of Standards Like MCP – Why MCP has been widely adopted and how it enables agent collaboration, tool use, and discovery.

The Future of AI Code Generation and Software Engineering – How AI-assisted coding is changing the role of software engineers and what skills will matter most in the future.

Domain Investing and Efficient Markets – Dharmesh’s approach to domain investing and how inefficiencies in digital asset markets create business opportunities.

The Philosophy of Saying No – Lessons from "Sorry, You Must Pass" and how prioritization leads to greater productivity and focus.

Timestamps

00:00 Introduction and Guest Welcome

02:29 Dharmesh Shah's Journey into AI

05:22 Defining AI Agents

06:45 The Evolution and Future of AI Agents

13:53 Graph Theory and Knowledge Representation

20:02 Engineering Practices and Overengineering

25:57 The Role of Junior Engineers in the AI Era

28:20 Multi-Agent Systems and MCP Standards

35:55 LinkedIn's Legal Battles and Data Scraping

37:32 The Future of AI and Hybrid Teams