Docker Model Runner

04/21/25 • 13 min

DevOps and Docker Talk: Cloud Native Interviews and Tooling

Docker launched "Docker Model Runner" to run LLMs through llama.cpp with a single "docker model" command. In this episode Bret details examples and some useful use cases for using this way to run LLMs. He breaks down the internals. How it works, when you should use it or not use it; and, how to get started using Open WebUI for a private ChatGPT-like experience.

★Topics★
Model Runner Docs
Hub Models
OCI Artifacts
Open WebUI
My Open WebUI Compose file

Creators & Guests

Cristi Cotovan - Editor
Beth Fisher - Producer
Bret Fisher - Host
(00:00) - Intro
(00:46) - Model Runner Elevator Pitch
(01:28) - Enabling Docker Model Runner
(04:28) - Self Promotion! Is that an ad? For me?
(05:03) - Downloading Models
(07:11) - Architectrure of Model Runner
(10:49) - ORAS
(11:09) - What's next for Model Runner?
(12:13) - Troubleshooting

You can also support my free material by subscribing to my YouTube channel and my weekly newsletter at bret.news!

Grab the best coupons for my Docker and Kubernetes courses.
Join my cloud native DevOps community on Discord.
Grab some merch at Bret's Loot Box
Homepage bretfisher.com

★Topics★
Model Runner Docs
Hub Models
OCI Artifacts
Open WebUI
My Open WebUI Compose file

Creators & Guests

Cristi Cotovan - Editor
Beth Fisher - Producer
Bret Fisher - Host
(00:00) - Intro
(00:46) - Model Runner Elevator Pitch
(01:28) - Enabling Docker Model Runner
(04:28) - Self Promotion! Is that an ad? For me?
(05:03) - Downloading Models
(07:11) - Architectrure of Model Runner
(10:49) - ORAS
(11:09) - What's next for Model Runner?
(12:13) - Troubleshooting

You can also support my free material by subscribing to my YouTube channel and my weekly newsletter at bret.news!

Grab the best coupons for my Docker and Kubernetes courses.
Join my cloud native DevOps community on Discord.
Grab some merch at Bret's Loot Box
Homepage bretfisher.com

Previous Episode

What's Coming in 2025?

This episode is about what I'm seeing and what I'm doing right now, and then for the rest of the year. There are three parts. First, I talk about what's about to happen for me for the next few weeks re going to London for KubeCon. Then what I'm planning to change in this podcast, as well as my other content on YouTube for the rest of the year. And lastly, I talk about some industry trends that I'm seeing that will force me, I think, to change the format of this show. I recorded the episode on March 22, 2025.
★Topics★
My work at KubeCon EU in London
What's next for this Podcast and my YouTube
What's up with AI for DevOps?

Creators & Guests

Beth Fisher - Producer
Bret Fisher - Host
(00:00) - What's Coming in 2025
(01:07) - Highlights I'm excited about re KubeCon
(04:35) - Changes to this Podcast
(05:58) - What's up with AI and "Agentic DevOps"?
(15:11) - Upcoming guests

You can also support my free material by subscribing to my YouTube channel and my weekly newsletter at bret.news!

Grab the best coupons for my Docker and Kubernetes courses.
Join my cloud native DevOps community on Discord.
Grab some merch at Bret's Loot Box
Homepage bretfisher.com

If you like this episode you’ll love

Geschichte ist Gegenwart! Der History & Politics Podcast der Körber-Stiftung

Aspen Ideas to Go

Joomla Beat Podcast | Web design, development, online marketing, social media & website management

History Defeats Itself

The Women's Eye with Stacey Gualandi | Women Leaders, Entrepreneurs, Authors and Global Changemakers

Episode Comments

Generate a badge

Get a badge for your website that links back to this episode

Select type & size

Copy