The Agile Embedded Podcast

Discussing Modern Development Techniques in Embedded Systems

In this episode of the Agile Embedded Podcast, hosts Jeff Gable and Luca Ingianni play buzzword bingo, discussing various modern development techniques, tools, and methodologies in the context of embedded systems. They explore the relevance and applicability of terms like microservices, serverless, Kubernetes, Docker, continuous integration, continuous delivery, continuous deployment, A/B testing, API-first design, behavior-driven development, and modern languages such as Rust and C++. They also touch upon the use of AI in development and deployment processes, along with debugging, observability, monitoring, and simulation tools like Memfault, Mender, and Renode. The conversation seeks to provide insights and opinions on which of these techniques and tools are worth investigating or applying to embedded projects.

00:00 Introduction to the Agile Embedded Podcast
00:06 Exploring Modern Development Buzzwords
01:26 Web Technologies in Embedded Systems
05:28 Diving into Docker for Embedded Development
07:49 Understanding Immutable Infrastructure
13:32 Continuous Integration, Delivery, and Deployment
19:53 A/B Testing in IoT Devices
23:06 API First Design in Embedded Systems
26:08 Modern Languages: The Case for Rust
27:54 Exploring Rust in Embedded Systems
28:37 Challenges with Modern C Programming
30:10 Embedded Linux and Language Flexibility
32:27 Model-Based Development with Simulink
33:53 Modern Tooling for Embedded Systems
40:25 AI in Embedded Systems
44:05 AI-Assisted Development
47:36 Conclusion and Contact Information

You can find Jeff at https://jeffgable.com.
You can find Luca at https://luca.engineer.

Want to join the agile Embedded Slack? Click here

The episode discusses the concept of Minimum Viable Continuous Delivery (Minimum CD), which represents a counter-movement to heavyweight frameworks like SAFe. The hosts explore how Minimum CD provides a set of essential practices for successfully building software-based products without unnecessary complexity. The approach focuses on core principles rather than rigid frameworks, making it particularly relevant for embedded systems development.

The discussion covers the fundamental requirements for continuous delivery, including automated testing, pipeline-driven deployments, and trunk-based development. The hosts emphasize that while these practices may seem challenging for embedded systems, they become increasingly important as devices become more sophisticated and connected.

A key theme throughout the episode is the importance of building trust in the development process through automation, consistent practices, and cultural commitment. The hosts stress that while some practices may seem difficult to implement in embedded systems, the more challenging they are, the more valuable they become when successfully implemented.

Timestamps and Topics:
00:00:00 - Introduction and overview of Minimum CD
00:02:00 - Discussion of Minimum CD as counter-movement to complex frameworks
00:03:45 - Continuous Integration fundamentals
00:15:35 - Pipeline as the only way to deploy
00:27:00 - Production-like test environments
00:29:45 - Rollback capabilities for connected devices
00:32:25 - Configuration deployment with artifacts
00:34:50 - Trunk-based development principles
00:39:30 - Automated testing requirements
00:41:10 - Maintaining delivered work integrity
00:45:55 - Wrap-up and closing thoughts

Shownotes:

Link to minimumcd.org: https://minimumcd.org/
Reference to Brian Finster as instigator of Minimum CD
Reference to Raymond Chen's blog about Windows backward compatibility
Reference to previous episode on trunk-based development
Reference to interviews with Philip Johnston from Embedded Artistry
Reference to interview with Francois from Mend Fault
Link to Agile Embedded Slack group

You can find Jeff at https://jeffgable.com.
You can find Luca at https://luca.engineer.

Want to join the agile Embedded Slack? Click here

In this episode, Jeff and Luca discuss trunk-based development, a software development practice where developers merge their work into the main branch (trunk) frequently - at least daily. They explain how this approach differs from traditional branching models like GitFlow, and address common objections and concerns. The hosts emphasize that while trunk-based development may seem risky, it actually reduces risk by exposing integration problems early and forcing teams to implement good engineering practices like automated testing and feature flags.

The discussion highlights how trunk-based development acts as a "forcing function" that encourages better development practices, smaller changes, and more frequent collaboration between team members. They explain that while this approach originated in web development, it's equally applicable to embedded systems. The hosts cite research from the book "Accelerate" showing that trunk-based development is a predictor of high-performing software teams.

The episode concludes by emphasizing that most objections to trunk-based development actually point to underlying process issues that need to be addressed, and that the benefits of early integration and feedback outweigh the perceived downsides.

Timestamps:
00:00:00 - Introduction and topic overview
00:03:00 - Basic version control concepts and branching
00:08:00 - Definition and principles of trunk-based development
00:13:00 - Feature flags explanation and implementation
00:20:00 - Common objections to trunk-based development
00:27:00 - Application to embedded systems
00:34:00 - Benefits of trunk-based development
00:40:00 - Impact on team dynamics and collaboration
00:47:00 - Research backing and evidence from "Accelerate"

Shownotes / Links:
MinimumCD: https://minimumcd.org/
Accelerate: https://www.goodreads.com/book/show/35747076-accelerate
The nvie branching model: https://nvie.com/posts/a-successful-git-branching-model/

You can find Jeff at https://jeffgable.com.
You can find Luca at https://luca.engineer.

Want to join the agile Embedded Slack? Click here

The Dangers of Magic in Software Abstractions

In this episode of the Agile Embedded Podcast, hosts Jeff Gable and Luca Ingianni discuss the pitfalls of relying on 'magic' in software development, particularly in tooling, build systems, and frameworks. They illuminate how seemingly helpful abstractions can become burdensome when they obscure necessary complexities or become hard to debug. Through various examples, including comparisons of Arduino and CMake, they explore what makes a good abstraction versus a bad one. The conversation also delves into the importance of composability, testability, and the real-world impact of overly simplistic or 'magical' solutions in long-term projects. The episode wraps up with practical advice for selecting and designing more effective coding abstractions to mitigate risks.

00:00 Introduction and Episode Theme
00:19 The Problem with Magic in Programming
02:54 Examples of Good and Bad Abstractions
04:45 Frameworks and Tools: Arduino vs. CMake
08:10 Vendor-Supplied Tools and Their Pitfalls
13:05 Characteristics of Good Abstractions
29:04 Testing and Abstractions
37:10 Final Thoughts and Contact Information

You can find Jeff at https://jeffgable.com.
You can find Luca at https://luca.engineer.

Want to join the agile Embedded Slack? Click here

## Key Topics

[00:30] Introduction to the listener's question about repository granularity in embedded development

[01:15] The listener's approach: separate repositories for different work products in safety-critical industries

[03:20] Luca's initial reaction and concerns about over-complication

[05:45] Discussion of monorepo approaches and configuration management

[08:10] The concept of micro-repositories and parallels to microservices

[11:30] Using feature flags and CI pipelines instead of repository separation

## Notable Quotes

> "You're splitting something which ought to be joined together into different repositories and hiding whatever is happening within the repositories from the different users, from the different developers." — Luca Ingianni

> "The risk of course is that you will not spot divergence early enough because people just don't merge because it's a chore and because things might break, and of course that is the point - the earlier you notice that something breaks, the easier it will be to fix it." — Luca Ingianni

> "I'm willing to guarantee that you're going to get the architecture wrong at least on the first try. You think you're being really smart and you cut it up into a bunch of microservices or micro-repositories, and you're just going to get the boundaries wrong." — Luca Ingianni

> "I would opt for fewer repositories and rather do configuration management within the repositories as opposed to between repositories. Use feature flags, use tagging, use whatever you want to insulate changes that might be breaking from the rest of the code base." — Luca Ingianni

## Resources Mentioned

John Taylor's Embedded Project Cookbook - A resource mentioned by the listener that discusses sequential events in embedded projects

Trunk-Based Development - Development methodology discussed throughout the episode

Minimum CD Podcast - Previous podcast episode referenced by the listener

You can find Jeff at https://jeffgable.com.
You can find Luca at https://luca.engineer.

Want to join the agile Embedded Slack? Click here