Reproducibility Is Powerful

August 20, 2024

Many respected voices in our industry talk about the concept of iteration velocity as a key metric to optimize for.

This seems to be true not just for software but for everything we do in life. But, iterating outside of software is very hard. Imagine you're a painter, and you're working on your masterpiece. In order to get there, you probably made hundreds or thousands of paintings. In the painting you're making right now, you probably want to try out many different ideas, iterate on the lighting, iterate on the subjects of your painting. You want to maybe add different kinds of objects in there, but of course you are limited by physical things. If you make a mistake or go somewhere that doesn't yield a good result, it's very hard to go back. Iterating and experimenting comes at a high risk. As you get closer and closer to finishing your painting, the risk keeps geting higher and higher. In fact, once the painting is done, you cannot touch it anymore. It is done.

The key concept that enables iteration velocity is reproducibility, or “how much effort does it take to jump to a specific state in my iteration cycle“.

We developers know that the hardest part of fixing a bug is finding a minimal repro. Once the repro is there, then the bug is toast.

Software’s edge is that moving bytes is very cheap [compared to moving physical objects], and therefore building reproducible features is possible. Think of undo/redo, of copy and paste, or of Git. These things enable people to build and iterate on software fearlessly.

Undo/redo helps us instantly go back to a moment ago. Copy and paste lets us clone the current state of our work and start fresh without altering the original copy. Git lets us save full snapshots of large amounts of work, branch out and try new ideas, go back in time, merge changes, etc. These features make iteration easy because they help reproducing different states in a program easy.

Some newer examples of great reproducible software include:

  • Vercel’s immutable deployments. They leverage Git very well and therefore give you reproducible infrastructure. Very powerful for finding bugs, experimenting with different infra settings, and overall just having a robuts backend.

  • Neon branches. They’ve built this by themselves (so not leveraging Git), and it introduces a paradigm shift for databases. Developers can debug with production data without altering production, which is mind blowing. Backups are cheap, etc.

  • Figma branches. Maybe too try-hard, as iterating on existing Figma designs doesn’t involve risk of breaking prod or similar—but a very interesting feature anyways.

  • BaseHub branches (coming soon!). Built by ourselves, but very closely inspired by Git; working on a workflow that makes sense for devs and marketers alike (more about our vision here).

Software isn't just good for iterating on software. In fact, software ate the world because every physical activity now uses software to create simulated, reproducible environments that let people experiment and iterate faster. This is true for car manufacturing, architecture, golf simulation, music production, films, and virtually everything and anything. Software enables high iteration velocity.

﹡﹡﹡

So, what's the takeaway? Well, understanding the underlying properties that make a piece of software good is important for getting better at our craft. Oh, and also, it’s fun to think about concepts like iteration velocity and reproducibility outside of software—as they’re principles that apply to everything. So go on, and iterate to greatness!