Data Intensive Applications — Agile Code Evolution, Data Encoding

Introduction

Welcome to my fourth article in the series, Designing Data Intensive Applications. This article is highly inspired by one of the best Data Engineering books available out there: Designing Data Intensive Applications by Martin Kleppmann. In this chapter, we’re going to learn the secret behind how we can release new code into production systems without breaking anything by introducing bugs. We’ll also see how to make sure that the old clients work, thanks to the understanding of data encoding foundations

Note: I have received no compensation for writing this piece. Please consider supporting my and others’ writing by becoming a Medium member with this link.

Rolling Update

New software features are released using rolling updates. You typically have a load balancer which would direct a small percentage of traffic to the new code that we’re trying to deploy and that’s where we can validate if things are working fine. We call it the Canary, we basically want to test out if everything’s working fine before we roll it out to most of the users. This functionality is very critical because this allows us for smaller and more frequent upgrades. In contrast, a one-shot deployment of bigger changes at the same time can break everything…