Data Engineering Explained: Data Serialization, backward and forward compatibility

This article belongs to my “Data Engineering Explained” series on Medium, in which I share with you the following:

• Fundamentals of Data Engineering
• Buzzwords demystified and explained
• Best practices of Data Engineering
• Hands-on Tutorials
• Interviews with Data Engineering professionals

Introduction

In today’s article, we will talk about several formats for encoding Data and discuss how those formats impact the way your write and update code.

Forward and Backward compatibility

Applications are continuously getting updated over time, features get added or modified and user requirements evolve or are better understood. Change of features / requirements implies a change in the data stored by your application. New fields may need to be captured, data might require to be represented differently. Those changes mean that old and new versions of the code and data formats may potentially coexist in your system at the…