Testing adventures in the data engineering landscape
Building a data platform from the ground up is an absolute privilege, but challenges abound. Throw in unique data platform requirements such as the need to ensure data reliability, avoid data downtime, build it to scale and remain flexible enough to grow with the company, and doing it all while you don’t yet have external customers is truly something else. Traditional software testing only gets you so far in the data engineering landscape before data quality assurance and observability come into play and that is where the adventures begin.