In order for analytics and machine learning projects to be useful, they require a high degree of data quality. To ensure that your pipelines are healthy you need a way to make them observable. In this episode Barr Moses and Lior Gavish, co-founders of Monte Carlo, share the leading causes of what they refer to as data downtime and how it manifests. They also discuss methods for gaining visibility into the flow of data through your infrastructure, how to diagnose and prevent potential problems, and what they are building at Monte Carlo to help you maintain your data’s uptime.
- How did you get involved in the area of data management?
- How did you come up with the idea to found Monte Carlo?
- What is “data downtime”?
- Can you start by giving your definition of observability in the context of data workflows?
- What are some of the contributing factors that lead to poor data quality at the different stages of the lifecycle?
- Monitoring and observability of infrastructure and software applications is a well understood problem. In what ways does observability of data applications differ from “traditional” software systems?
- What are some of the metrics or signals that we should be looking at to identify problems in our data applications?
- Why is this the year that so many companies are working to address the issue of data quality and observability?
- How are you addressing the challenge of bringing observability to data platforms at Monte Carlo?
- What are the areas of integration that you are targeting and how did you identify where to prioritize your efforts?
- For someone who is using Monte Carlo, how does the platform help them to identify and resolve issues in their data?
- What stage of the data lifecycle have you found to be the biggest contributor to downtime and quality issues?
- What are the most challenging systems, platforms, or tool chains to gain visibility into?
- What are some of the most interesting, innovative, or unexpected ways that you have seen teams address their observability needs?
- What are the most interesting, unexpected, or challenging lessons that you have learned while building the business and technology of Monte Carlo?
- What are the alternatives to Monte Carlo?
- What do you have planned for the future of the platform?
This article has been published from a wire agency feed without modifications to the text. Only the headline has been changed.