Summary Ad-hoc methods can be used to build a machine learning model once, but if you ever want to update it and serve it in production, you’ll need a way to repeatedly carry out a challenging set of tasks. With the help of the orchestration engine Dagster, you can go beyond simple task-based representations of your dependencies. Dagster comprehends the data that it is manipulating.
In this episode, Sandy Ryza discusses how his experience with machine learning has influenced his work on the Dagster project and the tenets upon which it is based, enabling collaboration between data engineering and machine learning issues.
