The lifecycle of machine learning & artificial intelligence

Technology

Keywords defining the session:

- AI

- Machine learning

- Governance

Takeaway points of the session:

- The audience will come away having learned about the importance and concepts of machine learning lifecycle management, and how tools LeVar and MLflow can be used to manage experiments, track model development and evolution, and deploy machine learning and AI pipelines in production scenarios.

Description:

As machine learning (ML) and artificial intelligence (AI) have risen to become core to the business of many large organizations, it’s become clear that the art of delivering data products is somewhat broken. It’s long been recognized that data engineering ends up taking up an inordinate amount of time [1], but until recently lifecycle management (experiment tracking, model management, governance & lineage, and deployment automation) has been downplayed, which is unfortunate because it ties into business value, de-risks future forward projects, and contributes to velocity and developer happiness.

In this talk, I’ll (1) review the good parts of mature machine learning and AI methodologies and ML/AI lifecycle management, discussing model development and model evaluation methodologies (2) introduce and demo two machine learning lifecycle management tools — LeVar [2] and MLflow [3] — and (3) talk about how the use of tools like these can help make the the full machine learning and AI development lifecycle more efficient.

The audience will come away having learned about the importance and concepts of machine learning lifecycle management, and how tools LeVar and MLflow can be used to manage experiments, track model development and evolution, and deploy machine learning and AI pipelines in production scenarios.

1. https://ai.google/research/pubs/pub43146
2. https://github.com/levarml/levar
3. https://github.com/mlflow/mlflow