Once a model has been trained that has a high model quality, the next step is to ask how the trained machine learning model can be used productively.
The challenges here include the automation of model training, the productive implementation of the model and monitoring.
This blog article explains how these challenges can be solved and which steps need to be taken to operationalize a machine learning solution. An example of an architecture for the operationalization of machine learning models based on the MLOps approach is presented.
It also looks at how the results can ultimately be made usable for end users.
Steps to operationalize machine learning models
The start of a data science project is an experimental phase. In this phase, the focus is on using exploratory data analysis to gain insights or correlations from the data and then training a model that serves the defined business objective.
Traditionally, exploratory data analysis and initial data processing and modeling take place in Jupyter notebooks. In addition to code, these contain markdowns, which are textual areas and are used to document the findings.
To operationalize a machine learning model, this initial manual workflow of model development must be automated. To automate this process or for continuous model development, it is necessary to transfer the code into a machine learning pipeline. A machine learning pipeline is a sequence of steps. The individual steps each carry out a task, such as data supply, data cleansing or model training.
The figure below shows an example of a machine learning pipeline for model development, which is summarized and described in detail below:
1. data supply
The first step is to set up interfaces to the source systems in which the data for training and testing a machine learning model is stored. There are two ways of making the data available for modeling. The first option is virtual live access to the data of a source system.
In this case, the data remains in the source system and is not physically transferred to the machine learning environment. The second option is to import the data from the source system into the machine learning environment. The advantage of this variant is the possibility of data versioning.
2. data processing and cleansing
- Removal of duplicates or incorrect data
- Remove or fill in missing data with statistical values (e.g. average)
- Transferring data to another format
- Generating new variables
- Aggregation of data
- Separation of data
3. model training
The third step contains the code for training the model. After dividing the data into training and test data sets, the variables and the machine learning algorithm are selected. A machine learning model is then trained on the basis of the training data, the variable selection and the selected machine learning algorithm.
For some algorithms, hyperparameters, which are parameters for model training, must first be defined. To do this, an algorithm is usually used to search for the optimum hyperparameters.
4. model evaluation
In the next step, the trained model is then evaluated using the test data. This means that forecasts are determined for the test data set using the model and these are compared with the actual values in order to assess the forecasting quality of the trained model.
5. model registration
In the final step of the machine learning pipeline, the model is saved in a model registry. This also involves versioning the model.
Machine learning ops approach
The automated execution of the machine learning pipeline can be data-based through the availability of new or changed data in the source system or time-based through scheduling. In addition to the execution of machine learning pipelines, the automation of the publication of machine learning pipelines after adjustments as well as the automation of the go-live and monitoring of machine learning solutions are further topics in operationalization. The MLOps approach has established itself in this context.
Machine Learning Ops (MLOps) is the application of the DevOpsWhat is DevOps? The term DevOps is made up of the words... More concept from software development to machine learning. In the context of MLOps, so-called MLOps pipelines are defined to automate the tasks described above. For example, an MLOps pipeline can be created for the automated, continuous publishing of the machine learning pipeline. This MLOps pipeline is triggered by changes to the machine learning pipeline and first leads to automated tests, such as unit tests and code quality tests. If the tests are successful or the changes are error-free, the code of the machine learning pipeline is automatically pushed to Git and the latest version of the training pipeline is published in the machine learning environment. This new version can then be used for model development.
Furthermore, an MLOps pipeline can be created for the automated, continuous deployment of a machine learning model. This pipeline can be triggered, for example, by the availability of a new model with a better forecast quality. The forecast quality is determined using a previously defined metric, such as the mean absolute forecast error. In a 2-system landscape, the model is first deployed and tested in the test environment, usually as a web service. If the tests are successful, the model is deployed in the production environment and also tested there. After successful testing, a productive machine learning model that provides forecasts is finally available.
Authentication and monitoring
An authentication method should be set up for the production machine learning model to prevent unauthorized access. If the model has been deployed as a web service, token-based or key-based authentication is usually used.
To ensure that failures of the productive machine learning model are avoided or downtimes are kept to a minimum, a service should be used to monitor the availability and utilization of the productive machine learning model. It can also be useful to monitor other parts of the architecture. For example, data drift, i.e. a significant change in the source data, can lead to a dramatic deterioration in the predictive quality of a machine learning model for new data. In many cases, it therefore makes sense to use a service that checks the data from the source systems for data drift.
The following is an example of an MLOps architecture for the operationalization of machine learning models.
Use of the results
As a rule, forecasts do not lead to automated decisions due to compliance in the company, but support the employees of a company in the decision-making process. For this reason, the forecasts generated by a machine learning model are usually made available to employees.
The results or forecasts can be made available to the end users from the specialist departments either by importing the forecasts back into an existing system or by creating a new application.
Depending on the requirements, the forecasts are generated in real time or in batch at specific times and transmitted to the target system. Real-time means that new forecasts are triggered directly when forecast-relevant input data changes or new forecast-relevant data is available in the source system.
In the classic deployment scenario, a machine learning model is deployed as a web service, e.g. in a Kubernetes cluster. HTTP requests, which provide the input arguments, are then used to request predictions from a target system.
Scenarios can also be implemented in which the machine learning model pulls the latest forecast-relevant input data and transmits forecasts to the target system (pull mechanism).
Summary
The MLOps approach is best practice for the operationalization of machine learning models. This approach meets the requirements for continuous integration as well as continuous training and deployment. A central step is the transfer of the code for training a model into a machine learning pipeline.
MLOps pipelines must then be defined to automate the execution and publication of the machine learning pipeline and to automate the productive implementation of the model. Further operationalization steps include setting up monitoring and an authentication method for the productive machine learning model. The final phase of operationalization involves making the results available to the end user and embedding the results in decision-making processes.
Many large software providers, including Microsoft, Google, AWS and SAP, offer solutions for the operationalization of machine learning models.
Author: Alexandros Xiagakis
CONTACT
Bernd Themann
Managing Consultant
SAP Information Management
