Machine Learning Workflow

9-step machine learning workflow at Microsoft

9 steps:

1. Model requirements: designers decide which features are feasible to implement with machine learning and which can be useful for a given existing product or for a new one. Most importantly, in this stage, they also decide what types of models are most appropriate for the given problem.
2. Data collection: the teams look for and integrate available datasets (e.g., internal or open source) or collect their own.
3. Data cleaning: the teams remove inaccurate or noisy records from the data, a common activity to all forms of data science.
4. Data labelling: assigns ground truth labels to each record. Labels can be provided either by engineers themselves, domain experts, or by crowd workers in online crowd-sourcing platforms.
5. Feature engineering: all activities that are performed to extract and select informative features for machine learning models.
6. Model training: the chosen models (using the selected features) are trained and tuned on clean, collected data and their respective labels.
7. Model evaluation: the engineers evaluate the output model on tested or safeguard datasets using pre-defined metrics.
8. Model deployment: the inference code of the model is then deployed on the targeted devices
9. Model monitoring: the deployed code is continously monitored for possible errors during real-world execution

Source: Software Engineering for Machine Learning: A Case Study, Proceedings - 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice, ICSE-SEIP 2019