Deployment Patterns in a ML project

Ali Issa
2 min readFeb 7, 2024
Generated by DALL-E 3

I recently encountered some deployment patterns that Iโ€™d like to share:

๐’๐ก๐š๐๐จ๐ฐ ๐Œ๐จ๐๐ž ๐ƒ๐ž๐ฉ๐ฅ๐จ๐ฒ๐ฆ๐ž๐ง๐ญ: This involves running a machine learning system in parallel with the current system, which could be a human or another AI model. The Human or the initial model will be used to process the given input, not the new ML model. However, we monitor how the model behaves for every given input to assess its performance.

The purpose of this shadow mode is to understand the modelโ€™s behaviour and accuracy. Then, we decide if we want to rely on the learning algorithm. This is an effective way to prevent the integration of an ML model that could potentially have issues. By having it work in the background and monitoring the results, we can avoid such problems.

๐‚๐š๐ง๐š๐ซ๐ฒ ๐ƒ๐ž๐ฉ๐ฅ๐จ๐ฒ๐ฆ๐ž๐ง๐ญ: We expose the model to a small fraction of the systemโ€™s traffic. The model will make decisions for each input coming from this small traffic segment. This technique allows us to spot issues and problems before they affect a large traffic volume. If the model is used for a wide range of users and a problem arises, this method can prevent significant issues. We can gradually increase the traffic when we notice that the model is performing well on a small portion of traffic.

๐๐ฅ๐ฎ๐ž-๐†๐ซ๐ž๐ž๐ง ๐ƒ๐ž๐ฉ๐ฅ๐จ๐ฒ๐ฆ๐ž๐ง๐ญ: Letโ€™s say we have two models, A and B. A is the new model, and B is the older one. In a given system, when an input is received from the user, the decisions are made by model B (the old version). When we want to use the new model, we suddenly switch to the newest model A (using a router). We can also rollback to the older version, which is model B. The advantage of this method is easy rollback. In case the new version is not performing well, we can rollback to the older version using the router.

If you like what you see, hit the follow button! You can also find me on LinkedIn, and we can follow each other there too.

--

--