I recently encountered some deployment patterns that Iโd like to share:
๐๐ก๐๐๐จ๐ฐ ๐๐จ๐๐ ๐๐๐ฉ๐ฅ๐จ๐ฒ๐ฆ๐๐ง๐ญ: This involves running a machine learning system in parallel with the current system, which could be a human or another AI model. The Human or the initial model will be used to process the given input, not the new ML model. However, we monitor how the model behaves for every given input to assess its performance.
The purpose of this shadow mode is to understand the modelโs behaviour and accuracy. Then, we decide if we want to rely on the learning algorithm. This is an effective way to prevent the integration of an ML model that could potentially have issues. By having it work in the background and monitoring the results, we can avoid such problems.
๐๐๐ง๐๐ซ๐ฒ ๐๐๐ฉ๐ฅ๐จ๐ฒ๐ฆ๐๐ง๐ญ: We expose the model to a small fraction of the systemโs traffic. The model will make decisions for each input coming from this small traffic segment. This technique allows us to spot issues and problems before they affect a large traffic volume. If the model is used for a wide range of users and a problem arises, this method can prevent significant issues. We can gradually increase the traffic when we notice that the model is performing well on a small portion of traffic.
๐๐ฅ๐ฎ๐-๐๐ซ๐๐๐ง ๐๐๐ฉ๐ฅ๐จ๐ฒ๐ฆ๐๐ง๐ญ: Letโs say we have two models, A and B. A is the new model, and B is the older one. In a given system, when an input is received from the user, the decisions are made by model B (the old version). When we want to use the new model, we suddenly switch to the newest model A (using a router). We can also rollback to the older version, which is model B. The advantage of this method is easy rollback. In case the new version is not performing well, we can rollback to the older version using the router.
If you like what you see, hit the follow button! You can also find me on LinkedIn, and we can follow each other there too.
