Building Ensemble Models with Web Services on Microservice Architecture

Abstract

The combination of machine learning with web services is not rare, as it is a possible way to make the models reachable to other applications. For example a mobile or web application with recommendation feature can send requests to query the model’s prediction. The advantage of this method is that it does not require to use the same platform or programming language on the model and application side. This paper investigates the building of ensemble models with web services, in a complex microservice architecture based application. The ensemble models are special, because they rely on other pre-trained models, so they can act as a wrapper model. The advantage of this approach is that it is applicable to multiple models that are written in different programming languages. When we have these wrapper models, all of them can be accessed through web services, which leads to many small services that can be managed together in an application on microservice architecture. In this paper, we combine models from Scikit Learn, Tensorflow, Weka and Deeplearning4j libraries to show how models written in different languages can work together. We propose two similar architecture variants involving machine learning and microservices to combine models from different platforms. The gateway variant uses patterns like API gateway or backends for frontends, the direct variant uses direct access to web services. The integration of these into existing web applications is also presented considering the server or client side computing load. The analysis shows that both of them can be used, but with different software systems. The direct is preferred when the application partly relies on machine learning services and thus only using few of them, and the gateway is preferred when the application is dependent on these services.