You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Next »

Motivation

Current wrappers such as standalone (JVM) or distributed (Flink) already allow us to develop new processors in the given runtime environment. More and more people from the community ask to also support Python based processors. Especially, data scientists are likely to use this.

Why Python wrapper?

* Python is a widely used language especially in the domain of data science
* Python is more concise and thus better to read
* We provide more options for standalone algorithms: It allows newcomers unfamiliar with Java to faster implement their algorithms

POC

We implemented a rudimentary prototype that involves a hybrid solution using the Java side for model declaration and registration as well as the Python side, where the actual logic is implement. The two programs can be deployed via Docker using supervisord to start two services in a Docker container. Java talks to Python via REST while Python is exposing Flask endpoints for starting and stopping processor threads.


  


  • No labels