This guide explains the steps of how to run a Flink application on the BDE platform.
Apache Flink is an open-source platform for distributed stream and batch processing.
In this post, we are going to see how to launch a Flink demo app in minutes, thanks to the Apache Flink docker image prepackaged and ready-to-use within the BDE platform.
A typical Flink Cluster consists of a Flink master and one or several Flink workers. A user can run an application on the cluster by submitting its executable using the run command. Such application can be scripted in either Java, Scala or Python. We’ll see, once the program has been implemented, how should the source code be packaged and submitted to Flink cluster.
The BDE platform provides a Flink master and a Flink worker images to get a Flink cluster up and running. They are both extensions of the Flink baseimage: a simple image that just includes Flink. Next, there is a Flink submitimage available allowing to submit an application to the Flink cluster. To ease the job for the developer, we went one step further and extended the Flink submit image with a Flink Maven template.