Native per job

[kinderyj]

Note:
We can hold off this PR and think it through, because the feature flink native setup session cluster on k8s is weak in Apache Flink 1.10, the feature flink native setup per job cluster on k8s has not be released in Apache Flink 1.10.

Prerequisites
Version >= 1.11 of Apache Flink
hadoopConfig of your hadoop cluster should be in the FlinkCluster Spec, especially for native per job cluster.

This PR support both the flink native setup session cluster on k8s and flink native setup per job cluster on k8s.

The flink native setp session cluster on cluster has been supported in PR

The flink native setup per job cluster on k8s needs the native per job feature of Apache Flink, which will be(maybe) relased in Apache Flink1.11.

usage:

cat flinkoperator_v1beta1_flinknativejobcluster.yaml

apiVersion: flinkoperator.k8s.io/v1beta1
kind: FlinkCluster
metadata:
  name: native-flinkjobcluster-sample
spec:
  image:
    name: flink1.11
    pullPolicy: IfNotPresent
  nativeJobClusterJob:
    flinkClusterID: native-flinksessioncluster-sample
    jarFile: /opt/flink/examples/streaming/WordCount.jar
  hadoopConfig:
    configMapName: hadoop
    mountPath: /etc/hadoop

kubectl create -f flinkoperator_v1beta1_flinknativejobcluster.yaml

kubectl get pod
NAME                                                             READY   STATUS      RESTARTS   AGE
native-flinkjobcluster-sample-5765b5b695-6lb56                   1/1     Running     0          21s
native-flinkjobcluster-sample-native-per-job-cluster-job-slwjl   0/1     Completed   0          24s
native-flinkjobcluster-sample-taskmanager-1-1                    1/1     Running     0          12s

kubectl get svc
NAME                                 TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)             AGE
native-flinkjobcluster-sample        ClusterIP      None            <none>           6123/TCP,6124/TCP   26s
native-flinkjobcluster-sample-rest   LoadBalancer   172.16.253.27   ****   8081:31390/TCP      26s

The job will be started and the flink kernel will talk with k8s api server to request taskmanager on-demand. After the job is done, all the resource(jobmanager svc, jobmanager deployment, taskmanager) will be deleted by the flink kernel automatically.

[functicons]

/gcbrun

[kinderyj]

Finally, the Native per job clustet in the Flink 1.11 is desined in another way, which is called "application" mode, it's not suit for this PR. So I will close thie PR.