Flink Native-mode control flow

Flink Native-mode control flow An architecture diagram generated by Archify. Operator · kubectl apply · Architecture component Operator kubectl apply Kubernetes API Server · kube-apiserver · EKS cluster Kubernetes API Server kube-apiserver Flink Kubernetes Operator · watches CRs · reconcile loop · EKS cluster Flink Kubernetes Operator watches CRs · reconcile loop JobManager Pod · embeds the ResourceManager · EKS cluster › Flink cluster (FlinkDeployment) JobManager Pod embeds the ResourceManager TaskManager Pod 1 · task slots · runs subtasks · EKS cluster › Flink cluster (FlinkDeployment) TaskManager Pod 1 task slots · runs subtasks TaskManager Pod 2 · task slots · runs subtasks · EKS cluster › Flink cluster (FlinkDeployment) TaskManager Pod 2 task slots · runs subtasks apply FlinkDeployment / FlinkSessionJob CR watch CR changes create / reconcile ResourceManager: request · release TaskManager pods subtasks · checkpoints subtasks · checkpoints EKS cluster Flink cluster (FlinkDeployment)

Flink Kubernetes Operator

  • • Supports Native and Standalone
  • • Manages cluster/job lifecycle and upgrade conditions

JobManager

  • • Graph preparation depends on submission mode
  • • Native ResourceManager uses the Kubernetes API

TaskManager

  • • Slot sharing and operator chaining
  • • Pod allocation and node capacity are separate layers