Kubernetes 위 컴포넌트 구성도

Kubernetes 위 컴포넌트 구성도 An architecture diagram generated by Archify. DAG bundle / code · Processor + worker 코드 · Airflow 3: logical components DAG bundle / code Processor + worker 코드 DAG processor · 파싱·직렬화 · Airflow 3: logical components DAG processor 파싱·직렬화 Metadata DB · PostgreSQL / MySQL · Airflow 3: logical components Metadata DB PostgreSQL / MySQL Triggerer (optional) · Async triggers · Airflow 3: logical components Triggerer (optional) Async triggers API server · UI / REST / Execution API · Airflow 3: logical components API server UI / REST / Execution API Scheduler + executor · 상태 관리·task 제출 · Airflow 3: logical components Scheduler + executor 상태 관리·task 제출 Kubernetes task pod · Task SDK + supervisor · Airflow 3: logical components Kubernetes task pod Task SDK + supervisor Celery worker pool · Task SDK + supervisor · Airflow 3: logical components Celery worker pool Task SDK + supervisor Celery broker · Redis 등 선택 · Airflow 3: logical components Celery broker Redis 등 선택 DAG code serialize trigger state state R/W DAG / state R/W queue tasks task delivery Execution API Execution API Kubernetes API: create task pod Airflow 3: logical components Legend 프런트엔드 백엔드 서비스 데이터 저장소 메시지 버스

DAG code

  • • Processor와 worker에 bundle/code 필요
  • • Versioning은 bundle에 따라 다름

Deployment choices

  • • DB/broker·replica 종류는 구성에 따름
  • • 유휴 control plane·노드 비용은 남음