Airflow 3 on EKS 핵심 아키텍처

Airflow 3 on EKS 핵심 아키텍처 An architecture diagram generated by Archify. DAG bundle / code · Processor + worker 코드 · Airflow 3: logical components DAG bundle / code Processor + worker 코드 DAG processor · 파싱·직렬화 · Airflow 3: logical components DAG processor 파싱·직렬화 Metadata DB · PostgreSQL / MySQL · Airflow 3: logical components Metadata DB PostgreSQL / MySQL Triggerer (optional) · Async triggers · Airflow 3: logical components Triggerer (optional) Async triggers API server · UI / REST / Execution API · Airflow 3: logical components API server UI / REST / Execution API Scheduler + executor · 상태 관리·task 제출 · Airflow 3: logical components Scheduler + executor 상태 관리·task 제출 Kubernetes task pod · Task SDK + supervisor · Airflow 3: logical components Kubernetes task pod Task SDK + supervisor Celery worker pool · Task SDK + supervisor · Airflow 3: logical components Celery worker pool Task SDK + supervisor Celery broker · Redis 등 선택 · Airflow 3: logical components Celery broker Redis 등 선택 DAG code serialize trigger state state R/W DAG / state R/W queue tasks task delivery Execution API Execution API Kubernetes API: create task pod Airflow 3: logical components Legend Frontend 백엔드 서비스 데이터 저장소 메시지 버스