Log Analysis and Visualization Pipeline

Log Analysis and Visualization Pipeline An architecture diagram generated by Archify. Control Plane · AWS Service Logs · sent directly to CloudWatch · Log Sources Control Plane · AWS Service Logs sent directly to CloudWatch Container · Application Logs · worker node stdout/stderr · Log Sources Container · Application Logs worker node stdout/stderr Log Collectors · Fluent Bit · Fluentd · Vector · CloudWatch Agent · Log Collection Log Collectors Fluent Bit · Fluentd · Vector · CloudWatch Agent CloudWatch Logs · Fluent Bit · CloudWatch Agent · Log Storage CloudWatch Logs Fluent Bit · CloudWatch Agent Amazon OpenSearch · Fluent Bit · Vector · Log Storage Amazon OpenSearch Fluent Bit · Vector Amazon S3 · Fluent Bit · long-term archive · Log Storage Amazon S3 Fluent Bit · long-term archive Grafana Loki · Fluentd · Log Storage Grafana Loki Fluentd CloudWatch Logs Insights · queries → custom dashboards · Log Analysis · Visualization CloudWatch Logs Insights queries → custom dashboards OpenSearch Dashboards · OpenSearch Visualizations · Log Analysis · Visualization OpenSearch Dashboards OpenSearch Visualizations Amazon Athena · SQL queries → QuickSight · Log Analysis · Visualization Amazon Athena SQL queries → QuickSight Grafana Explore · Grafana Dashboards · Log Analysis · Visualization Grafana Explore Grafana Dashboards direct Log Sources Log Collection Log Storage Log Analysis · Visualization Legend Frontend Backend Database Cloud External