v1.8.1 — Alert-Triggered AI Diagnosis + Major Docs Expansion

April 23, 2026 · 6 min read

A new pipeline receives events from external alerting systems and handles correlation → automatic AI root cause analysis → Slack notification end-to-end. Alongside it, 18 new ADRs, 5 runbooks, 11 module CLAUDE.md files, and 2 new Web guide pages (KO+EN) significantly expand the project documentation.

Key Changes

Alert-Triggered AI Diagnosis (ADR-009) · 4 Webhook Sources (CloudWatch/Alertmanager/Grafana/Generic) · HMAC Authentication · Slack Block Kit · 18 New ADRs · 5 New Runbooks · Web Guide Expansion

Alert-Triggered AI Diagnosis (ADR-009)

When a critical incident occurs, a scoped AI diagnosis runs automatically.

Pipeline Flow

[CloudWatch Alarm] → SNS Topic → SQS Queue
                                    ↓
                            [EC2 Poller (15s)]
                                    ↓
[Alertmanager/Grafana/Generic] → POST /api/alert-webhook
                                    ↓
                            [alert-correlation.ts]
                                    ↓
                         [alert-diagnosis.ts (AI)]
                                    ↓
                          [Slack/SNS Dispatch]

Supported Sources

Source	Transport	Normalization
CloudWatch Alarms	SNS → SQS → EC2 polling	CloudWatch event
Prometheus Alertmanager	Direct webhook (HMAC)	Alertmanager v4
Grafana Alerting	Direct webhook (HMAC)	Grafana unified
Generic JSON	Direct webhook (HMAC)	Custom schema

Correlation Engine

src/lib/alert-correlation.ts groups individual alerts into incidents:

Criterion	Default	Description
Time window	5 min	Merge alerts from same service within 5 min
Common service	≥1 match	`labels.service` or `resource` match
Common namespace	≥1 match	K8s `labels.namespace` match
Deduplication	1 min	Same `fingerprint` suppressed for 1 min
Severity escalation	3 warnings/5 min	→ promote to critical

Scoped Auto-Diagnosis

Unlike the full 15-section diagnosis, this runs only within the firing alert's scope and completes in 1–2 minutes:

AlertContext build — extract affected services/resources/namespaces + since timestamp
Scoped collection — CloudWatch metrics filtered to ±10 min around since and matched resources only
Section selection — run 3–5 of the 15 sections (Compute / Network / Container)
Change detection — compare against recent Terraform state / CloudTrail
Bedrock Sonnet analysis — root cause hypothesis + next steps

// src/lib/alert-diagnosis.ts (core interface)
interface AlertContext {
  services: string[];
  resources: string[];
  namespaces: string[];
  since: Date;     // firing time − 10 min
  until: Date;     // firing time + 10 min
}

Slack Notifications (Block Kit)

Severity drives channel routing and thread updates:

Severity	Default Channel	Color
`critical`	`#incidents`	🔴 red
`warning`	`#alerts`	🟠 orange
`info`	`#alerts-low`	🔵 blue

Initial alert is the main message; follow-ups (merged alerts, AI diagnosis results, resolution) post as replies in the same thread
Threading works in both Webhook and Bot Token modes (thread_ts is cached)
CloudWatch OK / Alertmanager resolved → ✅ resolution reply

HMAC Authentication

Shared secret is stored in alertWebhookSecret in data/config.json:

curl -X POST https://awsops.example.com/awsops/api/alert-webhook \
  -H 'X-Alert-Source: generic' \
  -H "X-Signature-256: sha256=$(printf '%s' "$BODY" | openssl dgst -sha256 -hmac "$SECRET" | awk '{print $2}')" \
  -H 'Content-Type: application/json' \
  -d "$BODY"

Alert Knowledge Base

Diagnosis records persist under data/alert-diagnosis/:

File	Content
`incidents/<id>.json`	Individual incident + AI diagnosis result
`summary-<YYYY-MM>.json`	Monthly stats (top services, alert names, resolution time)

The UI Knowledge Base tab searches past similar incidents, and new incidents get similarity-based recommendations automatically.

Major Documentation Expansion

18 New ADRs (011–028)

Design decisions behind features added in v1.8.0 are now formalized:

ADR	Topic
ADR-011	External datasource integration (SSRF defense + allowlist)
ADR-012	SNS notification strategy
ADR-013	Auto-collect investigation agents
ADR-014	Report generation and scheduling
ADR-015	AI routing priority
ADR-016	Bedrock model selection strategy
ADR-017	Cache warmer design
ADR-018	Cognito auth flow
ADR-019	SSE streaming
ADR-020	HMAC webhook authentication
ADR-021	Admin email permission model
ADR-022	CDK stack split
ADR-023	Multi-route parallel execution
ADR-024	i18n (ko/en) support
ADR-025	Code interpreter sandbox
ADR-026	CloudFront + Lambda@Edge
ADR-027	OpenCost + Prometheus (EKS cost)
ADR-028	Memory store (conversation history)

5 New Runbooks

Practical troubleshooting/setup guides for operational scenarios:

alert-pipeline-troubleshoot.md — webhooks not arriving, correlation not forming incidents, Slack silent
cache-warmer-operations.md — debug cache warmer, add/remove queries
cognito-auth-troubleshoot.md — Lambda@Edge, HttpOnly cookies, OAuth2 callback issues
deploy-flow.md — operational manual for the 11-step deployment scripts
multi-account-setup.md — Aggregator config, cross-account IAM role, verification

11 New Module CLAUDE.md Files

Each source module now documents its role/rules/key files as Claude context:

Path	Purpose
`docs/CLAUDE.md`	Documentation structure
`runbooks/CLAUDE.md`	Runbook index
`decisions/CLAUDE.md`	ADR index
`agent/CLAUDE.md`	Strands Agent + 19 Lambda
`scripts/CLAUDE.md`	11-step deployment scripts
`tests/CLAUDE.md`	Vitest structure
`infra-cdk/CLAUDE.md`	CDK stack composition
`src/app/ai-diagnosis/CLAUDE.md`	AI diagnosis page
`src/app/alert-settings/CLAUDE.md`	Alert settings page
`src/app/k8s/CLAUDE.md`	K8s page
`src/lib/collectors/CLAUDE.md`	7 auto-collect agents

Web Guide Expansion

The Docusaurus guide now reflects v1.8 features:

monitoring/ai-diagnosis.md — 15-section diagnosis, export formats (DOCX/MD/PDF/PPTX), scheduling, cost control tips
monitoring/alerts.md — full alert pipeline flow, HMAC, correlation, Slack threading, knowledge base
intro.md — updated to 40 pages / 18 APIs / 11 AI routes, added "AI Diagnosis & Alert Pipeline" feature block
faq/troubleshooting.md — Slack alert failure / AI diagnosis partial failure cases
Both Korean (docs/) and English (i18n/en/docusaurus-plugin-content-docs/current/) versions updated

Bug Fix Highlights

Item	Description
Webpack dynamic import	Added `/* webpackInclude: /\.(ts\|tsx\|js)$/ */` magic comment to collector dynamic imports — fixes build failure when CLAUDE.md was pulled into the webpack context
Bedrock model ID	Alert diagnosis uses `global.anthropic.claude-sonnet-4-6`
Slack thread reuse	Webhook mode now persists `thread_ts` for follow-up replies
Report download	Uses proxy URL instead of S3 presigned URL — resolves 404 after STS session expiry
SNS email	Markdown stripped to plaintext before dispatch
Cache warmer	Monitoring queries excluded (CloudWatch FDW calls exhaust pg pool)
Silent cookie deletion	HttpOnly cookies require server-side deletion via `POST /api/auth`

Key File Changes

File	Change
`src/app/api/alert-webhook/route.ts`	4-source webhook receiver + HMAC verification + correlation trigger + active incidents GET
`src/app/api/notification/route.ts`	Slack Block Kit / SNS dispatch (severity channel routing, markdown→plaintext)
`src/lib/alert-types.ts`	Per-source normalizers (CloudWatch/Alertmanager/Grafana/Generic)
`src/lib/alert-correlation.ts`	Correlation engine (30s buffer, time/service/resource matching)
`src/lib/alert-diagnosis.ts`	Diagnosis orchestrator (strategy selection, parallel collectors, AlertContext scope)
`src/lib/alert-knowledge.ts`	Knowledge base (JSONL storage, monthly summary, similarity search)
`src/lib/slack-notification.ts`	Slack client (Bot Token/Webhook, thread updates)
`src/lib/alert-sqs-poller.ts`	SQS background poller (SNS→SQS→EC2, DLQ, rate limit)
`src/app/alert-settings/page.tsx`	Alert settings admin page
`docs/decisions/ADR-011~028.md`	18 new ADRs
`docs/runbooks/*.md`	5 new runbooks
`web/docs/monitoring/ai-diagnosis.md`	AI comprehensive diagnosis guide
`web/docs/monitoring/alerts.md`	Alert pipeline guide

Version Comparison

Item	v1.8.0	v1.8.1	Change
Alert pipeline	—	Webhook + correlation + AI diagnosis	NEW
Supported alert sources	—	4 (CloudWatch/Alertmanager/Grafana/Generic)	NEW
Slack notifications	—	Block Kit + thread updates	NEW
ADRs	10 (001–010)	28 (001–028)	+18
Runbooks	—	5	NEW
Module CLAUDE.md	7	18	+11
Web guide pages	39	41	+2 (ai-diagnosis, alerts)

References

ADR-009: Alert-triggered AI diagnosis
ADR-012: SNS notification strategy
ADR-013: Auto-collect investigation agents
ADR-020: HMAC webhook authentication

Alert-Triggered AI Diagnosis (ADR-009)​

Pipeline Flow​

Supported Sources​

Correlation Engine​

Scoped Auto-Diagnosis​

Slack Notifications (Block Kit)​

HMAC Authentication​

Alert Knowledge Base​

Major Documentation Expansion​

18 New ADRs (011–028)​

5 New Runbooks​

11 New Module CLAUDE.md Files​

Web Guide Expansion​

Bug Fix Highlights​

Key File Changes​

Version Comparison​

References​