L6. Centralized Logging & Analysis: CloudWatch, Athena & Security Lake

Real-Time Alerting with CloudWatch

CloudWatch Logs centralizes log aggregation from AWS services and installed agents into log groups you can search and retain. On top of raw logs, metric filters turn matching log patterns into numeric CloudWatch metrics. For example, a metric filter that matches the string "AccessDenied" converts every occurrence into a data point on a metric. A CloudWatch Alarm watching that metric can then notify your team, typically via SNS, within minutes of a spike, giving you near-real-time detection without building a separate alerting pipeline.

Ad Hoc SQL Over Massive Log Archives

Amazon Athena runs standard SQL directly against log files sitting in S3, including CloudTrail logs, VPC Flow Logs, and ALB access logs, without provisioning any servers. When you need to answer a one-off historical question, such as "which IAM principal called this API last quarter", across terabytes of archived logs, Athena is the cost-effective choice: you pay per query, and there is no cluster to manage or keep running.

Normalizing Security Telemetry Across the Board

AWS Security Lake centralizes security telemetry from AWS services, on-premises sources, and supported third-party security tools into a single, customer-owned, S3-based data lake. Its defining feature is normalization: every source is converted into the Open Cybersecurity Schema Framework (OCSF), a common schema, so a single query language and a single set of detection logic works consistently across sources that would otherwise each have their own log format.

Security Lake uses AWS Lake Formation permissions to grant subscribers, such as a SIEM platform or an analytics service, the ability to query the centralized data directly. Subscribers read data in place; nothing needs to be copied or duplicated out to each consuming tool.

Balancing Investigation Speed Against Cost

High-volume raw logs like VPC Flow Logs are expensive to keep in hot storage indefinitely. The common pattern is a lifecycle policy: keep recent logs in CloudWatch Logs or S3 Standard for an initial investigation window, when queries need to be fast, then transition older logs to S3 Glacier once that window closes. This balances the operational need for quick access to recent activity against the long-term cost of retaining years of raw traffic data.

Exam tip: If a scenario mentions querying data from many different source types (AWS services, on-premises, third-party tools) in one consistent schema, that is Security Lake and OCSF. If it mentions a single one-off SQL query against S3 log files, that is Athena.