L18. Analyzing Telemetry: KQL Queries, Distributed Tracing & Performance Metrics
Distributed tracing pinpoints which hop in a multi-service call chain caused a slowdown, and KQL turns millions of log rows into a focused, actionable result set that manual scanning never could.
Infrastructure Performance Indicators: Your Diagnostic Baseline
When an application slows down, the first question is: "Is this an application code problem or an infrastructure constraint?" The answer comes from monitoring four fundamental infrastructure signals:
| Signal | What It Measures | Diagnosis |
|---|---|---|
| CPU Utilization | Percentage of processor time consumed | High CPU means application is compute-bound or not parallelized well |
| Memory Usage | RAM consumed relative to available capacity | High memory means memory leaks, caching issues, or undersized VMs |
| Disk I/O and Latency | Read/write speed and access time to storage | High latency means storage is a bottleneck (slow database, network filesystem) |
| Network Throughput | Bandwidth consumed relative to available capacity | High throughput means network saturation or inefficient data transfer |
Correlating Usage Against Performance
A critical diagnostic pattern is correlating usage (request volume) against performance (response time and error rate) over the same time period. If response time degrades but request volume stays flat, the application is genuinely degrading independent of load. If response time and request volume both spike together, the slowdown is simply load-driven and the application is performing as designed.
Without this correlation, teams often blame code quality when the real issue is just traffic growth, leading to pointless refactoring.
Distributed Tracing: End-to-End Operation Visibility
In microservices architectures, a single user request touches multiple services. A front-end calls an API, which calls a database, which queries another service. If the overall operation is slow, which hop is responsible? Distributed tracing in Application Insights answers this by assigning a unique operation ID to a logical operation and collecting telemetry from every service that operation touches. All the telemetry (requests, dependencies, exceptions) for that one operation are correlated and displayed as an end-to-end transaction view. You see exactly which downstream call caused the slowdown or failure, not just that something is slow.
Kusto Query Language (KQL) Fundamentals
At production scale, millions of log rows flow into Log Analytics every day. Manually scanning raw logs is impractical. Instead, you write KQL queries to aggregate, filter, and correlate across those millions of rows into a focused result set.
KQL is a pipe-based language. The pipe operator (|) chains operations:
requests
| where timestamp > ago(1h)
| where success == false
| summarize count() by resultCodeThis query filters requests from the last hour, then to failed requests only, then groups and counts by result code.
Essential KQL Operators
The AZ-400 exam tests recognizing what a given basic KQL query does, not writing complex queries from scratch. Know these:
where: Filter rows (like SQL WHERE)summarize: Aggregate (count, sum, avg, max, min)project: Select specific columnsorder by: Sort results (ascending or descending)join: Combine data from two tables on a shared key
- ✓CPU, memory, disk, and network are the baseline infrastructure performance indicators used to determine whether an infrastructure constraint, rather than application code, is causing a performance problem
- ✓Distributed tracing in Application Insights correlates all telemetry for one logical operation across multiple services using an operation ID, producing an end-to-end transaction view that isolated per-service logs cannot show on their own
- ✓Correlating usage (request volume) against performance (response time, error rate) distinguishes "the app is just getting more traffic" from "the app is degrading independent of load," a key diagnostic pattern
- ✓KQL is a pipe-based query language for interrogating Log Analytics/Application Insights data, using operators like where to filter, summarize to aggregate, project to select columns, and order by to sort results
- ✓KQL exists because manually scanning raw logs is impractical at production telemetry volume; a KQL query lets an engineer aggregate and filter across millions of rows into a focused, actionable result set
1. A single user request touches a web front end, an API, and a downstream database call, and the request is slow overall. Which Application Insights capability shows exactly which of those three hops is responsible for the slowdown?
2. Which basic KQL query would correctly count failed requests by result code over the last hour, given a table called requests with a success and resultCode column?
3. An application's response time has degraded, and the team wants to know whether this is simply due to a spike in request volume or a genuine performance regression independent of load. What should they correlate?
Recommended: Pluralsight
This free course covers the theory. Pluralsight adds structured DevOps Engineer Expert learning paths, hands-on Azure Pipelines and GitHub Actions labs, and timed practice exams to make it stick before exam day.