L8. Data Connectors: Ingesting Logs at Scale
Visual workflow edition
Worked example, diagrams and a question with time to think. Synthetic American-English narration; examples are fictional. Reviewed October 2026.
Read the video transcript
A connector says connected, but your investigation query is empty. That is a common moment to confuse configuration with evidence. We'll follow an event from its source to the destination table, using agent-based collection as one example, and separate ingestion failures from query mistakes.
Start with the connector's current documentation. A cloud service integration may need tenant permissions. Agent-based sources need the supported agent and collection configuration. Custom ingestion has its own schema and authentication requirements. Do not reuse instructions from a different source simply because both ultimately send data to Sentinel.
For sources collected through Azure Monitor Agent, verify agent health and the applicable data collection rule. Check which machines and events are included and where the data is sent. These are separate settings. A healthy agent with the wrong event selection can deliver no evidence for the detection you are trying to build.
In our fictional lab, an approved test generates one event at a recorded time. Query a broad enough time window and inspect the expected table. Check key fields and data types, not just the total row count. If the record arrives late, note that delay before setting a detection window that could miss it.
An empty result can have several causes. The event may never have been collected, it may be in another destination, or your filter may exclude it. Start with a small query that shows recent rows. Add filters one at a time. Be mindful of sensitive fields and query cost when expanding the investigation.
Which is stronger evidence of a working path: a green connector indicator, or a known test event found in the expected table with the right fields? Pause and explain the difference.
The known event is stronger evidence for that tested path. The status indicator is useful context, but it does not validate every source or field. Record the source, event time, destination, and observed delay. Reuse this small evidence checklist whenever you add a connector or change collection rules.
Course outlineLesson 8 of 18
Data connectors are the pipeline for getting security data into Microsoft Sentinel. This lesson covers connector types, the Azure Monitor Agent, CEF/Syslog ingestion, and custom log strategies for the SC-200 exam.
Data Connector Categories
Sentinel supports 300+ data connectors organized by type:
| Connector Type | Examples | Mechanism |
|---|---|---|
| Microsoft service-to-service | Microsoft 365, Entra ID, Defender XDR | Native API integration |
| Azure service | Azure Firewall, Key Vault, NSG Flow Logs | Diagnostic settings |
| Syslog/CEF | Linux hosts, network appliances, firewalls | Agent-based collection |
| REST API | Custom applications, third-party SaaS | API polling or webhook |
| Custom logs (DCR-based) | Any structured log source | Azure Monitor Agent + DCR |
| Codeless Connector Platform (CCP) | Partner-built connectors | Standardized API framework |
Microsoft Service Connectors
Microsoft first-party connectors are the simplest to configure. Key connectors:
- Microsoft Defender XDR: Ingests incidents, alerts, and raw event data from all Defender workloads
- Microsoft Entra ID: Sign-in logs, audit logs, provisioning logs, risky users, risk detections
- Office 365: Exchange, SharePoint, and Teams activity logs (free ingestion)
- Azure Activity: Subscription-level management operations (free ingestion)
Windows and Azure Sources
Windows Security Events via AMA
The Windows Security Events via AMA connector collects Security log events from Windows machines using the Azure Monitor Agent. You create a data collection rule (DCR) that picks which machines to collect from and which events to collect: a predefined set (All, Common or Minimal) or a custom selection using XPath queries. Events land in the SecurityEvent table. Choosing a smaller event set, or filtering with XPath, is the main lever for controlling ingestion cost.
Windows Event Forwarding (WEF)
When you can't (or don't want to) put the agent on every machine, Windows Event Forwarding sends events from source machines to a central Windows Event Collector server. Install the Azure Monitor Agent on the collector and use the Windows Forwarded Events connector with a DCR that collects the ForwardedEvents channel. Forwarded events land in the WindowsEvent table. Plan for collector sizing and for the subscription (which machines and which events are forwarded).
Azure Activity Logs and Resource Diagnostic Settings
The Azure Activity connector uses Azure Policy to assign a diagnostic setting to each subscription in scope, so subscription-level management events stream to the workspace into the AzureActivity table, including subscriptions created later. Resource logs (for example Key Vault or Azure Firewall) are sent with each resource's diagnostic settings, and you can enforce those at scale with Azure Policy instead of configuring resources one by one.
Exam tip: Updated for the SC-200 skills measured as of October 21, 2026, which add Windows Security Events via AMA, Windows Event Forwarding and Azure activity logs via Azure Policy to the Sentinel ingestion objectives.
Azure Monitor Agent (AMA)
The Azure Monitor Agent replaced the legacy Log Analytics Agent (MMA) for collecting logs from Windows and Linux machines. Key concepts:
- Data Collection Rules (DCR): Define which logs to collect, how to filter them, and where to send them
- Data Collection Endpoints (DCE): Network endpoints the agent connects to for uploading data
- Transformations: KQL-based transforms that filter or modify data during ingestion (before it hits the workspace)
DCR-based collection allows you to:
- Filter events at collection time (reducing ingestion costs)
- Route different log types to different workspaces
- Apply transformations to normalize or enrich data
Syslog and CEF Collection
For network appliances, firewalls, and Linux hosts:
- Syslog: Standard Unix logging protocol. Collected by the AMA on a Linux forwarder
- CEF (Common Event Format): Structured syslog format. Uses the same AMA-based collection but parses into the CommonSecurityLog table
Architecture for CEF/Syslog:
- Network appliance sends logs to a Linux log forwarder (rsyslog or syslog-ng)
- Azure Monitor Agent on the forwarder collects and sends to Sentinel
- Syslog data lands in the Syslog table; CEF data lands in CommonSecurityLog
Custom Logs and the Logs Ingestion API
For sources without a built-in connector, use the Logs Ingestion API:
- Create a custom table in the Log Analytics workspace
- Define a DCR with the table schema and any transformations
- Send data via the REST API using Microsoft Entra authentication
This is the recommended approach for custom application logs, third-party tools without native connectors, and IoT devices.
Connector Health
Monitor connector health through:
- Data connectors page: Shows last log received timestamp and status
- Health monitoring workbook: Visualizes ingestion latency and volume anomalies
- Heartbeat table: For agent-based connectors, tracks agent check-in status
- ✓The Defender XDR connector provides free alert/incident ingestion, but raw advanced hunting tables are billable.
- ✓Azure Monitor Agent (AMA) replaced the legacy Log Analytics Agent (MMA) for log collection.
- ✓Data Collection Rules (DCR) define what to collect, how to filter, and where to send logs.
- ✓Ingestion-time transformations use KQL in the DCR to filter or modify data before storage, reducing costs.
- ✓CEF logs land in the CommonSecurityLog table; standard Syslog lands in the Syslog table.
- ✓The Logs Ingestion API enables custom log sources to send data to Sentinel via REST.
- ✓Windows Security Events via AMA lands in SecurityEvent; use a DCR with All, Common, Minimal or custom XPath event selection.
- ✓WEF centralizes events on a collector; the Windows Forwarded Events connector (AMA on the collector) lands them in WindowsEvent.
- ✓Use Azure Policy to apply diagnostic settings at scale, both for subscription activity logs (AzureActivity) and for resource logs.
1. Which component defines what logs the Azure Monitor Agent collects and where it sends them?
2. A firewall appliance sends CEF-formatted logs to a Linux forwarder running the Azure Monitor Agent. Which Sentinel table receives these logs?
3. An organization wants to reduce Sentinel ingestion costs by filtering out informational Windows events before they reach the workspace. Which feature should they use?
4. An organization wants Security log events from 2,000 domain-joined servers in Sentinel but cannot install agents on them. Which approach fits?
5. Which table receives events collected by the Windows Forwarded Events connector?
6. You need activity logs from every current and future subscription in a management group sent to Sentinel. What should you use?
Keep learning
Use the knowledge check above to test your understanding. You can browse every lesson without signing in; opening another lesson does not mark this one complete.
Next lesson: KQL Fundamentals: Querying Security Data →