Back to Blog

AI-Driven Dynamic ETW Diagnostics

Christopher 9 min read
etwai-diagnosticswindowsarchitecture

A typical Windows host registers between 1,500 and 2,500 ETW trace providers covering kernel, .NET, networking, storage, security, and third-party drivers and services. For a given diagnostic question, only a small subset of providers produce relevant evidence; the remainder produce volume that has to be filtered or stored.

Two existing approaches to provider selection have known trade-offs. The first enables a broad keyword mask, captures the full event stream, and analyzes after the fact. This approach has higher network, storage, and query costs, and a broader capture surface increases the chance that PII or proprietary application data ends up in the trace. The second approach asks the operator to select providers manually. Tools such as WPR, WPA, and PerfView use this model. It requires the operator to know which providers correspond to which subsystems before the capture begins.

ET Ducky's dynamic ETW diagnostic agent selects providers programmatically per investigation. The AI decides which providers to enable for each round, observes the resulting event counts, and adjusts the selection in subsequent rounds based on what was captured. This document describes the architecture.

Round-based investigation

When an operator opens a guided troubleshooting session and submits a question (for example, "DNS resolution is intermittent on this host"), the orchestrator runs a multi-round investigation:

  1. The cloud-side investigation planner receives the question, the host's ETW provider catalog, and the engine's current budget headroom. It calls the AI to propose a set of providers to enable for round one.
  2. The agent enables those providers on a separate diagnostic ETW session distinct from the always-on baseline session. Capture runs for a bounded window. Events are counted into a per-(provider, event id, task, opcode) tally on the host.
  3. The counts-only snapshot is uploaded to the cloud. The planner calls the AI with the snapshot and the prior round's selection, and the AI returns the next round's plan (enables, disables, capture duration, or a "conclude" signal).
  4. The loop continues until the AI concludes or the round cap is reached.

Each capture window is configurable between 10 and 120 seconds. The default round cap is 5, with a hard ceiling of 10. Investigations typically complete within two to five minutes.

Per-agent provider catalog

The agent enumerates the host's registered ETW providers at startup and uploads a tagged inventory to the cloud. The inventory is tiered:

Vendor-specific providers vary across hosts. The per-agent catalog allows the AI to plan against the providers actually registered on the target host rather than a fixed list.

Engine-enforced safety caps

Budget limits are enforced inside the capture engine, not in the AI's prompt. The AI proposes changes; the engine accepts or refuses. A refused request returns a structured error to the AI for re-planning. Defaults:

CapDefault
Concurrent providers per investigation16
Aggregate events/sec per session25,000/s
Aggregate events/sec across all diagnostic sessions on the host50,000/s
In-flight buffer memory per session64 MiB
Concurrent investigations per host4
Hypothesis-loop rounds per investigation5 default, 10 hard ceiling
Per-round capture window10 to 120 seconds

An in-flight watchdog measures the rolling event-rate window. When the rate exceeds a configured cap, the engine automatically disables the most recently enabled provider and emits a structured scale-back event so the AI re-plans rather than retrying the same request. Investigations that go 15 minutes without a command from the cloud are removed by an on-agent sweeper.

Session isolation

The diagnostic session uses a separate TraceEventSession pair with a per-investigation name. It is distinct from the always-on baseline session that produces routine telemetry and behavioral security detections. The diagnostic and baseline sessions share no state, are configured independently, and are torn down independently. A configuration change on the diagnostic side cannot modify baseline session behavior.

Data handling

Raw events from diagnostic sessions are not transmitted to the cloud. The engine counts events into the per-(provider, event id, task, opcode) tally on the agent, and only the counts-only summary is uploaded. Tear-down at the end of the investigation purges event buffers and disposes both underlying TraceEventSession objects.

The conclusion returned to the cloud describes which providers fired, how many events of each type, and the AI's interpretation. The event payloads themselves remain on the endpoint.

Remediation script handoff

When an investigation concludes with a known-fix pattern, the AI generates a remediation script (PowerShell on Windows, Bash on Linux) and submits it to the org's Process Automations approval flow. The script is created with ApprovalStatus = "pending_review" and a paired review ticket is filed in the org's ticketing system. Execution requires organization administrator approval.

Two execution modes after approval:

The approval gate, per-command security policy, and organization administrator role check apply identically to AI-generated and human-written scripts. AI-generated scripts carry Origin = "process_automation" and CreatedBy = "dynamic-etw:<jobId>" markers in the audit trail.

Scope

The dynamic ETW diagnostic agent does not replace baseline always-on telemetry. The agent continues to produce metric heartbeats, behavioral security monitoring events, and the standard event stream. The diagnostic engine runs in addition to these and is active only during investigations.

The agent does not execute remediation actions autonomously. AI-generated remediation scripts go through the same approval gate, per-command security policy, and role checks as human-written scripts.

The per-provider cost catalog that gates admission control is seeded from sniffer measurements on a reference host. Production telemetry refines these estimates over time. The Tier 3 long-tail provider search is a planned addition.

ET Ducky

Documentation and pricing are available on this site. The dynamic ETW diagnostic agent is included in all plan tiers.

View Pricing