An Empty Hunt Needs an Evidence Check
An empty threat hunt looks reassuring. It can also mean that a workstation never generated the event, a collector never received it, or a search rule asked for a field that was never recorded.
That distinction has become a measurable problem. In the 2026 SANS survey, 50% of 500 respondents named data quality or quantity as their biggest barrier, ahead of skilled staffing at 45%. Only 40% said their programmes formally measure hunting outcomes. SANS released the survey page on 22 September and its summary on 23 September.
The survey records practitioners’ perceptions, not a measured failure rate for security tools. It cannot tell an individual organisation whether an intruder has arrived. Its useful warning is narrower: an analyst’s skill and a well-written query cannot recover evidence that was never collected.
A rule has more than one point of failure
For an unusual process on Windows, Security event 4688 records New Process Name and Creator Process ID. Version 2 adds Creator Process Name. Arguments are absent by default; recording them requires its own policy.
The search can return zero at several different stages. The endpoint may not have Audit Process Creation enabled. The event may exist locally but fall outside a forwarding subscription. It may reach a collector yet be dropped, delayed or parsed into different fields before indexing. Finally, the rule may filter it out because its assumptions about field names, event versions or timing are wrong. Each failure produces the same visible answer to the hunter: no match.
Windows Event Forwarding reads existing events, without enabling audit policies or disabled channels. By default, clients send only entries created after receiving a subscription. Logs overwritten during disconnection create gaps without notifying the collector. Collector health alone cannot prove complete coverage.
Prove the path with a harmless event
A safe validation exercise starts with one approved test endpoint, one known event and an expected arrival window. For a Windows process-creation hunt, an administrator can launch a benign programme on that endpoint and record the host, time, account and programme run. The goal is to test the evidence path, not to imitate an attack.
Check the stages in order:
- Generation. Confirm that the endpoint’s Security log contains the expected entry. If it does not, inspect the effective generation-auditing policy before changing a search rule. The event reference lists that policy as a prerequisite.
- Fields. Inspect the raw event, including its version, launched programme, creator information and timestamp. Do not assume recorded arguments are available. The reference documents the separate opt-in setting.
- Forwarding. Verify subscription selection, source connection and the expected ForwardedEvents entry. Respect configured batching and delivery timing: one event can await a count or time limit, without the subscription having failed.
- Indexing. Find the raw event in the hunting platform by its known host and time, then inspect the parsed fields and ingestion delay. This step is a BlackTree workflow recommendation: a collector receipt does not establish searchability under the schema a rule expects.
- Rule logic. Run the relevant rule against the known event and document which condition matched or suppressed it. A benign canary proves the pipeline can carry that event type. It does not prove that a malicious behaviour would be recognised.
To validate the detection itself, use a reviewed, non-destructive replay or a controlled exercise with the precise behaviour the rule is meant to identify. Include ordinary administrative activity as a negative case. If the rule flags every maintenance task, it will generate avoidable work; if broad exclusions hide the controlled test, it has lost its purpose. Record both outcomes before allowing an automated response to depend on the rule.
Measure evidence coverage, not just alert volume
The useful scorecard is small. For each priority hunt, record the systems expected to produce evidence, the share that actually did, required-field completeness, time from local event to searchable event, and the result of a repeatable positive and negative test. Re-run the check after audit-policy, endpoint-agent, collector, parser or rule changes. A fall in alert volume without a passing evidence test is ambiguous, not a success metric.
More collection has costs. The auditing guide warns that recorded arguments are stored in plain text, where they may contain passwords or private data. Limit who can read those events, review retention and fix applications that pass secrets in launch arguments before widening collection. BlackTree’s analysis of file-notification side channels raises the same broader issue: useful activity telemetry brings access and retention obligations.
This Windows example is only one way to make the problem tangible. A cloud identity hunt, a network hunt and an endpoint hunt need different events and controls, but the question is the same: can the team produce a known, safe signal and follow it all the way to the decision? Until it can, “nothing found” describes the search result, not the environment’s actual condition.
Sources
- SANS survey announcement, 23 September 2026. Primary source for sample and reported findings. Survey landing page dated 22 September; full PDF requires login and was not used for additional claims.
- Event-field reference, consulted 28 September 2026. Primary documentation for fields and event versions.
- Argument-recording guide, consulted 28 September 2026. Primary documentation for audit prerequisites and privacy implications.
- Microsoft forwarding guide, consulted 28 September 2026. Coverage gaps and limits.
- Microsoft, validating a source-initiated subscription, consulted 28 September 2026. Primary documentation for collector checks.
