Beyond Alerts: Building Detections That Actually Matter

Every SOC measures alert volume. Almost none measure whether a single one of those alerts would have caught the breach that matters. This is the story of how detection engineering went wrong, and the discipline that fixes it.

Walk into any security operations center and ask what the detection program produces. You will hear numbers: thousands of rules deployed, tens of thousands of alerts per week, dashboards glowing with activity. Now ask a harder question: of last quarter's alerts, how many led to a confirmed finding an attacker would have cared about? The room usually goes quiet.

The volume trap

Detection programs drift toward volume because volume is easy to measure and easy to defend in a budget review. A rule that fires is a rule that looks alive. But alert volume and detection quality are not just different metrics; past a threshold, they are opposed. Every low-quality alert costs analyst attention, and attention is the only resource a SOC cannot buy more of at 2 a.m.

The failure mode is familiar. A new CVE drops, someone ships a signature, it fires on every vulnerability scanner in the estate, and within a month the rule is either suppressed or ignored. The rule still counts toward coverage on paper. In reality it is dead weight with a pager attached.

An alert nobody investigates is not a detection. It is a liability with a timestamp.

Detect behavior, not indicators

Indicators expire. Hashes rotate per build, domains cost pennies, and IP blocks recycle weekly. Behavior is what attackers cannot cheaply change: they must authenticate, move, escalate, and take data out. A detection program anchored to the behavior layer survives infrastructure churn that kills indicator feeds overnight.

• Credential misuse: impossible travel, first-time source-to-target pairs, service accounts logging in interactively.
• Living off the land: signed binaries doing unsigned things, bitsadmin and certutil moving payloads, scheduled tasks appearing outside change windows.
• Quiet exfiltration: outbound volume that is normal for the org but abnormal for that host, that hour, that destination.
• Persistence in plain sight: new run keys, WMI subscriptions, and OAuth grants that no ticket ever asked for.

Prove it on your own history

The single highest-leverage practice in detection engineering is replaying a detector against months of your own real telemetry before it ever goes live. Not a lab dataset, not the vendor's demo data: your environment, with its weird build servers and its one contractor who logs in from three countries.

Replay answers the two questions that decide whether a detector deserves production: what would it have fired on, and what would it have missed? Tuning against real noise before deployment is the difference between shipping a detection and shipping a pager storm.

signal: authentication
where:
  account.type: service
  logon.type: interactive
baseline:
  learn: 30d            # what "normal" looks like per account
  suppress: known_admin_workflows
replay:
  window: 90d           # test against real history first
  require: precision >= 0.8
on_fire:
  create: detection     # evidence attached, scored, cited

Note the last line. In a mature program, the words matter: you build detectors. When a detector fires and finds something real, it creates a detection, with the evidence already attached. Keeping those two words straight keeps the pipeline honest, because it forces you to ask which detectors have never produced a detection worth reading.

Treat detectors like software

Detectors are code, and they deserve a software lifecycle: version control, review, staged rollout, and retirement. The graveyard matters as much as the release. A detector that has fired only false positives for ninety days should be retuned or deleted, and either way someone should be able to see who changed it, when, and why.

Close the loop with investigation

A detection that lands in a queue as a bare alert has done half its job. The other half is context: the entities involved, the surrounding timeline, the verdict, and the evidence a human can check. When triage findings flow back into tuning, precision compounds. When they do not, every false positive is paid for twice: once by the analyst who closes it, and again by the next analyst who sees the same one.

This is where an AI analyst changes the economics. Soc0 investigates every detection before a human sees it, attaches the evidence trail, and scores the verdict. The queue your team opens in the morning contains only the detections that survived investigation. That is what detection engineering is for.