DefendBench

Environment

Splunk and Microsoft Sentinel: GitHub, GitLab and Azure DevOps, with eight sources that all look like source control data, split across the two SIEMs.

Passing outcome

• Inventories all eight sources by their contents, catching a dead legacy connector and CI job logs named like an audit log.
• Pushes back on the GitHub source it was told to use: the audit stream never records which ref or files a push changed, so tag moves and workflow edits are blocked there.
• Never builds on job-log text or endpoint command lines as proof of a server-side change.

Environment

Microsoft Sentinel with Windows Security events and Microsoft Defender for Endpoint: a rule that finds ADFS servers, then alerts when remote WMI activity reaches one of them.

Passing outcome

• Verdict: Broken, with two independent defects. The Windows Security branch reads ProcessName, which is empty on every process-creation event (the executable path is in NewProcessName). The Sysmon branches read a table that holds no Sysmon data.
• Moves the rule to Defender for Endpoint rather than only patching the column: process activity joined to network logons on the same device, and WMI subscriptions and remote WMI operations from Defender's device events. Windows Security reaches only about half the servers, so a column-only fix stays blind to every Defender-only host.
• Reads the zero correctly. No ADFS server shows up anywhere in the environment, so a repaired rule that returns nothing is expected, not a failed fix.
• Avoids the tempting wrong answers: Healthy because it never fired, fixing the column and stopping, onboarding Sysmon when Defender already records the behavior, and calling the fix a failure because it returned zero.