Chaos Detection Engineering is necessary (and unheard of) due to a inherent problem with SOARs. I believe that SOAR made it easy for non-engineers to build code ("automations") via LC/NC (good!), but without knowing or applying the hard-won engineering solutions that help write good code, like Test-Driven Development, modular programming with functions instead of writing repetitive/hard-to-manage code, diff-style commit logs allowing precise change tracking, or pair programming. You can see this if you talk to a SOAR vendor - they emphasize how easy it is to *create* content and spend very little time talking about aiding maintenance/reliability of content.
Chaos Detection Engineering is such a brilliant concept! We steal from SRE all the time, but this one really clicks - intentionally popping the tire to see how fast you can change it. The five-step framework (steady state → hypothesis → experiment → verify → improve) maps perfectly to detection validation. I've never heard the term 'adversary injection' before but it captures exactly what BAS products are missing - they test coverage but not the full IR loop. The supply chain compromise survey was eye-opening too - 'control handoff' is a perfect name for attacks like xzutils. Good luck with Datadog Detect tomorrow! The 1000 chatters last time making it feel like a Twitch stream sounds like my kind of conferance :D
Great read as always. Thanks, Zack.
Chaos Detection Engineering is necessary (and unheard of) due to a inherent problem with SOARs. I believe that SOAR made it easy for non-engineers to build code ("automations") via LC/NC (good!), but without knowing or applying the hard-won engineering solutions that help write good code, like Test-Driven Development, modular programming with functions instead of writing repetitive/hard-to-manage code, diff-style commit logs allowing precise change tracking, or pair programming. You can see this if you talk to a SOAR vendor - they emphasize how easy it is to *create* content and spend very little time talking about aiding maintenance/reliability of content.
Chaos Detection Engineering is such a brilliant concept! We steal from SRE all the time, but this one really clicks - intentionally popping the tire to see how fast you can change it. The five-step framework (steady state → hypothesis → experiment → verify → improve) maps perfectly to detection validation. I've never heard the term 'adversary injection' before but it captures exactly what BAS products are missing - they test coverage but not the full IR loop. The supply chain compromise survey was eye-opening too - 'control handoff' is a perfect name for attacks like xzutils. Good luck with Datadog Detect tomorrow! The 1000 chatters last time making it feel like a Twitch stream sounds like my kind of conferance :D