Detection Engineering Weekly #23 - What's your true positive sweetspot?
Detection Engineering Weekly #23 - What's your true positive sweetspot?
Mine is 100% because I only alert when a computer turns on
Welcome to Issue #23 of Detection Engineering Weekly!
This week’s recap:
💎 by Adam Ostrich and his company’s approach to validating detections at scale
Hike Mount Thrunt with David Bianco as he describes the PEAK framework for threat hunting
LOLDrivers 1.0 release, True Positive and False Positive Rate sweet spots, and threat intel being a force multiplier
Pulling apart malware configs by Unit42, an AgentTesla dissection, BushidoToken IDs a scam ring and CISA squashes a snek
And much more!
Also, I started a Twitter list for detection engineering. Feel free to follow it and DM me/email me some people to follow to improve it!
🫵 Hey you! Do you have a blog post, social link, or open-source tool you want to see in this newsletter?
Leave a comment below or email me techy@detectionengineering.net
💎 Detection Engineering Gem 💎
The Validated Canary: Our validation philosophy by Adam Ostrich
I am a big fan of organizational structures related to detection functions. Red Canary researchers and detection engineers are some of the best I’ve seen when productizing and operationalizing the ethos of threat detection and detection efficacy. This post gives readers a closer look at how their business has curated detection rulesets AND continuously validates them. They’ve split up detection engineering into the core detection team (working on and curating new detections) with a validation team (testing detections end-to-end and improving or removing them).
State of the Art
Hypothesis-Driven Hunting with the PEAK Framework by David Bianco
Threat Hunting frameworks have become increasingly popular, evolving with threat detection and threat intelligence frameworks over the past few years. I will say that PEAK is probably my favorite from all I’ve read. In this post, Bianco uses PEAK to create a hypothesis around hunting for DNS exfiltration. Notice how their framework incorporates intelligence, telemetry, behavior (vs just indicators) and location of the attack, which to me reads like victimology. The best part? You feed results, including detection opportunities, into the backlog!
Living Off The Land Drivers 1.0 Release: New Features, Enrichments, and Community Contributions by Michael Haag
The LOLDrivers team makes a 1.0 release with massive community engagement, feedback, and contributions! The cool thing about a project like this is the community engagement. Folks at the Sigma and Velociraptor integrated detections for these drivers within their toolset. Several researchers added all kinds of hunting or detection rules out into the open-source ether. Nice work, team!
The Evolution of Business Email Compromise by Invictus Incident Response
“Hey techy, its $CEO_OF_YOUR_COMPANY, did you get those giftcards yet? Nevermind, click this link real quick and login to email. I’m with investors!!1”
Business Email Compromise (BEC) is an amalgamation of scenarios like the “CEO text” above that can result in some of the heftiest financial damage possible to a company. Don’t believe me? Check out the FBI’s latest Internet Crime Report, page 22. In this post, Invictus details several BEC incidents they’ve responded to, giving detection opportunities along the way.
Detecting and responding to ESXi compromise with Splunk by Alex John
This blog almost perfectly represents how threat-informed defense leads to detection opportunities. John reviews a diverse set of ransomware intelligence reports that began with an ESXi infection and distilled many of the tactics, techniques, and procedures into a list of interesting things to look out for. For example, almost all of the initial infections ran esxcli VM process list and esxcli VM process kill* . As always, I must call out the hunting queries/detection opportunities section at the bottom!
Fantastic Rootkits: And Where to Find Them (Part 1) by Rotem Salinas
Great deep dive on techniques for creating and detecting rootkits on the Windows platform. I’ve written plenty of Linux-based rootkits in my career/offtime, and have put Windows rootkits on the backburner. The wildest thing to me about any Windows malware development is the sheer level of undocumented APIs - is this security through obscurity? Linus would never allow that to happen!
Threat Intelligence vs Threat Hunting: What is the Perfect Pipeline? by Adam Goss
Interesting take on how threat intelligence should feed into threat hunting and detection efforts. This newsletter has posted many blogs and writeups talking about the detection backlog. This writeup is another great example of how information should flow into hunt and detection opportunities. The blog reads as if Goss assumes you have threat intel analysts doing proper dissemination, so if you don’t have threat intel analysts in your team, then staying on top of news in the threat landscape is probably your best avenue to do this at a smaller scale.
SIEM Content, False Positives and Engineering (Or Not) Security by Anton Chuvakin
As a vendor in the threat detection space, I can relate heavily to Chuvakin’s observations here about “out of the box” content shipped in a product to end customers. We (Datadog) have this debate almost daily - what is an acceptable amount of false positives for a default rule shipped to customers, and what is unacceptable? His polls were interesting because most people in both polls said that a false positive from an out-of-the-box detection means the user, not the vendor, should tune it.
In Defense of a Target Acceptable Global False Positive Rate for Threat Detections by Steve Roesser
Excellent follow-up post to Chuvakin’s blog above! Roesser gives their opinion on what a “sweet spot” true positive/false positive rate should be with a detection, with some nuanced observations around the context surrounding the alert firing. Is it “fast” to triage, or slow? Who triages the alert, SOC, IR, or Tier 1 analysts?
As I said above, as a vendor, it’s interesting to see how an internal team decides their “sweet spot” for these rates.
Detection Engineering on Social Media
https://twitter.com/arekfurt/status/1656049580830539780
https://twitter.com/chrissanders88/status/1655935856908312583
Threat Landscape
AgentTesla - Full Loader Analysis - Resolving API Hashes Using Conditional Breakpoints by Matthew/embee research
Awesome deep dive in an AgentTesla sample by Matthew/embee research here. They show their methodology of pulling apart the sample, with tips and tricks on using Ghidra, Dnsspy, X32dbg, detect-it-easy and much more.
New Cactus ransomware encrypts itself to evade antivirus by Ionut Ilascu
Who would win in a fight? A multi-billion dollar EDR vendor with a massive go-to-market strategy of stopping ransomware, or one ransomware family that just.. 7zips their ransomware binary? Well, according to Kroll Researchers, it turns out 7zip is beating out antivirus to prevent detection. Cactus has been active since at least March of this year and really likes finding open and/or vulnerable VPN devices to gain an initial foothold.
GreenMwizi - Kenyan scamming campaign using Twitter bots by BushidoToken
BushidoToken uncovers a scamming ring, likely from Kenya, that is entering Twitter threads and DMs of victims seeking help from Booking.com. I’m sure our new overlord on Twitter has staffed a team to help combat scams like this :
Teasing the Secrets From Threat Actors: Malware Configuration Parsing at Scale by Mark Lim, Daniel Raygoza and Bob Jung
The lovely thing about malware is that authors need to configure it in a scalable way. This means that researchers can be the “bad guys,” pulling apart information and reverse-engineering these configurations for large-scale analysis of criminal infrastructure. Unit42 researchers Lim, Raygoza, and Jung showcase an IcedID analysis example where they parse configuration data directly from the sample in memory and give analysis tips and detection ideas for you to do the same.
Packagist.org maintainer account takeover by Nils Adermann
A gray-hat netizen took over four Packagist accounts and published benign composer.json files to 14 PHP packages to demonstrate.. something. The kicker is that these four packages have totaled over 500 million installs to date. Interestingly, Adermann recommends “not re-use passwords,” as they were most likely credential stuffed from previous breaches. This is the same problem other package management ecosystems have with enforcing 2FA.
Hunting Russian Intelligence “Snake” Malware by CISA
CISA hits a home run with this incredible deep dive into the Russian-backed malware family, Snake. “We consider Snake to be the most sophisticated cyber espionage tool in the FSB’s arsenal,” if that doesn’t put chills down your spine, maybe you are a normal human being, unlike me. An impressive and sophisticated toolset focusing on stealth, CISA has found infections worldwide but is mostly used as P2P communication nodes to achieve its targeted, tactical objectives.
Open Source
Detection Engineering Pocket Guide by jshlbrd
Easy-to-reference guide on the most critical components of a Detection Engineering function. Latest update is on “Detection Metrics” at the bottom.
Freeze by optiv
Lots of EDR bypass toolsets have been published as of late! This one focuses on removing EDR hooks in userland and executing your code under the watchful eye of your favorite EDR platforms.
Awesome Cloud Security Labs by iknowjason
Another “awesome-” repo focusing on cloud security labs. I’ve done a number of these labs (CloudGoat, flaws.cloud and AWSGoat) and they are definitely worth your time.
Nidhogg by ldov31
Heavily customized Windows-based rootkit to add to your red team and emulation workflows. At least it doesn’t say “for educational use only!”