DEW #163 - Semantic Malware Detections, Microsoft's CTI REALM evals and Thrunting for Knowledge
if thrunting is for threat hunting what is the clobbered word for malware analysis?
Welcome to Issue #163 of Detection Engineering Weekly!
✍️ Musings from the life of Zack:
The sky and sun look crazy here in the Northeast U.S. due to fires a bit further north in Ontario. I hope anyone in that area stays safe and hopefully we get some rain here soon. I was driving towards the city here and drove by two brush fires, something I’ve NEVER seen before growing up in New York
I’ve been ramping up training for a 10k race here in Maine called the Beach to Beacon. I used to run races all of the time until my life was taken over by BJJ. Happy to say it’s been going really well and I’m feeling confident I can hit my 10k goal time of 50:00 min or less :D
I finished my build-your-own landline phone project. FreePBX took the longest since there’s no official Docker image so the VM install takes an hour, but other than that, the configuration of the phone and the trunk to Telnyx went super smooth. My kids now have a phone they can call family with!
Detection & Response Happy Hour @ Black Hat - ALMOST AT CAPACITY :O
If you are going to be in Vegas during Black Hat, come swing by Tom’s Watch Bar @ the NYNY Casino right on the strip on Tuesday!
I’m running it back after BSides SF/RSA with friends and supporters of the newsletter, Cotool.ai. We are almost at our 80 person capacity after one week, so register now before I have to figure out how to pay for more food and drinks :P
I opened up one more co-sponsor slot, so if you are interested in supporting the Happy Hour and having your company logo on the invite and the shoutouts in the newsletter & on social media, reach out to me at techy@detectionengineering.net
Otherwise, hit the register button below and come steal my stickers:
💎 Detection Engineering Gem 💎
Detection Engineering in the Era of Semantic Malware by Daniel Koifman
Semantic malware, or “promptware”, is malware attached to unstructured files and configurations used by coding agents. This research by Origin, which Koifman here referenced, shows that traditional threat detection strategies and mechanisms need to adapt to malware being injected into a non-deterministic process, a.k.a. coding agents. I talked about this type of malware with Agent Skills in last week’s issue. Koifman’s blog here highlights gaps in understanding and telemetry as we hunt for detection opportunities in these chaotic processes.
The fundamentals remain the same: we’ve seen work on detecting maliciousness in the agent's execution layer (also from last week’s post). Koifman points this out with areas to focus on in the underlying operating system, using process trees as an example. An exploit that runs on a piece of software and then causes PowerShell to run is an observable you can write a rule against. We are seeing this with IDEs and developer tools when they install packages. Now, we need to look at how coding agents themselves perform acts under malicious conditions, because some of our assumptions break:
Process tree analysis fares no better. In conventional detection, parent-child relationships are signal-rich: winword.exe spawning cmd.exe is suspicious; svchost.exe spawning powershell.exe is actionable. But an AI agent runtime is designed to spawn arbitrary processes. Claude Code calling curl, git, python, node, grep, and bash in rapid succession is not an anomaly — it is Tuesday. The agent’s process tree is a superset of almost every LOLBin chain an attacker would want. When the legitimate behavior of a process includes “execute any command the user or the context window tells it to,” there is no parent-child relationship left that constitutes an anomaly by itself.
He sheds some hope in the section afterward, where least privilege and some boring security controls can actually make a big difference. For example, he recommends file integrity monitoring (FIM) on critical coding agent configuration files and generating an alert if a process monitors a CLAUDE.md or similar file that is outside an allowed group of processes. IMHO, this is harder to do in practice. We’ve seen supply chain attacks target IDE extensions, so if your FIM rule allows disk writes to a CLAUDE.md if it came from Cursor, then a malicious extension can easily bypass the detection.
I don’t think this is a pure recommendation, but rather a challenge for us to consider the behavior and assumptions surrounding coding agents so we can build guardrails for them. EDR still works once the coding agent touches the operating system. Application allow listing, when configured correctly, can prevent the installation of post-exploitation malware, so you can build around the chaos of an agent. But that’s the thing: they are chaotic by design, and if you have a business that expects your engineering team to build and ship faster with these agents, you need to consider what friction you introduce into the environment that can slow them down as part of your risk calculation.
🔬 State of the Art
CTI-REALM: A new benchmark for end-to-end detection rule generation with AI agents by Arjun Chakraborty
CTI-REALM is an open-source benchmark for detection, ideation, and implementation from CTI reports. Chakraborty and Microsoft Researchers curated 30+ threat reports from industry sources (and they named Datadog Security Labs, where I work, as one of them n.b.d) and used them as a source of truth. They then measured how foundational models ingested these reports, built and explored telemetry associated with these attacks, and generated rules on the other end.
Interestingly, they contributed this dataset directly to the UK government’s AI Inspect repository, which hosts datasets and methodologies for evaluating AI tools.
Why I Spend More Time Proving Myself Wrong Than Hunting Threats by Smruti Ranjan Pradhan
This was a great post that helps readers demystify the true value of threat hunting. I think we teach threat hunting as the practice of finding security incidents and threat actors that our detection/SOC team missed, and that’s the only thing threat hunters do. There is a big problem with this: 99% of the time, you won’t find anything. But that is by design because there is a lot more to hunting that contributes even more value besides doing hunts that “fail” over and over again.
Pradhan hits the nail on the head, describing hunting as a way to check biases in your detection strategy, discover gaps, and learn more about your company’s environment. This sets threat hunting teams up for success because, in my humble opinion (and how I organize our threat hunting effort here at Datadog), the goal of threat hunting is to discover gaps. Testing a hypothesis by searching your telemetry to disprove it should lead you to identify control and detection gaps. These gaps should inform your security strategy and help answer the “are we covered?” question that every single leader and executive asks.
I’m super excited to see that my friend Matt Johansen (of the VulnU Newsletter, go subscribe!) and Ed Lowlevel (of lowlevel.tv) have started a new security podcast. They review the most interesting security news every week, and you can tell how much time and thoughtfulness they put into the production and quality here. What I appreciate here is how technical AND accessible they are.
The first half was the most interesting for me, and it revolved around the arrest of a Com member, Peter Stokes, because they stepped away from the marketing b.s. you see around “Scattered Spider”. Specifically, they called out how threat intel and cybercriminal investigations actually work, where you rely on the “bad guys” making OPSEC and privacy mistakes as soon as they lose interest in the problem space. They managed to nail Stokes using a GDID attached to his Windows O.S., and Microsoft helped correlate that ID with their web activity tied to Steam.
tl;dr: don’t play Steam games on the same operating system as your operational box :).
Defending SaaS-based applications against ShinyHunters OAuth abuse by Microsoft Security Research
Microsoft Security Research dropped a blog on TTP updates from ShinyHunters intrusions they’ve observed since 2025. The move to a supply chain compromise, paired with pure exfiltration, shows how effective this strategy can be without worrying about the ransom component. It certainly simplifies operations: you target SaaS environments and avoid EDRs altogether; you exploit nascent SaaS trust boundaries; and you rely on the lack of visibility into these attack paths to persist.
They certainly pitch their products as a way to showcase how they would protect against these attacks, but if you step away from the product pitch and focus on the solutions themselves, it all comes down to gaining visibility into third-party tooling. Unused, new, or risky applications granted via OAuth consent should be part of your risk calculation, and limiting the blast radius by removing these apps can really save you headaches if you fall victim to consent phishing or one of your vendors suffers a breach.
Hunting malware and malicious MCPs in memory on Kubernetes with FleetDM + Osquery + YARA by Ben Bornholm
TIL osquery can run YARA scans! Ben, a fellow RIT grad, created this step-by-step labs post to show how you can enroll devices into an osquery fleet and run YARA rules to scan for malware. He set up two scenarios to demonstrate the YARA plugin's scanning and alerting functionality. The first involved a K8S cluster with a Damn Vulnerable Web App container that he infected with Sliver, while the second involved a malicious MCP server.
The query is gnarly but effective: you left join your container queries with a remote, authenticated YARA rule and display the result if it finds a match:
From my experience with osquery, the thing you always have to worry about is latency causing problems by slowing endpoints down. YARA was purpose-built to be fast, so I’d be interested to see more research in this area and to hear how folks have deployed it at scale.
☣️ Threat Landscape
Alleged Member of Criminal Cyber Hacking Group “Scattered Spider” Arrested in Finland and Extradited to the United States by U.S. Department of Justice
Peter Stokes, an alleged member of The Com, was arrested in Finland and brought to the U.S. after being tied to several ransomware attacks. They racked up several charges across computer intrusion and fraud. The criminal complaint is unsealed, so you can go read it, but one thing I haven’t seen much in these announcements is that, about halfway down the blog, they tell a story about how Stokes failed to execute a ransom.
He was apparently involved in an intrusion into a luxury jeweler, and the security team at the firm managed to evict the threat actors from its network and prevent a major breach. It’s the first time I’ve seen a story about an intrusion's failure. They claimed it still cost $2 million in losses, but perhaps that was included to show that even carrying out these breaches can still result in a large loss for a company.
Compromised AsyncAPI npm packages: inside a CI supply-chain attack by Christophe Tafani-Dereeper, Eslam Salem and Sebastian Obregoso
My coworkers released a detailed campaign analysis where multiple asyncapi npm packages were compromised on July 14, with malicious versions published directly from the project’s own GitHub Actions workflow. A single commit injected obfuscated JavaScript into source files, resulting in four poisoned packages being published to npm. These four packages have a combined total of 3 million downloads per week.
This team always comes in the clutch with their timeline graphics, so here’s a great breakdown of everything they’ve found:
SpectrePaste by Joshua Platt and Jason Reaves
Walmart Threat Intel team were studying OSINT reports around DeepLoad campaigns and found a separate, previously undocumented PowerShell delivery system they call SpectrePaste. The fileless malware panel, internally labeled “PasteFast Panel” by the actors, has some impressive features that help operators manage large numbers of infections checking into the server without crashing it.
According to Platt and Reaves, the tool has AI-assisted development written all over it. But it’s not just things like clear Claude comments: they found several text files using Microsoft/GitHub’s spec-kit instruction sets. I linked spec-kit before, but you essentially use it to act as a product manager for your coding agents, and it’ll write code while checking back in with you on things like hitting milestones, getting clarification on features, and accepting changes.
NSA revives 'Tailored Access Operations' name for elite hacking unit by Martin Matishak
The National Security Agency’s famous “TAO” office has returned, according to the NSA Director. The announcement came last week and helps reunite “operators” and “developers”. This is NSA/government speak for security experts who specialize in the red team space and their counterparts who build and deploy tooling for the red teamers. I was always confused about why these groups were split off in the first place. I have a funny story about this group and Edward Snowden if y’all ever catch me in person at a meetup :).
🔗 Open Source
Detection Forge is an agent harness that ingests CTI reports and outputs detection rules in Sigma. It has several preprocessing steps to extract TTPs and IOCs, which then gets mapped to MITRE, and it iterates on building Sigma rules while self-correcting any errors that come along the way. After Sigma, it’ll convert to rule formats covered by Sigma converter backends, and it’ll open a P/R for review. Perhaps this would be a great candidate to do evals on using CTI-REALM from Microsoft above!
Chimera is a Rust-based sandbox for running untrusted code, specifically around security agents, to help reduce the blast radius of a malicious agent or a piece of malware being executed on your box. There are comparable projects like nono that perform similar functionality, but it looks like this implements a pseudo-hypervisor via a binary translator and converts them between architectures on the host and the guest.
Sighthound is a tree-sitter based static application security testing (SAST) tool similar to Semgrep or GitHub’s CodeQL. It has the features you would expect: rule writing, taint analysis and integration with GitHub Code Scanning. It looks like you need to purchase Corgea’s platform to get other features enabled such as the AI SAST components, but it’s good to see competitors in this space to give us all options.
UKGovernmentBEIS/inspect_evals
UK Government’s LLM evals repository where the Microsoft Security Research team contributed their CTI-REALM benchmark. There’s over 10 cybersecurity benchmarks and datasets ranging from capture the flag harnesses, vulnerability research and incident response.