Every day, security analysts investigate potential breaches in organizations’ digital infrastructure. Security Information and Event Management (SIEM) systems facilitate this by collecting and analysing security data in real time. Using analytics and automation, SIEMs help analysts detect threats and filter irrelevant information. At its core, SIEM systems rely on detection rules to analyse data. When a rule identifies a suspicious event, an alert is generated, drawing the attention of an analyst. These alerts form the basis of investigations, meaning that if an adversary evades a rule, a breach may go unnoticed.

To address this, we propose leveraging a transformer model to tokenize and embed executed PowerShell commands and PowerShell SIEM detection rules, enabling computation of cosine similarity between them. This allows for identification of evasions and determining which rules were evaded. Additionally, we introduce a regressor-based metamodel that selects the most suitable Large Language Model (LLM) from a pool and provides recommendations to improve rule coverage.

Using this approach, we found that at least 24% of the analysed rules in the 2025 SIGMA dataset were evadable. Our system detected 71% of hand-crafted evasions with a False Positive Rate (FPR) below 1% and correctly attributed 98% of the detected evasions to their corresponding detection rules, advancing the state-of-the-art system AMIDES. By selecting the most suitable LLM, the metamodel recommendations successfully updated 70% of the rules to detect previously unknown evasions. Additionally, security analysts, when presented with recommendations from our LLM-based metamodelling process reported that 31% of the recommendations could replace original rules with minor adjustments, while 47% offered key insights for manual refinement.