Purple Team Exercise Methodology: How to Run Collaborative Red-Blue Simulations That Actually Improve Detection

A purple team exercise without a scoped threat actor profile devolves into a generic technique checklist that produces unfocused results. The most valuable exercises are threat-actor-specific: select the adversary most likely to target your organization and test your detection coverage against their documented TTPs.

Step 1: Select the threat actor profile. Use threat intelligence to identify the 1-3 threat actors most likely to target your sector, geography, and technology stack. Consult:

• MITRE ATT&CK Groups page: each documented group has ATT&CK technique mappings
• Your threat intelligence platform's sector-specific reporting
• CISA alerts for your sector
• ISACs (Information Sharing and Analysis Centers) for your industry

Step 2: Extract ATT&CK techniques for the selected actor. For each selected threat actor, export their documented technique list from ATT&CK Navigator. Filter by: techniques with documented procedures (meaning specific implementation details are available, not just theoretical coverage), techniques relevant to your environment (discard cloud-specific techniques if the actor targets on-premises), and techniques where your current detection coverage is unknown or low.

Step 3: Prioritize by detection value. Not all techniques are equally worth testing. Prioritize techniques that:

• Appear in the initial access, execution, and persistence phases (catching early-stage activity stops the intrusion)
• Are used by multiple threat actors against your sector (technique diversity in the checklist)
• Map to existing detection rules you want to validate (confirming working detections is as valuable as finding gaps)
• Your SOC team has no visibility data for (unknown coverage is highest priority)

Step 4: Map to atomic test cases. For each prioritized technique, identify executable test cases from the Atomic Red Team library (github.com/redcanaryco/atomic-red-team). Atomic Red Team provides specific commands, scripts, and tools for executing each ATT&CK technique. Each test case documents expected logs and artifacts for validation.

Selecting the right execution platform determines the range of techniques you can test and the level of documentation generated during the exercise.

The core purple team execution loop is: execute one technique, observe detection outcome, document result, tune if needed, move to next technique. This loop is repeated for each technique in the exercise scope.

Pre-execution checklist:

• All exercise systems identified and tagged in SIEM (prevents analysts from escalating exercise activity as a real incident)
• Change freeze in effect for exercise environment during the exercise window
• Blue team has SIEM access and is actively monitoring
• Snapshot or clean baseline taken for exercise machines (enables reset after destructive techniques)
• Communication channel established between red and blue teams (Slack channel or call bridge)

Execution sequence for each technique:

1. Red team pre-announces: Red team notifies blue team: "We are about to execute T1059.001 (PowerShell) at 10:15 on HOST-01 using the following command: [exact command]." Pre-announcing eliminates ambiguity about what generated which alert.
2. Execute: Red team executes the technique at the agreed time.
3. Blue team observes (5-minute window): Blue team notes whether an alert fired, which rule triggered it, how quickly it fired, and the alert quality (true positive, tuning needed, or false positive).
4. Document outcome: Record the technique ID, test case used, timestamp, expected artifacts, observed artifacts, alert fired (yes/no), alert quality (high/medium/low), and detection gap (if no alert).
5. Tune if time allows: If a detection gap is identified and a fix is obvious, blue team tunes the rule in real time and re-executes the technique to validate the fix.

Exercise tracking template fields:

Technique ID | Technique Name | Test Case | Timestamp |
Artifacts Expected | Artifacts Observed | Alert Fired | Rule Name |
Alert Quality | Gap Category | Remediation Action | Owner | Due Date

Detection gaps identified during the exercise fall into distinct categories, each with a specific remediation approach.

Category 1: Telemetry gap — No log data exists for this activity. The technique executed but produced no logs in the SIEM. Remediation: deploy additional telemetry collection (Sysmon, EDR policy adjustment, Windows audit policy change, network sensor coverage expansion). Without the underlying log data, writing detection rules is impossible.

Example: PowerShell Script Block Logging is disabled — T1059.001 execution leaves no Script Block event in the SIEM. Fix: Enable PowerShell Script Block Logging (Event ID 4104) and Module Logging (Event ID 4103) via Group Policy.

Category 2: Ingestion gap — Log data is generated but not reaching the SIEM. The source system generates the expected log (visible in local Event Viewer or endpoint agent) but it is not appearing in the SIEM. Remediation: fix log forwarding configuration, WEC subscription, or SIEM ingestion pipeline.

Category 3: Parsing gap — Log data is ingested but not correctly parsed. Raw log data arrives in the SIEM but fields are not extracted correctly, preventing field-based detection rules from matching. Remediation: fix or add SIEM parsing logic for the log source.

Category 4: Rule gap — Telemetry exists and is parsed but no detection rule covers it. All log data is available and searchable, but no detection rule exists for this technique. Remediation: write and deploy a new detection rule. Use SigmaHQ community rules as a starting point and adapt for your environment.

Category 5: Tuning gap — Rule exists but does not fire, or fires with too many false positives. A detection rule exists but is either suppressed due to false positive volume or has a logic error. Remediation: tune the rule logic, reduce suppression scope, or split the rule into a high-confidence variant and a lower-confidence informational variant.

Category 6: Analyst gap — Alert fires but is suppressed or not investigated. Alerts are generated but are buried in alert queue or auto-closed by triage automation. Remediation: adjust alert priority, SOC triage procedures, or automation suppression logic.

The primary output of a purple team exercise is a detection coverage delta — the measurable improvement in detection capability. Report these metrics:

Coverage metrics:

• Techniques tested: total count of ATT&CK techniques executed
• Detection rate (pre-exercise): percentage of techniques that generated a timely, accurate alert before any tuning during the exercise
• Detection rate (post-exercise): percentage detected after real-time tuning during the exercise
• Detection rate (30-day target): percentage committed to be detectable within 30 days, including fixes requiring telemetry deployment

Quality metrics:

• Mean time to detect (MTTD): average time from technique execution to alert generation
• Alert accuracy rate: percentage of fired alerts that were true positives vs. required tuning
• Telemetry gap count: number of techniques with no underlying log data (measures logging maturity, not detection maturity)

Gap distribution by ATT&CK tactic: Break down gaps by tactic (Initial Access, Execution, Persistence, Privilege Escalation, etc.) to identify which phases of the kill chain have the weakest detection coverage. Most organizations find execution and persistence have better coverage than discovery and lateral movement.

Trend tracking across exercises: Run exercises quarterly and track detection rate trends over time. The goal is a consistent upward trend. Flat or declining detection rate despite investment signals that the environment is growing faster than detection coverage — a scaling problem requiring architectural response.