Security Logging Best Practices for SIEM, Compliance, and Forensics
Incident investigation without logs is archaeology: piecing together what happened from fragmentary evidence in system state. Organizations that discover a breach and find that critical logs were not collected, were overwritten after 7 days, or were not synchronized to a central platform face a severely limited ability to determine scope, root cause, and attacker activity. Security logging strategy determines your investigative capability before an incident happens. The goal is collecting the logs that matter for detection and investigation, retaining them appropriately, and making them searchable at the speed incident response requires.
What to Log: Priority Sources
Not all logs are equally valuable for security. Prioritize collection in this order:
Authentication and identity events
The highest-value log source for most security investigations. Windows: Event IDs 4624 (successful logon), 4625 (failed logon), 4648 (logon using explicit credentials), 4720 (account created), 4728/4732/4756 (user added to privileged group), 4771/4776 (Kerberos/NTLM failures). Linux: /var/log/auth.log or /var/log/secure. IdP authentication logs (Okta System Log, Entra ID Sign-in Logs). Retain all authentication logs for minimum 12 months.
Privileged activity
All actions performed by administrative accounts. Windows: Event ID 4672 (special privileges assigned at logon), 4688 with command line logging enabled (process creation), 4698/4702 (scheduled task creation/modification). Linux: auditd logs for sudo execution, su, and privileged command execution. PAM vault session logs. Cloud: CloudTrail/Activity Log events by privileged IAM roles.
Network traffic metadata
DNS query logs (all queries and responses), firewall/proxy connection logs (source IP, destination IP, port, bytes transferred, action taken), NetFlow or IPFIX from core network infrastructure. Full packet capture is ideal but resource-intensive; enriched flow metadata provides most investigative value at fraction of the storage cost.
Endpoint process execution
Enable Windows Sysmon (System Monitor) for detailed process creation, network connection, and file creation events with parent-child process relationships. Configure auditd on Linux for process execution and privilege escalation. EDR platforms provide this automatically; ensure EDR telemetry feeds your SIEM for correlation.
Cloud API activity
AWS CloudTrail (all management events, S3 data events for sensitive buckets), Azure Activity Log and Entra ID audit logs, GCP Cloud Audit Logs (Admin Activity, Data Access). These logs are the equivalent of authentication and privileged activity logs for cloud infrastructure. Retain for minimum 12 months, 1 year for regulated environments.
Windows Event Log Configuration
Default Windows audit policy does not log many security-relevant events. Configure Advanced Audit Policy via Group Policy to enable:
Process creation with command line
Account Management > Audit Process Creation: Success. Computer Configuration > Administrative Templates > System > Audit Process Creation > Include command line in process creation events: Enabled. This single setting dramatically increases investigation capability by capturing what commands were executed, not just which processes ran.
PowerShell script block logging
Enables logging of PowerShell script content before obfuscation is applied: Computer Configuration > Administrative Templates > Windows Components > Windows PowerShell > Turn on PowerShell Script Block Logging: Enabled. Also enable Module Logging and Transcription for comprehensive PowerShell visibility. Attackers rely heavily on PowerShell; these logs are essential for post-compromise investigation.
Security event log size
Default Windows Security event log size (20 MB) is insufficient and will overwrite rapidly on active systems. Set to minimum 1 GB for domain controllers and servers: Computer Configuration > Windows Settings > Security Settings > Event Log > Maximum security event log size.
Sysmon deployment
Deploy Microsoft Sysinternals Sysmon using the SwiftOnSecurity or olafhartong Sysmon configuration as a baseline. Sysmon provides network connection events, file creation events, registry modification events, and DNS query events that Windows native logging does not. Ship Sysmon logs to your SIEM for correlation.
Briefings like this, every morning before 9am.
Threat intel, active CVEs, and campaign alerts, distilled for practitioners. 50,000+ subscribers. No noise.
Log Retention Requirements
Retention requirements are driven by both investigation needs and compliance obligations:
Forensic investigation baseline
CISA recommends 30-day hot (immediately searchable) and 1-year cold (archival) retention. The SolarWinds breach had a 9-month dwell time before discovery: organizations with less than 9 months of log retention could not fully reconstruct attacker activity. For environments facing sophisticated adversaries, 12 to 24 months of retention is appropriate.
Compliance-driven retention
PCI DSS: 12 months (3 months immediately available). HIPAA: 6 years for audit logs. SOX: 7 years for financial systems logs. GDPR: retain only as long as necessary for the stated purpose; security logs typically fall under legitimate interest with a documented retention policy. Align your log retention policy with the most stringent applicable compliance requirement.
Cost-optimized tiering
Hot storage (immediately searchable, SIEM-indexed): 30 to 90 days for most log sources, 90 days for authentication and privileged activity logs. Warm storage (retrievable within hours): 90 to 365 days. Cold archive storage (retrievable within 24 hours): 1 to 7 years for compliance-driven sources. Cost per GB drops dramatically from hot to cold; tier appropriately rather than retaining everything in hot storage.
Log Quality: What Makes Logs Useful
Log collection is necessary but not sufficient. Log quality determines investigative value:
Consistent timestamps and NTP synchronization
All log sources must use synchronized time (NTP). Unsynchronized clocks make cross-source correlation impossible: an authentication event at 14:03:22 on System A and a network connection at 14:03:19 on System B (a 3-second difference due to clock drift) may be the same attack chain. Verify NTP sync for every log source.
Source identification
Every log event must unambiguously identify its source: hostname, IP address, and ideally asset ID from your CMDB. IP addresses change with DHCP; ensure your logging infrastructure captures the hostname at time of logging rather than relying on post-hoc IP-to-hostname resolution.
Structured logging for applications
Application logs should be structured (JSON format) rather than free-text. Structured logs are indexable, filterable, and parseable by SIEM without custom parsing rules. Log: timestamp, event type, user identity, source IP, target resource, action, outcome, and any relevant context. Avoid logging sensitive data (passwords, session tokens, PII) in application logs.
Log integrity protection
Logs that can be modified by an attacker who compromised the system have limited forensic value. Forward logs to a centralized platform (SIEM, syslog server) immediately: an attacker who compromises a server cannot modify logs already forwarded. Consider log signing for high-value sources where tamper evidence is required.
What Not to Log
Over-logging creates storage costs and SIEM noise that reduces the quality of security monitoring. Avoid: verbose application debug logs in production SIEM (collect them separately for application troubleshooting), health check and load balancer probe requests that generate thousands of routine log entries per hour, fully successful bulk read operations with no anomaly (reading 1,000 records in a data export workflow that runs every hour is not a security event), and sensitive data in logs (passwords, session tokens, credit card numbers, SSNs). Establish a log source tier list that defines which sources feed your SIEM (security-relevant) versus which go to application logging infrastructure only.
The bottom line
Security logging is an investment you make before an incident that determines your investigative capability during one. Enable process creation with command line on Windows, deploy Sysmon, ship IdP authentication logs and cloud audit trails to your SIEM with 12 months of retention, and synchronize all clocks. These changes cost less than a few hours of engineering time and dramatically improve your ability to reconstruct attacker activity.
Frequently asked questions
How much log data should we expect to generate and store?
Log volume varies enormously by environment. Rough estimates for planning: a Windows domain controller generates 1 to 5 GB of security event logs per day with standard auditing enabled. A busy web server generates 5 to 20 GB of access logs per day. Sysmon-enabled workstations generate 0.5 to 2 GB per day. Cloud API logs (CloudTrail) for a moderately active AWS account generate 1 to 10 GB per day. Run a 30-day volume measurement from your actual environment before sizing SIEM storage, as these estimates vary by an order of magnitude across different environments.
What is the difference between SIEM and a log management platform?
A log management platform (Splunk, Elastic, Graylog, Loki) stores, indexes, and makes logs searchable. A SIEM (Security Information and Event Management) adds correlation rules, alert generation, and security-specific analytics on top of log management. Some platforms serve both functions (Splunk can be used as log management or as a SIEM depending on configuration). For security operations, you need both functions: log storage and search for investigation, and correlation and alerting for real-time detection. These may be delivered by a single platform or separate tools.
Which Windows Event IDs are most important for security monitoring?
The highest-priority Windows Event IDs for SIEM monitoring: 4625 (failed logon - brute force indicator), 4648 (logon with explicit credentials - lateral movement indicator), 4672 (privileged logon - admin activity), 4688 with command line (process creation - detects malicious tools), 4698/4702 (scheduled task creation/modification - persistence), 4720 (user account created), 4728/4732/4756 (user added to privileged group), 7045 (new service installed - persistence/rootkit indicator), and 1102 (audit log cleared - attacker covering tracks). Prioritize building detections on these before expanding to lower-signal event IDs.
Should we log DNS queries and why?
Yes. DNS logging is one of the highest-value, lowest-cost log sources for security. DNS queries reveal: C2 beaconing (unusual query patterns, DGA-generated domain names), data exfiltration via DNS tunneling, malware communication with known-malicious domains (correlate against threat intelligence), and internal reconnaissance via reverse DNS lookups. Enable DNS query logging on your DNS resolvers (Windows DNS debug logging, BIND query logging, Unbound logging). Forward DNS logs to your SIEM. The query volume is high but the events themselves are small; DNS logging typically adds 5 to 15 GB per day per organization.
How do we handle GDPR compliance when logging user activity?
GDPR applies to logging of personal data, which includes usernames, email addresses, and IP addresses of EU residents. Security monitoring falls under the legitimate interests lawful basis (Article 6(1)(f)): maintaining security is a legitimate interest that justifies processing personal data in logs. Document your legitimate interest assessment in your Records of Processing Activities (ROPA). Minimize personal data in logs where possible: log user IDs rather than full names where the ID is sufficient for investigation. Apply the same retention policies to security logs as to other personal data: do not retain security logs containing EU personal data beyond what is necessary for the documented security purpose.
What log sources are most commonly missing in security programs?
The most commonly absent high-value log sources: DNS query logs (collected by fewer than 40 percent of organizations despite being free and high-value), PowerShell script block logs (requires GPO configuration, often not done), cloud management plane logs (CloudTrail/Activity Log not forwarded to SIEM), IdP authentication logs (Okta/Entra ID logs not integrated with SIEM), and network device syslog (routers and switches with syslog not configured or not forwarded). Audit your SIEM for these sources: their absence is a detection gap.
Sources & references
- NIST SP 800-92 Guide to Computer Security Log Management
- CISA Logging Made Easy
- NSA Cybersecurity Advisory - Logging Best Practices
- Microsoft Security Event Log Reference
- CIS Controls v8 - Audit Log Management
Free resources
Critical CVE Reference Card 2025–2026
25 actively exploited vulnerabilities with CVSS scores, exploit status, and patch availability. Print it, pin it, share it with your SOC team.
Ransomware Incident Response Playbook
Step-by-step 24-hour IR checklist covering detection, containment, eradication, and recovery. Built for SOC teams, IR leads, and CISOs.
Get threat intel before your inbox does.
50,000+ security professionals read Decryption Digest for early warnings on zero-days, ransomware, and nation-state campaigns. Free, weekly, no spam.
Unsubscribe anytime. We never sell your data.
Founder & Cybersecurity Evangelist, Decryption Digest
Cybersecurity professional with expertise in threat intelligence, vulnerability research, and enterprise security. Covers zero-days, ransomware, and nation-state operations for 50,000+ security professionals weekly.
The Mythos Brief is free.
AI that finds 27-year-old zero-days. What it means for your security program.