Data Encryption at Rest and in Transit: A Practitioner's Guide

Understanding when to use symmetric vs. asymmetric encryption is the foundation for sound encryption architecture.

Symmetric encryption (same key to encrypt and decrypt):

• Algorithm of choice: AES-256-GCM (authenticated encryption -- provides both confidentiality and integrity)
• Use for: encrypting data at rest (files, database columns, disk), bulk data encryption in transit
• Key management challenge: both parties need the same key, distributed securely
• Performance: very fast -- hardware AES acceleration is standard in modern CPUs

Asymmetric encryption (public/private key pair):

• Algorithm of choice: RSA-4096 or ECDSA P-384 (ECDSA preferred for performance at equivalent security)
• Use for: key exchange, digital signatures, TLS handshake, email encryption (S/MIME)
• Performance: orders of magnitude slower than symmetric -- never use to encrypt bulk data
• Key management: public key can be freely distributed; private key must be protected

The practical pattern -- hybrid encryption:

Most encryption in practice combines both:

1. Generate a random symmetric key (AES-256)
2. Encrypt the data with the symmetric key
3. Encrypt the symmetric key with the recipient's RSA or EC public key
4. Transmit/store the encrypted data + encrypted key

The recipient decrypts the symmetric key using their private key, then decrypts the data. This is exactly how TLS and PGP work. It is also the pattern behind cloud KMS envelope encryption.

Algorithm summary:

Database encryption approaches:

Transparent Data Encryption (TDE):

TDE encrypts the database files on disk -- the database engine decrypts data transparently as it reads and writes. If an attacker steals the database files without the encryption key, the data is unreadable.

• SQL Server: Enable TDE with a database encryption key (DEK) protected by a certificate in the master database. Use Azure Key Vault to protect the certificate.
• PostgreSQL: Use pgcrypto extension for column-level encryption; full-disk TDE via the OS layer (LUKS, dm-crypt) or cloud storage encryption
• MySQL/MariaDB: InnoDB Tablespace Encryption (INNODB_DEFAULT_ROW_FORMAT)

Column-level encryption (application-layer):

For fields with the highest sensitivity (SSNs, credit cards, health data), encrypt at the application layer before writing to the database. The database stores ciphertext and cannot query on the encrypted values directly.

# Python example using cryptography library with AES-256-GCM
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
import os

def encrypt_field(plaintext: str, key: bytes) -> bytes:
    aesgcm = AESGCM(key)
    nonce = os.urandom(12)  # 96-bit nonce for GCM
    ciphertext = aesgcm.encrypt(nonce, plaintext.encode(), None)
    return nonce + ciphertext  # prepend nonce for storage

def decrypt_field(ciphertext_with_nonce: bytes, key: bytes) -> str:
    aesgcm = AESGCM(key)
    nonce = ciphertext_with_nonce[:12]
    ciphertext = ciphertext_with_nonce[12:]
    return aesgcm.decrypt(nonce, ciphertext, None).decode()

File system encryption:

• Linux: LUKS/dm-crypt for full-disk; eCryptFS or fscrypt for per-directory encryption
• macOS: FileVault 2 (XTS-AES-128 -- AES-256 at hardware level via T2/M-series chip)
• Windows: BitLocker (AES-256 XTS mode) -- require TPM 2.0 + PIN for strongest protection

Cloud storage encryption:

All major cloud providers encrypt storage at rest by default with keys they manage. For stronger assurance, use Customer-Managed Keys (CMK):

• AWS S3: SSE-KMS with CMK; enforce with bucket policy requiring aws:RequestedEncryptionAlgorithm: "aws:kms"
• Azure Blob: Customer-Managed Keys via Azure Key Vault
• GCP Cloud Storage: CMEK with Cloud KMS

CMK gives you control over key rotation and the ability to immediately revoke access by disabling the key.

Envelope encryption is the standard pattern for enterprise key management. It solves the circular problem of "where do I store the encryption key?"

The pattern:

Data --> [Encrypt with Data Encryption Key (DEK)] --> Encrypted Data
DEK  --> [Encrypt with Key Encryption Key (KEK) in KMS] --> Encrypted DEK

Store: Encrypted Data + Encrypted DEK (together in database or object storage)
KMS stores: KEK (never leaves the KMS HSM boundary)

To decrypt: call KMS to decrypt the DEK using the KEK (requires IAM permission); use the plaintext DEK to decrypt the data; DEK exists in memory only for the duration of the operation.

AWS KMS envelope encryption implementation:

import boto3
import base64
from cryptography.hazmat.primitives.ciphers.aead import AESGCM

kms_client = boto3.client('kms', region_name='us-east-1')
KEY_ID = 'arn:aws:kms:us-east-1:123456789:key/mrk-xxxxx'

def encrypt_data(plaintext: str) -> dict:
    # Generate a data key (returns plaintext DEK + encrypted DEK)
    response = kms_client.generate_data_key(
        KeyId=KEY_ID,
        KeySpec='AES_256'
    )
    dek_plaintext = response['Plaintext']
    dek_encrypted = response['CiphertextBlob']

    # Encrypt data with the plaintext DEK
    aesgcm = AESGCM(dek_plaintext)
    nonce = os.urandom(12)
    ciphertext = aesgcm.encrypt(nonce, plaintext.encode(), None)

    return {
        'encrypted_dek': base64.b64encode(dek_encrypted).decode(),
        'nonce': base64.b64encode(nonce).decode(),
        'ciphertext': base64.b64encode(ciphertext).decode()
    }

Key hierarchy best practices:

• Master key (CMK): Lives in KMS HSM; never exported; rotated annually
• Data Encryption Keys (DEKs): Generated per object or per tenant; rotated per the sensitivity of the data
• Key context (encryption context): Use KMS encryption context to bind the DEK to a specific resource -- prevents a stolen encrypted DEK from being decrypted in a different context

Cloud KMS comparison:

TLS is the primary encryption mechanism for data in transit. Misconfigured TLS -- using deprecated versions, weak ciphers, or no certificate validation -- negates the protection entirely.

TLS version requirements:

Disable TLS 1.0 and 1.1 everywhere. TLS 1.3 is the target -- it eliminates weak cipher negotiation and requires Perfect Forward Secrecy (PFS) by design.

Nginx TLS 1.3 configuration:

server {
    listen 443 ssl http2;

    ssl_protocols TLSv1.3 TLSv1.2;         # allow 1.2 for older clients; prefer 1.3
    ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305';
    ssl_prefer_server_ciphers on;

    ssl_certificate /etc/ssl/certs/server.crt;
    ssl_certificate_key /etc/ssl/private/server.key;

    # HSTS: force HTTPS for 1 year including subdomains
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always;

    # OCSP Stapling
    ssl_stapling on;
    ssl_stapling_verify on;
    ssl_trusted_certificate /etc/ssl/certs/chain.pem;

    # DH parameters (for TLS 1.2 DHE suites)
    ssl_dhparam /etc/ssl/dhparam4096.pem;

    # Session resumption
    ssl_session_cache shared:SSL:50m;
    ssl_session_timeout 1d;
    ssl_session_tickets off;   # disable for PFS: session tickets use a static key
}

Certificate requirements:

• Key size: RSA 4096-bit or ECDSA P-384 minimum
• Signature algorithm: SHA-256 or better (SHA-1 certificates are rejected by all major browsers)
• Certificate Transparency: required by Chrome for all publicly-trusted certificates
• OCSP stapling: reduces revocation check latency; required in some compliance frameworks
• CAA DNS records: restrict which CAs can issue certificates for your domain

Internal service-to-service mTLS:

For internal microservice communication, use mutual TLS (mTLS) where both client and server present certificates. Service mesh platforms (Istio, Linkerd) automate mTLS for Kubernetes workloads -- enabling transparent mTLS without modifying application code.

Encryption is only as strong as your key management. Keys that never rotate, keys stored insecurely, and keys without access controls undermine even the strongest algorithms.

Key rotation requirements:

Automated key rotation in AWS KMS:

# Enable automatic annual rotation on a KMS key
aws kms enable-key-rotation --key-id arn:aws:kms:us-east-1:123456789:key/mrk-xxxxx

# Verify rotation is enabled
aws kms get-key-rotation-status --key-id arn:aws:kms:us-east-1:123456789:key/mrk-xxxxx

KMS keeps all previous key versions to decrypt data encrypted with older keys -- you do not need to re-encrypt all data during rotation. New data is encrypted with the current key version.

Cryptographic agility:

Design your encryption implementation so the algorithm can be swapped without rearchitecting the system. Include an algorithm identifier with stored ciphertext:

{
  "alg": "AES-256-GCM",
  "kms_key_id": "arn:aws:kms:...",
  "encrypted_dek": "base64...",
  "nonce": "base64...",
  "ciphertext": "base64..."
}

When you need to migrate to a new algorithm (post-quantum transition is coming), you can read old data with the old alg field and re-encrypt with the new algorithm during background processing, without a hard cutover.

Detecting encryption gaps:

Use DSPM (Data Security Posture Management) tools like Varonis, Cyera, or Normalyze to discover unencrypted sensitive data across your environment. These tools crawl cloud storage, databases, and file shares -- finding S3 buckets with PII that are not encrypted with CMKs, databases with sensitive columns stored in plaintext, and developer laptops with unencrypted backups.

Understanding which encryption controls each compliance framework requires prevents over-engineering (encrypting everything with HSMs when a software key store is compliant) and under-engineering (missing a required control that shows up in an audit).

PCI DSS 4.0 encryption requirements:

• Requirement 3.5: PAN must be rendered unreadable anywhere it is stored (AES-256, RSA-2048+, or equivalent)
• Requirement 4.2.1: Strong cryptography must be used on all cardholder data in transit (TLS 1.2 minimum, TLS 1.3 preferred)
• Requirement 3.7: Cryptographic key management procedures including key generation, distribution, storage, retirement, and replacement

HIPAA encryption requirements:

• HIPAA does not mandate specific algorithms -- it requires "reasonable and appropriate" encryption for ePHI
• HHS guidance recommends NIST-approved algorithms (AES-256, TLS 1.2+, SHA-256)
• Encrypted data that is stolen is not a reportable breach under the Safe Harbor provision (45 CFR 164.402)
• Key management must include documented procedures for key access and rotation

GDPR encryption:

• Article 32 requires "appropriate technical measures" including encryption -- algorithm specifics are not mandated
• Pseudonymization (encryption that can be reversed with the right key) counts as a technical safeguard
• Data breaches involving properly encrypted data may not require notification (Recital 83: "rendering the data unintelligible to any person not authorized to access it")

FedRAMP / FIPS 140-3:

• All cryptographic modules must be FIPS 140-3 validated
• AES-256 required; AES-128 does not meet FedRAMP High
• TLS 1.2 minimum with FIPS-approved cipher suites only; TLS 1.3 (FIPS mode) preferred
• Cloud HSMs (AWS CloudHSM, Azure Managed HSM, GCP Cloud HSM) provide FIPS 140-3 Level 3 validation

SOC 2 Type II:

• Encryption is mapped to CC6.1 (Logical and Physical Access Controls) and CC9.2 (Risk Mitigation)
• Auditors typically look for: encryption of data at rest for sensitive systems, TLS for data in transit, key management procedures, evidence of certificate rotation