Chapter 54 — Project 2: harden a server end to end

Brief

Take a single Linux host running SecureShop's web/API tier and harden it from a default install to something you'd defend in a review — OS, network, service, container, secrets, logging — measuring before and after against a benchmark.

Starting point

A fresh VM (Debian/Ubuntu or a RHEL derivative) running the SecureShop app in a container, with the Project 1 certificates. Snapshot it first so you can re-run.

Tasks

   1. BASELINE (measure first)
      - Run a CIS Benchmark scan: `lynis audit system`, and
        OpenSCAP with the CIS or STIG profile
        (`oscap xccdf eval --profile cis ...`).
      - Record the score and the top 20 findings.
      - nmap the box from outside: every open port -> justify or close.

   2. OS / HOST
      - Patch; enable unattended security updates.
      - Remove unused packages and services; disable IPv6 only if truly
        unused (else configure it).
      - Non-root service user; no login shells for service accounts.
      - sudo: least privilege, logged; no NOPASSWD for humans.
      - Kernel hardening: sysctl (rp_filter, tcp_syncookies,
        kptr_restrict, dmesg_restrict, disable unused protocols),
        `nf_conntrack` limits.
      - Filesystem: separate/`noexec,nodev,nosuid` on /tmp, /var/tmp,
        /dev/shm; auditd installed.
      - MAC: AppArmor/SELinux in enforcing mode for the service.
      - Time sync (chrony) -- needed for logs (Ch 50) and certs.

   3. SSH (Part 8)
      - Key-only, no root, no passwords, AllowGroups, modern
        Kex/Ciphers/MACs, reachable only from the bastion subnet.
      - `ssh-audit` -> clean.

   4. NETWORK (Part 9)
      - nftables default-deny inbound: 443 from anywhere, 22 from bastion.
      - Default-deny OUTBOUND: allow only DNS to your resolver, 443 to
        package mirrors + your APIs + monitoring. Log drops.
      - fail2ban or nftables rate-limit on 22/443.

   5. SERVICE / APP
      - Run as non-root in a container: read-only rootfs, `--cap-drop ALL`
        (add back only what's needed), `--security-opt no-new-privileges`,
        a seccomp profile, a tmpfs for scratch, resource limits.
      - Distroless/minimal base image, pinned by digest (Ch 47).
      - TLS via Project 1 certs; HSTS; security headers (Ch 41/43);
        server tokens/version banners off.
      - Healthcheck + auto-restart; graceful shutdown.

   6. SECRETS (Ch 49)
      - No secrets in the image, env file, or compose file. App pulls from
        a secrets manager using a host/workload identity.
      - Secret scan the build context.

   7. LOGGING (Ch 50)
      - Structured app audit logs + auditd + nftables drops shipped
        off-host over TLS to a central store the app host can't purge.
      - 3 detections with runbooks (e.g. outbound-drop spike, SSH auth
        anomaly, app authz-deny spike).

   8. RE-MEASURE
      - Re-run lynis / OpenSCAP -> new score; diff the findings.
      - Re-run external nmap -> attack surface delta.
      - Confirm the app still works end to end (don't harden it into a
        brick -- keep a functional smoke test).

Deliverable

~/sec-lab/projects/p2-harden/ with: the config (as scripts / Ansible / a Dockerfile + compose or systemd units — infrastructure as code, so it's repeatable), before.md and after.md scan outputs, and report.md with the score delta, the surface delta, each change and its rationale (mapped to a CIS control or a chapter), and anything you deliberately did not do (with why).

Definition of done

• Lynis/OpenSCAP score materially improved; you can explain every remaining finding (fix, accept-with-reason, or false positive).
• External nmap shows only 443 (and 22 from the bastion).
• Outbound is default-deny with a justified allowlist.
• Container runs non-root, read-only, least-capability, seccomp-confined.
• No secret is present in the image or compose/env files.
• Logs reach a store the host cannot tamper with; 3 detections fire in test.
• The app still passes a full functional smoke test.
• The whole build is reproducible from code in the repo.

Rubric (/100)

   Measured before/after with a real benchmark .. 15
   OS + kernel + MAC hardening ................. 20
   Network (in + egress) + SSH ................. 20
   Container/service hardening ................. 20
   Secrets handled correctly .................. 10
   Logging + working detections ............... 10
   IaC / reproducibility + report ............. 5