Frontend Labs
⋯

Part 5 — Microservices and distributed authorization

G1 taught mTLS and OAuth for one client and one server. A microservice system has hundreds of services, each of which must authenticate its callers, authorize every request, propagate the end-user's context safely across many hops, and keep tenants isolated — all without trusting the network. This Part is how.

Chapter 23 — Service identity: SPIFFE and SPIRE

In one sentence

SPIFFE gives every workload a cryptographic identity (a URI in a certificate or JWT) that it obtains automatically from a local API with no secret to distribute, and SPIRE is the implementation that issues and rotates those identities based on attested facts about the workload.

Where we are

G1 Ch 31 covered mTLS with certificates you issued by hand. That does not scale to hundreds of ephemeral services. SPIFFE/SPIRE is how you give every pod, VM, and function a short-lived, automatically-rotated identity that downstream services and policy engines can verify.

How it works

   SPIFFE ID    a URI:  spiffe://securesh.op/ns/core/sa/orders-api
                one per workload TYPE. The trust domain (securesh.op) is
                the root of trust.

   SVID         "SPIFFE Verifiable Identity Document" -- the credential:
                X.509-SVID  a short-lived cert with the SPIFFE ID as a URI
                            SAN. Used for mTLS.
                JWT-SVID    a short-lived JWT with sub = the SPIFFE ID and
                            aud = the intended recipient. Used where you
                            can't do mTLS (through an L7 proxy, a queue).

   TRUST BUNDLE the CA certs / JWKS for a trust domain -> how a verifier
                checks an SVID. Federation = trust domains exchange
                bundles so `securesh.op` workloads can verify
                `partner.example` workloads.

   WORKLOAD API a local gRPC API over a Unix domain socket. A workload
                calls it to GET its SVID + the trust bundle, and to be
                notified on rotation. NO token or secret is needed to call
                it -- the fact that the process can open that socket, plus
                kernel-level attestation of who the process is, IS the
                authentication. This is the key trick: no bootstrap
                secret.

SPIRE — the implementation:

   SPIRE SERVER   the CA + registrar. Signs SVIDs. Holds REGISTRATION
     ENTRIES (selector-set -> SPIFFE ID mappings). Its CA can be
     self-managed or "upstream'd" to Vault / AWS Private CA / disk / an
     HSM. HA via a shared datastore.

   SPIRE AGENT    one per node. Two-step attestation:
     1. NODE ATTESTATION -- the agent proves what node it runs on to the
        server, using a platform mechanism:
          aws_iid (EC2 instance identity doc), gcp_iit, azure_msi,
          k8s_psat (a projected SA token the server validates against the
          cluster), join_token, x509pop, tpm_devid.
     2. WORKLOAD ATTESTATION -- when a local process calls the Workload
        API, the agent inspects it via plugins:
          unix (uid/gid/path/sha256), k8s (namespace/serviceaccount/
          pod-label/pod-name/image-id), docker.
        The resulting SELECTORS (e.g. k8s:ns:core, k8s:sa:orders-api) are
        matched against registration entries -> the agent hands the
        process the right SVID.

   REGISTRATION ENTRY example:
     spiffe ID : spiffe://securesh.op/ns/core/sa/orders-api
     parent ID : spiffe://securesh.op/spire/agent/k8s_psat/<cluster>/<node>
     selectors : k8s:ns:core, k8s:sa:orders-api, k8s:pod-label:app:orders-api
     ttl       : 1h   (SVIDs auto-rotate at ~half-life)

How workloads use it:

   - Directly: link the SPIFFE library (go-spiffe, py-spiffe, etc.), call
     the Workload API, get an X.509-SVID, use it for mTLS; the library
     validates the peer's SVID against the trust bundle.
   - Via a mesh: Istio, Linkerd, Cilium, and Consul use SPIFFE identities
     natively. Istio's `spiffe://<trust-domain>/ns/<ns>/sa/<sa>` is a
     SPIFFE ID; its Citadel/istiod is effectively a SPIFFE CA. You get
     mTLS + identity for free, and AuthorizationPolicy keys off
     `source.principals` (the SPIFFE ID).
   - Via SDS: the mesh's Envoy sidecars fetch SVIDs from a SPIRE agent
     over the Envoy Secret Discovery Service.

Why this matters for the rest of the Part: every subsequent chapter assumes a service can answer "who is calling me?" with a verified SPIFFE ID. That is the foundation for service-to-service authz (Ch 25), safe token exchange (Ch 24), and "reachable but refused" (Ch 3).

Worked example

web cannot impersonate bff even with network reach.

   SecureShop mesh, SPIFFE trust domain securesh.op.
   Registration entries:
     spiffe://securesh.op/ns/edge/sa/bff     <- selectors k8s:ns:edge,
                                                k8s:sa:bff
     spiffe://securesh.op/ns/edge/sa/web     <- k8s:ns:edge, k8s:sa:web
     spiffe://securesh.op/ns/core/sa/orders-api

   orders-api's AuthorizationPolicy (Istio):
     action: ALLOW
     rules:
       - from: [{ source: { principals:
                 ["spiffe://securesh.op/ns/edge/sa/bff"] } }]
         to:   [{ operation: { methods: ["POST"], paths: ["/orders"] } }]

   ATTACK: `web` is popped. Attacker wants to call orders-api directly.
     - `web`'s Envoy presents `web`'s X.509-SVID (it can't get bff's --
       the SPIRE agent attests the calling process; a process in the
       `web` pod matches k8s:sa:web selectors, not bff's).
     - orders-api's sidecar checks source.principal == bff -> it's `web`
       -> 403 RBAC: access denied. Logged with both identities.
     - The attacker cannot forge bff's SVID: no CA key, SVIDs are
       short-lived and node-attested, and the Workload API only gives a
       process the identity it actually attests to.
   Even with full network reach, `web`'s blast radius toward orders-api is
   zero without a valid end-user context routed through bff.

Practice (60 min) — kind cluster

  1. Deploy SPIRE (server + agent DaemonSet) with the k8s_psat node attestor. Create registration entries for three workloads.
  2. Deploy a tiny Go/Python client and server using go-spiffe / py-spiffe that fetch X.509-SVIDs from the Workload API and do mTLS. Confirm the server logs the client's SPIFFE ID.
  3. Try to get workload A's SVID from workload B's pod — confirm the agent gives B its own identity, not A's (attestation is per-process).
  4. Install Istio (or Linkerd) with SPIRE integration; enable strict mTLS; write an AuthorizationPolicy scoping orders-api to bff's principal. Call from web (403) and bff (200).
  5. Rotate: shorten the SVID TTL to 2 minutes; watch SVIDs auto-renew with no downtime.
  6. Set up SPIFFE Federation between two trust domains (two kind clusters) and verify a cross-domain mTLS call.

Build it in Go (40 min) — SPIFFE-style identity with only the standard library

SPIRE issues and rotates the certificates. What a service has to do is small and worth seeing in code: require mTLS, pull the SPIFFE ID out of the peer certificate's URI SAN, check the trust domain, and authorize by ID.

// spiffe: workload identity the SPIFFE way, with only the standard library.
// A trust domain's CA issues short-lived certificates whose identity is a URI
// SAN (spiffe://shop.example/ns/prod/sa/orders). Services authenticate each
// other with mTLS and authorize by that ID -- not by IP, not by hostname.
//
//	go run ./spiffe
package main

import (
	"crypto/ecdsa"
	"crypto/elliptic"
	"crypto/rand"
	"crypto/tls"
	"crypto/x509"
	"crypto/x509/pkix"
	"errors"
	"fmt"
	"io"
	"log"
	"math/big"
	"net"
	"net/http"
	"net/url"
	"slices"
	"strings"
	"time"
)

type ca struct {
	cert *x509.Certificate
	key  *ecdsa.PrivateKey
}

func newCA(name string) *ca {
	key, _ := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
	tmpl := &x509.Certificate{
		SerialNumber: serial(), Subject: pkix.Name{CommonName: name},
		NotBefore: time.Now().Add(-time.Minute), NotAfter: time.Now().Add(24 * time.Hour),
		IsCA: true, BasicConstraintsValid: true,
		KeyUsage: x509.KeyUsageCertSign,
	}
	der, _ := x509.CreateCertificate(rand.Reader, tmpl, tmpl, &key.PublicKey, key)
	c, _ := x509.ParseCertificate(der)
	return &ca{c, key}
}

// issue mints an SVID: a 1-hour certificate whose ONLY identity is the URI.
func (a *ca) issue(spiffeID string) tls.Certificate {
	key, _ := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
	u, _ := url.Parse(spiffeID)
	tmpl := &x509.Certificate{
		SerialNumber: serial(),
		NotBefore:    time.Now().Add(-time.Minute),
		NotAfter:     time.Now().Add(time.Hour), // short-lived: rotation replaces revocation
		URIs:         []*url.URL{u},
		KeyUsage:     x509.KeyUsageDigitalSignature,
		ExtKeyUsage:  []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth, x509.ExtKeyUsageClientAuth},
	}
	der, _ := x509.CreateCertificate(rand.Reader, tmpl, a.cert, &key.PublicKey, a.key)
	return tls.Certificate{Certificate: [][]byte{der}, PrivateKey: key}
}

// spiffeID extracts the workload identity from a verified peer certificate.
func spiffeID(cs tls.ConnectionState) (string, error) {
	if len(cs.PeerCertificates) == 0 || len(cs.PeerCertificates[0].URIs) != 1 {
		return "", errors.New("no SPIFFE ID")
	}
	u := cs.PeerCertificates[0].URIs[0]
	if u.Scheme != "spiffe" || u.Host != "shop.example" {
		return "", fmt.Errorf("foreign trust domain %q", u.Host)
	}
	return u.String(), nil
}

func main() {
	trust := newCA("shop.example trust domain")
	pool := x509.NewCertPool()
	pool.AddCert(trust.cert)

	// The payments service: mTLS required, then an allow-list of callers.
	allowed := []string{"spiffe://shop.example/ns/prod/sa/orders"}
	srv := &http.Server{
		Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
			id, err := spiffeID(*r.TLS)
			if err != nil || !slices.Contains(allowed, id) {
				http.Error(w, "forbidden: "+id, http.StatusForbidden) // authenticated, not authorized
				return
			}
			fmt.Fprintf(w, "charge accepted for caller %s", id)
		}),
		TLSConfig: &tls.Config{
			Certificates: []tls.Certificate{trust.issue("spiffe://shop.example/ns/prod/sa/payments")},
			ClientAuth:   tls.RequireAndVerifyClientCert, // no client cert, no handshake
			ClientCAs:    pool,
			MinVersion:   tls.VersionTLS13,
		},
		ErrorLog: log.New(io.Discard, "", 0), // the clients below report the outcome
	}
	ln, _ := net.Listen("tcp", "127.0.0.1:0")
	go srv.ServeTLS(ln, "", "")
	addr := "https://" + ln.Addr().String() + "/charge"

	call := func(label string, cert *tls.Certificate) {
		cfg := &tls.Config{RootCAs: pool, MinVersion: tls.VersionTLS13,
			// The client verifies the SERVER's SPIFFE ID too: mutual, not one-way.
			InsecureSkipVerify: true, // hostname checks don't apply to SPIFFE IDs...
			VerifyConnection: func(cs tls.ConnectionState) error { // ...so verify chain + ID ourselves
				opts := x509.VerifyOptions{Roots: pool, KeyUsages: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth}}
				if _, err := cs.PeerCertificates[0].Verify(opts); err != nil {
					return err
				}
				if id, err := spiffeID(cs); err != nil || id != "spiffe://shop.example/ns/prod/sa/payments" {
					return fmt.Errorf("unexpected server identity %q", id)
				}
				return nil
			},
		}
		if cert != nil {
			// Send the certificate unconditionally. (With cfg.Certificates, Go's
			// client quietly sends NOTHING if the cert isn't from a CA the server
			// asked for -- an attacker's client won't be so polite.)
			cfg.GetClientCertificate = func(*tls.CertificateRequestInfo) (*tls.Certificate, error) {
				return cert, nil
			}
		}
		c := &http.Client{Transport: &http.Transport{TLSClientConfig: cfg}, Timeout: 3 * time.Second}
		resp, err := c.Post(addr, "text/plain", nil)
		if err != nil {
			var ue *url.Error
			if errors.As(err, &ue) {
				err = ue.Err
			}
			fmt.Printf("%-40s -> rejected in the TLS handshake: %v\n", label, err)
			return
		}
		body, _ := io.ReadAll(resp.Body)
		resp.Body.Close()
		fmt.Printf("%-40s -> %d %s\n", label, resp.StatusCode, strings.TrimSpace(string(body)))
	}

	orders := trust.issue("spiffe://shop.example/ns/prod/sa/orders")
	reports := trust.issue("spiffe://shop.example/ns/prod/sa/reports")
	attacker := newCA("attacker CA").issue("spiffe://shop.example/ns/prod/sa/orders")

	call("orders (allowed ID)", &orders)
	call("reports (valid cert, wrong ID)", &reports)
	call("forged orders ID from another CA", &attacker)
	call("no client certificate", nil)
}

func serial() *big.Int {
	n, _ := rand.Int(rand.Reader, big.NewInt(1<<62))
	return n
}
$ go run ./spiffe
orders (allowed ID)                      -> 200 charge accepted for caller spiffe://shop.example/ns/prod/sa/orders
reports (valid cert, wrong ID)           -> 403 forbidden: spiffe://shop.example/ns/prod/sa/reports
forged orders ID from another CA         -> rejected in the TLS handshake: remote error: tls: unknown certificate authority
no client certificate                    -> rejected in the TLS handshake: remote error: tls: certificate required

What to notice:

  • Authentication and authorization are separate outcomes. reports has a perfectly valid SVID from our trust domain, so the handshake succeeds, and it still gets a 403 because it isn't on the allow-list. A mesh that only "turns on mTLS" gives you the first half.
  • Identity is the URI, not the hostname or IP. That's why the client sets InsecureSkipVerify and re-implements verification in VerifyConnection: hostname checks don't apply to SPIFFE IDs, but chain verification and the ID check absolutely must. Copying only the first line of that pattern is a serious bug.
  • A finding from building this lab: given a certificate from a CA the server didn't ask for, Go's TLS client silently sends no certificate, so the server sees "certificate required" rather than "unknown authority". The lab uses GetClientCertificate so the forged certificate is really presented, as an attacker's client would.
  • One-hour lifetimes stand in for revocation, exactly as with SPIRE.

Exercises:

  1. Load the trust bundle and SVID from files and reload them on change (rotation every 30 minutes), using the GetCertificate pattern from the TCP/IP guide's certreload lab.
  2. Replace the allow-list with a call to the Chapter 25 policy engine: check("api:payments#caller@spiffe://...").
  3. Compare with github.com/spiffe/go-spiffe/v2, which does the same against a live SPIRE agent's Workload API.

Common mistakes

  • Treating SPIFFE IDs as secrets. They're identifiers, published freely. The SVID (the signed credential) is what matters, and it's short-lived.
  • Loose registration selectors. k8s:ns:core alone means any SA in core gets that identity. Add k8s:sa: (and label/image where useful).
  • Long SVID TTLs. Defeats the "leaked credential dies fast" property. Keep TTL short (minutes to an hour); rotation is automatic.
  • Upstreaming the SPIRE CA to nothing. Its CA key is the trust domain. Back it with Vault/AWS PCA/HSM, protect the datastore, and plan CA rotation.
  • mTLS without authorization. SPIFFE identity tells you who; you still need an AuthorizationPolicy / policy engine to decide what they may do (Ch 25).
  • Skipping node attestation strength. join_token is fine for a lab; use the platform attestor (aws_iid/gcp_iit/k8s_psat) in production.

Check yourself

  1. What is a SPIFFE ID vs an SVID, and why can the Workload API be called with no secret?
  2. Describe SPIRE's two-step attestation and give one node and one workload attestor.
  3. What's in a registration entry, and what goes wrong if the selectors are too loose?
  4. Why are short SVID TTLs a security property, and what makes them practical?
  5. How does a mesh like Istio use SPIFFE identities?
  6. SPIFFE identity establishes who; what still has to establish what they may do?

(Answers: Appendix G.)

Further reading

  • Spec: spiffe.io — the SPIFFE and SPIRE docs; the SPIFFE ID, SVID, and Workload API specs; SPIFFE Federation.
  • Book: Solving the Bottom Turtle (free, spiffe.io) — the "how do you bootstrap trust" problem and SPIFFE's answer.
  • Integration: Istio "Identity and certificate management", Linkerd "mTLS", the SPIRE + Istio and SPIRE + Envoy SDS guides.
  • Talks: "SPIFFE/SPIRE deep dive" from KubeCon; "Zero Trust workload identity" talks.

Generated from real-life-security-guide-v1.md · 8 Oct 2026 16:36