Chapter 20 — Signals: the OS's tap on the shoulder
In one sentence
A signal is a small, asynchronous notification the kernel (or another process) can send to a process — "please stop," "please reload your config," "you just tried to divide by zero" — and how a process chooses to handle (or ignore) each one determines whether it shuts down gracefully or just vanishes.
Why this matters
Every graceful shutdown, every "why didn't my cleanup code run," and every container orchestrator's stop sequence is built entirely on signals.
How it actually works
A SIGNAL is a number, delivered to a process, that INTERRUPTS its
normal execution to run a handler (either a DEFAULT action the
kernel provides, or a CUSTOM one your program registered). Common
ones:
SIGTERM (15) "please terminate" -- the POLITE request. A
well-behaved program catches this, cleans up
(closes files, finishes in-flight work, flushes
buffers), and exits on its own terms. This is what
`kill <pid>` sends by DEFAULT (no flag needed).
SIGKILL (9) "terminate NOW, no negotiation." The kernel
forcibly ends the process -- it CANNOT be caught,
blocked, or ignored by the program (this is by
design: it's the guaranteed last resort). This is
what `kill -9 <pid>` sends. No cleanup code runs.
SIGINT (2) "interrupt" -- sent when you press Ctrl-C in a
terminal. Catchable, like SIGTERM.
SIGHUP (1) historically "the terminal hung up"; commonly
repurposed by daemons to mean "reload your
configuration without restarting."
SIGSEGV (11) "you accessed invalid memory" (Chapter 11's
segfault) -- delivered BY THE KERNEL when it
detects this, not sent deliberately by anyone.
SIGCHLD (17) "one of your child processes changed state" (e.g.
exited) -- how a parent process is notified without
having to constantly poll (Chapter 3's wait()).
A PROCESS CAN, for most signals: use the DEFAULT action (often
"terminate," sometimes "ignore," depending on the signal), install a
CUSTOM HANDLER (run specific code when it arrives), or explicitly
IGNORE it. SIGKILL and SIGSTOP are the two exceptions -- deliberately
un-catchable and un-ignorable, so there's always a way to force a
process to stop, no matter how badly it's misbehaving.
Real-world example
Kubernetes' pod termination sequence (documented in detail in
Kubernetes' own official documentation) is this EXACT mechanism,
used deliberately and by design: when a pod needs to stop (a
deployment update, a scale-down, a node drain), Kubernetes sends
SIGTERM to the container's main process FIRST, then waits up to a
configurable "grace period" (30 seconds by default) for the process
to exit on its own, and ONLY IF IT HASN'T exited by then, sends
SIGKILL as a forced last resort.
This is precisely why a well-known, extremely common real production
bug is an application that does NOT properly handle SIGTERM
(ignoring it, or handling it too slowly): during a rolling deploy or
autoscaling event, in-flight requests get abruptly cut off when the
grace period expires and SIGKILL arrives, because the application
never used the polite warning it was given. This is documented
extensively across cloud-native engineering blogs and is one of the
most common causes of "why do we see a spike of errors during every
deploy" investigations -- and the fix is almost always: catch
SIGTERM in your application, stop accepting NEW work, finish
IN-FLIGHT work, then exit cleanly, well within the grace period.
Try it yourself
# Send signals and watch the difference
sleep 100 &
PID=$!
kill $PID # SIGTERM (default) -- sleep has no custom handler,
# so it just uses the default action: terminate
wait $PID 2>/dev/null; echo "exit status: $?"
# Write a tiny program that catches SIGTERM and cleans up gracefully
cat > sigdemo.py << 'EOF'
import signal, time, sys
def handle_sigterm(signum, frame):
print("caught SIGTERM -- cleaning up gracefully...")
time.sleep(1) # pretend to finish in-flight work
print("done, exiting cleanly")
sys.exit(0)
signal.signal(signal.SIGTERM, handle_sigterm)
print(f"PID {__import__('os').getpid()} running, send it SIGTERM")
while True:
time.sleep(1)
EOF
python3 sigdemo.py &
PID=$!
sleep 1
kill $PID # SIGTERM -- watch the graceful handler run
wait $PID
Build it in Go
In Go, signal.NotifyContext(ctx, syscall.SIGTERM, os.Interrupt) turns a
signal into a cancelled context, the cleanest way to start a graceful
shutdown. Complete examples: the
HTTPS guide's production server (fail readiness,
drain, exit) and the TCP/IP guide's draining backend,
where a SIGTERM-aware backend lost 0 of 300 requests and kill -9 lost 24.
Common mistakes
- "
killmeans force-kill." Plainkill <pid>sends SIGTERM (a request), not SIGKILL (a command) —kill -9is the forceful one. - Never handling SIGTERM in a long-running service. The process will be forcibly SIGKILLed after the grace period expires, with no chance to clean up — a direct, common cause of dropped requests during deploys.
- Doing SLOW cleanup work in a SIGTERM handler. If it takes longer than the grace period (Kubernetes' default: 30 seconds), you'll be SIGKILLed mid-cleanup anyway — keep the handler fast, or extend the grace period deliberately if the work genuinely needs longer.
Check yourself
- What's the difference between SIGTERM and SIGKILL, and why can't SIGKILL be caught or ignored?
- What does
kill <pid>(no flags) actually send? - Why does an application that ignores SIGTERM cause errors during a Kubernetes rolling deploy?
Further reading
- Reference:
man 7 signal— the authoritative list of every signal and its default action. - Book (free): OSTEP's process chapters mention signals; supplement with Kerrisk's The Linux Programming Interface for full depth.
- Docs: Kubernetes' own "Pod Lifecycle" documentation — the direct, real-world continuation of this chapter, covered again in Phase 3.