27. Signal Processing Basics

Level: Expert You'll use this for: understanding RF/wireless engineering, audio/video ML, and reading network jitter correctly.

The Fourier transform — decomposing a signal into frequencies

Any signal (audio waveform, radio signal, even a time series of request latencies) can be decomposed into a sum of sine waves of different frequencies. The Fourier transform converts a signal from the time domain (value vs. time) to the frequency domain (how much of each frequency is present).

import numpy as np

# A signal made of a 5Hz wave and a 50Hz wave mixed together
t = np.linspace(0, 1, 1000, endpoint=False)
signal = np.sin(2*np.pi*5*t) + 0.5*np.sin(2*np.pi*50*t)

fft_result = np.fft.fft(signal)
freqs = np.fft.fftfreq(len(t), d=t[1]-t[0])
magnitude = np.abs(fft_result)

# The two dominant frequencies show up as spikes at 5Hz and 50Hz
dominant_freqs = freqs[np.argsort(magnitude)[-4:]]
print(sorted(set(abs(f) for f in dominant_freqs if f > 0)))  # ~[5.0, 50.0]

This exact operation (FFT — Fast Fourier Transform, an O(n log n) algorithm for computing this decomposition) runs inside Wi-Fi/cellular modems (OFDM modulation splits data across many simultaneous frequency sub-carriers), audio codecs, and image compression (JPEG uses a related transform, the DCT).

The Nyquist-Shannon sampling theorem

To perfectly reconstruct a signal containing frequencies up to f_max, you must sample at a rate of at least 2 × f_max (the Nyquist rate). Sampling below this causes aliasing — high frequencies masquerade as false low frequencies in your sampled data (the classic "wagon wheel spinning backward" effect in old films is aliasing).

sample_rate >= 2 * highest_frequency_of_interest

Computing an alias frequency by hand. If you sample a 6Hz signal at only 5 samples/second (below the required 2×6=12Hz Nyquist rate), the signal doesn't just look choppy — it appears as a different, false frequency:

alias_frequency = | true_frequency - nearest_multiple_of(sample_rate) |
                 = | 6 - 5 |
                 = 1 Hz

Sampling a 6Hz wave at 5Hz makes it look exactly like a slow 1Hz wave in the recorded data — indistinguishable from the real thing once sampled, which is why aliasing is dangerous: nothing in the sampled data itself flags that anything went wrong.

This is why audio CDs sample at 44.1kHz (comfortably above 2×20kHz, the upper edge of human hearing), and it applies directly to any discrete sampling of a continuous signal — including monitoring systems: if you sample a metric (e.g., CPU usage) once a minute, you cannot detect spikes/oscillations happening faster than once every 2 minutes — they'll alias into misleading patterns in your dashboard.

Euler's formula: rotations as numbers

The Fourier transform is written with complex exponentials, and one identity makes them readable:

e^(iθ) = cos θ + i · sin θ

Multiplying by e^(iθ) rotates a point by θ. That's the rotation matrix from Ch. 10 packed into a single number. Set θ = π and you get e^(iπ) + 1 = 0, which links five fundamental constants in one line. A spinning phasor e^(2πi·f·t) goes round f times per second, so a sine wave of frequency f is just the vertical shadow of that rotation.

The Discrete Fourier Transform, symbol by symbol

X_k = Σₙ₌₀^(N-1)  x_n · e^(-2πi · k·n / N)          k = 0, 1, …, N-1

Every DFT coefficient answers one question: how well does the signal line up with a wave of frequency k? That's the same correlation idea as Ch. 9 and the same dot product as cosine similarity. The DFT is a change of basis, a matrix multiplication by an N × N matrix of rotations, and the FFT computes it in O(N log N) instead of O(N²) by reusing shared sub-products.

A 4-point DFT by hand. Signal x = [1, 0, -1, 0], which is one full cosine cycle. For N = 4, the probe values e^(-2πi·kn/4) are just 1, -i, -1, i raised to powers:

X₀ = 1 + 0 + (-1) + 0                  = 0     (no constant offset: the average is 0)
X₁ = 1·1 + 0·(-i) + (-1)·(-1) + 0·(i)  = 2     (strong match at 1 cycle)
X₂ = 1·1 + 0·(-1) + (-1)·(1)  + 0·(-1) = 0
X₃ = 1·1 + 0·(i)  + (-1)·(-1) + 0·(-i) = 2     (the mirror of X₁, always true for real signals)

The energy sits at k = 1 (and its mirror at k = 3), which correctly identifies "one cycle per window". np.fft.fft([1, 0, -1, 0]) returns [0, 2, 0, 2].

The convolution theorem: why the FFT is everywhere

Convolution slides one signal across another, multiplying and summing at each offset. It's what a filter, an echo, a blur and a CNN layer all do:

(f * g)[n] = Σₘ f[m] · g[n - m]

Theorem (convolution). Convolution in the time domain equals element-wise multiplication in the frequency domain: DFT(f * g) = DFT(f) ⊙ DFT(g).

Direct convolution of two length-N signals costs O(N²). Going through the FFT (transform, multiply, transform back) costs O(N log N). That's why audio effects, large-kernel image filters, radar matched filters and polynomial multiplication (including the number-theoretic transform at the heart of ML-KEM in Ch. 24) all run through FFTs. In AI, a CNN layer is a learned convolution. Long-sequence models such as Hyena and S4/Mamba-style state-space models use FFT convolutions to cover very long contexts in O(N log N) instead of attention's O(N²).

Why Nyquist's theorem holds

Sampling a signal every T = 1/f_s seconds makes its spectrum repeat every f_s hertz. Sampling is multiplication by a train of spikes, and by the convolution theorem that becomes convolution of the spectrum with spikes at multiples of f_s, which pastes copies of the spectrum at 0, ±f_s, ±2f_s, …. Each copy extends from -f_max to +f_max. The copies stay separate only if:

f_s - f_max > f_max    <=>    f_s > 2 · f_max

If they overlap, a high frequency lands on top of a low one and the two can never be told apart. That's aliasing. The |6 - 5| = 1 Hz alias computed above is the copy at f_s = 5 shifted back to 1 Hz. This is why every ADC has an analogue anti-aliasing filter in front of it to remove frequencies above f_s/2 before sampling, since afterwards it's too late. For monitoring, the equivalent is to scrape fast and then downsample using max or a percentile, not a single point sample.

Decibels: the logarithm that radio engineers live in

Signal powers span many orders of magnitude, so engineers use a log scale (Ch. 5):

gain (dB)   = 10 · log₁₀( P_out / P_in )
power (dBm) = 10 · log₁₀( P / 1 mW )

Multiplying ratios becomes adding decibels, so a link budget is plain addition: transmit power + antenna gains − losses = received power. Example: a Wi-Fi access point transmits at 20 dBm (100 mW). A laptop receiving at −70 dBm (10⁻¹⁰ W, a ten-billionth of the transmitted power) still has a usable link. That's a 90 dB loss, and you can only reason about a ratio of a billion comfortably on a log scale.

Free-space path loss: why 5 GHz has shorter range

FSPL (dB) = 20·log₁₀(d) + 20·log₁₀(f) - 147.55        (d in metres, f in Hz)

The 20·log₁₀ (not 10) comes from the inverse-square law: power spreads over a sphere of area 4πd². Worked out:

2.4 GHz at 10 m:  20·1 + 20·log₁₀(2.4·10⁹) - 147.55 = 20 + 187.6 - 147.55 ≈ 60.0 dB
5 GHz at 10 m:                                                             ≈ 66.4 dB
2.4 GHz at 20 m:                                                           ≈ 66.1 dB

Doubling distance costs 6 dB, and moving from 2.4 to 5 GHz also costs about 6 dB. 5 GHz reaches roughly half as far for the same transmit power, before you even count walls, which absorb higher frequencies more. That's why mesh Wi-Fi and 5G millimetre-wave cells are placed so densely.

QAM and OFDM: how Wi-Fi approaches the Shannon limit

QAM (quadrature amplitude modulation) sends log₂ M bits per symbol by choosing one of M points on a grid of amplitudes and phases:

The SNR column comes from inverting Shannon–Hartley (Ch. 23): b = log₂(1 + SNR) gives SNR = 2ᵇ - 1. Real radios need a few dB more than this floor, which is why your laptop drops to a lower modulation as you walk away: the SNR no longer supports the dense grid.

OFDM splits a channel into many narrow subcarriers, spaced exactly Δf = 1/T_symbol apart. That spacing makes them orthogonal: their dot product over one symbol is zero (Ch. 10), so they overlap in frequency without interfering. The transmitter builds a symbol with an inverse FFT and the receiver takes it apart with an FFT. Wi-Fi 6's peak rate, rebuilt from first principles (20 MHz channel, one spatial stream, top MCS 11):

data subcarriers           = 234
bits per subcarrier        = 10       (1024-QAM)
coding rate                = 5/6      (error-correction overhead)
symbol time                = 12.8 µs + 0.8 µs guard interval = 13.6 µs

rate = 234 · 10 · (5/6) / 13.6 µs  ≈  143.4 Mbit/s

That's exactly the figure in the 802.11ax rate tables. Double the channel width, double the subcarriers, add spatial streams (MIMO, a linear-algebra trick that solves y = H·x for several streams, Ch. 10), and you reach the headline multi-gigabit numbers. Each factor in the formula is maths from this guide: the FFT, the logarithm, Shannon's limit and a matrix inverse.

Real-life engineering ties

• Network: RF/wireless engineering (channel width, OFDM sub-carrier spacing) is direct Fourier-domain reasoning; understanding why wider Wi-Fi channels (Ch. 23's Shannon-Hartley) carry more data.
• AI: audio/speech models operate on spectrograms (a Fourier-domain representation of sound); positional encodings in transformers are literally built from sine/cosine functions of different frequencies — Fourier-style reasoning by design.
• Security: side-channel analysis (e.g., analyzing power-consumption or electromagnetic-emission traces to extract a cryptographic key) is fundamentally frequency-domain signal analysis; timing-attack detection also benefits from understanding sampling-rate limitations.
• SWE/SRE: understanding aliasing explains why your 1-minute-resolution monitoring can completely miss (or badly misrepresent) sub-minute latency spikes — a direct, practical consequence of the sampling theorem.

More real-life examples:

• Shazam's music-identification algorithm converts audio into a spectrogram (a Fourier-domain representation) and fingerprints the pattern of frequency peaks over time — matching a few seconds of noisy audio against a database of millions of songs.
• Wi-Fi (802.11a/g/n/ac/ax) and 4G/5G cellular both use OFDM (Orthogonal Frequency-Division Multiplexing), which splits a channel into dozens or hundreds of narrow sub-carriers computed via FFT — a massive-scale production use of the Fourier transform running in every phone and router.
• Speech-to-text systems (including OpenAI's Whisper) preprocess raw audio into a mel-spectrogram — a Fourier-transform-based, perceptually weighted frequency representation — before feeding it into the network.
• MRI machines acquire raw data directly in the frequency domain (called "k-space" in radiology) and apply an inverse Fourier transform to reconstruct the anatomical image radiologists actually view.

Common mistake

Assuming higher monitoring-sample-rate is "just more data, no downside" — it's the opposite problem people usually hit: under-sampling and believing the resulting (aliased) chart accurately reflects reality.