5. Exponents & Logarithms

Level: Beginner You'll use this for: understanding Big-O, decibels/signal strength, information content, and every "how many bits" question.

Exponent rules

x^a · x^b = x^(a+b)
x^a / x^b = x^(a-b)
(x^a)^b   = x^(ab)
x^0 = 1        x^(-a) = 1/x^a

Logarithms — the inverse of exponentiation

log_b(x) = y means b^y = x. "How many times do I multiply b by itself to reach x?"

log2(8)   = 3      (2^3 = 8)
log2(1024)= 10     (2^10 = 1024)
log10(1000)=3      (10^3 = 1000)

Why log₂ specifically matters to engineers: it answers "how many bits do I need?" log2(n) rounded up is the number of bits needed to represent n distinct values. This is the same number that shows up as the depth of a balanced binary search tree, the number of comparisons in binary search, and the number of "bits of entropy" in a random value.

import math
math.log2(1_000_000)   # ≈ 19.93 -> need 20 bits to enumerate a million things
math.log2(256)         # 8.0    -> a byte has 8 bits, 256 possible values

How to estimate `log2(x)` by hand — bracket between powers of 2

You don't need a calculator to get a useful estimate: find the two powers of 2 that x sits between, and interpolate.

Estimating log2(1,000,000):
  2^19 = 524,288        <- too small
  2^20 = 1,048,576      <- just above 1,000,000

  So log2(1,000,000) is between 19 and 20 — closer to 20, since
  1,000,000 is much nearer 1,048,576 than 524,288.
  (The exact value, ≈19.93, confirms the estimate.)

This "which two powers of 2 does it sit between" check is exactly how you mentally answer "roughly how many bits do I need for a million things" — you don't need the precise decimal, just the bracket, which is why engineers eyeball log2 constantly without reaching for a calculator.

Worked example: why binary search is O(log n)

Each comparison halves the search space. Starting with n items, after k halvings you have n / 2^k items left. Search ends when that's 1: n / 2^k = 1 → 2^k = n → k = log2(n). For a billion items, that's log2(10^9) ≈ 30 comparisons — not a billion.

Log rules that matter in practice

log(a·b) = log(a) + log(b)      <- multiplication becomes addition
log(a/b) = log(a) - log(b)
log(a^k) = k·log(a)

This is why decibels (dB = 10·log10(P/P_ref)), pH, the Richter scale, and perplexity/log-loss in ML all use logs: they compress a value that spans many orders of magnitude into a manageable additive scale, and they turn products of small probabilities (which underflow to 0 in floating point) into sums.

# Why ML loss functions use log-probabilities, not raw probabilities:
probs = [0.001] * 50          # 50 independent low-probability events
raw_product = 1.0
for p in probs: raw_product *= p
print(raw_product)            # 0.0  <- underflowed to zero!

log_sum = sum(math.log(p) for p in probs)
print(log_sum)                 # -345.4  <- still numerically meaningful

Real-life engineering ties

SWE: Big-O reasoning (O(log n) algorithms), pagination depth estimates.
AI: cross-entropy loss, log-likelihood, perplexity, log-sum-exp trick for numerical stability in softmax (Ch. 22–23).
Network: signal-to-noise ratio and channel capacity are log-based (Shannon-Hartley, Ch. 25/27); dB loss budgets in fiber links.
Security: password/key entropy is measured in bits, i.e., log2 of the keyspace size (Ch. 26).

More real-life examples:

git bisect finds a bad commit among thousands in O(log n) steps by binary search — the same halving logic that makes binary search fast, applied to version history instead of a sorted array.
Earthquake magnitude (Richter/moment scale) and sound intensity (decibels) are both logarithmic: a magnitude-7 earthquake releases roughly 32x more energy than a magnitude-6, not "1 more unit" of damage — exactly the compression effect log provides for values spanning huge ranges.
Large language model evaluation reports perplexity, which is literally 2^(cross-entropy loss) — a log-scale metric chosen for the same numerical-underflow reasons shown in the code example above.
Database index design assumes O(log n) lookups (B-trees) specifically so lookups stay fast as a table grows from thousands to billions of rows — the log term is why adding 10x more rows barely changes query time.

Common mistake

Confusing log base — most math textbooks default to ln (base e), most CS contexts default to log2, most engineering/audio contexts default to log10. Always check which base a formula assumes.