Chapter 6 — Base64, hex, and URL encoding: none of them protect anything

In one sentence

Encoding changes how data looks so it can travel safely through systems that would mangle it — and anyone can reverse it instantly, which is why it is not security.

The problem encoding solves

Some channels can't carry arbitrary bytes.

   Email (historically) was 7-bit ASCII only. A JPEG has bytes > 127.
   A URL can't contain a raw space, '&', or '#' -- they mean something.
   JSON can't hold arbitrary binary in a string.
   A terminal will beep, clear, or misbehave on control bytes.

So you re-express the same data using only "safe" characters. No secret is involved. That's the entire point — the receiver must be able to reverse it without any prior arrangement.

Base64

Takes 3 bytes (24 bits) and re-expresses them as 4 characters (6 bits each) from a 64-character alphabet: A–Z a–z 0–9 + /, padding with =.

   Input:  M         a         n
   ASCII:  77        97        110
   Binary: 01001101  01100001  01101110
           |------|--|----|----|--|------|
   6-bit:  010011   010110   000101   101110
   Value:  19       22       5        46
   Base64: T        W        F        u

   "Man" -> "TWFu"

Padding handles inputs that aren't a multiple of 3:

   "Man"  (3 bytes) -> "TWFu"      no padding
   "Ma"   (2 bytes) -> "TWE="      one '='
   "M"    (1 byte)  -> "TQ=="      two '='

Base64 makes data ~33% bigger (4 characters for every 3 bytes). That's the price of the safe alphabet.

Variants you'll meet:

Variant	Difference	Where
Standard	`+` and `/`, `=` padding	MIME, most APIs
URL-safe	`-` and `_` instead of `+` and `/`	JWTs, URLs, filenames
No padding	`=` omitted	JWTs

Using the wrong variant is a very common bug — a + in a URL is decoded as a space, silently corrupting the data.

Hexadecimal

Each byte becomes exactly two characters 0-9 a-f. Doubles the size, but is trivially readable and has no variants to get wrong.

   "Hello"  ->  48 65 6c 6c 6f

Used for hashes, keys, MAC addresses, and anywhere humans read raw bytes.

URL encoding (percent-encoding)

Unsafe characters become % plus two hex digits.

   space  -> %20    (or '+' in a query string, confusingly)
   /      -> %2F
   ?      -> %3F
   &      -> %26
   #      -> %23
   %      -> %25    <-- encoding the encoder

Double-encoding is a real attack. %252F decodes once to %2F and twice to /. If one component in a chain decodes and another decodes again, an attacker can smuggle a path separator past a filter. We'll use this in Chapter 45.

The thing to internalise

   +----------------------------------------------------------------+
   |  ENCODING IS NOT ENCRYPTION.                                    |
   |                                                                 |
   |  There is no key. There is no secret. There is no protection.    |
   |  Anyone with the encoded data can recover the original in        |
   |  microseconds, with a command that ships on every computer.      |
   |                                                                 |
   |  Base64 is an envelope, not a safe.                              |
   +----------------------------------------------------------------+

Real-world example: "we encrypt the password"

A production API sends:

   { "user": "alice", "password": "aHVudGVyMg==" }

The developer called this "encrypted". It is Base64 of hunter2:

echo 'aHVudGVyMg==' | base64 -d      # hunter2

This happens constantly. It appears in code review, in vendor documentation, and in breach post-mortems. If someone tells you data is protected by Base64, they have told you it is not protected.

The same mistake with a different costume: JWTs. A JWT looks cryptic and is just Base64url:

# The payload of any JWT is readable by anyone. Try one:
echo 'eyJzdWIiOiIxMjM0NSIsInJvbGUiOiJhZG1pbiJ9' | base64 -d
# {"sub":"12345","role":"admin"}

A JWT's signature provides integrity (Chapter 11), but the contents are public. Never put a secret in a JWT payload.

Practice (25 min)

cd ~/sec-lab/scratch

# --- Base64 both ways ---
printf 'Hello world' | base64                 # SGVsbG8gd29ybGQ=
echo 'SGVsbG8gd29ybGQ=' | base64 -d           # Hello world

# --- watch padding appear ---
for s in M Ma Man Many; do printf "%-5s -> " "$s"; printf "%s" "$s" | base64; done

# --- hex ---
printf 'Hello' | xxd -p                        # 48656c6c6f
echo '48656c6c6f' | xxd -r -p                  # Hello

# --- URL encoding ---
python3 -c "import urllib.parse as u; print(u.quote('hello world/&?'))"
python3 -c "import urllib.parse as u; print(u.unquote('%68%65%6C%6C%6F'))"

# --- double encoding ---
python3 -c "import urllib.parse as u; print(u.unquote('%252F'), '->', u.unquote(u.unquote('%252F')))"

# --- decode a real JWT payload ---
JWT='eyJhbGciOiJIUzI1NiJ9.eyJ1c2VyIjoiYWxpY2UiLCJyb2xlIjoiYWRtaW4ifQ.fakesig'
echo "$JWT" | cut -d. -f2 | tr '_-' '/+' | base64 -d 2>/dev/null; echo

Break it (this is the important part):

Base64-encode something you'd consider secret. Now decode it. How long did the "protection" last?
Take SGVsbG8= and change one character to SGVsbG9=. Decode it. Encoding has no integrity protection — nothing detects tampering.
Encode a string containing + and /, then put it in a URL. Watch it break. Now use URL-safe Base64 (base64 | tr '+/' '-_').

Common confusions

"Base64 is hashing." No — it's fully reversible.
"Base64 is compression." No — it makes data 33% bigger.
"The API needs the password Base64-encoded, so it's secure." The encoding is for transport. HTTPS (Chapter 27) provides the security. Without HTTPS, Base64 credentials are plaintext credentials. This is exactly what HTTP Basic auth is, and why it must never be used without TLS.
"I'll encode the data twice for extra security." Twice zero is zero.

Check yourself

Why does Base64 exist? What problem does it solve?
Why is Base64 output ~33% larger than the input?
What does the = at the end mean?
Can you tell if Base64-encoded data has been tampered with? Why not?
Is a JWT's payload secret?