Day 19 — Strings & Bytes

Learning Objectives

• Normalize URLs and headers safely — prevent WAF bypass via encoding tricks
• Use strings.Builder for efficient string construction in hot paths
• Understand the difference between string and []byte for security code

Key Concepts

// string is immutable — every concatenation allocates
// Use strings.Builder for building strings in loops
var sb strings.Builder
for _, part := range parts {
    sb.WriteString(part)
    sb.WriteByte('/')
}
result := sb.String() // one allocation at the end

// strings vs bytes: convert only when needed (costs allocation)
b := []byte(s)  // string → bytes copy
s := string(b)  // bytes → string copy

// Key functions for security code
strings.ToLower(s)                    // case normalization
strings.TrimSpace(s)                  // remove leading/trailing whitespace
strings.HasPrefix(s, "../")           // path traversal check
strings.Contains(s, "UNION SELECT")   // naive SQLi check
strings.ReplaceAll(s, "//", "/")      // normalize double slashes

Exercise: URL Normalizer (WAF Pre-Processing)

package main

import (
    "fmt"
    "net/url"
    "path"
    "sort"
    "strings"
)

// NormalizeURL prepares a URL for WAF rule matching.
// Attackers use encoding tricks to bypass naive string matching.
func NormalizeURL(rawURL string) (string, error) {
    // Step 1: Parse
    u, err := url.Parse(rawURL)
    if err != nil {
        return "", fmt.Errorf("invalid URL: %w", err)
    }

    // Step 2: Lowercase scheme and host
    u.Scheme = strings.ToLower(u.Scheme)
    u.Host = strings.ToLower(u.Host)

    // Step 3: Remove default ports
    host := u.Hostname()
    port := u.Port()
    switch {
    case u.Scheme == "http" && port == "80":
        u.Host = host
    case u.Scheme == "https" && port == "443":
        u.Host = host
    }

    // Step 4: Normalize path — resolve ./ and ../
    // url.Parse already handles %XX decoding, but path.Clean handles traversal
    cleaned := path.Clean(u.Path)
    if strings.HasSuffix(u.Path, "/") && cleaned != "/" {
        cleaned += "/"
    }
    u.Path = cleaned

    // Step 5: Sort query params (canonical order for matching)
    if u.RawQuery != "" {
        params := u.Query()
        var keys []string
        for k := range params {
            keys = append(keys, k)
        }
        sort.Strings(keys)
        var sb strings.Builder
        for i, k := range keys {
            if i > 0 { sb.WriteByte('&') }
            for j, v := range params[k] {
                if j > 0 { sb.WriteByte('&') }
                sb.WriteString(url.QueryEscape(k))
                sb.WriteByte('=')
                sb.WriteString(url.QueryEscape(v))
            }
        }
        u.RawQuery = sb.String()
    }

    return u.String(), nil
}

func main() {
    tests := []string{
        "HTTP://Example.COM:80/api/../admin/./users?b=2&a=1",
        "https://example.com:443/api//v1///resource",
        "https://example.com/path/%2e%2e/etc/passwd", // path traversal attempt
    }

    for _, raw := range tests {
        norm, err := NormalizeURL(raw)
        if err != nil {
            fmt.Printf("ERROR: %v\n", err)
            continue
        }
        fmt.Printf("IN:  %s\nOUT: %s\n\n", raw, norm)
    }
}

Real-World Context

OWASP A03 attacks (Injection) rely on encoding to bypass WAF rules. %2e%2e%2f is ../ URL-encoded. %252e%252e%252f is double-encoded. Your WAF normalization pipeline (Day 73) applies multiple decoding passes before rule matching.