Day 52 — TCP Internals: What Goes Wrong

Learning Objectives

• Diagnose CLOSE_WAIT and TIME_WAIT states in production
• Understand FIN_WAIT_2 and when it indicates a bug
• Know the difference between normal and pathological socket states

TCP State Machine Reference

Normal close sequence (server initiates):
  Server → FIN → Client   [server: FIN_WAIT_1 → FIN_WAIT_2]
  Client → ACK → Server   [client: CLOSE_WAIT]
  Client → FIN → Server   [server: TIME_WAIT → CLOSED]
  Server → ACK → Client   [client: LAST_ACK → CLOSED]

TIME_WAIT (normal):
  Lasts 2*MSL (60-120 seconds)
  Prevents duplicate packets from old connections confusing new ones
  SOLUTION: increase ephemeral port range (not your bug)

CLOSE_WAIT (application bug):
  Remote side sent FIN (closed their end)
  Your code received it but never called conn.Close()
  Connection stuck in CLOSE_WAIT forever
  SOLUTION: always defer conn.Close()

FIN_WAIT_2 (possible leak):
  You sent FIN, waiting for remote's FIN
  Remote is slow or never sends FIN
  Linux has FIN_WAIT_2 timeout (default 60s)
  SOLUTION: set ReadDeadline to force close

Exercise: Reproduce and Fix CLOSE_WAIT

package main

import (
    "bufio"
    "fmt"
    "net"
    "time"
)

// BUG: This server never closes the connection, causing CLOSE_WAIT
func buggyServer(ln net.Listener) {
    for {
        conn, err := ln.Accept()
        if err != nil { return }
        go func() {
            // MISSING: defer conn.Close()
            reader := bufio.NewReader(conn)
            line, _ := reader.ReadString('\n')
            conn.Write([]byte("ECHO: " + line))
            // Goroutine exits but conn is NOT closed
            // When client closes, conn enters CLOSE_WAIT
        }()
    }
}

// FIXED: Always defer conn.Close()
func fixedServer(ln net.Listener) {
    for {
        conn, err := ln.Accept()
        if err != nil { return }
        go func() {
            defer conn.Close() // FIXED: always close
            conn.SetDeadline(time.Now().Add(30 * time.Second))
            reader := bufio.NewReader(conn)
            line, _ := reader.ReadString('\n')
            conn.Write([]byte("ECHO: " + line))
        }()
    }
}

func main() {
    ln, _ := net.Listen("tcp", ":9090")
    fmt.Println("Buggy server on :9090")
    fmt.Println("After connecting a client and closing it:")
    fmt.Println("  netstat -an | grep 9090  # look for CLOSE_WAIT")

    // Run buggy server for 10s, then fixed
    go buggyServer(ln)
    time.Sleep(10 * time.Second)
    ln.Close()

    fmt.Println("\nNow running fixed server on :9090")
    ln2, _ := net.Listen("tcp", ":9090")
    go fixedServer(ln2)
    time.Sleep(10 * time.Second)
    ln2.Close()
}

Observe socket states:

# Run server, connect a client, then close client
go run . &
nc 127.0.0.1 9090
# type a line, press Enter, then Ctrl+C (close nc)
# Observe CLOSE_WAIT:
netstat -an | grep 9090 | grep CLOSE_WAIT
# After fix, this should not appear

Real-World Context

In a load balancer or proxy, CLOSE_WAIT exhaustion is a common incident. You get reports: "the service stops accepting connections" — and netstat shows thousands of CLOSE_WAIT sockets. The fix is always defer conn.Close() plus a SetDeadline. This is an OWASP A05 issue (Security Misconfiguration — resource exhaustion).