Frontend Labs
⋯

Chapter 79 -- Reading /proc from Go: build your own ps

In one sentence

ps, top, free, and most monitoring agents are parsers for files under /proc. Reading them yourself shows exactly what each number means and where the parsing traps are.

Why this matters

When a container has no ps, when you need one number in a health check, or when you're building an exporter or debugging an agent, you read /proc directly. Chapter 23 showed the files from the shell. This chapter shows the format details that break naive parsers.

How it actually works

File What it holds Trap
/proc/<pid>/stat one line: state, CPU ticks, threads, start time... field 2 is (comm) and may contain spaces and parentheses: split after the last )
/proc/<pid>/status human-readable: VmRSS, Threads, voluntary_ctxt_switches values carry units (kB)
/proc/<pid>/cmdline argv, NUL-separated empty for kernel threads
/proc/<pid>/fd/ one symlink per open descriptor needs permission for other users' processes
/proc/<pid>/limits effective rlimits the truth about Chapter 78's limits

CPU% requires two samples: (ticks_after − ticks_before) / USER_HZ / interval. USER_HZ is 100 on practically every Linux system.

Build it in Go (30 min) -- gops

//go:build linux

// gops: `ps` and a little of `top`, written from /proc alone -- everything
// those tools show comes from these files.
//
//	go run ./gops            # top 10 processes by CPU over 1 second
package main

import (
	"fmt"
	"os"
	"path/filepath"
	"sort"
	"strconv"
	"strings"
	"time"
)

type proc struct {
	pid      int
	comm     string
	state    string
	threads  int
	rssKB    int64
	cpuTicks int64 // utime + stime, in clock ticks
	cmdline  string
}

// read parses /proc/<pid>/stat (one line, space-separated, but the command
// name is in parentheses and may itself contain spaces) and /proc/<pid>/status.
func read(pid int) (*proc, error) {
	b, err := os.ReadFile(fmt.Sprintf("/proc/%d/stat", pid))
	if err != nil {
		return nil, err
	}
	s := string(b)
	l, r := strings.IndexByte(s, '('), strings.LastIndexByte(s, ')')
	f := strings.Fields(s[r+2:]) // fields after the comm: f[0] is field 3 ("state")
	p := &proc{pid: pid, comm: s[l+1 : r], state: f[0]}
	utime, _ := strconv.ParseInt(f[11], 10, 64) // field 14
	stime, _ := strconv.ParseInt(f[12], 10, 64) // field 15
	p.threads, _ = strconv.Atoi(f[17])          // field 20
	p.cpuTicks = utime + stime

	st, _ := os.ReadFile(fmt.Sprintf("/proc/%d/status", pid))
	for _, line := range strings.Split(string(st), "\n") {
		if strings.HasPrefix(line, "VmRSS:") {
			p.rssKB, _ = strconv.ParseInt(strings.Fields(line)[1], 10, 64)
		}
	}
	cl, _ := os.ReadFile(fmt.Sprintf("/proc/%d/cmdline", pid)) // NUL-separated argv
	p.cmdline = strings.TrimSpace(strings.ReplaceAll(string(cl), "\x00", " "))
	if p.cmdline == "" {
		p.cmdline = "[" + p.comm + "]" // kernel threads have no cmdline
	}
	return p, nil
}

func snapshot() map[int]*proc {
	out := map[int]*proc{}
	dirs, _ := filepath.Glob("/proc/[0-9]*")
	for _, d := range dirs {
		pid, _ := strconv.Atoi(filepath.Base(d))
		if p, err := read(pid); err == nil { // processes can vanish mid-scan: ignore
			out[pid] = p
		}
	}
	return out
}

func main() {
	const hz = 100 // USER_HZ: clock ticks per second on virtually all Linux systems
	interval := time.Second
	before := snapshot()
	time.Sleep(interval)
	after := snapshot()

	type row struct {
		p   *proc
		cpu float64
	}
	var rows []row
	for pid, p := range after {
		var delta int64
		if b, ok := before[pid]; ok {
			delta = p.cpuTicks - b.cpuTicks
		}
		rows = append(rows, row{p, float64(delta) / hz / interval.Seconds() * 100})
	}
	sort.Slice(rows, func(i, j int) bool { return rows[i].cpu > rows[j].cpu })

	states := map[string]int{}
	for _, r := range rows {
		states[r.p.state]++
	}
	fmt.Printf("%d processes; states: %v  (R running, S sleeping, D disk wait, Z zombie)\n\n", len(rows), states)
	fmt.Printf("%7s %-1s %6s %5s %9s  %s\n", "PID", "S", "%CPU", "THR", "RSS(MB)", "COMMAND")
	for _, r := range rows[:min(10, len(rows))] {
		cmd := r.p.cmdline
		if len(cmd) > 50 {
			cmd = cmd[:50] + "..."
		}
		fmt.Printf("%7d %-1s %6.1f %5d %9.1f  %s\n", r.p.pid, r.p.state, r.cpu, r.p.threads, float64(r.p.rssKB)/1024, cmd)
	}
}

Real output (in a container with --pid=host, a busy loop running):

311 processes; states: map[I:126 R:2 S:183]  (R running, S sleeping, D disk wait, Z zombie)

    PID S   %CPU   THR   RSS(MB)  COMMAND
  82877 R  101.0     1       0.5  sh -c (while :; do :; done) & sleep 0.5; /b/gops
  34206 S    0.0     1       3.5  nginx: worker process
    359 I    0.0     1       0.0  [kworker/11:1H-kblockd]
   1114 S    0.0     1      14.6  postgres: checkpointer
    295 S    0.0    20      88.9  /usr/bin/containerd --config /etc/containerd/conta...

What to notice:

  • I (idle) is a kernel-thread state for idle workers: 126 of them here, all in square brackets because kernel threads have no command line.
  • 101% CPU is one fully busy core, plus sampling error. top would show the same thing.
  • Processes vanish mid-scan. The code ignores read errors for that reason: any /proc walker must tolerate processes exiting between listing and reading.

Exercises:

  1. Add a -p PID mode that prints the process's open descriptors (resolving /proc/PID/fd/* symlinks) and its /proc/PID/limits.
  2. Add a zombie report: every Z process with its parent (field 4, ppid). Run it next to Chapter 74's pid1 lab.
  3. Export the numbers in Prometheus format (the TCP/IP guide's prober lab shows how).

Common mistakes

  • Splitting /proc/<pid>/stat on spaces from the start of the line.
  • Computing CPU% from a single sample.
  • Treating a failed read as fatal during a /proc scan.

Check yourself

  1. Why must the stat parser split after the last )?
  2. How do you compute a process's CPU% from /proc?
  3. Why do some processes have an empty cmdline?

Further reading

  • man 5 proc, the authoritative field-by-field reference.
  • Chapter 23 of this guide.

Generated from real-life-os-guide.md · 8 Oct 2026 16:36