53. Debugging drivers: printk/dmesg, an oops backtrace, and QEMU+gdb

53.1 printk log levels and dmesg

printk(KERN_EMERG   "system is unusable\n");   // level 0 — highest
printk(KERN_ALERT   "action must be taken immediately\n");
printk(KERN_CRIT    "critical condition\n");
printk(KERN_ERR     "error condition\n");
printk(KERN_WARNING "warning condition\n");
printk(KERN_NOTICE  "normal but significant\n");
printk(KERN_INFO    "informational\n");         // what Sections 50-52 used throughout
printk(KERN_DEBUG   "debug-level message\n");  // level 7 — lowest

dmesg | tail -20            # see recent kernel log messages, including every printk above
dmesg -w                    # follow, like `tail -f`, while you insmod/trigger your driver
cat /proc/sys/kernel/printk # which levels also go to the console, not just the ring buffer

53.2 Reading a kernel oops

A bug in a kernel module (a NULL dereference, an out-of-bounds access — Section 28's UB list, now with no memory-protection safety net at all) produces an oops: a backtrace printed to dmesg, naming the faulting function and an instruction-pointer address, then usually killing only the offending process (a full panic — Section 30's hardening context in user space has no real equivalent here — halts everything).

BUG: kernel NULL pointer dereference, address: 0000000000000008
...
RIP: 0010:chardev_read+0x12/0x40 [chardev]
Call Trace:
 vfs_read+0x9e/0x1a0
 ksys_read+0x67/0xe0
 do_syscall_64+0x38/0x90

chardev_read+0x12/0x40 [chardev] reads exactly like Section 14.4's bt output in gdb, just for kernel code: the faulting function, an offset into it, and which module it came from. addr2line (or the kernel source tree's scripts/faddr2line) turns that offset back into a source file and line number — the same "map an address back to a source location" workflow as any other crash in this guide, just with dmesg standing in for gdb's backtrace.

53.3 Iterating without risking real hardware: QEMU + gdb

You do not need to insmod untested driver code onto real hardware to debug it — booting a kernel under QEMU with gdb attached gives you a disposable "machine" you can crash as many times as you need:

qemu-system-x86_64 -kernel bzImage -initrd initrd.img \
    -append "console=ttyS0 nokaslr" -nographic -s -S
    # -s: open a gdb server on :1234    -S: pause the CPU before boot

# in another terminal:
gdb vmlinux
(gdb) target remote :1234
(gdb) break chardev_init
(gdb) continue

This is the same gdb from Section 14.4, attached over the network instead of to a local process — the entire "set a breakpoint, continue, inspect a variable" workflow you already know carries over unchanged; only the target (a whole virtual machine's kernel, paused before it even boots) is new.