67. Device and interface metrics: SNMP and its successors
67.1 SNMP in one page
Simple Network Management Protocol — the lingua franca of device monitoring since 1988. UDP/161 (queries), UDP/162 (traps).
MANAGER (your poller) AGENT (switch/router/server)
| |
| GET / GETNEXT / GETBULK (UDP 161) ------->|
|<--------------------------- response |
| |
|<-- TRAP / INFORM (UDP 162, unsolicited) ------| "my interface just went down"
OIDs and MIBs. Every value has a numeric OID (1.3.6.1.2.1.2.2.1.10.3),
and a MIB is the dictionary mapping names to OIDs
(IF-MIB::ifInOctets.3). The ones you'll use constantly:
| Name | OID root | Value |
|---|---|---|
sysUpTime |
1.3.6.1.2.1.1.3 |
Time since boot — a reset means the device rebooted |
ifDescr / ifName / ifAlias |
1.3.6.1.2.1.2.2.1.2 / 31.1.1.1.1 / 31.1.1.1.18 |
Interface naming; ifAlias is the human description |
ifHCInOctets / ifHCOutOctets |
1.3.6.1.2.1.31.1.1.1.6/.10 |
64-bit byte counters — use these, not the 32-bit ifInOctets |
ifInErrors / ifOutErrors |
...2.2.1.14/.20 |
Errors |
ifInDiscards / ifOutDiscards |
...2.2.1.13/.19 |
Drops (buffer/policy) |
ifOperStatus / ifAdminStatus |
...2.2.1.8/.7 |
Up/down, and whether a human shut it |
The 32-bit counter trap: a 32-bit octet counter wraps at ~4.3 GB. On a
10 Gbit/s link that is under 4 seconds. Always poll the 64-bit ifHC*
counters (SNMPv2c or later), and always compute rates as deltas with wrap
handling.
Versions: v1 (obsolete), v2c (community string in cleartext — treat the community as a password sent in the clear), v3 (authentication + encryption; use it). Restrict SNMP to a management network regardless.
67.2 Polling model and its limits
poll interval 60 s -> you cannot see a 10-second microburst
(it averages into invisibility)
poll 5,000 devices x 50 interfaces x 8 metrics every 60 s
-> 33,000 SNMP GETs/second -- and SNMP is a
request/response protocol over UDP
Consequences: use GETBULK, spread polls, and accept that SNMP is a minutes resolution tool. For sub-second visibility you need streaming telemetry.
67.3 Streaming telemetry (gNMI / NETCONF / OpenConfig)
The modern replacement for polling: the device pushes updates on subscription.
SNMP (pull): manager asks every 60 s text/ASN.1, UDP, per-OID
gNMI (push): device streams on change or at protobuf over gRPC/HTTP2/TLS,
a chosen cadence (e.g. 1 s) structured YANG paths
- YANG is the data model; OpenConfig is a vendor-neutral set of YANG
models. gNMI is the transport (
SubscribewithON_CHANGEorSAMPLEmodes). - Advantages: sub-second resolution, event-driven (no missed transitions), efficient encoding, structured data, secure by default.
- Reality: vendor support and model coverage vary; SNMP remains the universal fallback.
67.4 Host-level exporters
For servers, skip SNMP:
| Tool | Provides |
|---|---|
| node_exporter (Prometheus) | /proc/net/*, interface counters, conntrack, sockets |
| cAdvisor / kube-state-metrics | container and pod network stats |
eBPF exporters (ebpf_exporter, Cilium/Hubble, Pixie) |
per-socket RTT, retransmits, TCP state transitions, DNS latency — without packet capture |
eBPF is the significant recent shift: you can measure per-connection RTT and retransmits for every flow at low cost, which used to require sampling packets.
67.5 Real-world example
A 40 Gbit/s uplink showed 45% average utilisation and constant complaints of "slow at 09:00". Five-minute SNMP averages hid 10-second microbursts hitting line rate every morning as thousands of clients synced. Switching that interface to 1-second streaming telemetry made the bursts obvious; the fix was shaping and staggering the sync window. Averaging is how you hide the incident from yourself.
Try it yourself
# poll a device you own (v2c shown; prefer v3 in production)
snmpwalk -v2c -c public 192.168.1.1 1.3.6.1.2.1.1 # system info
snmpwalk -v2c -c public 192.168.1.1 IF-MIB::ifDescr
snmpget -v2c -c public 192.168.1.1 IF-MIB::ifHCInOctets.3
snmpbulkwalk -v2c -c public 192.168.1.1 IF-MIB::ifTable # efficient bulk read
# rate = (counter2 - counter1) / (t2 - t1) * 8 -> bits per second