Problem
When performance degrades, engineers guess and check random graphs. Without a checklist, bottlenecks in less obvious resources such as network interfaces or file descriptors are missed for hours.
Also known as: utilization-saturation-errors
Systematically find resource bottlenecks by checking Utilization, Saturation, and Errors for every resource.
When performance degrades, engineers guess and check random graphs. Without a checklist, bottlenecks in less obvious resources such as network interfaces or file descriptors are missed for hours.
The USE method, from Brendan Gregg, is a diagnostic checklist for resources. For each resource, ask three questions: how busy is it (Utilization), is work waiting (Saturation), and is it failing (Errors). It turns a vague performance hunt into a bounded sweep.
Enumerate resources, then check each metric type:
Resource Utilization Saturation Errors
CPU cpu busy per core run queue length machine check events
Memory used vs total swapping, OOM kills allocation failures
Disk device busy percent I/O queue depth device errors
Network bytes vs line rate drops, retransmits interface errors
DB pool in-use / max callers waiting timeouts
Threads busy / total task queue depth rejected tasks
Interpretation rules: utilization near 100 percent usually means a bottleneck; any sustained saturation means queued work and added latency; any errors deserve investigation. Because the sweep is finite, you can finish it, and you will have covered resources nobody thought of.
Low - node and pool metrics
Medium - maintain per-resource checklists
Medium - requires systems knowledge
Only utilization checked, so queued work goes unnoticed
Averaged CPU hides one saturated core
Software resources such as pools left off the checklist