Stackbook Logo
observabilityestablished · medium operational burden

USE Method

Also known as: utilization-saturation-errors

Intent

Systematically find resource bottlenecks by checking Utilization, Saturation, and Errors for every resource.

Problem

When performance degrades, engineers guess and check random graphs. Without a checklist, bottlenecks in less obvious resources such as network interfaces or file descriptors are missed for hours.

Forces

  • •Many resources, limited time during an incident
  • •Utilization alone hides queueing
  • •Errors on a resource are often not monitored
  • •Different resource types expose different counters

Solution

Overview

The USE method, from Brendan Gregg, is a diagnostic checklist for resources. For each resource, ask three questions: how busy is it (Utilization), is work waiting (Saturation), and is it failing (Errors). It turns a vague performance hunt into a bounded sweep.

How it works

Enumerate resources, then check each metric type:

Resource      Utilization            Saturation              Errors
CPU           cpu busy per core      run queue length        machine check events
Memory        used vs total          swapping, OOM kills     allocation failures
Disk          device busy percent    I/O queue depth         device errors
Network       bytes vs line rate     drops, retransmits      interface errors
DB pool       in-use / max           callers waiting         timeouts
Threads       busy / total           task queue depth        rejected tasks

Interpretation rules: utilization near 100 percent usually means a bottleneck; any sustained saturation means queued work and added latency; any errors deserve investigation. Because the sweep is finite, you can finish it, and you will have covered resources nobody thought of.

Pitfalls

  • Reading averaged CPU across cores. A single-threaded workload can saturate one core at 12 percent average.
  • Treating 70 percent disk utilization as safe. For a device that serves requests serially, queueing begins well below 100 percent.
  • Forgetting virtual limits such as cgroup CPU throttling, file descriptors, and ephemeral ports.
  • Ignoring errors because they look rare. A small count of retransmits can explain tail latency.

Key takeaways

  • Check utilization, saturation, and errors for every resource.
  • Saturation is the early warning that utilization misses.
  • Include software resources such as pools and locks.
  • Use USE for resources and RED for services.

✓ When to Use

  • •Diagnosing latency or throughput regressions
  • •Building infrastructure dashboards
  • •Capacity reviews for hosts and pools

✗ When Not to Use

  • •Describing request health of a service (use RED)
  • •Problems caused by logic bugs, not resources
  • •Systems where resources cannot be enumerated

Pros

  • +Finds bottlenecks quickly with a checklist
  • +Catches saturation that utilization hides
  • +Applies to hardware and software resources

Cons

  • −Needs an inventory of resources
  • −Saturation counters are inconsistent across platforms
  • −Does not show which request is affected

Cost Profile

Infrastructure

Low - node and pool metrics

Operational

Medium - maintain per-resource checklists

Cognitive

Medium - requires systems knowledge

Failure Modes

  • Only utilization checked, so queued work goes unnoticed

  • Averaged CPU hides one saturated core

  • Software resources such as pools left off the checklist

Real-World Examples

Alternatives

    Related Patterns

    • golden-signals
    • capacity-planning
    • observability

    Competency Domains

    reliability ops