Jaeger Tracing Guide

Jaeger is an open-source distributed tracing platform originally created at Uber and now a graduated CNCF project. It records the journey of a single request as it crosses services, so you can see where time is spent and where failures originate. Use Jaeger when a request touches several services and metrics alone cannot tell you which one is slow.

Core concepts

• Trace: the end-to-end record of one request, identified by a trace ID.
• Span: one unit of work inside a trace, such as an HTTP handler or a database query. Spans have a name, start time, duration, tags (attributes), and a parent span.
• Context propagation: trace and span IDs travel between services in headers, most commonly the W3C traceparent header. Without propagation, traces break into fragments.
• Sampling: the policy that decides which traces are recorded, since tracing everything is costly.
• OpenTelemetry (OTel): the vendor-neutral standard for instrumentation. Modern Jaeger accepts OTLP directly, so you instrument with OTel and use Jaeger as the backend.

Quick start

The all-in-one image bundles collector, storage (in memory), query service, and UI. It is for development only.

docker run -d --name jaeger \
  -p 16686:16686 \
  -p 4317:4317 \
  -p 4318:4318 \
  jaegertracing/all-in-one:latest

The UI is at http://localhost:16686. Port 4317 accepts OTLP over gRPC and 4318 accepts OTLP over HTTP.

Instrumenting a service

Many languages offer auto-instrumentation. For a Node.js service, install the OpenTelemetry packages and set environment variables:

npm install @opentelemetry/api @opentelemetry/auto-instrumentations-node \
  @opentelemetry/sdk-node @opentelemetry/exporter-trace-otlp-http

export OTEL_SERVICE_NAME=checkout-service
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
export OTEL_TRACES_SAMPLER=parentbased_traceidratio
export OTEL_TRACES_SAMPLER_ARG=0.1
node --require @opentelemetry/auto-instrumentations-node/register app.js

Python and Java follow the same pattern with their own agents. Auto-instrumentation covers HTTP, gRPC, and common database clients. Add manual spans for business logic:

const { trace } = require('@opentelemetry/api');
const tracer = trace.getTracer('checkout');

async function applyDiscount(cart) {
  return tracer.startActiveSpan('apply-discount', async (span) => {
    span.setAttribute('cart.items', cart.items.length);
    try {
      return await compute(cart);
    } catch (err) {
      span.recordException(err);
      throw err;
    } finally {
      span.end();
    }
  });
}

Reading a trace

In the Jaeger UI, choose a service, optionally filter by operation, tags such as http.status_code=500, or minimum duration, and click Find Traces. A trace view shows a waterfall of spans. Look for long bars, serial calls that could run in parallel, repeated identical queries (the classic N+1 problem), and spans marked with errors. The compare feature and the service dependency graph help you spot regressions and unexpected coupling.

Production deployment

The in-memory all-in-one image loses data on restart. For production, run the Jaeger collector and query components with a persistent backend such as Elasticsearch or OpenSearch, or Cassandra, and consider placing an OpenTelemetry Collector in front to batch, filter, and route spans. Keep retention short, often a few days, because traces are mostly useful for recent debugging.

Common pitfalls

• Broken propagation. A message queue, async job, or custom HTTP client that drops headers splits one trace into several. Inject and extract context explicitly.
• Wrong sampling. Head sampling at 1 percent can miss rare errors. Use parent-based sampling so downstream services honour upstream decisions, and consider tail sampling in the OTel Collector to keep slow or failed traces.
• High-cardinality or sensitive attributes. Do not put passwords, tokens, or personal data in span attributes.
• Too many tiny spans. Instrument boundaries, not every function.
• Clock skew between hosts, which makes child spans appear to start before parents.

Where Jaeger fits in an architecture

Tracing is the third pillar beside metrics and logs. A typical flow: a Grafana alert shows elevated p99 latency, logs narrow it to one endpoint, and a Jaeger trace reveals that a downstream inventory call is the bottleneck. Include the trace ID in log lines so you can jump between logs and traces. Grafana can also use Jaeger as a data source for unified investigation.

Key takeaways

• A trace is a tree of spans linked by propagated context.
• Instrument with OpenTelemetry and export OTLP to Jaeger.
• Use the all-in-one image for development and a persistent backend for production.
• Choose sampling deliberately and keep sensitive data out of spans.
• Correlate traces with logs and metrics for fast diagnosis.