OpenTelemetry Guide

What it is and when to use it

OpenTelemetry, often shortened to OTel, is a vendor-neutral standard for generating and exporting telemetry: traces, metrics and logs. It provides language SDKs, automatic instrumentation, a wire protocol called OTLP, and the OpenTelemetry Collector, a standalone pipeline for receiving, processing and exporting data.

Adopt it when you want observability without lock-in. Instrument once, then send data to Jaeger, Tempo, Prometheus, Grafana, Datadog or any other OTLP-capable backend, and change your mind later without touching application code.

Quick start

Zero-code instrumentation for a Python service:

pip install opentelemetry-distro opentelemetry-exporter-otlp
opentelemetry-bootstrap -a install

export OTEL_SERVICE_NAME=checkout
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
export OTEL_TRACES_SAMPLER=parentbased_traceidratio
export OTEL_TRACES_SAMPLER_ARG=0.1

opentelemetry-instrument python app.py

Node, Java and .NET offer comparable agents. For Java, attach the agent with -javaagent:opentelemetry-javaagent.jar.

Core concepts

Traces and spans. A trace follows one request across services. It is a tree of spans, each with a name, timing, attributes and a parent. Context propagation carries the trace id between services through the W3C traceparent header.

Metrics. Counters, up-down counters, gauges and histograms describe aggregated behavior over time.

Logs. Existing log records can be correlated with traces by attaching the trace and span ids.

Resource. Attributes such as service.name, service.version and deployment.environment describe the producer and make data searchable.

Semantic conventions standardize attribute names like http.request.method and db.system, so dashboards work across languages.

Manual spans

from opentelemetry import trace

tracer = trace.get_tracer("checkout")

def charge(order):
    with tracer.start_as_current_span("charge-card") as span:
        span.set_attribute("order.id", order.id)
        span.set_attribute("order.items", len(order.items))
        try:
            return gateway.pay(order)
        except Exception as exc:
            span.record_exception(exc)
            span.set_status(trace.Status(trace.StatusCode.ERROR))
            raise

The Collector

Run a Collector between applications and backends. It batches, retries, filters, redacts and fans out data.

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  memory_limiter:
    check_interval: 1s
    limit_mib: 512
  batch: {}

exporters:
  otlp/tempo:
    endpoint: tempo:4317
    tls:
      insecure: true
  prometheus:
    endpoint: 0.0.0.0:8889

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlp/tempo]
    metrics:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [prometheus]

Run it with otelcol-contrib --config otel-config.yaml or the matching container image.

Pitfalls

• Missing context propagation. Message queues, async jobs and custom HTTP clients break traces unless you inject and extract context.
• High-cardinality attributes such as user ids or full URLs as metric labels explode storage costs. Keep them on spans, not metrics.
• Sampling everything. Full tracing at scale is expensive. Use head sampling in the SDK, or tail sampling in the Collector to keep errors and slow traces.
• Skipping the memory limiter can let the Collector run out of memory under load.
• Sensitive data in attributes. Scrub tokens, emails and query parameters with an attributes or redaction processor.
• Inconsistent service names fragment service maps. Set OTEL_SERVICE_NAME everywhere.
• Version drift. Signal stability differs: traces and metrics are stable in most SDKs, logs vary by language.

How it fits an architecture

OTel is the collection layer of an observability stack. Applications emit OTLP to a local or sidecar Collector, an agent tier forwards to a gateway Collector that applies sampling and routing, and backends store traces, metrics and logs. Pair it with Grafana for visualization and use trace ids to jump from a metric spike to a trace to its related log lines.

Key takeaways

• OpenTelemetry standardizes instrumentation and decouples it from the backend.
• Start with auto-instrumentation, then add manual spans for business operations.
• Always deploy a Collector with a memory limiter and batching.
• Propagate context everywhere or traces fragment.
• Control cost with sampling and low-cardinality metric labels.
• Use resource attributes and semantic conventions consistently.