Stackbook Logo
← Back to System Architectures

Structured Logging Strategies

Logs are often the first thing engineers reach for when debugging. Poor logging makes debugging a nightmare.

3 min read

Logs are often the first thing engineers reach for when debugging. Poor logging makes debugging a nightmare.

Structured vs. Unstructured Logs

Unstructured Logs

2026-06-01T10:30:45 ERROR Failed to process order

Problems: Hard to query, hard to correlate across services, difficult to filter, can't easily extract timestamps or error codes.

Structured Logs (JSON Preferred)

{"timestamp":"2026-06-01T10:30:45Z","level":"error","service":"order-processor","message":"Failed to process order","order_id":"order_123","error_code":"DATABASE_TIMEOUT","trace_id":"trace_abc123"}

Benefits: Easy to query, easy to correlate, easy to filter, works with log analysis tools, consistent format.

Logging Strategy

1. Log What Matters

Don't log everything. Log:

  • Errors and exceptions with full stack traces at error level
  • Business events that are important for auditing or understanding flow
  • Performance issues slow operations, timeouts, high latency
  • Security events authentication failures, authorization violations
  • Startup and shutdown of services
  • Configuration changes that affect behavior

Don't log:

  • Every request (unless you're debugging a specific issue)
  • Every database query (use slow query log instead)
  • Heartbeat/ping messages (too noisy)
  • Information that's already in metrics

2. Consistent Structure

Use a consistent log structure across all services. JSON is the most common format. Include at minimum:

{
  "timestamp": "ISO8601",
  "level": "error|info|warn|debug",
  "service": "service-name",
  "instance": "instance-id-or-hostname",
  "trace_id": "correlation-id",
  "span_id": "within-service-span-id",
  "message": "human-readable description",
  "key_value_pairs": {"field": "value"}
}

Avoid: Free-form log messages that are difficult to query and correlate.

2. Correlation IDs

Every important request should have a correlation ID that flows through the entire system.

How it works:

  1. Generate correlation ID at the entry point (API gateway, entry controller)
  2. Pass it through all middleware and service calls
  3. Include in all logs, metrics, and traces related to that request
  4. Allow users/searchers to filter logs by correlation ID

Key: The correlation ID must propagate through all layers: frontend, API gateway, backend services, databases (if logging SQL), external service calls.

2. Log Levels Have Meaning

Define log levels consistently across the organization:

  • debug: Detailed information for development debugging. Don't in production.
  • info: Normal operation highlights. What you should see in normal operations.
  • warn: Potential issues, not yet errors. Worth watching.
  • error: Something went wrong. Requires attention.
  • critical: Severe issue. Immediate action required.

Key: Implementation should respect these levels. Don't log everything at error level and wonder why you're overwhelmed.

2. Log Destinations

Where do logs go?

  • Development: Stdout/stderr (viewable in container logs)
  • Staging: Centralized log system (ELK, Datadog, Loki)
  • Production: Centralized log system with search and analysis

Key: Consistent destinations across environments. Don't make developers guess where logs go.

2. Log Retention

How long do you keep logs?

  • Development: Keep as long as needed for debugging (often configurable)
  • Staging: 30-90 days typically
  • Production: 30-365 days depending on compliance requirements and cost

Key: Balance between useful debugging time and cost. Log storage is expensive at scale.

2. Log Analysis Tools

Tools make or break your logging strategy:

  • ELK Stack (Elasticsearch, Logstash, Kibana): Popular, flexible, self-hosted
  • Datadog: SaaS, excellent UX, expensive at scale
  • Loki: Grafana-backed, cost-effective for structured logs
  • Splunk: Enterprise-focused, powerful, expensive

Key: Choose based on budget, team expertise, and scale. Don't choose based on marketing alone.

The Honest Truth

Logging is often an afterthought until debugging becomes impossible. Structured logging from day one saves countless hours. But don't over-log—log what matters, structure it consistently, and make it searchable.

Published June 1, 2026 · 11 min read