Logs, metrics and traces: choosing the signal

What it is

Observability tooling distinguishes three signals. Logs are timestamped event records with arbitrary detail. Metrics are numeric time series (counters, gauges, histograms) that are cheap to store and query in aggregate. Traces represent a single request as a tree of spans, each with a start time, duration, attributes and a parent, propagated across service boundaries through context headers. OpenTelemetry defines APIs, SDKs and a wire protocol for all three.

Why it matters

Each signal answers different questions. Metrics tell you that error rate rose at 14:02; traces show which downstream call in which requests was slow; logs show the exact error text of one of them. Correlation identifiers shared across signals turn three views into one investigation.

How to apply

• Instrument request boundaries first (HTTP server and client, database calls); libraries often provide this automatically.
• Record a small number of metrics with bounded label cardinality (endpoint, status class), not per user or per id.
• Attach the trace id to log records and propagate it to downstream services.
• Sample traces in high-volume systems; keep all error traces.

Pitfalls

High-cardinality metric labels explode storage. Traces without propagation stop at the first service boundary. Collecting everything without a question in mind produces cost, not insight.