OpenTelemetry

First PublishedFeb 16, 2026ByAtif Alam

OpenTelemetry (OTel) is an open-source, vendor-neutral framework for generating, collecting, and exporting telemetry data — metrics, logs, and traces — from your applications. It’s becoming the industry standard for instrumentation, backed by the CNCF.

Why OpenTelemetry?

Without OTel, you instrument your app separately for each backend:

• Prometheus client library for metrics
• A logging library for logs
• A Jaeger/Zipkin SDK for traces

With OTel, you instrument once and export to any backend:

┌─────────────┐     OTel SDK      ┌──────────────┐     export     ┌──────────────┐
│  Your App   │──────────────────►│  OTel        │───────────────►│  Prometheus  │
│             │  metrics, logs,   │  Collector   │               │  Loki        │
│             │  traces           │              │               │  Jaeger/Tempo│
└─────────────┘                   └──────────────┘               └──────────────┘

Benefits

• Vendor-neutral — Switch backends (Prometheus → Datadog, Jaeger → Tempo) without changing application code.
• One SDK, three signals — Metrics, logs, and traces from a single library.
• Auto-instrumentation — Get traces and metrics from HTTP frameworks, databases, and gRPC with zero code changes.
• Correlation — Trace IDs link metrics, logs, and traces together so you can jump between signals.
• CNCF standard — Wide industry adoption; all major vendors support it.

Core Concepts

Concept	What It Is
Signal	A type of telemetry: metrics, logs, or traces
Span	A single operation within a trace (e.g. an HTTP request, a database query)
Trace	A tree of spans representing an end-to-end request across services
Exporter	Sends telemetry to a backend (Prometheus, OTLP, Jaeger, etc.)
Collector	A standalone service that receives, processes, and exports telemetry
Context propagation	Passing trace IDs between services (via HTTP headers, gRPC metadata)
Resource	Metadata about where telemetry comes from (service name, version, host)

Instrumentation

Manual Instrumentation (Python Example)

from opentelemetry import trace, metrics
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.exporter.otlp.proto.grpc.metric_exporter import OTLPMetricExporter
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader

# Set up tracing
trace.set_tracer_provider(TracerProvider())
trace.get_tracer_provider().add_span_processor(
    BatchSpanProcessor(OTLPSpanExporter(endpoint="http://otel-collector:4317"))
)

# Set up metrics
reader = PeriodicExportingMetricReader(
    OTLPMetricExporter(endpoint="http://otel-collector:4317")
)
metrics.set_meter_provider(MeterProvider(metric_readers=[reader]))

# Use in your code
tracer = trace.get_tracer("my-app")
meter = metrics.get_meter("my-app")

request_counter = meter.create_counter(
    "http_requests_total",
    description="Total HTTP requests",
)

@app.route("/api/data")
def get_data():
    with tracer.start_as_current_span("get_data") as span:
        span.set_attribute("http.method", "GET")
        span.set_attribute("http.route", "/api/data")
        request_counter.add(1, {"method": "GET", "status": "200"})
        return fetch_data()

Manual Instrumentation (Go Example)

import (
    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/trace"
)

tracer := otel.Tracer("my-app")

func handleRequest(w http.ResponseWriter, r *http.Request) {
    ctx, span := tracer.Start(r.Context(), "handleRequest")
    defer span.End()

    span.SetAttributes(
        attribute.String("http.method", r.Method),
        attribute.String("http.url", r.URL.Path),
    )

    result := fetchData(ctx)  // pass context to propagate trace
    json.NewEncoder(w).Encode(result)
}

Auto-Instrumentation (Zero Code Changes)

OTel provides auto-instrumentation that patches common libraries automatically:

Python:

pip install opentelemetry-distro opentelemetry-exporter-otlp
opentelemetry-bootstrap -a install    # auto-install instrumentation packages

# Run your app with auto-instrumentation
opentelemetry-instrument \
  --service_name my-app \
  --exporter_otlp_endpoint http://otel-collector:4317 \
  python app.py

This automatically instruments Flask/Django/FastAPI, requests, SQLAlchemy, psycopg2, redis, and more — no code changes.

Java (agent):

java -javaagent:opentelemetry-javaagent.jar \
  -Dotel.service.name=my-app \
  -Dotel.exporter.otlp.endpoint=http://otel-collector:4317 \
  -jar my-app.jar

Node.js:

npm install @opentelemetry/auto-instrumentations-node

node --require @opentelemetry/auto-instrumentations-node/register app.js

What Auto-Instrumentation Captures

Library Type	Examples	Signals
HTTP frameworks	Flask, Express, Spring	Traces + metrics
HTTP clients	requests, axios, HttpClient	Traces
Databases	SQLAlchemy, pg, mysql2	Traces
Message queues	Kafka, RabbitMQ, SQS	Traces
gRPC	grpc-python, grpc-java	Traces + metrics
Redis	redis-py, ioredis	Traces

The OTel Collector

The Collector is a proxy that sits between your apps and your backends. It receives telemetry, processes it, and exports it to one or more destinations.

App 1 ──► ┌──────────────────────────────────────┐ ──► Prometheus
App 2 ──► │  OTel Collector                       │ ──► Loki
App 3 ──► │  receivers → processors → exporters   │ ──► Tempo/Jaeger
          └──────────────────────────────────────┘

Why Use a Collector?

• Decouple apps from backends — Apps send to the Collector; the Collector routes to backends. Change backends without touching apps.
• Processing — Filter, sample, batch, add attributes, transform data before exporting.
• Multiple backends — Send traces to Tempo and Jaeger, metrics to Prometheus and Datadog, from one pipeline.
• Reduce app overhead — Offload batching and retry logic to the Collector.

Collector Configuration

otel-collector-config.yaml

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  batch:
    timeout: 5s
    send_batch_size: 1000
  memory_limiter:
    check_interval: 1s
    limit_mib: 512
  resource:
    attributes:
      - key: environment
        value: production
        action: upsert

exporters:
  prometheus:
    endpoint: 0.0.0.0:8889     # Prometheus scrapes this
  otlp/tempo:
    endpoint: tempo:4317         # send traces to Tempo
    tls:
      insecure: true
  loki:
    endpoint: http://loki:3100/loki/api/v1/push

service:
  pipelines:
    metrics:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [prometheus]
    traces:
      receivers: [otlp]
      processors: [memory_limiter, batch, resource]
      exporters: [otlp/tempo]
    logs:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [loki]

Pipeline Structure

Each pipeline has three stages:

Stage	Purpose	Examples
Receivers	Accept telemetry data	otlp, prometheus, jaeger, filelog
Processors	Transform, filter, batch	batch, memory_limiter, filter, resource, tail_sampling
Exporters	Send to backends	prometheus, otlp, loki, jaeger, datadog

Deployment Patterns

Agent (per-host sidecar):

┌──────────┐     ┌───────────┐
│  App Pod  │────►│ Collector │──► Backend
│           │     │ (sidecar) │
└──────────┘     └───────────┘

Each pod or host runs a Collector instance. Low latency, local processing.

Gateway (central):

App 1 ──►┐
App 2 ──►├──► Collector Gateway ──► Backend
App 3 ──►┘

One (or a few) Collector instances for the whole cluster. Simpler to manage, single point to configure exports.

Agent + Gateway (recommended for production):

App ──► Collector Agent (sidecar) ──► Collector Gateway ──► Backend

Agents handle local collection; the gateway handles routing, sampling, and export. Most flexible.

OTel vs Prometheus Client Libraries

	Prometheus Client	OpenTelemetry
Signals	Metrics only	Metrics + traces + logs
Pull/push	Pull (Prometheus scrapes)	Push (app → Collector)
Vendor lock	Prometheus format	Vendor-neutral (OTLP)
Auto-instrumentation	No	Yes
Trace correlation	No	Yes (trace IDs in metrics/logs)

You can use both: OTel for traces and the Prometheus client for metrics. Or go all-in on OTel — the Collector can export metrics in Prometheus format.

Key Takeaways

• OpenTelemetry gives you one SDK for metrics, logs, and traces — instrument once, export to any backend.
• Auto-instrumentation captures HTTP, database, and gRPC telemetry with zero code changes.
• The OTel Collector decouples your apps from backends: receivers → processors → exporters.
• Use the agent + gateway pattern in production for flexibility and reliability.
• OTel and Prometheus complement each other — you don’t have to choose one exclusively.