# OpenTelemetry Collector

A ready-made OpenTelemetry Collector gateway that batches telemetry from your fleet and forwards it to BugWatch. It uses the upstream `otel/opentelemetry-collector-contrib:0.103.0` image, pinned in both the Docker Compose file and the Kubernetes manifests. Point your fleet's OTLP exporters at the collector; it batches, applies memory limits, and forwards everything to BugWatch. Use it when you want one egress point, local buffering, or to keep API keys off application hosts. The project dashboard's Setup guide tab serves this config pre-filled.

## Quick start (Docker Compose)

```bash
BUGWATCH_API_KEY="sk_live_KEYID:secret" docker compose up -d
```

Compose refuses to start without `BUGWATCH_API_KEY` (the key needs `ingest:write`). `BUGWATCH_HTTP` defaults to `https://api.newinstance.cloud`; ports 4317 (gRPC) and 4318 (HTTP) are published; the container restarts unless stopped.

## Point your application at it

Standard OpenTelemetry env vars, any language:

```bash
OTEL_EXPORTER_OTLP_ENDPOINT=http://collector-host:4318
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
```

Go gotcha: `otlptracehttp.WithEndpoint` takes `host:port` only and appends `/v1/traces` itself; passing a full URL there breaks the exporter.

## Pipeline

OTLP receivers (gRPC 4317, HTTP 4318) then `memory_limiter` (512 MiB hard cap plus a 128 MiB spike allowance: on breach it drops the batch and logs instead of being OOM killed) then a `resource` upsert (`collector=bugwatch-otel-collector`) then `batch` (512 / 5 s) then the `otlphttp/bugwatch` exporter (gzip) to `/v1/logs`, `/v1/traces` and `/v1/metrics` with `x-api-key`. Health check on 13133. A `debug` exporter (verbosity basic or detailed) is defined but wired to no pipeline; add it temporarily to inspect payloads. The collector's own log level lives at `service.telemetry.logs.level`.

A gRPC exporter block (`otlp/bugwatch`, target `api.newinstance.cloud:4317`) ships commented out: to use it, uncomment it, add it to each pipeline, and set its TLS block deliberately.

## Kubernetes

Apply `namespace`, `secret`, `configmap`, then `deployment`: 2 replicas behind a ClusterIP, reachable in-cluster at `bugwatch-otel-collector.bugwatch.svc.cluster.local:4317|4318`; liveness and readiness probe 13133; requests 250m/256Mi, limits 1000m/768Mi; scale with `kubectl scale`.

## Config changes

Validate before shipping: `otelcol-contrib validate --config config.yaml` (or the same command through docker). Env references must use the `${env:VAR}` form. TLS: outbound HTTPS to BugWatch needs no config; a self-signed backend needs `tls.ca_file` mounted; in Kubernetes terminate inbound TLS at the ingress or mesh and keep the receivers plain. Tuning: raise `limit_mib` when you see `memory_limiter: dropping data`; raise `send_batch_size` toward 2048 to 4096 above roughly 10k spans/s.

## Alternatives without the collector

- Prometheus `remote_write` straight to `https://api.newinstance.cloud/api/v1/prom/write` with `x-api-key`; Grafana reads from `https://api.newinstance.cloud/api/v1/prom`.
- A Jaeger UI can point at `https://api.newinstance.cloud/jaeger/api`.
- Apps can also speak OTLP directly to the platform (see [OpenTelemetry, Prometheus & Tracing](/opentelemetry-prometheus-tracing)).

## Troubleshooting

- Reachability: `curl -v http://localhost:4318/v1/logs -H 'Content-Type: application/json' -d '{}'`; a 400 response (not a connection error) proves the collector is up.
- Kubernetes CrashLoopBackOff is usually a double base64-encoded secret; decode it and check it reads `keyId:secret`.