OpenTelemetry, Prometheus & Tracing

OpenTelemetry, Prometheus remote-write, Jaeger trace queries, and RUM web-vitals ingest for BugWatch.

Scope required for all write endpoints: ingest:write

Key facts:

  • OTLP/HTTP (/v1/logs, /v1/traces, /v1/metrics) accept both application/json and application/x-protobuf. The OpenTelemetry Collector sends protobuf by default - the requests in this folder use JSON for readability.
  • POST /api/v1/prom/write accepts Snappy-compressed Prometheus protobuf - this is always sent by the Prometheus remote-write adapter or the bugwatch-otel-collector. This folder documents the endpoint but does not include a fake binary body.
  • Prometheus GET query endpoints (/query, /query_range, /label/__name__/values) are fully runnable with real query params.
  • Jaeger GET endpoints (/jaeger/api/services, /jaeger/api/traces) are fully runnable.
  • RUM (/api/v1/rum) accepts x-bugwatch-session (browser) or x-api-key (server).

Auth header: x-api-key: {{apiKey}} or Authorization: Bearer {{apiKey}}

POST/v1/logs

OTLP/HTTP – ingest logs (JSON)

Send OpenTelemetry logs to BugWatch via OTLP/HTTP.

Scope: ingest:write

Content-Type options

  • application/json - JSON body (used here)
  • application/x-protobuf - protobuf ExportLogsServiceRequest (used by the OTel Collector)

Request body (JSON form)

{
  "resourceLogs": [{
    "resource": { "attributes": [{ "key": "service.name", "value": { "stringValue": "my-service" } }] },
    "scopeLogs": [{
      "logRecords": [{
        "timeUnixNano": "1750924800000000000",
        "severityNumber": 9,
        "severityText": "INFO",
        "body": { "stringValue": "User signed in" },
        "attributes": [{ "key": "user.id", "value": { "stringValue": "usr_123" } }]
      }]
    }]
  }]
}

Success - 200 OK (empty body on full success)

{}

On partial rejection:

{ "partialSuccess": { "rejectedLogRecords": 2, "errorMessage": "some records rejected" } }

Headers

x-api-key

Request body

application/json
{
  "resourceLogs": [
    {
      "resource": {
        "attributes": [
          {
            "key": "service.name",
            "value": {
              "stringValue": "my-service"
            }
          }
        ]
      },
      "scopeLogs": [
        {
          "logRecords": [
            {
              "timeUnixNano": "{{nowNano}}",
              "severityNumber": 9,
              "severityText": "INFO",
              "body": {
                "stringValue": "User signed in"
              },
              "attributes": [
                {
                  "key": "user.id",
                  "value": {
                    "stringValue": "usr_123"
                  }
                }
              ]
            }
          ]
        }
      ]
    }
  ]
}

Responses

200 – All accepted

null
boltTry it
env
POSThttps://api.newinstance.cloud/v1/logs

Headers

x-api-key

Request body

Code samples

curl -X POST 'https://api.newinstance.cloud/v1/logs' \
  -H 'Content-Type: application/json' \
  --data-raw '{
  "resourceLogs": [
    {
      "resource": {
        "attributes": [
          {
            "key": "service.name",
            "value": {
              "stringValue": "my-service"
            }
          }
        ]
      },
      "scopeLogs": [
        {
          "logRecords": [
            {
              "timeUnixNano": "{{nowNano}}",
              "severityNumber": 9,
              "severityText": "INFO",
              "body": {
                "stringValue": "User signed in"
              },
              "attributes": [
                {
                  "key": "user.id",
                  "value": {
                    "stringValue": "usr_123"
                  }
                }
              ]
            }
          ]
        }
      ]
    }
  ]
}'
POST/v1/traces

OTLP/HTTP – ingest traces (JSON)

Send OpenTelemetry traces to BugWatch via OTLP/HTTP.

Scope: ingest:write

Content-Type options

  • application/json - JSON body (used here)
  • application/x-protobuf - protobuf ExportTraceServiceRequest (used by the OTel Collector)

Request body (JSON form)

{
  "resourceSpans": [{
    "resource": { "attributes": [{ "key": "service.name", "value": { "stringValue": "my-service" } }] },
    "scopeSpans": [{
      "spans": [{
        "traceId": "4bf92f3577b34da6a3ce929d0e0e4736",
        "spanId": "00f067aa0ba902b7",
        "name": "HTTP GET /api/users",
        "kind": 2,
        "startTimeUnixNano": "1750924800000000000",
        "endTimeUnixNano": "1750924800050000000",
        "status": { "code": 1 }
      }]
    }]
  }]
}

Success - 200 OK (empty body on full success; partialSuccess on rejection)

Headers

x-api-key

Request body

application/json
{
  "resourceSpans": [
    {
      "resource": {
        "attributes": [
          {
            "key": "service.name",
            "value": {
              "stringValue": "my-service"
            }
          }
        ]
      },
      "scopeSpans": [
        {
          "spans": [
            {
              "traceId": "4bf92f3577b34da6a3ce929d0e0e4736",
              "spanId": "00f067aa0ba902b7",
              "name": "HTTP GET /api/users",
              "kind": 2,
              "startTimeUnixNano": "{{nowNano}}",
              "endTimeUnixNano": "{{nowNano}}",
              "status": {
                "code": 1
              },
              "attributes": [
                {
                  "key": "http.method",
                  "value": {
                    "stringValue": "GET"
                  }
                },
                {
                  "key": "http.status_code",
                  "value": {
                    "intValue": 200
                  }
                }
              ]
            }
          ]
        }
      ]
    }
  ]
}

Responses

200 – All accepted

null
boltTry it
env
POSThttps://api.newinstance.cloud/v1/traces

Headers

x-api-key

Request body

Code samples

curl -X POST 'https://api.newinstance.cloud/v1/traces' \
  -H 'Content-Type: application/json' \
  --data-raw '{
  "resourceSpans": [
    {
      "resource": {
        "attributes": [
          {
            "key": "service.name",
            "value": {
              "stringValue": "my-service"
            }
          }
        ]
      },
      "scopeSpans": [
        {
          "spans": [
            {
              "traceId": "4bf92f3577b34da6a3ce929d0e0e4736",
              "spanId": "00f067aa0ba902b7",
              "name": "HTTP GET /api/users",
              "kind": 2,
              "startTimeUnixNano": "{{nowNano}}",
              "endTimeUnixNano": "{{nowNano}}",
              "status": {
                "code": 1
              },
              "attributes": [
                {
                  "key": "http.method",
                  "value": {
                    "stringValue": "GET"
                  }
                },
                {
                  "key": "http.status_code",
                  "value": {
                    "intValue": 200
                  }
                }
              ]
            }
          ]
        }
      ]
    }
  ]
}'
POST/v1/metrics

OTLP/HTTP – ingest metrics (JSON)

Send OpenTelemetry metrics to BugWatch via OTLP/HTTP.

Scope: ingest:write

Content-Type options

  • application/json - JSON body (used here)
  • application/x-protobuf - protobuf ExportMetricsServiceRequest (used by the OTel Collector)

Request body (JSON form)

{
  "resourceMetrics": [{
    "resource": { "attributes": [{ "key": "service.name", "value": { "stringValue": "my-service" } }] },
    "scopeMetrics": [{
      "metrics": [{
        "name": "http.server.request.duration",
        "unit": "ms",
        "gauge": {
          "dataPoints": [{
            "timeUnixNano": "1750924800000000000",
            "asDouble": 123.4,
            "attributes": [{ "key": "http.method", "value": { "stringValue": "GET" } }]
          }]
        }
      }]
    }]
  }]
}

Success - 200 OK (empty on full success; partialSuccess on rejection)

Headers

x-api-key

Request body

application/json
{
  "resourceMetrics": [
    {
      "resource": {
        "attributes": [
          {
            "key": "service.name",
            "value": {
              "stringValue": "my-service"
            }
          }
        ]
      },
      "scopeMetrics": [
        {
          "metrics": [
            {
              "name": "http.server.request.duration",
              "unit": "ms",
              "gauge": {
                "dataPoints": [
                  {
                    "timeUnixNano": "{{nowNano}}",
                    "asDouble": 123.4,
                    "attributes": [
                      {
                        "key": "http.method",
                        "value": {
                          "stringValue": "GET"
                        }
                      }
                    ]
                  }
                ]
              }
            }
          ]
        }
      ]
    }
  ]
}

Responses

200 – All accepted

null
boltTry it
env
POSThttps://api.newinstance.cloud/v1/metrics

Headers

x-api-key

Request body

Code samples

curl -X POST 'https://api.newinstance.cloud/v1/metrics' \
  -H 'Content-Type: application/json' \
  --data-raw '{
  "resourceMetrics": [
    {
      "resource": {
        "attributes": [
          {
            "key": "service.name",
            "value": {
              "stringValue": "my-service"
            }
          }
        ]
      },
      "scopeMetrics": [
        {
          "metrics": [
            {
              "name": "http.server.request.duration",
              "unit": "ms",
              "gauge": {
                "dataPoints": [
                  {
                    "timeUnixNano": "{{nowNano}}",
                    "asDouble": 123.4,
                    "attributes": [
                      {
                        "key": "http.method",
                        "value": {
                          "stringValue": "GET"
                        }
                      }
                    ]
                  }
                ]
              }
            }
          ]
        }
      ]
    }
  ]
}'
POST/api/v1/prom/write

Prometheus remote-write (DOCUMENTATION ONLY)

Prometheus remote-write ingestion endpoint.

Scope: ingest:write

This endpoint is NOT called directly by humans. It is invoked by the Prometheus remote-write adapter or the bugwatch-otel-collector. The body is a Snappy-compressed protobuf WriteRequest - you cannot send a readable JSON body here.

To use Prometheus remote-write with BugWatch:

  1. Deploy the bugwatch-otel-collector (or configure Prometheus remote_write)
  2. Set the remote-write URL to: {{baseUrl}}/api/v1/prom/write
  3. Add header: x-api-key: sk_live_KEYID:secret

Success - 204 No Content

Common errors

  • 400 - malformed Snappy/protobuf body
  • 401 - invalid API key
  • 429 - rate limited

Headers

x-api-key
X-Prometheus-Remote-Write-Version

Request body

text/plain
<<< BINARY SNAPPY-COMPRESSED PROTOBUF - sent by Prometheus/collector, not a manual JSON request >>>

Responses

204 – Accepted

null
boltTry it
env
POSThttps://api.newinstance.cloud/api/v1/prom/write

Headers

x-api-key
X-Prometheus-Remote-Write-Version

Request body

Code samples

curl -X POST 'https://api.newinstance.cloud/api/v1/prom/write' \
  -H 'Content-Type: application/json' \
  --data-raw '<<< BINARY SNAPPY-COMPRESSED PROTOBUF - sent by Prometheus/collector, not a manual JSON request >>>'
GET/api/v1/prom/query

Prometheus query (instant)

Query stored Prometheus metrics using a PromQL-compatible selector.

Scope: ingest:write (same key used for write is accepted for query)

Query params

  • query (required) - metric name or label selector e.g. http_requests_total or {service="api-gateway"}
  • time (optional) - Unix timestamp (seconds, float); defaults to now

Success - 200 OK

{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [{ "metric": { "__name__": "http_requests_total", "service": "api" }, "value": [1750924800, "42"] }]
  }
}

Headers

x-api-key

Parameters

queryquerystringdefault: Metric name or label selector
timequerystringdefault: Unix timestamp seconds (optional, defaults to now)

Responses

200 – Instant vector

{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "__name__": "http_requests_total",
          "service": "api"
        },
        "value": [
          1785933561,
          "42"
        ]
      }
    ]
  }
}
boltTry it
env
GEThttps://api.newinstance.cloud/api/v1/prom/query

Query parameters

query
time

Headers

x-api-key

Code samples

curl -X GET 'https://api.newinstance.cloud/api/v1/prom/query'
GET/api/v1/prom/query_range

Prometheus query_range (over time)

Query stored Prometheus metrics over a time range.

Scope: ingest:write

Query params

  • query (required) - metric selector
  • start (required) - range start as Unix seconds (float)
  • end (required) - range end as Unix seconds (float)

Success - 200 OK

{
  "status": "success",
  "data": {
    "resultType": "matrix",
    "result": [{ "metric": { "__name__": "http_requests_total" }, "values": [[1750924800, "42"], [1750924860, "47"]] }]
  }
}

Headers

x-api-key

Parameters

queryquerystringdefault: Metric name or label selector
startquerystringdefault: Range start Unix seconds
endquerystringdefault: Range end Unix seconds

Responses

200 – Matrix result

{
  "status": "success",
  "data": {
    "resultType": "matrix",
    "result": [
      {
        "metric": {
          "__name__": "http_requests_total"
        },
        "values": [
          [
            1785929961,
            "38"
          ],
          [
            1785933561,
            "42"
          ]
        ]
      }
    ]
  }
}
boltTry it
env
GEThttps://api.newinstance.cloud/api/v1/prom/query_range

Query parameters

query
start
end

Headers

x-api-key

Code samples

curl -X GET 'https://api.newinstance.cloud/api/v1/prom/query_range'
GET/api/v1/prom/label/__name__/values

Prometheus label __name__ values (list metric names)

List all distinct metric names stored for this project+environment.

Scope: ingest:write

Success - 200 OK

{ "status": "success", "data": ["http_requests_total", "rum.lcp", "rum.cls"] }

Headers

x-api-key

Responses

200 – Metric names

{
  "status": "success",
  "data": [
    "http_requests_total",
    "rum.lcp",
    "rum.cls",
    "rum.inp"
  ]
}
boltTry it
env
GEThttps://api.newinstance.cloud/api/v1/prom/label/__name__/values

Headers

x-api-key

Code samples

curl -X GET 'https://api.newinstance.cloud/api/v1/prom/label/__name__/values'
GET/jaeger/api/services

Jaeger – list service names

List distinct service names that have sent traces to this BugWatch project.

Scope: ingest:write (same API key)

Success - 200 OK

{ "data": ["api-gateway", "payment-service", "user-service"], "total": 3, "limit": 3, "offset": 0, "errors": null }

Headers

x-api-key

Responses

200 – Services

{
  "data": [
    "api-gateway",
    "payment-service",
    "user-service"
  ],
  "total": 3,
  "limit": 3,
  "offset": 0,
  "errors": null
}
boltTry it
env
GEThttps://api.newinstance.cloud/jaeger/api/services

Headers

x-api-key

Code samples

curl -X GET 'https://api.newinstance.cloud/jaeger/api/services'
GET/jaeger/api/traces

Jaeger – list traces

List recent distributed traces for this project+environment.

Scope: ingest:write

Query params

  • service (optional) - filter by service name
  • limit (optional, default 20, max 100)
  • lookback (optional) - 1h | 24h | 7d | 30d (default 1h)

Success - 200 OK - returns Jaeger-compatible trace list with spans.

Headers

x-api-key

Parameters

limitquerystringdefault: Max traces (default 20, max 100)
lookbackquerystringdefault: 1h | 24h | 7d | 30d

Responses

200 – Traces list

{
  "data": [],
  "total": 0,
  "limit": 20,
  "offset": 0,
  "errors": null
}
boltTry it
env
GEThttps://api.newinstance.cloud/jaeger/api/traces

Query parameters

limit
lookback

Headers

x-api-key

Code samples

curl -X GET 'https://api.newinstance.cloud/jaeger/api/traces'
POST/api/v1/rum

RUM – ingest web-vitals

Ingest Real User Monitoring (RUM) web-vitals metrics (LCP, CLS, INP, FID, TTFB).

Auth: Either x-bugwatch-session (browser) or x-api-key (server-side / testing)

Scope (for x-api-key): ingest:write

Request body - three accepted shapes:

  1. Wrapped array:
{ "vitals": [{ "name": "LCP", "value": 1250.5, "url": "https://example.com/", "rating": "good" }] }
  1. Bare array:
[{ "name": "CLS", "value": 0.05, "rating": "good" }, { "name": "INP", "value": 180, "rating": "needs-improvement" }]
  1. Single vital object:
{ "name": "TTFB", "value": 320, "url": "https://example.com/checkout", "rating": "needs-improvement" }

Vital field schema:

FieldTypeNotes
namestring (required)LCP, CLS, INP, FID, TTFB
valuenumber (required)Metric value in ms (or unitless for CLS)
urlstring (optional)Page URL
ratingstring (optional)good
idstring (optional)Client-side unique id

Success - 202 Accepted

{ "ingested": 2 }

Stored as rum.<name_lower> metric points (e.g. rum.lcp, rum.cls).

Common errors

  • 400 - malformed JSON
  • 401 - invalid session token or API key
  • 403 - origin not in allowedOrigins (when using session token)
  • 429 - rate limited

Headers

x-api-keyOr use x-bugwatch-session for browser calls

Request body

application/json
{
  "vitals": [
    {
      "name": "LCP",
      "value": 1250.5,
      "url": "https://example.com/",
      "rating": "good"
    },
    {
      "name": "CLS",
      "value": 0.05,
      "url": "https://example.com/",
      "rating": "good"
    },
    {
      "name": "INP",
      "value": 180,
      "url": "https://example.com/",
      "rating": "needs-improvement"
    }
  ]
}

Responses

202 – Vitals ingested

{
  "ingested": 3
}
boltTry it
env
POSThttps://api.newinstance.cloud/api/v1/rum

Headers

x-api-key

Request body

Code samples

curl -X POST 'https://api.newinstance.cloud/api/v1/rum' \
  -H 'Content-Type: application/json' \
  --data-raw '{
  "vitals": [
    {
      "name": "LCP",
      "value": 1250.5,
      "url": "https://example.com/",
      "rating": "good"
    },
    {
      "name": "CLS",
      "value": 0.05,
      "url": "https://example.com/",
      "rating": "good"
    },
    {
      "name": "INP",
      "value": 180,
      "url": "https://example.com/",
      "rating": "needs-improvement"
    }
  ]
}'