ANW-42 Telemetry: take export config from OTEL_ environment variables only
Delete the uptrace DSN parser and the {endpoint}/v1/* concatenation that
produced the dead export, and build the OTLP exporters with no endpoint,
headers, or protocol so the SDK reads OTEL_EXPORTER_OTLP_* itself.
The SDK concatenates as naively as we did: measured against a local sink,
a ?query base sends POST /?query/v1/metrics and a #fragment base sends
POST /. So one check stays -- a query or fragment on
OTEL_EXPORTER_OTLP_ENDPOINT fails at startup. Per-signal variables are
used verbatim and need none.
A protocol variable is the second startup check. The exporter picks its
transport at build time and this binary ships OTLP/HTTP alone, so
OTEL_EXPORTER_OTLP_PROTOCOL=grpc kept exporting over HTTP with nothing in
the log -- the same silence this change removes.
service.name stays a default rather than policy: an attribute set on the
resource builder wins over the SDK detectors that read OTEL_SERVICE_NAME
and OTEL_RESOURCE_ATTRIBUTES, so anwesen supplies its own only for keys
the environment leaves alone.
Telemetry-off is our own check on the endpoint variables: with none set
the exporter would still build and aim at the SDK default localhost:4318.
--uptrace-dsn, --otlp-endpoint, and --otlp-header stay parsed but hidden,
so a deployment upgrading with them set fails naming the OTEL_
replacement instead of going quiet.
Assumed hidden clap stubs are the right migration shape; clap rejecting
the flags as unknown would say nothing about the replacement. Flag if a
plain unknown-argument error is wanted instead.
Assumed rejecting http/json alongside grpc is right: neither has a
transport in this build. Flag if a build with both features is wanted
instead.
This commit is contained in:
parent
8894ea0c7b
commit
dc113b8249
5 changed files with 563 additions and 227 deletions
59
README.md
59
README.md
|
|
@ -56,8 +56,7 @@ anwesen merge --vault /path/to/vault --query 'tags=adr&__anw-order=title' > ADRs
|
||||||
|
|
||||||
```
|
```
|
||||||
anwesen serve --vault <path> [--bind <addr:port>] [--log-level <level>]
|
anwesen serve --vault <path> [--bind <addr:port>] [--log-level <level>]
|
||||||
[--uptrace-dsn <dsn> | --otlp-endpoint <url>]
|
[--otlp-slow-request-ms <n>]
|
||||||
[--otlp-header <key=value>]... [--otlp-slow-request-ms <n>]
|
|
||||||
anwesen doctor --vault <path>
|
anwesen doctor --vault <path>
|
||||||
anwesen merge --vault <path> --query <query-string>
|
anwesen merge --vault <path> --query <query-string>
|
||||||
anwesen version
|
anwesen version
|
||||||
|
|
@ -69,24 +68,62 @@ anwesen version
|
||||||
| `--bind <addr:port>` | `ANWESEN_BIND` | `127.0.0.1:8080` | Listen address for `serve`. |
|
| `--bind <addr:port>` | `ANWESEN_BIND` | `127.0.0.1:8080` | Listen address for `serve`. |
|
||||||
| `--log-level <level>` | `ANWESEN_LOG_LEVEL` | `info` | `error`, `warn`, `info`, `debug`, or `trace`. |
|
| `--log-level <level>` | `ANWESEN_LOG_LEVEL` | `info` | `error`, `warn`, `info`, `debug`, or `trace`. |
|
||||||
| `--query <query-string>` | -- | _required for `merge`_ | A `/query` query string: frontmatter predicates plus `__anw-` controls. |
|
| `--query <query-string>` | -- | _required for `merge`_ | A `/query` query string: frontmatter predicates plus `__anw-` controls. |
|
||||||
| `--uptrace-dsn <dsn>` | `ANWESEN_UPTRACE_DSN` | unset | uptrace DSN (`https://<token>@api.uptrace.dev`). Excludes `--otlp-endpoint`. |
|
|
||||||
| `--otlp-endpoint <url>` | `ANWESEN_OTLP_ENDPOINT` | unset | OTLP/HTTP base URL. Excludes `--uptrace-dsn`. |
|
|
||||||
| `--otlp-header <key=value>` | `ANWESEN_OTLP_HEADERS` | none | Extra export header, repeatable. The env var takes a comma-separated list. |
|
|
||||||
| `--otlp-slow-request-ms` | `ANWESEN_OTLP_SLOW_REQUEST_MS` | `500` | Requests at or over this duration, or answering 5xx, also export a span. |
|
| `--otlp-slow-request-ms` | `ANWESEN_OTLP_SLOW_REQUEST_MS` | `500` | Requests at or over this duration, or answering 5xx, also export a span. |
|
||||||
|
|
||||||
Every flag has a matching `ANWESEN_<UPPER>` environment variable, except
|
Every flag has a matching `ANWESEN_<UPPER>` environment variable. CLI flags win
|
||||||
`--otlp-header`, whose env var is the plural `ANWESEN_OTLP_HEADERS` because it
|
over env vars.
|
||||||
takes a list. CLI flags win over env vars.
|
|
||||||
|
|
||||||
Telemetry is off unless `--uptrace-dsn` or `--otlp-endpoint` is set. With
|
`--bind` and `--otlp-slow-request-ms` apply to `serve` only. Where telemetry is
|
||||||
neither, nothing is exported and no exporter is built. The four telemetry
|
exported is configured entirely through the standard `OTEL_` variables below.
|
||||||
flags apply to `serve` only.
|
|
||||||
|
|
||||||
- **`serve`** -- run the daemon: walk the vault, build the index, watch for changes, serve the API.
|
- **`serve`** -- run the daemon: walk the vault, build the index, watch for changes, serve the API.
|
||||||
- **`doctor`** -- walk the vault once and report what would stop clean ingestion: unreadable files, unparseable YAML, path collisions on the HTTP surface, and frontmatter type drift (the same key carrying incompatible types across notes). Read-only; non-zero exit if any issue is found.
|
- **`doctor`** -- walk the vault once and report what would stop clean ingestion: unreadable files, unparseable YAML, path collisions on the HTTP surface, and frontmatter type drift (the same key carrying incompatible types across notes). Read-only; non-zero exit if any issue is found.
|
||||||
- **`merge`** -- one-shot local generation: walk the vault, evaluate `--query`, and write the merged markdown document to stdout. No server, no HTTP. See [Local generation](#local-generation).
|
- **`merge`** -- one-shot local generation: walk the vault, evaluate `--query`, and write the merged markdown document to stdout. No server, no HTTP. See [Local generation](#local-generation).
|
||||||
- **`version`** -- print version and exit.
|
- **`version`** -- print version and exit.
|
||||||
|
|
||||||
|
## Telemetry
|
||||||
|
|
||||||
|
anwesen exports OTLP metrics for every request, and a span for requests at or
|
||||||
|
over `--otlp-slow-request-ms` or answering a 5xx. Export is configured through
|
||||||
|
the standard OpenTelemetry environment variables, which the SDK reads directly:
|
||||||
|
|
||||||
|
| Variable | Meaning |
|
||||||
|
| -------------------------------------- | ---------------------------------------------------------------- |
|
||||||
|
| `OTEL_EXPORTER_OTLP_ENDPOINT` | Base URL. The SDK appends `/v1/metrics` and `/v1/traces`. |
|
||||||
|
| `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT` | Full metrics URL, used as given. Overrides the base for metrics. |
|
||||||
|
| `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` | Full traces URL, used as given. Overrides the base for traces. |
|
||||||
|
| `OTEL_EXPORTER_OTLP_HEADERS` | Export headers, `key=value` comma-separated. |
|
||||||
|
| `OTEL_EXPORTER_OTLP_PROTOCOL` | `http/protobuf`, the default and the only value this build speaks. |
|
||||||
|
| `OTEL_SERVICE_NAME`, `OTEL_RESOURCE_ATTRIBUTES` | Override the `anwesen` service identity. |
|
||||||
|
|
||||||
|
The per-signal `OTEL_EXPORTER_OTLP_METRICS_PROTOCOL` and
|
||||||
|
`OTEL_EXPORTER_OTLP_TRACES_PROTOCOL` are read the same way.
|
||||||
|
|
||||||
|
With none of the three endpoint variables set, telemetry is off: no exporter is
|
||||||
|
built and the request middleware is not installed.
|
||||||
|
|
||||||
|
Two settings fail at startup rather than export nowhere in silence:
|
||||||
|
|
||||||
|
- A query or a fragment on `OTEL_EXPORTER_OTLP_ENDPOINT`. The base URL must be
|
||||||
|
a base URL; the SDK appends the signal path after whatever it is given.
|
||||||
|
- A protocol other than `http/protobuf`. The binary ships the OTLP/HTTP
|
||||||
|
transport alone, so `grpc` would keep exporting over HTTP with nothing in the
|
||||||
|
log. Point the endpoint at the collector's HTTP port, not its gRPC one.
|
||||||
|
|
||||||
|
uptrace:
|
||||||
|
|
||||||
|
```
|
||||||
|
OTEL_EXPORTER_OTLP_ENDPOINT=https://api.uptrace.dev
|
||||||
|
OTEL_EXPORTER_OTLP_HEADERS=uptrace-dsn=https://TOKEN@api.uptrace.dev?grpc=4317
|
||||||
|
```
|
||||||
|
|
||||||
|
Paste the DSN from Project Settings into the header verbatim, tail and all --
|
||||||
|
it is a credential there, not an address. The endpoint is the host alone.
|
||||||
|
|
||||||
|
Removed in 0.3.0: `--uptrace-dsn`, `--otlp-endpoint`, `--otlp-header` and their
|
||||||
|
`ANWESEN_` variables. Passing any of them fails at startup with the `OTEL_`
|
||||||
|
replacement to use.
|
||||||
|
|
||||||
## HTTP API
|
## HTTP API
|
||||||
|
|
||||||
All endpoints are `GET` and return JSON unless noted.
|
All endpoints are `GET` and return JSON unless noted.
|
||||||
|
|
|
||||||
63
src/cli.rs
63
src/cli.rs
|
|
@ -4,18 +4,22 @@
|
||||||
//!
|
//!
|
||||||
//! ```text
|
//! ```text
|
||||||
//! anwesen serve --vault <path> [--bind <addr:port>] [--log-level <level>]
|
//! anwesen serve --vault <path> [--bind <addr:port>] [--log-level <level>]
|
||||||
//! [--uptrace-dsn <dsn> | --otlp-endpoint <url>]
|
//! [--otlp-slow-request-ms <n>]
|
||||||
//! [--otlp-header <key=value>]... [--otlp-slow-request-ms <n>]
|
|
||||||
//! anwesen doctor --vault <path> [--log-level <level>]
|
//! anwesen doctor --vault <path> [--log-level <level>]
|
||||||
//! anwesen merge --vault <path> [--query <string>] [--log-level <level>]
|
//! anwesen merge --vault <path> [--query <string>] [--log-level <level>]
|
||||||
//! anwesen version
|
//! anwesen version
|
||||||
//! ```
|
//! ```
|
||||||
//!
|
//!
|
||||||
//! `--bind` and the OTLP telemetry flags are `serve`-only ([ANW-37]);
|
//! `--bind` and `--otlp-slow-request-ms` are `serve`-only ([ANW-37]);
|
||||||
//! `doctor` and `merge` do not bind a port; `version` takes no flags. Each
|
//! `doctor` and `merge` do not bind a port; `version` takes no flags. Each
|
||||||
//! flag has a matching `ANWESEN_<UPPER>` environment variable and CLI wins
|
//! flag has a matching `ANWESEN_<UPPER>` environment variable and CLI wins
|
||||||
//! over env per the manual. With no `--otlp-endpoint`/`--uptrace-dsn`,
|
//! over env per the manual.
|
||||||
//! telemetry is off and the server behaves exactly as without these flags.
|
//!
|
||||||
|
//! Telemetry export is configured through the standard `OTEL_EXPORTER_OTLP_*`
|
||||||
|
//! environment variables ([ANW-42](https://crvrs.youtrack.cloud/issue/ANW-42)).
|
||||||
|
//! With none of them set, telemetry is off and the server behaves as it did
|
||||||
|
//! before telemetry existed. The removed flags below are still parsed so an
|
||||||
|
//! upgrade fails loudly instead of dropping export config in silence.
|
||||||
|
|
||||||
use std::net::SocketAddr;
|
use std::net::SocketAddr;
|
||||||
use std::path::PathBuf;
|
use std::path::PathBuf;
|
||||||
|
|
@ -56,25 +60,22 @@ pub struct ServeArgs {
|
||||||
#[arg(long, env = "ANWESEN_LOG_LEVEL", default_value = "info")]
|
#[arg(long, env = "ANWESEN_LOG_LEVEL", default_value = "info")]
|
||||||
pub log_level: LogLevel,
|
pub log_level: LogLevel,
|
||||||
|
|
||||||
/// uptrace DSN shorthand (`https://<token>@api.uptrace.dev`), parsed
|
/// Removed (ANW-42): use `OTEL_EXPORTER_OTLP_ENDPOINT` plus
|
||||||
/// into the OTLP endpoint plus an `uptrace-dsn` header. Mutually
|
/// `OTEL_EXPORTER_OTLP_HEADERS=uptrace-dsn=<dsn>`. Still accepted so
|
||||||
/// exclusive with --otlp-endpoint. When this and --otlp-endpoint are
|
/// startup fails with that message rather than exporting nowhere.
|
||||||
/// both unset, telemetry is fully off (ANW-37).
|
#[arg(long, env = "ANWESEN_UPTRACE_DSN", hide = true)]
|
||||||
#[arg(long, env = "ANWESEN_UPTRACE_DSN")]
|
|
||||||
pub uptrace_dsn: Option<String>,
|
pub uptrace_dsn: Option<String>,
|
||||||
|
|
||||||
/// Generic OTLP/HTTP endpoint base URL for telemetry export. The
|
/// Removed (ANW-42): use `OTEL_EXPORTER_OTLP_ENDPOINT`.
|
||||||
/// per-signal path (`/v1/metrics`, `/v1/traces`) is appended by the
|
#[arg(long, env = "ANWESEN_OTLP_ENDPOINT", hide = true)]
|
||||||
/// exporter. Mutually exclusive with --uptrace-dsn.
|
|
||||||
#[arg(long, env = "ANWESEN_OTLP_ENDPOINT")]
|
|
||||||
pub otlp_endpoint: Option<String>,
|
pub otlp_endpoint: Option<String>,
|
||||||
|
|
||||||
/// Extra OTLP export header as `key=value`, repeatable. On the env var
|
/// Removed (ANW-42): use `OTEL_EXPORTER_OTLP_HEADERS`.
|
||||||
/// (`ANWESEN_OTLP_HEADERS`) pass a comma-separated `key=value` list.
|
|
||||||
#[arg(
|
#[arg(
|
||||||
long = "otlp-header",
|
long = "otlp-header",
|
||||||
env = "ANWESEN_OTLP_HEADERS",
|
env = "ANWESEN_OTLP_HEADERS",
|
||||||
value_delimiter = ','
|
value_delimiter = ',',
|
||||||
|
hide = true
|
||||||
)]
|
)]
|
||||||
pub otlp_headers: Vec<String>,
|
pub otlp_headers: Vec<String>,
|
||||||
|
|
||||||
|
|
@ -172,6 +173,34 @@ mod tests {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The removed telemetry flags still parse, hidden from `--help`. Clap
|
||||||
|
/// rejecting them as unknown would say nothing about the `OTEL_`
|
||||||
|
/// replacement; the migration error in `telemetry::TelemetryConfig`
|
||||||
|
/// needs the values to reach it (ANW-42).
|
||||||
|
#[test]
|
||||||
|
fn serve_still_parses_the_removed_telemetry_flags() {
|
||||||
|
let cli = parse(&[
|
||||||
|
"serve",
|
||||||
|
"--vault",
|
||||||
|
"/tmp/v",
|
||||||
|
"--uptrace-dsn",
|
||||||
|
"https://tok@api.uptrace.dev",
|
||||||
|
"--otlp-endpoint",
|
||||||
|
"https://collector.example.com",
|
||||||
|
"--otlp-header",
|
||||||
|
"authorization=Bearer xyz",
|
||||||
|
])
|
||||||
|
.expect("parse");
|
||||||
|
match cli.command {
|
||||||
|
Command::Serve(a) => {
|
||||||
|
assert!(a.uptrace_dsn.is_some());
|
||||||
|
assert!(a.otlp_endpoint.is_some());
|
||||||
|
assert_eq!(a.otlp_headers, vec!["authorization=Bearer xyz".to_string()]);
|
||||||
|
}
|
||||||
|
_ => panic!("expected serve"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn serve_rejects_malformed_bind_at_parse_time() {
|
fn serve_rejects_malformed_bind_at_parse_time() {
|
||||||
let err = parse(&["serve", "--vault", "/tmp/v", "--bind", "not-an-addr"]).unwrap_err();
|
let err = parse(&["serve", "--vault", "/tmp/v", "--bind", "not-an-addr"]).unwrap_err();
|
||||||
|
|
|
||||||
19
src/http.rs
19
src/http.rs
|
|
@ -819,18 +819,23 @@ mod tests {
|
||||||
}
|
}
|
||||||
|
|
||||||
/// With telemetry installed, a normal request is answered byte-for-byte
|
/// With telemetry installed, a normal request is answered byte-for-byte
|
||||||
/// as without it. The exporter points at an unreachable local port, so
|
/// as without it. The test process sets no `OTEL_EXPORTER_OTLP_*`
|
||||||
/// export fails instantly in the background and never touches the
|
/// variables, so the exporter aims at the SDK default and fails in the
|
||||||
/// response path.
|
/// background without ever touching the response path.
|
||||||
#[tokio::test]
|
#[tokio::test]
|
||||||
async fn telemetry_layer_does_not_alter_responses() {
|
async fn telemetry_layer_does_not_alter_responses() {
|
||||||
use crate::telemetry::{self, RawTelemetryArgs, TelemetryConfig};
|
use crate::telemetry::{self, OtelEnv, RawTelemetryArgs, TelemetryConfig};
|
||||||
|
|
||||||
let cfg = TelemetryConfig::resolve(RawTelemetryArgs {
|
let cfg = TelemetryConfig::resolve(
|
||||||
otlp_endpoint: Some("http://127.0.0.1:9".into()),
|
&RawTelemetryArgs {
|
||||||
slow_request_ms: 500,
|
slow_request_ms: 500,
|
||||||
..Default::default()
|
..Default::default()
|
||||||
})
|
},
|
||||||
|
OtelEnv {
|
||||||
|
endpoint: Some("http://127.0.0.1:9".into()),
|
||||||
|
..OtelEnv::default()
|
||||||
|
},
|
||||||
|
)
|
||||||
.unwrap()
|
.unwrap()
|
||||||
.expect("telemetry on");
|
.expect("telemetry on");
|
||||||
let tel = Arc::new(telemetry::init(cfg).expect("telemetry init"));
|
let tel = Arc::new(telemetry::init(cfg).expect("telemetry init"));
|
||||||
|
|
|
||||||
14
src/main.rs
14
src/main.rs
|
|
@ -10,7 +10,7 @@ use std::sync::Arc;
|
||||||
use anwesen::app::Anwesen;
|
use anwesen::app::Anwesen;
|
||||||
use anwesen::doctor;
|
use anwesen::doctor;
|
||||||
use anwesen::merge;
|
use anwesen::merge;
|
||||||
use anwesen::telemetry::{self, RawTelemetryArgs, TelemetryConfig};
|
use anwesen::telemetry::{self, OtelEnv, RawTelemetryArgs, TelemetryConfig};
|
||||||
use anyhow::Result;
|
use anyhow::Result;
|
||||||
use clap::Parser;
|
use clap::Parser;
|
||||||
use hydra::Application;
|
use hydra::Application;
|
||||||
|
|
@ -25,14 +25,18 @@ fn main() -> Result<()> {
|
||||||
match cli.command {
|
match cli.command {
|
||||||
Command::Serve(args) => {
|
Command::Serve(args) => {
|
||||||
init_logging(args.log_level);
|
init_logging(args.log_level);
|
||||||
// Resolve telemetry config before the supervisor starts; an
|
// Resolve telemetry config before the supervisor starts; no
|
||||||
// unset endpoint leaves it `None` (export off, behaves as today).
|
// OTEL_EXPORTER_OTLP_* endpoint leaves it `None` (export off,
|
||||||
let telemetry = match TelemetryConfig::resolve(RawTelemetryArgs {
|
// no middleware). A removed flag is a startup error (ANW-42).
|
||||||
|
let telemetry = match TelemetryConfig::resolve(
|
||||||
|
&RawTelemetryArgs {
|
||||||
uptrace_dsn: args.uptrace_dsn,
|
uptrace_dsn: args.uptrace_dsn,
|
||||||
otlp_endpoint: args.otlp_endpoint,
|
otlp_endpoint: args.otlp_endpoint,
|
||||||
otlp_headers: args.otlp_headers,
|
otlp_headers: args.otlp_headers,
|
||||||
slow_request_ms: args.otlp_slow_request_ms,
|
slow_request_ms: args.otlp_slow_request_ms,
|
||||||
})? {
|
},
|
||||||
|
OtelEnv::from_env(),
|
||||||
|
)? {
|
||||||
Some(cfg) => Some(Arc::new(telemetry::init(cfg)?)),
|
Some(cfg) => Some(Arc::new(telemetry::init(cfg)?)),
|
||||||
None => None,
|
None => None,
|
||||||
};
|
};
|
||||||
|
|
|
||||||
587
src/telemetry.rs
587
src/telemetry.rs
|
|
@ -12,140 +12,288 @@
|
||||||
//! the propagated context. Every other request stays metrics-only, so the
|
//! the propagated context. Every other request stays metrics-only, so the
|
||||||
//! ~3.6k requests/min steady state does not drown the trace backend.
|
//! ~3.6k requests/min steady state does not drown the trace backend.
|
||||||
//!
|
//!
|
||||||
//! When no OTLP endpoint (or uptrace DSN) is configured, [`init`] returns
|
//! Transport configuration comes from the standard `OTEL_EXPORTER_OTLP_*`
|
||||||
|
//! environment variables only, read by the `OpenTelemetry` SDK itself
|
||||||
|
//! ([ANW-42](https://crvrs.youtrack.cloud/issue/ANW-42)). anwesen parses no
|
||||||
|
//! addresses and appends no per-signal paths. It checks two things at
|
||||||
|
//! startup, both cases the SDK would otherwise export nowhere in silence:
|
||||||
|
//!
|
||||||
|
//! - `OTEL_EXPORTER_OTLP_ENDPOINT` carries no query or fragment, because the
|
||||||
|
//! SDK's own concatenation mangles those (measured against a sink: a
|
||||||
|
//! `?tail` base sends `POST /?tail/v1/metrics`, a `#frag` base sends
|
||||||
|
//! `POST /`).
|
||||||
|
//! - No protocol variable asks for anything but `http/protobuf`, the only
|
||||||
|
//! transport this binary is built with.
|
||||||
|
//!
|
||||||
|
//! When no endpoint variable is set, [`TelemetryConfig::resolve`] returns
|
||||||
//! `None`, the request middleware is not installed, and the server behaves
|
//! `None`, the request middleware is not installed, and the server behaves
|
||||||
//! exactly as it did before this module existed. External installs run
|
//! exactly as it did before this module existed. External installs run
|
||||||
//! unchanged.
|
//! unchanged.
|
||||||
//!
|
|
||||||
//! Config mirrors the gestell uptrace surface (see the gestell PDR-GES-136
|
|
||||||
//! `[otel]` section): a `uptrace_dsn` shorthand, or a generic endpoint plus
|
|
||||||
//! headers.
|
|
||||||
|
|
||||||
use std::collections::HashMap;
|
|
||||||
use std::time::{Duration, SystemTime};
|
use std::time::{Duration, SystemTime};
|
||||||
|
|
||||||
use anyhow::{Context as _, anyhow, bail};
|
use anyhow::{Context as _, bail};
|
||||||
use axum::http::HeaderMap;
|
use axum::http::HeaderMap;
|
||||||
use opentelemetry::KeyValue;
|
use opentelemetry::KeyValue;
|
||||||
use opentelemetry::metrics::{Counter, Histogram, MeterProvider as _};
|
use opentelemetry::metrics::{Counter, Histogram, MeterProvider as _};
|
||||||
use opentelemetry::propagation::{Extractor, TextMapPropagator};
|
use opentelemetry::propagation::{Extractor, TextMapPropagator};
|
||||||
use opentelemetry::trace::{Span, SpanKind, Tracer, TracerProvider as _};
|
use opentelemetry::trace::{Span, SpanKind, Tracer, TracerProvider as _};
|
||||||
use opentelemetry_otlp::{
|
use opentelemetry_otlp::{MetricExporter, SpanExporter};
|
||||||
MetricExporter, Protocol, SpanExporter, WithExportConfig, WithHttpConfig,
|
|
||||||
};
|
|
||||||
use opentelemetry_sdk::Resource;
|
use opentelemetry_sdk::Resource;
|
||||||
use opentelemetry_sdk::metrics::{PeriodicReader, SdkMeterProvider};
|
use opentelemetry_sdk::metrics::{PeriodicReader, SdkMeterProvider};
|
||||||
use opentelemetry_sdk::propagation::TraceContextPropagator;
|
use opentelemetry_sdk::propagation::TraceContextPropagator;
|
||||||
use opentelemetry_sdk::trace::Sampler;
|
use opentelemetry_sdk::trace::Sampler;
|
||||||
use opentelemetry_sdk::trace::{SdkTracer, SdkTracerProvider};
|
use opentelemetry_sdk::trace::{SdkTracer, SdkTracerProvider};
|
||||||
use opentelemetry_semantic_conventions::resource::SERVICE_VERSION;
|
use opentelemetry_semantic_conventions::resource::{SERVICE_NAME, SERVICE_VERSION};
|
||||||
|
|
||||||
/// Raw telemetry options as parsed by clap on the `serve` command. Resolved
|
/// The removed telemetry options, still parsed so their presence is an error
|
||||||
/// into an [`Option<TelemetryConfig>`] by [`TelemetryConfig::resolve`].
|
/// rather than a silent config drop on upgrade
|
||||||
|
/// ([ANW-42](https://crvrs.youtrack.cloud/issue/ANW-42)), plus the one
|
||||||
|
/// surviving option. Resolved into an [`Option<TelemetryConfig>`] by
|
||||||
|
/// [`TelemetryConfig::resolve`].
|
||||||
#[derive(Debug, Default)]
|
#[derive(Debug, Default)]
|
||||||
pub struct RawTelemetryArgs {
|
pub struct RawTelemetryArgs {
|
||||||
/// `--uptrace-dsn` / `ANWESEN_UPTRACE_DSN`.
|
/// Removed `--uptrace-dsn` / `ANWESEN_UPTRACE_DSN`.
|
||||||
pub uptrace_dsn: Option<String>,
|
pub uptrace_dsn: Option<String>,
|
||||||
/// `--otlp-endpoint` / `ANWESEN_OTLP_ENDPOINT`.
|
/// Removed `--otlp-endpoint` / `ANWESEN_OTLP_ENDPOINT`.
|
||||||
pub otlp_endpoint: Option<String>,
|
pub otlp_endpoint: Option<String>,
|
||||||
/// `--otlp-header` / `ANWESEN_OTLP_HEADERS`, each `key=value`.
|
/// Removed `--otlp-header` / `ANWESEN_OTLP_HEADERS`.
|
||||||
pub otlp_headers: Vec<String>,
|
pub otlp_headers: Vec<String>,
|
||||||
/// `--otlp-slow-request-ms` / `ANWESEN_OTLP_SLOW_REQUEST_MS`.
|
/// `--otlp-slow-request-ms` / `ANWESEN_OTLP_SLOW_REQUEST_MS`. Kept: it
|
||||||
|
/// decides when anwesen emits a span, not where the export goes, and no
|
||||||
|
/// standard variable covers it.
|
||||||
pub slow_request_ms: u64,
|
pub slow_request_ms: u64,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// A resolved, telemetry-on configuration. Built only when an endpoint or a
|
/// The OTLP transport variables the SDK reads, captured once at startup so
|
||||||
/// DSN is present; absence is represented by `Ok(None)` from [`resolve`].
|
/// the telemetry-on decision and the startup checks stay pure functions of
|
||||||
|
/// them. Values are the raw strings; anwesen does not parse them beyond the
|
||||||
|
/// checks in [`check_generic_endpoint`] and [`check_protocols`].
|
||||||
|
#[derive(Debug, Default, Clone, PartialEq, Eq)]
|
||||||
|
pub struct OtelEnv {
|
||||||
|
/// `OTEL_EXPORTER_OTLP_ENDPOINT`: base URL, per-signal path appended by
|
||||||
|
/// the SDK.
|
||||||
|
pub endpoint: Option<String>,
|
||||||
|
/// `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT`: full URL, used verbatim.
|
||||||
|
pub metrics_endpoint: Option<String>,
|
||||||
|
/// `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`: full URL, used verbatim.
|
||||||
|
pub traces_endpoint: Option<String>,
|
||||||
|
/// `OTEL_EXPORTER_OTLP_PROTOCOL`.
|
||||||
|
pub protocol: Option<String>,
|
||||||
|
/// `OTEL_EXPORTER_OTLP_METRICS_PROTOCOL`.
|
||||||
|
pub metrics_protocol: Option<String>,
|
||||||
|
/// `OTEL_EXPORTER_OTLP_TRACES_PROTOCOL`.
|
||||||
|
pub traces_protocol: Option<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl OtelEnv {
|
||||||
|
/// Read the transport variables from the process environment. An empty or
|
||||||
|
/// whitespace-only value counts as unset: an empty endpoint would
|
||||||
|
/// otherwise turn telemetry on and export to the SDK's `localhost:4318`
|
||||||
|
/// default.
|
||||||
|
#[must_use]
|
||||||
|
pub fn from_env() -> Self {
|
||||||
|
let var = |name: &str| {
|
||||||
|
std::env::var(name)
|
||||||
|
.ok()
|
||||||
|
.map(|v| v.trim().to_string())
|
||||||
|
.filter(|v| !v.is_empty())
|
||||||
|
};
|
||||||
|
Self {
|
||||||
|
endpoint: var("OTEL_EXPORTER_OTLP_ENDPOINT"),
|
||||||
|
metrics_endpoint: var("OTEL_EXPORTER_OTLP_METRICS_ENDPOINT"),
|
||||||
|
traces_endpoint: var("OTEL_EXPORTER_OTLP_TRACES_ENDPOINT"),
|
||||||
|
protocol: var("OTEL_EXPORTER_OTLP_PROTOCOL"),
|
||||||
|
metrics_protocol: var("OTEL_EXPORTER_OTLP_METRICS_PROTOCOL"),
|
||||||
|
traces_protocol: var("OTEL_EXPORTER_OTLP_TRACES_PROTOCOL"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether any endpoint variable is set. No endpoint means telemetry off.
|
||||||
|
fn any_endpoint(&self) -> bool {
|
||||||
|
self.endpoint.is_some() || self.metrics_endpoint.is_some() || self.traces_endpoint.is_some()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The endpoint to name in the startup log: the generic base when set,
|
||||||
|
/// otherwise whichever per-signal URL is.
|
||||||
|
fn describe(&self) -> &str {
|
||||||
|
self.endpoint
|
||||||
|
.as_deref()
|
||||||
|
.or(self.metrics_endpoint.as_deref())
|
||||||
|
.or(self.traces_endpoint.as_deref())
|
||||||
|
.unwrap_or("")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A resolved, telemetry-on configuration. Built only when an endpoint
|
||||||
|
/// variable is set; absence is represented by `Ok(None)` from [`resolve`].
|
||||||
|
/// It carries no transport settings: the exporters read those from the
|
||||||
|
/// environment themselves.
|
||||||
///
|
///
|
||||||
/// [`resolve`]: TelemetryConfig::resolve
|
/// [`resolve`]: TelemetryConfig::resolve
|
||||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
pub struct TelemetryConfig {
|
pub struct TelemetryConfig {
|
||||||
/// OTLP/HTTP base URL. The exporter appends the per-signal path
|
|
||||||
/// (`/v1/metrics`, `/v1/traces`).
|
|
||||||
pub endpoint: String,
|
|
||||||
/// Export headers (for uptrace, the `uptrace-dsn` entry) as ordered
|
|
||||||
/// `(name, value)` pairs.
|
|
||||||
pub headers: Vec<(String, String)>,
|
|
||||||
/// A request at or over this duration, or answering a 5xx, is recorded
|
/// A request at or over this duration, or answering a 5xx, is recorded
|
||||||
/// as a server span.
|
/// as a server span.
|
||||||
pub slow_request: Duration,
|
pub slow_request: Duration,
|
||||||
|
/// The transport variables, kept for the startup log line only.
|
||||||
|
env: OtelEnv,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl TelemetryConfig {
|
impl TelemetryConfig {
|
||||||
/// Resolve raw clap options into an optional config.
|
/// Resolve the surviving option plus the OTLP transport variables into an
|
||||||
|
/// optional config.
|
||||||
///
|
///
|
||||||
/// - Both `uptrace_dsn` and `otlp_endpoint` set is an error (they are
|
/// - A removed flag or `ANWESEN_` variable is an error naming its `OTEL_`
|
||||||
/// two ways to name the same endpoint).
|
/// replacement: a deployment exporting today must fail one restart
|
||||||
/// - Neither set means telemetry is off: `Ok(None)`.
|
/// rather than go quiet.
|
||||||
/// - A malformed `key=value` header or an unparseable DSN is an error,
|
/// - No endpoint variable set means telemetry is off: `Ok(None)`.
|
||||||
/// surfaced at startup rather than silently dropping export.
|
/// - A query or fragment on `OTEL_EXPORTER_OTLP_ENDPOINT` is an error.
|
||||||
|
/// - A protocol this binary cannot speak is an error.
|
||||||
///
|
///
|
||||||
/// # Errors
|
/// # Errors
|
||||||
/// Returns an error when the two endpoint sources conflict, a header is
|
/// Returns an error when a removed option is present, when the generic
|
||||||
/// not `key=value`, or the uptrace DSN cannot be parsed.
|
/// endpoint carries a query or a fragment, or when a protocol variable
|
||||||
pub fn resolve(raw: RawTelemetryArgs) -> anyhow::Result<Option<Self>> {
|
/// asks for anything but `http/protobuf`.
|
||||||
let slow_request = Duration::from_millis(raw.slow_request_ms);
|
pub fn resolve(raw: &RawTelemetryArgs, env: OtelEnv) -> anyhow::Result<Option<Self>> {
|
||||||
let mut headers = parse_headers(&raw.otlp_headers)?;
|
check_removed(raw)?;
|
||||||
|
if !env.any_endpoint() {
|
||||||
match (raw.uptrace_dsn, raw.otlp_endpoint) {
|
return Ok(None);
|
||||||
(Some(_), Some(_)) => {
|
|
||||||
bail!("--uptrace-dsn and --otlp-endpoint are mutually exclusive");
|
|
||||||
}
|
}
|
||||||
(Some(dsn), None) => {
|
if let Some(endpoint) = &env.endpoint {
|
||||||
let (endpoint, dsn_header) = parse_uptrace_dsn(&dsn)?;
|
check_generic_endpoint(endpoint)?;
|
||||||
// The DSN header leads; any explicit --otlp-header follows.
|
}
|
||||||
headers.insert(0, dsn_header);
|
check_protocols(&env)?;
|
||||||
Ok(Some(Self {
|
Ok(Some(Self {
|
||||||
endpoint,
|
slow_request: Duration::from_millis(raw.slow_request_ms),
|
||||||
headers,
|
env,
|
||||||
slow_request,
|
|
||||||
}))
|
}))
|
||||||
}
|
}
|
||||||
(None, Some(endpoint)) => Ok(Some(Self {
|
|
||||||
endpoint,
|
|
||||||
headers,
|
|
||||||
slow_request,
|
|
||||||
})),
|
|
||||||
(None, None) => Ok(None),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Parse `key=value` header specs. Whitespace around key and value is
|
/// Fail on any removed telemetry option, naming the `OTEL_` variable that
|
||||||
/// trimmed; an empty key or a spec with no `=` is an error.
|
/// replaces it. Silence is the expensive failure here: an upgrade that drops
|
||||||
fn parse_headers(specs: &[String]) -> anyhow::Result<Vec<(String, String)>> {
|
/// the export config would stop telemetry with nothing in the log.
|
||||||
let mut out = Vec::with_capacity(specs.len());
|
fn check_removed(raw: &RawTelemetryArgs) -> anyhow::Result<()> {
|
||||||
for spec in specs {
|
let removed: [(&str, &str, bool); 3] = [
|
||||||
let (k, v) = spec
|
(
|
||||||
.split_once('=')
|
"--uptrace-dsn / ANWESEN_UPTRACE_DSN",
|
||||||
.ok_or_else(|| anyhow!("OTLP header {spec:?} is not key=value"))?;
|
"OTEL_EXPORTER_OTLP_ENDPOINT=https://api.uptrace.dev plus \
|
||||||
let k = k.trim();
|
OTEL_EXPORTER_OTLP_HEADERS=uptrace-dsn=<the DSN, verbatim>",
|
||||||
if k.is_empty() {
|
raw.uptrace_dsn.is_some(),
|
||||||
bail!("OTLP header {spec:?} has an empty key");
|
),
|
||||||
|
(
|
||||||
|
"--otlp-endpoint / ANWESEN_OTLP_ENDPOINT",
|
||||||
|
"OTEL_EXPORTER_OTLP_ENDPOINT",
|
||||||
|
raw.otlp_endpoint.is_some(),
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"--otlp-header / ANWESEN_OTLP_HEADERS",
|
||||||
|
"OTEL_EXPORTER_OTLP_HEADERS",
|
||||||
|
!raw.otlp_headers.is_empty(),
|
||||||
|
),
|
||||||
|
];
|
||||||
|
for (option, replacement, present) in removed {
|
||||||
|
if present {
|
||||||
|
bail!("{option} was removed in anwesen 0.3.0; use {replacement} instead");
|
||||||
}
|
}
|
||||||
out.push((k.to_string(), v.trim().to_string()));
|
|
||||||
}
|
}
|
||||||
Ok(out)
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Parse an uptrace DSN (`https://<token>@host[:port]`) into the OTLP
|
/// Reject a query or fragment on `OTEL_EXPORTER_OTLP_ENDPOINT`. The SDK
|
||||||
/// endpoint base URL and the `uptrace-dsn` header uptrace expects. The full
|
/// appends the per-signal path to this value textually, so a `?grpc=4317`
|
||||||
/// DSN is echoed as the header value per uptrace's ingest contract.
|
/// tail sends `POST /?grpc=4317/v1/metrics` and a `#frag` tail sends
|
||||||
fn parse_uptrace_dsn(dsn: &str) -> anyhow::Result<(String, (String, String))> {
|
/// `POST /` -- both dead exports, neither logged. Measured against a sink on
|
||||||
let dsn = dsn.trim();
|
/// opentelemetry-otlp 0.32 ([ANW-42]). A path prefix composes correctly and
|
||||||
let (scheme, rest) = dsn
|
/// is left alone.
|
||||||
.split_once("://")
|
///
|
||||||
.context("uptrace DSN has no scheme (expected https://<token>@host)")?;
|
/// [ANW-42]: https://crvrs.youtrack.cloud/issue/ANW-42
|
||||||
// Host is whatever follows the credentials `@`; a DSN with no `@` is
|
fn check_generic_endpoint(endpoint: &str) -> anyhow::Result<()> {
|
||||||
// treated as endpoint-only (lenient, though real uptrace DSNs carry a
|
if let Some(bad) = endpoint.find(['?', '#']) {
|
||||||
// token).
|
let tail = &endpoint[bad..];
|
||||||
let host = rest.rsplit_once('@').map_or(rest, |(_, h)| h);
|
bail!(
|
||||||
let host = host.trim_end_matches('/');
|
"OTEL_EXPORTER_OTLP_ENDPOINT {endpoint:?} has a trailing {tail:?}; \
|
||||||
if host.is_empty() {
|
the exporter would append the signal path after it and export nowhere. \
|
||||||
bail!("uptrace DSN has no host");
|
Pass the base URL alone, and put an uptrace DSN in \
|
||||||
|
OTEL_EXPORTER_OTLP_HEADERS=uptrace-dsn=<the DSN, verbatim>"
|
||||||
|
);
|
||||||
}
|
}
|
||||||
let endpoint = format!("{scheme}://{host}");
|
Ok(())
|
||||||
Ok((endpoint, ("uptrace-dsn".to_string(), dsn.to_string())))
|
}
|
||||||
|
|
||||||
|
/// The one OTLP protocol this binary speaks. `opentelemetry-otlp` is built
|
||||||
|
/// with the `http-proto` feature alone (Cargo.toml), so neither `grpc` nor
|
||||||
|
/// `http/json` has a transport behind it.
|
||||||
|
const SUPPORTED_PROTOCOL: &str = "http/protobuf";
|
||||||
|
|
||||||
|
/// Reject a protocol variable this binary cannot honor. The exporter builder
|
||||||
|
/// picks its transport at compile time, so `OTEL_EXPORTER_OTLP_PROTOCOL=grpc`
|
||||||
|
/// does not switch anything: the export keeps going out as HTTP protobuf,
|
||||||
|
/// with nothing in the log (measured, [ANW-42]). An operator who follows
|
||||||
|
/// uptrace's console to the gRPC port would get exactly the dead-silent
|
||||||
|
/// export this issue exists to remove, so it fails at startup instead.
|
||||||
|
///
|
||||||
|
/// [ANW-42]: https://crvrs.youtrack.cloud/issue/ANW-42
|
||||||
|
fn check_protocols(env: &OtelEnv) -> anyhow::Result<()> {
|
||||||
|
let vars = [
|
||||||
|
("OTEL_EXPORTER_OTLP_PROTOCOL", env.protocol.as_deref()),
|
||||||
|
(
|
||||||
|
"OTEL_EXPORTER_OTLP_METRICS_PROTOCOL",
|
||||||
|
env.metrics_protocol.as_deref(),
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"OTEL_EXPORTER_OTLP_TRACES_PROTOCOL",
|
||||||
|
env.traces_protocol.as_deref(),
|
||||||
|
),
|
||||||
|
];
|
||||||
|
for (name, value) in vars {
|
||||||
|
let Some(value) = value else { continue };
|
||||||
|
if value != SUPPORTED_PROTOCOL {
|
||||||
|
bail!(
|
||||||
|
"{name}={value:?} is not supported; this build speaks \
|
||||||
|
{SUPPORTED_PROTOCOL} only. Unset the variable, and point the \
|
||||||
|
endpoint at the collector's HTTP port rather than its gRPC one"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The resource attributes anwesen supplies as defaults, minus every key the
|
||||||
|
/// environment already sets.
|
||||||
|
///
|
||||||
|
/// `service.name=anwesen` and the crate version are defaults, not policy:
|
||||||
|
/// contract point 6 of [ANW-42] keeps `OTEL_SERVICE_NAME` and
|
||||||
|
/// `OTEL_RESOURCE_ATTRIBUTES` working. Attaching them unconditionally blocks
|
||||||
|
/// the override, because an attribute set on the builder wins over the one
|
||||||
|
/// the SDK's own detectors read from those variables (measured by sie).
|
||||||
|
///
|
||||||
|
/// Only key presence matters here; the values stay the SDK's to parse.
|
||||||
|
///
|
||||||
|
/// [ANW-42]: https://crvrs.youtrack.cloud/issue/ANW-42
|
||||||
|
fn default_resource_attrs(
|
||||||
|
service_name: Option<&str>,
|
||||||
|
resource_attributes: Option<&str>,
|
||||||
|
) -> Vec<KeyValue> {
|
||||||
|
let from_attrs: Vec<&str> = resource_attributes
|
||||||
|
.unwrap_or_default()
|
||||||
|
.split(',')
|
||||||
|
.filter_map(|entry| entry.split_once('='))
|
||||||
|
.map(|(key, _)| key.trim())
|
||||||
|
.collect();
|
||||||
|
let set_by_env = |key: &str| {
|
||||||
|
(key == SERVICE_NAME && service_name.is_some_and(|v| !v.trim().is_empty()))
|
||||||
|
|| from_attrs.contains(&key)
|
||||||
|
};
|
||||||
|
[
|
||||||
|
(SERVICE_NAME, "anwesen"),
|
||||||
|
(SERVICE_VERSION, env!("CARGO_PKG_VERSION")),
|
||||||
|
]
|
||||||
|
.into_iter()
|
||||||
|
.filter(|(key, _)| !set_by_env(key))
|
||||||
|
.map(|(key, value)| KeyValue::new(key, value))
|
||||||
|
.collect()
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Semantic route bucket for the `http.route` label. Coarser than the axum
|
/// Semantic route bucket for the `http.route` label. Coarser than the axum
|
||||||
|
|
@ -332,28 +480,29 @@ struct TelemetryInner {
|
||||||
|
|
||||||
impl TelemetryInner {
|
impl TelemetryInner {
|
||||||
fn new(config: TelemetryConfig) -> anyhow::Result<Self> {
|
fn new(config: TelemetryConfig) -> anyhow::Result<Self> {
|
||||||
let TelemetryConfig {
|
let TelemetryConfig { slow_request, env } = config;
|
||||||
endpoint,
|
|
||||||
headers,
|
|
||||||
slow_request,
|
|
||||||
} = config;
|
|
||||||
// `.with_endpoint()` is used verbatim by the exporter (the `/v1/*`
|
|
||||||
// suffix is only auto-appended for the generic OTEL env var), so we
|
|
||||||
// append the per-signal path ourselves.
|
|
||||||
let endpoint = endpoint.trim_end_matches('/').to_string();
|
|
||||||
let header_map: HashMap<String, String> = headers.into_iter().collect();
|
|
||||||
|
|
||||||
let resource = Resource::builder()
|
// Service identity is a default only: a key OTEL_SERVICE_NAME or
|
||||||
.with_service_name("anwesen")
|
// OTEL_RESOURCE_ATTRIBUTES already carries is left to the SDK's own
|
||||||
.with_attribute(KeyValue::new(SERVICE_VERSION, env!("CARGO_PKG_VERSION")))
|
// detectors, because a builder attribute would win over them.
|
||||||
.build();
|
let mut resource = Resource::builder();
|
||||||
|
for attr in default_resource_attrs(
|
||||||
|
std::env::var("OTEL_SERVICE_NAME").ok().as_deref(),
|
||||||
|
std::env::var("OTEL_RESOURCE_ATTRIBUTES").ok().as_deref(),
|
||||||
|
) {
|
||||||
|
resource = resource.with_attribute(attr);
|
||||||
|
}
|
||||||
|
let resource = resource.build();
|
||||||
|
|
||||||
// -- metrics: thread-based periodic reader over an OTLP/HTTP exporter.
|
// -- metrics: thread-based periodic reader over an OTLP/HTTP exporter.
|
||||||
|
// No endpoint, headers, or protocol here: the SDK reads
|
||||||
|
// OTEL_EXPORTER_OTLP_* itself and composes the per-signal URL
|
||||||
|
// (ANW-42). `.with_http()` picks the only transport this binary is
|
||||||
|
// built with, http/protobuf -- the default, and what anwesen sent
|
||||||
|
// before. Any other OTEL_EXPORTER_OTLP_PROTOCOL value would be
|
||||||
|
// ignored here, so `check_protocols` rejects it at startup.
|
||||||
let metric_exporter = MetricExporter::builder()
|
let metric_exporter = MetricExporter::builder()
|
||||||
.with_http()
|
.with_http()
|
||||||
.with_protocol(Protocol::HttpBinary)
|
|
||||||
.with_endpoint(format!("{endpoint}/v1/metrics"))
|
|
||||||
.with_headers(header_map.clone())
|
|
||||||
.build()
|
.build()
|
||||||
.context("build OTLP metric exporter")?;
|
.context("build OTLP metric exporter")?;
|
||||||
let reader = PeriodicReader::builder(metric_exporter)
|
let reader = PeriodicReader::builder(metric_exporter)
|
||||||
|
|
@ -386,9 +535,6 @@ impl TelemetryInner {
|
||||||
// -- traces: thread-based batch processor over an OTLP/HTTP exporter.
|
// -- traces: thread-based batch processor over an OTLP/HTTP exporter.
|
||||||
let span_exporter = SpanExporter::builder()
|
let span_exporter = SpanExporter::builder()
|
||||||
.with_http()
|
.with_http()
|
||||||
.with_protocol(Protocol::HttpBinary)
|
|
||||||
.with_endpoint(format!("{endpoint}/v1/traces"))
|
|
||||||
.with_headers(header_map)
|
|
||||||
.build()
|
.build()
|
||||||
.context("build OTLP span exporter")?;
|
.context("build OTLP span exporter")?;
|
||||||
let tracer_provider = SdkTracerProvider::builder()
|
let tracer_provider = SdkTracerProvider::builder()
|
||||||
|
|
@ -401,7 +547,7 @@ impl TelemetryInner {
|
||||||
let tracer = tracer_provider.tracer("anwesen");
|
let tracer = tracer_provider.tracer("anwesen");
|
||||||
|
|
||||||
tracing::info!(
|
tracing::info!(
|
||||||
endpoint = %endpoint,
|
endpoint = %env.describe(),
|
||||||
slow_request_ms = slow_request.as_millis(),
|
slow_request_ms = slow_request.as_millis(),
|
||||||
"telemetry: OTLP export enabled"
|
"telemetry: OTLP export enabled"
|
||||||
);
|
);
|
||||||
|
|
@ -489,93 +635,208 @@ mod tests {
|
||||||
RawTelemetryArgs::default()
|
RawTelemetryArgs::default()
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
fn generic(endpoint: &str) -> OtelEnv {
|
||||||
fn resolve_off_when_nothing_set() {
|
OtelEnv {
|
||||||
let cfg = TelemetryConfig::resolve(raw()).unwrap();
|
endpoint: Some(endpoint.into()),
|
||||||
assert!(cfg.is_none(), "no endpoint/DSN means telemetry off");
|
..OtelEnv::default()
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn resolve_generic_endpoint() {
|
fn resolve_off_when_no_endpoint_variable_is_set() {
|
||||||
let cfg = TelemetryConfig::resolve(RawTelemetryArgs {
|
let cfg = TelemetryConfig::resolve(&raw(), OtelEnv::default()).unwrap();
|
||||||
otlp_endpoint: Some("https://collector.example.com".into()),
|
assert!(cfg.is_none(), "no OTEL_ endpoint means telemetry off");
|
||||||
otlp_headers: vec!["authorization=Bearer xyz".into()],
|
|
||||||
slow_request_ms: 500,
|
|
||||||
..raw()
|
|
||||||
})
|
|
||||||
.unwrap()
|
|
||||||
.expect("telemetry on");
|
|
||||||
assert_eq!(cfg.endpoint, "https://collector.example.com");
|
|
||||||
assert_eq!(
|
|
||||||
cfg.headers,
|
|
||||||
vec![("authorization".to_string(), "Bearer xyz".to_string())]
|
|
||||||
);
|
|
||||||
assert_eq!(cfg.slow_request, Duration::from_millis(500));
|
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn resolve_uptrace_dsn_splits_endpoint_and_header() {
|
fn resolve_on_for_each_endpoint_variable() {
|
||||||
let cfg = TelemetryConfig::resolve(RawTelemetryArgs {
|
let per_signal = |field: fn(&mut OtelEnv)| {
|
||||||
uptrace_dsn: Some("https://SECRET_TOKEN@api.uptrace.dev".into()),
|
let mut env = OtelEnv::default();
|
||||||
|
field(&mut env);
|
||||||
|
env
|
||||||
|
};
|
||||||
|
for env in [
|
||||||
|
generic("https://collector.example.com"),
|
||||||
|
per_signal(|e| {
|
||||||
|
e.metrics_endpoint = Some("https://collector.example.com/v1/metrics".into());
|
||||||
|
}),
|
||||||
|
per_signal(|e| {
|
||||||
|
e.traces_endpoint = Some("https://collector.example.com/v1/traces".into());
|
||||||
|
}),
|
||||||
|
] {
|
||||||
|
let cfg = TelemetryConfig::resolve(
|
||||||
|
&RawTelemetryArgs {
|
||||||
slow_request_ms: 250,
|
slow_request_ms: 250,
|
||||||
..raw()
|
..raw()
|
||||||
})
|
},
|
||||||
.unwrap()
|
env.clone(),
|
||||||
|
)
|
||||||
|
.unwrap_or_else(|e| panic!("{env:?}: {e}"))
|
||||||
.expect("telemetry on");
|
.expect("telemetry on");
|
||||||
assert_eq!(cfg.endpoint, "https://api.uptrace.dev");
|
|
||||||
assert_eq!(
|
|
||||||
cfg.headers,
|
|
||||||
vec![(
|
|
||||||
"uptrace-dsn".to_string(),
|
|
||||||
"https://SECRET_TOKEN@api.uptrace.dev".to_string()
|
|
||||||
)]
|
|
||||||
);
|
|
||||||
assert_eq!(cfg.slow_request, Duration::from_millis(250));
|
assert_eq!(cfg.slow_request, Duration::from_millis(250));
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The tails the SDK mangles: a query lands in the query string with the
|
||||||
|
/// signal path behind it, a fragment drops the signal path entirely.
|
||||||
#[test]
|
#[test]
|
||||||
fn resolve_dsn_header_leads_explicit_headers() {
|
fn resolve_rejects_a_query_or_fragment_on_the_generic_endpoint() {
|
||||||
let cfg = TelemetryConfig::resolve(RawTelemetryArgs {
|
for endpoint in [
|
||||||
uptrace_dsn: Some("https://tok@api.uptrace.dev".into()),
|
"https://api.uptrace.dev?grpc=4317",
|
||||||
otlp_headers: vec!["x-extra=1".into()],
|
"https://api.uptrace.dev/?grpc=4317",
|
||||||
..raw()
|
"https://api.uptrace.dev:4318?grpc=4317",
|
||||||
})
|
"https://api.uptrace.dev#frag",
|
||||||
|
] {
|
||||||
|
let err = TelemetryConfig::resolve(&raw(), generic(endpoint)).unwrap_err();
|
||||||
|
assert!(err.to_string().contains("trailing"), "{endpoint}: {err}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A path prefix composes correctly (`/otlp` -> `POST /otlp/v1/metrics`),
|
||||||
|
/// and the per-signal variables are used verbatim, tail and all.
|
||||||
|
#[test]
|
||||||
|
fn resolve_accepts_a_path_prefix_and_per_signal_tails() {
|
||||||
|
for env in [
|
||||||
|
generic("https://collector.example.com/otlp"),
|
||||||
|
generic("https://collector.example.com/"),
|
||||||
|
OtelEnv {
|
||||||
|
metrics_endpoint: Some("https://api.uptrace.dev/v1/metrics?grpc=4317".into()),
|
||||||
|
..OtelEnv::default()
|
||||||
|
},
|
||||||
|
] {
|
||||||
|
TelemetryConfig::resolve(&raw(), env.clone())
|
||||||
|
.unwrap_or_else(|e| panic!("{env:?}: {e}"))
|
||||||
|
.expect("telemetry on");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// `.with_http()` fixes the transport at compile time, so a protocol this
|
||||||
|
/// build cannot speak is a dead export with nothing in the log.
|
||||||
|
#[test]
|
||||||
|
fn resolve_rejects_a_protocol_this_build_cannot_speak() {
|
||||||
|
let with_protocol = |field: fn(&mut OtelEnv)| {
|
||||||
|
let mut env = generic("https://collector.example.com");
|
||||||
|
field(&mut env);
|
||||||
|
env
|
||||||
|
};
|
||||||
|
for env in [
|
||||||
|
with_protocol(|e| e.protocol = Some("grpc".into())),
|
||||||
|
with_protocol(|e| e.protocol = Some("http/json".into())),
|
||||||
|
with_protocol(|e| e.metrics_protocol = Some("grpc".into())),
|
||||||
|
with_protocol(|e| e.traces_protocol = Some("grpc".into())),
|
||||||
|
] {
|
||||||
|
let err = TelemetryConfig::resolve(&raw(), env.clone())
|
||||||
|
.unwrap_err()
|
||||||
|
.to_string();
|
||||||
|
assert!(err.contains("http/protobuf"), "{env:?}: {err}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The default protocol is the one this build speaks, so naming it
|
||||||
|
/// explicitly is not an error.
|
||||||
|
#[test]
|
||||||
|
fn resolve_accepts_the_supported_protocol_spelled_out() {
|
||||||
|
let env = OtelEnv {
|
||||||
|
protocol: Some("http/protobuf".into()),
|
||||||
|
metrics_protocol: Some("http/protobuf".into()),
|
||||||
|
..generic("https://collector.example.com")
|
||||||
|
};
|
||||||
|
TelemetryConfig::resolve(&raw(), env)
|
||||||
.unwrap()
|
.unwrap()
|
||||||
.expect("telemetry on");
|
.expect("telemetry on");
|
||||||
assert_eq!(cfg.headers[0].0, "uptrace-dsn");
|
}
|
||||||
assert_eq!(cfg.headers[1], ("x-extra".to_string(), "1".to_string()));
|
|
||||||
|
/// Contract point 6: `service.name` stays anwesen's default only while
|
||||||
|
/// the environment supplies none. A builder attribute wins over the SDK's
|
||||||
|
/// detectors, so the default has to step aside for the override to work.
|
||||||
|
#[test]
|
||||||
|
fn service_identity_defaults_step_aside_for_the_environment() {
|
||||||
|
let keys = |attrs: &[KeyValue]| {
|
||||||
|
attrs
|
||||||
|
.iter()
|
||||||
|
.map(|kv| kv.key.as_str().to_string())
|
||||||
|
.collect::<Vec<_>>()
|
||||||
|
};
|
||||||
|
|
||||||
|
let plain = default_resource_attrs(None, None);
|
||||||
|
assert_eq!(keys(&plain), [SERVICE_NAME, SERVICE_VERSION]);
|
||||||
|
|
||||||
|
let named = default_resource_attrs(Some("svcname-override"), None);
|
||||||
|
assert_eq!(keys(&named), [SERVICE_VERSION]);
|
||||||
|
|
||||||
|
let attrs = default_resource_attrs(
|
||||||
|
None,
|
||||||
|
Some("deployment.environment=prod,service.name=attrs-override"),
|
||||||
|
);
|
||||||
|
assert_eq!(keys(&attrs), [SERVICE_VERSION]);
|
||||||
|
|
||||||
|
let versioned = default_resource_attrs(None, Some("service.version=9.9.9"));
|
||||||
|
assert_eq!(keys(&versioned), [SERVICE_NAME]);
|
||||||
|
|
||||||
|
// An empty OTEL_SERVICE_NAME supplies nothing; the SDK ignores it too.
|
||||||
|
let empty = default_resource_attrs(Some(" "), None);
|
||||||
|
assert_eq!(keys(&empty), [SERVICE_NAME, SERVICE_VERSION]);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn resolve_rejects_both_endpoint_sources() {
|
fn resolve_rejects_the_removed_uptrace_dsn() {
|
||||||
let err = TelemetryConfig::resolve(RawTelemetryArgs {
|
let err = TelemetryConfig::resolve(
|
||||||
|
&RawTelemetryArgs {
|
||||||
uptrace_dsn: Some("https://tok@api.uptrace.dev".into()),
|
uptrace_dsn: Some("https://tok@api.uptrace.dev".into()),
|
||||||
otlp_endpoint: Some("https://collector.example.com".into()),
|
|
||||||
..raw()
|
..raw()
|
||||||
})
|
},
|
||||||
.unwrap_err();
|
OtelEnv::default(),
|
||||||
assert!(err.to_string().contains("mutually exclusive"));
|
)
|
||||||
|
.unwrap_err()
|
||||||
|
.to_string();
|
||||||
|
assert!(err.contains("--uptrace-dsn"), "{err}");
|
||||||
|
assert!(err.contains("OTEL_EXPORTER_OTLP_HEADERS"), "{err}");
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn resolve_rejects_malformed_header() {
|
fn resolve_rejects_the_removed_otlp_endpoint() {
|
||||||
let err = TelemetryConfig::resolve(RawTelemetryArgs {
|
let err = TelemetryConfig::resolve(
|
||||||
|
&RawTelemetryArgs {
|
||||||
otlp_endpoint: Some("https://collector.example.com".into()),
|
otlp_endpoint: Some("https://collector.example.com".into()),
|
||||||
otlp_headers: vec!["no-equals-sign".into()],
|
|
||||||
..raw()
|
..raw()
|
||||||
})
|
},
|
||||||
.unwrap_err();
|
OtelEnv::default(),
|
||||||
assert!(err.to_string().contains("key=value"));
|
)
|
||||||
|
.unwrap_err()
|
||||||
|
.to_string();
|
||||||
|
assert!(err.contains("--otlp-endpoint"), "{err}");
|
||||||
|
assert!(err.contains("OTEL_EXPORTER_OTLP_ENDPOINT"), "{err}");
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn resolve_rejects_dsn_without_scheme() {
|
fn resolve_rejects_the_removed_otlp_header() {
|
||||||
let err = TelemetryConfig::resolve(RawTelemetryArgs {
|
let err = TelemetryConfig::resolve(
|
||||||
uptrace_dsn: Some("tok@api.uptrace.dev".into()),
|
&RawTelemetryArgs {
|
||||||
|
otlp_headers: vec!["authorization=Bearer xyz".into()],
|
||||||
..raw()
|
..raw()
|
||||||
})
|
},
|
||||||
|
OtelEnv::default(),
|
||||||
|
)
|
||||||
|
.unwrap_err()
|
||||||
|
.to_string();
|
||||||
|
assert!(err.contains("--otlp-header"), "{err}");
|
||||||
|
assert!(err.contains("OTEL_EXPORTER_OTLP_HEADERS"), "{err}");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A removed option fails even with a correct `OTEL_` endpoint alongside
|
||||||
|
/// it: the operator's intent is in the flag, and half-applied config is
|
||||||
|
/// the silent failure this issue removes.
|
||||||
|
#[test]
|
||||||
|
fn resolve_rejects_a_removed_option_alongside_a_good_endpoint() {
|
||||||
|
let err = TelemetryConfig::resolve(
|
||||||
|
&RawTelemetryArgs {
|
||||||
|
uptrace_dsn: Some("https://tok@api.uptrace.dev".into()),
|
||||||
|
..raw()
|
||||||
|
},
|
||||||
|
generic("https://api.uptrace.dev"),
|
||||||
|
)
|
||||||
.unwrap_err();
|
.unwrap_err();
|
||||||
assert!(err.to_string().contains("scheme"));
|
assert!(err.to_string().contains("removed"), "{err}");
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue