Run the handler with a local OpenTelemetry tracer, collect its finished spans in memory, and assert both approved attribute names and approved values. An allowlist that permits http.route must still reject an email address copied into that field.
Consider a checkout handler whose span may contain only the route template /checkout and an integer HTTP response status. Adding user.email should fail its contract test. Moving that same email into http.route should fail too, even though the attribute name is approved.
The Stack Overflow episode description connects an influx of AI-generated code with changing system specifications and discusses the boundary between observability and surveillance. It reports a discussion, not a demonstrated telemetry leak. The practical decision for instrumentation engineers is narrower: which data may this handler emit, and what check will reject an expansion?
For this handler, I would enforce a small, explicit data contract rather than rely on a sensitive-looking-key scan. The contract applies equally to human-written and AI-generated instrumentation.
Approve values, not just field names
Use this attribute contract as the review artifact:
| Surface | Approved value | Rejected example |
|---|---|---|
http.route |
Exactly /checkout, a fixed route template |
An email address, raw URL, or query string |
http.response.status_code |
An integer from 100 through 599 | A string, boolean, or out-of-range number |
| Any additional span attribute | Nothing | user.email, request bodies, arbitrary metadata |
| Synthetic email sentinel | Must not appear in any attribute value | trace-contract@example.invalid |
The status range is this example’s policy, not a claim that every handler should emit every status in that range. A handler with a smaller outcome set can use a stricter enumeration.
An approved key can still carry an unapproved value.
The sentinel gives the test a recognizable sensitive input. It is a tripwire, not an email detector: the route constraint rejects arbitrary strings, while the sentinel check produces a direct diagnostic if that particular input reaches an attribute.
Build an isolated exporter fixture
The example below uses a provider owned by the test rather than changing the process-wide tracer provider. It configures only an in-memory exporter, so the spans produced through this tracer are not sent to a telemetry backend.
Install the dependencies in an isolated environment:
python -m venv .venv
. .venv/bin/activate
python -m pip install pytest opentelemetry-api opentelemetry-sdk
These commands do not pin versions. For repeatable CI dependency resolution, commit the resolved versions using your repository’s lockfile process. The example has not been executed here; the outcomes below are expectations derived from the assertions.
Save this as test_trace_contract.py:
import pytest
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.sdk.trace.export.in_memory_span_exporter import (
InMemorySpanExporter,
)
EMAIL_SENTINEL = "trace-contract@example.invalid"
REQUIRED_KEYS = {"http.route", "http.response.status_code"}
@pytest.fixture
def trace_capture():
exporter = InMemorySpanExporter()
provider = TracerProvider()
provider.add_span_processor(SimpleSpanProcessor(exporter))
tracer = provider.get_tracer("checkout-contract-test")
try:
yield tracer, exporter
finally:
provider.shutdown()
def handle_checkout(request, tracer):
# Minimal handler: no network, database, or customer data.
with tracer.start_as_current_span("checkout") as span:
span.set_attribute("http.route", "/checkout")
status = 201
span.set_attribute("http.response.status_code", status)
return status
def assert_attribute_contract(span):
attributes = dict(span.attributes or {})
keys = set(attributes)
errors = []
missing = REQUIRED_KEYS - keys
unexpected = keys - REQUIRED_KEYS
if missing:
errors.append(f"missing keys: {sorted(missing)}")
if unexpected:
errors.append(f"unexpected keys: {sorted(unexpected)}")
if "http.route" in attributes:
if attributes["http.route"] != "/checkout":
errors.append("http.route must equal '/checkout'")
if "http.response.status_code" in attributes:
status = attributes["http.response.status_code"]
# bool is a subclass of int, so use an exact type check.
if type(status) is not int or not 100 <= status <= 599:
errors.append(
"http.response.status_code must be an integer from 100 to 599"
)
for key, value in attributes.items():
if EMAIL_SENTINEL in str(value):
errors.append(f"synthetic email sentinel found in {key}")
if errors:
raise AssertionError("\n".join(errors))
def test_checkout_span_attributes(trace_capture):
tracer, exporter = trace_capture
request = {
"method": "POST",
"path": "/checkout",
"email": EMAIL_SENTINEL,
}
assert handle_checkout(request, tracer) == 201
# The handler's context manager has ended the span before collection.
spans = exporter.get_finished_spans()
assert len(spans) == 1, "expected exactly one handler span"
assert spans[0].name == "checkout"
assert_attribute_contract(spans[0])
Run the gate with:
python -m pytest -q test_trace_contract.py
The baseline is expected to pass: the handler emits exactly the two required attributes, the route is fixed, the status is an integer in range, and neither value contains the sentinel.
The unused request fields are intentional. Having data available to a handler does not make it approved telemetry.
SimpleSpanProcessor sends ended spans to the exporter synchronously in this setup. Collecting after the handler returns avoids a timing-dependent sleep. Each test receives its own provider and exporter, so this fixture does not depend on spans left behind by another test.
Make two instrumentation regressions fail
First, insert this line immediately before return status, inside the span context:
span.set_attribute("user.email", request["email"])
Rerun the same command. The test is expected to fail with both diagnostics:
unexpected keys: ['user.email']
synthetic email sentinel found in user.email
The unexpected-key check catches the expanded schema. The sentinel check identifies the synthetic sensitive value without printing it into the failure message.
Next, remove that line and replace the original route assignment with:
span.set_attribute("http.route", request["email"])
This mutation keeps the approved key set unchanged. The expected failure is:
http.route must equal '/checkout'
synthetic email sentinel found in http.route
A key-only allowlist would miss this regression. The value constraint catches it because a route template has a narrower meaning than “any string.”
Restore the fixed route assignment after checking the failure. Keep the passing test as the CI gate; if you retain mutation cases as tests, make them explicitly assert that the contract rejects the bad span.
Keep the gate strict without overstating its coverage
This example requires exactly one span because that is the handler’s intended output. It will fail if instrumentation adds another span, forcing a review. If your handler legitimately emits child spans, define a contract for each expected span role rather than silently filtering away spans you did not expect.
Checking fewer spans creates a smaller guarantee.
Use this checklist when adapting the gate:
- Choose stable fields. Prefer route templates and bounded status values over raw paths, query strings, or request objects.
- Specify allowed values. Use exact constants, small enumerations, or explicit type-and-range constraints. An unrestricted string leaves the data policy undefined.
- Seed synthetic inputs. Put recognizable sentinels in request fields that must stay out of telemetry, then exercise the branches that read those fields.
- Inventory other surfaces. Give logs, span events, resource attributes, span names, and baggage their own checks where relevant.
This test covers span attributes captured through the selected tracer on the exercised handler path. It does not inspect logs, span events, resource attributes, other exporters, or every production execution path. Hashing, encoding, or splitting a sentinel also defeats the substring check, although the strict route constraint still rejects those values in this example.
I would use this gate for a small, stable instrumentation contract. I would not treat one passing request as evidence that an application is free of sensitive telemetry. Additional branches and telemetry surfaces need additional cases.
Try the smallest useful gate today
In five minutes, choose one handler, approve its attribute keys and value constraints, and run one synthetic request through a local exporter. Deliberately add one forbidden attribute and confirm that the check fails before restoring the baseline.
That exercise verifies the test’s ability to reject one known regression, not the safety of the whole application. Once it behaves as expected, add the passing command to your existing CI job.
For the next step, put a verification gate on your pipeline.