Seventy-three percent of post-incident reviews at engineering organizations cite "code quality issues" as a contributing factor, yet almost none of those organizations have a single dashboard tracking code quality over time. That gap is the article.
Uptime gets a full stack: Datadog, PagerDuty, SLOs, error budgets, burn-rate alerts, on-call rotations. Code quality gets a pre-commit hook that half the team has disabled locally and a quarterly reminder to "pay down tech debt." The asymmetry is embarrassing once you see it.
What We Actually Mean by Code Quality Observability
Observability, in the systems sense, means you can ask arbitrary questions about your system's behavior without deploying new instrumentation. You emit structured signals, store them, and query them. The key word is arbitrary: you do not have to know the question in advance.
Applying that definition to code quality means treating your codebase as a system that emits signals on every change. Complexity scores, duplication ratios, coupling metrics, test coverage deltas, static analysis violations, type-check error counts. Not as one-off gate conditions that block a PR, but as time-series data you can trend, alert on, and correlate with production incidents.
The distinction matters. A gate says "this PR is bad, fix it." Observability says "coupling in the payments module has increased 18% over six weeks, correlated with three latency regressions in the checkout service." One is a speed bump. The other is a signal that lets you make a staffed engineering decision before the incident.
Most teams stop at the speed bump.
The Instrumentation Gap
Here is what a typical CI pipeline looks like from a data perspective:
# .github/workflows/ci.yml (typical)
jobs:
test:
steps:
- run: pytest --cov=src
- run: ruff check .
- run: mypy src/
All three of those steps produce structured output. pytest gives you a coverage percentage, pass/fail counts, and per-module coverage. ruff gives you violation counts by rule. mypy gives you error counts by file and error code. None of that is stored anywhere after the job finishes, except as a binary pass/fail artifact in your CI provider's UI, inaccessible to any time-series query.
The fix is not complicated.
# scripts/emit_quality_metrics.py
import subprocess, json, time, httpx
def collect_mypy_errors(src_path: str) -> int:
result = subprocess.run(
["mypy", src_path, "--output=json"],
capture_output=True, text=True
)
lines = [l for l in result.stdout.splitlines() if l.strip()]
return len(lines)
def push_metric(name: str, value: float, tags: dict) -> None:
# Push to Datadog statsd, InfluxDB, or your preferred sink
payload = {"metric": name, "value": value, "tags": tags, "timestamp": time.time()}
httpx.post("http://metrics-collector:9090/ingest", json=payload, timeout=5)
if __name__ == "__main__":
errors = collect_mypy_errors("src/")
push_metric(
"code_quality.mypy_errors",
errors,
{"repo": "payments-service", "branch": "main", "env": "ci"}
)
Add a step like this after every analysis tool in CI, point it at a time-series sink (Datadog custom metrics, InfluxDB, even a Postgres table with a recorded_at timestamp), and you have the raw signal layer. From here, everything else is queries and dashboards you already know how to build.
The reason almost nobody does this is not technical. Code quality metrics do not have a natural owner. SRE owns uptime. Nobody owns code quality observability, so it does not happen.
What Signals Actually Matter
Not every metric is worth collecting. Here is where I would spend the signal budget:
Cyclomatic complexity per module, trended over time. Radon in Python, lizard for polyglot repos. A single spike is noise. A sustained upward trend in a specific module is a staffing or architectural problem. The number itself is less interesting than the slope.
Type coverage percentage, not just error count. If your codebase is gradually shedding type annotations because nobody is enforcing the trend, mypy's error count stays flat (no errors on untyped code) while your actual safety net shrinks. Track mypy --txt-report output's typed-vs-untyped ratio explicitly.
Test coverage delta per PR, not absolute coverage. Absolute coverage is easy to game and hard to act on. Delta per PR tells you whether coverage is trending in the right direction on the code that is actually changing. A PR that adds 200 lines and drops net coverage by 4% is a concrete conversation.
Static analysis violation counts by category, not just total. Aggregating ruff, ESLint, or Semgrep violations into a single number loses the signal. A spike in security-category violations is a different conversation than a spike in style violations.
Skip cyclomatic complexity per function (too granular, too noisy) and lines-of-code trends (almost always misleading). Senior engineers optimize for signal-to-noise, not metric count.
Correlating Quality Signals with Production Incidents
This is where the investment pays off, and it requires one deliberate data modeling decision: timestamp your quality metrics with the same precision as your incident timeline.
When you have a production incident in the payments service, you can now ask what was happening to the quality signals for that service in the two weeks before the incident. In my experience running this kind of retrospective, the answer is almost always one of two patterns.
Pattern one: coupling metrics spiked because a cross-team dependency was refactored hastily under deadline pressure. The service started importing internals it should not have touched. Static analysis caught the import style but not the architectural coupling. A dedicated coupling metric (fan-in/fan-out ratios via tools like pydeps or dependency-cruiser) would have flagged it.
Pattern two: test coverage for a specific subsystem dropped 12 percentage points over three weeks as new features landed without matching tests. The overall coverage number looked fine because the rest of the codebase was well-covered. Only per-module trending revealed the gap.
Neither of these is obvious from incident review alone. They become obvious when you have the time-series data to draw the line between the quality degradation and the production failure.
OpenThunder takes this correlation work seriously: it runs static, dynamic, and behavioral checks on every change, which means the signal history exists at the PR level, not just at the nightly batch level. That granularity matters when you are trying to bisect a quality regression to a specific merge.
Making Quality Dashboards Actionable
A dashboard nobody acts on is a monument to good intentions. Quality dashboards get ignored because they show state without showing responsibility or recommended action.
Three things that fix this:
Ownership labels on every metric. Every module's metrics should map to a team. In Datadog, this is a tag. In Grafana, it is a template variable. When the payments module's complexity spikes, the payments team gets the alert, not the entire engineering organization.
Threshold-based alerts with a decay period. Alert when a metric crosses a threshold and stays there for five business days. This filters out PR-level spikes from temporary work-in-progress and surfaces actual trends. Alerting on every spike teaches teams to ignore the alerts.
Link alerts to the relevant PR range. If your CI pipeline is emitting metrics tagged with commit SHA, an alert can automatically link to the GitHub compare view for the relevant range. Engineers should not have to go hunting for what caused the trend.
The goal is a weekly engineering sync where someone can say "complexity in the auth module is up 22% this sprint, here are the three PRs that drove it, the team lead is aware" rather than "we should really look at code quality sometime."
OpenThunder surfaces this kind of finding at the PR level, which is the right moment: before the merge, when the cost of fixing it is lowest and the author is still context-loaded.
The Organizational Case
If you are trying to sell this internally, the argument is not "code quality is important." Every engineering leader already believes that and it has not moved the needle. The argument is: we already have the instrumentation investment model for production systems, we just have not applied it to the system that produces those systems.
Frame it as extending your existing observability stack, not adding a new process. You probably already have Datadog or Grafana. You already have CI. You are not asking for new tooling budget; you are asking for a few CI script additions and a new dashboard in the tool you already pay for.
The ask is modest. The payoff, when you can show that a quality regression preceded a production incident by two weeks and you now have the instrumentation to catch it earlier, is not modest at all.
Put a verification gate on your pipeline
If you want the signal layer without building it yourself, OpenThunder runs static, dynamic, and behavioral checks on every change and turns failures into fixable findings with the historical record attached. Try it here.
The code that runs your product is a system; treat it like one and instrument it accordingly.