CommitFest 6804 Review: log_postmaster_stats

report · generated from canonical Markdown

Type
report
Source
postgresql-master
Slug
commitfest-6804-review
Tags
commitfest-6804coveragepg-review-harnessreview-report-v2waiting-on-author
Updated
2026-08-31T01:50:51.422375+00:00
Verdict / Summarywaiting-on-author
Front matter
schema: review-report-v2
target: "log_postmaster_stats GUC - periodically log postmaster connection/CPU statistics (CommitFest 6804)"
base: 92819e57945d106317eaaa4273180dd8663f91bd
head: 92819e57945d106317eaaa4273180dd8663f91bd
scope_identity: diff-e20ef1973d1da42128e5ea581ef718fe2ed14374
diff_sha256: 27de5b7e9dc78b7a3eb45b503b38dc2fa47e9df22c6d4a0165dc5d194572598b
diff_blob_sha: e20ef1973d1da42128e5ea581ef718fe2ed14374
verdict: waiting-on-author
blocker_counts:
  verified: 1
  unverified: 0
model: glm-5.2:cloud
model_policy_version: 1
model_policy_hash: 997c035512fa1b35aa17149ea386a861af531ec05839ce06810f38268a3caebe
source: postgresql-master
slug: commitfest-6804-review

CommitFest 6804 Review — log_postmaster_stats

Verdict

PostgreSQL readiness: waiting-on-author

Scope

Wiki "Reviewing a Patch" six-heading summary

  1. What it does: Adds a SIGHUP GUC log_postmaster_stats (seconds, default 0=off). When positive, the postmaster periodically emits a LOG line with connection rate, disconnection rate, parallel-worker-start rate, and postmaster CPU/elapsed usage (pg_rusage_show). DetermineSleepTime caps wakeups to the configured interval.
  2. Does it work: Yes, mechanically. Compiles cleanly against HEAD; the logic is straightforward and consistent with existing periodic-block patterns in ServerLoop. No crash/assertion path found. Verified non-findings: lockless parallel_register_count read (PAT-6/SEC-1) and DetermineSleepTime cap (PAT-5) are correct.
  3. Performance: One getrusage + gettimeofday per interval (≥1s); negligible vs fork() cost. Unlocked shared-memory read is the established postmaster pattern. No hot-path cost when =0.
  4. Security: No new external input, no privilege change, no catalog access, getrusage(RUSAGE_SELF) is process-local. No security finding (SEC-1/SEC-4 refuted as security issues; SEC-2/SEC-5/SEC-6 informational non-findings).
  5. Smarter approaches: The unresolved idle-flood (PAT-2) is the main design smell; the author's own later thread direction (overload-oriented GUCs) and Robert Haas's M7 cumulative-stats/connection-rate proposal are competing, arguably smarter approaches. The simple periodic GUC and the overload GUCs should be reconciled on-list (PAT-1).
  6. Conclusion: Sound, low-risk implementation hampered by an open design/process question and a weak test. Address PAT-1 (thread confirm), PAT-2 (idle flood), PAT-3 (test strength), then ready-for-committer.

Findings

Severity scale: BLOCKER > MAJOR > MINOR > NIT.

PAT-1

— BLOCKER (process, UNVERIFIED) — thread alignment / scope drift

  • Where: whole patch (design), referencing thread 646913 (msgs 2026-07-27 M6, 2026-07-28 M7, 2026-07-30 M8).
  • Mechanism: The worktree implements the original M1 log_postmaster_stats interval-logger design. The author's latest postings (M6/M8) renamed/expanded the concept to log_postmaster_overloads + log_postmaster_excess_connections, and committer Robert Haas (M7) proposed dropping the CPU/rusage path for a pure log_excessive_connection_rate=N. There is no on-list message confirming the simple variant is still the intended landing target.
  • verification_status: cannot-verify (process/design alignment question; no code patch can verify or resolve it). Labelled UNVERIFIED. Classified BLOCKER-class per the harness verdict rules.
  • verification_reason: A code proposal here would be unsafe/misleading — committing without the author's on-list confirmation risks landing a design the thread no longer endorses.
  • Repair (waived, process): Post an on-list message asking the author (Jakub Wartak) to confirm whether the simple single-GUC log_postmaster_stats variant is still the intended landing target, or whether the worktree is stale relative to the M6-M8 renamed two-GUC direction. No commit should proceed until resolved on-list. No code patch, no validation, no test (typed safety waivers: design-choice).
PAT-2

— MAJOR (behavior) — idle-database log flood (committer-corroborated, unaddressed)

  • Where: src/backend/postmaster/postmaster.c ServerLoop stats block (~1854-1888); GUC log_postmaster_stats.
  • Mechanism: When the cluster is idle, conn_delta == disc_delta == pqw_delta == 0 on every interval, yet the block unconditionally emits a LOG line. Reviewer Quan Zongliang raised this (2026-06-10); committer Robert Haas corroborated the zero-counter suppression fix (2026-07-20: "omitting the line altogether when all the counters were zero would be sensible"). The worktree does NOT implement suppression.
  • verification_status: verified (method: two-run TAP gate).
  • verification_evidence: Two-run gate (independent, verify-scratch): un-repaired 006 idle-skip FAIL (Tests=2 Failed=2; Phase 1 saw idle 0.00-rate line; Phase 2 elapsed=2.10s < 3s floor); repaired 006 PASS (Tests=2 6s).
  • Repair (passed): Suppress the ereport when all three deltas are zero AND leave baselines/last_pmstats_time/pg_rusage_init untouched so the next non-idle window covers the full quiet period. Add an else branch on the elapsed > 0 guard that resets baselines defensively for the time-jumped-backwards case.
  • Patch: src/backend/postmaster/postmaster.c (unified-diff bound to HEAD 92819e57945d106317eaaa4273180dd8663f91bd); sha256 8aa94a546036e0de6dcc233e85fbf450b2cbf3de91488c9af011049e1ef1a053.
  • Validation: apply-check/apply/diff-check OK; postmaster.o compiles with -Wall -Werror=vla (gcc 15.2.0) no warnings; relink OK; 006 two-run gate FAIL→PASS.
  • Test example: src/test/postmaster/t/006_log_postmaster_stats_idle_skip.pl (new, 98 lines, in test-patches.patch). Oracle: Phase 1 no line while idle ≥5s; Phase 2 first non-idle line elapsed ≥3s.
PAT-3

— MAJOR (tests) — TAP test fragile / under-covers

  • Where: src/test/postmaster/t/005_log_postmaster_stats.pl.
  • Mechanism: (a) $offset captured before safe_psql but postmaster may emit during startup; (b) loose regex only checks conns/sec, never validates disconns/parallel/CPU; (c) no interval-timing assertion; (d) redundant pass().
  • verification_status: verified (method: inspection + two-run gate).
  • verification_evidence: Reconciled strengthened 005 PASSED on the repaired build (8 tests, 10-11s). The test-author's original strengthened 005 was reconciled (Test 1 issues SELECT 1 before waiting) to remain compatible with the PAT-2 idle-skip repair; the original would time out at Test 1 on the repaired build. Actual patched bytes validated.
  • Repair (passed): Replace 005 with the reconciled strengthened version (full-field STATS_RE; disconnection/parallel/SIGHUP coverage; Test 1 issues SELECT 1 before waiting). Add 006 idle-skip negative test. Register 006 in meson.build.
  • Patch: src/test/postmaster/t/005_log_postmaster_stats.pl + t/006_... + meson.build (unified-diff bound to HEAD); sha256 1cfb94614c9d88bf8313700fafdc40218c773154a958cd60110d180f373013a9.
  • Validation: apply-check/diff-check OK; on fully-patched build 005 PASS (8 tests 11s), 006 PASS (2 tests 6s); 006 two-run gate FAIL un-repaired / PASS repaired.
PAT-4

— MINOR (docs) — pg_rusage elapsed vs time_t elapsed divergence

  • Where: src/backend/postmaster/postmaster.c:1861 (time_t elapsed) vs :1880 (pg_rusage_show).
  • Mechanism: Rates use coarse time_t seconds; CPU elapsed uses gettimeofday microseconds. The two "elapsed" figures can diverge by up to ~1s. Cosmetic; author self-noted on-list (M1).
  • verification_status: verified (inspection). Stands under the worktree's M1 design; likely moot under Robert M7's drop-CPU direction.
  • Repair (passed): Combined SGML patch adds a precision note. Patch sha256 49ff579803ac070261bb168a495bdc41127c981f0a62b95408dd0f2bff81c081 (with PAT-7/PAT-8). Validation: apply/diff-check OK; make -C doc/src/sgml postgres.html succeeds with xmllint --valid.
PAT-7

— MINOR (docs, new) — SGML disconnection wording loose

  • Where: doc/src/sgml/config.sgml (~8500); code postmaster.c:2690 + miscadmin.h:418.
  • Mechanism: SGML says the connection rate counts regular+replication but does not state the disconnection rate is likewise limited to B_BACKEND/B_WAL_SENDER exits.
  • verification_status: verified (inspection). Genuinely new; no thread discussion.
  • Repair (passed): Combined SGML patch rewords to cover both rates. Patch sha256 49ff579803ac070261bb168a495bdc41127c981f0a62b95408dd0f2bff81c081. Validation: doc-build PASS.
PAT-8

— NIT (docs, new) — SGML sample wrapped across two physical lines

  • Where: doc/src/sgml/config.sgml (~8497-8498).
  • Mechanism: The <screen> sample wraps postmaster stats: across two lines while the actual ereport produces a single-line LOG.
  • verification_status: verified (inspection). Genuinely new; no thread discussion.
  • Repair (passed): Combined SGML patch renders the sample as one line. Patch sha256 49ff579803ac070261bb168a495bdc41127c981f0a62b95408dd0f2bff81c081. Validation: doc-build PASS.
COV-1

— MINOR (tests, verified-by-coverage) — disabled-GUC default path untested

  • Where: src/backend/postmaster/postmaster.c:1656 (and line 1653 false branch).
  • Mechanism: The disabled-GUC default path (log_postmaster_stats == 0) is untested. DetermineSleepTime line 1653 false branch and line 1656 (return 60 * 1000) are gcov count 0 under the shipped 005 test; reachable with a standard TAP cluster started with the GUC at its default. This is the symmetric negative case and a regression guard that the disabled GUC is a true no-op.
  • verification_status: verified (method: verified-by-coverage).
  • verification_evidence: Fresh-epoch gcov (coverage seat): line 1656 count 0; line 1653 false branch count 0. Re-validated (cov-revalidate handoff): 008 test passed twice (4.55s, 5.07s) on the patched tree; fresh-epoch gcov shows line 1656 count 0→12 and line 1653 false branch 0→12. Coverage gap closed. Existing 005 coverage retained.
  • coverage_reference: src/backend/postmaster/postmaster.c:1656 (line).
  • Repair (passed): Add src/test/postmaster/t/008_log_postmaster_stats_disabled.pl + register in meson.build. Patch sha256 1458e1e28bb74ad55f52b6e39ed980a044bfe68907c6974ef2bfc594cd61abdc. Validation: apply/diff-check OK; coverage build OK (2401/2401 targets); 008 PASS twice; full postmaster suite 6/6 OK; fresh-epoch gcov confirms line 1656 0→12 and line 1653 false branch 0→12.

Refuted findings (kept in report, not in counts)

Verification Summary

#FindingSeverityverification_statusmethodrepair.validation
PAT-1scope drift / processBLOCKERcannot-verify (UNVERIFIED)inspection (thread ledger)waived (process)
PAT-2idle-database log floodMAJORverifiedtwo-run (TAP)passed
PAT-3005 fragile/under-coversMAJORverifiedinspection + two-runpassed
PAT-4rusage vs time_t elapsed divergenceMINORverifiedinspectionpassed (doc-build)
PAT-7SGML disconnection wording looseMINORverifiedinspectionpassed (doc-build)
PAT-8SGML sample wrapped across two linesNITverifiedinspectionpassed (doc-build)
COV-1disabled-GUC default path untestedMINORverifiedverified-by-coveragepassed
SEC-1 / PAT-6unlocked read of parallel_register_countMINORrefuted-as-security-issueinspectionnot-applicable-refuted
SEC-4tiny-interval self-DoSMINORrefuted-as-security-issueinspectionnot-applicable-refuted
SEC-2pqw_delta uint32 wraparoundNITrefutedinspectionnot-applicable-refuted
SEC-5time_t backwards jumpNITrefutedinspectionnot-applicable-refuted
SEC-6pg_rusage_show static bufferNITrefutedinspectionnot-applicable-refuted

Counts: verified = 6 (PAT-2, PAT-3, PAT-4, PAT-7, PAT-8, COV-1); cannot-verify = 1 (PAT-1, UNVERIFIED); refuted = 5 (SEC-1, SEC-4, SEC-2, SEC-5, SEC-6). Verified BLOCKER/MAJOR: PAT-1 (blocker-class process), PAT-2 (MAJOR behavior), PAT-3 (MAJOR tests). Unverified BLOCKER/MAJOR: 0.

Repair Validation Summary

All verified code/test findings carry concrete HEAD-bound unified diffs validated in disposable scratch (apply-check, apply, diff-check, build, targeted-test, two-run gate / coverage as applicable). PAT-1 is a process repair with a typed safety waiver (design-choice) and no code patch.

Patchfilessha256validation
pat2-repair.patchsrc/backend/postmaster/postmaster.c8aa94a546036e0de6dcc233e85fbf450b2cbf3de91488c9af011049e1ef1a053passed: apply, compile, 006 two-run gate (FAIL→PASS)
test-patches.patchsrc/test/postmaster/t/005_…, t/006_…, meson.build1cfb94614c9d88bf8313700fafdc40218c773154a958cd60110d180f373013a9passed: apply, 005+006 run on repaired; 006 two-run gate
sgml-repair.patchdoc/src/sgml/config.sgml (PAT-4/PAT-7/PAT-8)49ff579803ac070261bb168a495bdc41127c981f0a62b95408dd0f2bff81c081passed: apply, doc-build (postgres.html + xmllint --valid)
test-cov1.patchsrc/test/postmaster/t/008_…, meson.build1458e1e28bb74ad55f52b6e39ed980a044bfe68907c6974ef2bfc594cd61abdcpassed: apply, coverage build, 008 PASS (×2), fresh-epoch gcov 0→12

A successful repair does not prove a finding, and a failed repair does not refute it. Finding verification and repair validation never override one another.

Changed-Code Coverage

Coverage requirement: required (changed instrumentable implementation in src/backend/postmaster/postmaster.c and src/backend/postmaster/bgworker.c).

Execution lane: host-disposable (passed). Fresh meson coverage build (b_coverage=true, buildtype=debug, cassert=true, tap_tests=enabled, readline=disabled, icu=disabled) at frozen HEAD + scope diff; postgresql:postmaster/005_log_postmaster_stats OK 1 subtest passed 2.32s; durable test.success marker present; "postmaster stats:" LOG lines present in primary log. No nested-sandbox attempt needed.

Fresh disposable counter epoch: disposable source = fresh git clone of HEAD 92819e57945d106317eaaa4273180dd8663f91bd, scope diff applied with git apply --index (scope diff sha256 verified 27de5b7e…, matches scope identity). .gcno count 1650. After --suite postgresql:setup produced 1016 .gcda, all .gcda were deleted and asserted zero immediately before the measured test run: FRESH_EPOCH_PRE_TEST: gcda_count=0. The measured test then ran exactly once under LC_ALL=C LANG=C TZ=UTC; after the run 974 .gcda files including postmaster_postmaster.c.gcda and postmaster_bgworker.c.gcda. No shared/user build counters were touched.

Test set (exact executed command):

meson test -C <cov-scratch>/mbuild --no-rebuild -j1 --print-errorlogs postgresql:postmaster/005_log_postmaster_stats

Per-test status: selected=1, started=1, passed=1, failed=0, skipped/gated=0, timed-out=0. PRIMARY measurement is against the patch-as-shipped 25-line 005 test (the test set the harness is measuring). The strengthened reconciled 005 + 006 in verify-scratch are candidate repairs, not part of this measurement.

Integrity cross-check: filtered gcovr JSON vs raw gcov --json-format --stdout --branch-probabilities --branch-counts — 1669 common line coordinates, 0 disagreements → PASS. integrity_errors: [].

Raw metrics:

metricvalue
changed executable lines32
covered changed lines29 (90.6%)
compiler-uncovered changed lines3
semantic uncovered changed lines3
structural uncovered changed lines0
changed lines without coverage coordinates46 (declarations, comments, braces, #include, GUC table rows — non-instrumentable)
changed branches10
covered changed branches6 (60.0%)
uncovered branches4

Invariants: covered(29) + compiler-uncovered(3) = changed executable(32); semantic(3) + structural(0) = compiler-uncovered(3); covered-branches(6) + uncovered-branch-records(4) = changed-branches(10).

Touched-function diagnostics (whole-function, diagnostics only):

functionfileenteredwhole-function lines (cov/total)whole-function branches (cov/total)
GetParallelWorkerRegisterCountbgworker.cyes (count 3)2/20/0
DetermineSleepTimepostmaster.cyes (count 13)8/325/22
ServerLooppostmaster.cyes (count 1)49/6623/36
CleanupBackendpostmaster.cyes (count 2)31/3915/22

Uncovered changed regions (semantic):

Uncovered changed branches (zero-count decision outcomes):

Excluded changed coordinates (bounded, non-actionable): 8 compiler-no-coverage-coordinate in bgworker.c (comments, signature, braces); 38 in postmaster.c (#include, file-scope/static declarations, local declarations, comments, braces); 4 compiler-branch-attribution at postmaster.c lines 1859/1875 (raw compiler CFG arcs without a matching source-level conditional decision). These 46 compiler-no-coverage-coordinate lines are non-instrumentable and do not produce COV-* findings.

Non-instrumented changes (separate): doc/src/sgml/config.sgml (docs), guc_parameters.dat (catalog metadata), postgresql.conf.sample (sample config), postmaster.h/bgworker_internals.h (declarations), meson.build (build file), 005_log_postmaster_stats.pl (the measured test itself).

Residual limitations: branch coverage from gcovr --decisions (experimental heuristic; raw gcov used for integrity cross-check only); measured test set is exactly {postgresql:postmaster/005_log_postmaster_stats}; percentages qualified to this set; DetermineSleepTime whole-function branch coverage low because bgworker-restart/io-worker branches not exercised by idle SELECT 1 (only changed lines in scope); single fresh-epoch invocation, no counter accumulation.

The normalized machine object is in review-artifact-v2.json (coverage), validated by validate_review_artifact.py (exit 0).

Thread ledger (pgsql-hackers)

Status: reconciled. Ledger report slug: commitfest-6804-review-thread-ledger. Single thread: all 8 messages share imported thread_id 646913 (subject "log_postmaster_stats"). Catversion exception applied (GUC-only development-branch patch; no catalog/OID/.bki change; no catversion/CATALOG_VERSION_NO impact).

#Date (UTC)Message-IDSenderKind
12026-05-27CAKZiRmxeTNAe_VZ9m=rt=5155mvL=CFsQ4ENJWhuUdgMe=_Udg@mail.gmail.comJakub Wartakpatch v1
22026-06-10164cb42a-52da-44b1-88d0-3f505ca5238b@yeah.netQuan Zongliangreview
32026-06-10CAKZiRmzBZo7Ycm_f6z1+PEewmfaby9Tu=Jhm0TOYSa+nzjZmmA@mail.gmail.comJakub Wartakauthor reply
42026-07-20CA+TgmoZLAr2Y8ZRpxCg5oQa8qGUh3pXTB2mMkM9EnDoUaSUV9Q@mail.gmail.comRobert Haascommitter reply
52026-07-23CAKZiRmxEz9zQKHdb3VNiZVh9PT2JDBo8sivBEY0giecSAaFxrA@mail.gmail.comJakub Wartakauthor reply
62026-07-27CAKZiRmy+qqox8XSk+cd+c7XmbGZHHpnAHOg3D-Ho-atuQRP4fg@mail.gmail.comJakub Wartakpatch v2/v3
72026-07-28CA+Tgmob7xpq3-ds536jo5vdenGJJ_rm5-okr7G-1F=5sU6HF9g@mail.gmail.comRobert Haascommitter reply
82026-07-30CAKZiRmx996hSedQ9BW6v2qj092tUoZZiUHWHyD9YRyhxT8gvXg@mail.gmail.comJakub Wartakpatch v3

Per-finding dispositions: PAT-1 already-raised (M6/M7/M8); PAT-2 committer-corroborated (M2/M3/M4); PAT-3 already-raised (M6); PAT-4 author-deferred (M1/M7); PAT-7/PAT-8/COV-1 new; SEC-1 new; SEC-4 already-raised (subsumed by PAT-2); SEC-2/SEC-5/SEC-6 new.

Thread issues not in the review seats: Robert Haas's cumulative-stats architecture objection (M4/M7) — a committer stated preference that this kind of metric belong in pgstats, not logs — should be answered on-list.

Aggregated Test Plan

ArtifactTypeStatusValidates
src/test/postmaster/t/005_log_postmaster_stats.pl (reconciled)TAPPASS on repaired (8 tests)all fields, disconnections, parallel workers, SIGHUP
src/test/postmaster/t/006_log_postmaster_stats_idle_skip.pl (new)TAP negativetwo-run gate (FAIL un-repaired / PASS repaired)PAT-2 idle-skip + quiet-period accumulation
src/test/postmaster/t/008_log_postmaster_stats_disabled.pl (new)TAPPASS (×2); full postmaster suite 6/6 OKCOV-1 disabled-GUC no-op + line 1656 coverage

Security and test-gap sections

Per-commit rollup / verified-clean / residual risk

Provenance

UI