CommitFest 6121 Review - ATPostAlterTypeCleanup double-call

report · generated from canonical Markdown

Type
report
Source
postgresql-master
Slug
commitfest-6121-review
Tags
alter-tablecatalog-consistencycommitfest-6121generated-columnspatch-reviewpostgresql
Updated
2026-08-17T18:32:55.658669+00:00
Verdict / Summarywaiting-on-author
Front matter
schema: review-report-v2
source: postgresql-master
slug: commitfest-6121-review
target: CommitFest patch 6121 - avoid calling ATPostAlterTypeCleanup twice for the same relation
head: 0499c6ccf5ce510ad8c57211ee8c9414e92c71b7
base: 0499c6ccf5ce510ad8c57211ee8c9414e92c71b7
diff_content_sha256: ae00941b5935f9324f8644efc582a05792e57e1114bc23a3d8458d1ca39f4d51
diff_blob_sha: ca728cfe0d12aabcc0854dbeee544ee30af0c2d7
verdict: waiting-on-author
blocker_counts:
  verified: 1
  unverified: 0
reviewer_model: glm-5.2:cloud
scope_identity: cf-6121/head-0499c6ccf5c/diff-ca728cfe
thread_id: "623416"

CommitFest 6121 Review — ATPostAlterTypeCleanup double-call on generated columns

Patch summary

The staged diff modifies 5 files (+16/-1):

FileChange
src/backend/commands/tablecmds.cIn ATRewriteCatalogs(), add List *relids = NIL and guard ATPostAlterTypeCleanup(wqueue, tab, lockmode) with if (!list_member_oid(relids, tab->relid)) so it runs only once per relation across the AT_PASS_ALTER_TYPE and AT_PASS_SET_EXPRESSION passes.
src/test/regress/sql/generated_stored.sqlAdd ALTER TABLE gtest25 ADD CONSTRAINT cc CHECK(b > 9) NOT VALID; then ALTER TABLE gtest25 ALTER COLUMN b SET DATA TYPE INT8, ALTER COLUMN b SET EXPRESSION AS (a * 3); --ok.
src/test/regress/sql/generated_virtual.sqlSame two statements.
src/test/regress/expected/generated_stored.outEcho the two new statements.
src/test/regress/expected/generated_virtual.outSame.

The original bug (pgsql-hackers thread 623416, Jian He, 2025-09-27): a single ALTER TABLE combining ALTER COLUMN ... SET DATA TYPE and ALTER COLUMN ... SET EXPRESSION on the same generated column errors with cache lookup failed for constraint because ATRewriteCatalogs calls ATPostAlterTypeCleanup twice for the same relation and the second call re-processes already-dropped constraint OIDs.

Verdict

waiting-on-author — one verified BLOCKER.

The worktree contains the v1 dedup-by-relid approach, which David Rowley rejected on 2025-09-28 and the author replaced with v6 (reset the changed_* lists to NIL at the end of ATPostAlterTypeCleanup) on 2026-08-02. The v6 approach was confirmed correct by Evan Li on 2026-08-03. The worktree must be updated to v6 before this patch can proceed.

Findings

SEC-1

— BLOCKER — correctness — src/backend/commands/tablecmds.c:5457

Mechanism: The dedup-by-relid guard skips ATPostAlterTypeCleanup for the AT_PASS_SET_EXPRESSION pass when AT_PASS_ALTER_TYPE already ran cleanup for the same relation. This leaves constraints/indexes registered *only* by RememberWholeRowDependentForRebuilding (and per-column dependents of the expression-changed column) un-rebuilt.

Why this is wrong: ATPostAlterTypeCleanup processes tab->changedConstraintOids, changedIndexOids, changedStatisticsOids, replicaIdentityIndex, and clusterOnIndex. These lists are *accumulated across passes*:

  1. During AT_PASS_ALTER_TYPE, ATExecAlterColumnType registers

constraints/indexes dependent on the type-changed column.

  1. ATPostAlterTypeCleanup runs, drops those objects via

performMultipleDeletions, and queues re-add commands. Under v1 the relid is recorded.

  1. During AT_PASS_SET_EXPRESSION, ATExecSetExpression calls

RememberAllDependentForRebuilding and RememberWholeRowDependentForRebuilding (tablecmds.c:8836/:8843), registering *additional* constraints/indexes dependent on the expression-changed column — including whole-row CHECK constraints (pg_depend refobjsubid = 0) that the per-column pg_depend scan in step 1 cannot find (RememberWholeRowDependentForRebuilding has Assert(subtype == AT_SetExpression) and is called *only* from ATExecSetExpression).

  1. Under v1, ATPostAlterTypeCleanup is skipped because relid is already

in relids. The newly-registered objects from step 3 are never dropped, never reparsed, and never queued for re-addition.

Empirical reproduction (coordinator-executed, disposable build):

Cross-column whole-row counter-example against the v1 worktree patch:

sql
CREATE TABLE gtest_blocker3 (a int, z int, b int GENERATED ALWAYS AS (a*2) STORED, c text);
ALTER TABLE gtest_blocker3 ADD CONSTRAINT cc_col CHECK (z > 0);       -- per-column dep on z
ALTER TABLE gtest_blocker3 ADD CONSTRAINT cc_wr CHECK (gtest_blocker3 IS NOT NULL); -- whole-row (refobjsubid=0)
ALTER TABLE gtest_blocker3 ALTER COLUMN z SET DATA TYPE bigint, ALTER COLUMN b SET EXPRESSION AS (a*3);

Under v1: cc_col OID changed 17919 → 17922 (rebuilt by ALTER_TYPE pass), but cc_wr OID stayed 17920 → 17920 (NOT rebuilt — the SET_EXPRESSION cleanup was skipped). pg_depend for cc_wr has refobjsubid = 0, so only RememberWholeRowDependentForRebuilding (SET_EXPRESSION pass) registers it.

David Rowley's per-column counter-example (thread 623416, 2025-09-28) shows the same defect for a non-whole-row constraint on the generated column when only a *different* column's type is altered: cc check (b > 0) is not recreated.

Security relevance: CHECK constraints may silently fail to be rebuilt against the new column type/expression; replica identity and CLUSTER index markings may be lost, affecting logical replication correctness. Privilege neutral (any table owner can trigger it); the impact is catalog consistency and constraint enforcement. CWE-863 (incorrect enforcement) / catalog inconsistency.

Verification status: verified (by reproduction + pgsql-hackers thread corroboration).

Repair (v6): Revert the dedup-by-relid guard and instead reset the changed_* lists to NIL/NULL at the end of ATPostAlterTypeCleanup, after performMultipleDeletions + free_object_addresses. This allows the second cleanup call to process only objects registered by the SET_EXPRESSION pass.

Suggested code (exact postimage, overlaps the v6 repair patch):

c
tab->changedConstraintOids = NIL;
tab->changedConstraintDefs = NIL;
tab->changedIndexOids = NIL;
tab->changedIndexDefs = NIL;
tab->changedStatisticsOids = NIL;
tab->changedStatisticsDefs = NIL;
tab->changedStatisticsOwners = NIL;
tab->replicaIdentityIndex = NULL;
tab->clusterOnIndex = NULL;

Concrete HEAD-bound unified diff (sha256 08169af996d9855a984307f2d15235d7c6b4a7a7a418b1ff9326fd50b9f08038):

diff
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -16342,8 +16342,22 @@
 
 	/*
 	 * The objects will get recreated during subsequent passes over the work
-	 * queue.
+	 * queue.  Reset the changed-object lists so that a subsequent
+	 * ATPostAlterTypeCleanup call for this same tab (which happens when one
+	 * ALTER TABLE combines ALTER COLUMN TYPE and ALTER COLUMN SET EXPRESSION)
+	 * only processes objects registered by later passes, instead of
+	 * re-processing the already-dropped objects and ERRORing on a stale
+	 * SearchSysCache1(CONSTROID) lookup.
 	 */
+	tab->changedConstraintOids = NIL;
+	tab->changedConstraintDefs = NIL;
+	tab->changedIndexOids = NIL;
+	tab->changedIndexDefs = NIL;
+	tab->changedStatisticsOids = NIL;
+	tab->changedStatisticsDefs = NIL;
+	tab->changedStatisticsOwners = NIL;
+	tab->replicaIdentityIndex = NULL;
+	tab->clusterOnIndex = NULL;
 }
TEST-1

— MAJOR — tests — src/test/regress/sql/generated_stored.sql:601

The added regression test only exercises the same-column case (ALTER TYPE + SET EXPRESSION on column b), which does not trigger SEC-1. It does not cover the cross-column case (David Rowley's counter-example) and performs no post-condition verification (\d or pg_constraint SELECT) that the constraint survived and was rebuilt. A regression that silently drops cc would pass the current test.

Verification status: cannot-verify (the test-adequacy gap is confirmed by inspection, but the concrete expected/*.out repair must be captured by running pg_regress in the author environment; hand-authoring expected output is not permitted).

Suggested test additions (the .out deltas must be generated by pg_regress):

sql
-- cross-column counter-example (sec-1)
CREATE TABLE gtest25_dr (a0 int, z int, a int, b int GENERATED ALWAYS AS (a * 2 + a0) STORED);
ALTER TABLE gtest25_dr ADD CONSTRAINT check_z CHECK (z > 0);
ALTER TABLE gtest25_dr ADD CONSTRAINT cc_dr CHECK (b > 0);
ALTER TABLE gtest25_dr ALTER COLUMN z SET DATA TYPE numeric, ALTER COLUMN b SET EXPRESSION AS (z + 0);
\d gtest25_dr
SELECT conname, pg_get_constraintdef(oid) FROM pg_constraint WHERE conrelid = 'gtest25_dr'::regclass ORDER BY conname;
DOC-1

— MINOR — maintainability — src/backend/commands/tablecmds.c:5455

The block comment above the cleanup call was not updated. It still says "done only once if multiple columns of a table are altered", which is misleading under the v6 repair (cleanup runs once per pass, and the list reset makes the second call safe). Verification status: cannot-verify (comment-only; no runtime validation).

Verification Summary

FindingSeverityClassVerification statusMethod
SEC-1BLOCKERcorrectnessverifiedrepro (disposable build + cross-column counter-example) + pgsql-hackers thread 623416 corroboration
TEST-1MAJORtestscannot-verifystatic-check (inspection of expected/*.out)
DOC-1MINORmaintainabilitycannot-verifystatic-check (inspection of comment)

reasons (expected output must be captured; comment is prose), not for finding-existence doubt. The findings themselves are confirmed by inspection.

Repair Validation Summary

FindingRepairValidation statusEvidence
SEC-1v6: reset changed_* lists to NIL/NULL at end of ATPostAlterTypeCleanup; revert dedup-by-relid guardpassedApplied v6 to disposable build, rebuilt backend, re-ran (1) generated_stored+generated_virtual regression tests (All 2 tests passed); (2) David Rowley cross-column counter-example: cc 17901→17904 and check_z 17900→17903 both recreated; (3) whole-row counter-example: cc_wr 16391→16394 and cc_col 16390→16393 both recreated. Methods: apply-check, diff-check, compile, targeted-test, repro.
TEST-1Add cross-column counter-example + \d/SELECT verification; regenerate .outwaivedExpected output must be captured by pg_regress in author environment; coordinator did not modify the read-only worktree's expected output.
DOC-1Update comment to reflect per-pass cleanup + list resetwaivedComment-only change; committer amends wording during commit.

A successful repair does not prove a finding, and a failed repair does not refute it. The SEC-1 finding is verified independently by reproduction and thread corroboration; the v6 repair validation confirms the proposed fix resolves the reproduced defect.

Changed-Code Coverage

Coverage requirement: required — changed instrumentable implementation in src/backend/commands/tablecmds.c (ATRewriteCatalogs).

Execution lane: host-disposable (passed). Fresh disposable PostgreSQL build under the run scratch directory; tablecmds.c compiled with -fprofile-arcs -ftest-coverage, backend relinked with -fprofile-arcs; fresh initdb cluster on port 55644; pg_regress ran generated_stored and generated_virtual (ok 1, ok 2; All 2 tests passed); gcov -b produced per-line and per-branch counts.

Counter epoch: fresh. Integrity errors: none (empty list).

Test commands:

make -j4 -C src-patched (CFLAGS='-g -O0') then recompile tablecmds.c with -fprofile-arcs -ftest-coverage and relink backend with -fprofile-arcs
initdb -D pgdata (fresh disposable cluster)
pg_ctl -D pgdata -o '-p 55644' start
pg_regress --port=55644 --user=ubuntu --dbname=regression generated_stored generated_virtual
gcov -b tablecmds.c -o src-patched/src/backend/commands

Raw metrics (changed executable lines/branches in ATRewriteCatalogs):

MetricCount
changed executable lines4
covered changed lines4
compiler-uncovered changed lines0
uncovered changed lines0
structural uncovered changed lines0
changed lines without coverage coordinates0
changed branches2
covered changed branches2

Covered changed lines (gcov counts):

LineTextgcov count
5417List *relids = NIL;276 (function entered)
5457if (!list_member_oid(relids, tab->relid))59 evaluations — branch true (enter block) 57, branch false (skip) 2
5459ATPostAlterTypeCleanup(wqueue, tab, lockmode);57
5460relids = lappend_oid(relids, tab->relid);57

Both branches of the new dedup guard (line 5457) are exercised: the true branch (relid not yet recorded → run cleanup) is taken 57 times, and the false branch (relid already recorded → skip) is taken 2 times (the dedup skip that triggers SEC-1).

Uncovered regions: none. Uncovered branches: none. Excluded changed coordinates: none.

Touched functions:

PathFunctionSpanEnteredLines covered/totalBranches covered/total
src/backend/commands/tablecmds.cATRewriteCatalogs5412–5488yes4/42/2

Non-instrumented changes: src/test/regress/expected/generated_stored.out, src/test/regress/expected/generated_virtual.out, src/test/regress/sql/generated_stored.sql, src/test/regress/sql/generated_virtual.sql — test/expected files, not C implementation, not gcov-instrumentable.

Residual limitation: coverage was measured only for src/backend/commands/tablecmds.c (the sole changed implementation file); the regression suite exercised the same-column dedup path but not the cross-column path that exposes SEC-1 (see TEST-1).

pgsql-hackers thread ledger

Thread: 623416 — "bug, ALTER TABLE call ATPostAlterTypeCleanup twice for the same relation" (reconciled).

DateSenderKindSummary
2025-09-27Jian HepatchOriginal bug report + v1 dedup-by-relid patch.
2025-09-28David RowleyreviewRejects v1; cross-column counter-example shows cc not recreated.
2026-08-02Jian Hepatchv6: reset changed_* lists to NULL after performMultipleDeletions.
2026-08-03Evan LireviewConfirms v6 is correct.

Finding-link dispositions: SEC-1 → already-raised (Rowley 2025-09-28, Li 2026-08-03); TEST-1 → new; DOC-1 → new. Ledger report slug: commitfest-6121-review-handoff-security-reviewer.

Provenance

commitfest-6121-review-handoff-patch-reviewer (source postgresql-master); markdown_sha256 bb8f85d54643df384e119088feb4775c4effb5c81d5cc709995d9f1ec1ccecc6.

commitfest-6121-review-handoff-security-reviewer (source postgresql-master); read back and consumed via get_report.

gcov epoch); pg-change-coverage-reviewer analysis folded into the coordinator artifact.

Maintainer CC

Residual risk

The v1 worktree patch fixes the originally-reported same-column ERROR but introduces (or, for the dedup-by-relid approach, retains) the cross-column catalog-consistency defect. Adopting the v6 repair (reset lists) resolves both. The test-adequacy gap (TEST-1) means the regression suite would not catch a future regression of the cross-column path; the author should add the cross-column counter-example with post-condition verification.