sql/schemachanger: support ALTER COLUMN ... [SET | SET NOT] VISIBLE in declarative schema changer

[rafiss]

This tracks support for statements such as ALTER TABLE t ALTER COLUMN c SET VISIBLE in the declarative schema changer.

The plan below was created from a conversation with Claude Code, and reviewed by @rafiss.

To learn about key concepts in the schema changer implementation, first start by reading this guide to the declarative schema changer code

---

Implementation Plan: ALTER COLUMN SET [NOT] VISIBLE in Declarative Schema Changer

This feature will allow users to control column visibility using ALTER TABLE ... ALTER COLUMN ... SET [NOT] VISIBLE. The implementation follows the pattern established by PR #128892 (ColumnComputeExpression migration) and the ColumnNotNull element.

Summary: Create a new ColumnHidden element to replace the is_hidden boolean field in scpb.Column, then implement full DSC support for ALTER TABLE ... ALTER COLUMN ... SET [NOT] VISIBLE syntax. The parser support already exists via tree.AlterTableSetVisible. Support for this syntax in the declarative schema changer should be added so this only works when the CRDB cluster version is v26.1 or later.

Implementation Checklist

PR 1: Migrate is_hidden field to ColumnHidden element

This PR establishes mixed-version compatibility and must be merged first.

Files to modify:

• pkg/sql/schemachanger/scpb/elements.proto:

  • Add new ColumnHidden element with table_id, column_id fields
  • Add to ElementProto oneof with parent tag "parent:\"Column\""
  • Mark Column.is_hidden as [deprecated = true] with comment
  • Add in_26_1_or_later field to ElementCreationMetadata

• pkg/sql/schemachanger/scpb/migration.go:

  • Add migration logic in migrateDeprecatedFields() to convert is_hidden to ColumnHidden element
  • Pattern: Check if column.IsHidden == true, create new ColumnHidden target, set column.IsHidden = false
  • Match the pattern from compute expression migration (lines 61-77)

• pkg/sql/schemachanger/scpb/migration_test.go:

  • Add test cases for migrating deprecated is_hidden field

• pkg/sql/schemachanger/scplan/internal/opgen/opgen_column_hidden.go (NEW):

  • Implement state transitions: ABSENT → WRITE_ONLY → PUBLIC
  • Implement reverse: PUBLIC → WRITE_ONLY → ABSENT
  • No validation needed (unlike ColumnNotNull)
  • Pattern similar to opgen_column_not_null.go but simpler

• pkg/sql/schemachanger/scop/immediate_mutation.go:

  • Add operations: MakeColumnHidden, MakeColumnNotHidden

• pkg/sql/schemachanger/scexec/scmutationexec/column.go:

  • Implement mutation executors for new scop operations
  • Set/unset Hidden field in column descriptor

• pkg/sql/schemachanger/scbuild/internal/scbuildstmt/alter_table_add_column.go:

  • When adding column with IsHidden, create ColumnHidden element (if version >= 26.1)
  • Preserve old behavior for older versions using ElementCreationMetadata

• pkg/sql/schemachanger/scbuild/internal/scbuildstmt/alter_table_drop_column.go:

  • Update walkColumnDependencies at line 324-329 to include *scpb.ColumnHidden in the list

• pkg/sql/schemachanger/scbuild/internal/scbuildstmt/alter_table_alter_column_set_visible.go:

  • Add the handler

• pkg/sql/schemachanger/scplan/internal/rules/current/helpers.go:

  • Update isColumnDependent() to include *scpb.ColumnHidden

• pkg/sql/schemachanger/scdecomp/decomp.go:

  • Add logic to decompose existing hidden columns into ColumnHidden elements

• pkg/sql/schemachanger/screl/attr.go:

  • Check if any new attributes are needed (likely not - ColumnHidden only uses TableID and ColumnID)
  • Need to register in elementSchemaOptions

• Run ./dev generate to update generated files

Testing:

• Add tests for migration from old to new element representation
• Add tests for mixed-version behavior
• Update existing scbuild/scplan testdata files

---

PR 2: Implement ALTER COLUMN SET VISIBLE support

Combines scbuild support, dependency rules, and end-to-end tests.

Files to modify:

• pkg/sql/schemachanger/scbuild/internal/scbuildstmt/alter_table_set_visible.go (NEW):

  func alterTableSetVisible(
      b BuildCtx, tn *tree.TableName, tbl *scpb.Table,
      stmt tree.Statement, t *tree.AlterTableSetVisible,
  ) {
      alterColumnPreChecks(b, tn, tbl, t.Column)
      columnID := getColumnIDFromColumnName(b, tbl.TableID, t.Column, true)
      panicIfSystemColumn(mustRetrieveColumnElem(b, tbl.TableID, columnID), t.Column)
  
      if t.Visible {
          // SET VISIBLE: Drop ColumnHidden element if it exists
      } else {
          // SET NOT VISIBLE: Add ColumnHidden element
      }
  }

• pkg/sql/schemachanger/scbuild/internal/scbuildstmt/alter_table.go:

  • Add entry to supportedAlterTableStatements map:

    reflect.TypeOf((*tree.AlterTableSetVisible)(nil)): {
        fn: alterTableSetVisible, on: true, checks: isV261Active,
    }

• pkg/sql/schemachanger/scbuild/internal/scbuildstmt/helpers.go:

  • It may be useful to add helper functions for hidden columns here, but only if you find that they are non-trivial and are needed in multiple places.

• The generic column dependent rules should already cover the new element, but if testing reveals an issue, new rules may be added in pkg/sql/schemachanger/scplan/internal/rules/current/

Testing:

• pkg/sql/schemachanger/scbuild/testdata/alter_table_set_visible (NEW):

  • Test setting column visible/not visible

• pkg/sql/schemachanger/testdata/end_to_end/alter_table_set_visible (NEW):

  • Test setting column visible/not visible
  • Test rollback scenarios
  • Test with various column types

• Verify existing logic tests pass:

  • Run tests in pkg/sql/logictest/testdata/logic_test/alter_table
  • These should cover visibility behavior (SELECT * excludes hidden columns, explicit access still works)

---

Key Implementation Notes

1. Element naming: ColumnHidden (presence = hidden, absence = visible)

2. Mixed-version compatibility:

   • Use ElementCreationMetadata.in_26_1_or_later
   • Older versions continue using Column.is_hidden field
   • Migration logic converts old field to new element on upgrade

3. State transitions:

   • Adding hidden: ABSENT → WRITE_ONLY → PUBLIC
   • Removing hidden: PUBLIC → WRITE_ONLY → ABSENT
   • No validation phase needed (unlike NOT NULL constraint)

4. Scope:

   • Only handles is_hidden field (user-controlled visibility)
   • Does NOT handle is_inaccessible (system columns, expression indexes)

5. Behavior:

   • SET NOT VISIBLE: Creates ColumnHidden element (sets is_hidden = true)
   • SET VISIBLE: Drops ColumnHidden element (sets is_hidden = false)
   • Hidden columns excluded from SELECT * but can be explicitly referenced

6. Dependency handling:

   • Update isColumnDependent() helper in all rules versions (current, release_25_2, release_25_3)
   • Update walkColumnDependencies() to include ColumnHidden
   • Existing generic column dependency rules should handle ColumnHidden automatically

The first PR is critical for establishing the migration foundation.

Jira issue: CRDB-46731

Jira epic: CRDB-31283