GCP: Support write-time object context annotations in GCS FileIO (equivalent of S3 write.tags)

[revanthgss]

Feature Request / Improvement

Background

The AWS module supports attaching arbitrary key-value tags to S3 objects at write time via
s3.write.tags.<key>=<value> catalog properties (see S3FileIOProperties and S3OutputStream).
These tags are embedded in the upload request with no extra API call and can drive S3 lifecycle
policies.

GCS has a direct equivalent called object contexts — key-value metadata embedded
during the object write. The GCS Java SDK supports this via
BlobInfo.Builder.setContexts(ObjectContexts) since
commit 485aefd
(google-cloud-storage ≥ 2.57).

Proposed change

Add gcs.write.object-context.<key>=<value> catalog property support, mirroring the S3 design. This would be useful to add additional metadata like environment, team and table info to the catalog files that are created in GCS.

• GCPProperties — new constant GCS_WRITE_OBJECT_CONTEXT_PREFIX = "gcs.write.object-context.";
  constructor collects all matching properties into an immutable Map<String, String> writeObjectContexts;
  new accessor writeObjectContexts()
• GCSOutputStream — new private buildBlobInfo(BlobId, GCPProperties) that calls
  builder.setContexts(ObjectContexts) when contexts are configured before passing BlobInfo to
  storage.writer(). No changes to the stream mechanics.
• No changes to GCSFileIO, GCSOutputFile, or any module outside gcp/

Example usage

spark.sql.catalog.my_catalog.gcs.write.object-context.env=prod
spark.sql.catalog.my_catalog.gcs.write.object-context.team=data-platform
spark.sql.catalog.my_catalog.gcs.write.object-context.iceberg-table=orders

Related

• S3 write-tags: AWS: Add S3 Tagging #4259
• GCS object contexts SDK commit: [googleapis/java-storage@485aefd](googleapis/java-storage@485aefd)
• GCS object context docs: https://cloud.google.com/storage/docs/use-object-contexts

AI Disclosure

LLM is used to research the details of S3 write tags and draft the issue description.

Query engine

None

Willingness to contribute

• I can contribute this improvement/feature independently
• I would be willing to contribute this improvement/feature with guidance from the Iceberg community
• I cannot contribute this improvement/feature at this time