[Bug]: BigQuery Copy jobs do not set write disposition to WRITE_APPEND after first copy

[Polber]

What happened?

There was a bug filed for a similar issue with BQ load jobs, #24535, that appears to still have an error path on Python SDK when copying the jobs from temporary tables to their final destination.

When using BigQueryIO Write with WRITE_TRUNCATE disposition, the first bundle is copied correctly, but the subsequent bundle does not set the disposition to WRITE_APPEND for itself and future bundles as intended. This leads to a "table exists" error thrown by BQ as the call to BQ tries to create a new table for the table already created by the first bundle.

Issue Priority

Priority: 1 (data loss / total loss of function)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner