Spark <> Iceberg bug integration test

[kevinjqliu]

Apache Iceberg version

None

Please describe the bug 🐞

While working on #444, I ran into a weird bug with Spark integration test.

Particularly here
https://github.com/apache/iceberg-python/compare/main...kevinjqliu:iceberg-python:kevinjqliu/weird-spark-bug?expand=1#diff-ae89704e133e5eb800112d7a84557f2976819b2c5d989a62af97bf922865631bR459

The current snapshot contains 10 data files, as verified in the assert statement just above.

assert tbl.current_snapshot().summary['added-data-files'] == '10'

But the Spark metadata table still returns with 1 file.

The issue goes away completely if I use a new table identifier in L452.