[HUDI-5269] Enhancing spark-sql write tests for some of the core user flows

[jonvex]

Change Logs

We realized we don't have good test coverage for some of the core user flows w/ spark data source writes. So, enhancing the tests to cover the scenarios.

Tests coverage added w/ spark-data source writes:

COW and MOR * (w/ and w/o metadata)
    Partitioned(BLOOM, SIMPLE, GLOBAL_BLOOM), non-partitioned(GLOBAL_BLOOM).
        Immutable data. pure bulk_insert row writing. 
        Immutable w/ file sizing. pure inserts. 
        initial bulk ingest, followed by updates.

This pr is modeled after #7179
This uses a different sql schema that only has regular and struct fields because the the schema in the test above was too complicated to create in sql

Impact

Will help catch any bugs around core user flows upfront.

Risk level (write none, low medium or high below)

low

Documentation Update

N/A

Contributor's checklist

• Read through contributor's guide
• Change Logs and Impact were stated clearly
• Adequate tests were added if applicable
• CI passed

[nsivabalan]

Please create a jira and fill in PR description.

[hudi-bot]

CI report:

• 36b9c85 UNKNOWN
• ed4aa78 Azure: FAILURE

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build