perf: use dataset-level scan for indexed vector search to avoid per-fragment redundancy

[shmilygkd]

Problem

Lance's IVF index is a global structure: centroids are computed over the full dataset and cannot be restricted to individual fragment boundaries. As a result, each `Fragment.newScan()` with a nearest query executes a complete global index search rather than a fragment-local one. With N fragments this has two consequences:

1. Incorrect recall: Each Spark partition retrieves globally-optimal candidates. Unioning N such result sets inflates recall relative to a single global search at the same `nprobes`, producing results inconsistent with a direct pylance query.
2. N-fold overhead: Each task independently loads the same global index metadata and `nprobes` partition files. Index I/O and compute scale as N×.

Fix

Introduce `LanceSplit.isIndexedVectorSearch()` to distinguish indexed vector search (`nearest` with `useIndex=true`) from brute-force KNN (`useIndex=false`).

For indexed search, all fragments are merged into a single `LanceSplit` and `Dataset.newScan()` is used instead of `Fragment.newScan()`, executing one global index search on a single executor. Recall is now consistent with pylance.

For brute-force KNN (`useIndex=false`), per-fragment splits are preserved so each partition physically scans its own fragment in parallel.

Additional fixes:

• Guard against empty datasets in the indexed path
• Fix SPJ partition key computation: previously skipped for all `nearest` queries; now skipped only for indexed search, preserving valid partition keys for brute-force KNN

Testing

Added `LanceSplitVectorSearchTest` with integration tests verifying that `useIndex=true` produces exactly one split and `useIndex=false` produces one split per fragment.