Repositories: Batch WHERE IN queries to avoid SQL Server 2100-parameter limit

[AndyButland]

Description

We've had a few issues over the years with WHERE IN queries exceeding the SQL Server parameter limit.

This PR is a preventive update of a few other cases I found where we haven't had issues reported, but depending on configuration and customer data, could in theory trip up on this in future.

Also included are CLAUDE.md updates in an attempt that this oversight in future is caught in development or review.

The updates are:

• DatabaseCacheRepository.GetContentSourcesAsync / GetMediaSourcesAsync and the five private rebuild ForNodes helpers now batch their WHERE IN queries with InGroupsOf(Constants.Sql.MaxParameterCount). These were driven by CacheSettings.DocumentSeedBatchSize, CacheSettings.MediaSeedBatchSize and NuCacheSettings.SqlPageSize — defaults are safe but the settings are documented as tunable, and raising any of them past 2000 would break seeding/rebuilds with this exception.
• RedirectUrlRepository.PerformGetAll no longer throws when given more than MaxParameterCount ids. It now batches the fetch, and treats null/empty ids as "fetch all" (matching the convention used by other repositories like DomainRepository.PerformGetAll).

Testing

Additional integration tests have been added to ensure there's no regression of the expected and happy path of a small number of IDs for which entities are retrieved.

[claude]

Claude finished @AndyButland's task in 6m 6s —— View job

---

PR Review

Target: origin/v17/dev · Based on commit: c4ece856

Adds batching to WHERE IN queries across DatabaseCacheRepository and RedirectUrlRepository to prevent SqlException 8003 on SQL Server when configurable batch sizes exceed 2000 IDs, and fixes RedirectUrlRepository.PerformGetAll to return all rows on empty/null input rather than producing invalid SQL.

• Other changes: RedirectUrlRepository.PerformGetAll behavior change — passing null or an empty array now performs a full-table fetch (previously produced an unbounded WHERE IN (NULL/empty), undefined/broken behavior). This aligns with the FullDataSetRepositoryCachePolicy contract and the pattern used by DomainRepository.

---

Suggestions

• src/Umbraco.PublishedCache.HybridCache/Persistence/DatabaseCacheRepository.cs:221: The ORDER BY (level, parentId, sortOrder) is now per-batch rather than global — results within each 2000-item batch are sorted, but the accumulated list across batches is not. Benign for the current seeding caller (which just iterates and stores), but worth noting if a future caller expects globally level-sorted output.

• tests/Umbraco.Tests.Integration/Umbraco.Infrastructure/Persistence/Repositories/RedirectUrlRepositoryTests.cs:291: New test method names (GetMany_With_Ids_*) differ from the project-wide Can_/Cannot_ prefix convention used in the surrounding tests. Same applies to the new DatabaseCacheRepositoryTests file.

---

Approved with Suggestions for improvement

Good defensive fix that closes a real production footgun across several high-traffic rebuild/seeding paths. The CLAUDE.md documentation and the InGroupsOf pattern codified here will help future contributors spot unbounded WHERE IN risks early. Suggestions above are non-blocking.

[Copilot]

Pull request overview

This PR hardens repository queries against SQL Server’s 2100-parameter limit by batching WHERE IN clauses using InGroupsOf(Constants.Sql.MaxParameterCount), preventing runtime SqlException 8003 when batch-size settings or datasets grow beyond safe defaults.

Changes:

• Batch WHERE IN queries in DatabaseCacheRepository (content/media source reads + rebuild “ForNodes” helpers) to stay under Constants.Sql.MaxParameterCount.
• Update RedirectUrlRepository.PerformGetAll to (a) treat null/empty ids as “fetch all” and (b) batch id lookups instead of throwing above the parameter ceiling.
• Add/extend integration tests covering the common (“small N”) paths for both repositories, and document safe batching patterns in the repo CLAUDE guides.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
tests/Umbraco.Tests.Integration/Umbraco.PublishedCache.HybridCache/DatabaseCacheRepositoryTests.cs	Adds integration coverage for hybrid-cache source lookups (keys + empty keys) and rebuild pipeline path.
tests/Umbraco.Tests.Integration/Umbraco.Infrastructure/Persistence/Repositories/RedirectUrlRepositoryTests.cs	Adds tests for GetMany with ids and for the “no ids => fetch all” convention.
src/Umbraco.PublishedCache.HybridCache/Persistence/DatabaseCacheRepository.cs	Implements batching for GUID/id WHERE IN queries to avoid exceeding SQL Server parameter limits.
src/Umbraco.Infrastructure/Persistence/Repositories/Implement/RedirectUrlRepository.cs	Implements batched PerformGetAll and aligns empty/null ids behavior with “fetch all” expectations.
src/Umbraco.Infrastructure/CLAUDE.md	Documents decision rules and safe batching patterns for the 2100-parameter limit.
src/Umbraco.Core/CLAUDE.md	Adds cross-reference to Constants.Sql.MaxParameterCount and batching helpers.
CLAUDE.md	Adds top-level guidance pointing contributors to the detailed batching section.

[Zeegaan]

Probably a good preventative measure 😁