Add Wolfgang.Etl.ErrorPolicies package (lockstep 0.21.0)

[Chris-Wolfgang]

Adds a second package built from this repo, Wolfgang.Etl.ErrorPolicies, versioned in lockstep with Wolfgang.Etl.Abstractions (same <Version>, one tag — the TestKit + TestKit.Xunit pattern). It hosts a shared generic ItemErrorPolicy factory for the family's OnError hook, so the ready-made error policies are defined once instead of duplicated per format library.

Why a separate package (decision: "C′")

The policies are format-agnostic and pair with the #84 ItemError* types already in Abstractions. But two of them need Microsoft.Extensions.Logging.Abstractions (SkipAndLog) and System.Threading.Channels (the channel dead-letter overloads). Rather than push those deps onto the deliberately-lean core (every Abstractions consumer would inherit them), they live only in this package. The core assembly's dependency set is unchanged; the six format leaves reference nothing new (OnError is Func<ItemErrorContext, ItemErrorAction>? — pure Abstractions types); end users opt into ErrorPolicies for the ready-mades or write their own lambda.

Surface (ItemErrorPolicy)

• Skip / Abort — Func<ItemErrorContext, ItemErrorAction> properties
• SkipAndLog(ILogger)
• SkipAndDeadLetter(ICollection<ItemErrorContext>) / SkipAndDeadLetter(ChannelWriter<ItemErrorContext>)
• SkipDeadLetterAndLog(ICollection<…>, ILogger) / SkipDeadLetterAndLog(ChannelWriter<…>, ILogger)

Dead-letter sinks are caller-owned (bounded by the caller); the channel overload is the thread-safe concurrent-producer path (TryWrite). The ICollection overloads add without locking — documented as safe for a single serial stage.

Dependencies (this package only)

• Wolfgang.Etl.Abstractions 0.21.0 (lockstep, all TFMs)
• Microsoft.Extensions.Logging.Abstractions 10.0.10 (all TFMs)
• System.Threading.Channels 10.0.10 (net462/net472/net48/net481/netstandard2.0 only — in-box net5.0+)

Release mechanics

• Bumps Abstractions <Version> 0.20.0 → 0.21.0 (lockstep); PackageValidation baseline stays at last-published 0.20.0. New package has no baseline on its first release (add next cycle).
• No release.yaml change needed: it already packs every src/ project, validates the tag against any src <Version>, and pushes all *.nupkg.

Verification (local, full matrix net462 → net10.0)

• Package + tests build clean, 0 warnings / 0 errors (strict analyzers + RS0016 PublicAPI)
• 14 unit tests pass on net462 (Channels package path), net8.0, net10.0
• dotnet pack emits correct per-TFM dependency groups (verified Abstractions 0.21.0 lockstep dep on every TFM; Channels only on netFx/ns2.0)

Release routing is yours — this bumps to 0.21.0 for a lockstep release; route via vNext/tag per the per-repo-release-pilot flow (standalone, or bundled with other 0.21.0 work).

🤖 Generated with Claude Code

[github-actions]

BenchmarkDotNet

Details

Benchmark suite	Current: 3d506c0	Previous: 5e5e958	Ratio
Wolfgang.Etl.Abstractions.Benchmarks.ExtractorBenchmarks.Extract_NoProgress(RecordCount: 1000)	31393.145802815754 ns (± 197.45291732008621)	31583.873189290363 ns (± 115.28689686427228)	0.99
Wolfgang.Etl.Abstractions.Benchmarks.ExtractorBenchmarks.Extract_WithProgress(RecordCount: 1000)	34512.90861002604 ns (± 247.65949195033753)	35160.96792602539 ns (± 104.53835631316898)	0.98
Wolfgang.Etl.Abstractions.Benchmarks.ExtractorBenchmarks.Extract_NoProgress(RecordCount: 100000)	3066479.3606770835 ns (± 12488.839573303052)	3107534.1640625 ns (± 17584.772051305485)	0.99
Wolfgang.Etl.Abstractions.Benchmarks.ExtractorBenchmarks.Extract_WithProgress(RecordCount: 100000)	3367265.0989583335 ns (± 2229.19415989582)	3448047.61328125 ns (± 4145.027564723044)	0.98
Wolfgang.Etl.Abstractions.Benchmarks.PipelineBenchmarks.FluentPipeline(RecordCount: 1000)	29232.722712198894 ns (± 80.0839446999554)	29626.396423339844 ns (± 122.33354576748134)	0.99
Wolfgang.Etl.Abstractions.Benchmarks.PipelineBenchmarks.ManualComposition(RecordCount: 1000)	28957.610438028973 ns (± 22.5270248379915)	28745.347229003906 ns (± 56.763752733836895)	1.01
Wolfgang.Etl.Abstractions.Benchmarks.PipelineBenchmarks.BaseClassComposition(RecordCount: 1000)	67740.15051269531 ns (± 250.4486726511416)	69584.28100585938 ns (± 220.36328762445214)	0.97
Wolfgang.Etl.Abstractions.Benchmarks.PipelineBenchmarks.FluentPipeline(RecordCount: 100000)	2894727.7265625 ns (± 1317.622871382007)	2835179.5182291665 ns (± 5152.038095929195)	1.02
Wolfgang.Etl.Abstractions.Benchmarks.PipelineBenchmarks.ManualComposition(RecordCount: 100000)	2913623.6536458335 ns (± 716.0158873807541)	2925295.2018229165 ns (± 3396.484964473814)	1.00
Wolfgang.Etl.Abstractions.Benchmarks.PipelineBenchmarks.BaseClassComposition(RecordCount: 100000)	6695888.294270833 ns (± 7163.478585605702)	6793926.755208333 ns (± 24027.364375768884)	0.99

This comment was automatically generated by workflow using github-action-benchmark.

[Copilot]

🟡 Not ready to approve

The new package/docs repeatedly describe an "OnError" hook, but the Abstractions API names the policy hook OnItemError, so the current wording is misleading and should be corrected for accuracy.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Pull request overview

This PR introduces a new lockstep-versioned package, Wolfgang.Etl.ErrorPolicies, to centralize reusable item-error policy delegates (skip/abort/log/dead-letter) so downstream format-specific ETL libraries don’t duplicate the same On*Error policy implementations. It also bumps Wolfgang.Etl.Abstractions to 0.21.0 to keep both packages aligned for release.

Changes:

• Added new Wolfgang.Etl.ErrorPolicies package (multi-TFM) with ItemErrorPolicy + cached LoggerMessage delegates.
• Added a new unit test project and coverage for the policy factory methods across TFMs.
• Bumped Wolfgang.Etl.Abstractions version to 0.21.0 and updated solution + changelog for the new package.

File summaries

File	Description
tests/Wolfgang.Etl.ErrorPolicies.Tests.Unit/Wolfgang.Etl.ErrorPolicies.Tests.Unit.csproj	Adds new multi-targeted unit test project for the ErrorPolicies package.
tests/Wolfgang.Etl.ErrorPolicies.Tests.Unit/ItemErrorPolicyTests.cs	Adds tests for Skip/Abort/Log/DeadLetter policy behaviors and null-guarding.
src/Wolfgang.Etl.ErrorPolicies/Wolfgang.Etl.ErrorPolicies.csproj	Introduces the new packable library project and its package dependencies/metadata.
src/Wolfgang.Etl.ErrorPolicies/PublicAPI.Unshipped.txt	Declares the initial public API surface for the new package.
src/Wolfgang.Etl.ErrorPolicies/PublicAPI.Shipped.txt	Establishes shipped API baseline header for first release.
src/Wolfgang.Etl.ErrorPolicies/ItemErrorPolicyLog.cs	Adds cached LoggerMessage delegates to minimize per-item logging allocations.
src/Wolfgang.Etl.ErrorPolicies/ItemErrorPolicy.cs	Implements the ready-made policy factory methods (skip/abort/log/dead-letter).
src/Wolfgang.Etl.Abstractions/Wolfgang.Etl.Abstractions.csproj	Bumps Abstractions version to 0.21.0 for lockstep release.
ETL-Abstractions.sln	Adds the new library + test projects to the solution.
CHANGELOG.md	Documents the new ErrorPolicies package and its surface area in Unreleased.

Review details

Suppressed comments (1)

src/Wolfgang.Etl.ErrorPolicies/ItemErrorPolicy.cs:35

• This comment refers to leaving "OnError" unset, but the core Abstractions API uses OnItemError (protected virtual) as the policy hook. Aligning the terminology here avoids confusion.

    /// A policy that re-throws the failure and stops the run. Equivalent to leaving <c>OnError</c>
    /// unset, provided for symmetry and explicitness.

• Files reviewed: 10/10 changed files
• Comments generated: 3
• Review effort level: Lite

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

[Chris-Wolfgang]

Update: added ErrorPolicy to the base stages

Folded in the piece that makes this package's assign-a-policy design actually work. ExtractorBase / LoaderBase / TransformerBase now expose:

public Func<ItemErrorContext, ItemErrorAction> ErrorPolicy { get; init; }   // non-null, default fail-fast, throw-on-null
protected virtual ItemErrorAction OnItemError(ItemErrorContext context) => ErrorPolicy(context);

So ErrorPolicy = ItemErrorPolicy.SkipAndLog(logger) works on any stage — no per-type property or OnItemError override needed (overriding is still available for stage-internal logic). The convenience bases (#344) inherit it.

This supersedes the earlier "the base classes deliberately expose no public error-handling property" decision in OnItemError's rationale comment — that comment is rewritten, and the resume caveat is documented (on a format that can't resume after a bad record, Skip means swallow-and-stop, not skip-and-continue; such leaves document it on their own type). The worker still owns the try/catch — that fact is unchanged.

Package docs / csproj description / CHANGELOG moved from the interim "override-and-invoke" wording back to the ErrorPolicy = … form. Skip/Abort naming settled (over OnError, which reads as an event and collides with the OnItemError method).

Verified: full-matrix build 0/0, Abstractions PackageValidation passes vs 0.20.0 (additive), Abstractions unit suite 463 green, ErrorPolicies 14 green.