Skip to content

Increase vectorized Tan test tolerance #130469

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kotlarmilos merged 4 commits into from
Jul 13, 2026
Merged

Increase vectorized Tan test tolerance #130469

Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4c413f5
Initial plan
Copilot Jul 10, 2026
e67a446
Increase FMA trig test tolerance
Copilot Jul 10, 2026
9f6b3e9
Use separate tolerance for vectorized Tan tests
kotlarmilos Jul 10, 2026
76ae504
Keep shared trig tolerance unchanged
kotlarmilos Jul 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions src/libraries/System.Numerics.Tensors/tests/TensorPrimitives.Generic.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -457,8 +457,11 @@ public static IEnumerable<object[]> SpanDestinationFunctionsToTest()
// The current trigonometric algorithm depends on hardware FMA support for best precision.
// Even with FMA, ARM64 vectorized trig can diverge a few tens of ULPs from scalar.
T? trigTolerance = IsFmaSupported
? Helpers.DetermineTolerance<T>(doubleTolerance: 1e-14, floatTolerance: 1e-5f)
? Helpers.DetermineTolerance<T>(doubleTolerance: 3e-14, floatTolerance: 1e-5f)
: Helpers.DetermineTolerance<T>(doubleTolerance: 1e-10, floatTolerance: 1e-4f);
T? tanTolerance = IsFmaSupported
? Helpers.DetermineTolerance<T>(doubleTolerance: 3e-13, floatTolerance: 1e-4f)
: trigTolerance;
Comment thread
kotlarmilos marked this conversation as resolved.
Comment thread
kotlarmilos marked this conversation as resolved.

yield return Create(TensorPrimitives.Acosh, T.Acosh);
yield return Create(TensorPrimitives.AcosPi, T.AcosPi);
Expand Down Expand Up @@ -507,7 +510,7 @@ public static IEnumerable<object[]> SpanDestinationFunctionsToTest()
yield return Create(TensorPrimitives.Sinh, T.Sinh, Helpers.DetermineTolerance<T>(doubleTolerance: 1e-14));
yield return Create(TensorPrimitives.SinPi, T.SinPi, Helpers.DetermineTolerance<T>(doubleTolerance: 1e-13, floatTolerance: 1e-4f));
yield return Create(TensorPrimitives.Sqrt, T.Sqrt);
yield return Create(TensorPrimitives.Tan, T.Tan, trigTolerance);
yield return Create(TensorPrimitives.Tan, T.Tan, tanTolerance);
yield return Create(TensorPrimitives.Tanh, T.Tanh);
yield return Create(TensorPrimitives.TanPi, T.TanPi);
yield return Create(TensorPrimitives.Truncate, T.Truncate);
Expand Down
Loading