diff --git a/offload/unittests/Conformance/README.md b/offload/unittests/Conformance/README.md new file mode 100644 index 0000000000000..0202242c99a0d --- /dev/null +++ b/offload/unittests/Conformance/README.md @@ -0,0 +1,83 @@ +# GPU Math Conformance Tests + +## Overview + +This test suite provides a framework to systematically measure the accuracy of math functions on GPUs and verify their conformance with standards like OpenCL. + +While the primary focus is validating the implementations in the C standard math library (LLVM-libm), these tests can also be executed against other math library providers, such as CUDA Math and HIP Math, for comparison. + +The goals of this project are to empower LLVM-libm contributors with a robust tool for validating their implementations and to build trust with end-users by providing transparent accuracy data. + +### Table of Contents + +- [Getting Started](#getting-started) +- [Running the Tests](#running-the-tests) +- [Adding New Tests](#adding-new-tests) + +## Getting Started + +This guide covers how to build the necessary dependencies, which include the new Offload API and the C standard library for both host and GPU targets. + +### System Requirements + +Before you begin, ensure your system meets the following requirements: + +- A system with an AMD or NVIDIA GPU. +- The latest proprietary GPU drivers installed. +- The corresponding development SDK for your hardware: + - **AMD:** [ROCm SDK](https://rocm.docs.amd.com) + - **NVIDIA:** [CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit) + +### Building the Dependencies + +The official documentation for building LLVM-libc for GPUs provides a detailed guide and should be considered the primary reference. Please follow the instructions in the **"Standard runtimes build"** section of that guide: + +- [Building the GPU C library (Official Documentation)](https://libc.llvm.org/gpu/building.html) + +> [!IMPORTANT] +> For the conformance tests, the standard `cmake` command from the official documentation must be adapted slightly. You must also add `libc` to the main `-DLLVM_ENABLE_RUNTIMES` list. This is a crucial step because the tests need a host-side build of `libc` to use as the reference oracle for validating GPU results. + +## Running the Tests + +### Default Test + +To build and run the conformance test for a given function (e.g., `logf`) against the default C standard math library `llvm-libm` provider, use the following command. This will execute the test on all available and supported platforms. + +```bash +ninja -C build/runtimes/runtimes-bins offload.conformance.logf +``` + +### Testing Other Providers + +Once the test binary has been built, you can run it against other math library providers using the `--test-configs` flag. + +- **For `cuda-math` on an NVIDIA GPU:** + + ```bash + ./build/runtimes/runtimes-bins/offload/logf.conformance --test-configs=cuda-math:cuda + ``` + +- **For `hip-math` on an AMD GPU:** + + ```bash + ./build/runtimes/runtimes-bins/offload/logf.conformance --test-configs=hip-math:amdgpu + ``` + +You can also run all available configurations for a test with: + +```bash +./build/runtimes/runtimes-bins/offload/logf.conformance --test-configs=all +``` + +## Adding New Tests + +To add a conformance test for a new math function, follow these steps: + +1. **Implement the Device Kernels**: Create a kernel wrapper for the new function in each provider's source file. For CUDA Math and HIP Math, you must also add a forward declaration for the vendor function in `/device_code/DeviceAPIs.hpp`. + +2. **Implement the Host Test**: Create a new `.cpp` file in `/tests`. This file defines the `FunctionConfig` (function and kernel names, as well as ULP tolerance) and the input generation strategy. + + - Use **exhaustive testing** (`ExhaustiveGenerator`) for functions with small input spaces (e.g., half-precision functions and single-precision univariate functions). This strategy iterates over every representable point in the input space, ensuring complete coverage. + - Use **randomized testing** (`RandomGenerator`) for functions with large input spaces (e.g., single-precision bivariate and double-precision functions), where exhaustive testing is computationally infeasible. Although not exhaustive, this strategy is deterministic, using a fixed seed to sample a large, reproducible subset of points from the input space. + +3. **Add the Build Target**: Add a new `add_conformance_test(...)` entry to `/tests/CMakeLists.txt` to make the test buildable.