[Accuracy diff No.70] Fix accuracy diff for topk API #217
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for your contribution! |
Cutelemon6
reviewed
May 31, 2025
Cutelemon6
requested changes
Jun 3, 2025
tester/api_config/config_analyzer.py
Outdated
| elif self.dtype in {"int32", "int64"}: | ||
| self.numpy_tensor = numpy.random.choice(numpy.arange(-x_numel, x_numel), size=self.shape, replace=False).astype(self.dtype) | ||
| else: | ||
| raise ValueError(f"Unsupported dtype {self.dtype} for paddle.topk") |
Collaborator
There was a problem hiding this comment.
f"Unsupported dtype {self.dtype} for paddle.topk / paddle.Tensor.topk"
Comment on lines
+1675
to
+1681
| if self.dtype in {"bfloat16", "float32", "float64"}: | ||
| dtype = "float32" if self.dtype == "bfloat16" else self.dtype | ||
| self.numpy_tensor = numpy.linspace(-x_numel, x_numel, x_numel, dtype=dtype).reshape(self.shape) | ||
| if numpy.unique(self.numpy_tensor).size < x_numel: | ||
| self.numpy_tensor = generate_unique_array(x_numel, dtype).reshape(self.shape) | ||
| elif self.dtype == "float16": | ||
| self.numpy_tensor = generate_unique_array(x_numel, self.dtype).reshape(self.shape) |
Collaborator
There was a problem hiding this comment.
非 float 16 经过 numpy.linspace 也会产生相同的元素吗（舍入错误），考虑将范围放在 dtype 的最大范围内再生成呢
numpy.linspace(numpy.finfo(dtype).min, numpy.finfo(dtype).max, x_numel, dtype=dtype).reshape(self.shape)float16 也试试。看了下有一些 float16 配置 numel 很大，但是 float16 表示范围很有限。
Collaborator
There was a problem hiding this comment.
upd：float16 最多表示 63488 个有限值，超出这个元素数量的 tensor 具有随机输出。
同学可以了解一下 float16 的表示范围，元素数量1亿以内的的 float16 tensor 理论上都能通过，没有随机性。
print(f"float16 max: {numpy.finfo(numpy.float16).max}, float16 min: {numpy.finfo(numpy.float16).min}, float16 eps: {numpy.finfo(numpy.float16).eps}")
print(f"max tensor numel est: {(numpy.finfo(numpy.float16).max.astype(numpy.float64) - numpy.finfo(numpy.float16).min.astype(numpy.float64)) / numpy.finfo(numpy.float16).eps}")output
float16 max: 65504.0, float16 min: -65504.0, float16 eps: 0.0009765625
max tensor numel est: 134152192.0
Cutelemon6
requested changes
Jun 5, 2025
| paddle.put_along_axis(Tensor([7, 8000],"float32"), Tensor([7, 799],"int64"), Tensor([7, 799],"float32"), 1, ) | ||
| paddle.put_along_axis(Tensor([8, 8000],"float32"), Tensor([8, 799],"int64"), Tensor([8, 799],"float32"), 1, ) | ||
| paddle.put_along_axis(Tensor([9, 8000],"float32"), Tensor([9, 799],"int64"), Tensor([9, 799],"float32"), 1, ) | ||
| paddle.topk(Tensor([128, 1000],"float16"), k=5, ) No newline at end of file |
Collaborator
There was a problem hiding this comment.
这个 case 在 config_analyzer.py 中添加如下初始化，经过多次运行均可 pass
elif api_config.api_name == "paddle.topk":
if self.check_arg(api_config, 0, "x"):
self.numpy_tensor = numpy.linspace(numpy.finfo(self.dtype).min, numpy.finfo(self.dtype).max, num=self.numel()).astype(self.dtype).reshape(self.shape)
Contributor
Author
There was a problem hiding this comment.
import numpy
dtype = numpy.float16
numel = 128*1000
out = numpy.linspace(numpy.finfo(dtype).min, numpy.finfo(dtype).max, num=numel).astype(dtype)
print(len(numpy.unique(out)))打印出来实际上只有两个数呀，小数不是浮点数，浮点数是有限的（二进制位数是确定的），小数是无限的，浮点数有精度，两个小数间隔在精度范围之外不能用两个浮点数来表示，浮点数越大，相邻的间隔也会越大
Collaborator
There was a problem hiding this comment.
确实嗷，是我没考虑全抱歉，float16 最多也只能表示六万多个不同的数了，对于这个 case 确实会表现随机性。
Comment on lines
+1675
to
+1681
| if self.dtype in {"bfloat16", "float32", "float64"}: | ||
| dtype = "float32" if self.dtype == "bfloat16" else self.dtype | ||
| self.numpy_tensor = numpy.linspace(-x_numel, x_numel, x_numel, dtype=dtype).reshape(self.shape) | ||
| if numpy.unique(self.numpy_tensor).size < x_numel: | ||
| self.numpy_tensor = generate_unique_array(x_numel, dtype).reshape(self.shape) | ||
| elif self.dtype == "float16": | ||
| self.numpy_tensor = generate_unique_array(x_numel, self.dtype).reshape(self.shape) |
Collaborator
There was a problem hiding this comment.
upd：float16 最多表示 63488 个有限值，超出这个元素数量的 tensor 具有随机输出。
同学可以了解一下 float16 的表示范围，元素数量1亿以内的的 float16 tensor 理论上都能通过，没有随机性。
print(f"float16 max: {numpy.finfo(numpy.float16).max}, float16 min: {numpy.finfo(numpy.float16).min}, float16 eps: {numpy.finfo(numpy.float16).eps}")
print(f"max tensor numel est: {(numpy.finfo(numpy.float16).max.astype(numpy.float64) - numpy.finfo(numpy.float16).min.astype(numpy.float64)) / numpy.finfo(numpy.float16).eps}")output
float16 max: 65504.0, float16 min: -65504.0, float16 eps: 0.0009765625
max tensor numel est: 134152192.0
Collaborator
|
所有 topk 的用例都通过了吗 |
Contributor
Author
是滴是滴，唯一没过的已经放在 random_calculation.txt 里面了 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
topk 返回的索引，相同元素的情况不保证是稳定的，更加关注前面的值