Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error while parsing precomputed field for NativeFunctions YAML in torchgen #127411

Closed
apach301 opened this issue May 29, 2024 · 0 comments
Closed
Labels
module: codegen Issues related to the codegen for Aten and Autograd triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@apach301
Copy link
Contributor

apach301 commented May 29, 2024

🐛 Describe the bug

Hi,
I found a bug during testing parse_native_yaml function.

An error occured in Argument parser, but the bug is in missed validation for 'precomputed' field.
Seed to reproduce bug:

- func: index.Tensor_out(Tensor self, Tensor?[] indices, *, Tensor(a!) out) -> Tensor(a!)
  device_check: NoCheck
  structured: True
  structured_inherits: TensorIteratorBase
  precomputed:
  - indices ->dnn_linear(Tensor self, Tensor weight, Tensor? bias=None) -> Tensor
  python_module: nn

Error:

 === Uncaught Python exception: ===
ValueError: not enough values to unpack (expected 2, got 1)
  in /tmp/tmptn11fmou.yaml:3017:
    index.Tensor_out(Tensor self, Tensor?[] indices, *, Tensor(a!) out) -> Tensor(a!)
Traceback (most recent call last):
  File "/parse_native_yaml.py", line 40, in TestOneInput
    parse_native_yaml(filename, "/pytorch/aten/src/ATen/native/tags.yaml")
  File "/usr/local/lib/python3.9/dist-packages/torchgen/gen.py", line 247, in parse_native_yaml
    _GLOBAL_PARSE_NATIVE_YAML_CACHE[path] = parse_native_yaml_struct(
  File "/usr/local/lib/python3.9/dist-packages/torchgen/gen.py", line 169, in parse_native_yaml_struct
    func, m = NativeFunction.from_yaml(e, loc, valid_tags, ignore_keys)
  File "/usr/local/lib/python3.9/dist-packages/torchgen/model.py", line 670, in from_yaml
    precomputed = Precompute.parse(precomputed_dict) if precomputed_dict else None
  File "/usr/local/lib/python3.9/dist-packages/torchgen/model.py", line 2773, in parse
    with_list_args = [Argument.parse(name.strip()) for name in with_list]
  File "/usr/local/lib/python3.9/dist-packages/torchgen/model.py", line 2773, in <listcomp>
    with_list_args = [Argument.parse(name.strip()) for name in with_list]
  File "/usr/local/lib/python3.9/dist-packages/torchgen/model.py", line 1990, in parse
    type_and_annot, name_and_default = arg.rsplit(" ", 1)
ValueError: not enough values to unpack (expected 2, got 1)
  in /tmp/tmptn11fmou.yaml:3017:
    index.Tensor_out(Tensor self, Tensor?[] indices, *, Tensor(a!) out) -> Tensor(a!)

Versions

PyTorch version: 2.3.0, current main

OS: Ubuntu 20.04.6 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
Clang version: 14.0.6
CMake version: version 3.29.2
Libc version: glibc-2.31

Python version: 3.9.5 (default, Nov 23 2021, 15:27:38) [GCC 9.3.0] (64-bit runtime)
Python platform: Linux-5.4.0-174-generic-x86_64-with-glibc2.31

cc @ezyang @bhosmer @bdhirsh @kadeng

@ezyang ezyang added triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module module: codegen Issues related to the codegen for Aten and Autograd labels May 30, 2024
pytorchmergebot pushed a commit that referenced this issue May 30, 2024
Fixing multiple errors in parse_native_yaml when loading NativeFunctions from Yaml file.

Add assertions that validates parsed data.

Fixes #127404, #127405, #127406, #127407, #127408, #127409, #127410, #127411

Pull Request resolved: #127413
Approved by: https://github.com/ezyang
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module: codegen Issues related to the codegen for Aten and Autograd triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Development

No branches or pull requests

2 participants