Collecting multiple attribute error #4243

Cheukting · 2024-06-13T09:54:13Z

~~Attempt to clos e #2892~~ (Edit by @davidhewitt - I think there's more we can do with #[pymethods] probably, though might be a fair bit more work.)

I am not sure how to add the test in CI but I have tested manually with the following:

use pyo3::prelude::*;

#[pyclass]
struct Example {
    #[pyo3(foo)]
    #[pyo3(blah)]
    x: i32,
    #[pyo3(pop)]
    y: i32,
}

Originally it gave:

error: expected one of: `get`, `set`, `name`
 --> pytests/src/pyclasses.rs:7:12
  |
7 |     #[pyo3(foo)]
  |            ^^^

error: cannot find attribute `pyo3` in this scope
  --> pytests/src/pyclasses.rs:10:7
   |
10 |     #[pyo3(pop)]
   |       ^^^^
   |
   = note: `pyo3` is in scope, but it is a crate, not an attribute

error: could not compile `pyo3-pytests` (lib) due to 2 previous errors

Now it gives:

error: expected one of: `get`, `set`, `name`
 --> pytests/src/pyclasses.rs:7:12
  |
7 |     #[pyo3(foo)]
  |            ^^^

error: expected one of: `get`, `set`, `name`
 --> pytests/src/pyclasses.rs:8:12
  |
8 |     #[pyo3(blah)]
  |            ^^^^

error: expected one of: `get`, `set`, `name`
  --> pytests/src/pyclasses.rs:10:12
   |
10 |     #[pyo3(pop)]
   |            ^^^

error: could not compile `pyo3-pytests` (lib) due to 3 previous errors

davidhewitt

Thanks, this looks like a great discovery!

To test this, we should update the UI tests to make use of these attributes. For example, see tests/ui/invalid_pyclass_args.rs.

At the moment it has a separate class for each invalid entry. We could instead write pieces which can be combined together, for example:

#[pyclass(
    extend = pyo3::types::PyDict,  // typo in the key
    extends = "PyDict", // Needs to be an ident, not a string
)]
struct InvalidExtends;

... and so on.

See also invalid_pymethods.rs - we have loads of #[pymethods] impl MyClass blocks, and we should be able to combine them all into one with this functionality!

davidhewitt · 2024-06-14T11:04:54Z

pyo3-macros-backend/src/attributes.rs

+    if !allerr.is_empty() {
+        let mut error = allerr[0].clone();
+        for err in &allerr[1..] {
+            error.combine(err.clone());
+        }
+        Err(error)
+    } else {
+        Ok(out)
+    }


Ah, this error.combine() function is extremely handy!

I think now that we know about this, to avoid the allerr vec we can probably have something like this:

struct ErrorCombiner(Option<syn::Error>); impl ErrorCombiner { fn combine(&mut self, error: syn::Error) { if let Some(existing) = &mut self.0 { existing.combine(error); } else { self.0 = Some(error); } } fn ensure_empty(self) -> Result<()> { if let Some(error) = self.0 { Err(error) } else { None } }

Above, instead of extending the Vec we would combine the error immediately, and then at the end of the function we could just check the combiner is empty:

Suggested change

if !allerr.is_empty() {

let mut error = allerr[0].clone();

for err in &allerr[1..] {

error.combine(err.clone());

}

Err(error)

} else {

Ok(out)

}

error_combiner.ensure_empty()?;

Ok(out)

I think this pattern would be useful in a lot of places in the macro code!

Cheukting · 2024-06-14T17:15:17Z

Thanks for the suggestions @davidhewitt i have added the test and have refactored as suggested.

davidhewitt

Thanks, I love these kinds of UX improvements! Just a couple of thoughts related to the implementation still.

(And sorry it took me a few days here, was focussing on #4266 for a while.)

davidhewitt · 2024-06-21T14:15:26Z

tests/ui/invalid_pyclass_args.rs

+#[pyclass]
+struct Example {
+    #[pyo3(foo)]
+    #[pyo3(blah)]
+    x: i32,
+    #[pyo3(pop)]
+    y: i32,
+}


Let's call this something more descriptive:

Suggested change

#[pyclass]

struct Example {

#[pyo3(foo)]

#[pyo3(blah)]

x: i32,

#[pyo3(pop)]

y: i32,

}

#[pyclass]

struct MultipleErrors {

#[pyo3(foo)]

#[pyo3(blah)]

x: i32,

#[pyo3(pop)]

y: i32,

}

davidhewitt · 2024-06-21T14:19:07Z

pyo3-macros-backend/src/attributes.rs

-    take_attributes(attrs, |attr| {
-        if let Some(options) = get_pyo3_options(attr)? {
-            out.extend(options);
+    let mut all_error = ErrorCombiner(None);


Personally I prefer a plural name:

Suggested change

let mut all_error = ErrorCombiner(None);

let mut all_errors = ErrorCombiner(None);

davidhewitt · 2024-06-21T14:23:05Z

pyo3-macros-backend/src/pyclass.rs

+    let mut field_options_res: Vec<Result<(&syn::Field, FieldPyO3Options)>> =
+        match &mut class.fields {
+            syn::Fields::Named(fields) => fields
+                .named
+                .iter_mut()
+                .map(|field| {
+                    FieldPyO3Options::take_pyo3_options(&mut field.attrs)
+                        .map(move |options| (&*field, options))
+                })
+                .collect::<Vec<_>>(),
+            syn::Fields::Unnamed(fields) => fields
+                .unnamed
+                .iter_mut()
+                .map(|field| {
+                    FieldPyO3Options::take_pyo3_options(&mut field.attrs)
+                        .map(move |options| (&*field, options))
+                })
+                .collect::<Vec<_>>(),
+            syn::Fields::Unit => {
+                if let Some(attr) = args.options.set_all {
+                    return Err(syn::Error::new_spanned(attr, UNIT_SET));
+                };
+                if let Some(attr) = args.options.get_all {
+                    return Err(syn::Error::new_spanned(attr, UNIT_GET));
+                };
+                // No fields for unit struct
+                Vec::new()
+            }
+        };
+
+    // handle error here
+
+    let mut all_error = ErrorCombiner(None);
+
+    let mut field_options: Vec<(&syn::Field, FieldPyO3Options)> = field_options_res
+        .drain(..)
+        .filter_map(|result| match result {
+            Err(err) => {
+                all_error.combine(err);
+                None
+            }
+            Ok(options) => Some(options),
+        })
+        .collect::<Vec<_>>();


I think we should now be able to simplify here and go back to something more like the original implementation and drop the need for the intermediate field_options_res.

I think the idea would be to set up all_errors before the match &mut class.fields. And then instead of collecting into vec-of-results I think it would be possible to swap the .map for .filter_map.

Something like

.filter_map(|field| { match FieldPyO3Options::take_pyo3_options(&mut field.attrs) { Ok(options) => Some((&*field, options)), Err(e) => { all_errors.combine(e); None } } })

davidhewitt · 2024-06-21T14:38:17Z

newsfragments/4243.changed.md

@@ -0,0 +1 @@
+Reporting multiple errors from proc-macros attributes.


Strictly speaking, this only covers some of the attributes we could yet cover:

Suggested change

Reporting multiple errors from proc-macros attributes.

Report multiple errors from `#[pyclass]` and `#[pyo3(..)]` attributes.

Cheukting added 3 commits June 13, 2024 00:03

collecting multiple errors

6231d2a

collecting errors from different fileds

b9da077

adding changelog

978c2f1

Cheukting changed the title ~~Collecting multiple attribute errore~~ Collecting multiple attribute error Jun 13, 2024

davidhewitt reviewed Jun 14, 2024

View reviewed changes

Adding UI test

5ccd8a4

Cheukting force-pushed the mul_error branch from 83f14f4 to 5ccd8a4 Compare June 14, 2024 13:36

refactoring

1289c9b

davidhewitt reviewed Jun 21, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Collecting multiple attribute error #4243

Collecting multiple attribute error #4243

Cheukting commented Jun 13, 2024 •

edited by davidhewitt

Loading

davidhewitt left a comment

davidhewitt Jun 14, 2024

Cheukting commented Jun 14, 2024

davidhewitt left a comment

davidhewitt Jun 21, 2024

davidhewitt Jun 21, 2024

davidhewitt Jun 21, 2024

davidhewitt Jun 21, 2024

	let mut all_error = ErrorCombiner(None);
	let mut all_errors = ErrorCombiner(None);

		@@ -0,0 +1 @@
		Reporting multiple errors from proc-macros attributes.

	Reporting multiple errors from proc-macros attributes.
	Report multiple errors from `#[pyclass]` and `#[pyo3(..)]` attributes.

Collecting multiple attribute error #4243

Are you sure you want to change the base?

Collecting multiple attribute error #4243

Conversation

Cheukting commented Jun 13, 2024 • edited by davidhewitt Loading

davidhewitt left a comment

Choose a reason for hiding this comment

davidhewitt Jun 14, 2024

Choose a reason for hiding this comment

Cheukting commented Jun 14, 2024

davidhewitt left a comment

Choose a reason for hiding this comment

davidhewitt Jun 21, 2024

Choose a reason for hiding this comment

davidhewitt Jun 21, 2024

Choose a reason for hiding this comment

davidhewitt Jun 21, 2024

Choose a reason for hiding this comment

davidhewitt Jun 21, 2024

Choose a reason for hiding this comment

Cheukting commented Jun 13, 2024 •

edited by davidhewitt

Loading