__getitem__ iteration.

[fsh]

>>> a=PyZZ(17)
>>> len(a)
5
>>> [a[k] for k in range(len(a))]
[True, False, False, False, True]
>>> list(a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'builtins.PyZZ' object is not iterable

I have both __len__() __getitem__() which I think is normally enough for Python to do iteration without me needing to create a separate iter object, at least that's how it's worked for me in other bindings.

From Python docs:

Setting a special method to None indicates that the corresponding operation is not available. For example, if a class sets iter() to None, the class is not iterable, so calling iter() on its instances will raise a TypeError (without falling back to getitem()).

Does PyO3 do something like that to block the fallback behavior? Is it possible to stop it from doing it?

I tried returning PyRef<Self> from __iter__ but then I get the error that a non-iterable was returned. If I define also __next__ to some dummy, of course it just doesn't iterate. I'd like to avoid having to use the __iter__ __next__ if I can, because I have a lot of small types and merely want them to be iterable out of convenience, and the desired behavior in my case is for Python to fall back to using __getitem__[] iteration.

For reference my actual __getitem__ is as:

  fn __getitem__(&self, idx: Either2<&PySlice,isize>) -> PyResult<bool> {
    let bz = self.el.significant_bits() as isize;
    match idx {
      Either2::A(sl) => todo!(),
      Either2::B(i) => {
        let i = if i < 0 { bz + i } else { i };
        if i < 0 || i >= bz {
          Err(PyIndexError::new_err("out of bounds"))
        } else {
          Ok(self.el.get_bit(i as u32))
        }
      }
    }
  }

Possibly related to #1855 or the ongoing migration work?

[fsh]

Oops, I definitely skimmed the other issue too quickly. This is indeed a duplicate of #1855. Feel free to mark as such.

[davidhewitt]

Hmm, thanks for the head-up. I don't think this is a duplicate.

Looks like there are four methods with special fallbacks implemented in CPython: https://docs.python.org/3/reference/datamodel.html#id9: __hash__, __iter__, __reversed__, and __contains__

We don't implement equivalent fallbacks in PyO3, but should consider doing so.

[fsh]

Hmmm, I thought that this was not something that PyO3 needs to do, as it was handled automatically by CPython. Because I seem to recall these automatic fallbacks working in Cython for example, so long as I didn't define an __iter__ nor set it to None... But perhaps Cython had special handling of it then?

(I.e. what I thought was happening is that PyO3 explicitly disables the protocols that aren't defined, thus signalling to CPython that the class it not iterable, and so CPython doesn't try the fallback? But is setting those xx_func pointers to null (C API-side) equivalent to setting them to None in a Python class declaration?)

[davidhewitt]

You're right, on further investigation these automatic fallbacks are generated by CPython.

The reason that the __iter__ fallback is not working is because __getitem__ in #[pymethods] currently doesn't fill the sequence slot. This is part of the ongoing work in #1884. I have an idea how to fix this, however it may lead to changes in behaviour in edge cases (e.g. types which weren't iterable suddenly become iterable) so this will be done for PyO3 0.16.

[davidhewitt]

From PyO3 0.16 the sequence slot will now be filled (#2065).