[internal] Port MergeDigests to Rust

[Eric-Arellano]

See #13646 for the motivation.

[ci skip-build-wheels]

[Eric-Arellano]

Note this clone. It's because we want to move the Vec<Digest> out of the Python::with_gil() closure so that we can use it in an async call to store.merge() right after.

I think I'm confused what is actually happening before. We before had a &PyAny and were extracting out Vec<&PyAny>, then iterating over and calling .extract() on each element into a PyDigest, unwrapped into a Digest. Then collecting into a Vec<Digest.

I'm confused if that involved cloning each PyDigest. I'm pretty sure the answer is yes because this line extracts as a PyDigest rather than PyRef<PyDigest>:

pants/src/rust/engine/src/nodes.rs

Lines 268 to 271 in 06efefc

	pub fn lift_directory_digest(digest: &PyAny) -> Result<hashing::Digest, String> {
	let py_digest: externs::fs::PyDigest = digest.extract().map_err(|e| format!("{}", e))?;
	Ok(py_digest.0)
	}

I found in #13660 that calling extract on a #[pyclass] struct without wrapping in PyRef<> results in cloning it. And indeed, we derive Clone on PyDigest, so that all compiles.

So, I think this is no worse than before? Only more explicit.

[Eric-Arellano]

Hm, is it important to us that the constructor remains Iterable[Digest] instead of Sequence[Digest]? Concretely, without that, you can't pass a generator expression.

It doesn't look like PyO3 has a conversion trait for Iterable; only Iterator, which requires calling iter() on sequences like lists.

[tdyas]

It doesn't look like PyO3 has a conversion trait for Iterable; only Iterator, which requires calling iter() on sequences like lists.

Maybe just take a PyObject and convert it using PyIterator::from_object which returns a PyResult so will error if it cannot convert. https://github.com/PyO3/pyo3/blob/8a03778ca32c22e6bd158cac4596e0b5541a4f41/src/types/iterator.rs#L36