feat(binding/java): Add list with recursive support

[cuichenli]

Rationale for this change

Currently, the java binding only support list current directory, it would be nice to have the ability to list recursively. Given opendal itself already supported this, I think it is reasonable to add this to the java binding as well. But I am not sure if that is suitable, so please feel free to correct me if I missed anything. It is a little bit tricky to add the unit test directly, given fs service it self does not support recursively list. But I have tested in my java program with s3 and gcs, they are working as expected.

What changes are included in this PR?

Added one paramater to support list recursively. Also fixed one unit test.

Are there any user-facing changes?

Java binding's list API changed.

[cuichenli]

i also fixed one unit test failing on my local machine, but looks like it is failing in ci. i will revert it later

[Xuanwo]

Considering that we will be adding more arguments for the list, how about following the same pattern to add this? #5664

It is a little bit tricky to add the unit test directly, given fs service it self does not support recursively list.

fs should support recursive list since we have applied some wrapper for it

[cuichenli]

fs should support recursive list since we have applied some wrapper for it

which service should i use? do i need to verify the capability as well? i observed error when specify the capability.list_with_recursive.

i will update the code later to align with the pr you provided.

[Xuanwo]

which service should i use? do i need to verify the capability as well? i observed error when specify the capability.list_with_recursive.

I remember that all services implementing list will automatically support recursive. I will double-check this later.

[cuichenli]

I remember that all services implementing list will automatically support recursive. I will double-check this later.

again, i may be wrong, but looks like the option for fs list is not used. so i think there is no recursive supprot for fs

opendal/core/src/services/fs/backend.rs

Line 364 in a6b0478

async fn list(&self, path: &str, _: OpList) -> Result<(RpList, Self::Lister)> {

[Xuanwo]

again, i may be wrong, but looks like the option for fs list is not used. so i think there is no recursive supprot for fs

Hi, it's implemeneted outsides at

opendal/core/src/layers/complete.rs

Lines 249 to 292 in a6b0478

	async fn complete_list(
	&self,
	path: &str,
	args: OpList,
	) -> Result<(RpList, CompleteLister<A, A::Lister>)> {
	let cap = self.info.native_capability();
	let recursive = args.recursive();
	match (recursive, cap.list_with_recursive) {
	// - If service can list_with_recursive, we can forward list to it directly.
	(_, true) => {
	let (rp, p) = self.inner.list(path, args).await?;
	Ok((rp, CompleteLister::One(p)))
	}
	// If recursive is true but service can't list_with_recursive
	(true, false) => {
	// Forward path that ends with /
	if path.ends_with('/') {
	let p = FlatLister::new(self.inner.clone(), path);
	Ok((RpList::default(), CompleteLister::Two(p)))
	} else {
	let parent = get_parent(path);
	let p = FlatLister::new(self.inner.clone(), parent);
	let p = PrefixLister::new(p, path);
	Ok((RpList::default(), CompleteLister::Four(p)))
	}
	}
	// If recursive and service doesn't support list_with_recursive, we need to handle
	// list prefix by ourselves.
	(false, false) => {
	// Forward path that ends with /
	if path.ends_with('/') {
	let (rp, p) = self.inner.list(path, args).await?;
	Ok((rp, CompleteLister::One(p)))
	} else {
	let parent = get_parent(path);
	let (rp, p) = self.inner.list(parent, args).await?;
	let p = PrefixLister::new(p, path);
	Ok((rp, CompleteLister::Three(p)))
	}
	}
	}
	}

Sorry for making you feel confused. We have a plan to refactor this part to make it more clear.

[cuichenli]

i had some trouble on passing the options to the async block so that is the best way i can do. please let me know if that is not suitable, i may need some help on how to pass this arround. thanks

[Xuanwo]

Hi, I'm a bit confused why we need to apply CompleteLayer?

[Xuanwo]

CompleteLayer is applied by default. We don't need to apply it again.

[Xuanwo]

Thank you @cuichenli for working on this. This PR looks good to me. Wait for @tisonkun and @G-XD for another look.

[tisonkun]

Make some code style change and approved. Keep all the binding mappings follow a similar structure and avoid some unnecessary clone.

[Xuanwo]

Thank you @cuichenli and @tisonkun for working on this!