Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

list-objects-*: implement allow unordered option #1270

Merged

Conversation

snosratiershad
Copy link
Contributor

As discussed in this issue

@snosratiershad
Copy link
Contributor Author

@fviard, if there is anything I can help about it, feel free to contact me.

@fviard
Copy link
Contributor

fviard commented Aug 15, 2022

Sorry for the delay of my reply.
Instead of adding the parameter to all the function prototypes, i think that you can use directly self.config.allow_unordered inside def bucket_list_noparse(...).

Also, it will not be limited to "ls" anymore, but I think that it is ok, as s3cmd sync logic never assumed that file list would be in order for remote or local side.

@snosratiershad snosratiershad force-pushed the list-objects/support-unordered-list branch from 6a98067 to d5fc25c Compare August 22, 2022 10:00
@snosratiershad
Copy link
Contributor Author

@fviard, as you requested, I changed it to a config variable.
I couldn't found any of documentation files in the repository, but I think it must be included in somewhere as there is no help message for now.

@fviard
Copy link
Contributor

fviard commented Aug 22, 2022

@snosratiershad You could have kept the "command line switch" in addition with the help, that was good.

What is happening is that the config comes from the config file, but command line switches allow to override the config files.
It has no relation with how you use it inside the code after that.

@snosratiershad
Copy link
Contributor Author

snosratiershad commented Aug 22, 2022

@fviard, should command line switch literally overide config file, or just ignoring the value comes from self.config.list_objects_allow_unordered when the `--ls-allow-unordered' switch called?

@snosratiershad
Copy link
Contributor Author

I realized what you mean. also going to change "list-objects-allow-unordered" to "list-allow-onordered".

@snosratiershad
Copy link
Contributor Author

snosratiershad commented Aug 22, 2022

@fviard Done.
Update: the force push is because my typo in first push.

@snosratiershad snosratiershad force-pushed the list-objects/support-unordered-list branch from b0fd9ec to 26b3495 Compare August 22, 2022 12:33
@fviard fviard added this to the 2.3.0 milestone Aug 27, 2022
@fviard fviard merged commit 027e96b into s3tools:master Aug 27, 2022
@fviard
Copy link
Contributor

fviard commented Aug 27, 2022

Merged, thank you!

@visbits
Copy link

visbits commented Nov 9, 2023

Delimiter cannot be used with allow-unordered.

The code must not set a limiter if allow-unordered is true:
From line 400 in S3.py
if not self.config.recursive and not recursive:
To
if not self.config.recursive and not recursive and not self.config.list_allow_unordered:

@snosratiershad
Copy link
Contributor Author

@visbits Thanks for pointing this out. I'll check it out and fix it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants