Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 Make file processing with ClassDefVisitor deterministic #118

Merged
merged 3 commits into from
Aug 14, 2023

Conversation

slafs
Copy link
Contributor

@slafs slafs commented Aug 13, 2023

Related to #117 (non-determinism part)

  • Change queue creation and element addition while processing
  • Sort files before processing (Path.glob doesn't guarantee any order)

By using `set` for specifying files in queue
we loose the order in which the files will be processed
(in `missing_files` var).
So we process the queue left to right.

This way the `appendleft` call makes sense
and we process the same file as first one
as with the previous implementation.
Turns out `.glob(...)` yields results
in "arbitrary order",
so the file list for processing
might be in different order on different OSes.

See docs for `os.scandir` (the thing `glob` uses under the hood):
https://docs.python.org/3/library/os.html?highlight=arbitrary%20order#os.scandir
@slafs slafs changed the title Fix finding pydantic models Make file processing with ClassDefVisitor deterministic Aug 14, 2023
@Kludex
Copy link
Member

Kludex commented Aug 14, 2023

Thanks @slafs ❤️

@Kludex Kludex changed the title Make file processing with ClassDefVisitor deterministic 🐛 Make file processing with ClassDefVisitor deterministic Aug 14, 2023
@Kludex Kludex merged commit b08f445 into pydantic:main Aug 14, 2023
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants