Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix preflight check util #480

Merged
merged 2 commits into from
Mar 20, 2024
Merged

Fix preflight check util #480

merged 2 commits into from
Mar 20, 2024

Conversation

eu9ene
Copy link
Collaborator

@eu9ene eu9ene commented Mar 15, 2024

  • fix config path
  • make it compatible with earlier versions of python (we use a variety of python versions and there's no reason to make it incompatible)

@eu9ene eu9ene requested a review from gregtatum March 15, 2024 19:00
Copy link
Member

@gregtatum gregtatum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix! I'm a bit sad I can't use the new syntax here. Question: What's the story on our oldest version of python we support? Is there an opportunity to increase it to get access to newer language features?

@eu9ene
Copy link
Collaborator Author

eu9ene commented Mar 20, 2024

Thanks for the fix! I'm a bit sad I can't use the new syntax here. Question: What's the story on our oldest version of python we support? Is there an opportunity to increase it to get access to newer language features?

It has to do with inconsistent environments we run this in (see #264). Ideally we should have just two docker images with everything installed (CPU and GPU ones, see also #391) that we can also run locally and have just one python version for everything. I see some of the prod tasks run with Python version: 3.10.12.

IMO in general new Python versions come out periodically and usually, they're not worth switching to unless you do some low-level Python optimization. All the new syntactic sugar doesn't help much with our problems. The bottlenecks we have are usually not in python but in IO, CPU/GPU, the amount of processed data etc. If say, there is a groundbreaking optimization in python that improves something like memory allocation 2x+, it's worth considering. For now I would even recommend downgraing poetry env to 3.10 to be consistent with production.

Also, I don't see a big difference in using switch syntax vs standard for python if else one.

@eu9ene eu9ene merged commit a359723 into main Mar 20, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants