Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upgrade to python3.11 #4059

Closed
jonathanmetzman opened this issue Jun 21, 2024 · 8 comments
Closed

Upgrade to python3.11 #4059

jonathanmetzman opened this issue Jun 21, 2024 · 8 comments
Assignees
@vitorguidi

Comments

@jonathanmetzman
Copy link
Collaborator

All PRs related to this effort should link here.
@jonathanmetzman jonathanmetzman mentioned this issue Jun 21, 2024
Merged
jonathanmetzman added a commit that referenced this issue Jun 21, 2024
@jonathanmetzman
Fix deploy. (#4060)
99ef33c 
Metric setup is broken because of Python package upgrade.

Fixes: #4058
Related: #4059
This was referenced Jun 24, 2024
Merged
Merged
Merged
Merged
vitorguidi added a commit that referenced this issue Jun 26, 2024
@vitorguidi
Chore/py37 proto bump reattempt (#4062)
7e57e88 
Reattempt at #4029 

This PR belongs to [this
initiative](#4059)
@jonathanmetzman jonathanmetzman mentioned this issue Jun 27, 2024
Merged
This was referenced Jun 27, 2024
Merged
Merged
@eduarddfinity
Copy link

eduarddfinity commented Jul 18, 2024 •
edited
Loading

Hello clusterfuzz team

do you have an ETA for migrating to python3.11?
Unfortunately we've been locked out of our clusterfuzz instance since last month, because cross origin redirect login is no longer supported in firebase. I know this has been fixed in clusterfuzz but we cannot update our instance as app engine no longer allows python3.7 apps to be deployed.

Thanks in advance!

@oliverchang
Copy link
Collaborator

I'll let @vitorguidi comment in more detail on this, but we are actively working on this upgrade and should hopefully be completing this in the coming weeks / month.

@vitorguidi
Copy link
Collaborator

Hello clusterfuzz team

do you have an ETA for migrating to python3.11? Unfortunately we've been locked out of our clusterfuzz instance since last month, because cross origin redirect login is no longer supported in firebase. I know this has been fixed in clusterfuzz but we cannot update our instance as app engine no longer allows python3.7 apps to be deployed.

Thanks in advance!

We already have all dependencies bumped, the next step is switching the interpreter and performing a canary with production instances. This should be done next week, and it will be possible to more accurately gauge how complex the process will be, but Oliver's estimate is what I am expecting.

@vitorguidi vitorguidi mentioned this issue Jul 29, 2024
Merged
vitorguidi added a commit that referenced this issue Jul 29, 2024
@vitorguidi
Logging skipped files on normalized file set (#4127)
b5203e4 
### Motivation

During the python 3.11 upgrade, a regresion in os.path.realpath made
clusterfuzz delete all its files when paths with mismatching case in the
C: drive were compared raw. In order to recover historical context as to
why we choose to delete files that have a mismatch between real and
absolute path, this log will be introduced.

Part of this
[initiative](#4059)
@vitorguidi
Copy link
Collaborator

#4025 has already been merged and we have been running clusterfuzz in production for >1 week at Python3.11, so it should be good now @eduarddfinity

@vitorguidi vitorguidi mentioned this issue Aug 12, 2024
Merged
@nguyendon
Copy link

@vitorguidi are you all running from a stable release branch or are you just running off of master?

@jonathanmetzman
Copy link
Collaborator Author

We run off master. I don't think we've done releases in at least a year, don't have the cycles for this sadly.

@nguyendon
Copy link

@jonathanmetzman is running off of master just for an internal instance or is that what OSS-Fuzz runs on as well?

@jonathanmetzman
Copy link
Collaborator Author

@jonathanmetzman is running off of master just for an internal instance or is that what OSS-Fuzz runs on as well?

OSS-Fuzz was running on a very old version of master but now is pretty much on latest master. Things for oss-fuzz are not in a very stable state but your internal deployment is probably more like the internal instance which is stable on 3.11

@vitorguidi vitorguidi mentioned this issue Aug 26, 2024
Merged
vitorguidi added a commit that referenced this issue Aug 26, 2024
@vitorguidi
Adding CLUSTERFUZZ_RELEASE env var to batch docker container (#4210)
2b882d4 
### Motivation

The last move to upgrade our fleet to 3.11 is switch batch. This PR adds
the CLUSTERFUZZ_RELEASE env var to batch config, so we can roll
forward/back between 3.7 and 3.11 on batch by maneuvering between the
appropriate docker image and code release

This PR belongs to this
[initiative](#4059)

### Deployment strategy

This code should be deployed to uworker instances, which are the actual
ones that create batch jobs. Deploy this using --release=candidate, with
the appropriate clusterfuzz-config
@vitorguidi vitorguidi mentioned this issue Sep 26, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vitorguidi vitorguidi

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@oliverchang @vitorguidi @nguyendon @jonathanmetzman @eduarddfinity