kubeflow-pipelines-visualization-server: bump to use Python 3.11

[sil2100]

Another package that was causing some images to pull in multiple Python bases.

This only bumps the used Python version to 3.11 as this is the max that google-cloud-sdk seems to support for now (see google-cloud-sdk.yaml). This also required bumping some pip dependencies.

[sil2100]

Also, wanted to bump the tensorflow requirement to what we use, aka. 2.18.0, but actually one of the pip dependencies, I guess that was even tensorflow-data-validation 1.16.1 (latest), claimed it only supported tensorflow <2.17.0. So my bump in the requirements.txt had to be to 2.16.2 instead.

[dannf]

Wow, this one seems like a difficult case - thanks for tackling it!

I'm concerned with diverging from upstream to move to 3.11 support. I wondered why upstream hadn't moved forward, and took a quick look at the project. It seems very active. But where there used to be both 2.3.0 and sdk-2.3.0 tags, there's now been 9 releases of just sdk-* tags (2.4->2.11). I wonder if maybe they have restructured things somewhat, and we're stuck on a dead release.

As for how to validate these changes, the best way I can think of is to build them into the target image and run the image test case.

[sil2100]

Let me try building the image against the PR presubmit repo and see if that works as expected.

[sil2100]

I'm concerned with diverging from upstream to move to 3.11 support. I wondered why upstream hadn't moved forward, and took a quick look at the project. It seems very active. But where there used to be both 2.3.0 and sdk-2.3.0 tags, there's now been 9 releases of just sdk-* tags (2.4->2.11). I wonder if maybe they have restructured things somewhat, and we're stuck on a dead release.

Regarding this: I don't think we're necessarily 'diverging' from upstream by moving from 3.10 to 3.11. I didn't dig down
too deeply of course, but from all the release notes that I saw, there doesn't seem to be a recommended Python version to use. I think we used 3.10 because it was probably what google-cloud-sdk was using before. The only requirement that pipelines has is for Python to be 3.7 or above.
Did you see any mention of 3.10 being recommended that I missed?

[sil2100]

hm, for unknown reasons I can't get the relevant image to build - both with the old and new package. It just gets stuck on "Creating...". Will investigate.