Notebook Culling for VSCode/CodeServer style notebooks

[tu-infra-tests]

/kind feature

Why you need this feature:
We use VSCode/CodeServer style notebooks (https://github.com/coder/code-server) where it is critical to have a mechanism to terminate underutilized resources. If such a mechanism does not exist, it can be easy to have notebooks bound to expensive GPU resources even while no one is actively using them.

Describe the solution you'd like:
For Jupyter style notebooks there is a notebook culling feature where underutilized resources can be automatically terminated by the notebook controller if unused for a certain amount of time. It does not appear this feature works when using VSCode style notebooks, but the same type of feature would solve this problem as well.

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

[apo-ger]

I've sent a PR to extend the existing implementation of the culling controller (#6807 and proposals/20220121-jupyter-notebook-idleness.md) to extract Istio HTTP metrics from VSCode and RStudio Νotebook Pods and use them to decide on their idleness.

[juliusvonkohout]

@apo-ger will this also work for vscode and rstudio within jupyterlab via jupyter-server-proxy?

[juliusvonkohout]

#7186 is related

[wjhhuizi]

I believe This is due to the Culling controller is only considering notebook kernel's last_activity but not terminal's.

Here's my proposal for fixing this issue:

User shall be able to config Culling controller to consider terminals workload by adding options to the notebook-controller-config, for example:

CULL_OPTION: KERNEL | TERMINAL | BOTH | NETWORK
--------
# KERNEL corresponds to: `api/kernels`
# TERMINAL corresponds to: `api/terminals`
# BOTH took above 2 into consideration
# NETWORK corresponds to: `api/status`

[wadhah101]

@wjhhuizi api/status doesn't work due to
components/proposals/20220121-jupyter-notebook-idleness.md

check the ## Alternative Considered Approaches section

[wjhhuizi]

@wjhhuizi api/status doesn't work due to components/proposals/20220121-jupyter-notebook-idleness.md

check the ## Alternative Considered Approaches section

It seems that proposals was created 2 years ago, and I tested running a long execution code block in the notebook then close the browser tab, it seems the last_activity from api/status is still updating properly.

Code Tested:

import time
for i in range(60):
    print (i)
    time.sleep(1)

[wjhhuizi]

Okay it seems if I remove the line print (i) then the last_activity won't get updated for neither api/kernels nor api/status, however the execution_state in api/kernels would remains busy. This might be point I think...

[wjhhuizi]

Updating proposal to

CULL_OPTION: KERNEL | TERMINAL | BOTH | NETWORK | NETWORK+
--------
# KERNEL corresponds to: `api/kernels`
# TERMINAL corresponds to: `api/terminals`
# BOTH took above 2 into consideration
# NETWORK corresponds to: `api/status`
# NETWORK+ took consideration on both api/status and KERNEL idleness

[juliusvonkohout]

We will probably just replace this one here with #7156

[em-le-ts]

+1