Propagate local Devfile changes to the UI #6970

rm3l · 2023-07-17T06:36:23Z

What type of PR is this:
/kind feature
/area dev

What does this PR do / why we need it:

This PR introduces a new /notifications API endpoint that is able to send SSE events to connected clients. Currently, if sends DevfileUpdated events only (along with a Heartbeat comment) whenever the local Devfile is updated on the local filesystem.
The UI listens on this DevfileUpdated event and automatically overwrites the Devfile in the YAML view (and in other related tabs).
This is the first part for propagating changes from the local filesystem to the UI. Another PR will follow after to warn users whenever a DevfileUpdated event is received and allow them to either overwrite or discard the new Devfile.

Which issue(s) this PR fixes:
Fixes acceptance criteria 1 of #6902

PR acceptance criteria:

Unit test
Integration test
Documentation

How to test changes / Special notes to the reviewer:

netlify · 2023-07-17T06:36:28Z

✅ Deploy Preview for odo-docusaurus-preview canceled.

Name	Link
🔨 Latest commit	`0fcad96`
🔍 Latest deploy log	https://app.netlify.com/sites/odo-docusaurus-preview/deploys/64b66093f31da600086ee57f

openshift-ci · 2023-07-17T06:36:29Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

odo-robot · 2023-07-17T06:43:28Z

NoCluster Tests on commit 7b915e4 finished successfully.
View logs: TXT HTML

odo-robot · 2023-07-17T06:43:51Z

OpenShift Unauthenticated Tests on commit 7b915e4 finished successfully.
View logs: TXT HTML

odo-robot · 2023-07-17T06:46:38Z

Unit Tests on commit 7b915e4 finished successfully.
View logs: TXT HTML

odo-robot · 2023-07-17T06:52:00Z

Validate Tests on commit 7b915e4 finished successfully.
View logs: TXT HTML

odo-robot · 2023-07-17T07:03:00Z

Kubernetes Tests on commit 7b915e4 finished successfully.
View logs: TXT HTML

odo-robot · 2023-07-17T07:04:11Z

Windows Tests (OCP) on commit 7b915e4 finished with errors.
View logs: TXT HTML

odo-robot · 2023-07-17T07:08:22Z

OpenShift Tests on commit 7b915e4 finished successfully.
View logs: TXT HTML

odo-robot · 2023-07-17T07:14:11Z

Kubernetes Docs Tests on commit 9fd65cd finished successfully.
View logs: TXT HTML

feloy · 2023-07-17T07:18:45Z

Add '/notifications' endpoint for subscribing to server-sent events

I would prefer to add a watch parameter to the GET /devfile endpoint, to have an interface similar to the Kubernetes API's one.

I had created this PoC a while ago to get events from a watch K8s API's endpoint: https://github.com/feloy/kubernetes-explorer/blob/master/src/app/services/kubernetes.service.ts

ui/src/app/services/sse.service.ts

pkg/apiserver-impl/sse/event.go

rm3l · 2023-07-17T13:36:41Z

Add '/notifications' endpoint for subscribing to server-sent events

I would prefer to add a watch parameter to the GET /devfile endpoint, to have an interface similar to the Kubernetes API's one.

I had created this PoC a while ago to get events from a watch K8s API's endpoint: feloy/kubernetes-explorer@master/src/app/services/kubernetes.service.ts

For the record, and as discussed together earlier today, the main reason for having this /notifications endpoint is to have a single EventSource connection from the client side that will receive any event. This is because browsers enforce a pretty-low limit on the number of persistent connections per domain. We might be quickly limited if we were to keep a permanent connection per type of event.

This does not seem to work because the generated server always responds with application/json, and it is not possible to respond with a different content-type. This reverts commit cf3ce83677649763b8166c4847501c37246dd757.

This reverts commit b985c007a0561edbe185adc3b9582e12aa3f072b.

…events" This reverts commit c5c9033.

…otify Devfile changes

Here, this is being used to automatically reload the Devfile in the YAML view whenever the API server notifies of filesystem changes in the Devfile (and related resources).

This is needed to be able to persist Devfiles from the UI to the filesystem

Co-authored-by: Philippe Martin <[email protected]>

…fileUpdated event data Co-authored-by: Philippe Martin <[email protected]>

… page Co-authored-by: Philippe Martin <[email protected]>

…he home page

sonarqubecloud · 2023-07-18T09:51:53Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
3 Code Smells

No Coverage information
0.0% Duplication

rm3l · 2023-07-18T14:39:41Z


 V  Creating the project "cmd-namespace-test104clj" [290ms]
  [odo]  -  Waiting for the project to come up  ...
  [odo] I0718 04:57:47.375318    1372 kclient.go:237] Checking if "projects" resource is supported
  [odo] I0718 04:57:47.508771    1372 kclient.go:237] Checking if "projects" resource is supported
  [odo] I0718 04:57:47.603715    1372 kclient.go:237] Checking if "projects" resource is supported
  [odo] I0718 04:57:47.796100    1372 kclient.go:237] Checking if "projects" resource is supported
  [odo] I0718 04:57:48.014446    1372 kclient.go:237] Checking if "projects" resource is supported
  [odo] I0718 04:57:48.198345    1372 kclient.go:237] Checking if "projects" resource is supported
  [odo] 
 X  Waiting for the project to come up [1s]
  [odo] I0718 04:57:48.353275    1372 implem.go:114] The path for preference file is C:\Users\Administrator.ANSIBLE-TEST-VS\AppData\Local\Temp\2497318144\preference.yaml
  [odo] I0718 04:57:48.353275    1372 segment.go:264] Checking telemetry enable status
  [odo] I0718 04:57:48.353275    1372 segment.go:282] Sending telemetry disabled by env variable
  [odo]  X  timeout while waiting for project "cmd-namespace-test104clj" to be ready; you can change the timeout preference by running `odo preference set timeout <duration>`
  [FAILED] in [BeforeEach] - C:/Users/Administrator.ANSIBLE-TEST-VS/4377/tests/helper/helper_cmd_wrapper.go:101 @ 07/18/23 04:57:48.423
  Deleting project: cmd-namespace-test104axt
  Running oc.exe with args [oc delete project cmd-namespace-test104axt --wait=false] and odo env: []
  [oc] project.project.openshift.io "cmd-namespace-test104axt" deleted
  Setting current dir to: C:\Users\Administrator.ANSIBLE-TEST-VS\4377\tests\integration
  Deleting dir: C:\Users\Administrator.ANSIBLE-TEST-VS\AppData\Local\Temp\1538575290
  Deleting dir: C:\Users\Administrator.ANSIBLE-TEST-VS\AppData\Local\Temp\2497318144
  << Timeline

  [FAILED] No future change is possible.  Bailing out early after 1.845s.
  Running odo.exe with args [odo create project cmd-namespace-test104clj -w -v4] and odo env: []
  Expected
      <int>: 1
  to match exit code:
      <int>: 0
  In [BeforeEach] at: C:/Users/Administrator.ANSIBLE-TEST-VS/4377/tests/helper/helper_cmd_wrapper.go:101 @ 07/18/23 04:57:48.423
...
...
  [odo]   __
  [odo]  /  \__     Developing using the "rhgepc" Devfile
  [odo]  \__/  \    Namespace: cmd-dev-api-server-test169ryv
  [odo]  /  \__/    odo version: v3.12.0
  [odo]  \__/
  [odo] 
  [odo] - Running on the cluster in Dev mode
  [odo] I0718 05:02:55.020410    6328 content.go:48] converted devfile YAML to JSON
  [odo] I0718 05:02:55.020410    6328 apiVersion.go:60] devfile schemaVersion: '2.2.0'
  [odo] I0718 05:02:55.020410    6328 helper.go:56] devfile apiVersion '2.2.0' is supported
  [odo] I0718 05:02:55.023346    6328 schema.go:61] validated devfile schema
  [odo] I0718 05:02:55.023953    6328 writer.go:55] devfile yaml created at: 'devfile.yaml'
  [odo] I0718 05:02:55.046665    6328 starterserver.go:82] API Server started at localhost:20000/api/v1
  [odo] I0718 05:02:55.046786    6328 kubedev.go:86] Creating new adapter
  [odo] I0718 05:02:55.046786    6328 kubedev.go:94] Creating inner-loop resources for the component
  [odo] I0718 05:02:55.046786    6328 watch.go:101] starting WatchAndPush, path: C:\Users\Administrator.ANSIBLE-TEST-VS\AppData\Local\Temp\3384580862, component: rhgepc, ignores [.git .odo .odo\odo-file-index.json]
  [odo] I0718 05:02:55.047386    6328 starterserver.go:102] Stopping the API server; encountered error: listen tcp :20000: bind: Only one usage of each socket address (protocol/network address/port) is normally permitted.
...

Summarizing 2 Failures:
  [FAIL] odo create/delete/list/set namespace/project tests delete project when force-deleting a valid project [BeforeEach] should successfully delete the project synchronously with --wait
  C:/Users/Administrator.ANSIBLE-TEST-VS/4377/tests/helper/helper_cmd_wrapper.go:101
  [FAIL] odo dev command with api server tests when the component is bootstrapped when odo dev is run with --api-server flag [BeforeEach] when /instance endpoint is DELETEd should terminate the dev session
  C:/Users/Administrator.ANSIBLE-TEST-VS/4377/tests/helper/helper_run.go:50

Ran 382 of 940 Specs in 1261.460 seconds
FAIL! -- 380 Passed | 2 Failed | 0 Pending | 558 Skipped

Timeout when waiting for Project creation on OpenShift - unrelated to the changes here, but let's keep an eye out for this.
Ports conflicting issue on Windows (Ports conflict when running api server tests on Windows #6939)

/override windows-integration-test/Windows-test

openshift-ci · 2023-07-18T14:39:45Z

@rm3l: Overrode contexts on behalf of rm3l: windows-integration-test/Windows-test

In response to this:


V  Creating the project "cmd-namespace-test104clj" [290ms]
 [odo]  -  Waiting for the project to come up  ...
 [odo] I0718 04:57:47.375318    1372 kclient.go:237] Checking if "projects" resource is supported
 [odo] I0718 04:57:47.508771    1372 kclient.go:237] Checking if "projects" resource is supported
 [odo] I0718 04:57:47.603715    1372 kclient.go:237] Checking if "projects" resource is supported
 [odo] I0718 04:57:47.796100    1372 kclient.go:237] Checking if "projects" resource is supported
 [odo] I0718 04:57:48.014446    1372 kclient.go:237] Checking if "projects" resource is supported
 [odo] I0718 04:57:48.198345    1372 kclient.go:237] Checking if "projects" resource is supported
 [odo] 
X  Waiting for the project to come up [1s]
 [odo] I0718 04:57:48.353275    1372 implem.go:114] The path for preference file is C:\Users\Administrator.ANSIBLE-TEST-VS\AppData\Local\Temp\2497318144\preference.yaml
 [odo] I0718 04:57:48.353275    1372 segment.go:264] Checking telemetry enable status
 [odo] I0718 04:57:48.353275    1372 segment.go:282] Sending telemetry disabled by env variable
 [odo]  X  timeout while waiting for project "cmd-namespace-test104clj" to be ready; you can change the timeout preference by running `odo preference set timeout <duration>`
 [FAILED] in [BeforeEach] - C:/Users/Administrator.ANSIBLE-TEST-VS/4377/tests/helper/helper_cmd_wrapper.go:101 @ 07/18/23 04:57:48.423
 Deleting project: cmd-namespace-test104axt
 Running oc.exe with args [oc delete project cmd-namespace-test104axt --wait=false] and odo env: []
 [oc] project.project.openshift.io "cmd-namespace-test104axt" deleted
 Setting current dir to: C:\Users\Administrator.ANSIBLE-TEST-VS\4377\tests\integration
 Deleting dir: C:\Users\Administrator.ANSIBLE-TEST-VS\AppData\Local\Temp\1538575290
 Deleting dir: C:\Users\Administrator.ANSIBLE-TEST-VS\AppData\Local\Temp\2497318144
 << Timeline

 [FAILED] No future change is possible.  Bailing out early after 1.845s.
 Running odo.exe with args [odo create project cmd-namespace-test104clj -w -v4] and odo env: []
 Expected
     <int>: 1
 to match exit code:
     <int>: 0
 In [BeforeEach] at: C:/Users/Administrator.ANSIBLE-TEST-VS/4377/tests/helper/helper_cmd_wrapper.go:101 @ 07/18/23 04:57:48.423
...
...
 [odo]   __
 [odo]  /  \__     Developing using the "rhgepc" Devfile
 [odo]  \__/  \    Namespace: cmd-dev-api-server-test169ryv
 [odo]  /  \__/    odo version: v3.12.0
 [odo]  \__/
 [odo] 
 [odo] - Running on the cluster in Dev mode
 [odo] I0718 05:02:55.020410    6328 content.go:48] converted devfile YAML to JSON
 [odo] I0718 05:02:55.020410    6328 apiVersion.go:60] devfile schemaVersion: '2.2.0'
 [odo] I0718 05:02:55.020410    6328 helper.go:56] devfile apiVersion '2.2.0' is supported
 [odo] I0718 05:02:55.023346    6328 schema.go:61] validated devfile schema
 [odo] I0718 05:02:55.023953    6328 writer.go:55] devfile yaml created at: 'devfile.yaml'
 [odo] I0718 05:02:55.046665    6328 starterserver.go:82] API Server started at localhost:20000/api/v1
 [odo] I0718 05:02:55.046786    6328 kubedev.go:86] Creating new adapter
 [odo] I0718 05:02:55.046786    6328 kubedev.go:94] Creating inner-loop resources for the component
 [odo] I0718 05:02:55.046786    6328 watch.go:101] starting WatchAndPush, path: C:\Users\Administrator.ANSIBLE-TEST-VS\AppData\Local\Temp\3384580862, component: rhgepc, ignores [.git .odo .odo\odo-file-index.json]
 [odo] I0718 05:02:55.047386    6328 starterserver.go:102] Stopping the API server; encountered error: listen tcp :20000: bind: Only one usage of each socket address (protocol/network address/port) is normally permitted.
...

Summarizing 2 Failures:
 [FAIL] odo create/delete/list/set namespace/project tests delete project when force-deleting a valid project [BeforeEach] should successfully delete the project synchronously with --wait
 C:/Users/Administrator.ANSIBLE-TEST-VS/4377/tests/helper/helper_cmd_wrapper.go:101
 [FAIL] odo dev command with api server tests when the component is bootstrapped when odo dev is run with --api-server flag [BeforeEach] when /instance endpoint is DELETEd should terminate the dev session
 C:/Users/Administrator.ANSIBLE-TEST-VS/4377/tests/helper/helper_run.go:50

Ran 382 of 940 Specs in 1261.460 seconds
FAIL! -- 380 Passed | 2 Failed | 0 Pending | 558 Skipped

Timeout when waiting for Project creation on OpenShift - unrelated to the changes here, but let's keep an eye out for this.
Ports conflicting issue on Windows (Ports conflict when running api server tests on Windows #6939)

/override windows-integration-test/Windows-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. Required by Prow. label Jul 17, 2023

openshift-ci bot added the kind/feature Categorizes issue as a feature request. For PRs, that means that the PR is the implementation label Jul 17, 2023

openshift-ci bot added the area/dev Issues or PRs related to `odo dev` label Jul 17, 2023

rm3l force-pushed the 6902-propagate-changes-from-the-local-devfile-to-the-odo-dev-web-ui-and-vice-versa branch from 878cbc3 to 33f8ba3 Compare July 17, 2023 07:45

feloy reviewed Jul 17, 2023

View reviewed changes

ui/src/app/services/sse.service.ts Outdated Show resolved Hide resolved

feloy reviewed Jul 17, 2023

View reviewed changes

pkg/apiserver-impl/sse/event.go Show resolved Hide resolved

rm3l mentioned this pull request Jul 17, 2023

Generate static UI files from within a container #6972

Merged

3 tasks

openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. Required by Prow. label Jul 18, 2023

rm3l added 10 commits July 18, 2023 11:16

Add '/notifications' endpoint for subscribing to server-sent events

3285153

Generate server and client

a61a8be

Try implementing the notification service endpoint

8655073

Revert "Try implementing the notification service endpoint"

4aa09f7

This does not seem to work because the generated server always responds with application/json, and it is not possible to respond with a different content-type. This reverts commit cf3ce83677649763b8166c4847501c37246dd757.

Revert "Generate server and client"

5c004de

This reverts commit b985c007a0561edbe185adc3b9582e12aa3f072b.

Revert "Add '/notifications' endpoint for subscribing to server-sent …

d686826

…events" This reverts commit c5c9033.

Implement 'GET /notifications' SSE endpoint and logic to detect and n…

ce3b2f5

…otify Devfile changes

Leverage EventSource to subscribe to Server Sent Events

7c0d0ba

Here, this is being used to automatically reload the Devfile in the YAML view whenever the API server notifies of filesystem changes in the Devfile (and related resources).

Add Preference Client to apiserver CLI

856fb30

This is needed to be able to persist Devfiles from the UI to the filesystem

Add E2E test case

812c828

rm3l and others added 4 commits July 18, 2023 11:16

fixup! Leverage EventSource to subscribe to Server Sent Events

6cd5739

Co-authored-by: Philippe Martin <[email protected]>

Limit the round-trips by sending the whole Devfile content in the Dev…

a5ea27e

…fileUpdated event data Co-authored-by: Philippe Martin <[email protected]>

[Cypress] Make sure to wait for APi responses after visiting the home…

aa0bbfe

… page Co-authored-by: Philippe Martin <[email protected]>

Generate static UI

9083f80

rm3l force-pushed the 6902-propagate-changes-from-the-local-devfile-to-the-odo-dev-web-ui-and-vice-versa branch from 26f5667 to 9083f80 Compare July 18, 2023 09:18

openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. Required by Prow. label Jul 18, 2023

fixup! [Cypress] Make sure to wait for APi responses after visiting t…

0fcad96

…he home page

feloy approved these changes Jul 18, 2023

View reviewed changes

openshift-ci bot assigned feloy Jul 18, 2023

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. Required by Prow. label Jul 18, 2023

rm3l changed the title ~~[WIP] Propagate local Devfile changes to the UI~~ Propagate local Devfile changes to the UI Jul 18, 2023

rm3l marked this pull request as ready for review July 18, 2023 14:21

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. Required by Prow. label Jul 18, 2023

openshift-ci bot requested review from anandrkskd and feloy July 18, 2023 14:22

rm3l removed the request for review from anandrkskd July 18, 2023 14:23

rm3l added this to the v3.13.0 🚀 milestone Jul 18, 2023

openshift-merge-robot merged commit 8c9bcde into redhat-developer:main Jul 18, 2023

rm3l deleted the 6902-propagate-changes-from-the-local-devfile-to-the-odo-dev-web-ui-and-vice-versa branch July 18, 2023 15:09

rm3l mentioned this pull request Jul 18, 2023

Propagate changes from the local Devfile to the odo dev web UI #6902

Closed

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Propagate local Devfile changes to the UI #6970

Propagate local Devfile changes to the UI #6970

rm3l commented Jul 17, 2023 •

edited

Loading

netlify bot commented Jul 17, 2023 •

edited

Loading

openshift-ci bot commented Jul 17, 2023

odo-robot bot commented Jul 17, 2023 •

edited

Loading

odo-robot bot commented Jul 17, 2023 •

edited

Loading

odo-robot bot commented Jul 17, 2023 •

edited

Loading

odo-robot bot commented Jul 17, 2023 •

edited

Loading

odo-robot bot commented Jul 17, 2023 •

edited

Loading

odo-robot bot commented Jul 17, 2023 •

edited

Loading

odo-robot bot commented Jul 17, 2023 •

edited

Loading

odo-robot bot commented Jul 17, 2023 •

edited

Loading

feloy commented Jul 17, 2023

rm3l commented Jul 17, 2023

sonarqubecloud bot commented Jul 18, 2023

rm3l commented Jul 18, 2023

openshift-ci bot commented Jul 18, 2023

Propagate local Devfile changes to the UI #6970

Propagate local Devfile changes to the UI #6970

Conversation

rm3l commented Jul 17, 2023 • edited Loading

netlify bot commented Jul 17, 2023 • edited Loading

✅ Deploy Preview for odo-docusaurus-preview canceled.

openshift-ci bot commented Jul 17, 2023

odo-robot bot commented Jul 17, 2023 • edited Loading

odo-robot bot commented Jul 17, 2023 • edited Loading

odo-robot bot commented Jul 17, 2023 • edited Loading

odo-robot bot commented Jul 17, 2023 • edited Loading

odo-robot bot commented Jul 17, 2023 • edited Loading

odo-robot bot commented Jul 17, 2023 • edited Loading

odo-robot bot commented Jul 17, 2023 • edited Loading

odo-robot bot commented Jul 17, 2023 • edited Loading

feloy commented Jul 17, 2023

rm3l commented Jul 17, 2023

sonarqubecloud bot commented Jul 18, 2023

rm3l commented Jul 18, 2023

openshift-ci bot commented Jul 18, 2023

rm3l commented Jul 17, 2023 •

edited

Loading

netlify bot commented Jul 17, 2023 •

edited

Loading

odo-robot bot commented Jul 17, 2023 •

edited

Loading

odo-robot bot commented Jul 17, 2023 •

edited

Loading

odo-robot bot commented Jul 17, 2023 •

edited

Loading

odo-robot bot commented Jul 17, 2023 •

edited

Loading

odo-robot bot commented Jul 17, 2023 •

edited

Loading

odo-robot bot commented Jul 17, 2023 •

edited

Loading

odo-robot bot commented Jul 17, 2023 •

edited

Loading

odo-robot bot commented Jul 17, 2023 •

edited

Loading