Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support telemetry batching and move WebSocket handling to worker #7391

Merged
merged 26 commits into from
Jan 29, 2024

Conversation

akhenry
Copy link
Contributor

@akhenry akhenry commented Jan 19, 2024

Closes #7390

Describe your changes:

  • Implements new BatchingWebSocket class which handles a lot of boilerplate involved in initiating and managing WebSockets from inside a worker. This is intended as a convenience for Telemetry Providers. More detailed documentation is coming here.
  • Modifies the Telemetry API to allow views to optionally specify a strategy when subscribing to telemetry. The supported strategies are:
    • latest - This is the default strategy. The latest strategy will always return the most recent value for a given telemetry point. Depending upon the implementation details of the telemetry provider, for performance reasons a telemetry provider may drop intermediate telemetry values in order to prioritize the latest value when using the latest strategy. This is appropriate for Alphanumeric views, LAD Tables, etc.
    • batch - The batch strategy invokes subscription callbacks with an array containing all of the telemetry values received since the last batch. This strategy is used by plots and tables, where intermediate values matter and should not typically be discarded.
  • Modifies plots and tables to accept a batch of telemetry values as an array, where supported by the telemetry provider.

Note on the worker implementation

The worker implementation uses ES6 classes to organize the code, but because of the method used to inline the worker all of the classes have to be in the same file. I am open to suggestions on how to inline workers in a cleaner way.

Note on backwards compatibility

The changes in this PR are intended to be entirely backwards compatible. The BatchingWebSocket and the new batching subscribe strategy are opt-in. Any incompatibilities discovered should be treated as bugs.

All Submissions:

  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?
  • Is this change backwards compatible? For example, developers won't need to change how they are calling the API or how they've extended core plugins such as Tables or Plots.

Author Checklist

  • Changes address original issue?
  • Tests included and/or updated with changes? Unit tests included. Issue filed for additional testing backfill here.
  • Has this been smoke tested?
  • Have you associated this PR with a type: label? Note: this is not necessarily the same as the original issue.
  • Have you associated a milestone with this PR? Note: leave blank if unsure.
  • Is this a breaking change to be called out in the release notes?
  • Testing instructions included in associated issue OR is this a dependency/testcase change?

Reviewer Checklist

  • Changes appear to address issue?
  • Reviewer has tested changes by following the provided instructions?
  • Changes appear not to be breaking changes?
  • Appropriate automated tests included?
  • Code style and in-line documentation are appropriate?

@akhenry akhenry marked this pull request as draft January 19, 2024 19:54
src/api/telemetry/WebSocketWorker.js Dismissed Show dismissed Hide dismissed
src/api/telemetry/WebSocketWorker.js Dismissed Show resolved Hide resolved
src/api/telemetry/WebSocketWorker.js Dismissed Show resolved Hide resolved
src/api/telemetry/WebSocketWorker.js Dismissed Show resolved Hide resolved
Copy link

codecov bot commented Jan 19, 2024

Codecov Report

Attention: 161 lines in your changes are missing coverage. Please review.

Comparison is base (0eea2e0) 55.82% compared to head (cdf549b) 55.52%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7391      +/-   ##
==========================================
- Coverage   55.82%   55.52%   -0.31%     
==========================================
  Files         666      668       +2     
  Lines       26514    26701     +187     
  Branches     2585     2585              
==========================================
+ Hits        14801    14825      +24     
- Misses      10998    11160     +162     
- Partials      715      716       +1     
Flag Coverage Δ *Carryforward flag
e2e-full 41.75% <ø> (-0.07%) ⬇️ Carriedforward from b93647b
e2e-stable 59.10% <20.90%> (-0.69%) ⬇️
unit 48.60% <26.36%> (-0.26%) ⬇️

*This pull request uses carry forward flags. Click here to find out more.

Files Coverage Δ
src/api/telemetry/TelemetryCollection.js 84.84% <100.00%> (+0.15%) ⬆️
src/plugins/plot/configuration/Model.js 92.85% <ø> (ø)
src/plugins/plot/configuration/PlotSeries.js 85.12% <100.00%> (+0.15%) ⬆️
src/plugins/remoteClock/RemoteClock.js 84.78% <100.00%> (+1.80%) ⬆️
src/api/telemetry/TelemetryAPI.js 89.21% <91.66%> (-0.15%) ⬇️
src/api/telemetry/BatchingWebSocket.js 2.85% <2.85%> (ø)
src/api/telemetry/WebSocketWorker.js 0.00% <0.00%> (ø)

... and 2 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0eea2e0...cdf549b. Read the comment docs.

@akhenry akhenry added this to the Target:4.0.0 milestone Jan 19, 2024
@akhenry akhenry changed the title Subscriptions support array callbacks Support telemetry batching Jan 21, 2024
@akhenry akhenry changed the title Support telemetry batching Support telemetry batching and move WebSocket handling to worker Jan 21, 2024
@akhenry akhenry marked this pull request as ready for review January 21, 2024 03:21
@akhenry akhenry requested a review from davetsay January 21, 2024 03:22
Copy link
Contributor

@scottbell scottbell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work @akhenry! I'm putting a general code review here, but will check this code against YAMCS in the other PR for more thorough testing. I had a few questions below.

src/api/telemetry/BatchingWebSocket.js Show resolved Hide resolved
src/api/telemetry/BatchingWebSocket.js Outdated Show resolved Hide resolved
src/api/telemetry/BatchingWebSocket.js Show resolved Hide resolved
src/api/telemetry/TelemetryAPI.js Show resolved Hide resolved
src/api/telemetry/TelemetryAPI.js Outdated Show resolved Hide resolved
src/api/telemetry/WebSocketWorker.js Dismissed Show resolved Hide resolved
src/api/telemetry/WebSocketWorker.js Show resolved Hide resolved
src/plugins/plot/configuration/PlotSeries.js Show resolved Hide resolved
src/plugins/remoteClock/RemoteClock.js Outdated Show resolved Hide resolved
src/api/telemetry/WebSocketWorker.js Show resolved Hide resolved
@akhenry
Copy link
Contributor Author

akhenry commented Jan 24, 2024

Review comments addressed, just doing some regression testing...

@akhenry
Copy link
Contributor Author

akhenry commented Jan 24, 2024

Review comments addressed, just doing some regression testing...

Done, ready for re-review.

Copy link
Contributor

@scottbell scottbell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested again and works great!

@akhenry akhenry removed the request for review from davetsay January 26, 2024 19:38
scottbell
scottbell approved these changes Jan 26, 2024
@unlikelyzero
Copy link
Collaborator

Blocked on legit test failure

@unlikelyzero unlikelyzero added pr:e2e:couchdb npm run test:e2e:couchdb pr:e2e:perf Trigger perf tests labels Jan 29, 2024
@github-actions github-actions bot removed pr:e2e:perf Trigger perf tests pr:e2e:couchdb npm run test:e2e:couchdb labels Jan 29, 2024
@unlikelyzero unlikelyzero added notable_change A change which should be noted in the changelog type:feature Feature. Required intentional design pr:e2e:couchdb npm run test:e2e:couchdb pr:e2e:perf Trigger perf tests and removed type:enhancement labels Jan 29, 2024
@github-actions github-actions bot removed the pr:e2e:couchdb npm run test:e2e:couchdb label Jan 29, 2024
@unlikelyzero unlikelyzero merged commit 5c21c34 into master Jan 29, 2024
65 of 73 checks passed
@unlikelyzero unlikelyzero deleted the subscriptions-support-array-callbacks branch January 29, 2024 23:17
@github-actions github-actions bot removed the pr:e2e:perf Trigger perf tests label Jan 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
notable_change A change which should be noted in the changelog type:feature Feature. Required intentional design
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Provide support for worker-based WebSocket management
3 participants