Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Send one full update from y-websocket #6650

Merged
merged 16 commits into from
Nov 26, 2024
Merged

Conversation

max-nextcloud
Copy link
Collaborator

@max-nextcloud max-nextcloud commented Nov 14, 2024

Always send full diff to server state.

  • Inline y-websocket, so we can adjust it to our needs.
  • Track the server state by updating a ydoc with all the updates
    received from the server.
  • Compute updates based on that server state.
  • Only send the last update from the queue (as it includes all previous).

Todo:

  • Check why it does not work with document state provided.
  • Confirm it works for new files with content.

@max-nextcloud max-nextcloud requested a review from mejo- as a code owner November 14, 2024 13:50
@max-nextcloud max-nextcloud marked this pull request as draft November 14, 2024 13:54
@max-nextcloud max-nextcloud force-pushed the fix/y-websocket-with-full-updates branch from 2cfb371 to 4675077 Compare November 14, 2024 13:55
@juliusknorr juliusknorr added this to the Nextcloud 31 milestone Nov 14, 2024
@max-nextcloud max-nextcloud self-assigned this Nov 14, 2024
@max-nextcloud max-nextcloud force-pushed the fix/y-websocket-with-full-updates branch 4 times, most recently from 1495034 to f08a168 Compare November 18, 2024 11:01
@max-nextcloud max-nextcloud marked this pull request as ready for review November 18, 2024 11:32
Copy link
Member

@juliusknorr juliusknorr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gave this a first try and basic syncing is still working fine, also some offline online switching on two browser sessions worked as expected.

One small difference I noticed (without reading the code yet in detail):

  • When loading the file it seems we no longer wait until the content is loaded to hide the loading indicator, we start with an empty document and can type before any document state is propagated to us. @max-nextcloud is this expected or can it cause any side effects?

$getStepsSinceVersion = count($querySteps) > 0 ? 0 : $version;
// If there were any queries in the steps send all steps since last save.
$getStepsSinceVersion = count($querySteps) > 0
? $document->getLastSavedVersion()
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this needs to be min($document->getLastSavedVersion(), $version) to make sure we don't skip any steps

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed in the next commit by including the document state that matches $document->getLastSavedVersion

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still using the smaller would mean we have a redundancy in case the client that did the save was out of sync.

@max-nextcloud
Copy link
Collaborator Author

max-nextcloud commented Nov 21, 2024

What's missing:

  • license for y-websocket
  • remove $queue from Editor.vue
  • loading spinner
  • ensure to only store remote state in documentState.

@max-nextcloud max-nextcloud force-pushed the fix/y-websocket-with-full-updates branch from f08a168 to e786a18 Compare November 25, 2024 09:34
Keep an internal ydoc tracking updates that came from the server.
Send updates, that would sync this doc with the current doc state.

Signed-off-by: Max <[email protected]>
The pushed steps are echoed back with all other steps since version immediately.
Processing them reduces the size of the following pushes and syncs.

Signed-off-by: Max <[email protected]>
It has always been publicly called from the PollingBackend.

Signed-off-by: Max <[email protected]>
Updates now include all the local structs
that were not yet received from remote.

No need to compute a separate update message anymore.

Signed-off-by: Max <[email protected]>
Apply document state as a step.
Process it like other steps received from the remote.
In particular include it in the tracking of steps already applied
and set the version accordingly.

Signed-off-by: Max <[email protected]>
This was a very inefficient attempt to resync
that we did not even process on the client side.

Only the steps since the last save may not be enough
to get back in sync.
However we can expand this by including the document state
or storing it as the first step after a save.

Signed-off-by: Max <[email protected]>
Do not process document state from create response.
During create the editor has not been initialized fully
and the cursor position is 0 - which is invalid
as it is not inside a node with inline content.
(It is inside the doc before the initial paragraph.)

This also allows processing document state later on
in order to recover from out of sync situations.
But we do not make use of that yet.

Signed-off-by: Max <[email protected]>
@max-nextcloud max-nextcloud force-pushed the fix/y-websocket-with-full-updates branch 3 times, most recently from ea1a4fb to 91450a2 Compare November 26, 2024 09:23
Copy link
Member

@mejo- mejo- left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments.

src/services/SyncService.js Outdated Show resolved Hide resolved
src/services/Outbox.ts Outdated Show resolved Hide resolved
src/services/Outbox.ts Outdated Show resolved Hide resolved
@max-nextcloud
Copy link
Collaborator Author

  • When loading the file it seems we no longer wait until the content is loaded to hide the loading indicator, we start with an empty document and can type before any document state is propagated to us. @max-nextcloud is this expected or can it cause any side effects?

I cannot reproduce this right now.

@max-nextcloud max-nextcloud force-pushed the fix/y-websocket-with-full-updates branch 2 times, most recently from c0b1bd6 to 3bc7015 Compare November 26, 2024 11:00
max-nextcloud and others added 4 commits November 26, 2024 12:44
Store the steps that need to be send
where we also do the debouncing.

They will be updated whenever there is a new message from y-websocket.

Signed-off-by: Max <[email protected]>
@mejo- mejo- merged commit 1c25d7e into main Nov 26, 2024
66 checks passed
@mejo- mejo- deleted the fix/y-websocket-with-full-updates branch November 26, 2024 14:46
@mejo-
Copy link
Member

mejo- commented Nov 26, 2024

/backport to stable30

@mejo-
Copy link
Member

mejo- commented Nov 26, 2024

/backport to stable29

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: ☑️ Done
Development

Successfully merging this pull request may close these issues.

3 participants