Skip to content

DRIVERS-1975: Add showExpandedEvents flag and new change stream events #1220

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
baileympearson merged 12 commits into from
May 31, 2022
Merged

DRIVERS-1975: Add showExpandedEvents flag and new change stream events #1220

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
23d4bb8
Add new change stream events to spec
baileympearson May 11, 2022
8fe1300
reformat
baileympearson May 18, 2022
8dbc64d
fix wording in spec and update tests
baileympearson May 19, 2022
74df404
update server version compatibilty
baileympearson May 19, 2022
a7f0e60
merge master to resolve merge conflicts
baileympearson May 19, 2022
2ec068c
add operationDescription to create/modify events
baileympearson May 23, 2022
6064dea
revert changes to walltime
baileympearson May 23, 2022
3ee0aa3
adjust wording in changelog
baileympearson May 23, 2022
7695114
Address comments on tests
baileympearson May 23, 2022
9c713d2
Address comments on PR
baileympearson May 26, 2022
9b70252
fix: specify Binary instead of UUID for collectionUUID
baileympearson May 31, 2022
81449d4
fix: move comment about binary type
baileympearson May 31, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 75 additions & 7 deletions source/change-streams/change-streams.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ Change Streams
:Status: Accepted
:Type: Standards
:Minimum Server Version: 3.6
:Last Modified: 2022-04-13
:Version: 1.14
:Last Modified: 2022-05-17
:Version: 1.15
Comment thread
benjirewis marked this conversation as resolved.
Outdated

.. contents::

Expand Down Expand Up @@ -137,7 +137,21 @@ If an aggregate command with a ``$changeStream`` stage completes successfully, t
* MUST NOT err when they encounter a new `operationType`. Unknown `operationType`
* values may be represented by "unknown" or the literal string value.
*/
operationType: "insert" | "update" | "replace" | "delete" | "invalidate" | "drop" | "dropDatabase" | "rename";
operationType: "insert"
| "update"
| "replace"
| "delete"
| "invalidate"
| "drop"
| "dropDatabase"
| "rename"
| "createIndexes"
| "dropIndexes"
| "modify"
| "create"
| "shardCollection"
| "refineCollectionShardKey"
| "reshardCollection";
Comment thread
benjirewis marked this conversation as resolved.

Comment on lines +153 to 155

@baileympearson baileympearson May 19, 2022

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

refineCollectionShardKey and reshardCollection events are behind a (currently unsupported) feature flag. I included them here for completeness, but notice that there are no spec tests testing for these new events. If there are no objections, I can file a follow up ticket to add tests for these events once they're available.

@benjirewis benjirewis May 23, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha. Sounds like the feature flag won't be supported until 6.1.0. I agree that we should file a follow-up ticket and add tests once the feature flag can be enabled and drivers can actually test.

/**
* Contains two fields: "db" and "coll" containing the database and
Expand All @@ -155,6 +169,16 @@ If an aggregate command with a ``$changeStream`` stage completes successfully, t
*/
to: Optional<Document>;

/**
* Only present for ops of type 'rename', 'createIndexes', 'dropIndexes', 'shardCollection', 'reshardCollection', 'refineCollectionShardKey'.
Comment thread
benjirewis marked this conversation as resolved.
Outdated
* Only present when the `showExpandedEvents` change stream option is enabled.
*
* An description of the operation.
Comment thread
baileympearson marked this conversation as resolved.
Outdated
*
* @since 6.1.0
*/
operationDescription: Optional<Document>

/**
* Only present for ops of type ‘insert’, ‘update’, ‘replace’, and
* ‘delete’.
Expand All @@ -168,11 +192,8 @@ If an aggregate command with a ``$changeStream`` stage completes successfully, t

/**
* Only present for ops of type ‘update’.
*
* Contains a description of updated and removed fields in this
* operation.
*/
updateDescription: Optional<UpdateDescription>;
updateDescription: Optional<UpdateDescription | Document>;

@abr-egn abr-egn May 25, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a breaking change for the Rust driver (going from a struct to an enum in a public API); from the syntax doc, it looks like the underlying change is the addition of a specialFields field?

@baileympearson baileympearson May 26, 2022

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually this change was unintentional. Originally I wrote the spec update to include the showRawUpdateDescription and showSystemEvents flags but these flags are internal and not intended to be exposed so drivers don't have to support them.

The specialFields and rawUpdateDescription fields on the UpdateDescription are guarded behind showRawUpdateDescription, and so don't need to be added.


/**
* Always present for operations of type 'insert' and 'replace'. Also
Expand Down Expand Up @@ -203,6 +224,35 @@ If an aggregate command with a ``$changeStream`` stage completes successfully, t
* pre-image is unavailable, this will be explicitly set to null.
*/
fullDocumentBeforeChange: Document | null;

/**
* The `wall` time from the oplog entry corresponding to the change event.
*
* Only present when the `showExpandedEvents` change stream option is enabled.
*
* @since 6.0.0
*/
wallTime: Date;

/**
* The `ui` field from the oplog entry corresponding to the change event.
*
* Only present when the `showExpandedEvents` change stream option is enabled and for the following events
* - 'insert'
* - 'update'
* - 'delete'
* - 'createIndexes'
* - 'dropIndexes'
* - 'modify'
* - 'drop'
* - 'create'
* - 'shardCollection'
* - 'reshardCollection'
* - 'refineCollectionShardKey'
*
* @since 6.0.0
*/
collectionUUID: UUID;
}

class UpdateDescription {
Expand Down Expand Up @@ -490,6 +540,22 @@ Driver API
* @note this is an aggregation command option
*/
comment: Optional<any>

/**
* Enables the server to send the 'expanded' list of change stream events.
* The list of events included with this flag set are
Comment thread
baileympearson marked this conversation as resolved.
Outdated
* - createIndexes
* - dropIndexes
* - modify
* - create
* - shardCollection
* - reshardCollection
* - refineCollectionShardKey
*
* @since 6.0.0
* @note this is an option of the change stream pipeline stage
*/
showExpandedEvents: Optional<Boolean>
}

**NOTE:** The set of ``ChangeStreamOptions`` may grow over time.
Expand Down Expand Up @@ -999,3 +1065,5 @@ Changelog
| 2022-04-13 | Support returning point-in-time pre and post-images with |
| | ``fullDocumentBeforeChange`` and ``fullDocument``. |
+------------+------------------------------------------------------------+
| 2022-05-18 | Support new change stream events for C2C replication |
+------------+------------------------------------------------------------+
Loading