Skip to content

Add dbref field support for mongodb connector;#8549

Merged
ebyhr merged 1 commit intotrinodb:masterfrom
academy-codex:mr_mongo_dbref_support
Jul 19, 2021
Merged

Add dbref field support for mongodb connector;#8549
ebyhr merged 1 commit intotrinodb:masterfrom
academy-codex:mr_mongo_dbref_support

Conversation

@academy-codex
Copy link
Copy Markdown
Contributor

@academy-codex academy-codex commented Jul 14, 2021

Fixes #3134.

Description: Added explicit case for handling if the type of a field is DBRef. In such case, we extract the id, database name and collection name from the object and convert that into a row. This change enables support at the MongoSession level. With this change, the column starts showing up in the information schema. However, all the values come as null because the writeBlock() method still needs to add the support.

To do this, we need add support at the MongoPageSource. This is done by aadding a case in the writeBlock() method.

@cla-bot cla-bot bot added the cla-signed label Jul 14, 2021
@academy-codex academy-codex requested a review from ebyhr July 14, 2021 05:46
Copy link
Copy Markdown
Member

@ebyhr ebyhr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this!

Please add a test to TestMongoIntegrationSmokeTest. The test should create DBRef using MongoDB Java library and validate the result in Trino SQL.

CI failure cause is

2021-07-14T06:17:29.7767740Z [ERROR] src/main/java/io/trino/plugin/mongodb/MongoPageSource.java:[84] (regexp) RegexpMultiline: Multiple consecutive blank lines
2021-07-14T06:17:29.7769978Z [ERROR] src/main/java/io/trino/plugin/mongodb/MongoSession.java:[656,69] (whitespace) ParenPad: '(' is followed by whitespace.

If you haven't yet configured codestyle in IntelliJ, I would recommend setting https://github.com/trinodb/trino/blob/master/DEVELOPMENT.md#code-style.

Also, please remove ; from the commit message.

@academy-codex
Copy link
Copy Markdown
Contributor Author

academy-codex commented Jul 15, 2021

@ebyhr Please have a look. Made the changes and added the test case. :)

Test case description: The test case ensures that the column of dbref type is poppulated as well as has a value parsed. This is done by asserting that the count of documents where the dbref column is not null is equal to 1, as we have inserted only one document in the test case.

If column was not found or value was not parsed, the test case would fail.

The test case passes and works and confirms that the dbref value is getting poppulated.

@academy-codex academy-codex requested a review from ebyhr July 15, 2021 07:52
@academy-codex
Copy link
Copy Markdown
Contributor Author

Thanks for the review. Let me fix them and get back in a day or two.

@academy-codex academy-codex requested a review from ebyhr July 16, 2021 11:34
Copy link
Copy Markdown
Member

@ebyhr ebyhr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost good to me. Please squash commits into one and update the commit title:
Add DBRef field support for MongoDB connector

@academy-codex academy-codex force-pushed the mr_mongo_dbref_support branch from 90ed9c9 to 7037eca Compare July 16, 2021 16:14
Copy link
Copy Markdown
Member

@ebyhr ebyhr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good except for minor comments.

@academy-codex academy-codex force-pushed the mr_mongo_dbref_support branch from 7037eca to a93a712 Compare July 17, 2021 06:35
@academy-codex
Copy link
Copy Markdown
Contributor Author

@ebyhr all good in this one now ?

@ebyhr ebyhr merged commit 41f8c03 into trinodb:master Jul 19, 2021
@ebyhr
Copy link
Copy Markdown
Member

ebyhr commented Jul 19, 2021

Merged, thanks!

@ebyhr ebyhr mentioned this pull request Jul 19, 2021
11 tasks
@ebyhr ebyhr added this to the 360 milestone Jul 19, 2021
@messaoudi-mounir
Copy link
Copy Markdown

I'm testing latest version of trino (docker image 371). He dosn't support BinData as DBRef $id

example
"referencedObject" : DBRef("rig", BinData(3, "lUDptLrw6lXQGoEMLPJ/iQ==")),

@ebyhr
Copy link
Copy Markdown
Member

ebyhr commented Feb 21, 2022

@messaoudi-mounir It will be supported in #11122.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

Add support for MongoDB Database References

3 participants