Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Aggregate supports group by date fields #5538

Merged
merged 4 commits into from
Apr 27, 2019

Conversation

davimacedo
Copy link
Member

This PR aims to affirm that Parse Server currently DOES support group by date fields.

I started working on this because I've just setup a brand new environment for contributing to Parse Server and the test below keeps failing to me when I run coverage npm script on master:
https://github.com/parse-community/parse-server/blob/master/spec/ParseQuery.Aggregate.spec.js#L381

I found out that the group by date fields actually works for normal cases and only does not work when you have dirty data, meaning a non date value in a field that is expected to be a date. I was only able to simulate this problem by doing:

  1. I dropped the test database
  2. I used the api to save a data in the TestObject class with value '' in the field dateField
  3. I run the tests

After this, I've just dropped the database again and run the tests. Then the group by was working again.

Probably this test is passing in Travis and maintainers' environments because they have legacy data in their databases. According to my tests, if any of them drop the current test database and run the tests again, this test will fail.

In summary, what I did in this PR:

@dplewis do you mind to take a look on this and check if you agree?

@davimacedo davimacedo closed this Apr 26, 2019
@dplewis
Copy link
Member

dplewis commented Apr 26, 2019

Did you figure something out?

@davimacedo
Copy link
Member Author

The tests are passing in my environment but didn't pass in Travis. I'm trying to understand why.

@davimacedo davimacedo reopened this Apr 26, 2019
.then(results => {
const counts = results.map(result => result.count);
expect(counts.length).toBe(3);
expect(counts.sort()).toEqual([1, 2, 4]);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If group by date works, wouldn't the results be obj1 + obj3 is one group and obj2 is another.

count would be [1, 2] what does the 4 come from?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4 come from the default data that is added before each test. It returns [ objectId: undefined, count: 4]

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dplewis
Copy link
Member

dplewis commented Apr 26, 2019

Which version of mongo are you running locally?

@davimacedo
Copy link
Member Author

Antonios-MacBook-Pro-2:parse-server davimacedo$ mongod --version
db version v4.0.9
git version: fc525e2d9b0e4bceff5c2201457e564362909765
allocator: system
modules: none
build environment:
distarch: x86_64
target_arch: x86_64

@davimacedo
Copy link
Member Author

davimacedo commented Apr 26, 2019

I think in this new version I'm running mongo works for undefined values, but it doesn't for version 4.0.4. I just added the match stage to ensure that the undefined dates (the ones added before each test) are not processed. Let's see if it will pass in Travis now.

@davimacedo
Copy link
Member Author

It passed. Conclusion:

  • for mongo 4.0.9 it works except in the case that you have values that are not date, but works even if you have values that are undefined
  • for mongo <=4.0.4 it works except in the case that you have values that are not date or are undefined

In both scenario you can add a match stage to filter out the data before the groupBy.

@@ -379,18 +379,23 @@ describe('Parse.Query Aggregate testing', () => {
});

it_exclude_dbs(['postgres'])(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should work on postgres right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure. I will check.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The group by date field works, but the test fails because the $exists is not implemented for match stage on postgres. I included an additional test only for postgres.

@dplewis dplewis changed the title it actually supports group by date fields Aggregate supports group by date fields Apr 27, 2019
@dplewis
Copy link
Member

dplewis commented Apr 27, 2019

@davimacedo Thanks!

@dplewis dplewis merged commit db994ed into parse-community:master Apr 27, 2019
UnderratedDev pushed a commit to UnderratedDev/parse-server that referenced this pull request Mar 21, 2020
* it actually supports group by date fields

* Changing the field name again to see Travis logs

* Adding match stage to the test

* Adding test for group by date fields on postgres
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants