Skip to content

Use std::map for computing json object hash#11875

Merged
ggreenway merged 5 commits intoenvoyproxy:masterfrom
greenhouse-org:fix-json-hash
Jul 8, 2020
Merged

Use std::map for computing json object hash#11875
ggreenway merged 5 commits intoenvoyproxy:masterfrom
greenhouse-org:fix-json-hash

Conversation

@wrowe
Copy link
Contributor

@wrowe wrowe commented Jul 2, 2020

Commit Message: Use std::map for computing json object hash
Additional Description:

  • Reviewing all unordered_map, we discovered this test was flawed,
    because it assumed ordering of unordered map elements. This is true
    of libc++ and msvc, but not consistent with one another, and not
    true absl::node_hash_map at all, resulting in test failing 50%
    of the time when converted to absl.

  • Reviewing the RFC, the hash comparison itself did not follow json
    which insists they are compared without respect to order.

  • Note this could be updated with the equivalent absl collection type,
    if and when all std::map's are refactored, but was out of scope to
    addressing std::unordered_map.

Co-authored-by: Sunjay Bhatia sunjayb@vmware.com
Co-authored-by: William A Rowe Jr wrowe@vmware.com
Signed-off-by: Sunjay Bhatia sunjayb@vmware.com
Signed-off-by: William A Rowe Jr wrowe@vmware.com

For an explanation of how to fill out the fields, please see the relevant section
in PULL_REQUESTS.md

Risk Level: low
Testing: local, windows & linux docker containers
Docs Changes: n/a
Release Notes: n/a

- Reviewing all unordered_map, we discovered this test was flawed,
  because it assumed ordering of map elements. This is true of
  libc++ and msvc, but not consistant with one another, and not
  true absl::node_hash_map at all, resulting in test failing 50%
  of the time.

- Reviewing the RFC, the hash comparison itself did not follow json.

- Note this could be updated with the equivilant absl collection type
  if and when all std::map's are refactored.

Co-authored-by: Sunjay Bhatia <sunjayb@vmware.com>
Co-authored-by: William A Rowe Jr <wrowe@vmware.com>
Signed-off-by: Sunjay Bhatia <sunjayb@vmware.com>
Signed-off-by: William A Rowe Jr <wrowe@vmware.com>
Rekick for test/integration/http2_integration_test flake on master

Signed-off-by: William A Rowe Jr <wrowe@vmware.com>
ggreenway
ggreenway previously approved these changes Jul 6, 2020
@ggreenway ggreenway self-assigned this Jul 6, 2020
ObjectSharedPtr json2 = Factory::loadFromString("{\"value2\": -12.3, \"value1\": 10.5}");
ObjectSharedPtr json3 = Factory::loadFromString(" { \"value2\": -12.3, \"value1\": 10.5} ");
EXPECT_NE(json1->hash(), json2->hash());
EXPECT_EQ(json1->hash(), json2->hash());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should these tests be stricter and verify that the hash has a specific value?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could to make it v strict, though changing the underlying container/hash algorithm changes could make this test brittle and require repeated changes

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed.

possible nit: there are no remaining EXPECT_NE for the hash function. It may make sense to add something like:

ObjectSharedPtr json4 = Factory::loadFromString("{"value1": 10.5}");
EXPECT_EQ(json1->hash(), json4->hash());

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point! will add that test assertion

Signed-off-by: Sunjay Bhatia <sunjayb@vmware.com>
Also add comments clarifying test cases

Signed-off-by: Sunjay Bhatia <sunjayb@vmware.com>
Co-authored-by: Sunjay Bhatia <sunjayb@vmware.com>
Co-authored-by: William A Rowe Jr <wrowe@vmware.com>
Signed-off-by: Sunjay Bhatia <sunjayb@vmware.com>
Signed-off-by: William A Rowe Jr <wrowe@vmware.com>
@ggreenway ggreenway merged commit 2afd9db into envoyproxy:master Jul 8, 2020
@wrowe wrowe deleted the fix-json-hash branch July 8, 2020 17:14
scheler pushed a commit to scheler/envoy that referenced this pull request Aug 4, 2020
- Reviewing all unordered_map, we discovered this test was flawed,
  because it assumed ordering of map elements. This is true of
  libc++ and msvc, but not consistant with one another, and not
  true absl::node_hash_map at all, resulting in test failing 50%
  of the time.

- Reviewing the RFC, the hash comparison itself did not follow json.

- Note this could be updated with the equivilant absl collection type
  if and when all std::map's are refactored.

Co-authored-by: Sunjay Bhatia <sunjayb@vmware.com>
Co-authored-by: William A Rowe Jr <wrowe@vmware.com>
Signed-off-by: Sunjay Bhatia <sunjayb@vmware.com>
Signed-off-by: William A Rowe Jr <wrowe@vmware.com>
Signed-off-by: scheler <santosh.cheler@appdynamics.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants