Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Change] DataCache to Match LevelDbStore & RocksDbStore #3709

Open
wants to merge 22 commits into
base: master
Choose a base branch
from
Open

[Change] DataCache to Match LevelDbStore & RocksDbStore #3709

wants to merge 22 commits into from
+364 −418

Conversation

cschuchardt88
Copy link
Member

@cschuchardt88 cschuchardt88 commented Feb 3, 2025 •
edited
Loading

Description

Mostly bug fixes and same data results as the Leveldb and Rocksdb.

Fixes # (issue)

Type of change

  • Optimization (the change is only an optimization)
  • Style (the change is only a code style for better maintenance or standard purpose)
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

image

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules

@cschuchardt88 cschuchardt88 changed the title ['Change'] DataCache to Match LevelDbStore & RocksDbStore [Change] DataCache to Match LevelDbStore & RocksDbStore Feb 3, 2025
@cschuchardt88 cschuchardt88 added Discussion Initial issue state - proposed but not yet accepted Waiting for Review labels Feb 3, 2025
vncoelho
vncoelho reviewed Feb 3, 2025
View reviewed changes
src/Neo.Extensions/ByteArrayComparer.cs Outdated
return y.AsSpan().SequenceCompareTo(x.AsSpan());
return x.AsSpan().SequenceCompareTo(y.AsSpan());

return x.AsSpan().SequenceCompareTo(y.AsSpan()) * _direction;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about submitting the changes in this first in another PR?
Perhaps this can be discussed faster and merged quicker.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe something like

        public int Compare(byte[]? x, byte[]? y)
        {
            if (ReferenceEquals(x, y)) return 0;
            if (x is null) return y is null ? 0 : -_direction;
            if (y is null) return _direction;

            return _direction * x.AsSpan().SequenceCompareTo(y.AsSpan());
        }

Is the length multiplication really needed?

byte[] a = new byte[10];
byte[] b = new byte[20];
ByteArrayComparer comparer = ByteArrayComparer.Default;

Console.WriteLine(comparer.Compare(null, a)); // Expected: -1, Original: -10
Console.WriteLine(comparer.Compare(null, b)); // Expected: -1, Original: -20

Copy link
Member Author

@cschuchardt88 cschuchardt88 Feb 3, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Im sorry, I mean MemorySnapshot was returning all results for null.

shargon
shargon reviewed Feb 3, 2025
View reviewed changes
tests/Neo.UnitTests/Persistence/UT_MemoryStore.cs Outdated Show resolved Hide resolved
tests/Neo.UnitTests/SmartContract/UT_InteropService.cs
@@ -547,7 +546,7 @@ public void TestBlockchain_ListContracts()
Assert.AreEqual(state.Hash, NativeContract.ContractManagement.GetContract(engine.SnapshotCache, state.Hash).Hash);

var list2 = NativeContract.ContractManagement.ListContracts(engine.SnapshotCache);
Assert.AreEqual(list.Count(), list2.Count());
Assert.AreEqual(list.Count(), list2.Count() - 1);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this changed?

Copy link
Member Author

@cschuchardt88 cschuchardt88 Feb 3, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you look up a couple of lines there is a new contract that got added. So the test would be invalid.

10 != 11

image

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test didn't change, this line was before

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was an issue with DataCache being bugged. Let me find out what happening, so I can go into more detail about it.

tests/Neo.UnitTests/Persistence/UT_DataCache.cs
@@ -290,7 +290,7 @@ public void TestFindRange()
Assert.IsTrue(items[0].Value.EqualsTo(value5));
Assert.AreEqual(key4, items[1].Key);
Assert.IsTrue(items[1].Value.EqualsTo(value4));
Assert.AreEqual(2, items.Length);
Assert.AreEqual(3, items.Length);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this changed, and not in the other PR?

Copy link
Member Author

@cschuchardt88 cschuchardt88 Feb 3, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because it from starting key to ending end. It's not in between. We don't skip the 1st one.

// OLD
FindRange([0x00, 0x00, 0x04], [0x00, 0x00, 0x03], Backwards);
// Results: Empty

// NEW
FindRange([0x00, 0x00, 0x04], [0x00, 0x00, 0x03], Backwards);
// Results:
// [0x00, 0x00, 0x04]
// [0x00, 0x00, 0x03]

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you move this fix into the other pr, and here only the DataCache refactor?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this part of the DataCache?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I rewrote all the code. So it would be hard for me to put back the bug.

shargon
shargon reviewed Feb 3, 2025
View reviewed changes
src/Neo/Persistence/StorageKeyEqualityComparer.cs Outdated Show resolved Hide resolved
src/Neo/Persistence/MemoryStore.cs Outdated Show resolved Hide resolved
@shargon
Copy link
Member

shargon commented Feb 6, 2025

Replace the name, and change the logic, makes harder the review, I reverted the name replacement, you can do it in the next pr (i'm not against to) but in the same PR, makes this review harder and slower.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vncoelho vncoelho vncoelho left review comments

@shargon shargon shargon left review comments

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
Discussion Initial issue state - proposed but not yet accepted Waiting for Review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@cschuchardt88 @shargon @vncoelho