Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Header version cleanup #2809

Merged
merged 3 commits into from
May 8, 2019
Merged

Conversation

antiochp
Copy link
Member

@antiochp antiochp commented May 7, 2019

  • Introduce HeaderVersion(u16) for type safety around header versioning
  • Cleanup pow ser/deser (header version is unused)

Header version is converted to the underlying u16 during json serialization for the api.
Header version is serialized/deserialized as the inner u16 vie read/write.
Default header version is 1 (currently the only version in use).
pow ser/deser was passing an unused version around - cleaned this up so we can now implement Readable/Writeable traits on ProofOfWork struct.

This is (should be) purely a refactor - no change to behavior or functionality.

I went down this rabbit hole because I was investigating "protocol version" specific serialization/deserialization and discovered we are already using "versioning" for block headers (even if not for the same reason).

#[derive(Clone, Copy, Debug, Eq, PartialEq, Serialize)]
pub struct HeaderVersion(pub u16);

impl Default for HeaderVersion {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure it makes sense to implement Default for HeaderVersion. In 2 months, all new blocks will have to be version 2, and having a default at that point seems like a good way to introduce logical errors.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Default post-HF should be version 2 at that point. At least that's the intent here.
We may want to make it more explicit though and not use "default".

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's assume for a second that it will be version 2 post hard fork. How do you manage that? Do we want default to look up the current height to know if we're >= block 250000? I would assume no. My vote is to remove default.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was slightly agreeing with @DavidBurkett but then realized versions are rarely built by themselves, they're built with their headers. And then we have a bunch of code (mostly tests) that just want a header without caring much.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The way I was thinking about this was in terms of two ways we need to create headers -

  • new/current headers - we want the default to be version 2 post hark fork
    • for tests (as @ignopeverell points out) it is reasonable to default to version 2 here
  • historical headers (say during fast sync) - we want to create non-default headers taking height into consideration
    • any tests touching pre and post hard fork behavior will want to use non-default versions

But to be honest I have not thought too deeply about this -

  • we only have a single version today
  • its not unreasonable to have a default now, until we decide otherwise
  • its easy to rework this as necessary in the future

@@ -239,9 +239,19 @@ impl Default for ProofOfWork {
}
}

impl ProofOfWork {
/// Read implementation, can't define as trait impl as we need a version
pub fn read(_ver: u16, reader: &mut dyn Reader) -> Result<ProofOfWork, ser::Error> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think you can do this. You're going to be adding that version back for our scheduled hard fork, aren't you?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the check is going to happen here during deserialization.

We have consensus::valid_header_version() and this is happening orthogonal to the ser/deser process (called via pipe::validate_header()).

I think we originally had this passed in here when we first introduced the AR/AT dual PoW impl or when we opened up the C31+ flexibility.

That said - this change was one of the things I wanted to discuss via this PR.

Copy link
Contributor

@DavidBurkett DavidBurkett May 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know much about the format of our PoW, but I assumed the hardfork would change how we have to serialize/deserialize ProofOfWork. If that's not true, then I agree with your change.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd leaning toward simplifying and standardizing how we read (to use our existing traits) for now. If and when we discover this needs to be adjusted we can revisit this decision.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally the PoW change wouldn't change its format, but just details about graph construction or cycle finding. Not sure we'll be able to keep it that way every time but still a good goal to start with.

@antiochp
Copy link
Member Author

antiochp commented May 8, 2019

Note: I added a header version validation step to the header deserialization stage.
We're going to want to halt header (and block) processing as early as possible if we know the header is an incorrect version.

We definitely want to revisit this as we get closer to the hardfork - I'm guessing (but to be discussed) that a pre-HF peer will want to be banned to keep the network healthy.
Currently a serialization Error during read will simply drop the connection (but the peer will not be banned).

@@ -239,9 +239,19 @@ impl Default for ProofOfWork {
}
}

impl ProofOfWork {
/// Read implementation, can't define as trait impl as we need a version
pub fn read(_ver: u16, reader: &mut dyn Reader) -> Result<ProofOfWork, ser::Error> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally the PoW change wouldn't change its format, but just details about graph construction or cycle finding. Not sure we'll be able to keep it that way every time but still a good goal to start with.

#[derive(Clone, Copy, Debug, Eq, PartialEq, Serialize)]
pub struct HeaderVersion(pub u16);

impl Default for HeaderVersion {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was slightly agreeing with @DavidBurkett but then realized versions are rarely built by themselves, they're built with their headers. And then we have a bunch of code (mostly tests) that just want a header without caring much.

@antiochp antiochp force-pushed the header_version_cleanup branch from 349831a to 540681d Compare May 8, 2019 19:56
@antiochp antiochp merged commit fabff51 into mimblewimble:master May 8, 2019
@antiochp antiochp deleted the header_version_cleanup branch May 8, 2019 20:10
@antiochp antiochp added this to the 1.1.0 milestone Jun 5, 2019
@antiochp antiochp added the release notes To be included in release notes (of relevant milestone). label Jun 5, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release notes To be included in release notes (of relevant milestone).
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants