Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use integer for root intra in TxnExtra #763

Merged
merged 2 commits into from
Nov 3, 2021

Conversation

tolikzinovyev
Copy link
Contributor

Summary

Found a way use integer for root intra. This doesn't change the format of json.

Test Plan

Added usual encoding tests.

@codecov-commenter
Copy link

codecov-commenter commented Oct 29, 2021

Codecov Report

Merging #763 (1d1565c) into develop (242e7eb) will decrease coverage by 0.10%.
The diff coverage is 45.83%.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop     #763      +/-   ##
===========================================
- Coverage    55.20%   55.09%   -0.11%     
===========================================
  Files           26       26              
  Lines         3817     3828      +11     
===========================================
+ Hits          2107     2109       +2     
- Misses        1435     1443       +8     
- Partials       275      276       +1     
Impacted Files Coverage Δ
idb/idb.go 57.44% <23.52%> (-27.85%) ⬇️
api/converter_utils.go 87.18% <100.00%> (+0.48%) ⬆️
idb/postgres/internal/writer/writer.go 78.07% <100.00%> (ø)
idb/postgres/internal/encoding/encoding.go 75.73% <0.00%> (+1.31%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 242e7eb...1d1565c. Read the comment docs.

Copy link
Contributor

@winder winder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure this change is worth it, seems like a lot of extra work/code for what we get.

Maybe we could use json.Marshal for this object and use a pointer?

@tolikzinovyev
Copy link
Contributor Author

Our current codec without the RecursiveEmptyCheck option and a pointer also achieves it, but I thought the reason why with this option nil and &0 are both treated as empty, is because we are not supposed to use a pointer for an optional value.

The only extra code here is the marshal/unmarshal functions plus tests. I think this is worth having to isolate encoding and logic.

Copy link
Contributor

@gmalouf gmalouf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Biggest ask - what is the problem this is solving? Is it a driver of our mismatches/or being done to more closely align with go-algorand?

}

// UnmarshalText implements TextUnmarshaler interface.
func (ou *OptionalUint) UnmarshalText(text []byte) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels more like a utility function that should live independently of idb and have its own testing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a method on type OptionalUint, so we can't separate them. Do you think I should move the type elsewhere?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I was referring to the overall OptionalUint type as being general purpose - though we should have at least 1 other use-case before moving it (I'm used to optionals etc from functional languages).

idb/idb.go Outdated

return nil
}

// TxnExtra is some additional metadata needed for a transaction.
type TxnExtra struct {
AssetCloseAmount uint64 `codec:"aca,omitempty"`
// RootIntra is set on inner transactions. Combined with the confirmation
// round it can be used to lookup the root transaction.
// The type is string to allow distinguishing between 0 and empty.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update the comment to match code changes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@tolikzinovyev
Copy link
Contributor Author

Biggest ask - what is the problem this is solving? Is it a driver of our mismatches/or being done to more closely align with go-algorand?

This change is too separate encoding from the business logic. As you can see, before we were doing integer <-> string conversions in the business logic. Functionally, this PR doesn't change anything.

@tolikzinovyev
Copy link
Contributor Author

More concretely, the json codec will invoke MarshalText() / UnmarshalText() methods when encoding / decoding json.

// round it can be used to lookup the root transaction.
// The type is string to allow distinguishing between 0 and empty.
RootIntra string `codec:"root-intra,omitempty"`
Copy link
Contributor

@winder winder Nov 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It just occurred to me that we can use RootTxid to determine if this field exists or not. We can avoid special encoding/decode code entirely and use a uint64 type here and use len(RootTxid) > 0 as the "Present" condition.

From there we just need to make sure RootIntra has a comment saying "0 is a valid value when RootTxid is set."

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The query would need to change though to handle the special case when t.extra->>'root-intra' is empty string. https://github.com/algorand/indexer/pull/721/files#diff-8fbec00be6e6b2f77a294cbe9a3ee8e3fbdcb5eb13be3046e5da8277302264c2R637
If you would like to change it, I will not object.

By the way, I'm afraid this query has quadratic complexity. I'm not sure postgres is smart enough to look up the right root txn with one query. Possibly, for each txn row there will be a scan of all rows for this round.

@tolikzinovyev tolikzinovyev merged commit ad96531 into develop Nov 3, 2021
@tolikzinovyev tolikzinovyev deleted the tolik/root-intra-encoding branch November 3, 2021 18:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants