Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HIP-1056 Add block file transformer #10147

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

edwin-greene
Copy link
Contributor

Description:
Adds the block file transformer for the cryptotransfer transaction type.

Related issue(s):

Fixes #9896

Notes for reviewer:

Checklist

  • Documented (Code comments, README, etc.)
  • Tested (unit, integration, etc.)

Signed-off-by: Edwin Greene <[email protected]>
@edwin-greene edwin-greene added enhancement Type: New feature importer Area: Importer labels Jan 15, 2025
@edwin-greene edwin-greene self-assigned this Jan 15, 2025
@edwin-greene edwin-greene linked an issue Jan 15, 2025 that may be closed by this pull request
Signed-off-by: Edwin Greene <[email protected]>
Copy link

codecov bot commented Jan 16, 2025

Codecov Report

Attention: Patch coverage is 93.39623% with 7 lines in your changes missing coverage. Please review.

Project coverage is 92.31%. Comparing base (0a8166b) to head (5bb5c1f).

Files with missing lines Patch % Lines
...mporter/downloader/block/BlockFileTransformer.java 91.30% 6 Missing ⚠️
...r/block/transformer/CryptoTransferTransformer.java 90.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##               main   #10147    +/-   ##
==========================================
  Coverage     92.30%   92.31%            
- Complexity     7821     7840    +19     
==========================================
  Files           947      951     +4     
  Lines         32847    32951   +104     
  Branches       4151     4155     +4     
==========================================
+ Hits          30321    30420    +99     
- Misses         1550     1555     +5     
  Partials        976      976            

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@edwin-greene edwin-greene added this to the 0.122.0 milestone Jan 16, 2025
@edwin-greene edwin-greene marked this pull request as ready for review January 16, 2025 22:29
@edwin-greene edwin-greene requested a review from a team as a code owner January 16, 2025 22:29
Comment on lines 43 to 45
private final PersistProperties persistProperties = new PersistProperties();
private final Predicate<EntityId> entityTransactionPredicate = persistProperties::shouldPersistEntityTransaction;
private final Predicate<EntityId> contractTransactionPredicate = e -> persistProperties.isContractTransaction();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we create a PersistProperties object here? In addition, what to persist is in the scope of parser, not downloader.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An artifact from comparing normal record files to transformed record files. Thanks it is not needed, removed.

Comment on lines 83 to 85
.softwareVersionMajor(major)
.softwareVersionMinor(minor)
.softwareVersionPatch(patch)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

software version is not hapi version, there is a separate software version field in block header.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks I have corrected it.

.bytes(blockFile.getBytes())
.consensusEnd(blockFile.getConsensusEnd())
.consensusStart(blockFile.getConsensusStart())
.count((long) blockItems.size())
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
.count((long) blockItems.size())
.count(blockFile.count())

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, fixed

.index(blockHeader.getNumber())
.name(blockFile.getName())
.nodeId(blockFile.getNodeId())
.previousHash(blockHeader.getPreviousBlockHash().toStringUtf8())
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
.previousHash(blockHeader.getPreviousBlockHash().toStringUtf8())
.previousHash(blockFile.getPreviousHash())

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks I have fixed it.

Comment on lines 101 to 102
.contractTransactionPredicate(contractTransactionPredicate)
.entityTransactionPredicate(entityTransactionPredicate)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the predicates are set and used in parser, leave them unset here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good, removed.

return recordItems;
}

@SneakyThrows
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't think it's good practice to SneakyThrows here, should limit its usage to test only code

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, I have removed it.

Comment on lines 40 to 42
// Note on TransactionHash:
// There is an Open Issue in HIP 1056 whether the transaction hash will be included in the block stream
// or if it will need to be calculated by block stream consumers.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is that still an open issue? transaction hash is straightforward to calculate and the intention is to save bytes. my recollection is whether or not to do the same for ethereum transaction hash is undecided.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, updated this to include calculating the transaction hash.

public TransactionRecord getTransactionRecord(BlockItem blockItem, TransactionBody transactionBody) {
var transactionResult = blockItem.transactionResult();
var receiptBuilder = TransactionReceipt.newBuilder().setStatus(transactionResult.getStatus());
var transactionRecordBuilder = TransactionRecord.newBuilder()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add these

  • paid_staking_rewards
  • parent_consensus_timestamp
  • schedule_ref

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, these are now present

// or if it will need to be calculated by block stream consumers.
.setReceipt(receiptBuilder);

updateTransactionRecord(blockItem.transactionOutput(), transactionRecordBuilder);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

state changes? should pass the whole blockItem

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good I have updated this.

import org.junit.jupiter.params.provider.EnumSource;

@RequiredArgsConstructor
public class BlockFileTransformerTest extends ImporterIntegrationTest {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is it an integration test?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is using the domainBuilder from the ImporterIntegrationTest.

Signed-off-by: Edwin Greene <[email protected]>
Signed-off-by: Edwin Greene <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Type: New feature importer Area: Importer
Projects
None yet
Development

Successfully merging this pull request may close these issues.

HIP-1056 Block to record transformer
2 participants