Issue 5544 - Tessera as internal process

[gtebrean]

PR description

The key changes are in the TesseraInternalProcessTestHarness class. This starts Tessera nodes as processes.
For unzziping Tessera I created a gradle task.

Also got inspired from #4466

Fixed Issue(s)

Fixes #5544

[github-actions]

• I thought about documentation and added the doc-change-required label to this PR if updates are required.
• I thought about the changelog and included a changelog update if required.
• If my PR includes database changes (e.g. KeyValueSegmentIdentifier) I have thought about compatibility and performed forwards and backwards compatibility tests

[non-fungible-nelson]

Hey @gtebrean - this would be a huge help in our CI simplification and testing processes. Anything you need to get this across the line?

[gtebrean]

Hey @gtebrean - this would be a huge help in our CI simplification and testing processes. Anything you need to get this across the line?

Hey Mat, the only thing left here is getting Tessera JVM started properly. Last time, I encountered some issues with the ports. I've been busy with work over the past few months, but now I'm getting back to this.

[macfarla]

@gtebrean it's failing locally for me because the tessera tar file doesn't exist at this location

  systemProperty 'tessera-dist', "${buildDir}/tessera/dist/tessera-${tesseraDist}/bin/tessera"

[macfarla]

can this dependency go in versions.gradle instead

[macfarla]

also should be able to delete the testcontainers dependency

dependency 'org.testcontainers:testcontainers:1.19.3'

[macfarla]

@gtebrean I played around with circle CI config for the privacy ATs task today to see if I could make it more efficient - most recent run worked but took 43 min which isn't really feasible so I changed it back

[gtebrean]

@macfarla I tried to address the tx pool issue that we are facing, in the last commit. I followed the approach which was adopted in #6384 but I update it for the privacy node use case creating a new PrivateSyncCondition, using the PrivConditios. However the issue still persists locally and in github pipeline. Could you please look at my last commit? As I haven't interacted that much with synchronisation, maybe I missed something.

[macfarla]

I think your latest commit fixes the "Transaction pool not enabled" issue. (could you push commit this as a separate PR - I think we can merge that ahead of this one to fix the nightly). This PR though still has the occasional "Tessera did not start" which seems to be flaky for a different reason.

[macfarla]

Also - unit test failures is files missing headers

• What went wrong:
  Execution failed for task ':checkSpdxHeader'.

Files without headers: /home/circleci/project/acceptance-tests/dsl/src/main/java/org/hyperledger/besu/tests/acceptance/dsl/condition/priv/PrivateSyncingStatusCondition.java
/home/circleci/project/acceptance-tests/dsl/src/main/java/org/hyperledger/besu/tests/acceptance/dsl/transaction/privacy/PrivSyncingTransactions.java

[macfarla]

@gtebrean for me it's still missing the part that will download the tessera dependency and install it into the build dir

[gtebrean]

@macfarla please mention in here the tasks you are trying to run to check exactly, in my case the tessera is added to the build folder.
Sorry also for inactivity but I was really busy on projects.
I tried to update this to latest changes, but BesuConfig class was updated and now all the privacy tests needs to be updated...at least all of them are failing when I run the job locally.

[macfarla]

@gtebrean I think we can close this PR, you have other methods to test privacy right, and we aren't going to spend time trying to get this working again with Tessera within Besu.