Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(test-tooling): migrate AIO image to Fabric v2.5.6, add constants #3054

Conversation

petermetz
Copy link
Contributor

@petermetz petermetz commented Mar 4, 2024

  1. The Fabric v2 image has been migrated to the current LTS release
    which has increased stability and of course adheres to general best
    practices more thoroughly since now we are in sync with the Fabric
    maintainers in terms of LTS.
  2. The newer versions of the fabric-samples test-net containers have
    some of the configuration files under different paths and this had to be
    reverse engineered by manually inspecting the containers at runtime and
    searching for the same files in different directories. To make this
    easier in the future for people who are using the AIO image, we've added
    a collection of constants to the test-tooling package that stores the
    paths hardcoded that are exported via verbose variable names that pin these
    to the specific Fabric version they are related to so that in the future
    if these paths change again, we can accommodate the change in a way that
    is not too confusing by exporting more variables with slightly different
    names and values.
  3. The image built from the updated Dockerfile_v2 is accessible on GHCR
    as: ghcr.io/hyperledger/cactus-fabric2-all-in-one:2024-03-03--issue-2945-fabric-v2-5-6
    which has the v2.5.6 versioned container images pre-cached (embedded) so that
    network transfer does not need to occur and rate limiting doesn't produce
    CI flakes (DockerHub has rate limits for image downloading that we regularly
    hit when we don't embed the Fabric container images this way...).
  4. Why can't we just pull the values for these constants directly from the
    container at runtime?
    4.1. The list of constants and their names/values change based on the
    Fabric version that's being used by the AIO image.
    4.2. The only ones that could be pulled are the ones that belong to the
    first organization because this is the one that the CLI container uses.
    4.3. Configuration files that store the constants' values for the second
    organization are also located in different directories depending on the
    Fabric version being used.
    4.4. The fabric-samples repo have been known to make breaking changes
    to older releases which would then make it even harder to debug if we
    had logic for identifying the constants that suddenly broke (this specific
    incident has happened in the past unfortunately)
    4.5. In short, there is definitely a case for applying the DRY principle
    here, but in this particular case it appears to be not worth it due to
    the increased complexity that the convoluted additional logic would bring.

Related but does not fix #1899

Signed-off-by: Peter Somogyvari [email protected]

Pull Request Requirements

  • Rebased onto upstream/main branch and squashed into single commit to help maintainers review it more efficient and to avoid spaghetti git commit graphs that obfuscate which commit did exactly what change, when and, why.
  • Have git sign off at the end of commit message to avoid being marked red. You can add -s flag when using git commit command. You may refer to this link for more information.
  • Follow the Commit Linting specification. You may refer to this link for more information.

Character Limit

  • Pull Request Title and Commit Subject must not exceed 72 characters (including spaces and special characters).
  • Commit Message per line must not exceed 80 characters (including spaces and special characters).

A Must Read for Beginners
For rebasing and squashing, here's a must read guide for beginners.

@petermetz petermetz enabled auto-merge (rebase) March 4, 2024 19:14
petermetz added a commit to petermetz/cacti that referenced this pull request Mar 4, 2024
The deployContractGoSourceV1() method now assumes that the underlying
test ledger is Fabric 2.5 (current LTS).

This will allow us to upgrade the contracts that are being used by the
Supply chain app to Fabric 2.x from Fabric 1.x which will then implicitly
fix a large number of other issues at the same time.

This change is part of laying the foundation for that follow-up work.

Primary changes:
-----------------

1. Added a new, standalone utility function to deploy go source contracts
with the name of `deployContractGoSourceImplFabricV256()`.
2. The code of this function was derived from the original Fabric v1
compatible deployContractGoSourceV1 method of the Fabric connector.
3. 2 organizations are supported for deployment via the endpoint.
4. The endpoint is only used by the supply chain app example at the moment
and there is no test coverage of it due to dependencies that will be
resolved in a follow-up pull request that is coming soon.

Secondary changes:
1. Also extracted the SSH execution function from the fabric connector
into a standalone function that can be used without having to have a
Fabric connector instance created first.
2. Also extracted/abstracted some logic into a utility function for
similar reasons that is used to replace logging configuration environment
variables in shell commands that we use to perform contract deployment
onto the Fabric test ledgers.

Depends on hyperledger-cacti#3054

Signed-off-by: Peter Somogyvari <[email protected]>
Copy link
Contributor

@izuru0 izuru0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

petermetz added a commit to petermetz/cacti that referenced this pull request Mar 5, 2024
The deployContractGoSourceV1() method now assumes that the underlying
test ledger is Fabric 2.5 (current LTS).

This will allow us to upgrade the contracts that are being used by the
Supply chain app to Fabric 2.x from Fabric 1.x which will then implicitly
fix a large number of other issues at the same time.

This change is part of laying the foundation for that follow-up work.

Primary changes:
-----------------

1. Added a new, standalone utility function to deploy go source contracts
with the name of `deployContractGoSourceImplFabricV256()`.
2. The code of this function was derived from the original Fabric v1
compatible deployContractGoSourceV1 method of the Fabric connector.
3. 2 organizations are supported for deployment via the endpoint.
4. The endpoint is only used by the supply chain app example at the moment
and there is no test coverage of it due to dependencies that will be
resolved in a follow-up pull request that is coming soon.

Secondary changes:
1. Also extracted the SSH execution function from the fabric connector
into a standalone function that can be used without having to have a
Fabric connector instance created first.
2. Also extracted/abstracted some logic into a utility function for
similar reasons that is used to replace logging configuration environment
variables in shell commands that we use to perform contract deployment
onto the Fabric test ledgers.

Depends on hyperledger-cacti#3054

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit to petermetz/cacti that referenced this pull request Mar 5, 2024
The supply chain app's build and execution scripts should finally be
working after this and also be much more stable than before due to the
flakiness of the Fabric V1 test ledger not being an issue anymore.

1.The new contract is compiled with go v1.20 and therefore has to have
the contract method names capitalized insted of camelCase, otherwise
the methods are not possible to be exported and the contract deployment
fails even if everything else is correct.
2. The supply chain app now uses the newest edition of the Fabric v2 AIO
test ledger container image which uses Fabric v2.5.6 (current LTS at the
time of this writing).
3. The shipment contract's source code has been migrated to Fabric v2
meaning that instead of a stub object we get a context object for each
method's first parameter and then the stub can be acquired from that
context object.
4. The method arguments no longer need to be passed around as an array
of strings and instead the contract method's input arguments are first-class
go method parameters.
5. Re-enabled a test case that was being skipped until now due to flakiness:
...`src/test/typescript/integration/supply-chain-backend-api-calls.test.ts`

Depends on hyperledger-cacti#3058
Depends on hyperledger-cacti#3054

Fixes hyperledger-cacti#2945
Fixes hyperledger-cacti#2969
Fixes hyperledger-cacti#1899
Fixes hyperledger-cacti#1521
Fixes hyperledger-cacti#1518

Signed-off-by: Peter Somogyvari <[email protected]>
1. The Fabric v2 image has been migrated to the current LTS release
which has increased stability and of course adheres to general best
practices more thoroughly since now we are in sync with the Fabric
maintainers in terms of LTS.
2. The newer versions of the fabric-samples test-net containers have
some of the configuration files under different paths and this had to be
reverse engineered by manually inspecting the containers at runtime and
searching for the same files in different directories. To make this
easier in the future for people who are using the AIO image, we've added
a collection of constants to the test-tooling package that stores the
paths hardcoded that are exported via verbose variable names that pin these
to the specific Fabric version they are related to so that in the future
if these paths change again, we can accommodate the change in a way that
is not too confusing by exporting more variables with slightly different
names and values.
3. The image built from the updated `Dockerfile_v2` is accessible on GHCR
as: `ghcr.io/hyperledger/cactus-fabric2-all-in-one:2024-03-03--issue-2945-fabric-v2-5-6`
which has the v2.5.6 versioned container images pre-cached (embedded) so that
network transfer does not need to occur and rate limiting doesn't produce
CI flakes (DockerHub has rate limits for image downloading that we regularly
hit when we don't embed the Fabric container images this way...).
4. Why can't we just pull the values for these constants directly from the
container at runtime?
4.1. The list of constants and their names/values change based on the
Fabric version that's being used by the AIO image.
4.2. The only ones that could be pulled are the ones that belong to the
first organization because this is the one that the CLI container uses.
4.3. Configuration files that store the constants' values for the second
organization are also located in different directories depending on the
Fabric version being used.
4.4. The fabric-samples repo have been known to make breaking changes
to older releases which would then make it even harder to debug if we
had logic for identifying the constants that suddenly broke (this specific
incident has happened in the past unfortunately)
4.5. In short, there is definitely a case for applying the DRY principle
here, but in this particular case it appears to be not worth it due to
the increased complexity that the convoluted additional logic would bring.

Related but does not fix hyperledger-cacti#1899

Signed-off-by: Peter Somogyvari <[email protected]>
@petermetz petermetz force-pushed the test-tooling-fabric-v256-lts-support branch from 2113302 to f6eb342 Compare March 5, 2024 20:36
@petermetz petermetz requested a review from outSH March 5, 2024 20:39
petermetz added a commit to petermetz/cacti that referenced this pull request Mar 5, 2024
The deployContractGoSourceV1() method now assumes that the underlying
test ledger is Fabric 2.5 (current LTS).

This will allow us to upgrade the contracts that are being used by the
Supply chain app to Fabric 2.x from Fabric 1.x which will then implicitly
fix a large number of other issues at the same time.

This change is part of laying the foundation for that follow-up work.

Primary changes:
-----------------

1. Added a new, standalone utility function to deploy go source contracts
with the name of `deployContractGoSourceImplFabricV256()`.
2. The code of this function was derived from the original Fabric v1
compatible deployContractGoSourceV1 method of the Fabric connector.
3. 2 organizations are supported for deployment via the endpoint.
4. The endpoint is only used by the supply chain app example at the moment
and there is no test coverage of it due to dependencies that will be
resolved in a follow-up pull request that is coming soon.

Secondary changes:
1. Also extracted the SSH execution function from the fabric connector
into a standalone function that can be used without having to have a
Fabric connector instance created first.
2. Also extracted/abstracted some logic into a utility function for
similar reasons that is used to replace logging configuration environment
variables in shell commands that we use to perform contract deployment
onto the Fabric test ledgers.

Depends on hyperledger-cacti#3054

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit to petermetz/cacti that referenced this pull request Mar 5, 2024
The supply chain app's build and execution scripts should finally be
working after this and also be much more stable than before due to the
flakiness of the Fabric V1 test ledger not being an issue anymore.

1.The new contract is compiled with go v1.20 and therefore has to have
the contract method names capitalized insted of camelCase, otherwise
the methods are not possible to be exported and the contract deployment
fails even if everything else is correct.
2. The supply chain app now uses the newest edition of the Fabric v2 AIO
test ledger container image which uses Fabric v2.5.6 (current LTS at the
time of this writing).
3. The shipment contract's source code has been migrated to Fabric v2
meaning that instead of a stub object we get a context object for each
method's first parameter and then the stub can be acquired from that
context object.
4. The method arguments no longer need to be passed around as an array
of strings and instead the contract method's input arguments are first-class
go method parameters.
5. Re-enabled a test case that was being skipped until now due to flakiness:
...`src/test/typescript/integration/supply-chain-backend-api-calls.test.ts`

Depends on hyperledger-cacti#3058
Depends on hyperledger-cacti#3054

Fixes hyperledger-cacti#2945
Fixes hyperledger-cacti#2969
Fixes hyperledger-cacti#1899
Fixes hyperledger-cacti#1521
Fixes hyperledger-cacti#1518

Signed-off-by: Peter Somogyvari <[email protected]>
Copy link
Contributor

@jagpreetsinghsasan jagpreetsinghsasan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@petermetz petermetz merged commit f59f369 into hyperledger-cacti:main Mar 7, 2024
126 of 149 checks passed
@petermetz petermetz deleted the test-tooling-fabric-v256-lts-support branch March 7, 2024 07:19
petermetz added a commit to petermetz/cacti that referenced this pull request Mar 7, 2024
The deployContractGoSourceV1() method now assumes that the underlying
test ledger is Fabric 2.5 (current LTS).

This will allow us to upgrade the contracts that are being used by the
Supply chain app to Fabric 2.x from Fabric 1.x which will then implicitly
fix a large number of other issues at the same time.

This change is part of laying the foundation for that follow-up work.

Primary changes:
-----------------

1. Added a new, standalone utility function to deploy go source contracts
with the name of `deployContractGoSourceImplFabricV256()`.
2. The code of this function was derived from the original Fabric v1
compatible deployContractGoSourceV1 method of the Fabric connector.
3. 2 organizations are supported for deployment via the endpoint.
4. The endpoint is only used by the supply chain app example at the moment
and there is no test coverage of it due to dependencies that will be
resolved in a follow-up pull request that is coming soon.

Secondary changes:
1. Also extracted the SSH execution function from the fabric connector
into a standalone function that can be used without having to have a
Fabric connector instance created first.
2. Also extracted/abstracted some logic into a utility function for
similar reasons that is used to replace logging configuration environment
variables in shell commands that we use to perform contract deployment
onto the Fabric test ledgers.

Depends on hyperledger-cacti#3054

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit that referenced this pull request Mar 7, 2024
The deployContractGoSourceV1() method now assumes that the underlying
test ledger is Fabric 2.5 (current LTS).

This will allow us to upgrade the contracts that are being used by the
Supply chain app to Fabric 2.x from Fabric 1.x which will then implicitly
fix a large number of other issues at the same time.

This change is part of laying the foundation for that follow-up work.

Primary changes:
-----------------

1. Added a new, standalone utility function to deploy go source contracts
with the name of `deployContractGoSourceImplFabricV256()`.
2. The code of this function was derived from the original Fabric v1
compatible deployContractGoSourceV1 method of the Fabric connector.
3. 2 organizations are supported for deployment via the endpoint.
4. The endpoint is only used by the supply chain app example at the moment
and there is no test coverage of it due to dependencies that will be
resolved in a follow-up pull request that is coming soon.

Secondary changes:
1. Also extracted the SSH execution function from the fabric connector
into a standalone function that can be used without having to have a
Fabric connector instance created first.
2. Also extracted/abstracted some logic into a utility function for
similar reasons that is used to replace logging configuration environment
variables in shell commands that we use to perform contract deployment
onto the Fabric test ledgers.

Depends on #3054

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit to petermetz/cacti that referenced this pull request Mar 7, 2024
The supply chain app's build and execution scripts should finally be
working after this and also be much more stable than before due to the
flakiness of the Fabric V1 test ledger not being an issue anymore.

1.The new contract is compiled with go v1.20 and therefore has to have
the contract method names capitalized insted of camelCase, otherwise
the methods are not possible to be exported and the contract deployment
fails even if everything else is correct.
2. The supply chain app now uses the newest edition of the Fabric v2 AIO
test ledger container image which uses Fabric v2.5.6 (current LTS at the
time of this writing).
3. The shipment contract's source code has been migrated to Fabric v2
meaning that instead of a stub object we get a context object for each
method's first parameter and then the stub can be acquired from that
context object.
4. The method arguments no longer need to be passed around as an array
of strings and instead the contract method's input arguments are first-class
go method parameters.
5. Re-enabled a test case that was being skipped until now due to flakiness:
...`src/test/typescript/integration/supply-chain-backend-api-calls.test.ts`

Depends on hyperledger-cacti#3058
Depends on hyperledger-cacti#3054

Fixes hyperledger-cacti#2945
Fixes hyperledger-cacti#2969
Fixes hyperledger-cacti#1899
Fixes hyperledger-cacti#1521
Fixes hyperledger-cacti#1518

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit to petermetz/cacti that referenced this pull request Mar 8, 2024
The supply chain app's build and execution scripts should finally be
working after this and also be much more stable than before due to the
flakiness of the Fabric V1 test ledger not being an issue anymore.

The container image is now published as:

`ghcr.io/hyperledger/cactus-example-supply-chain-app:2024-03-08--pr-3059-1`

1.The new contract is compiled with go v1.20 and therefore has to have
the contract method names capitalized insted of camelCase, otherwise
the methods are not possible to be exported and the contract deployment
fails even if everything else is correct.
2. The supply chain app now uses the newest edition of the Fabric v2 AIO
test ledger container image which uses Fabric v2.5.6 (current LTS at the
time of this writing).
3. The shipment contract's source code has been migrated to Fabric v2
meaning that instead of a stub object we get a context object for each
method's first parameter and then the stub can be acquired from that
context object.
4. The method arguments no longer need to be passed around as an array
of strings and instead the contract method's input arguments are first-class
go method parameters.
5. Re-enabled a test case that was being skipped until now due to flakiness:
...`src/test/typescript/integration/supply-chain-backend-api-calls.test.ts`

The supply chain app container image was built with this command:
```sh
DOCKER_BUILDKIT=1 docker build \
  --build-arg="NPM_PKG_VERSION=2.0.0-2945-supply-chain-app-build-failed.241+b2c306ea0" \
  -f ./examples/cactus-example-supply-chain-backend/Dockerfile \
  . \
  -t scaeb
```

The NPM_PKG_VERSION build arg is important because the image defaults to
"latest" which at the moment is packages that do not contain the fixes
made by this commit, so re-building the image without that extra arg
will not work until we issue the next official non-canary release.

Depends on hyperledger-cacti#3058
Depends on hyperledger-cacti#3054

Fixes hyperledger-cacti#2945
Fixes hyperledger-cacti#2969
Fixes hyperledger-cacti#1899
Fixes hyperledger-cacti#1521
Fixes hyperledger-cacti#1518

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit that referenced this pull request Mar 9, 2024
The supply chain app's build and execution scripts should finally be
working after this and also be much more stable than before due to the
flakiness of the Fabric V1 test ledger not being an issue anymore.

The container image is now published as:

`ghcr.io/hyperledger/cactus-example-supply-chain-app:2024-03-08--pr-3059-1`

1.The new contract is compiled with go v1.20 and therefore has to have
the contract method names capitalized insted of camelCase, otherwise
the methods are not possible to be exported and the contract deployment
fails even if everything else is correct.
2. The supply chain app now uses the newest edition of the Fabric v2 AIO
test ledger container image which uses Fabric v2.5.6 (current LTS at the
time of this writing).
3. The shipment contract's source code has been migrated to Fabric v2
meaning that instead of a stub object we get a context object for each
method's first parameter and then the stub can be acquired from that
context object.
4. The method arguments no longer need to be passed around as an array
of strings and instead the contract method's input arguments are first-class
go method parameters.
5. Re-enabled a test case that was being skipped until now due to flakiness:
...`src/test/typescript/integration/supply-chain-backend-api-calls.test.ts`

The supply chain app container image was built with this command:
```sh
DOCKER_BUILDKIT=1 docker build \
  --build-arg="NPM_PKG_VERSION=2.0.0-2945-supply-chain-app-build-failed.241+b2c306ea0" \
  -f ./examples/cactus-example-supply-chain-backend/Dockerfile \
  . \
  -t scaeb
```

The NPM_PKG_VERSION build arg is important because the image defaults to
"latest" which at the moment is packages that do not contain the fixes
made by this commit, so re-building the image without that extra arg
will not work until we issue the next official non-canary release.

Depends on #3058
Depends on #3054

Fixes #2945
Fixes #2969
Fixes #1899
Fixes #1521
Fixes #1518

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit to petermetz/cacti that referenced this pull request Mar 10, 2024
The deployContractGoSourceV1() method now assumes that the underlying
test ledger is Fabric 2.5 (current LTS).

This will allow us to upgrade the contracts that are being used by the
Supply chain app to Fabric 2.x from Fabric 1.x which will then implicitly
fix a large number of other issues at the same time.

This change is part of laying the foundation for that follow-up work.

Primary changes:
-----------------

1. Added a new, standalone utility function to deploy go source contracts
with the name of `deployContractGoSourceImplFabricV256()`.
2. The code of this function was derived from the original Fabric v1
compatible deployContractGoSourceV1 method of the Fabric connector.
3. 2 organizations are supported for deployment via the endpoint.
4. The endpoint is only used by the supply chain app example at the moment
and there is no test coverage of it due to dependencies that will be
resolved in a follow-up pull request that is coming soon.

Secondary changes:
1. Also extracted the SSH execution function from the fabric connector
into a standalone function that can be used without having to have a
Fabric connector instance created first.
2. Also extracted/abstracted some logic into a utility function for
similar reasons that is used to replace logging configuration environment
variables in shell commands that we use to perform contract deployment
onto the Fabric test ledgers.

Depends on hyperledger-cacti#3054

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit to petermetz/cacti that referenced this pull request Mar 10, 2024
The supply chain app's build and execution scripts should finally be
working after this and also be much more stable than before due to the
flakiness of the Fabric V1 test ledger not being an issue anymore.

The container image is now published as:

`ghcr.io/hyperledger/cactus-example-supply-chain-app:2024-03-08--pr-3059-1`

1.The new contract is compiled with go v1.20 and therefore has to have
the contract method names capitalized insted of camelCase, otherwise
the methods are not possible to be exported and the contract deployment
fails even if everything else is correct.
2. The supply chain app now uses the newest edition of the Fabric v2 AIO
test ledger container image which uses Fabric v2.5.6 (current LTS at the
time of this writing).
3. The shipment contract's source code has been migrated to Fabric v2
meaning that instead of a stub object we get a context object for each
method's first parameter and then the stub can be acquired from that
context object.
4. The method arguments no longer need to be passed around as an array
of strings and instead the contract method's input arguments are first-class
go method parameters.
5. Re-enabled a test case that was being skipped until now due to flakiness:
...`src/test/typescript/integration/supply-chain-backend-api-calls.test.ts`

The supply chain app container image was built with this command:
```sh
DOCKER_BUILDKIT=1 docker build \
  --build-arg="NPM_PKG_VERSION=2.0.0-2945-supply-chain-app-build-failed.241+b2c306ea0" \
  -f ./examples/cactus-example-supply-chain-backend/Dockerfile \
  . \
  -t scaeb
```

The NPM_PKG_VERSION build arg is important because the image defaults to
"latest" which at the moment is packages that do not contain the fixes
made by this commit, so re-building the image without that extra arg
will not work until we issue the next official non-canary release.

Depends on hyperledger-cacti#3058
Depends on hyperledger-cacti#3054

Fixes hyperledger-cacti#2945
Fixes hyperledger-cacti#2969
Fixes hyperledger-cacti#1899
Fixes hyperledger-cacti#1521
Fixes hyperledger-cacti#1518

Signed-off-by: Peter Somogyvari <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

TypeError: "pkcs8" must be PCKS8 formatted string - cactus-example-supply-chain-app:v1.0.0-rc.3
4 participants