Skip to content

Staging integration workflow for syncing Polkadot + Kusama nodes#1751

Merged
timwu20 merged 19 commits intodevelopmentfrom
tim-staging
Aug 31, 2021
Merged

Staging integration workflow for syncing Polkadot + Kusama nodes#1751
timwu20 merged 19 commits intodevelopmentfrom
tim-staging

Conversation

@timwu20
Copy link
Copy Markdown
Contributor

@timwu20 timwu20 commented Aug 23, 2021

Changes

  • AWS Task definitions for the Kusama and Polkadot services. Services are deployed to an ECS cluster called gossamer-stg
  • Github actions workflow to build image, push image to ECR, update task definition and register task definition to service.
  • One container to instance at this time. Uses host networking, which actually binds to the EC2 instance ports. Required to get libp2p working.
  • Dockerfile.staging which is a revised Dockerfile that runs the node with --publish-metrics and also installs the Datadog agent and forwards the metrics to Dataog as custom metrics.
  • Metrics are prefixed with gossamer.staging.polkadot/kusama.

Tests

Ensure that merging to staging branch successfully deploys to ECS via github actions.

Issues

Primary Reviewer

@timwu20 timwu20 changed the title Tim staging Staging integration workflow for syncing Polkadot + Kusama nodes Aug 23, 2021
@codecov
Copy link
Copy Markdown

codecov bot commented Aug 23, 2021

Codecov Report

Merging #1751 (2860964) into development (071fe44) will increase coverage by 0.02%.
The diff coverage is n/a.

Impacted file tree graph

@@               Coverage Diff               @@
##           development    #1751      +/-   ##
===============================================
+ Coverage        59.02%   59.04%   +0.02%     
===============================================
  Files              184      184              
  Lines            19390    19390              
===============================================
+ Hits             11444    11448       +4     
+ Misses            5959     5955       -4     
  Partials          1987     1987              
Flag Coverage Δ
unit-tests 59.04% <ø> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
lib/blocktree/leaves.go 90.24% <0.00%> (-4.88%) ⬇️
lib/blocktree/blocktree.go 55.61% <0.00%> (ø)
dot/network/sync.go 64.10% <0.00%> (+0.39%) ⬆️
dot/state/block_notify.go 73.86% <0.00%> (+1.13%) ⬆️
lib/blocktree/node.go 74.39% <0.00%> (+3.65%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 071fe44...2860964. Read the comment docs.

Comment thread Dockerfile.staging
RUN ["sh", "-c", "DD_AGENT_MAJOR_VERSION=7 DD_INSTALL_ONLY=true DD_API_KEY=${DD_API_KEY} DD_SITE=\"datadoghq.com\" bash -c \"$(curl -L https://s3.amazonaws.com/dd-agent/scripts/install_script.sh)\""]

RUN mkdir /gossamer
WORKDIR /gossamer
Copy link
Copy Markdown
Member

@EclesioMeloJunior EclesioMeloJunior Aug 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the WORKDIR command creates the path if it doesn't exist

Copy link
Copy Markdown
Contributor

@noot noot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks proper - is there a way i can test this locally, or do we need to merge it in first?

@timwu20
Copy link
Copy Markdown
Contributor Author

timwu20 commented Aug 25, 2021

looks proper - is there a way i can test this locally, or do we need to merge it in first?

You can push and deploy if you had AWS access keys and proper permissions. Otherwise, it's just building a specific docker image via github actions.

@timwu20 timwu20 merged commit 210a68e into development Aug 31, 2021
@timwu20 timwu20 deleted the tim-staging branch August 31, 2021 15:08
@github-actions
Copy link
Copy Markdown

github-actions bot commented Dec 3, 2021

🎉 This PR is included in version 0.6.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

timwu20 added a commit to timwu20/gossamer that referenced this pull request Dec 6, 2021
…inSafe#1751)

* add wip staging workflow

* wip

* wip

* wip

* wip

* wip

* wip

* wip matrix

* wip render and deploy

* wip render and deploy

* wip

* wip

* fix typo

* fix svc

* temp remove ebs volume

* add dd agent to Dockerfile and openmetrics conf

* update workflow

* update branch

* remove mkdir
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants