feat(devnet): add substrate docker images to dockerfile #2263
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| services: | ||
| prometheus: | ||
|
Contributor
There was a problem hiding this comment. why are we running prometheus in this docker-compose?
Member
Author
There was a problem hiding this comment. I added Prometheus while testing the exported metric data from authority nodes, I decided to keep it in the same file to easily start a Prometheus server with authorities nodes, should I remove it? |
||
| image: prom/prometheus:v2.33.1 | ||
| volumes: | ||
| - ./../prometheus.yml:/etc/prometheus/prometheus.yml:ro | ||
| command: | ||
| - '--storage.tsdb.path=/prometheus' | ||
| - '--web.console.libraries=/usr/share/prometheus/console_libraries' | ||
| - '--web.console.templates=/usr/share/prometheus/consoles' | ||
| ports: | ||
| - 9090 | ||
| restart: always | ||
|
|
||
| alice: | ||
| platform: linux/amd64 | ||
| build: | ||
| context: ./.. | ||
| dockerfile: devnet/substrate_alice.Dockerfile | ||
| args: | ||
| DD_API_KEY: ${DD_API_KEY} | ||
| ports: | ||
| - 7001 | ||
| - 8545 | ||
| - 8546 | ||
| - 9876 | ||
|
|
||
| bob: | ||
| platform: linux/amd64 | ||
| build: | ||
| context: ./.. | ||
| dockerfile: devnet/substrate_bob.Dockerfile | ||
| args: | ||
| key: bob | ||
| DD_API_KEY: ${DD_API_KEY} | ||
| ports: | ||
| - 7001 | ||
| - 8545 | ||
| - 8546 | ||
| - 9876 | ||
| depends_on: | ||
| - alice | ||
|
|
||
| charlie: | ||
| platform: linux/amd64 | ||
| build: | ||
| context: ./.. | ||
| dockerfile: devnet/substrate_bob.Dockerfile | ||
| args: | ||
| key: charlie | ||
| DD_API_KEY: ${DD_API_KEY} | ||
| ports: | ||
| - 7001 | ||
| - 8545 | ||
| - 8546 | ||
| - 9876 | ||
| depends_on: | ||
| - alice | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| FROM parity/polkadot:v0.9.10 AS polkadot | ||
| FROM golang:1.17 | ||
|
|
||
| ARG CHAIN=cross-client | ||
|
Contributor
There was a problem hiding this comment. what is the difference between the
Member
Author
There was a problem hiding this comment. Both have 3 authority nodes, however, the
Contributor
There was a problem hiding this comment. Ideally we should have one chain all cross client, and gossamer only setups. We may end up having multiple of these chains that differ based on number of authority nodes. @kishansagathiya is working on secondary vrf slot mode for babe #2307. We need to ensure that the genesis file is using this mode and we should be updating to v0.9.16 of polkadot. It would be nice to have some sort of readme or something in the devnet readme that describes how to generate these files. I know you were following the steps provided by Kishan, but where did the original genesis-spec.json come from? What pallets are in the runtime? These are questions we should have answers to. It may make sense to have another repo and associated github CI workflows to generate these genesis files that the devnets can pull down as part of the docker image build process. Rather than just putting arbitrary files in this repo.
Contributor
There was a problem hiding this comment.
And then you run something like this to get a genesis.json file Will make a pr to add this to read me. New versions have a different genesis-spec file than 0.9.10.
Contributor
There was a problem hiding this comment. can we rename this to something more descriptive.
Contributor
There was a problem hiding this comment. Also we could have i.e. ARG POLKADOT_VERSION=v0.9.10
FROM parity/polkadot:${POLKADOT_VERSION} AS polkadot
FROM golang:1.17
ARG CHAIN=3-auth-node-${POLKADOT_VERSION}
Member
Author
There was a problem hiding this comment. @qdm12 I was having problems trying to use arguments declared before Basically, all the |
||
| ARG VERSION=v0.9.10 | ||
| ARG DD_API_KEY=somekey | ||
| ARG METRICS_NAMESPACE=substrate.local.devnet | ||
|
|
||
| ENV CHAIN=${CHAIN} | ||
|
qdm12 marked this conversation as resolved.
Outdated
|
||
| ENV DD_API_KEY=${DD_API_KEY} | ||
|
|
||
| RUN DD_AGENT_MAJOR_VERSION=7 DD_INSTALL_ONLY=true DD_SITE="datadoghq.com" bash -c "$(curl -L https://s3.amazonaws.com/dd-agent/scripts/install_script.sh)" | ||
|
|
||
| COPY --from=polkadot /usr/bin/polkadot /usr/bin/polkadot | ||
|
|
||
| WORKDIR /gossamer | ||
|
|
||
| COPY go.mod go.sum ./ | ||
| RUN go mod download | ||
|
|
||
| COPY . . | ||
|
|
||
| WORKDIR /gossamer/devnet | ||
|
|
||
| RUN go run cmd/update-dd-agent-confd/main.go -n=${METRICS_NAMESPACE} -t=key:alice > /etc/datadog-agent/conf.d/openmetrics.d/conf.yaml | ||
|
Contributor
There was a problem hiding this comment. what port is prometheus running on?
Member
Author
There was a problem hiding this comment. Changed substrate nodes to expose Prometheus at port 9876
Contributor
There was a problem hiding this comment. What port does substrate uses? Shouldn't we use the standard
Member
Author
There was a problem hiding this comment. by default the port 9090 is the one Prometheus docker image prom/prometheus uses to bootstrap its own server to collect metrics. I just standardized the port which substrate and gossamer will expose metrics
Contributor
There was a problem hiding this comment. Yes but by default what's the port substrate/polkadot uses for Prometheus? We force it to As far as I know, applications usually have their Prom registry server listening on |
||
|
|
||
| ENTRYPOINT service datadog-agent start && /usr/bin/polkadot \ | ||
| --chain chain/$CHAIN/genesis-raw.json \ | ||
| --alice \ | ||
| --port 7001 \ | ||
| --rpc-port 8545 \ | ||
| --ws-port 8546 \ | ||
| --node-key "93ce444331ced4d2f7bfb8296267544e20c2591dbf310c7ea3af672f2879cf8f" \ | ||
|
EclesioMeloJunior marked this conversation as resolved.
Outdated
|
||
| --tmp \ | ||
| --prometheus-external \ | ||
|
qdm12 marked this conversation as resolved.
|
||
| --prometheus-port 9876 \ | ||
| --unsafe-rpc-external \ | ||
| --unsafe-ws-external | ||
|
|
||
| EXPOSE 7001/tcp 8545/tcp 8546/tcp 9876/tcp | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| FROM parity/polkadot:v0.9.10 AS polkadot | ||
| FROM golang:1.17 | ||
|
qdm12 marked this conversation as resolved.
Outdated
|
||
|
|
||
| ARG key | ||
| ARG CHAIN=cross-client | ||
| ARG DD_API_KEY=somekey | ||
| ARG METRICS_NAMESPACE=substrate.local.devnet | ||
|
|
||
| ENV key=${key} | ||
| ENV CHAIN=${CHAIN} | ||
| ENV DD_API_KEY=${DD_API_KEY} | ||
|
qdm12 marked this conversation as resolved.
|
||
|
|
||
| RUN test -n "${key}" | ||
| RUN DD_AGENT_MAJOR_VERSION=7 DD_INSTALL_ONLY=true DD_SITE="datadoghq.com" bash -c "$(curl -L https://s3.amazonaws.com/dd-agent/scripts/install_script.sh)" | ||
|
|
||
| COPY --from=polkadot /usr/bin/polkadot /usr/bin/polkadot | ||
|
|
||
| WORKDIR /gossamer | ||
|
|
||
| COPY go.mod go.sum ./ | ||
| RUN go mod download | ||
|
|
||
| COPY . . | ||
|
|
||
| WORKDIR /gossamer/devnet | ||
|
qdm12 marked this conversation as resolved.
Outdated
|
||
|
|
||
| RUN go run cmd/update-dd-agent-confd/main.go -n=${METRICS_NAMESPACE} -t=key:$key > /etc/datadog-agent/conf.d/openmetrics.d/conf.yaml | ||
|
|
||
| ENTRYPOINT service datadog-agent start && /usr/bin/polkadot \ | ||
| --bootnodes /dns/alice/tcp/7001/p2p/12D3KooWMER5iow67nScpWeVqEiRRx59PJ3xMMAYPTACYPRQbbWU \ | ||
| --chain chain/$CHAIN/genesis-raw.json \ | ||
| --port 7001 \ | ||
| --rpc-port 8545 \ | ||
| --ws-port 8546 \ | ||
| --${key} \ | ||
| --tmp \ | ||
| --prometheus-external \ | ||
| --prometheus-port 9876 \ | ||
| --unsafe-rpc-external \ | ||
| --unsafe-ws-external | ||
|
|
||
| EXPOSE 7001/tcp 8545/tcp 8546/tcp 9876/tcp | ||
Uh oh!
There was an error while loading. Please reload this page.