[Question] How to build image with client as a separate machine?

[chendave]

server and client are deployed on different machines, I tried to build the image with the below command but failed with exceptions,

buildctl --addr http://10.169.36.51/run/buildkit/buildkitd.sock build --frontend dockerfile.v0 --local dockerfile=. --local context=. --output type=image,name=jungler/test
[+] Building 0.0s (0/0)
error: failed to get status: rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = "transport: error while dialing: dial http: unknown network http"

I have not found any doc to explain how to build image in this case, would like to get some help for this case, thanks!

[AkihiroSuda]

Server: buildkitd --addr tcp://0.0.0.0:1234 --tlscert /path/to/tlscert --tlskey /path/to/tlskey --tlscacert /path/to/tlscacert
Client: buildctl --addr tcp://10.169.36.51 --tlscert /path/to/tlscert --tlskey /path/to/tlskey --tlscacert /path/to/tlscacert --tlsservername 10.169.36.51

[AkihiroSuda]

or ssh -L if you don't want to configure TLS stuff.

Maybe we can add buildctl --addr ssh://10.169.36.51 connection helper if it is useful.

[AkihiroSuda]

You may also want to try buildctl --addr docker-container://containername or buildctl --addr kube-pod://podname if you have remote Docker or Kubernetes.

[chendave]

@AkihiroSuda thank you! I will have a try!

[chendave]

@AkihiroSuda , I still have not made it through, here is what I tried,

openssl genrsa -des3 -out tlsca.key 4096
openssl req -x509 -new -nodes -key tlsca.key -sha256 -days 1024 -out tlsca.crt
openssl genrsa -out tls.key 2048
openssl req -new -key tls.key -out tls.csr
openssl x509 -req -in tls.csr -CA tlsca.crt -CAkey tlsca.key -CAcreateserial -out tls.crt -days 500 -sha256

And then start the service,

buildkitd --addr tcp://0.0.0.0:1234  --tlscert /home/dave/pass/tls.crt --tlskey /home/dave/pass/tls.key --tlscacert /home/dave/pass/tlsca.crt  --debug --root /var/lib/buildkit --oci-worker=false --containerd-worker=true

copy the certificate to client and then build the in this way,

buildctl --addr tcp://10.169.36.51:1234 --tlscert /home/dave/pass/tls.crt --tlskey /home/dave/pass/tls.key --tlscacert /home/dave/pass/tlsca.crt --tlsservername 10.169.36.51 build --frontend dockerfile.v0 --local dockerfile=. --local context=. --output type=image,name=jungler/multiarch
[+] Building 0.0s (0/0)
error: failed to dial gRPC: rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = "transport: authentication handshake failed: x509: cannot validate certificate for 10.169.36.51 because it doesn't contain any IP SANs"

Do you know what's wrong here?

[chendave]

or ssh -L if you don't want to configure TLS stuff.

I think ssh -L is used for port forwarding, could you pls elaborate a little, cause I don't know what the port will be forwarded in this case.

Maybe we can add buildctl --addr ssh://10.169.36.51 connection helper if it is useful.

wan to confirm with you, with ssh -L, the addr here will be "ssh://10.169.36.51" instead of "tcp://10.169.36.51", right?

[AkihiroSuda]

Do you know what's wrong here?

because it doesn't contain any IP SANs https://security.stackexchange.com/questions/74345/provide-subjectaltname-to-openssl-directly-on-the-command-line

I think ssh -L is used for port forwarding, could you pls elaborate a little, cause I don't know what the port will be forwarded in this case.

It can also forward UNIX sockets: ssh -nNT -L ./buildkitd.sock:/run/buildkit/buildkitd.sock USER@HOST

wan to confirm with you, with ssh -L, the addr here will be "ssh://10.169.36.51" instead of "tcp://10.169.36.51", right?

If we are going to add ssh helper (just as in DOCKER_HOST=ssh://USER@HOST, the URL schema will be ssh:// rather than tcp://, but probably it would invoke ssh USER@HOST -- buildctl dial-stdio rather than ssh -L. (implementation detail)

[chendave]

@AkihiroSuda , I managed to build with tls certificate successfully, and will try other approaches later. Thanks for your help and those good information/great suggestion. I do think there should be some doc to explain those features :)