Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CRM crashed after building with go1.17 #2079

Open
gainsley opened this issue Mar 25, 2022 · 0 comments
Open

CRM crashed after building with go1.17 #2079

gainsley opened this issue Mar 25, 2022 · 0 comments

Comments

@gainsley
Copy link
Contributor

After platform was built with go1.17 and deployed, we saw a crash in the CRM. Seeing this issue in 2022-02-03-1 load.

crmserver logs from automationBerlinCloudlet

fatal error: unreachable method called. linker bug

goroutine 180 [running]:
runtime.throw({0x2afbd97, 0x7f4c945dbf14})
  /usr/local/src/runtime/panic.go:1198 +0x71 fp=0xc0009ac648 sp=0xc0009ac618 pc=0xe88d71
runtime.unreachableMethod()
  /usr/local/src/runtime/iface.go:561 +0x25 fp=0xc0009ac668 sp=0xc0009ac648 pc=0xe5e6c5
golang.org/x/crypto/ssh.(*connectionState).readPacket(0xc0003f9d40, 0x7f4c945dbde0)
  /go/pkg/mod/golang.org/x/[email protected]/ssh/transport.go:130 +0x42 fp=0xc0009ac6e8 sp=0xc0009ac668 pc=0x7f4c94b1dc82
golang.org/x/crypto/ssh.(*transport).readPacket(0xc0003f9d40)
  /go/pkg/mod/golang.org/x/[email protected]/ssh/transport.go:114 +0x2c fp=0xc0009ac708 sp=0xc0009ac6e8 pc=0x7f4c94b1dbec
golang.org/x/crypto/ssh.(*handshakeTransport).readOnePacket(0xc0000c6f20, 0x1)
  /go/pkg/mod/golang.org/x/[email protected]/ssh/handshake.go:374 +0x3d fp=0xc0009ac788 sp=0xc0009ac708 pc=0x7f4c94b0485d
golang.org/x/crypto/ssh.(*handshakeTransport).readLoop(0xc0000c6f20)
  /go/pkg/mod/golang.org/x/[email protected]/ssh/handshake.go:197 +0x45 fp=0xc0009ac7c8 sp=0xc0009ac788 pc=0x7f4c94b03d25
golang.org/x/crypto/ssh.newClientTransport·dwrap·7()
  /go/pkg/mod/golang.org/x/[email protected]/ssh/handshake.go:134 +0x26 fp=0xc0009ac7e0 sp=0xc0009ac7c8 pc=0x7f4c94b03b26
runtime.goexit()
  /usr/local/src/runtime/asm_amd64.s:1581 +0x1 fp=0xc0009ac7e8 sp=0xc0009ac7e0 pc=0x7f4c94638f81
created by golang.org/x/crypto/ssh.newClientTransport
  /go/pkg/mod/golang.org/x/[email protected]/ssh/handshake.go:134 +0x21f

goroutine 1 [chan receive, 1 minutes]:
main.main()
  /go/src/github.com/mobiledgex/edge-cloud/cloud-resource-manager/cmd/crmserver/main.go:373 +0x14e5

goroutine 7 [chan receive]: k8s.io/klog.(*loggingT).flushDaemon(0xc00008e180)
  /go/pkg/mod/k8s.io/[email protected]/klog.go:1010 +0x6a
created by k8s.io/klog.init.0
  /go/pkg/mod/k8s.io/[email protected]/klog.go:411 +0xf4


angshuman@abhattacharjee-mac cloudlet % qmcc cloudlet show region=EU cloudlet=automationBerlinCloudlet

key:
organization: TDG
name: automationBerlinCloudlet
location:
latitude: 52.520007
longitude: 13.404954
ipsupport: Dynamic
numdynamicips: 254
state: CrmInitok <========
platformtype: Openstack
notifysrvaddr: 127.0.0.1:0
flavor:
name: DefaultPlatformFlavor
physicalname: berlin
containerversion: 2022-02-03-1
deployment: docker
crmaccesspublickey:
createdat: "2022-02-03T15:57:18.915690447+05:30"
defaultresourcealertthreshold: 80
gpuconfig:
driver:
name: nvidia-450
organization: TDG
secondarynotifysrvaddr: 127.0.0.1:0
dnslabel: automationberlincloudlet-tdg
rootlbfqdn: [shared.automationberlincloudlet-tdg.eu.mobiledgex.net](http://shared.automationberlincloudlet-tdg.eu.mobiledgex.net/)
angshuman@abhattacharjee-mac cloudlet %

80.187.131.187 is the IP of platformvm

The controller had also crashed once

angshuman@abhattacharjee-mac cloudlet % kubectl get pods -l app=controller
NAME READY STATUS RESTARTS AGE
controller-99ccc4cdc-dq4tf 1/1 Running 1 3h19m
controller-99ccc4cdc-tbtkt 1/1 Running 1 3h19m
angshuman@abhattacharjee-mac cloudlet %

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant