Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[bug] gobgp cost many cpu if dynamic neighbors contain self #2167

Open
wu0407 opened this issue Sep 22, 2019 · 3 comments
Open

[bug] gobgp cost many cpu if dynamic neighbors contain self #2167

wu0407 opened this issue Sep 22, 2019 · 3 comments

Comments

@wu0407
Copy link

wu0407 commented Sep 22, 2019 •
edited
Loading

gobgp like Infinite loop, if dynamic neighbors contain it self.

  1. start first gobgp not use many cpu

[global.config]
  router-id = "192.168.189.213"
  as = 64513

[[neighbors]]
  [neighbors.config]
    neighbor-address = "192.168.251.145"
    peer-as = 64513
[neighbors.route-reflector.config]
    route-reflector-client = true
    route-reflector-cluster-id = "1.1.1.1"


[[neighbors]]
  [neighbors.config]
    neighbor-address = "192.168.189.214"
    peer-as = 64513
[neighbors.route-reflector.config]
    route-reflector-client = true
    route-reflector-cluster-id = "1.1.1.1"


[[neighbors]]
  [neighbors.config]
    neighbor-address = "192.168.189.215"
    peer-as = 64513
[neighbors.route-reflector.config]
    route-reflector-client = true
    route-reflector-cluster-id = "1.1.1.1"


[[peer-groups]]
  [peer-groups.config]
    peer-group-name = "uscasv2"
    peer-as = 64513
  [peer-groups.route-reflector.config]
    route-reflector-client = true
    route-reflector-cluster-id = "1.1.1.1"  
  [[peer-groups.afi-safis]]
    [peer-groups.afi-safis.config]
      afi-safi-name = "ipv4-unicast"


[[dynamic-neighbors]]
  [dynamic-neighbors.config]
    prefix = "192.168.189.0/24"
    peer-group = "uscasv2"


[zebra]
 [zebra.config]
    enabled = true
    url = "unix:/var/run/frr/zserv.api"
    redistribute-route-type-list = ["connect"]
    version = 6
    software-name="frr7.1"

[global.apply-policy.config]
  export-policy-list = ["not-export-local-subnet"]

[[policy-definitions]]
    name = "not-export-local-subnet"
 	[[policy-definitions.statements]]
      name = "statement1"
      [policy-definitions.statements.conditions.match-prefix-set]
        prefix-set = "local-subnet"
    	[policy-definitions.statements.actions]
        route-disposition = "reject-route" 

[[defined-sets.prefix-sets]]
  prefix-set-name ="local-subnet"
  [[defined-sets.prefix-sets.prefix-list]]
    ip-prefix = "192.168.189.0/24"
  1. start second gobgp peer with first gobgp, gobgp use 500% cpu
[global.config]
  router-id = "192.168.189.214"
  as = 64513

[[neighbors]]
  [neighbors.config]
    neighbor-address = "192.168.251.145"
    peer-as = 64513
[neighbors.route-reflector.config]
    route-reflector-client = true
    route-reflector-cluster-id = "1.1.1.2"


[[neighbors]]
  [neighbors.config]
    neighbor-address = "192.168.189.213"
    peer-as = 64513
[neighbors.route-reflector.config]
    route-reflector-client = true
    route-reflector-cluster-id = "1.1.1.2"


[[neighbors]]
  [neighbors.config]
    neighbor-address = "192.168.189.215"
    peer-as = 64513
[neighbors.route-reflector.config]
    route-reflector-client = true
    route-reflector-cluster-id = "1.1.1.2"

[[peer-groups]]
  [peer-groups.config]
    peer-group-name = "uscasv2"
    peer-as = 64513
  [peer-groups.route-reflector.config]
    route-reflector-client = true
    route-reflector-cluster-id = "1.1.1.2"  
  [[peer-groups.afi-safis]]
    [peer-groups.afi-safis.config]
      afi-safi-name = "ipv4-unicast"


[[dynamic-neighbors]]
  [dynamic-neighbors.config]
    prefix = "192.168.189.0/24"
    peer-group = "uscasv2"


[zebra]
 [zebra.config]
    enabled = true
    url = "unix:/var/run/frr/zserv.api"
    redistribute-route-type-list = ["connect"]
    version = 6
    software-name="frr7.1"

[global.apply-policy.config]
  export-policy-list = ["not-export-local-subnet"]

[[policy-definitions]]
    name = "not-export-local-subnet"
 	[[policy-definitions.statements]]
      name = "statement1"
      [policy-definitions.statements.conditions.match-prefix-set]
        prefix-set = "local-subnet"
    	[policy-definitions.statements.actions]
        route-disposition = "reject-route" 

[[defined-sets.prefix-sets]]
  prefix-set-name ="local-subnet"
  [[defined-sets.prefix-sets.prefix-list]]
    ip-prefix = "192.168.189.0/24"
  1. change dynamic-neighbors config exclude 192.168.189.213 192.168.189.214, gobgp cost a few cpu
[[dynamic-neighbors]]
  [dynamic-neighbors.config]
    prefix = '192.168.189.0/25'
    peer-group = "uscasv2"

[[dynamic-neighbors]]
  [dynamic-neighbors.config]
    prefix = '192.168.189.128/26'
    peer-group = "uscasv2"

 [[dynamic-neighbors]]
  [dynamic-neighbors.config]
    prefix = '192.168.189.192/28'
    peer-group = "uscasv2"


 [[dynamic-neighbors]]
  [dynamic-neighbors.config]
    prefix = '192.168.189.208/30'
    peer-group = "uscasv2"


[[dynamic-neighbors]]
  [dynamic-neighbors.config]
    prefix = '192.168.189.212/32'
    peer-group = "uscasv2"

 [[dynamic-neighbors]]
  [dynamic-neighbors.config]
    prefix = '192.168.189.216/29'
    peer-group = "uscasv2"


 [[dynamic-neighbors]]
  [dynamic-neighbors.config]
    prefix = '192.168.189.224/27'
    peer-group = "uscasv2"
  1. syscall.syscall use most of cpu
  2. log output
{"Key":"10.3.0.128/25","Topic":"Table","level":"debug","msg":"Removing withdrawals","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"computeKnownBestPath knownPathList: 1","time":"2019-09-20T08:55:45Z"}
{"Key":"192.168.189.214","Topic":"Peer","attributes":[],"level":"debug","msg":"received update","nlri":[],"time":"2019-09-20T08:55:45Z","withdrawals":[{"prefix":"10.3.1.128/25"}]}
{"Body":{"Type":9,"Instance":0,"Flags":5,"Message":5,"SAFI":1,"Prefix":{"Family":0,"PrefixLen":25,"Prefix":"10.3.0.128"},"SrcPrefix":{"Family":0,"PrefixLen":0,"Prefix":""},"Nexthops":[{"Type":0,"VrfId":0,"Ifindex":0,"Gate":"192.168.189.217","BlackholeType":0,"LabelNum":0,"MplsLabels":null,"Onlink":0}],"Distance":0,"Metric":0,"Mtu":0,"Tag":0,"Rmac":[0,0,0,0,0,0],"Api":0},"Header":{"Len":10,"Marker":254,"Version":6,"VrfId":0,"Command":7},"Topic":"Zebra","level":"debug","msg":"send command to zebra","time":"2019-09-20T08:55:45Z"}
{"Key":"10.3.1.128/25","Topic":"Table","level":"debug","msg":"Removing withdrawals","time":"2019-09-20T08:55:45Z"}
{"Key":"192.168.189.217","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":null,"level":"debug","msg":"sent update","nlri":null,"time":"2019-09-20T08:55:45Z","withdrawals":[{"prefix":"10.3.0.128/25"}]}
{"Topic":"Table","level":"debug","msg":"computeKnownBestPath knownPathList: 1","time":"2019-09-20T08:55:45Z"}
{"Key":"192.168.189.215","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.217"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.217"},{"type":10,"value":["1.1.1.1"]}],"level":"debug","msg":"sent update","nlri":[{"prefix":"10.3.0.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":null}
{"Key":"192.168.251.145","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.217"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.217"},{"type":10,"value":["1.1.1.1"]}],"level":"debug","msg":"sent update","nlri":[{"prefix":"10.3.0.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":null}
{"Key":"192.168.189.214","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.217"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.217"},{"type":10,"value":["1.1.1.1"]}],"level":"debug","msg":"sent update","nlri":[{"prefix":"10.3.0.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":null}
{"Key":"192.168.189.214","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.215"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.215"},{"type":10,"value":["1.1.1.2"]}],"level":"debug","msg":"received update","nlri":[{"prefix":"10.3.1.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":[]}
{"Key":"192.168.189.215","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":null,"level":"debug","msg":"sent update","nlri":null,"time":"2019-09-20T08:55:45Z","withdrawals":[{"prefix":"10.3.1.128/25"}]}
{"Key":"192.168.251.145","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.215"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.215"},{"type":10,"value":["1.1.1.1"]}],"level":"debug","msg":"sent update","nlri":[{"prefix":"10.3.1.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":null}
{"Key":"192.168.189.217","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.215"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.215"},{"type":10,"value":["1.1.1.1"]}],"level":"debug","msg":"sent update","nlri":[{"prefix":"10.3.1.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":null}
{"Body":{"Type":9,"Instance":0,"Flags":5,"Message":5,"SAFI":1,"Prefix":{"Family":0,"PrefixLen":25,"Prefix":"10.3.1.128"},"SrcPrefix":{"Family":0,"PrefixLen":0,"Prefix":""},"Nexthops":[{"Type":0,"VrfId":0,"Ifindex":0,"Gate":"192.168.189.215","BlackholeType":0,"LabelNum":0,"MplsLabels":null,"Onlink":0}],"Distance":0,"Metric":0,"Mtu":0,"Tag":0,"Rmac":[0,0,0,0,0,0],"Api":0},"Header":{"Len":10,"Marker":254,"Version":6,"VrfId":0,"Command":7},"Topic":"Zebra","level":"debug","msg":"send command to zebra","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"computeKnownBestPath knownPathList: 2","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByReachableNexthop -- path1: { 10.3.1.128/25 | src: { 192.168.189.214 | as: 64513, id: 192.168.189.214, cluster-id: 1.1.1.1 }, nh: 192.168.189.215 }, path2: { 10.3.1.128/25 | src: { 192.168.189.215 | as: 64513, id: 192.168.189.215, cluster-id: 1.1.1.1 }, nh: 192.168.189.215 }","time":"2019-09-20T08:55:45Z"}
{"Key":"192.168.189.214","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.215"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.215"},{"type":10,"value":["1.1.1.1"]}],"level":"debug","msg":"sent update","nlri":[{"prefix":"10.3.1.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":null}
{"Topic":"Table","level":"debug","msg":"enter compareByHighestWeight -- path1: { 10.3.1.128/25 | src: { 192.168.189.214 | as: 64513, id: 192.168.189.214, cluster-id: 1.1.1.1 }, nh: 192.168.189.215 }, path2: { 10.3.1.128/25 | src: { 192.168.189.215 | as: 64513, id: 192.168.189.215, cluster-id: 1.1.1.1 }, nh: 192.168.189.215 }","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByLocalPref","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByLocalOrigin","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByASPath","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"compareByASPath -- l1: 0, l2: 0","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByOrigin","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"compareByOrigin -- origin1: 0, origin2: 0","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByMED","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"compareByMED -- med1: 0, med2: 0","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByASNumber","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"compareByASNumber -- p1Asn: 64513, p2Asn: 64513","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByIGPCost -- path1: { 10.3.1.128/25 | src: { 192.168.189.214 | as: 64513, id: 192.168.189.214, cluster-id: 1.1.1.1 }, nh: 192.168.189.215 }, path2: { 10.3.1.128/25 | src: { 192.168.189.215 | as: 64513, id: 192.168.189.215, cluster-id: 1.1.1.1 }, nh: 192.168.189.215 }","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByRouterID","time":"2019-09-20T08:55:45Z"}
{"Key":"192.168.251.145","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.215"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.215"},{"type":10,"value":["1.1.1.1","1.1.1.2"]}],"level":"debug","msg":"sent update","nlri":[{"prefix":"10.3.1.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":null}
{"Key":"192.168.189.217","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.215"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.215"},{"type":10,"value":["1.1.1.1","1.1.1.2"]}],"level":"debug","msg":"sent update","nlri":[{"prefix":"10.3.1.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":null}
{"Body":{"Type":9,"Instance":0,"Flags":5,"Message":5,"SAFI":1,"Prefix":{"Family":0,"PrefixLen":25,"Prefix":"10.3.1.128"},"SrcPrefix":{"Family":0,"PrefixLen":0,"Prefix":""},"Nexthops":[{"Type":0,"VrfId":0,"Ifindex":0,"Gate":"192.168.189.215","BlackholeType":0,"LabelNum":0,"MplsLabels":null,"Onlink":0}],"Distance":0,"Metric":0,"Mtu":0,"Tag":0,"Rmac":[0,0,0,0,0,0],"Api":0},"Header":{"Len":10,"Marker":254,"Version":6,"VrfId":0,"Command":7},"Topic":"Zebra","level":"debug","msg":"send command to zebra","time":"2019-09-20T08:55:45Z"}
{"Key":"192.168.189.214","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.217"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.217"},{"type":10,"value":["1.1.1.2"]}],"level":"debug","msg":"received update","nlri":[{"prefix":"10.3.0.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":[]}
{"Key":"192.168.189.215","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.215"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.215"},{"type":10,"value":["1.1.1.1","1.1.1.2"]}],"level":"debug","msg":"sent update","nlri":[{"prefix":"10.3.1.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":null}
{"Key":"192.168.189.214","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":null,"level":"debug","msg":"sent update","nlri":null,"time":"2019-09-20T08:55:45Z","withdrawals":[{"prefix":"10.3.1.128/25"}]}
{"Topic":"Table","level":"debug","msg":"computeKnownBestPath knownPathList: 2","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByReachableNexthop -- path1: { 10.3.0.128/25 | src: { 192.168.189.214 | as: 64513, id: 192.168.189.214, cluster-id: 1.1.1.1 }, nh: 192.168.189.217 }, path2: { 10.3.0.128/25 | src: { 192.168.189.217 | as: 64513, id: 192.168.189.217, cluster-id: 1.1.1.1 }, nh: 192.168.189.217 }","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByHighestWeight -- path1: { 10.3.0.128/25 | src: { 192.168.189.214 | as: 64513, id: 192.168.189.214, cluster-id: 1.1.1.1 }, nh: 192.168.189.217 }, path2: { 10.3.0.128/25 | src: { 192.168.189.217 | as: 64513, id: 192.168.189.217, cluster-id: 1.1.1.1 }, nh: 192.168.189.217 }","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByLocalPref","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByLocalOrigin","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByASPath","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"compareByASPath -- l1: 0, l2: 0","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByOrigin","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"compareByOrigin -- origin1: 0, origin2: 0","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByMED","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"compareByMED -- med1: 0, med2: 0","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByASNumber","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"compareByASNumber -- p1Asn: 64513, p2Asn: 64513","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByIGPCost -- path1: { 10.3.0.128/25 | src: { 192.168.189.214 | as: 64513, id: 192.168.189.214, cluster-id: 1.1.1.1 }, nh: 192.168.189.217 }, path2: { 10.3.0.128/25 | src: { 192.168.189.217 | as: 64513, id: 192.168.189.217, cluster-id: 1.1.1.1 }, nh: 192.168.189.217 }","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByRouterID","time":"2019-09-20T08:55:45Z"}
{"Key":"192.168.189.214","Topic":"Peer","attributes":[],"level":"debug","msg":"received update","nlri":[],"time":"2019-09-20T08:55:45Z","withdrawals":[{"prefix":"10.3.0.128/25"}]}
{"Body":{"Type":9,"Instance":0,"Flags":5,"Message":5,"SAFI":1,"Prefix":{"Family":0,"PrefixLen":25,"Prefix":"10.3.0.128"},"SrcPrefix":{"Family":0,"PrefixLen":0,"Prefix":""},"Nexthops":[{"Type":0,"VrfId":0,"Ifindex":0,"Gate":"192.168.189.217","BlackholeType":0,"LabelNum":0,"MplsLabels":null,"Onlink":0}],"Distance":0,"Metric":0,"Mtu":0,"Tag":0,"Rmac":[0,0,0,0,0,0],"Api":0},"Header":{"Len":10,"Marker":254,"Version":6,"VrfId":0,"Command":7},"Topic":"Zebra","level":"debug","msg":"send command to zebra","time":"2019-09-20T08:55:45Z"}
{"Key":"10.3.0.128/25","Topic":"Table","level":"debug","msg":"Removing withdrawals","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"computeKnownBestPath knownPathList: 1","time":"2019-09-20T08:55:45Z"}
{"Key":"192.168.251.145","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.217"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.217"},{"type":10,"value":["1.1.1.1","1.1.1.2"]}],"level":"debug","msg":"sent update","nlri":[{"prefix":"10.3.0.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":null}
{"Key":"192.168.189.214","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":null,"level":"debug","msg":"sent update","nlri":null,"time":"2019-09-20T08:55:45Z","withdrawals":[{"prefix":"10.3.0.128/25"}]}
{"Key":"192.168.189.215","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.217"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.217"},{"type":10,"value":["1.1.1.1","1.1.1.2"]}],"level":"debug","msg":"sent update","nlri":[{"prefix":"10.3.0.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":null}
{"Key":"192.168.189.217","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.217"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.217"},{"type":10,"value":["1.1.1.1","1.1.1.2"]}],"level":"debug","msg":"sent update","nlri":[{"prefix":"10.3.0.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":null}
{"Body":{"Type":9,"Instance":0,"Flags":5,"Message":5,"SAFI":1,"Prefix":{"Family":0,"PrefixLen":25,"Prefix":"10.3.0.128"},"SrcPrefix":{"Family":0,"PrefixLen":0,"Prefix":""},"Nexthops":[{"Type":0,"VrfId":0,"Ifindex":0,"Gate":"192.168.189.217","BlackholeType":0,"LabelNum":0,"MplsLabels":null,"Onlink":0}],"Distance":0,"Metric":0,"Mtu":0,"Tag":0,"Rmac":[0,0,0,0,0,0],"Api":0},"Header":{"Len":10,"Marker":254,"Version":6,"VrfId":0,"Command":7},"Topic":"Zebra","level":"debug","msg":"send command to zebra","time":"2019-09-20T08:55:45Z"}
{"Key":"192.168.189.214","Topic":"Peer","attributes":[],"level":"debug","msg":"received update","nlri":[],"time":"2019-09-20T08:55:45Z","withdrawals":[{"prefix":"10.3.1.128/25"}]}
{"Key":"192.168.251.145","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.217"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.217"},{"type":10,"value":["1.1.1.1"]}],"level":"debug","msg":"sent update","nlri":[{"prefix":"10.3.0.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":null}
{"Key":"10.3.1.128/25","Topic":"Table","level":"debug","msg":"Removing withdrawals","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"computeKnownBestPath knownPathList: 1","time":"2019-09-20T08:55:45Z"}
{"Key":"192.168.189.215","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.217"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.217"},{"type":10,"value":["1.1.1.1"]}],"level":"debug","msg":"sent update","nlri":[{"prefix":"10.3.0.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":null}
{"Key":"192.168.189.217","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":null,"level":"debug","msg":"sent update","nlri":null,"time":"2019-09-20T08:55:45Z","withdrawals":[{"prefix":"10.3.0.128/25"}]}
{"Key":"192.168.189.214","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.217"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.217"},{"type":10,"value":["1.1.1.1"]}],"level":"debug","msg":"sent update","nlri":[{"prefix":"10.3.0.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":null}
{"Body":{"Type":9,"Instance":0,"Flags":5,"Message":5,"SAFI":1,"Prefix":{"Family":0,"PrefixLen":25,"Prefix":"10.3.1.128"},"SrcPrefix":{"Family":0,"PrefixLen":0,"Prefix":""},"Nexthops":[{"Type":0,"VrfId":0,"Ifindex":0,"Gate":"192.168.189.215","BlackholeType":0,"LabelNum":0,"MplsLabels":null,"Onlink":0}],"Distance":0,"Metric":0,"Mtu":0,"Tag":0,"Rmac":[0,0,0,0,0,0],"Api":0},"Header":{"Len":10,"Marker":254,"Version":6,"VrfId":0,"Command":7},"Topic":"Zebra","level":"debug","msg":"send command to zebra","time":"2019-09-20T08:55:45Z"}
{"Key":"192.168.189.214","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.215"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.215"},{"type":10,"value":["1.1.1.2"]}],"level":"debug","msg":"received update","nlri":[{"prefix":"10.3.1.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":[]}
{"Key":"192.168.251.145","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.215"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.215"},{"type":10,"value":["1.1.1.1"]}],"level":"debug","msg":"sent update","nlri":[{"prefix":"10.3.1.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":null}
{"Key":"192.168.189.214","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.215"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.215"},{"type":10,"value":["1.1.1.1"]}],"level":"debug","msg":"sent update","nlri":[{"prefix":"10.3.1.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":null}
{"Key":"192.168.189.215","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":null,"level":"debug","msg":"sent update","nlri":null,"time":"2019-09-20T08:55:45Z","withdrawals":[{"prefix":"10.3.1.128/25"}]}
{"Key":"192.168.189.217","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.215"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.215"},{"type":10,"value":["1.1.1.1"]}],"level":"debug","msg":"sent update","nlri":[{"prefix":"10.3.1.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":null}
{"Topic":"Table","level":"debug","msg":"computeKnownBestPath knownPathList: 2","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByReachableNexthop -- path1: { 10.3.1.128/25 | src: { 192.168.189.214 | as: 64513, id: 192.168.189.214, cluster-id: 1.1.1.1 }, nh: 192.168.189.215 }, path2: { 10.3.1.128/25 | src: { 192.168.189.215 | as: 64513, id: 192.168.189.215, cluster-id: 1.1.1.1 }, nh: 192.168.189.215 }","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByHighestWeight -- path1: { 10.3.1.128/25 | src: { 192.168.189.214 | as: 64513, id: 192.168.189.214, cluster-id: 1.1.1.1 }, nh: 192.168.189.215 }, path2: { 10.3.1.128/25 | src: { 192.168.189.215 | as: 64513, id: 192.168.189.215, cluster-id: 1.1.1.1 }, nh: 192.168.189.215 }","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByLocalPref","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByLocalOrigin","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByASPath","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"compareByASPath -- l1: 0, l2: 0","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByOrigin","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"compareByOrigin -- origin1: 0, origin2: 0","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByMED","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"compareByMED -- med1: 0, med2: 0","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByASNumber","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"compareByASNumber -- p1Asn: 64513, p2Asn: 64513","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByIGPCost -- path1: { 10.3.1.128/25 | src: { 192.168.189.214 | as: 64513, id: 192.168.189.214, cluster-id: 1.1.1.1 }, nh: 192.168.189.215 }, path2: { 10.3.1.128/25 | src: { 192.168.189.215 | as: 64513, id: 192.168.189.215, cluster-id: 1.1.1.1 }, nh: 192.168.189.215 }","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByRouterID","time":"2019-09-20T08:55:45Z"}
{"Key":"192.168.189.214","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.217"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.217"},{"type":10,"value":["1.1.1.2"]}],"level":"debug","msg":"received update","nlri":[{"prefix":"10.3.0.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":[]}
{"Body":{"Type":9,"Instance":0,"Flags":5,"Message":5,"SAFI":1,"Prefix":{"Family":0,"PrefixLen":25,"Prefix":"10.3.1.128"},"SrcPrefix":{"Family":0,"PrefixLen":0,"Prefix":""},"Nexthops":[{"Type":0,"VrfId":0,"Ifindex":0,"Gate":"192.168.189.215","BlackholeType":0,"LabelNum":0,"MplsLabels":null,"Onlink":0}],"Distance":0,"Metric":0,"Mtu":0,"Tag":0,"Rmac":[0,0,0,0,0,0],"Api":0},"Header":{"Len":10,"Marker":254,"Version":6,"VrfId":0,"Command":7},"Topic":"Zebra","level":"debug","msg":"send command to zebra","time":"2019-09-20T08:55:45Z"}
{"Key":"192.168.189.214","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":null,"level":"debug","msg":"sent update","nlri":null,"time":"2019-09-20T08:55:45Z","withdrawals":[{"prefix":"10.3.1.128/25"}]}
{"Topic":"Table","level":"debug","msg":"computeKnownBestPath knownPathList: 2","time":"2019-09-20T08:55:45Z"}
{"Key":"192.168.189.217","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.215"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.215"},{"type":10,"value":["1.1.1.1","1.1.1.2"]}],"level":"debug","msg":"sent update","nlri":[{"prefix":"10.3.1.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":null}
{"Key":"192.168.251.145","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.215"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.215"},{"type":10,"value":["1.1.1.1","1.1.1.2"]}],"level":"debug","msg":"sent update","nlri":[{"prefix":"10.3.1.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":null}
{"Topic":"Table","level":"debug","msg":"enter compareByReachableNexthop -- path1: { 10.3.0.128/25 | src: { 192.168.189.214 | as: 64513, id: 192.168.189.214, cluster-id: 1.1.1.1 }, nh: 192.168.189.217 }, path2: { 10.3.0.128/25 | src: { 192.168.189.217 | as: 64513, id: 192.168.189.217, cluster-id: 1.1.1.1 }, nh: 192.168.189.217 }","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByHighestWeight -- path1: { 10.3.0.128/25 | src: { 192.168.189.214 | as: 64513, id: 192.168.189.214, cluster-id: 1.1.1.1 }, nh: 192.168.189.217 }, path2: { 10.3.0.128/25 | src: { 192.168.189.217 | as: 64513, id: 192.168.189.217, cluster-id: 1.1.1.1 }, nh: 192.168.189.217 }","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByLocalPref","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByLocalOrigin","time":"2019-09-20T08:55:45Z"}
{"Key":"192.168.189.215","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.215"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.215"},{"type":10,"value":["1.1.1.1","1.1.1.2"]}],"level":"debug","msg":"sent update","nlri":[{"prefix":"10.3.1.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":null}
{"Topic":"Table","level":"debug","msg":"enter compareByASPath","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"compareByASPath -- l1: 0, l2: 0","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByOrigin","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"compareByOrigin -- origin1: 0, origin2: 0","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByMED","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"compareByMED -- med1: 0, med2: 0","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByASNumber","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"compareByASNumber -- p1Asn: 64513, p2Asn: 64513","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByIGPCost -- path1: { 10.3.0.128/25 | src: { 192.168.189.214 | as: 64513, id: 192.168.189.214, cluster-id: 1.1.1.1 }, nh: 192.168.189.217 }, path2: { 10.3.0.128/25 | src: { 192.168.189.217 | as: 64513, id: 192.168.189.217, cluster-id: 1.1.1.1 }, nh: 192.168.189.217 }","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"enter compareByRouterID","time":"2019-09-20T08:55:45Z"}
{"Key":"192.168.189.214","Topic":"Peer","attributes":[],"level":"debug","msg":"received update","nlri":[],"time":"2019-09-20T08:55:45Z","withdrawals":[{"prefix":"10.3.0.128/25"}]}
{"Key":"192.168.251.145","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.217"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.217"},{"type":10,"value":["1.1.1.1","1.1.1.2"]}],"level":"debug","msg":"sent update","nlri":[{"prefix":"10.3.0.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":null}
{"Key":"192.168.189.217","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.217"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.217"},{"type":10,"value":["1.1.1.1","1.1.1.2"]}],"level":"debug","msg":"sent update","nlri":[{"prefix":"10.3.0.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":null}
{"Key":"192.168.189.215","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.217"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.217"},{"type":10,"value":["1.1.1.1","1.1.1.2"]}],"level":"debug","msg":"sent update","nlri":[{"prefix":"10.3.0.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":null}
{"Key":"10.3.0.128/25","Topic":"Table","level":"debug","msg":"Removing withdrawals","time":"2019-09-20T08:55:45Z"}
{"Body":{"Type":9,"Instance":0,"Flags":5,"Message":5,"SAFI":1,"Prefix":{"Family":0,"PrefixLen":25,"Prefix":"10.3.0.128"},"SrcPrefix":{"Family":0,"PrefixLen":0,"Prefix":""},"Nexthops":[{"Type":0,"VrfId":0,"Ifindex":0,"Gate":"192.168.189.217","BlackholeType":0,"LabelNum":0,"MplsLabels":null,"Onlink":0}],"Distance":0,"Metric":0,"Mtu":0,"Tag":0,"Rmac":[0,0,0,0,0,0],"Api":0},"Header":{"Len":10,"Marker":254,"Version":6,"VrfId":0,"Command":7},"Topic":"Zebra","level":"debug","msg":"send command to zebra","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"computeKnownBestPath knownPathList: 1","time":"2019-09-20T08:55:45Z"}
{"Key":"192.168.189.214","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":null,"level":"debug","msg":"sent update","nlri":null,"time":"2019-09-20T08:55:45Z","withdrawals":[{"prefix":"10.3.0.128/25"}]}
{"Body":{"Type":9,"Instance":0,"Flags":5,"Message":5,"SAFI":1,"Prefix":{"Family":0,"PrefixLen":25,"Prefix":"10.3.0.128"},"SrcPrefix":{"Family":0,"PrefixLen":0,"Prefix":""},"Nexthops":[{"Type":0,"VrfId":0,"Ifindex":0,"Gate":"192.168.189.217","BlackholeType":0,"LabelNum":0,"MplsLabels":null,"Onlink":0}],"Distance":0,"Metric":0,"Mtu":0,"Tag":0,"Rmac":[0,0,0,0,0,0],"Api":0},"Header":{"Len":10,"Marker":254,"Version":6,"VrfId":0,"Command":7},"Topic":"Zebra","level":"debug","msg":"send command to zebra","time":"2019-09-20T08:55:45Z"}
{"Key":"192.168.189.214","Topic":"Peer","attributes":[],"level":"debug","msg":"received update","nlri":[],"time":"2019-09-20T08:55:45Z","withdrawals":[{"prefix":"10.3.1.128/25"}]}
{"Key":"192.168.189.217","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":null,"level":"debug","msg":"sent update","nlri":null,"time":"2019-09-20T08:55:45Z","withdrawals":[{"prefix":"10.3.0.128/25"}]}
{"Key":"192.168.251.145","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.217"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.217"},{"type":10,"value":["1.1.1.1"]}],"level":"debug","msg":"sent update","nlri":[{"prefix":"10.3.0.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":null}
{"Key":"192.168.189.214","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.217"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.217"},{"type":10,"value":["1.1.1.1"]}],"level":"debug","msg":"sent update","nlri":[{"prefix":"10.3.0.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":null}
{"Key":"192.168.189.215","State":"BGP_FSM_ESTABLISHED","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.217"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.217"},{"type":10,"value":["1.1.1.1"]}],"level":"debug","msg":"sent update","nlri":[{"prefix":"10.3.0.128/25"}],"time":"2019-09-20T08:55:45Z","withdrawals":null}
{"Key":"10.3.1.128/25","Topic":"Table","level":"debug","msg":"Removing withdrawals","time":"2019-09-20T08:55:45Z"}
{"Topic":"Table","level":"debug","msg":"computeKnownBestPath knownPathList: 1","time":"2019-09-20T08:55:45Z"}
{"Key":"192.168.189.214","Topic":"Peer","attributes":[{"type":1,"value":0},{"type":2,"as_paths":null},{"type":3,"nexthop":"192.168.189.215"},{"type":4,"metric":0},{"type":5,"value":100},{"type":9,"value":"192.168.189.215"},{"type":10,"value":["1.1.1.2"]}],"level":"debug","msg":"received update","nlri":[{"prefix":"10.3.1.128/25"}],"time":"2019-09-20T08:55:45Z"
@wu0407
Copy link
Author

wu0407 commented Sep 22, 2019 •
edited
Loading

go pprof file
pprof.gobgpd.samples.cpu.001.pb.gz

@fujita
Copy link
Member

fujita commented Sep 26, 2019 •
edited
Loading

I can't reproduce this. Please send the minimum configurations that cause this. For example, this happens without route reflector configuration? how about zebra? What neighbors matter? Please drop all the unrelated configurations.

@wu0407
Copy link
Author

wu0407 commented Oct 7, 2019 •
edited
Loading

use zebra of frr 7.1 daemon run with " -A 127.0.0.1 -s 90000000 --log stdout --log-level info -k 60 -r -z /var/run/frr/zserv.api" .
without dynamic group anything is ok.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@fujita @wu0407