Not able to see http requests transmitted from loadbalancer at endpoint

[megarajan]

hi

I created a loadbalancer docker container in one of my ubuntu vm and have a http server in another docker container in another vm.

Created the loadbalancer using below cmd :

loxicmd create lb 10.34.0.100 --tcp=2020:8080 --endpoints=10.37.0.10:1

Added the VIP to loxilb container using the below cmd:

sudo ip netns exec loxilb ip addr add 10.34.0.100/32 dev lo

Verified VIP is added in loxilb container

root@aef9cc10ec19:/# ip addr show lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet 10.10.10.3/32 scope global lo
valid_lft forever preferred_lft forever
inet 10.34.0.100/32 scope global lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
root@aef9cc10ec19:/#

root@aef9cc10ec19:/# loxicmd get lb -o wide

EXT IP	SEC IPS	HOST	PORT	PROTO	NAME	MARK	SEL	MODE	ENDPOINT	EPORT	WEIGHT	STATE	COUNTERS
10.34.0.100			2020	tcp		0	rr	default	10.37.0.10	8080	1	-	19:1406
root@aef9cc10ec19:/#
root@aef9cc10ec19:/#

Connectivty to endpoint from lb is ok

sudo ip netns exec loxilb ping 10.37.0.10
PING 10.37.0.10 (10.37.0.10) 56(84) bytes of data.
64 bytes from 10.37.0.10: icmp_seq=1 ttl=64 time=1.98 ms
^C
--- 10.37.0.10 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.980/1.980/1.980/0.000 ms
root@pn-distributor-3:/home/cps#

But the request does not reach the endpoint

sudo ip netns exec loxilb tcpdump -i ethwe port 2020
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ethwe, link-type EN10MB (Ethernet), capture size 262144 bytes

11:46:32.110960 IP 10.32.0.2.44080 > 10.34.0.100.2020: Flags [S], seq 2803844390, win 65535, options [mss 1336,sackOK,TS val 3636009498 ecr 0,nop,wscale 9], length 0
11:46:33.122365 IP 10.32.0.2.44080 > 10.34.0.100.2020: Flags [S], seq 2803844390, win 65535, options [mss 1336,sackOK,TS val 3636010510 ecr 0,nop,wscale 9], length 0
11:46:35.138391 IP 10.32.0.2.44080 > 10.34.0.100.2020: Flags [S], seq 2803844390, win 65535, options [mss 1336,sackOK,TS val 3636012526 ecr 0,nop,wscale 9], length 0
11:46:39.174356 IP 10.32.0.2.44080 > 10.34.0.100.2020: Flags [S], seq 2803844390, win 65535, options [mss 1336,sackOK,TS val 3636016562 ecr 0,nop,wscale 9], length 0

I dont see the loadbalancer requests at the endpoint container

any clues on what I could be missing ?

[TrekkieCoder]

Can you please check :

1. tcpdump on port 8080
2. loxicmd get ct in loxilb docker

[megarajan]

thanks. It seems the requests indeed reach the endpoint container.

tcpdump at loxilb container

sudo ip netns exec loxilb tcpdump port 2020 or port 8080
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ethwe, link-type EN10MB (Ethernet), capture size 262144 bytes
12:51:52.011150 IP 10.32.0.2.39220 > 10.34.0.100.2020: Flags [S], seq 2270447176, win 65535, options [mss 1336,sackOK,TS val 3726329396 ecr 0,nop,wscale 9], length 0
12:51:52.011203 IP 10.32.0.2.39220 > 10.37.0.10.http-alt: Flags [S], seq 2270447176, win 65535, options [mss 1336,sackOK,TS val 3726329396 ecr 0,nop,wscale 9], length 0
12:51:53.032229 IP 10.32.0.2.39220 > 10.34.0.100.2020: Flags [S], seq 2270447176, win 65535, options [mss 1336,sackOK,TS val 3726330418 ecr 0,nop,wscale 9], length 0
12:51:53.032263 IP 10.32.0.2.39220 > 10.37.0.10.http-alt: Flags [S], seq 2270447176, win 65535, options [mss 1336,sackOK,TS val 3726330418 ecr 0,nop,wscale 9], length 0
12:51:55.044238 IP 10.32.0.2.39220 > 10.34.0.100.2020: Flags [S], seq 2270447176, win 65535, options [mss 1336,sackOK,TS val 3726332430 ecr 0,nop,wscale 9], length 0
12:51:55.044268 IP 10.32.0.2.39220 > 10.37.0.10.http-alt: Flags [S], seq 2270447176, win 65535, options [mss 1336,sackOK,TS val 3726332430 ecr 0,nop,wscale 9], length 0
12:51:59.300487 IP 10.32.0.2.39220 > 10.34.0.100.2020: Flags [S], seq 2270447176, win 65535, options [mss 1336,sackOK,TS val 3726336686 ecr 0,nop,wscale 9], length 0
12:51:59.300542 IP 10.32.0.2.39220 > 10.37.0.10.http-alt: Flags [S], seq 2270447176, win 65535, options [mss 1336,sackOK,TS val 3726336686 ecr 0,nop,wscale 9], length 0

endpoint container

root@diameter-endpoint-s106:/# tcpdump src 10.32.0.2 and port 8080
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ethwe, link-type EN10MB (Ethernet), capture size 262144 bytes
12:51:52.011380 IP orchestrator.weave.local.39220 > diameter-endpoint-s106.weave.local.http-alt: Flags [S], seq 2270447176, win 65535, options [mss 1336,sackOK,TS val 3726329396 ecr 0,nop,wscale 9], length 0
12:51:52.011999 IP orchestrator.weave.local.39220 > diameter-endpoint-s106.weave.local.http-alt: Flags [R], seq 2270447177, win 0, length 0
12:51:53.032418 IP orchestrator.weave.local.39220 > diameter-endpoint-s106.weave.local.http-alt: Flags [S], seq 2270447176, win 65535, options [mss 1336,sackOK,TS val 3726330418 ecr 0,nop,wscale 9], length 0
12:51:53.032806 IP orchestrator.weave.local.39220 > diameter-endpoint-s106.weave.local.http-alt: Flags [R], seq 2270447177, win 0, length 0
12:51:55.044411 IP orchestrator.weave.local.39220 > diameter-endpoint-s106.weave.local.http-alt: Flags [S], seq 2270447176, win 65535, options [mss 1336,sackOK,TS val 3726332430 ecr 0,nop,wscale 9], length 0
12:51:55.044773 IP orchestrator.weave.local.39220 > diameter-endpoint-s106.weave.local.http-alt: Flags [R], seq 2270447177, win 0, length 0
12:51:59.300714 IP orchestrator.weave.local.39220 > diameter-endpoint-s106.weave.local.http-alt: Flags [S], seq 2270447176, win 65535, options [mss 1336,sackOK,TS val 3726336686 ecr 0,nop,wscale 9], length 0
12:51:59.301438 IP orchestrator.weave.local.39220 > diameter-endpoint-s106.weave.local.http-alt: Flags [R], seq 2270447177, win 0, length 0

[megarajan]

loxicmd get ct is not available in my image.

but the counters is increased in get lb output

root@ee1a4a14006e:/# loxicmd get lb -o wide

EXT IP	SEC IPS	HOST	PORT	PROTO	NAME	MARK	SEL	MODE	ENDPOINT	EPORT	WEIGHT	STATE	COUNTERS
10.34.0.100			2020	tcp		0	rr	default	10.37.0.10	8080	1	-	39:2886
root@ee1a4a14006e:/#
root@ee1a4a14006e:/#

[TrekkieCoder]

In my opinion, you should create the rule with additional parameter --mode=onearm.

[TrekkieCoder]

Also, please check this post for further info.

[megarajan]

thanks let me check

[megarajan]

thanks adding onearm worked