Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stress tests have a lot of errors #72

Closed
NicoSpec opened this issue Dec 22, 2017 · 6 comments
Closed

Stress tests have a lot of errors #72

NicoSpec opened this issue Dec 22, 2017 · 6 comments

Comments

@NicoSpec
Copy link

we used tsung to test dpvs appearing lots of errors

Name |Highest Rate | Total number

error_abort_max_conn_retries | 99.3 / sec | 2557
error_abort_max_send_retries | 3.6 / sec | 73
error_connect_eaddrinuse | 21.7 / sec | 779
error_connect_etimedout | 133 / sec | 20857
error_connection_closed | 17.2 / sec | 360
error_timeout | 39.5 / sec | 811

@ywc689
Copy link
Collaborator

ywc689 commented Dec 27, 2017

How much traffic used for the stress tests? Was there any error logs in the system?
Possibly not enough local IPs configured. Try 'dpip addr show' to find if there exists sa_miss in stress test. If so, configure more local IPs and try again.

@beacer
Copy link
Contributor

beacer commented Jan 5, 2018

Any update @NicoSpec . Can you show us the test env and configure? for example the client tool, server, DPVS's configure file. and dpvs's log.

@NicoSpec
Copy link
Author

NicoSpec commented Jan 8, 2018

our dpvs is DR mode.
our test tool is tsung,concurrency is 50,000,test page is 10 bytes.
DPVS hardware:
NIC is Intel Gigabit Ethernet.
CPU is Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz.

DPVS's configure is default configure file.

dpvs's log have some error's log,for example:
IPVS: dp_vs_sched_persist: persist-schedule: no dest found.
IPVS: dp_vs_sched_persist: persist-schedule: no dest found.
IPVS: dp_vs_sched_persist: persist-schedule: no dest found.
IPVS: dp_vs_sched_persist: persist-schedule: no dest found.
IPVS: dp_vs_sched_persist: persist-schedule: no dest found.
IPVS: dp_vs_sched_persist: persist-schedule: no dest found.
IPVS: dp_vs_sched_persist: persist-schedule: no dest found.
IPVS: dp_vs_sched_persist: persist-schedule: no dest found.
IPVS: dp_vs_sched_persist: persist-schedule: no dest found.
IPVS: dp_vs_sched_persist: persist-schedule: no dest found.
IPVS: dp_vs_sched_persist: persist-schedule: no dest found.
IPVS: dp_vs_sched_persist: persist-schedule: no dest found.

This is tsung's error reports.
default
This is tsung's RPS reports:
default

@beacer
Copy link
Contributor

beacer commented Jan 8, 2018

Can you show the output of dpip addr show, ipvsadm -ln, ipvsadm -G, dpip -s link show , dpip -s link show cpu ?
The log persist-schedule: no dest found. could be warning not error, the tool we're using are wrk, ab and webbench, while wrk is highly recommend.

@ywc689
Copy link
Collaborator

ywc689 commented Jan 8, 2018

1no dest found indicates your backend servers may be the bottleneck of the stress test. Add more backend servers, or use high performance servers instead.

Are you really need persistent connections? If not, turn off it may improve performance.

@NicoSpec
Copy link
Author

NicoSpec commented Jan 9, 2018

Hi,
@beacer @ywc689
Now,we use "Intel(R) 10 Gigabit PCI Express Network" to test dpvs and close nginx's persistent connection.dpvs's log haven't any warning and tsung's report almost no error.
PPS also reached the official test results.
Thank you very much for your support.

@NicoSpec NicoSpec closed this as completed Jan 9, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants