Skip to content

Conversation

@khluu
Copy link
Contributor

@khluu khluu commented Nov 6, 2024

REGRESSION 10.62%: tasks_per_second (THROUGHPUT) regresses from 524.9117000820374 to 469.18167402268733 in benchmarks/many_tasks.json
REGRESSION 8.06%: pgs_per_second (THROUGHPUT) regresses from 22.52798235601382 to 20.71303274352877 in benchmarks/many_pgs.json
REGRESSION 7.44%: multi_client_put_gigabytes (THROUGHPUT) regresses from 46.31338079710033 to 42.866645929309996 in microbenchmark.json
REGRESSION 7.35%: client__1_1_actor_calls_concurrent (THROUGHPUT) regresses from 1051.493235107139 to 974.1707706196564 in microbenchmark.json
REGRESSION 6.17%: client__1_1_actor_calls_async (THROUGHPUT) regresses from 1045.2530250056586 to 980.7790835269525 in microbenchmark.json
REGRESSION 4.68%: single_client_get_object_containing_10k_refs (THROUGHPUT) regresses from 12.639316158564123 to 12.047828247398513 in microbenchmark.json
REGRESSION 3.91%: n_n_async_actor_calls_async (THROUGHPUT) regresses from 23929.36598717011 to 22994.669871663224 in microbenchmark.json
REGRESSION 3.52%: single_client_put_gigabytes (THROUGHPUT) regresses from 17.802093535903047 to 17.176054266128194 in microbenchmark.json
REGRESSION 3.10%: tasks_per_second (THROUGHPUT) regresses from 360.10290607052235 to 348.9277364300741 in benchmarks/many_nodes.json
REGRESSION 2.53%: 1_n_actor_calls_async (THROUGHPUT) regresses from 8623.73827147855 to 8405.676715400055 in microbenchmark.json
REGRESSION 1.83%: single_client_tasks_async (THROUGHPUT) regresses from 7997.547070810859 to 7850.804688217522 in microbenchmark.json
REGRESSION 1.80%: multi_client_tasks_async (THROUGHPUT) regresses from 22222.678238606593 to 21823.517428742365 in microbenchmark.json
REGRESSION 0.58%: n_n_actor_calls_async (THROUGHPUT) regresses from 27090.36785253979 to 26933.018765424662 in microbenchmark.json
REGRESSION 191.94%: dashboard_p95_latency_ms (LATENCY) regresses from 8.773 to 25.612 in benchmarks/many_nodes.json
REGRESSION 176.87%: dashboard_p99_latency_ms (LATENCY) regresses from 362.19 to 1002.78 in benchmarks/many_pgs.json
REGRESSION 99.14%: time_to_broadcast_1073741824_bytes_to_50_nodes (LATENCY) regresses from 31.073800462999998 to 61.879495881000025 in scalability/object_store.json
REGRESSION 21.93%: stage_2_avg_iteration_time (LATENCY) regresses from 66.36001825332642 to 80.91288795471192 in stress_tests/stress_test_many_tasks.json
REGRESSION 14.32%: dashboard_p99_latency_ms (LATENCY) regresses from 84.553 to 96.658 in benchmarks/many_nodes.json
REGRESSION 10.41%: stage_0_time (LATENCY) regresses from 8.909741163253784 to 9.837347507476807 in stress_tests/stress_test_many_tasks.json
REGRESSION 9.94%: dashboard_p50_latency_ms (LATENCY) regresses from 4.308 to 4.736 in benchmarks/many_nodes.json
REGRESSION 8.74%: 107374182400_large_object_time (LATENCY) regresses from 28.976551088000008 to 31.510281453000005 in scalability/single_node.json
REGRESSION 6.48%: dashboard_p50_latency_ms (LATENCY) regresses from 126.428 to 134.62 in benchmarks/many_tasks.json
REGRESSION 5.17%: dashboard_p50_latency_ms (LATENCY) regresses from 90.11 to 94.771 in benchmarks/many_actors.json
REGRESSION 5.04%: stage_3_time (LATENCY) regresses from 3249.9383351802826 to 3413.631863594055 in stress_tests/stress_test_many_tasks.json
REGRESSION 1.37%: stage_4_spread (LATENCY) regresses from 0.3168361946930436 to 0.32116315011886526 in stress_tests/stress_test_many_tasks.json
REGRESSION 0.26%: 3000_returns_time (LATENCY) regresses from 5.854871097 to 5.869913475000004 in scalability/single_node.json

@jjyao jjyao self-assigned this Nov 6, 2024
@jjyao
Copy link
Collaborator

jjyao commented Nov 6, 2024

REGRESSION 10.62%: tasks_per_second (THROUGHPUT) regresses from 524.9117000820374 to 469.18167402268733 in benchmarks/many_tasks.json

Infra side issue.

REGRESSION 99.14%: time_to_broadcast_1073741824_bytes_to_50_nodes (LATENCY) regresses from 31.073800462999998 to 61.879495881000025 in scalability/object_store.json

Noise.

REGRESSION 21.93%: stage_2_avg_iteration_time (LATENCY) regresses from 66.36001825332642 to 80.91288795471192 in stress_tests/stress_test_many_tasks.json

Noise.

@jjyao jjyao added the go add ONLY when ready to merge, run all tests label Nov 6, 2024
@jjyao jjyao merged commit a23bb65 into master Nov 12, 2024
1 check passed
@jjyao jjyao deleted the 2.39.0_perf_metrics branch November 12, 2024 21:45
JP-sDEV pushed a commit to JP-sDEV/ray that referenced this pull request Nov 14, 2024
mohitjain2504 pushed a commit to mohitjain2504/ray that referenced this pull request Nov 15, 2024
Signed-off-by: Jiajun Yao <[email protected]>
Signed-off-by: mohitjain2504 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-backlog go add ONLY when ready to merge, run all tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants