Skip to content

mark flaky Exception Replay integration test #9970

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Nov 18, 2025
Merged

mark flaky Exception Replay integration test #9970

merged 3 commits into from
Nov 18, 2025

Conversation

@jpbempel
Copy link
Member

@jpbempel jpbempel commented Nov 14, 2025

ExceptionDebuggerIntegrationTest::test3CapturedRecursiveFrames

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@jpbempel jpbempel requested a review from a team as a code owner November 14, 2025 13:40
@jpbempel jpbempel added type: bug Bug report and fix tag: no release notes Changes to exclude from release notes labels Nov 14, 2025
@jpbempel jpbempel requested review from shatzi and removed request for a team November 14, 2025 13:40
@jpbempel jpbempel added the comp: debugger Dynamic Instrumentation label Nov 14, 2025
@jpbempel
mark flaky Exception Replay integration test
e3c7eda 
ExceptionDebuggerIntegrationTest::test3CapturedRecursiveFrames
@jpbempel jpbempel force-pushed the jpbempel/mark-flaky branch from 967d961 to e3c7eda Compare November 14, 2025 13:45
@datadog-datadog-prod-us1

This comment has been minimized.

Sign in to view
@pr-commenter
Copy link

pr-commenter bot commented Nov 17, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/mark-flaky
git_commit_date 1763479872 1763489865
git_commit_sha fb40aba 0731038
release_version 1.56.0-SNAPSHOT~fb40abaeee 1.56.0-SNAPSHOT~0731038baa
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1763491726 1763491726
ci_job_id 1242874618 1242874618
ci_pipeline_id 83002123 83002123
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-5zdycb1z 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-5zdycb1z 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 59 metrics, 6 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.56.0-SNAPSHOT~0731038baa, baseline=1.56.0-SNAPSHOT~fb40abaeee

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.101 s) : 0, 1101084
Total [baseline] (10.817 s) : 0, 10817288
Agent [candidate] (1.096 s) : 0, 1095841
Total [candidate] (10.758 s) : 0, 10757918
section appsec
Agent [baseline] (1.275 s) : 0, 1274689
Total [baseline] (11.01 s) : 0, 11010046
Agent [candidate] (1.292 s) : 0, 1291618
Total [candidate] (11.035 s) : 0, 11034771
section iast
Agent [baseline] (1.235 s) : 0, 1234729
Total [baseline] (11.184 s) : 0, 11184114
Agent [candidate] (1.236 s) : 0, 1236037
Total [candidate] (11.163 s) : 0, 11162753
section profiling
Agent [baseline] (1.227 s) : 0, 1227122
Total [baseline] (11.066 s) : 0, 11066279
Agent [candidate] (1.225 s) : 0, 1224870
Total [candidate] (11.067 s) : 0, 11067000
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.101 s -
Agent appsec 1.275 s 173.604 ms (15.8%)
Agent iast 1.235 s 133.645 ms (12.1%)
Agent profiling 1.227 s 126.038 ms (11.4%)
Total tracing 10.817 s -
Total appsec 11.01 s 192.758 ms (1.8%)
Total iast 11.184 s 366.826 ms (3.4%)
Total profiling 11.066 s 248.991 ms (2.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.096 s -
Agent appsec 1.292 s 195.777 ms (17.9%)
Agent iast 1.236 s 140.196 ms (12.8%)
Agent profiling 1.225 s 129.028 ms (11.8%)
Total tracing 10.758 s -
Total appsec 11.035 s 276.854 ms (2.6%)
Total iast 11.163 s 404.836 ms (3.8%)
Total profiling 11.067 s 309.082 ms (2.9%) 
gantt
    title petclinic - break down per module: candidate=1.56.0-SNAPSHOT~0731038baa, baseline=1.56.0-SNAPSHOT~fb40abaeee

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.449 ms) : 0, 1449
crashtracking [candidate] (1.46 ms) : 0, 1460
BytebuddyAgent [baseline] (706.752 ms) : 0, 706752
BytebuddyAgent [candidate] (703.629 ms) : 0, 703629
GlobalTracer [baseline] (249.0 ms) : 0, 249000
GlobalTracer [candidate] (247.742 ms) : 0, 247742
AppSec [baseline] (32.429 ms) : 0, 32429
AppSec [candidate] (32.013 ms) : 0, 32013
Debugger [baseline] (64.184 ms) : 0, 64184
Debugger [candidate] (63.749 ms) : 0, 63749
Remote Config [baseline] (640.481 µs) : 0, 640
Remote Config [candidate] (634.177 µs) : 0, 634
Telemetry [baseline] (8.144 ms) : 0, 8144
Telemetry [candidate] (8.211 ms) : 0, 8211
Flare Poller [baseline] (3.659 ms) : 0, 3659
Flare Poller [candidate] (3.675 ms) : 0, 3675
section appsec
crashtracking [baseline] (1.441 ms) : 0, 1441
crashtracking [candidate] (1.462 ms) : 0, 1462
BytebuddyAgent [baseline] (726.791 ms) : 0, 726791
BytebuddyAgent [candidate] (737.904 ms) : 0, 737904
GlobalTracer [baseline] (239.327 ms) : 0, 239327
GlobalTracer [candidate] (242.656 ms) : 0, 242656
AppSec [baseline] (174.131 ms) : 0, 174131
AppSec [candidate] (174.703 ms) : 0, 174703
Debugger [baseline] (60.546 ms) : 0, 60546
Debugger [candidate] (61.51 ms) : 0, 61510
Remote Config [baseline] (659.407 µs) : 0, 659
Remote Config [candidate] (699.126 µs) : 0, 699
Telemetry [baseline] (8.173 ms) : 0, 8173
Telemetry [candidate] (8.368 ms) : 0, 8368
Flare Poller [baseline] (3.768 ms) : 0, 3768
Flare Poller [candidate] (3.919 ms) : 0, 3919
IAST [baseline] (24.974 ms) : 0, 24974
IAST [candidate] (25.177 ms) : 0, 25177
section iast
crashtracking [baseline] (1.439 ms) : 0, 1439
crashtracking [candidate] (1.445 ms) : 0, 1445
BytebuddyAgent [baseline] (825.578 ms) : 0, 825578
BytebuddyAgent [candidate] (828.545 ms) : 0, 828545
GlobalTracer [baseline] (237.319 ms) : 0, 237319
GlobalTracer [candidate] (237.136 ms) : 0, 237136
AppSec [baseline] (34.208 ms) : 0, 34208
AppSec [candidate] (35.075 ms) : 0, 35075
Debugger [baseline] (61.744 ms) : 0, 61744
Debugger [candidate] (60.875 ms) : 0, 60875
Remote Config [baseline] (562.442 µs) : 0, 562
Remote Config [candidate] (539.28 µs) : 0, 539
Telemetry [baseline] (7.876 ms) : 0, 7876
Telemetry [candidate] (7.648 ms) : 0, 7648
Flare Poller [baseline] (3.534 ms) : 0, 3534
Flare Poller [candidate] (3.437 ms) : 0, 3437
IAST [baseline] (27.651 ms) : 0, 27651
IAST [candidate] (26.501 ms) : 0, 26501
section profiling
ProfilingAgent [baseline] (97.206 ms) : 0, 97206
ProfilingAgent [candidate] (96.93 ms) : 0, 96930
crashtracking [baseline] (1.43 ms) : 0, 1430
crashtracking [candidate] (1.423 ms) : 0, 1423
BytebuddyAgent [baseline] (730.302 ms) : 0, 730302
BytebuddyAgent [candidate] (728.236 ms) : 0, 728236
GlobalTracer [baseline] (221.467 ms) : 0, 221467
GlobalTracer [candidate] (221.333 ms) : 0, 221333
AppSec [baseline] (32.082 ms) : 0, 32082
AppSec [candidate] (32.143 ms) : 0, 32143
Debugger [baseline] (62.693 ms) : 0, 62693
Debugger [candidate] (62.828 ms) : 0, 62828
Remote Config [baseline] (642.084 µs) : 0, 642
Remote Config [candidate] (650.872 µs) : 0, 651
Telemetry [baseline] (7.892 ms) : 0, 7892
Telemetry [candidate] (8.006 ms) : 0, 8006
Flare Poller [baseline] (3.79 ms) : 0, 3790
Flare Poller [candidate] (3.786 ms) : 0, 3786
Profiling [baseline] (97.799 ms) : 0, 97799
Profiling [candidate] (97.513 ms) : 0, 97513
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.56.0-SNAPSHOT~0731038baa, baseline=1.56.0-SNAPSHOT~fb40abaeee

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.096 s) : 0, 1096131
Total [baseline] (8.844 s) : 0, 8844068
Agent [candidate] (1.105 s) : 0, 1104590
Total [candidate] (8.844 s) : 0, 8843663
section iast
Agent [baseline] (1.241 s) : 0, 1241284
Total [baseline] (9.528 s) : 0, 9527978
Agent [candidate] (1.237 s) : 0, 1237453
Total [candidate] (9.533 s) : 0, 9533018
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.096 s -
Agent iast 1.241 s 145.153 ms (13.2%)
Total tracing 8.844 s -
Total iast 9.528 s 683.91 ms (7.7%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.105 s -
Agent iast 1.237 s 132.862 ms (12.0%)
Total tracing 8.844 s -
Total iast 9.533 s 689.355 ms (7.8%) 
gantt
    title insecure-bank - break down per module: candidate=1.56.0-SNAPSHOT~0731038baa, baseline=1.56.0-SNAPSHOT~fb40abaeee

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.465 ms) : 0, 1465
crashtracking [candidate] (1.454 ms) : 0, 1454
BytebuddyAgent [baseline] (704.169 ms) : 0, 704169
BytebuddyAgent [candidate] (710.097 ms) : 0, 710097
GlobalTracer [baseline] (247.833 ms) : 0, 247833
GlobalTracer [candidate] (249.667 ms) : 0, 249667
AppSec [baseline] (32.219 ms) : 0, 32219
AppSec [candidate] (32.607 ms) : 0, 32607
Debugger [baseline] (63.197 ms) : 0, 63197
Debugger [candidate] (63.615 ms) : 0, 63615
Remote Config [baseline] (642.643 µs) : 0, 643
Remote Config [candidate] (629.284 µs) : 0, 629
Telemetry [baseline] (8.136 ms) : 0, 8136
Telemetry [candidate] (8.005 ms) : 0, 8005
Flare Poller [baseline] (3.669 ms) : 0, 3669
Flare Poller [candidate] (3.596 ms) : 0, 3596
section iast
crashtracking [baseline] (1.47 ms) : 0, 1470
crashtracking [candidate] (1.463 ms) : 0, 1463
BytebuddyAgent [baseline] (833.5 ms) : 0, 833500
BytebuddyAgent [candidate] (830.531 ms) : 0, 830531
GlobalTracer [baseline] (237.715 ms) : 0, 237715
GlobalTracer [candidate] (236.602 ms) : 0, 236602
AppSec [baseline] (34.157 ms) : 0, 34157
AppSec [candidate] (33.173 ms) : 0, 33173
Debugger [baseline] (60.207 ms) : 0, 60207
Debugger [candidate] (60.782 ms) : 0, 60782
Remote Config [baseline] (547.431 µs) : 0, 547
Remote Config [candidate] (576.261 µs) : 0, 576
Telemetry [baseline] (7.613 ms) : 0, 7613
Telemetry [candidate] (7.729 ms) : 0, 7729
Flare Poller [baseline] (3.451 ms) : 0, 3451
Flare Poller [candidate] (3.487 ms) : 0, 3487
IAST [baseline] (27.587 ms) : 0, 27587
IAST [candidate] (28.117 ms) : 0, 28117
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/mark-flaky
git_commit_date 1763479872 1763489865
git_commit_sha fb40aba 0731038
release_version 1.56.0-SNAPSHOT~fb40abaeee 1.56.0-SNAPSHOT~0731038baa
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1763492284 1763492284
ci_job_id 1242874620 1242874620
ci_pipeline_id 83002123 83002123
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-3-wwryacfo 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-3-wwryacfo 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 5 performance improvements and 1 performance regressions! Performance is the same for 14 metrics, 16 unstable metrics.

scenario Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p95 Δ mean throughput candidate mean agg_http_req_duration_p50 candidate mean agg_http_req_duration_p95 candidate mean throughput baseline mean agg_http_req_duration_p50 baseline mean agg_http_req_duration_p95 baseline mean throughput
scenario:load:insecure-bank:iast_GLOBAL:high_load better
[-283.438µs; -164.013µs] or [-9.581%; -5.544%]		 better
[-857.552µs; -442.123µs] or [-10.279%; -5.299%]		 unstable
[-38.029op/s; +236.029op/s] or [-3.101%; +19.245%]		 2.735ms 7.693ms 1325.438op/s 2.958ms 8.343ms 1226.438op/s
scenario:load:insecure-bank:tracing:high_load better
[-111.191µs; -31.849µs] or [-7.087%; -2.030%]		 unstable
[-787.600µs; -41.044µs] or [-17.969%; -0.936%]		 unstable
[-122.775op/s; +417.900op/s] or [-5.209%; +17.729%]		 1.497ms 3.969ms 2504.719op/s 1.569ms 4.383ms 2357.156op/s
scenario:load:insecure-bank:iast:high_load better
[-134.981µs; -55.573µs] or [-5.163%; -2.126%]		 same
[-365.657µs; +65.649µs] or [-4.830%; +0.867%]		 unstable
[-117.825op/s; +176.137op/s] or [-8.576%; +12.820%]		 2.519ms 7.421ms 1403.062op/s 2.614ms 7.571ms 1373.906op/s
scenario:load:petclinic:no_agent:high_load worse
[+1.258ms; +2.691ms] or [+7.527%; +16.104%]		 unstable
[+0.121ms; +3.181ms] or [+0.418%; +10.948%]		 unstable
[-50.480op/s; +2.292op/s] or [-18.774%; +0.853%]		 18.682ms 30.704ms 244.781op/s 16.708ms 29.053ms 268.875op/s
scenario:load:petclinic:profiling:high_load better
[-1.603ms; -0.612ms] or [-8.284%; -3.164%]		 unsure
[-1.987ms; -0.533ms] or [-6.449%; -1.729%]		 unstable
[-11.099op/s; +36.599op/s] or [-4.636%; +15.287%]		 18.248ms 29.557ms 252.156op/s 19.356ms 30.817ms 239.406op/s
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.56.0-SNAPSHOT~0731038baa, baseline=1.56.0-SNAPSHOT~fb40abaeee
    dateFormat X
    axisFormat %s
section baseline
no_agent (17.352 ms) : 17174, 17529
.   : milestone, 17352,
appsec (18.521 ms) : 18332, 18710
.   : milestone, 18521,
code_origins (17.688 ms) : 17514, 17861
.   : milestone, 17688,
iast (17.907 ms) : 17726, 18088
.   : milestone, 17907,
profiling (19.5 ms) : 19302, 19698
.   : milestone, 19500,
tracing (17.763 ms) : 17588, 17939
.   : milestone, 17763,
section candidate
no_agent (19.066 ms) : 18873, 19259
.   : milestone, 19066,
appsec (18.565 ms) : 18372, 18757
.   : milestone, 18565,
code_origins (18.139 ms) : 17957, 18321
.   : milestone, 18139,
iast (17.735 ms) : 17560, 17910
.   : milestone, 17735,
profiling (18.508 ms) : 18322, 18694
.   : milestone, 18508,
tracing (17.546 ms) : 17371, 17721
.   : milestone, 17546,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 17.352 ms [17.174 ms, 17.529 ms] -
appsec 18.521 ms [18.332 ms, 18.71 ms] 1.169 ms (6.7%)
code_origins 17.688 ms [17.514 ms, 17.861 ms] 336.37 µs (1.9%)
iast 17.907 ms [17.726 ms, 18.088 ms] 555.88 µs (3.2%)
profiling 19.5 ms [19.302 ms, 19.698 ms] 2.149 ms (12.4%)
tracing 17.763 ms [17.588 ms, 17.939 ms] 411.53 µs (2.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 19.066 ms [18.873 ms, 19.259 ms] -
appsec 18.565 ms [18.372 ms, 18.757 ms] -501.309 µs (-2.6%)
code_origins 18.139 ms [17.957 ms, 18.321 ms] -926.884 µs (-4.9%)
iast 17.735 ms [17.56 ms, 17.91 ms] -1.33 ms (-7.0%)
profiling 18.508 ms [18.322 ms, 18.694 ms] -557.76 µs (-2.9%)
tracing 17.546 ms [17.371 ms, 17.721 ms] -1.52 ms (-8.0%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.56.0-SNAPSHOT~0731038baa, baseline=1.56.0-SNAPSHOT~fb40abaeee
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.197 ms) : 1186, 1209
.   : milestone, 1197,
iast (3.334 ms) : 3291, 3376
.   : milestone, 3334,
iast_FULL (5.969 ms) : 5833, 6105
.   : milestone, 5969,
iast_GLOBAL (3.743 ms) : 3690, 3795
.   : milestone, 3743,
profiling (2.053 ms) : 2035, 2071
.   : milestone, 2053,
tracing (1.91 ms) : 1893, 1927
.   : milestone, 1910,
section candidate
no_agent (1.212 ms) : 1200, 1223
.   : milestone, 1212,
iast (3.262 ms) : 3218, 3306
.   : milestone, 3262,
iast_FULL (5.883 ms) : 5786, 5979
.   : milestone, 5883,
iast_GLOBAL (3.458 ms) : 3406, 3510
.   : milestone, 3458,
profiling (2.015 ms) : 1998, 2031
.   : milestone, 2015,
tracing (1.794 ms) : 1780, 1808
.   : milestone, 1794,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.197 ms [1.186 ms, 1.209 ms] -
iast 3.334 ms [3.291 ms, 3.376 ms] 2.136 ms (178.4%)
iast_FULL 5.969 ms [5.833 ms, 6.105 ms] 4.772 ms (398.5%)
iast_GLOBAL 3.743 ms [3.69 ms, 3.795 ms] 2.545 ms (212.6%)
profiling 2.053 ms [2.035 ms, 2.071 ms] 855.223 µs (71.4%)
tracing 1.91 ms [1.893 ms, 1.927 ms] 712.584 µs (59.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.212 ms [1.2 ms, 1.223 ms] -
iast 3.262 ms [3.218 ms, 3.306 ms] 2.051 ms (169.3%)
iast_FULL 5.883 ms [5.786 ms, 5.979 ms] 4.671 ms (385.6%)
iast_GLOBAL 3.458 ms [3.406 ms, 3.51 ms] 2.246 ms (185.4%)
profiling 2.015 ms [1.998 ms, 2.031 ms] 803.147 µs (66.3%)
tracing 1.794 ms [1.78 ms, 1.808 ms] 582.858 µs (48.1%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/mark-flaky
git_commit_date 1763479872 1763489865
git_commit_sha fb40aba 0731038
release_version 1.56.0-SNAPSHOT~fb40abaeee 1.56.0-SNAPSHOT~0731038baa
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1763491790 1763491790
ci_job_id 1242874621 1242874621
ci_pipeline_id 83002123 83002123
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-zap0chrh 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-zap0chrh 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.56.0-SNAPSHOT~0731038baa, baseline=1.56.0-SNAPSHOT~fb40abaeee
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.477 ms) : 1465, 1488
.   : milestone, 1477,
appsec (3.738 ms) : 3517, 3960
.   : milestone, 3738,
iast (2.213 ms) : 2148, 2277
.   : milestone, 2213,
iast_GLOBAL (2.252 ms) : 2187, 2317
.   : milestone, 2252,
profiling (2.086 ms) : 2032, 2139
.   : milestone, 2086,
tracing (2.037 ms) : 1987, 2088
.   : milestone, 2037,
section candidate
no_agent (1.479 ms) : 1467, 1490
.   : milestone, 1479,
appsec (3.735 ms) : 3515, 3955
.   : milestone, 3735,
iast (2.214 ms) : 2150, 2279
.   : milestone, 2214,
iast_GLOBAL (2.279 ms) : 2213, 2345
.   : milestone, 2279,
profiling (2.127 ms) : 2070, 2183
.   : milestone, 2127,
tracing (2.05 ms) : 1999, 2101
.   : milestone, 2050,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.477 ms [1.465 ms, 1.488 ms] -
appsec 3.738 ms [3.517 ms, 3.96 ms] 2.261 ms (153.1%)
iast 2.213 ms [2.148 ms, 2.277 ms] 735.783 µs (49.8%)
iast_GLOBAL 2.252 ms [2.187 ms, 2.317 ms] 775.082 µs (52.5%)
profiling 2.086 ms [2.032 ms, 2.139 ms] 608.749 µs (41.2%)
tracing 2.037 ms [1.987 ms, 2.088 ms] 560.291 µs (37.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.479 ms [1.467 ms, 1.49 ms] -
appsec 3.735 ms [3.515 ms, 3.955 ms] 2.256 ms (152.6%)
iast 2.214 ms [2.15 ms, 2.279 ms] 735.652 µs (49.7%)
iast_GLOBAL 2.279 ms [2.213 ms, 2.345 ms] 800.016 µs (54.1%)
profiling 2.127 ms [2.07 ms, 2.183 ms] 647.915 µs (43.8%)
tracing 2.05 ms [1.999 ms, 2.101 ms] 571.173 µs (38.6%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.56.0-SNAPSHOT~0731038baa, baseline=1.56.0-SNAPSHOT~fb40abaeee
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.96 s) : 14960000, 14960000
.   : milestone, 14960000,
appsec (15.203 s) : 15203000, 15203000
.   : milestone, 15203000,
iast (18.407 s) : 18407000, 18407000
.   : milestone, 18407000,
iast_GLOBAL (17.994 s) : 17994000, 17994000
.   : milestone, 17994000,
profiling (14.938 s) : 14938000, 14938000
.   : milestone, 14938000,
tracing (14.87 s) : 14870000, 14870000
.   : milestone, 14870000,
section candidate
no_agent (15.064 s) : 15064000, 15064000
.   : milestone, 15064000,
appsec (14.634 s) : 14634000, 14634000
.   : milestone, 14634000,
iast (18.243 s) : 18243000, 18243000
.   : milestone, 18243000,
iast_GLOBAL (17.973 s) : 17973000, 17973000
.   : milestone, 17973000,
profiling (15.022 s) : 15022000, 15022000
.   : milestone, 15022000,
tracing (14.847 s) : 14847000, 14847000
.   : milestone, 14847000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.96 s [14.96 s, 14.96 s] -
appsec 15.203 s [15.203 s, 15.203 s] 243.0 ms (1.6%)
iast 18.407 s [18.407 s, 18.407 s] 3.447 s (23.0%)
iast_GLOBAL 17.994 s [17.994 s, 17.994 s] 3.034 s (20.3%)
profiling 14.938 s [14.938 s, 14.938 s] -22.0 ms (-0.1%)
tracing 14.87 s [14.87 s, 14.87 s] -90.0 ms (-0.6%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.064 s [15.064 s, 15.064 s] -
appsec 14.634 s [14.634 s, 14.634 s] -430.0 ms (-2.9%)
iast 18.243 s [18.243 s, 18.243 s] 3.179 s (21.1%)
iast_GLOBAL 17.973 s [17.973 s, 17.973 s] 2.909 s (19.3%)
profiling 15.022 s [15.022 s, 15.022 s] -42.0 ms (-0.3%)
tracing 14.847 s [14.847 s, 14.847 s] -217.0 ms (-1.4%)

@jpbempel
Copy link
Member Author

jpbempel commented Nov 18, 2025

@AlexeyKuznetsov-DD please do not update the branch for others

@AlexeyKuznetsov-DD
Copy link
Contributor

AlexeyKuznetsov-DD commented Nov 18, 2025

@AlexeyKuznetsov-DD please do not update the branch for others

Sorry about that, will avoid doing that in future.

@jpbempel jpbempel merged commit 0f63e5e into master Nov 18, 2025
538 checks passed
@jpbempel jpbempel deleted the jpbempel/mark-flaky branch November 18, 2025 19:30
@github-actions github-actions bot added this to the 1.56.0 milestone Nov 18, 2025
amarziali pushed a commit that referenced this pull request Nov 21, 2025
@jpbempel @amarziali
mark flaky Exception Replay integration test (#9970)
395e2b1 
mark flaky Exception Replay integration tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD AlexeyKuznetsov-DD approved these changes

@shatzi shatzi Awaiting requested review from shatzi shatzi is a code owner automatically assigned from DataDog/debugger-java

Assignees

No one assigned

Labels

comp: debugger Dynamic Instrumentation tag: no release notes Changes to exclude from release notes type: bug Bug report and fix

Projects

None yet

Milestone

1.56.0

Development

Successfully merging this pull request may close these issues.

3 participants

@jpbempel @AlexeyKuznetsov-DD