Antithesis poc #9993
Draft
Antithesis poc #9993
+274
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 4 performance regressions! Performance is the same for 55 metrics, 6 unstable metrics.
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.56.0-SNAPSHOT~e750f61cfc, baseline=1.56.0-SNAPSHOT~8d538789b9
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.114 s) : 0, 1113820
Total [baseline] (8.895 s) : 0, 8894964
Agent [candidate] (1.098 s) : 0, 1098100
Total [candidate] (8.752 s) : 0, 8751745
section iast
Agent [baseline] (1.234 s) : 0, 1234061
Total [baseline] (9.538 s) : 0, 9537955
Agent [candidate] (1.234 s) : 0, 1234401
Total [candidate] (9.436 s) : 0, 9436123
gantt
title insecure-bank - break down per module: candidate=1.56.0-SNAPSHOT~e750f61cfc, baseline=1.56.0-SNAPSHOT~8d538789b9
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.478 ms) : 0, 1478
crashtracking [candidate] (1.449 ms) : 0, 1449
BytebuddyAgent [baseline] (714.826 ms) : 0, 714826
BytebuddyAgent [candidate] (704.425 ms) : 0, 704425
GlobalTracer [baseline] (252.105 ms) : 0, 252105
GlobalTracer [candidate] (248.982 ms) : 0, 248982
AppSec [baseline] (32.919 ms) : 0, 32919
AppSec [candidate] (32.352 ms) : 0, 32352
Debugger [baseline] (64.375 ms) : 0, 64375
Debugger [candidate] (63.25 ms) : 0, 63250
Remote Config [baseline] (682.932 µs) : 0, 683
Remote Config [candidate] (628.386 µs) : 0, 628
Telemetry [baseline] (8.39 ms) : 0, 8390
Telemetry [candidate] (8.398 ms) : 0, 8398
Flare Poller [baseline] (3.82 ms) : 0, 3820
Flare Poller [candidate] (3.653 ms) : 0, 3653
section iast
crashtracking [baseline] (1.445 ms) : 0, 1445
crashtracking [candidate] (1.447 ms) : 0, 1447
BytebuddyAgent [baseline] (827.304 ms) : 0, 827304
BytebuddyAgent [candidate] (826.767 ms) : 0, 826767
GlobalTracer [baseline] (237.589 ms) : 0, 237589
GlobalTracer [candidate] (238.212 ms) : 0, 238212
IAST [baseline] (29.297 ms) : 0, 29297
IAST [candidate] (26.593 ms) : 0, 26593
AppSec [baseline] (32.264 ms) : 0, 32264
AppSec [candidate] (34.602 ms) : 0, 34602
Debugger [baseline] (59.803 ms) : 0, 59803
Debugger [candidate] (60.032 ms) : 0, 60032
Remote Config [baseline] (545.231 µs) : 0, 545
Remote Config [candidate] (526.21 µs) : 0, 526
Telemetry [baseline] (7.584 ms) : 0, 7584
Telemetry [candidate] (7.775 ms) : 0, 7775
Flare Poller [baseline] (3.445 ms) : 0, 3445
Flare Poller [candidate] (3.493 ms) : 0, 3493
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.56.0-SNAPSHOT~e750f61cfc, baseline=1.56.0-SNAPSHOT~8d538789b9
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.112 s) : 0, 1111965
Total [baseline] (10.876 s) : 0, 10875886
Agent [candidate] (1.099 s) : 0, 1099107
Total [candidate] (10.728 s) : 0, 10727672
section appsec
Agent [baseline] (1.28 s) : 0, 1279750
Total [baseline] (11.029 s) : 0, 11028787
Agent [candidate] (1.286 s) : 0, 1286097
Total [candidate] (10.925 s) : 0, 10925065
section iast
Agent [baseline] (1.238 s) : 0, 1237872
Total [baseline] (11.184 s) : 0, 11183840
Agent [candidate] (1.236 s) : 0, 1235731
Total [candidate] (11.266 s) : 0, 11265881
section profiling
Agent [baseline] (1.23 s) : 0, 1229602
Total [baseline] (11.034 s) : 0, 11033773
Agent [candidate] (1.236 s) : 0, 1235764
Total [candidate] (11.195 s) : 0, 11195310
gantt
title petclinic - break down per module: candidate=1.56.0-SNAPSHOT~e750f61cfc, baseline=1.56.0-SNAPSHOT~8d538789b9
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.465 ms) : 0, 1465
crashtracking [candidate] (1.434 ms) : 0, 1434
BytebuddyAgent [baseline] (713.465 ms) : 0, 713465
BytebuddyAgent [candidate] (705.709 ms) : 0, 705709
GlobalTracer [baseline] (251.199 ms) : 0, 251199
GlobalTracer [candidate] (248.602 ms) : 0, 248602
AppSec [baseline] (32.872 ms) : 0, 32872
AppSec [candidate] (32.155 ms) : 0, 32155
Debugger [baseline] (65.175 ms) : 0, 65175
Debugger [candidate] (63.746 ms) : 0, 63746
Remote Config [baseline] (642.324 µs) : 0, 642
Remote Config [candidate] (629.347 µs) : 0, 629
Telemetry [baseline] (8.309 ms) : 0, 8309
Telemetry [candidate] (8.284 ms) : 0, 8284
Flare Poller [baseline] (3.735 ms) : 0, 3735
Flare Poller [candidate] (3.601 ms) : 0, 3601
section appsec
crashtracking [baseline] (1.458 ms) : 0, 1458
crashtracking [candidate] (1.462 ms) : 0, 1462
BytebuddyAgent [baseline] (730.133 ms) : 0, 730133
BytebuddyAgent [candidate] (733.623 ms) : 0, 733623
GlobalTracer [baseline] (240.484 ms) : 0, 240484
GlobalTracer [candidate] (242.167 ms) : 0, 242167
IAST [baseline] (24.885 ms) : 0, 24885
IAST [candidate] (25.099 ms) : 0, 25099
AppSec [baseline] (174.059 ms) : 0, 174059
AppSec [candidate] (174.553 ms) : 0, 174553
Debugger [baseline] (60.869 ms) : 0, 60869
Debugger [candidate] (60.822 ms) : 0, 60822
Remote Config [baseline] (736.129 µs) : 0, 736
Remote Config [candidate] (676.997 µs) : 0, 677
Telemetry [baseline] (8.253 ms) : 0, 8253
Telemetry [candidate] (8.58 ms) : 0, 8580
Flare Poller [baseline] (3.807 ms) : 0, 3807
Flare Poller [candidate] (3.933 ms) : 0, 3933
section iast
crashtracking [baseline] (1.449 ms) : 0, 1449
crashtracking [candidate] (1.453 ms) : 0, 1453
BytebuddyAgent [baseline] (829.177 ms) : 0, 829177
BytebuddyAgent [candidate] (827.367 ms) : 0, 827367
GlobalTracer [baseline] (238.022 ms) : 0, 238022
GlobalTracer [candidate] (238.291 ms) : 0, 238291
IAST [baseline] (30.089 ms) : 0, 30089
IAST [candidate] (30.86 ms) : 0, 30860
AppSec [baseline] (31.433 ms) : 0, 31433
AppSec [candidate] (30.409 ms) : 0, 30409
Debugger [baseline] (61.047 ms) : 0, 61047
Debugger [candidate] (60.538 ms) : 0, 60538
Remote Config [baseline] (559.008 µs) : 0, 559
Remote Config [candidate] (532.549 µs) : 0, 533
Telemetry [baseline] (7.623 ms) : 0, 7623
Telemetry [candidate] (7.81 ms) : 0, 7810
Flare Poller [baseline] (3.497 ms) : 0, 3497
Flare Poller [candidate] (3.541 ms) : 0, 3541
section profiling
crashtracking [baseline] (1.443 ms) : 0, 1443
crashtracking [candidate] (1.451 ms) : 0, 1451
BytebuddyAgent [baseline] (731.626 ms) : 0, 731626
BytebuddyAgent [candidate] (733.208 ms) : 0, 733208
GlobalTracer [baseline] (222.101 ms) : 0, 222101
GlobalTracer [candidate] (222.345 ms) : 0, 222345
AppSec [baseline] (32.162 ms) : 0, 32162
AppSec [candidate] (32.476 ms) : 0, 32476
Debugger [baseline] (62.955 ms) : 0, 62955
Debugger [candidate] (63.057 ms) : 0, 63057
Remote Config [baseline] (628.872 µs) : 0, 629
Remote Config [candidate] (650.518 µs) : 0, 651
Telemetry [baseline] (7.904 ms) : 0, 7904
Telemetry [candidate] (8.308 ms) : 0, 8308
Flare Poller [baseline] (3.684 ms) : 0, 3684
Flare Poller [candidate] (3.866 ms) : 0, 3866
ProfilingAgent [baseline] (97.375 ms) : 0, 97375
ProfilingAgent [candidate] (100.506 ms) : 0, 100506
Profiling [baseline] (97.965 ms) : 0, 97965
Profiling [candidate] (101.126 ms) : 0, 101126
LoadParameters
See matching parameters
SummaryFound 4 performance improvements and 0 performance regressions! Performance is the same for 13 metrics, 19 unstable metrics.
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.56.0-SNAPSHOT~e750f61cfc, baseline=1.56.0-SNAPSHOT~8d538789b9
dateFormat X
axisFormat %s
section baseline
no_agent (18.314 ms) : 18126, 18502
. : milestone, 18314,
appsec (18.513 ms) : 18324, 18701
. : milestone, 18513,
code_origins (17.528 ms) : 17354, 17702
. : milestone, 17528,
iast (18.868 ms) : 18672, 19064
. : milestone, 18868,
profiling (19.571 ms) : 19371, 19772
. : milestone, 19571,
tracing (17.587 ms) : 17411, 17763
. : milestone, 17587,
section candidate
no_agent (18.092 ms) : 17905, 18279
. : milestone, 18092,
appsec (18.594 ms) : 18406, 18783
. : milestone, 18594,
code_origins (17.584 ms) : 17409, 17759
. : milestone, 17584,
iast (17.972 ms) : 17792, 18153
. : milestone, 17972,
profiling (18.755 ms) : 18567, 18943
. : milestone, 18755,
tracing (17.911 ms) : 17732, 18091
. : milestone, 17911,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.56.0-SNAPSHOT~e750f61cfc, baseline=1.56.0-SNAPSHOT~8d538789b9
dateFormat X
axisFormat %s
section baseline
no_agent (1.218 ms) : 1206, 1230
. : milestone, 1218,
iast (3.201 ms) : 3156, 3246
. : milestone, 3201,
iast_FULL (5.553 ms) : 5499, 5606
. : milestone, 5553,
iast_GLOBAL (3.712 ms) : 3645, 3778
. : milestone, 3712,
profiling (2.176 ms) : 2155, 2196
. : milestone, 2176,
tracing (1.837 ms) : 1822, 1853
. : milestone, 1837,
section candidate
no_agent (1.217 ms) : 1204, 1230
. : milestone, 1217,
iast (3.191 ms) : 3152, 3230
. : milestone, 3191,
iast_FULL (5.67 ms) : 5614, 5727
. : milestone, 5670,
iast_GLOBAL (3.693 ms) : 3634, 3752
. : milestone, 3693,
profiling (1.958 ms) : 1940, 1975
. : milestone, 1958,
tracing (1.81 ms) : 1795, 1825
. : milestone, 1810,
DacapoParameters
See matching parameters
SummaryFound 1 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 1 unstable metrics.
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.56.0-SNAPSHOT~e750f61cfc, baseline=1.56.0-SNAPSHOT~8d538789b9
dateFormat X
axisFormat %s
section baseline
no_agent (14.822 s) : 14822000, 14822000
. : milestone, 14822000,
appsec (15.235 s) : 15235000, 15235000
. : milestone, 15235000,
iast (18.036 s) : 18036000, 18036000
. : milestone, 18036000,
iast_GLOBAL (17.834 s) : 17834000, 17834000
. : milestone, 17834000,
profiling (14.935 s) : 14935000, 14935000
. : milestone, 14935000,
tracing (14.804 s) : 14804000, 14804000
. : milestone, 14804000,
section candidate
no_agent (15.689 s) : 15689000, 15689000
. : milestone, 15689000,
appsec (15.366 s) : 15366000, 15366000
. : milestone, 15366000,
iast (18.25 s) : 18250000, 18250000
. : milestone, 18250000,
iast_GLOBAL (18.201 s) : 18201000, 18201000
. : milestone, 18201000,
profiling (14.703 s) : 14703000, 14703000
. : milestone, 14703000,
tracing (14.53 s) : 14530000, 14530000
. : milestone, 14530000,
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.56.0-SNAPSHOT~e750f61cfc, baseline=1.56.0-SNAPSHOT~8d538789b9
dateFormat X
axisFormat %s
section baseline
no_agent (1.481 ms) : 1469, 1492
. : milestone, 1481,
appsec (3.706 ms) : 3487, 3925
. : milestone, 3706,
iast (2.22 ms) : 2157, 2283
. : milestone, 2220,
iast_GLOBAL (2.255 ms) : 2191, 2319
. : milestone, 2255,
profiling (2.486 ms) : 2322, 2649
. : milestone, 2486,
tracing (2.047 ms) : 1997, 2097
. : milestone, 2047,
section candidate
no_agent (1.485 ms) : 1474, 1497
. : milestone, 1485,
appsec (2.465 ms) : 2414, 2516
. : milestone, 2465,
iast (2.224 ms) : 2160, 2287
. : milestone, 2224,
iast_GLOBAL (2.265 ms) : 2200, 2329
. : milestone, 2265,
profiling (2.094 ms) : 2041, 2146
. : milestone, 2094,
tracing (2.042 ms) : 1992, 2092
. : milestone, 2042,
|
6 tasks
|
🎯 Code Coverage 🔗 Commit SHA: e750f61 | Docs | Datadog PR Page | Was this helpful? Give us feedback! |
bric3
reviewed
Nov 25, 2025
Comment on lines
+152
to
+158
| ObjectNode httpResponseDetails = JsonNodeFactory.instance.objectNode(); | ||
| httpResponseDetails.put("trace_count", payload.traceCount()); | ||
| httpResponseDetails.put("payload_size_bytes", sizeInBytes); | ||
| httpResponseDetails.put("http_status", response.code()); | ||
| httpResponseDetails.put("http_message", response.message()); | ||
| httpResponseDetails.put("success", response.code() == 200); | ||
| httpResponseDetails.put("agent_url", tracesUrl.toString()); |
Contributor
There was a problem hiding this comment.
thought: I'm very uncomfortable with using jackson API in these parts. And the Assert methods rely on it.
Comment on lines
+175
to
+178
| log.debug("ANTITHESIS_ASSERT: Non-200 HTTP response (unreachable) - code: {}, message: {}, traces: {}", response.code(), response.message(), payload.traceCount()); | ||
| Assert.unreachable( | ||
| "Non-200 HTTP response from agent indicates API failure - traces may be lost", | ||
| errorDetails); |
Contributor
There was a problem hiding this comment.
thought: Another thing is the human cost to maintain these assertions.
bric3
reviewed
Nov 25, 2025
| implementation(project(":internal-api")) | ||
|
|
||
| // Antithesis SDK for assertions and property testing - bundled in tracer JAR | ||
| implementation(group = "com.antithesis", name = "sdk", version = "1.4.5") |
Contributor
There was a problem hiding this comment.
thought: Another concern, is how much more weight it adds to the jar.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What Does This Do
Add Antithesis assertions in the tracer
Motivation
Additional Notes
Contributor Checklist
type:and (comp:orinst:) labels in addition to any useful labelsclose,fixor any linking keywords when referencing an issue.Use
solvesinstead, and assign the PR milestone to the issueJira ticket: [PROJ-IDENT]