test/access_loggers/stats: replace tcmalloc consumedBytes assertions with sizeof structural bounds#44708
Closed
test/access_loggers/stats: replace tcmalloc consumedBytes assertions with sizeof structural bounds#44708
Conversation
…ds in stats_test.cc Agent-Logs-Url: https://github.com/envoyproxy/envoy/sessions/636ffcaf-9841-4c03-8ab2-af7842e7e936 Co-authored-by: phlax <454682+phlax@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Fix memory footprint tests in access logger
test/access_loggers/stats: replace tcmalloc consumedBytes assertions with sizeof structural bounds
Apr 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
GaugeKeyTest.ExactMemoryFootprintandStatsAccessLoggerTest.AccessLogStateMemoryFootprintusedMemory::TestUtil::MemoryTest::consumedBytes()with hand-tuned tcmalloc fudge factors ("384 gives a generous 1.5x buffer for allocator page alignment","We allow 4096 bytes for allocator caches"). This is the same anti-pattern removed forStringMatcherin #44701 — it produces CI flakes and needs re-tuning on every tcmalloc/abseil/protobuf bump.Changes
GaugeKeyTest::ExactMemoryFootprint→GaugeKeyTest::SizeIsBounded: drop all threeMemoryTest/consumedBytes()blocks; replace with a single portable structural bound:StatsAccessLoggerTest::AccessLogStateMemoryFootprint→StatsAccessLoggerTest::AccessLogStateSizeIsBounded: remove the 10 K-item insertion/removal loop and allconsumedBytes()assertions (including the two that bypassedEXPECT_MEMORY_LEwith rawEXPECT_LE); replace with:Comments document the
InflightGaugelayout (private struct) and explain that leak-after-destruction is already covered by ASAN/MSAN/LSAN in CI.Add
MATCHER_P(MemNotMoreThan, sz, ...)(identical to test: replace flaky StringMatcher.Memory with deterministic sizeof bounds #44701): failure message prompts the reviewer to think carefully before raising the bound.Remove
#include "test/common/memory/memory_test_utility.h"— no remaining callers.All bounds are expressed in
sizeof(void*)units for portability across libc++/libstdc++ and 32/64-bit builds.Warning
Firewall rules blocked me from connecting to one or more addresses (expand for details)
I tried to connect to the following addresses, but was blocked by firewall rules:
dl.google.com/build/bazel_root/install/fb2a7f6d344d2f4e335882534df59296/embedded_tools/jdk/bin/java bazel(envoy) --add-opens=java.base/java.lang=ALL-UNNAMED -Xverify:none -Djava.util.logging.config.file=/build/bazel_root/base/javalog.properties -Dcom.google.devtools.build.lib.util.LogHandlerQuerier.class=com.google.devtools.build.lib.util.SimpleLogHandler$HandlerQuerier -XX:-MaxFDLimit -Djava.library.path=/build/bazel_root/install/fb2a7f6d344d2f4e335882534df59296/embedded_tools/jdk/lib:/build/bazel_root/install/fb2a7f6d344d2f4e335882534df59296/embedded_tools/jdk/lib/server:/build/bazel_root/install/fb2a7f6d344d2f4e335882534df59296/ -Dfile.encoding=ISO-8859-1 -Duser.country= -Duser.language= -Duser.variant= -Xmx3g -DBAZEL_TRACK_SOURCE_DIRECTORIES=1 -Djavax.net.ssl.trustStore=/tmp/custom-cacerts -Djavax.net.ssl.trustStorePassword=changeit -jar /build/bazel_root/install/fb2a7f6d344d2f4e335882534df59296/A-server.jar --max_idle_secs=10800 --noshutdown_on_low_sys_mem --connect_timeout_secs=30(dns block)If you need me to access, download, or install something from one of these locations, you can either:
Original prompt
Background
PR #44701 replaced
TEST_F(StringMatcher, Memory)withStringMatcher.SizeIsBounded, removing tcmalloc page-levelconsumedBytes()assertions with hand-tuned ceilings (which produced ~5/1000 CI flakes and had to be re-tuned on every tcmalloc/abseil/protobuf bump — see #37782, #43467, #44528).The same anti-pattern exists in
test/extensions/access_loggers/stats/stats_test.cc, in two places:EXPECT_MEMORY_LE(memory_test.consumedBytes(), 64);block (just beforeAccessLogStateMemoryFootprint).TEST_F(StatsAccessLoggerTest, AccessLogStateMemoryFootprint)— which mixes a deterministicEXPECT_LE(sizeof(AccessLogState), 128)static check with multipleconsumedBytes()-based assertions whose comments explicitly call out allocator fudge factors:"384 gives a generous 1.5x buffer for allocator page alignment.""We allow 4096 bytes for allocator caches.""approximately 1.6 Megabytes for 10,000 items ... threshold of 2 Megabytes here to account for this capacity and allocator page alignment."It also bypasses
EXPECT_MEMORY_LEin two spots with rawEXPECT_LE(static_cast<int64_t>(memory_test.consumedBytes()), …)andEXPECT_LE(memory_test.consumedBytes(), 4096), which skips the canonical-platform guard thatEXPECT_MEMORY_LEprovides — so on non-canonical platforms those lines are effectively asserting tcmalloc page math directly, which is exactly the flake source we want to remove.The test is young (added Feb 2026 in #42226) so it hasn't been re-tuned yet, but it will start needing the same toolchain-bump tuning that #44701 retired.
Task
Apply the same approach used in #44701 to
test/extensions/access_loggers/stats/stats_test.cc:Memory::TestUtil::MemoryTest/consumedBytes()based assertions with deterministicsizeof-based structural bounds, expressed insizeof(std::string)/sizeof(void*)/sizeof(some_type)for libc++ / libstdc++ / 32-bit / 64-bit portability.MATCHER_P(MemNotMoreThan, sz, ...)(or equivalent) similar to the one introduced in PR test: replace flaky StringMatcher.Memory with deterministic sizeof bounds #44701 so the failure message tells the maintainer to "think carefully before increasing this, and if you're sure, update the corresponding expectation."AccessLogStateand theGaugeKey/InflightGaugeslot types stay small). The existing comments documenting why the bounds exist (slot composition, what the per-item overhead is, etc.) should be kept and re-cast in terms ofsizeofof the relevant types instead of measured tcmalloc bytes.consumedBytes(), drop the tcmalloc-based check entirely if it cannot be re-expressed deterministically. The destructor invariants the test was trying to assert are better covered by ASAN/MSAN/LSAN, which already run on this code in CI; preserving a flaky leak check here adds no real safety net. Add a brief comment explaining this.AccessLogStateMemoryFootprinttoAccessLogStateSizeIsBoundedor similar, mirroring the rename in test: replace flaky StringMatcher.Memory with deterministic sizeof bounds #44701).#include "test/common/memory/memory_test_utility.h"if no other test in the file still needs it after the change.gmock/gmock.his not already included, add it (needed forEXPECT_THAT/MATCHER_P).Reference
The PR to mirror: #44701 — see the diff to
test/common/common/matchers_test.ccfor the exact style, theMemNotMoreThanmatcher, and the bound expressions to follow.Out of scope
test/integration/stats_integration_test.ccClusterMemoryTest,test/common/stats/thread_local_store_test.ccMemoryWith*RealSymbolTable,test/common/stats/symbol_table_impl_test.ccSymbolTableTest.Memory). Those are intentional golden-value trackers and are correctly usingEXPECT_MEMORY_EQ/EXPECT_MEMORY_LEagainst pinned values.test/common/common/packed_struct_test.ccin this PR.Validation
bazel test //test/extensions/access_loggers/stats:stats_testshould pass.Memory::TestUtil::MemoryTest/consumedBytes()/EXPECT_MEMORY_LE/EXPECT_MEMORY_EQin the modified file.The following is the prior conversation context from the user's chat exploration (may be truncated):
User: we just landed this one - which i think is a great improvement
i think there is another similar (perhaps ~adjacent) test that does the same anti-pattern and should be fixed
Assistant: Looking at the PR you just landed, the anti-pattern was: **using
Memory::TestUtil::MemoryTest::consumedBytes()(tcmalloc page-level accounting) with hand-tuned ceilings + alignment fu...This pull request was created from Copilot chat.