Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix a stack size warning in ros3 VFD code #4625

Merged
merged 1 commit into from
Jul 3, 2024

Conversation

derobins
Copy link
Member

@derobins derobins commented Jul 3, 2024

Just makes a stack array dynamic. Valgrind shows no memory leaks.

Just makes a stack array dynamic. Valgrind shows no memory leaks.
@derobins derobins added Merge - To 1.14 Priority - 2. Medium ⏹ It would be nice to have this in the next release Component - C Library Core C library issues (usually in the src directory) Type - Improvement Improvements that don't add a new feature or functionality labels Jul 3, 2024
@derobins
Copy link
Member Author

derobins commented Jul 3, 2024

FWIW, Valgrind does show unfreed memory, but this is a known curl issue and not really considered a memory leak. This predates the changes here and is not due to mismatched curl init/cleanup calls.

$ libtool --mode=execute valgrind --leak-check=full --track-origins=yes --show-leak-kinds=all --num-callers=50 ./ros3
==257163== Memcheck, a memory error detector
==257163== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==257163== Using Valgrind-3.21.0 and LibVEX; rerun with -h for copyright info
==257163== Command: /home/derobins/hdf5_devel/build/test/.libs/ros3
==257163== 
Testing ros3 VFD functionality.
Testing ros3 fapl configuration validation                             PASSED
Testing ros3 driver flags                                              PASSED
Testing ros3 VFD-level open                                            PASSED
Testing ROS3 eof/eoa gets and sets                                     PASSED
Testing ros3 VFD read/range-gets                                       PASSED
Testing ros3 VFD read-eoa temporal coupling library limitation         PASSED
Testing ros3 VFD always-fail and no-op routines                        PASSED
Testing ros3 cmp (comparison)                                          PASSED
Testing ros3 access modes                                              PASSED
All ros3 tests passed.
==257163== 
==257163== HEAP SUMMARY:
==257163==     in use at exit: 1,506 bytes in 4 blocks
==257163==   total heap usage: 1,416,931 allocs, 1,416,927 frees, 133,954,184 bytes allocated
==257163== 
==257163== 48 bytes in 1 blocks are still reachable in loss record 1 of 4
==257163==    at 0x4845828: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==257163==    by 0x4027D7E: malloc (rtld-malloc.h:56)
==257163==    by 0x4027D7E: strdup (strdup.c:42)
==257163==    by 0x4016805: _dl_load_cache_lookup (dl-cache.c:515)
==257163==    by 0x40091DB: _dl_map_object (dl-load.c:2132)
==257163==    by 0x400D058: dl_open_worker_begin (dl-open.c:534)
==257163==    by 0x40014DC: _dl_catch_exception (dl-catch.c:237)
==257163==    by 0x400C569: dl_open_worker (dl-open.c:784)
==257163==    by 0x40014DC: _dl_catch_exception (dl-catch.c:237)
==257163==    by 0x400C91B: _dl_open (dl-open.c:886)
==257163==    by 0x5014DA0: do_dlopen (dl-libc.c:95)
==257163==    by 0x40014DC: _dl_catch_exception (dl-catch.c:237)
==257163==    by 0x4001602: _dl_catch_error (dl-catch.c:256)
==257163==    by 0x5015199: dlerror_run (dl-libc.c:45)
==257163==    by 0x5015199: __libc_dlopen_mode (dl-libc.c:162)
==257163==    by 0x4FF51AE: module_load (nss_module.c:187)
==257163==    by 0x4FF567C: __nss_module_load (nss_module.c:302)
==257163==    by 0x4FF567C: __nss_module_get_function (nss_module.c:328)
==257163==    by 0x4F8FCC0: get_nss_addresses (getaddrinfo.c:640)
==257163==    by 0x4F8FCC0: gaih_inet (getaddrinfo.c:1179)
==257163==    by 0x4F8FCC0: getaddrinfo (getaddrinfo.c:2385)
==257163==    by 0x4E05DC6: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.8.0)
==257163==    by 0x4E09F9B: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.8.0)
==257163==    by 0x4F33B59: start_thread (pthread_create.c:444)
==257163==    by 0x4FC4463: clone (clone.S:100)
==257163== 
==257163== 48 bytes in 1 blocks are still reachable in loss record 2 of 4
==257163==    at 0x4845828: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==257163==    by 0x400C2C8: malloc (rtld-malloc.h:56)
==257163==    by 0x400C2C8: _dl_new_object (dl-object.c:199)
==257163==    by 0x400752E: _dl_map_object_from_fd (dl-load.c:1053)
==257163==    by 0x4008F70: _dl_map_object (dl-load.c:2265)
==257163==    by 0x400D058: dl_open_worker_begin (dl-open.c:534)
==257163==    by 0x40014DC: _dl_catch_exception (dl-catch.c:237)
==257163==    by 0x400C569: dl_open_worker (dl-open.c:784)
==257163==    by 0x40014DC: _dl_catch_exception (dl-catch.c:237)
==257163==    by 0x400C91B: _dl_open (dl-open.c:886)
==257163==    by 0x5014DA0: do_dlopen (dl-libc.c:95)
==257163==    by 0x40014DC: _dl_catch_exception (dl-catch.c:237)
==257163==    by 0x4001602: _dl_catch_error (dl-catch.c:256)
==257163==    by 0x5015199: dlerror_run (dl-libc.c:45)
==257163==    by 0x5015199: __libc_dlopen_mode (dl-libc.c:162)
==257163==    by 0x4FF51AE: module_load (nss_module.c:187)
==257163==    by 0x4FF567C: __nss_module_load (nss_module.c:302)
==257163==    by 0x4FF567C: __nss_module_get_function (nss_module.c:328)
==257163==    by 0x4F8FCC0: get_nss_addresses (getaddrinfo.c:640)
==257163==    by 0x4F8FCC0: gaih_inet (getaddrinfo.c:1179)
==257163==    by 0x4F8FCC0: getaddrinfo (getaddrinfo.c:2385)
==257163==    by 0x4E05DC6: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.8.0)
==257163==    by 0x4E09F9B: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.8.0)
==257163==    by 0x4F33B59: start_thread (pthread_create.c:444)
==257163==    by 0x4FC4463: clone (clone.S:100)
==257163== 
==257163== 168 bytes in 1 blocks are still reachable in loss record 3 of 4
==257163==    at 0x484A993: calloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==257163==    by 0x4015F8B: calloc (rtld-malloc.h:44)
==257163==    by 0x4015F8B: _dl_check_map_versions (dl-version.c:280)
==257163==    by 0x400D2BC: dl_open_worker_begin (dl-open.c:602)
==257163==    by 0x40014DC: _dl_catch_exception (dl-catch.c:237)
==257163==    by 0x400C569: dl_open_worker (dl-open.c:784)
==257163==    by 0x40014DC: _dl_catch_exception (dl-catch.c:237)
==257163==    by 0x400C91B: _dl_open (dl-open.c:886)
==257163==    by 0x5014DA0: do_dlopen (dl-libc.c:95)
==257163==    by 0x40014DC: _dl_catch_exception (dl-catch.c:237)
==257163==    by 0x4001602: _dl_catch_error (dl-catch.c:256)
==257163==    by 0x5015199: dlerror_run (dl-libc.c:45)
==257163==    by 0x5015199: __libc_dlopen_mode (dl-libc.c:162)
==257163==    by 0x4FF51AE: module_load (nss_module.c:187)
==257163==    by 0x4FF567C: __nss_module_load (nss_module.c:302)
==257163==    by 0x4FF567C: __nss_module_get_function (nss_module.c:328)
==257163==    by 0x4F8FCC0: get_nss_addresses (getaddrinfo.c:640)
==257163==    by 0x4F8FCC0: gaih_inet (getaddrinfo.c:1179)
==257163==    by 0x4F8FCC0: getaddrinfo (getaddrinfo.c:2385)
==257163==    by 0x4E05DC6: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.8.0)
==257163==    by 0x4E09F9B: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.8.0)
==257163==    by 0x4F33B59: start_thread (pthread_create.c:444)
==257163==    by 0x4FC4463: clone (clone.S:100)
==257163== 
==257163== 1,242 bytes in 1 blocks are still reachable in loss record 4 of 4
==257163==    at 0x484A993: calloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==257163==    by 0x400BF9C: calloc (rtld-malloc.h:44)
==257163==    by 0x400BF9C: _dl_new_object (dl-object.c:92)
==257163==    by 0x400752E: _dl_map_object_from_fd (dl-load.c:1053)
==257163==    by 0x4008F70: _dl_map_object (dl-load.c:2265)
==257163==    by 0x400D058: dl_open_worker_begin (dl-open.c:534)
==257163==    by 0x40014DC: _dl_catch_exception (dl-catch.c:237)
==257163==    by 0x400C569: dl_open_worker (dl-open.c:784)
==257163==    by 0x40014DC: _dl_catch_exception (dl-catch.c:237)
==257163==    by 0x400C91B: _dl_open (dl-open.c:886)
==257163==    by 0x5014DA0: do_dlopen (dl-libc.c:95)
==257163==    by 0x40014DC: _dl_catch_exception (dl-catch.c:237)
==257163==    by 0x4001602: _dl_catch_error (dl-catch.c:256)
==257163==    by 0x5015199: dlerror_run (dl-libc.c:45)
==257163==    by 0x5015199: __libc_dlopen_mode (dl-libc.c:162)
==257163==    by 0x4FF51AE: module_load (nss_module.c:187)
==257163==    by 0x4FF567C: __nss_module_load (nss_module.c:302)
==257163==    by 0x4FF567C: __nss_module_get_function (nss_module.c:328)
==257163==    by 0x4F8FCC0: get_nss_addresses (getaddrinfo.c:640)
==257163==    by 0x4F8FCC0: gaih_inet (getaddrinfo.c:1179)
==257163==    by 0x4F8FCC0: getaddrinfo (getaddrinfo.c:2385)
==257163==    by 0x4E05DC6: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.8.0)
==257163==    by 0x4E09F9B: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.8.0)
==257163==    by 0x4F33B59: start_thread (pthread_create.c:444)
==257163==    by 0x4FC4463: clone (clone.S:100)
==257163== 
==257163== LEAK SUMMARY:
==257163==    definitely lost: 0 bytes in 0 blocks
==257163==    indirectly lost: 0 bytes in 0 blocks
==257163==      possibly lost: 0 bytes in 0 blocks
==257163==    still reachable: 1,506 bytes in 4 blocks
==257163==         suppressed: 0 bytes in 0 blocks
==257163== 
==257163== For lists of detected and suppressed errors, rerun with: -s
==257163== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

size_t cr_len = 0; /* working length of canonical request str */
size_t sh_len = 0; /* working length of signed headers str */
char *tmpstr = NULL;
const size_t TMP_STR_SIZE = sizeof(char) * H5FD_ROS3_MAX_SECRET_TOK_LEN;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the extra variable? Seems simpler if you just used H5FD_ROS3_MAX_SECRET_TOK_LEN everywhere.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even if you wanted another symbol to talk about the tmpstr size, you are using a C++ idiom in the middle of the C... A #define for TMP_STR_SIZE would be more idiomatic for C.

@derobins derobins merged commit 49feed3 into HDFGroup:develop Jul 3, 2024
59 checks passed
lrknox pushed a commit to lrknox/hdf5 that referenced this pull request Jul 19, 2024
Just makes a stack array dynamic. Valgrind shows no memory leaks.
lrknox added a commit that referenced this pull request Jul 19, 2024
* Test fixes for log-based vol (#4618)

* fixes to address failures in the log-based VOL

* moved file cleanup to tests proper

* skipped index API test if not supported

* Add 'try' parameter to H5Z_find, and remove calls to H5E_clear_stack() (#4609)

* Bump the github-actions group with 4 updates (#4620)

Bumps the github-actions group with 4 updates: [actions/checkout](https://github.com/actions/checkout), [aws-actions/configure-aws-credentials](https://github.com/aws-actions/configure-aws-credentials), [softprops/action-gh-release](https://github.com/softprops/action-gh-release) and [github/codeql-action](https://github.com/github/codeql-action).

Updates `actions/checkout` from 4.1.1 to 4.1.7
- [Release notes](https://github.com/actions/checkout/releases)
- [Commits](actions/checkout@v4.1.1...v4.1.7)

Updates `aws-actions/configure-aws-credentials` from 1 to 4
- [Release notes](https://github.com/aws-actions/configure-aws-credentials/releases)
- [Changelog](https://github.com/aws-actions/configure-aws-credentials/blob/main/CHANGELOG.md)
- [Commits](aws-actions/configure-aws-credentials@v1...v4)

Updates `softprops/action-gh-release` from 2.0.5 to 2.0.6
- [Release notes](https://github.com/softprops/action-gh-release/releases)
- [Changelog](https://github.com/softprops/action-gh-release/blob/master/CHANGELOG.md)
- [Commits](softprops/action-gh-release@69320db...a74c6b7)

Updates `github/codeql-action` from 3.25.7 to 3.25.11
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](github/codeql-action@f079b84...b611370)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-actions
- dependency-name: aws-actions/configure-aws-credentials
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
- dependency-name: softprops/action-gh-release
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-actions
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Fix a stack size warning in ros3 VFD code (#4625)

Just makes a stack array dynamic. Valgrind shows no memory leaks.

* Reworked cleaning up API test files (#4626)

* Reworked cleaning up test files, only removing test files if present to account for skipped tests

* changed to using H5Fis_accessible

* update to full use of remove_test_file

* corrected offset type in C wrapper (#4622)

* Remove some internal use of API calls and H5E_BEGIN_TRY/H5E_END_TRY (#4624)

* Remove auto NSIS check because of issues with CI (#4646)

* Add python testing for examples (#4628)

* Clean up Fortran __float128 configure-time checks (#4649)

* Always use DECIMAL_DIG instead of LDBL_DIG. This was controlled by
  an ifdef that is always true in C99 or greater

It's confusing to use float.h C constants as variable names in
configure.ac and the PAC_FC_LDBL_DIG macro.

* Directly compare MY_FLT128_DIG and MY_LDBL_DIG

* Make uniform across CMake and Autotools
* Don't export quadmath.h variables to H5pubconf.h

* Feature/awesome (#4604)

* Added Doxygen Awesome and fixed a few quirks.

* Fixed unterminated strings.

* Added Doxygen Awesome by copy.

* Add tools usage text as doxygen  for Tools UG (#4602)

* Add h5* compiler wrapper testing for CMake #4605 (#4613)

* Add show option

* remove non-static libs and correct names of static libs

* Fixup the pkg-config libs and comp builds

* Fix commands and add fortran pkg-config test scripts

* Add help usage option

* Add temporary fix for ARM64 Mac _Float16 build failure (#4639)

* Correct H5VL_t ref count on H5O_refresh_metadata failure (#4636)

* Fix bad H5VL_t rc on H5O_refresh_metadata fail

* Decrement nrefs before raising error

* Update doxygen Learn Basics / example refs. Add Reference sections (#4640)

* Fixed messed up table captions.  (#4653)

* Fixed messed up table captions. Browsers don't seem to respect relative values for width. Hardcoding 800px for now.

* Fixed FetchContent usage for new CMake reqs. (#4650)

CMake version 3.30 changed the behavior of the FetchContent module to deprecate
the use of FetchContent_Populate() in favor of FetchContent_MakeAvailable(). Therefore,
the copying of HDF specialized CMakeLists.txt files to the dependent project's source
was implemented in the FetchContent_Declare() call.

* Fixed usage issue with FindZLIB.cmake module (#4655)

* Add a comment on the FindZLIB.cmake module usage

* Allow choice of static/shared compression libs for Find Module

* Added new option to INSTALL_CMake file and changed option text

* Eliminate more H5E_BEGIN/END_TRY macros and H5E_clear_stack() calls (#4648)

* Correct name of zlib_ng option (#4658)

* Fix the examples for testing java with binaries (#4660)

* Update filename in RELEASE_PROCESS.md to current name
INSTALL_autotools.txt.

* Remove reference to V116 in tools/src/h5repack/h5repack.h.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component - C Library Core C library issues (usually in the src directory) Priority - 2. Medium ⏹ It would be nice to have this in the next release Type - Improvement Improvements that don't add a new feature or functionality
Projects
Status: Needs Merged
Development

Successfully merging this pull request may close these issues.

3 participants