forked from lttng/lttng-tools
-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix: set GLOBAL buffer type for kernel domain in list #1
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
MI is using the list command reponse's buffer type, even when listing the kernel domain. Not setting .buf_type here resulted in MI reporting a wrong buffer type for the kernel domain. Signed-off-by: Philippe Proulx <[email protected]>
Wrong user, sorry. |
jgalar
pushed a commit
that referenced
this pull request
May 17, 2016
Found by Coverity: CID 1321742 (#1 of 2): Buffer not null terminated (BUFFER_SIZE_WARNING)21. buffer_size_warning: Calling strncpy with a maximum size argument of 256 bytes on destination array attr->name of size 256 bytes might leave the destination string unterminated. CID 1321742 (#2 of 2): Buffer not null terminated (BUFFER_SIZE_WARNING)22. buffer_size_warning: Calling strncpy with a maximum size argument of 256 bytes on destination array attr->name of size 256 bytes might leave the destination string unterminated. Signed-off-by: Mathieu Desnoyers <[email protected]> Signed-off-by: Jérémie Galarneau <[email protected]>
jgalar
pushed a commit
that referenced
this pull request
May 17, 2016
Found by Coverity: CID 1321741 (#1 of 1): Buffer not null terminated (BUFFER_SIZE_WARNING)1. buffer_size_warning: Calling strncpy with a maximum size argument of 256 bytes on destination array ust_event->attr.name of size 256 bytes might leave the destination string unterminated. Signed-off-by: Mathieu Desnoyers <[email protected]>
jgalar
pushed a commit
that referenced
this pull request
May 17, 2016
Found by Coverity: CID 1243038 (#1 of 1): Buffer not null terminated (BUFFER_SIZE_WARNING)15. buffer_size_warning: Calling strncpy with a maximum size argument of 4096 bytes on destination array consumer->dst.trace_path of size 4096 bytes might leave the destination string unterminated. Signed-off-by: Mathieu Desnoyers <[email protected]>
jgalar
pushed a commit
that referenced
this pull request
May 17, 2016
Found by Coverity: CID 1243033 (#1 of 1): Buffer not null terminated (BUFFER_SIZE_WARNING)16. buffer_size_warning: Calling strncpy with a maximum size argument of 256 bytes on destination array msg.name of size 256 bytes might leave the destination string unterminated. Signed-off-by: Mathieu Desnoyers <[email protected]>
jgalar
pushed a commit
that referenced
this pull request
May 17, 2016
Found by Coverity: CID 1243016 (#1 of 1): Buffer not null terminated (BUFFER_SIZE_WARNING)14. buffer_size_warning: Calling strncpy with a maximum size argument of 256 bytes on destination array msg.name of size 256 bytes might leave the destination string unterminated. Signed-off-by: Mathieu Desnoyers <[email protected]>
jgalar
pushed a commit
that referenced
this pull request
May 17, 2016
Found by Coverity: CID 1243022 (#1 of 1): Buffer not null terminated (BUFFER_SIZE_WARNING)23. buffer_size_warning: Calling strncpy with a maximum size argument of 256 bytes on destination array (tmp_events + i).name of size 256 bytes might leave the destination string unterminated. Signed-off-by: Mathieu Desnoyers <[email protected]>
jgalar
pushed a commit
that referenced
this pull request
May 17, 2016
Found by Coverity: CID 1243031 (#1 of 2): Buffer not null terminated (BUFFER_SIZE_WARNING)22. buffer_size_warning: Calling strncpy with a maximum size argument of 4096 bytes on destination array (list + idx).ctrl_url of size 4096 bytes might leave the destination string unterminated. CID 1243031 (#2 of 2): Buffer not null terminated (BUFFER_SIZE_WARNING)26. buffer_size_warning: Calling strncpy with a maximum size argument of 255 bytes on destination array (list + idx).name of size 255 bytes might leave the destination string unterminated. Signed-off-by: Mathieu Desnoyers <[email protected]>
jgalar
pushed a commit
that referenced
this pull request
May 17, 2016
Found by Coverity: CID 1243029 (#1 of 1): Buffer not null terminated (BUFFER_SIZE_WARNING)31. buffer_size_warning: Calling strncpy with a maximum size argument of 4096 bytes on destination array obj->subdir of size 4096 bytes might leave the destination string unterminated. Signed-off-by: Mathieu Desnoyers <[email protected]>
jgalar
pushed a commit
that referenced
this pull request
May 17, 2016
Found by Coverity: CID 1243028 (#1 of 2): Buffer not null terminated (BUFFER_SIZE_WARNING)5. buffer_size_warning: Calling strncpy with a maximum size argument of 255 bytes on destination array output->name of size 255 bytes might leave the destination string unterminated. CID 1243028 (#2 of 2): Buffer not null terminated (BUFFER_SIZE_WARNING)10. buffer_size_warning: Calling strncpy with a maximum size argument of 4096 bytes on destination array output->consumer->dst.trace_path of size 4096 bytes might leave the destination string unterminated. Signed-off-by: Mathieu Desnoyers <[email protected]>
jgalar
pushed a commit
that referenced
this pull request
May 17, 2016
Found by Coverity: CID 1243027 (#1 of 1): Buffer not null terminated (BUFFER_SIZE_WARNING)20. buffer_size_warning: Calling strncpy with a maximum size argument of 255 bytes on destination array tmp_output.name of size 255 bytes might leave the destination string unterminated. Signed-off-by: Mathieu Desnoyers <[email protected]>
jgalar
pushed a commit
that referenced
this pull request
May 17, 2016
Found by Coverity: CID 1243018 (#1 of 1): Buffer not null terminated (BUFFER_SIZE_WARNING)11. buffer_size_warning: Calling strncpy with a maximum size argument of 256 bytes on destination array (channels + i).name of size 256 bytes might leave the destination string unterminated. Signed-off-by: Mathieu Desnoyers <[email protected]>
jgalar
pushed a commit
that referenced
this pull request
May 17, 2016
Found by Coverity: CID 1243023 (#1 of 1): Buffer not null terminated (BUFFER_SIZE_WARNING)3. buffer_size_warning: Calling strncpy with a maximum size argument of 4096 bytes on destination array pidfile_path of size 4096 bytes might leave the destination string unterminated. Signed-off-by: Mathieu Desnoyers <[email protected]>
jgalar
pushed a commit
that referenced
this pull request
May 17, 2016
Found by Coverity: CID 1243021 (#1 of 1): Buffer not null terminated (BUFFER_SIZE_WARNING)25. buffer_size_warning: Calling strncpy with a maximum size argument of 255 bytes on destination array (syscall_table + index).name of size 255 bytes might leave the destination string unterminated. Signed-off-by: Mathieu Desnoyers <[email protected]>
jgalar
pushed a commit
that referenced
this pull request
May 17, 2016
Found by Coverity: CID 1243015 (#1 of 1): Buffer not null terminated (BUFFER_SIZE_WARNING)8. buffer_size_warning: Calling strncpy with a maximum size argument of 4096 bytes on destination array consumer->subdir of size 4096 bytes might leave the destination string unterminated. Signed-off-by: Mathieu Desnoyers <[email protected]>
jgalar
pushed a commit
that referenced
this pull request
May 17, 2016
Found by Coverity: CID 1323138 (#1 of 2): Buffer not null terminated (BUFFER_SIZE_WARNING)3. buffer_size_warning: Calling strncpy with a maximum size argument of 64 bytes on destination array session->hostname of size 64 bytes might leave the destination string unterminated. CID 1323138 (#2 of 2): Buffer not null terminated (BUFFER_SIZE_WARNING)3. buffer_size_warning: Calling strncpy with a maximum size argument of 255 bytes on destination array session->session_name of size 255 bytes might leave the destination string unterminated. Signed-off-by: Mathieu Desnoyers <[email protected]>
jgalar
pushed a commit
that referenced
this pull request
May 17, 2016
Found by Coverity: CID 1243024 (#1 of 2): Buffer not null terminated (BUFFER_SIZE_WARNING)2. buffer_size_warning: Calling strncpy with a maximum size argument of 255 bytes on destination array msg.session_name of size 255 bytes might leave the destination string unterminated. CID 1243024 (#2 of 2): Buffer not null terminated (BUFFER_SIZE_WARNING)3. buffer_size_warning: Calling strncpy with a maximum size argument of 64 bytes on destination array msg.hostname of size 64 bytes might leave the destination string unterminated. Signed-off-by: Mathieu Desnoyers <[email protected]>
jgalar
pushed a commit
that referenced
this pull request
May 17, 2016
Found by Coverity: CID 1243017 (#1 of 4): Buffer not null terminated (BUFFER_SIZE_WARNING)14. buffer_size_warning: Calling strncpy with a maximum size argument of 264 bytes on destination array msg.channel_name of size 264 bytes might leave the destination string unterminated. ID 1243017 (#2 of 4): Buffer not null terminated (BUFFER_SIZE_WARNING)14. buffer_size_warning: Calling strncpy with a maximum size argument of 264 bytes on destination array msg_2_2.channel_name of size 264 bytes might leave the destination string unterminated. CID 1243017 (#3 of 4): Buffer not null terminated (BUFFER_SIZE_WARNING)15. buffer_size_warning: Calling strncpy with a maximum size argument of 4096 bytes on destination array msg.pathname of size 4096 bytes might leave the destination string unterminated. CID 1243017 (#4 of 4): Buffer not null terminated (BUFFER_SIZE_WARNING)15. buffer_size_warning: Calling strncpy with a maximum size argument of 4096 bytes on destination array msg_2_2.pathname of size 4096 bytes might leave the destination string unterminated. Signed-off-by: Mathieu Desnoyers <[email protected]>
jgalar
pushed a commit
that referenced
this pull request
May 17, 2016
Found by Coverity: CID 1243025 (#1 of 2): Buffer not null terminated (BUFFER_SIZE_WARNING)17. buffer_size_warning: Calling strncpy with a maximum size argument of 64 bytes on destination array send_session->hostname of size 64 bytes might leave the destination string unterminated. CID 1243025 (#2 of 2): Buffer not null terminated (BUFFER_SIZE_WARNING)17. buffer_size_warning: Calling strncpy with a maximum size argument of 255 bytes on destination array send_session->session_name of size 255 bytes might leave the destination string unterminated. Signed-off-by: Mathieu Desnoyers <[email protected]>
jgalar
pushed a commit
that referenced
this pull request
May 17, 2016
Found by Coverity: CID 1243037 (#1 of 2): Buffer not null terminated (BUFFER_SIZE_WARNING)18. buffer_size_warning: Calling strncpy with a maximum size argument of 4096 bytes on destination array send_stream.path_name of size 4096 bytes might leave the destination string unterminated. CID 1243037 (#2 of 2): Buffer not null terminated (BUFFER_SIZE_WARNING)18. buffer_size_warning: Calling strncpy with a maximum size argument of 255 bytes on destination array send_stream.channel_name of size 255 bytes might leave the destination string unterminated. Signed-off-by: Mathieu Desnoyers <[email protected]>
jgalar
pushed a commit
that referenced
this pull request
May 17, 2016
Found by Coverity: CID 1243032 (#1 of 1): Buffer not null terminated (BUFFER_SIZE)1. buffer_size: Calling strncpy with a source string whose length (8 chars) is greater than or equal to the size argument (8) will fail to null-terminate attr.name. Signed-off-by: Mathieu Desnoyers <[email protected]>
jgalar
pushed a commit
that referenced
this pull request
May 17, 2016
Found by Coverity: CID 1243030 (#1 of 1): Buffer not null terminated (BUFFER_SIZE_WARNING)1. buffer_size_warning: Calling strncpy with a maximum size argument of 256 bytes on destination array ev.name of size 256 bytes might leave the destination string unterminated. Signed-off-by: Mathieu Desnoyers <[email protected]>
jgalar
pushed a commit
that referenced
this pull request
May 17, 2016
Found by Coverity: CID 1243026 (#1 of 1): Buffer not null terminated (BUFFER_SIZE_WARNING)1. buffer_size_warning: Calling strncpy with a maximum size argument of 256 bytes on destination array ev.name of size 256 bytes might leave the destination string unterminated. Signed-off-by: Mathieu Desnoyers <[email protected]>
jgalar
pushed a commit
that referenced
this pull request
May 17, 2016
Found by Coverity: CID 1243019 (#1 of 1): Buffer not null terminated (BUFFER_SIZE_WARNING)1. buffer_size_warning: Calling strncpy with a maximum size argument of 256 bytes on destination array ev.name of size 256 bytes might leave the destination string unterminated. Signed-off-by: Mathieu Desnoyers <[email protected]>
jgalar
pushed a commit
that referenced
this pull request
May 17, 2016
Found by Coverity: CID 1321742 (#1 of 2): Buffer not null terminated (BUFFER_SIZE_WARNING)21. buffer_size_warning: Calling strncpy with a maximum size argument of 256 bytes on destination array attr->name of size 256 bytes might leave the destination string unterminated. CID 1321742 (#2 of 2): Buffer not null terminated (BUFFER_SIZE_WARNING)22. buffer_size_warning: Calling strncpy with a maximum size argument of 256 bytes on destination array attr->name of size 256 bytes might leave the destination string unterminated. Signed-off-by: Mathieu Desnoyers <[email protected]> Signed-off-by: Jérémie Galarneau <[email protected]>
jgalar
pushed a commit
that referenced
this pull request
May 17, 2016
Found by Coverity: CID 1321741 (#1 of 1): Buffer not null terminated (BUFFER_SIZE_WARNING)1. buffer_size_warning: Calling strncpy with a maximum size argument of 256 bytes on destination array ust_event->attr.name of size 256 bytes might leave the destination string unterminated. Signed-off-by: Mathieu Desnoyers <[email protected]> Signed-off-by: Jérémie Galarneau <[email protected]>
jgalar
pushed a commit
that referenced
this pull request
May 17, 2016
Found by Coverity: CID 1243038 (#1 of 1): Buffer not null terminated (BUFFER_SIZE_WARNING)15. buffer_size_warning: Calling strncpy with a maximum size argument of 4096 bytes on destination array consumer->dst.trace_path of size 4096 bytes might leave the destination string unterminated. Signed-off-by: Mathieu Desnoyers <[email protected]> Signed-off-by: Jérémie Galarneau <[email protected]>
jgalar
pushed a commit
that referenced
this pull request
May 17, 2016
Found by Coverity: CID 1243033 (#1 of 1): Buffer not null terminated (BUFFER_SIZE_WARNING)16. buffer_size_warning: Calling strncpy with a maximum size argument of 256 bytes on destination array msg.name of size 256 bytes might leave the destination string unterminated. Signed-off-by: Mathieu Desnoyers <[email protected]> Signed-off-by: Jérémie Galarneau <[email protected]>
jgalar
pushed a commit
that referenced
this pull request
Jan 13, 2022
Observed issue ============== lttng-consumerd segfaults for the following scenario: $ lttng create test --snapshot $ lttng enable-event -u -an $ lttng start # Run an app just to have some events $ lttng regenerate metadata $ lttng snapshot record The following backtrace is obtained: (gdb) bt #0 __GI___pthread_mutex_lock (mutex=0x130) at ../nptl/pthread_mutex_lock.c:67 #1 0x000055b383cfaed3 in lttng_ustconsumer_recv_metadata (sock=29, key=4, offset=0, len=12391, version=1, channel=0x7fe90000a760, timer=0, wait=1) at ust-consumer.c:1347 #2 0x000055b383d00197 in lttng_ustconsumer_request_metadata (ctx=0x55b3855a1e50, channel=0x7fe90000a760, timer=0, wait=1) at ust-consumer.c:3336 #3 0x000055b383cf9e76 in snapshot_metadata (metadata_channel=0x7fe90000a760, key=4, path=0x7fe911a09944 "uid/1000/64-bit", relayd_id=18446744073709551615, ctx=0x55b3855a1e50) at ust-consum #4 0x000055b383cfbe73 in lttng_ustconsumer_recv_cmd (ctx=0x55b3855a1e50, sock=28, consumer_sockpoll=0x7fe911a0dbb0) at ust-consumer.c:1853 #5 0x000055b383ccf9b7 in lttng_consumer_recv_cmd (ctx=0x55b3855a1e50, sock=28, consumer_sockpoll=0x7fe911a0dbb0) at consumer.c:2097 #6 0x000055b383cd3bfd in consumer_thread_sessiond_poll (data=0x55b3855a1e50) at consumer.c:3284 #7 0x00007fe914c22609 in start_thread (arg=<optimized out>) at pthread_create.c:477 #8 0x00007fe914b47293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 (gdb) up #1 0x000055b383cfaed3 in lttng_ustconsumer_recv_metadata (sock=29, key=4, offset=0, len=12391, version=1, channel=0x7fe90000a760, timer=0, wait=1) at ust-consumer.c:1347 1347 pthread_mutex_lock(&channel->metadata_stream->lock); (gdb) print channel->metadata_stream $1 = (struct lttng_consumer_stream *) 0x0 Note that the following scenario also leads to a similar backtrace: $ lttng create test --snapshot $ lttng enable-event -u -an $ lttng start # Run an app just to have some events with a short duration $ lttng regenerate metadata # Run a second app just to have some events and a metadata flush while # the metadata cache status is set as `invalidated`. ^ lttng-consumerd segfault on app termination. The backtrace: (gdb) bt #0 __GI___pthread_mutex_lock (mutex=0x130) at ../nptl/pthread_mutex_lock.c:67 #1 0x00005612a5a13ed3 in lttng_ustconsumer_recv_metadata (sock=28, key=2, offset=0, len=12391, version=1, channel=0x7f3978005400, timer=0, wait=1) at ust-consumer.c:1347 #2 0x00005612a5a14d0a in lttng_ustconsumer_recv_cmd (ctx=0x5612a6feee50, sock=28, consumer_sockpoll=0x7f3989494bb0) at ust-consumer.c:1818 #3 0x00005612a59e89b7 in lttng_consumer_recv_cmd (ctx=0x5612a6feee50, sock=28, consumer_sockpoll=0x7f3989494bb0) at consumer.c:2097 #4 0x00005612a59ecbfd in consumer_thread_sessiond_poll (data=0x5612a6feee50) at consumer.c:3284 #5 0x00007f398c6a9609 in start_thread (arg=<optimized out>) at pthread_create.c:477 #6 0x00007f398c5ce293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 (gdb) up #1 0x00005612a5a13ed3 in lttng_ustconsumer_recv_metadata (sock=28, key=2, offset=0, len=12391, version=1, channel=0x7f3978005400, timer=0, wait=1) at ust-consumer.c:1347 1347 pthread_mutex_lock(&channel->metadata_stream->lock); (gdb) print channel->metadata_stream $1 = (struct lttng_consumer_stream *) 0x0 (gdb) Cause ===== For session configured in snapshot mode the metadata channel metadata_stream field is NULL except for a "short" window during the actual snapshot record action (snapshot_metadata). The `regenerate metadata` effectively flag the metadata cache as invalid leading to handling the cache invalidation state (`CONSUMER_METADATA_CACHE_WRITE_STATUS_INVALIDATED`) in `lttng_ustconsumer_recv_metadata`. This was introduced by b1316da [1]. At that point the function expects the `channel->metadata_stream` to be non null. This is simply not true for snapshot session metadata channels. Note that we cannot simply swap `lttng_ustconsumer_request_metadata` and `create_ust_streams` in `snapshot_metadata` to ensure that the `channel->metadata_stream` is non null since `lttng_ustconsumer_recv_metadata` ends up being called on metadata flush when an app quit. This sequence of events corresponds to the second scenario put forward in the `Observed Issue` section. Solution ======== Null check `channel->metadata_stream` and perform only the operation when it is non null. This partly mirror what is done in `consumer_metadata_wakeup_pipe`. I am not sure if the check on `channel->monitor` is required but it seems irrelevant to the notion of resetting the stream consumed position when the stream exists. With this taken care off, we find ourself with another backtrace: #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 [93/122] #1 0x00007f75cf9b3859 in __GI_abort () at abort.c:79 #2 0x00007f75cf9b3729 in __assert_fail_base (fmt=0x7f75cfb49588 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x55ab004e9c68 "pthread_mutex_trylock(&stream->lock)", file=0x55ab004 #3 0x00007f75cf9c4f36 in __GI___assert_fail (assertion=0x55ab004e9c68 "pthread_mutex_trylock(&stream->lock)", file=0x55ab004e8d7a "ust-consumer.c", line=1285, function=0x55ab004eb8a0 <__PR #4 0x000055ab004b1b9c in metadata_stream_reset_cache_consumed_position (stream=0x7f75b400a850) at ust-consumer.c:1285 #5 0x000055ab004b4fef in commit_one_metadata_packet (stream=0x7f75b400a850) at ust-consumer.c:2551 #6 0x000055ab004b5f46 in get_next_subbuffer_metadata (stream=0x7f75b400a850, subbuffer=0x7f75cc972630) at ust-consumer.c:2927 #7 0x000055ab0048b6a9 in lttng_consumer_read_subbuffer (stream=0x7f75b400a850, ctx=0x55ab01d4ee50, locked_by_caller=true) at consumer.c:3522 #8 0x000055ab004b0f66 in snapshot_metadata (metadata_channel=0x7f75b4005400, key=2, path=0x7f75cc972944 "uid/1000/64-bit", relayd_id=18446744073709551615, ctx=0x55ab01d4ee50) at ust-consum lttng#9 0x000055ab004b2e86 in lttng_ustconsumer_recv_cmd (ctx=0x55ab01d4ee50, sock=28, consumer_sockpoll=0x7f75cc976bb0) at ust-consumer.c:1861 lttng#10 0x000055ab004869b7 in lttng_consumer_recv_cmd (ctx=0x55ab01d4ee50, sock=28, consumer_sockpoll=0x7f75cc976bb0) at consumer.c:2097 lttng#11 0x000055ab0048abfd in consumer_thread_sessiond_poll (data=0x55ab01d4ee50) at consumer.c:3284 lttng#12 0x00007f75cfb8b609 in start_thread (arg=<optimized out>) at pthread_create.c:477 lttng#13 0x00007f75cfab0293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 Which is also caused in part to the invalidation of the cache. `metadata_stream_reset_cache_consumed_position` expect that the stream at that point be locked. Which is not the case despite what the last argument to `lttng_consumer_read_subbuffer` indicates. To alleviate that we hold the lock during the call to `lttng_consumer_read_subbuffer`. Known drawbacks ========= None. References ========= [1] https://git.lttng.org/?p=lttng-tools.git;a=commit;h=b1316da1ffbd276fc8271e7a9438e683ad352781 Signed-off-by: Jonathan Rajotte <[email protected]> Signed-off-by: Jérémie Galarneau <[email protected]> Change-Id: Ib09fdc989b1baae26d0f496afeabc3e7cb27228c
jgalar
pushed a commit
that referenced
this pull request
Jan 13, 2022
Observed issue ============== While running tests/regression/tools/clear/test_ust test in a loop we eventually witness the following error: The symptom on the Babeltrace side is a Connection reset by peer. This is caused by a relayd abort after an assertion failure due to a reference count being lower than 0. Test case: # Test ust streaming live clear with viewer # Parameters: tracing_active=0, clear_twice=0, buffer_type=pid Tools commit: https://review.lttng.org/c/lttng-tools/+/6754/1 Relay daemon assertion failure stack trace: (gdb) bt #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 #1 0x00007ffa2e850859 in __GI_abort () at abort.c:79 #2 0x00007ffa2e850729 in __assert_fail_base (fmt=0x7ffa2e9e6588 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x56221bdb96f4 "res >= 0", file=0x56221bdb97a0 "/root/virtenv/usr/include/urcu/ref.h", line=66, function=<optimized out>) at assert.c:92 #3 0x00007ffa2e861f36 in __GI___assert_fail (assertion=assertion@entry=0x56221bdb96f4 "res >= 0", file=file@entry=0x56221bdb97a0 "/root/virtenv/usr/include/urcu/ref.h", line=line@entry=66, function=function@entry=0x56221bdbd638 <__PRETTY_FUNCTION__.7554> "urcu_ref_put") at assert.c:101 #4 0x000056221bd6a1cc in urcu_ref_put (release=<optimized out>, ref=0x7ffa24008cb0) at /root/virtenv/usr/include/urcu/ref.h:66 #5 viewer_stream_put (vstream=vstream@entry=0x7ffa24008cb0) at viewer-stream.c:279 #6 0x000056221bd5e4c5 in viewer_get_metadata (conn=conn@entry=0x7ffa0c000fc0) at live.c:2211 #7 0x000056221bd63778 in process_control (conn=0x7ffa0c000fc0, recv_hdr=0x7ffa297c5af0) at live.c:2376 #8 thread_worker (data=<optimized out>) at live.c:2541 lttng#9 0x00007ffa2ea28609 in start_thread (arg=<optimized out>) at pthread_create.c:477 lttng#10 0x00007ffa2e94d293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 Babeltrace crash: 11-17 05:13:05.197 47007 47007 D PLUGIN/SRC.CTF.LTTNG-LIVE/VIEWER [email protected]:1494 [live] Requesting next index for stream: cmd=GET_NEXT_INDEX, viewer-stream-id=3352 11-17 05:13:05.215 47007 47007 E PLUGIN/SRC.CTF.LTTNG-LIVE/VIEWER [email protected]:245 [live] Error receiving from Relay: Connection reset by peer. 11-17 05:13:05.215 47007 47007 E PLUGIN/SRC.CTF.LTTNG-LIVE/VIEWER [email protected]:1522 [live] Error receiving get next index reply 11-17 05:13:05.215 47007 47007 D PLUGIN/SRC.CTF.LTTNG-LIVE [email protected]:1069 [live] Returning from advancing live stream iterator: status=LTTNG_LIVE_ITERATOR_STATUS_ERROR, stream-name="stream-3352", viewer-stream-id=3352 11-17 05:13:05.215 47007 47007 E PLUGIN/SRC.CTF.LTTNG-LIVE [email protected]:1802 [live] Error preparing the next batch of messages: live-iter-status=LTTNG_LIVE_ITERATOR_STATUS_ERROR 11-17 05:13:05.216 47007 47007 W LIB/MSG-ITER [email protected]:866 Component input port message iterator's "next" method failed: iter-addr=0x563b7dd07700, iter-upstream-comp-name="live", iter-upstream-comp-log-level=TRACE, iter-upstream-comp-class-type=SOURCE, iter-upstream-comp-class-name="lttng-live", iter-upstream-comp-class-partial-descr="Connect to an LTTng relay daemon", iter-upstream-port-type=OUT PUT, iter-upstream-port-name="out", status=ERROR 11-17 05:13:05.216 47007 47007 E PLUGIN/FLT.UTILS.MUXER [email protected]:446 [mux] Upstream iterator's next method returned an error: status=ERROR 11-17 05:13:05.216 47007 47007 E PLUGIN/FLT.UTILS.MUXER [email protected]:989 [mux] Cannot validate muxer's upstream message iterator wrapper: muxer-msg-iter-addr=0x563b7dd04740, muxer-upstream-msg-iter-wrap-addr=0x563b7dd0db30 11-17 05:13:05.216 47007 47007 E PLUGIN/FLT.UTILS.MUXER [email protected]:1417 [mux] Cannot get next message: comp-addr=0x563b7dd1a760, muxer-comp-addr=0x563b7dd1b930, muxer-msg-iter-addr=0x563b7dd04740, msg-iter-addr=0x563b7dd07590, status=ERROR 11-17 05:13:05.216 47007 47007 W LIB/MSG-ITER [email protected]:866 Component input port message iterator's "next" method failed: iter-addr=0x563b7dd07590, iter-upstream-comp-name="mux", iter-upstream-comp-log-level=WARNING, iter-upstream-comp-class-type=FILTER, iter-upstream-comp-class-name="muxer", iter-upstream-comp-class-partial-descr="Sort messages from multiple inpu", iter-upstream-port-type=OUTPUT, iter-upstream-port-name="out", status=ERROR 11-17 05:13:05.216 47007 47007 W LIB/GRAPH [email protected]:462 Component's "consume" method failed: status=ERROR, comp-addr=0x563b7dd06d20, comp-name="pretty", comp-log-level=WARNING, comp-class-type=SINK, comp-class-name="pretty", comp-class-partial-descr="Pretty-print messages (`text` fo", comp-class-is-frozen=1, comp-class-so-handle-addr=0x563b7dd051b0, comp-class-so-handle-path="/root/virtenv/usr/lib/babeltr ace2/plugins/babeltrace-plugin-text.so", comp-input-port-count=1, comp-output-port-count=0 11-17 05:13:05.217 47007 47007 E CLI [email protected]:2537 Graph failed to complete successfully Cause ===== Both relay and live threads can put the ownership reference on the metadata viewer stream concurrently without synchronization, thus leading to a reference count going lower than 0. The viewer stream ownership design initially planned for being owned by the live thread, thus allowing the live thread to put the ownership reference as soon as the associated relay stream is observed as closed, and the viewer stream is considered as hung up. However, in the specific case of the metadata viewer stream, the responsibility of closing the metadata viewer stream is shared between the relay and live threads, because the viewers expect to observe a LTTNG_VIEWER_NO_NEW_METADATA message before the metadata stream hangs up (see comment in viewer_get_metadata()). Therefore, if viewer_get_metadata() is done before the metadata stream is closed, the viewer will receive the LTTNG_VIEWER_NO_NEW_METADATA message, and set the no_new_metadata_notified state to true. It's then the relay thread's relay_close_stream() which will invoke the ownership put. However, the live thread may concurrently try to put the viewer stream ownership as well from a subsequent viewer_get_metadata(), thus leading to a reference count < 0. Solution ======== Fix this by putting the ownership reference from the live viewer thread rather than the relay thread. This can be done by tracking the state of no_new_metadata_notified within the live viewer thread. Known drawbacks =============== This will postpone reclaim of the metadata viewer stream from the relay stream close to the following viewer_get_metadata (after a LTTNG_VIEWER_NO_NEW_METADATA message has been replied), which I don't think is an issue. Signed-off-by: Mathieu Desnoyers <[email protected]> Signed-off-by: Jérémie Galarneau <[email protected]> Change-Id: I9783d3a8402dddaa364a7a69c0764eaf30c16fde
jgalar
pushed a commit
that referenced
this pull request
May 25, 2022
If the parsing completed but is not the expected size, the *out_event_context object leaks. Reported by Coverity: CID 1475821 (#1 of 1): Resource leak (RESOURCE_LEAK) Variable event_context going out of scope leaks the storage it points to. Change-Id: I8198f7f3f1e1af3dcb377d89674402ec49cb788b Signed-off-by: Jonathan Rajotte <[email protected]> Signed-off-by: Jérémie Galarneau <[email protected]>
jgalar
added a commit
that referenced
this pull request
May 25, 2022
The following leak is reported by LeakSanitizer when setup_metadata() fails to send the metadata stream to the relay daemon: ==3050181==ERROR: LeakSanitizer: detected memory leaks Direct leak of 240 byte(s) in 5 object(s) allocated from: #0 0x7f5fce02cfb9 in __interceptor_calloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:154 #1 0x7f5fcdd95a7a in zmalloc ../../../src/common/macros.h:23 #2 0x7f5fcdd95a7a in lttng_ust_ctl_create_stream /home/jgalar/EfficiOS/src/lttng-ust/src/lib/lttng-ust-ctl/ustctl.c:1649 A consumer stream can have an allocated `struct lttng_ust_ctl_consumer_stream *` (ustream) even if it is not globally visible at the time of its teardown. In the case of the user space consumer, the only site that creates consumer stream instances ensures that the allocation of the lttng_ust_ctl_consumer_stream succeeded, ensuring that the consumer stream's 'ustream' is always set. Signed-off-by: Jérémie Galarneau <[email protected]> Change-Id: Ia4be7969e85bd8812ae13b042e1e100812a63c1d
jgalar
added a commit
that referenced
this pull request
May 25, 2022
Observed issue ============== When consumer_stream_destroy() is called from, for example, the error path in setup_metadata(), consumer_stream_free() can end up being called twice on the same stream. Since the stream->metadata_bucket is not set to NULL after being destroyed, it leads to a use-after-free: ERROR: AddressSanitizer: heap-use-after-free on address 0x604000000318 READ of size 8 at 0x604000000318 thread T7 #0 in metadata_bucket_destroy #1 in consumer_stream_free #2 in consumer_stream_destroy #3 in setup_metadata #4 in lttng_ustconsumer_recv_cmd #5 in lttng_consumer_recv_cmd #6 in consumer_thread_sessiond_poll #7 in start_thread nptl/pthread_create.c:481 #8 in clone (/lib/x86_64-linux-gnu/libc.so.6+0xfcbde) 0x604000000318 is located 8 bytes inside of 48-byte region [0x604000000310,0x604000000340) freed by thread T7 here: #0 in __interceptor_free #1 in metadata_bucket_destroy #2 in consumer_stream_free #3 in consumer_stream_destroy #4 in clean_channel_stream_list #5 in consumer_del_channel #6 in consumer_stream_destroy #7 in setup_metadata #8 in lttng_ustconsumer_recv_cmd lttng#9 in lttng_consumer_recv_cmd lttng#10 in consumer_thread_sessiond_poll lttng#11 in start_thread nptl/pthread_create.c:481 previously allocated by thread T7 here: #0 in __interceptor_calloc #1 in zmalloc #2 in metadata_bucket_create #3 in consumer_stream_enable_metadata_bucketization #4 in lttng_ustconsumer_set_stream_ops #5 in lttng_ustconsumer_on_recv_stream #6 in lttng_consumer_on_recv_stream #7 in create_ust_streams #8 in ask_channel lttng#9 in lttng_ustconsumer_recv_cmd lttng#10 in lttng_consumer_recv_cmd lttng#11 in consumer_thread_sessiond_poll lttng#12 in start_thread nptl/pthread_create.c:481 Thread T7 created by T0 here: #0 in __interceptor_pthread_create #1 in main #2 in __libc_start_main ../csu/libc-start.c:332 SUMMARY: AddressSanitizer: heap-use-after-free in metadata_bucket_destroy This can be easily reproduced by forcing a failure during the setup of the metadata reproducible using the following change: diff --git a/src/common/ust-consumer/ust-consumer.c b/src/common/ust-consumer/ust-consumer.c index fa1c71299..97ed59632 100644 --- a/src/common/ust-consumer/ust-consumer.c +++ b/src/common/ust-consumer/ust-consumer.c @@ -908,8 +908,7 @@ static int setup_metadata(struct lttng_consumer_local_data *ctx, uint64_t key) /* Send metadata stream to relayd if needed. */ if (metadata->metadata_stream->net_seq_idx != (uint64_t) -1ULL) { - ret = consumer_send_relayd_stream(metadata->metadata_stream, - metadata->pathname); + ret = -1; if (ret < 0) { ret = LTTCOMM_CONSUMERD_ERROR_METADATA; goto error; Cause ===== Channels have a list of streams that are being "setup" and are not yet monitored for consumption. During this setup phase, the streams are owned by the channel. On destruction of the channel, any stream in that list will thus be cleaned-up. When destroying a consumer stream, a reference to its channel is 'put'. This can result in the destruction of the channel. In the situation described above, the release of the channel's reference is done before the stream is removed from the channel's stream list. This causes the channel's clean-up to invoke (again) the current stream's clean-up, resulting in the double-free of the metadata bucket. This problem is present in a number of error paths. Solution ======== Some error paths already manually removed the consumer stream from it's channel's stream list before invoking consumer_stream_destroy(). The various error paths that have to deal with this possible situation are changed to simply invoke consumer_stream_destroy(). consumer_stream_destroy() is modified to always remove the stream from its channel's list before performing the rest of the clean-up. This ensures that those double clean-ups can't occur. Drawbacks ========= None. Reported-by: Vincent Whitchurch <[email protected]> Tested-by: Vincent Whitchurch <[email protected]> Signed-off-by: Jérémie Galarneau <[email protected]> Change-Id: Ibeca9b675b86fc46be3f57826f7158de4da43df8
jgalar
pushed a commit
that referenced
this pull request
May 25, 2022
Observed issue ============== Scenario: gdb lttng-sessiond set non-stop break rotation-thread.cpp:584 ^ simulates a slow rotation thread or not scheduled thread. lttng create test1 lttng enable-event -u -a lttng start test1 lttng create test2 lttng enable-event -u -a lttng start test2 lttng destroy test1 This will hang on rotation pending checks on the CLI side. In another shell: lttng destroy test2 This will hang on rotation pending checks on the CLI side. Back to gdb thread 7 continue Results in: #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 #1 0x00007ffff786c859 in __GI_abort () at abort.c:79 #2 0x00007ffff786c729 in __assert_fail_base (fmt=0x7ffff7a02588 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x5555556bb148 "count == lttng_ht_get_count(ltt_sessions_ht_by_name)", file=0x5555556bae9f "session.cpp", line=395, function=<optimized out>) at assert.c:92 #3 0x00007ffff787e006 in __GI___assert_fail (assertion=0x5555556bb148 "count == lttng_ht_get_count(ltt_sessions_ht_by_name)", file=0x5555556bae9f "session.cpp", line=395, function=0x5555556bb129 "int ltt_sessions_ht_empty()") at assert.c:101 #4 0x0000555555586d59 in ltt_sessions_ht_empty () at session.cpp:395 #5 0x0000555555586e53 in del_session_ht (ls=0x7fffdc000c30) at session.cpp:418 #6 0x0000555555588a95 in session_release (ref=0x7fffdc001e50) at session.cpp:999 #7 0x000055555558620f in urcu_ref_put (ref=0x7fffdc001e50, release=0x5555555886eb <session_release(urcu_ref*)>) at /home/joraj/lttng/master/install/include/urcu/ref.h:68 #8 0x0000555555588c8f in session_put (session=0x7fffdc000c30) at session.cpp:1048 lttng#9 0x00005555555bf995 in handle_job_queue (handle=0x55555575d260, state=0x7fffeeffc240, queue=0x555555758960) at rotation-thread.cpp:612 lttng#10 0x00005555555c05da in thread_rotation (data=0x55555575d260) at rotation-thread.cpp:847 lttng#11 0x00005555555c3b1c in launch_thread (data=0x55555575d2f0) at thread.cpp:66 lttng#12 0x00007ffff7a46609 in start_thread (arg=<optimized out>) at pthread_create.c:477 lttng#13 0x00007ffff7969163 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 Other scenarios can lead to a similar backtrace when using the `--no-wait` lttng destroy option. Cause ===== Since ed41e57 [1], hash table removal for the session object for the `ltt_sessions_ht_by_name` and `ltt_sessions_ht_by_name` are "decoupled". Removal from `ltt_sessions_ht_by_name` is done early in `session_destroy()` while removal from `ltt_sessions_ht_by_id` is done during `session_release` when the last reference of a session object is released. This can leads to `imbalances` between the size of the two hash tables when multiple sessions are at play. Solution ======== Rework `ltt_sessions_ht_empty()` to exit early when `ltt_sessions_ht_by_id` is not empty. Perform a sanity check on `ltt_sessions_ht_by_name` only when `ltt_sessions_ht_by_id` is empty. Note ======== Ideally both hash tables' lifetime would be managed separately but it seems easier in term of initialization to bundle them together for now considering the limited scope of the `ltt_sessions_ht_by_name` hash table. Signed-off-by: Jonathan Rajotte <[email protected]> Signed-off-by: Jérémie Galarneau <[email protected]> Change-Id: I66c459f80298f929add703ac977cccd1da6dd556
jgalar
pushed a commit
that referenced
this pull request
May 25, 2022
Observed issue ============== Using the notification client from doc/examples/trigger-condition-event-matches/notification-client.cpp, an assert is hit when the notification subsystem is under load. #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 #1 0x00007f69eab58859 in __GI_abort () at abort.c:79 #2 0x00007f69eab58729 in __assert_fail_base (fmt=0x7f69eacee588 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x7f69eae1d5dd "len > 0", file=0x7f69eae1d5cb "unix.cpp", line=179, function=<optimized out>) at assert.c:92 #3 0x00007f69eab6a006 in __GI___assert_fail (assertion=0x7f69eae1d5dd "len > 0", file=0x7f69eae1d5cb "unix.cpp", line=179, function=0x7f69eae1d598 "ssize_t lttcomm_recv_unix_sock(int, void*, size_t)") at assert.c:101 #4 0x00007f69eadd5fe6 in lttcomm_recv_unix_sock (sock=3, buf=0x55da9ecd5f89, len=0) at unix.cpp:179 #5 0x00007f69ead7df3f in receive_message (channel=0x55da9ecd6ee0) at channel.cpp:64 #6 0x00007f69ead7e478 in lttng_notification_channel_get_next_notification (channel=0x55da9ecd6ee0, _notification=0x7ffdefed2570) at channel.cpp:279 #7 0x000055da9e0e742f in main (argc=2, argv=0x7ffdefed2698) at notification-client.cpp:506 (gdb) frame #5 0x00007f69ead7df3f in receive_message (channel=0x55da9ecd6ee0) at channel.cpp:64 64 ret = lttcomm_recv_unix_sock(channel->socket, (gdb) print msg $2 = {type = 5 '\005', size = 0, fds = 0, payload = 0x7ffdefed24a8 ""} The msg type 5 is `LTTNG_NOTIFICATION_CHANNEL_MESSAGE_TYPE_NOTIFICATION_DROPPED` Cause ===== The msg portion of a `LTTNG_NOTIFICATION_CHANNEL_MESSAGE_TYPE_NOTIFICATION_DROPPED` is indeed zero. There is no extra payload. Solution ======== When the msg size is zero, skip the 'payload' reception phase. Known drawbacks ========= None. Signed-off-by: Jonathan Rajotte <[email protected]> Signed-off-by: Jérémie Galarneau <[email protected]> Change-Id: Ibabb922d0e410c9902414a5eabbe04738861d772
jgalar
pushed a commit
that referenced
this pull request
May 25, 2022
Observed issue ============== During high throughput event notification generation scenarios the following deadlock happens: Thread 14 (Thread 0x7f74b4ff9700 (LWP 76062)): #0 __lll_lock_wait (futex=futex@entry=0x56408765dde8, private=0) at lowlevellock.c:52 #1 0x00007f74c941a0a3 in __GI___pthread_mutex_lock (mutex=0x56408765dde8) at ../nptl/pthread_mutex_lock.c:80 #2 0x000056408704b207 in run_command_wait (handle=0x56408765ddd0, cmd=0x7f74b4ff7f70) at notification-thread-commands.cpp:31 #3 0x000056408704bcef in notification_thread_command_remove_tracer_event_source (handle=0x56408765ddd0, tracer_event_source_fd=54) at notification-thread-commands.cpp:319 #4 0x000056408708a0c1 in delete_ust_app (app=0x7f749c000bf0) at ust-app.cpp:1059 #5 0x000056408708a511 in delete_ust_app_rcu (head=0x7f749c000ca0) at ust-app.cpp:1122 #6 0x00007f74c988b4a7 in call_rcu_thread (arg=0x7f74b8004a80) at ../src/urcu-call-rcu-impl.h:369 #7 0x00007f74c9417609 in start_thread (arg=<optimized out>) at pthread_create.c:477 #8 0x00007f74c933a163 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 Thread 13 (Thread 0x7f74b57fa700 (LWP 76047)): #0 0x00007f74c933a49e in epoll_wait (epfd=48, events=0x7f74a4000b60, maxevents=2, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30 #1 0x00005640870eafa6 in compat_epoll_wait (events=0x7f74b57f9240, timeout=-1, interruptible=false) at compat/poll.cpp:280 #2 0x00005640870abb65 in thread_agent_management (data=0x56408765f0b0) at agent-thread.cpp:424 #3 0x0000564087062b1a in launch_thread (data=0x56408765f150) at thread.cpp:66 #4 0x00007f74c9417609 in start_thread (arg=<optimized out>) at pthread_create.c:477 #5 0x00007f74c933a163 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 Thread 12 (Thread 0x7f74b5ffb700 (LWP 76046)): [630/709] #0 0x00007f74c933a49e in epoll_wait (epfd=47, events=0x7f74a0000b60, maxevents=2, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30 #1 0x00005640870eafa6 in compat_epoll_wait (events=0x7f74b5ffa170, timeout=-1, interruptible=false) at compat/poll.cpp:280 #2 0x00005640870a4095 in thread_application_notification (data=0x56408765ee40) at notify-apps.cpp:78 #3 0x0000564087062b1a in launch_thread (data=0x56408765eed0) at thread.cpp:66 #4 0x00007f74c9417609 in start_thread (arg=<optimized out>) at pthread_create.c:477 #5 0x00007f74c933a163 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 Thread 11 (Thread 0x7f74b67fc700 (LWP 76045)): #0 0x00007f74c933a49e in epoll_wait (epfd=44, events=0x7f74ac000b60, maxevents=2, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30 #1 0x00005640870eafa6 in compat_epoll_wait (events=0x7f74b67fb170, timeout=-1, interruptible=false) at compat/poll.cpp:280 #2 0x00005640870723db in thread_application_management (data=0x56408765ebd0) at manage-apps.cpp:93 #3 0x0000564087062b1a in launch_thread (data=0x56408765ec60) at thread.cpp:66 #4 0x00007f74c9417609 in start_thread (arg=<optimized out>) at pthread_create.c:477 #5 0x00007f74c933a163 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 Thread 10 (Thread 0x7f74b6ffd700 (LWP 76044)): #0 0x00007f74c933a49e in epoll_wait (epfd=39, events=0x7f74a8000b60, maxevents=2, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30 #1 0x00005640870eafa6 in compat_epoll_wait (events=0x7f74b6ffc130, timeout=-1, interruptible=false) at compat/poll.cpp:280 #2 0x0000564087070a27 in thread_application_registration (data=0x56408765e940) at register.cpp:214 #3 0x0000564087062b1a in launch_thread (data=0x56408765e9f0) at thread.cpp:66 #4 0x00007f74c9417609 in start_thread (arg=<optimized out>) at pthread_create.c:477 #5 0x00007f74c933a163 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 Thread 9 (Thread 0x7f74b77fe700 (LWP 76043)): [654/709] #0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38 #1 0x00005640870c8e25 in futex (uaddr=0x5640871e2800 <ust_cmd_queue>, op=0, val=-1, timeout=0x0, uaddr2=0x0, val3=0) at /home/joraj/lttng/master/install/include/urcu/futex.h:72 #2 0x00005640870c8e6d in futex_async (uaddr=0x5640871e2800 <ust_cmd_queue>, op=0, val=-1, timeout=0x0, uaddr2=0x0, val3=0) at /home/joraj/lttng/master/install/include/urcu/futex.h:104 #3 0x00005640870c939a in futex_nto1_wait (futex=0x5640871e2800 <ust_cmd_queue>) at futex.cpp:77 #4 0x000056408706f2af in thread_dispatch_ust_registration (data=0x56408765e740) at dispatch.cpp:453 #5 0x0000564087062b1a in launch_thread (data=0x56408765e760) at thread.cpp:66 #6 0x00007f74c9417609 in start_thread (arg=<optimized out>) at pthread_create.c:477 #7 0x00007f74c933a163 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 Thread 8 (Thread 0x7f74b7fff700 (LWP 76042)): #0 0x00007f74c933a49e in epoll_wait (epfd=33, events=0x7f74b0000b60, maxevents=2, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30 #1 0x00005640870eafa6 in compat_epoll_wait (events=0x7f74b7ffad40, timeout=-1, interruptible=false) at compat/poll.cpp:280 #2 0x000056408706c424 in thread_manage_clients (data=0x56408765e4f0) at client.cpp:2528 #3 0x0000564087062b1a in launch_thread (data=0x56408765e560) at thread.cpp:66 #4 0x00007f74c9417609 in start_thread (arg=<optimized out>) at pthread_create.c:477 #5 0x00007f74c933a163 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 Thread 7 (Thread 0x7f74c4b8f700 (LWP 76041)): [672/709] #0 0x00007f74c933a49e in epoll_wait (epfd=31, events=0x7f74bc000b60, maxevents=3, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30 #1 0x00005640870eafa6 in compat_epoll_wait (events=0x7f74c4b8e240, timeout=-1, interruptible=false) at compat/poll.cpp:280 #2 0x000056408705f2b6 in thread_rotation (data=0x56408765e280) at rotation-thread.cpp:804 #3 0x0000564087062b1a in launch_thread (data=0x56408765e310) at thread.cpp:66 #4 0x00007f74c9417609 in start_thread (arg=<optimized out>) at pthread_create.c:477 #5 0x00007f74c933a163 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 Thread 6 (Thread 0x7f74c5390700 (LWP 76040)): #0 0x00007f74c925f1d2 in __GI___sigtimedwait (set=0x7f74c538f090, info=0x7f74c538f110, timeout=0x0) at ../sysdeps/unix/sysv/linux/sigtimedwait.c:29 #1 0x000056408706138a in thread_timer (data=0x7ffc1fcbe3f0) at timer.cpp:359 #2 0x0000564087062b1a in launch_thread (data=0x56408765e0a0) at thread.cpp:66 #3 0x00007f74c9417609 in start_thread (arg=<optimized out>) at pthread_create.c:477 #4 0x00007f74c933a163 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 Thread 5 (Thread 0x7f74c5b91700 (LWP 76039)): #0 __libc_write (nbytes=8, buf=0x7f74c5b8fc88, fd=24) at ../sysdeps/unix/sysv/linux/write.c:26 #1 __libc_write (fd=24, buf=0x7f74c5b8fc88, nbytes=8) at ../sysdeps/unix/sysv/linux/write.c:24 #2 0x00005640870eeb4f in lttng_write (fd=24, buf=0x7f74c5b8fc88, count=8) at readwrite.cpp:77 #3 0x000056408704b535 in run_command_no_wait (handle=0x56408765ddd0, in_cmd=0x7f74c5b8fdf0) at notification-thread-commands.cpp:92 #4 0x000056408704bf49 in notification_thread_client_communication_update (handle=0x56408765ddd0, id=2, transmission_status=CLIENT_TRANSMISSION_STATUS_QUEUED) at notification-thread-command #5 0x000056408707bc62 in client_handle_transmission_status (client=0x7f74b80050d0, status=CLIENT_TRANSMISSION_STATUS_QUEUED, user_data=0x7f74b8004410) at action-executor.cpp:258 #6 0x0000564087057525 in notification_client_list_send_evaluation (client_list=0x7f74b8004df0, trigger=0x7f74b0001030, evaluation=0x7f74b815d1d0, source_object_creds=0x0, client_report=0x5 #7 0x000056408707bce9 in action_executor_notify_handler (executor=0x7f74b8004410, work_item=0x7f74b815d430, item=0x7f74b80e48e0) at action-executor.cpp:269 #8 0x000056408707dd6d in action_executor_generic_handler (executor=0x7f74b8004410, work_item=0x7f74b815d430, item=0x7f74b80e48e0) at action-executor.cpp:670 lttng#9 0x000056408707df01 in action_work_item_execute (executor=0x7f74b8004410, work_item=0x7f74b815d430) at action-executor.cpp:689 lttng#10 0x000056408707e525 in action_executor_thread (_data=0x7f74b8004410) at action-executor.cpp:771 [698/709] lttng#11 0x0000564087062b1a in launch_thread (data=0x7f74b80044b0) at thread.cpp:66 lttng#12 0x00007f74c9417609 in start_thread (arg=<optimized out>) at pthread_create.c:477 lttng#13 0x00007f74c933a163 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 Thread 4 (Thread 0x7f74c6392700 (LWP 76038)): #0 __lll_lock_wait (futex=futex@entry=0x56408765dde8, private=0) at lowlevellock.c:52 #1 0x00007f74c941a0a3 in __GI___pthread_mutex_lock (mutex=0x56408765dde8) at ../nptl/pthread_mutex_lock.c:80 #2 0x0000564087053c89 in handle_notification_thread_command (handle=0x56408765ddd0, state=0x7f74c63911b0) at notification-thread-events.cpp:3142 #3 0x000056408704ac81 in thread_notification (data=0x56408765ddd0) at notification-thread.cpp:715 #4 0x0000564087062b1a in launch_thread (data=0x56408765dec0) at thread.cpp:66 #5 0x00007f74c9417609 in start_thread (arg=<optimized out>) at pthread_create.c:477 #run_command_no_wait6 0x00007f74c933a163 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 Thread 3 (Thread 0x7f74c6b93700 (LWP 76037)): #0 0x00007f74c933a49e in epoll_wait (epfd=21, events=0x7f74c0000b60, maxevents=2, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30 #1 0x00005640870eafa6 in compat_epoll_wait (events=0x7f74c6b92170, timeout=-1, interruptible=false) at compat/poll.cpp:280 #2 0x000056408706400a in thread_manage_health (data=0x56408765db50) at health.cpp:140 #3 0x0000564087062b1a in launch_thread (data=0x56408765dbf0) at thread.cpp:66 #4 0x00007f74c9417609 in start_thread (arg=<optimized out>) at pthread_create.c:477 #5 0x00007f74c933a163 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 Thread 2 (Thread 0x7f74c7394700 (LWP 76036)): #0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38 #1 0x00007f74c987d238 in futex (uaddr=0x564087659b10, op=0, val=-1, timeout=0x0, uaddr2=0x0, val3=0) at ../include/urcu/futex.h:72 #2 futex_async (uaddr=0x564087659b10, op=0, val=-1, timeout=0x0, uaddr2=0x0, val3=0) at ../include/urcu/futex.h:104 #3 futex_wait (futex=0x564087659b10) at workqueue.c:136 #4 0x00007f74c987ced2 in workqueue_thread (arg=0x564087659ad0) at workqueue.c:237 #5 0x00007f74c9417609 in start_thread (arg=<optimized out>) at pthread_create.c:477 #6 0x00007f74c933a163 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 Thread 1 (Thread 0x7f74c73cd300 (LWP 76034)): #0 0x00007f74c933a49e in epoll_wait (epfd=50, events=0x564087666880, maxevents=1, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30 #1 0x00005640870eafa6 in compat_epoll_wait (events=0x7ffc1fcbe280, timeout=-1, interruptible=false) at compat/poll.cpp:280 #2 0x0000564087062244 in sessiond_wait_for_quit_pipe (timeout_ms=-1) at thread-utils.cpp:83 #3 0x00005640870127dc in main (argc=1, argv=0x7ffc1fcbe668) at main.cpp:1921 Cause ===== The event_pipe used to notify the notification poll loop is full and the lttng_write call blocks with the locks for both the client and the cmd_queue held. Solution ======== Go back to using eventfd but without the use of EFD_SEMAPHORE (linux 2.6.30) to continue supporting kernel between 2.6.27 and 2.6.29. The EFD_SEMAPHORE is emulated with a read, decrement, write as explained by the initial committer of EFD_SEMAPHORE [1]. Known drawbacks ========= This does not solve the actual block+lock problem but simply push it back further. The lttng_write on the eventfd can block when reaching UINT64_MAX. This would represent, at 1 command queued per ns (which is ridiculous), ~584 years of queueing without a dequeue operation. Reference ======= [1] https://lwn.net/Articles/318151/ Signed-off-by: Jonathan Rajotte <[email protected]> Signed-off-by: Jérémie Galarneau <[email protected]> Change-Id: Ie749c4169708f57463fe3cfab2366f1015bae4e0
jgalar
added a commit
that referenced
this pull request
May 25, 2022
Issue observed ============== Address sanitizer reports the following invalid accesses while running the test_mi test. ❯ ASAN_OPTIONS=detect_odr_violation=0 lttng-sessiond ================================================================= ==289173==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60400000e280 at pc 0x55cbbe35e2e0 bp 0x7f01672f1550 sp 0x7f01672f1540 WRITE of size 4 at 0x60400000e280 thread T13 #0 0x55cbbe35e2df in mark_thread_as_ready /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/manage-consumer.cpp:32 #1 0x55cbbe360160 in thread_consumer_management /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/manage-consumer.cpp:267 #2 0x55cbbe336ac4 in launch_thread /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/thread.cpp:66 #3 0x7f01729c15c1 in start_thread (/usr/lib/libc.so.6+0x8d5c1) #4 0x7f0172a46583 in __clone (/usr/lib/libc.so.6+0x112583) 0x60400000e280 is located 8 bytes to the right of 40-byte region [0x60400000e250,0x60400000e278) allocated by thread T7 here: #0 0x7f01733b1fb9 in __interceptor_calloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:154 #1 0x55cbbe33adf3 in zmalloc_internal ../../../src/common/macros.hpp:60 #2 0x55cbbe33ae03 in thread_notifiers* zmalloc<thread_notifiers>() ../../../src/common/macros.hpp:89 #3 0x55cbbe3617f9 in launch_consumer_management_thread(consumer_data*) /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/manage-consumer.cpp:440 #4 0x55cbbe33cf49 in spawn_consumer_thread /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/client.cpp:188 #5 0x55cbbe33f7cf in start_consumerd /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/client.cpp:394 #6 0x55cbbe345713 in process_client_msg /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/client.cpp:1277 #7 0x55cbbe34d74b in thread_manage_clients /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/client.cpp:2622 #8 0x55cbbe336ac4 in launch_thread /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/thread.cpp:66 lttng#9 0x7f01729c15c1 in start_thread (/usr/lib/libc.so.6+0x8d5c1) Thread T13 created by T7 here: #0 0x7f0173353eb7 in __interceptor_pthread_create /usr/src/debug/gcc/libsanitizer/asan/asan_interceptors.cpp:216 #1 0x55cbbe336f9e in lttng_thread_create(char const*, void* (*)(void*), bool (*)(void*), void (*)(void*), void*) /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/thread.cpp:106 #2 0x55cbbe3618cc in launch_consumer_management_thread(consumer_data*) /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/manage-consumer.cpp:453 #3 0x55cbbe33cf49 in spawn_consumer_thread /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/client.cpp:188 #4 0x55cbbe33f7cf in start_consumerd /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/client.cpp:394 #5 0x55cbbe345713 in process_client_msg /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/client.cpp:1277 #6 0x55cbbe34d74b in thread_manage_clients /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/client.cpp:2622 #7 0x55cbbe336ac4 in launch_thread /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/thread.cpp:66 #8 0x7f01729c15c1 in start_thread (/usr/lib/libc.so.6+0x8d5c1) Thread T7 created by T0 here: #0 0x7f0173353eb7 in __interceptor_pthread_create /usr/src/debug/gcc/libsanitizer/asan/asan_interceptors.cpp:216 #1 0x55cbbe336f9e in lttng_thread_create(char const*, void* (*)(void*), bool (*)(void*), void (*)(void*), void*) /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/thread.cpp:106 #2 0x55cbbe34eebf in launch_client_thread() /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/client.cpp:2756 #3 0x55cbbe27f31a in main /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/main.cpp:1838 #4 0x7f017296130f in __libc_start_call_main (/usr/lib/libc.so.6+0x2d30f) SUMMARY: AddressSanitizer: heap-buffer-overflow /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/manage-consumer.cpp:32 in mark_thread_as_ready Shadow bytes around the buggy address: 0x0c087fff9c00: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fa 0x0c087fff9c10: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fa 0x0c087fff9c20: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fa 0x0c087fff9c30: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fa 0x0c087fff9c40: fa fa fd fd fd fd fd fa fa fa 00 00 00 00 00 fa =>0x0c087fff9c50:[fa]fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c087fff9c60: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c087fff9c70: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c087fff9c80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c087fff9c90: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c087fff9ca0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb Shadow gap: cc ==289173==ABORTING Cause ===== The start functions of the various worker threads of the session daemon are implemented in separate translation units (TU). To make use of the lttng_thread API, they all define different control structures to control their shutdown. Those structures are all named 'thread_notifiers' and are all allocated using zmalloc<>. The various instances of zmalloc<thread_notifiers> all end up having the same mangled name (e.g. _Z7zmallocI16thread_notifiersEPT_v). At link time, only one instance of zmalloc<thread_notifiers> is kept. Since those structures all have different layout/sizes, this is problematic. However, it is an acceptable behaviour according to the ODR [1]. I first considered making the various memory allocation functions in macros.hpp 'static' which results in each TU holding the appropriate specialization of the various functions. While this works, it doesn't make us ODR-compliant. To make a long story short, a program defining multiple types sharing the same name, in the same namespace, is ill-formed. Another concern is that marking all templated free-functions as static will eventually result in code bloat. Solution ======== All structures defined in TUs (but not in a header) are placed in unnamed namespaces (also called anonymous namespaces) [2]. This results in separate copies of the templated functions being generated when specialized using a structure in an anonymous namespace (e.g. _Z7zmallocIN12_GLOBAL__N_116thread_notifiersEEPT_v). We could have renamed the various `thread_notifiers` structures to give them different names. However, I found those are not the only structures sharing a name in different TUs. For instance, the same problem applies to `struct lttng_index` (index in a stream, index in a map). I propose we systematically namespace structures defined in TUs in the future. This will also save us trouble if those POD structures eventually become non-POD: we would experience the same "clashes" if those structures had constructors, for example. References ========== [1] https://en.cppreference.com/w/cpp/language/definition [2] https://en.cppreference.com/w/cpp/language/namespace Signed-off-by: Jérémie Galarneau <[email protected]> Change-Id: I867e5a287ad8cf3ada617335bc1a80b800bf0833
jgalar
added a commit
that referenced
this pull request
May 25, 2022
LeakSanitizer reports the following leak: ==974957==ERROR: LeakSanitizer: detected memory leaks Direct leak of 32 byte(s) in 1 object(s) allocated from: #0 0x7fdb86fcd1b2 in __interceptor_realloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:164 #1 0x7fdb86d7c296 in lttng_dynamic_buffer_set_capacity(lttng_dynamic_buffer*, unsigned long) /home/jgalar/EfficiOS/src/lttng-tools/src/common/dynamic-buffer.cpp:159 #2 0x7fdb86d7c060 in lttng_dynamic_buffer_set_size(lttng_dynamic_buffer*, unsigned long) /home/jgalar/EfficiOS/src/lttng-tools/src/common/dynamic-buffer.cpp:112 #3 0x7fdb86d2589a in recv_payload_sessiond /home/jgalar/EfficiOS/src/lttng-tools/src/lib/lttng-ctl/lttng-ctl.cpp:230 #4 0x7fdb86d26fa5 in lttng_ctl_ask_sessiond_payload(lttng_payload_view*, lttng_payload*) /home/jgalar/EfficiOS/src/lttng-tools/src/lib/lttng-ctl/lttng-ctl.cpp:662 #5 0x7fdb86d2cd8d in lttng_list_tracepoint_fields /home/jgalar/EfficiOS/src/lttng-tools/src/lib/lttng-ctl/lttng-ctl.cpp:1767 #6 0x56481623cb4c in list_ust_event_fields commands/list.cpp:850 #7 0x5648162448d9 in cmd_list(int, char const**) commands/list.cpp:2394 #8 0x56481628fb3e in handle_command /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng/lttng.cpp:238 lttng#9 0x564816290601 in parse_args /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng/lttng.cpp:427 lttng#10 0x564816290908 in main /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng/lttng.cpp:476 lttng#11 0x7fdb8661730f in __libc_start_call_main (/usr/lib/libc.so.6+0x2d30f) SUMMARY: AddressSanitizer: 32 byte(s) leaked in 1 allocation(s). The session daemon's reply is indeed never released in lttng_list_tracepoint_fields. Signed-off-by: Jérémie Galarneau <[email protected]> Change-Id: Idd244b52a69f3b74e5c131c1c36c6ee6d76f4285
jgalar
added a commit
that referenced
this pull request
May 25, 2022
==1175545==ERROR: LeakSanitizer: detected memory leaks Direct leak of 8696 byte(s) in 1 object(s) allocated from: #0 0x7efed0f39fb9 in __interceptor_calloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:154 #1 0x55707ddc6004 in zmalloc_internal ../../../src/common/macros.hpp:60 #2 0x55707ddceb17 in ltt_ust_session* zmalloc<ltt_ust_session>() ../../../src/common/macros.hpp:89 #3 0x55707ddc81e7 in trace_ust_create_session(unsigned long) /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/trace-ust.cpp:274 #4 0x55707ddc2bea in test_create_one_ust_session /home/jgalar/EfficiOS/src/lttng-tools/tests/unit/test_ust_data.cpp:63 #5 0x55707ddc4941 in main /home/jgalar/EfficiOS/src/lttng-tools/tests/unit/test_ust_data.cpp:283 #6 0x7efed04f930f in __libc_start_call_main (/usr/lib/libc.so.6+0x2d30f) Indirect leak of 24672 byte(s) in 1 object(s) allocated from: #0 0x7efed0f39fb9 in __interceptor_calloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:154 #1 0x55707dee4ec1 in zmalloc_internal ../../../src/common/macros.hpp:60 #2 0x55707def774e in consumer_output* zmalloc<consumer_output>() ../../../src/common/macros.hpp:89 #3 0x55707dee90df in consumer_create_output(consumer_dst_type) /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/consumer.cpp:523 #4 0x55707ddc8821 in trace_ust_create_session(unsigned long) /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/trace-ust.cpp:321 #5 0x55707ddc2bea in test_create_one_ust_session /home/jgalar/EfficiOS/src/lttng-tools/tests/unit/test_ust_data.cpp:63 #6 0x55707ddc4941 in main /home/jgalar/EfficiOS/src/lttng-tools/tests/unit/test_ust_data.cpp:283 #7 0x7efed04f930f in __libc_start_call_main (/usr/lib/libc.so.6+0x2d30f) Indirect leak of 1024 byte(s) in 1 object(s) allocated from: #0 0x7efed0f39fb9 in __interceptor_calloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:154 #1 0x7efed0bf985f in alloc_split_items_count /home/jgalar/EfficiOS/src/userspace-rcu/src/rculfhash.c:688 #2 0x7efed0bf985f in _cds_lfht_new /home/jgalar/EfficiOS/src/userspace-rcu/src/rculfhash.c:1642 Indirect leak of 656 byte(s) in 1 object(s) allocated from: #0 0x7efed0f39fb9 in __interceptor_calloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:154 #1 0x7efed0bfac68 in __default_alloc_cds_lfht ../src/rculfhash-internal.h:172 #2 0x7efed0bfac68 in alloc_cds_lfht /home/jgalar/EfficiOS/src/userspace-rcu/src/rculfhash-mm-order.c:81 Indirect leak of 48 byte(s) in 2 object(s) allocated from: #0 0x7efed0f39fb9 in __interceptor_calloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:154 #1 0x7efed0bfabd4 in cds_lfht_alloc_bucket_table /home/jgalar/EfficiOS/src/userspace-rcu/src/rculfhash-mm-order.c:35 #2 0x7efed0bfabd4 in cds_lfht_alloc_bucket_table /home/jgalar/EfficiOS/src/userspace-rcu/src/rculfhash-mm-order.c:28 Indirect leak of 24 byte(s) in 1 object(s) allocated from: #0 0x7efed0f39fb9 in __interceptor_calloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:154 #1 0x55707de3a9af in zmalloc_internal ../../src/common/macros.hpp:60 #2 0x55707de3a9bf in lttng_ht* zmalloc<lttng_ht>() ../../src/common/macros.hpp:89 #3 0x55707de38461 in lttng_ht_new(unsigned long, lttng_ht_type) hashtable/hashtable.cpp:113 #4 0x55707dee9340 in consumer_create_output(consumer_dst_type) /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/consumer.cpp:535 #5 0x55707ddc8821 in trace_ust_create_session(unsigned long) /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/trace-ust.cpp:321 #6 0x55707ddc2bea in test_create_one_ust_session /home/jgalar/EfficiOS/src/lttng-tools/tests/unit/test_ust_data.cpp:63 #7 0x55707ddc4941 in main /home/jgalar/EfficiOS/src/lttng-tools/tests/unit/test_ust_data.cpp:283 #8 0x7efed04f930f in __libc_start_call_main (/usr/lib/libc.so.6+0x2d30f) Indirect leak of 16 byte(s) in 1 object(s) allocated from: #0 0x7efed0f39fb9 in __interceptor_calloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:154 #1 0x7efed0bfac15 in cds_lfht_alloc_bucket_table /home/jgalar/EfficiOS/src/userspace-rcu/src/rculfhash-mm-order.c:31 Signed-off-by: Jérémie Galarneau <[email protected]> Change-Id: Ib2ad82a197f2a4ccb86ae5799c1d93ff059888e3
jgalar
added a commit
that referenced
this pull request
May 25, 2022
==1190137==ERROR: LeakSanitizer: detected memory leaks Direct leak of 8 byte(s) in 1 object(s) allocated from: #0 0x7f40a9d4c1b2 in __interceptor_realloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:164 #1 0x55ab716e1def in lttng_dynamic_buffer_set_capacity(lttng_dynamic_buffer*, unsigned long) /home/jgalar/EfficiOS/src/lttng-tools/src/common/dynamic-buffer.cpp:159 #2 0x55ab716e1956 in lttng_dynamic_buffer_append(lttng_dynamic_buffer*, void const*, unsigned long) /home/jgalar/EfficiOS/src/lttng-tools/src/common/dynamic-buffer.cpp:52 #3 0x55ab716ca64e in lttng_log_level_rule_serialize(lttng_log_level_rule const*, lttng_payload*) /home/jgalar/EfficiOS/src/lttng-tools/src/common/log-level-rule.cpp:177 #4 0x55ab716c760f in test_log_level_rule_serialize_deserialize /home/jgalar/EfficiOS/src/lttng-tools/tests/unit/test_log_level_rule.cpp:60 #5 0x55ab716c8457 in test_log_level_rule_at_least_as_severe_as /home/jgalar/EfficiOS/src/lttng-tools/tests/unit/test_log_level_rule.cpp:177 #6 0x55ab716c84d3 in main /home/jgalar/EfficiOS/src/lttng-tools/tests/unit/test_log_level_rule.cpp:185 #7 0x7f40a938830f in __libc_start_call_main (/usr/lib/libc.so.6+0x2d30f) Direct leak of 8 byte(s) in 1 object(s) allocated from: #0 0x7f40a9d4c1b2 in __interceptor_realloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:164 #1 0x55ab716e1def in lttng_dynamic_buffer_set_capacity(lttng_dynamic_buffer*, unsigned long) /home/jgalar/EfficiOS/src/lttng-tools/src/common/dynamic-buffer.cpp:159 #2 0x55ab716e1956 in lttng_dynamic_buffer_append(lttng_dynamic_buffer*, void const*, unsigned long) /home/jgalar/EfficiOS/src/lttng-tools/src/common/dynamic-buffer.cpp:52 #3 0x55ab716ca64e in lttng_log_level_rule_serialize(lttng_log_level_rule const*, lttng_payload*) /home/jgalar/EfficiOS/src/lttng-tools/src/common/log-level-rule.cpp:177 #4 0x55ab716c760f in test_log_level_rule_serialize_deserialize /home/jgalar/EfficiOS/src/lttng-tools/tests/unit/test_log_level_rule.cpp:60 #5 0x55ab716c8135 in test_log_level_rule_exactly /home/jgalar/EfficiOS/src/lttng-tools/tests/unit/test_log_level_rule.cpp:154 #6 0x55ab716c84ce in main /home/jgalar/EfficiOS/src/lttng-tools/tests/unit/test_log_level_rule.cpp:184 #7 0x7f40a938830f in __libc_start_call_main (/usr/lib/libc.so.6+0x2d30f) Signed-off-by: Jérémie Galarneau <[email protected]> Change-Id: I2d1eafabbd5c101c188bad8a2137615b29c0ef68
jgalar
added a commit
that referenced
this pull request
May 25, 2022
==1198508==ERROR: LeakSanitizer: detected memory leaks Direct leak of 56 byte(s) in 1 object(s) allocated from: #0 0x7f8b62634fb9 in __interceptor_calloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:154 #1 0x557871869adb in zmalloc_internal ../../src/common/macros.hpp:60 #2 0x55787186c8a0 in zmalloc<(anonymous namespace)::lttng_rate_policy_once_after_n> ../../src/common/macros.hpp:89 #3 0x55787186c173 in lttng_rate_policy_once_after_n_create actions/rate-policy.cpp:707 #4 0x55787186a368 in lttng_rate_policy_once_after_n_create_from_payload actions/rate-policy.cpp:183 #5 0x55787186ad02 in lttng_rate_policy_create_from_payload(lttng_payload_view*, lttng_rate_policy**) actions/rate-policy.cpp:287 #6 0x557871865b5b in test_rate_policy_once_after_n /home/jgalar/EfficiOS/src/lttng-tools/tests/unit/test_rate_policy.cpp:231 #7 0x557871865dc9 in main /home/jgalar/EfficiOS/src/lttng-tools/tests/unit/test_rate_policy.cpp:250 #8 0x7f8b61c7130f in __libc_start_call_main (/usr/lib/libc.so.6+0x2d30f) Direct leak of 56 byte(s) in 1 object(s) allocated from: #0 0x7f8b62634fb9 in __interceptor_calloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:154 #1 0x557871869adb in zmalloc_internal ../../src/common/macros.hpp:60 #2 0x55787186c890 in zmalloc<(anonymous namespace)::lttng_rate_policy_every_n> ../../src/common/macros.hpp:89 #3 0x55787186b6cd in lttng_rate_policy_every_n_create actions/rate-policy.cpp:492 #4 0x55787186a699 in lttng_rate_policy_every_n_create_from_payload actions/rate-policy.cpp:220 #5 0x55787186ad02 in lttng_rate_policy_create_from_payload(lttng_payload_view*, lttng_rate_policy**) actions/rate-policy.cpp:287 #6 0x557871864cae in test_rate_policy_every_n /home/jgalar/EfficiOS/src/lttng-tools/tests/unit/test_rate_policy.cpp:122 #7 0x557871865dc4 in main /home/jgalar/EfficiOS/src/lttng-tools/tests/unit/test_rate_policy.cpp:249 #8 0x7f8b61c7130f in __libc_start_call_main (/usr/lib/libc.so.6+0x2d30f) SUMMARY: AddressSanitizer: 112 byte(s) leaked in 2 allocation(s). Signed-off-by: Jérémie Galarneau <[email protected]> Change-Id: I3a9b4d99e93f355ddb8623a289f8397907486ab0
jgalar
added a commit
that referenced
this pull request
May 25, 2022
==1245463==ERROR: LeakSanitizer: detected memory leaks Direct leak of 5 byte(s) in 1 object(s) allocated from: #0 0x7fe7c494fdd9 in __interceptor_malloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:145 #1 0x7fe7c44a5c09 (/usr/lib/libpopt.so.0+0x3c09) Arguments obtained with poptGetOptArg() must be free'd. Signed-off-by: Jérémie Galarneau <[email protected]> Change-Id: I5a65ca6fbaa18f7717ea918a5bc7f42daeb1009a
jgalar
added a commit
that referenced
this pull request
May 25, 2022
==1429021==ERROR: LeakSanitizer: detected memory leaks Direct leak of 8 byte(s) in 1 object(s) allocated from: #0 0x7fe305f031b2 in __interceptor_realloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:164 #1 0x559f1b022238 in lttng_dynamic_buffer_set_capacity(lttng_dynamic_buffer*, unsigned long) /home/jgalar/EfficiOS/src/lttng-tools/src/common/dynamic-buffer.cpp:159 #2 0x559f1b021d9f in lttng_dynamic_buffer_append(lttng_dynamic_buffer*, void const*, unsigned long) /home/jgalar/EfficiOS/src/lttng-tools/src/common/dynamic-buffer.cpp:52 #3 0x559f1b02144a in lttng_dynamic_array_add_element(lttng_dynamic_array*, void const*) /home/jgalar/EfficiOS/src/lttng-tools/src/common/dynamic-array.cpp:58 #4 0x559f1b07d07b in lttng_action_path_copy(lttng_action_path const*, lttng_action_path*) actions/path.cpp:116 #5 0x559f1b02383f in lttng_error_query_action_create /home/jgalar/EfficiOS/src/lttng-tools/src/common/error-query.cpp:232 #6 0x559f1b02760e in lttng_error_query_create_from_payload(lttng_payload_view*, lttng_error_query**) /home/jgalar/EfficiOS/src/lttng-tools/src/common/error-query.cpp:911 #7 0x559f1af5c361 in receive_lttng_error_query /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/client.cpp:740 #8 0x559f1af64eba in process_client_msg /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/client.cpp:2336 lttng#9 0x559f1af67378 in thread_manage_clients /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/client.cpp:2624 lttng#10 0x559f1af50642 in launch_thread /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/thread.cpp:68 lttng#11 0x7fe3055225c1 in start_thread (/usr/lib/libc.so.6+0x8d5c1) Signed-off-by: Jérémie Galarneau <[email protected]> Change-Id: I7a6f7d2a9746124581eebf30877466f16db67a6b
jgalar
added a commit
that referenced
this pull request
May 25, 2022
==1480456==ERROR: LeakSanitizer: detected memory leaks Direct leak of 112 byte(s) in 1 object(s) allocated from: #0 0x7fdb9260cfb9 in __interceptor_calloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:154 #1 0x7fdb9242348d in zmalloc_internal ../../src/common/macros.hpp:60 #2 0x7fdb924295a9 in lttng_trigger* zmalloc<lttng_trigger>() ../../src/common/macros.hpp:89 #3 0x7fdb92423dbe in lttng_trigger_create /home/jgalar/EfficiOS/src/lttng-tools/src/common/trigger.cpp:58 #4 0x56304832331f in register_trigger /home/jgalar/EfficiOS/src/lttng-tools/tests/regression/tools/trigger/utils/register-some-triggers.cpp:24 #5 0x5630483233f1 in register_trigger_action_list_notify /home/jgalar/EfficiOS/src/lttng-tools/tests/regression/tools/trigger/utils/register-some-triggers.cpp:46 #6 0x5630483239a0 in test_session_rotation_conditions /home/jgalar/EfficiOS/src/lttng-tools/tests/regression/tools/trigger/utils/register-some-triggers.cpp:246 #7 0x563048323d4d in main /home/jgalar/EfficiOS/src/lttng-tools/tests/regression/tools/trigger/utils/register-some-triggers.cpp:309 #8 0x7fdb91c6630f in __libc_start_call_main (/usr/lib/libc.so.6+0x2d30f) Signed-off-by: Jérémie Galarneau <[email protected]> Change-Id: Ie163989a70f65f9c2c4e93c36cc9fc6ba6bdeeb5
jgalar
added a commit
that referenced
this pull request
May 25, 2022
==1501334==ERROR: LeakSanitizer: detected memory leaks Indirect leak of 16386 byte(s) in 1 object(s) allocated from: #0 0x7f95efc3cdd9 in __interceptor_malloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:145 #1 0x55acb0681ed3 in lttng_filter_yyalloc(unsigned long, void*) filter/filter-lexer.cpp:2511 #2 0x55acb067f2f2 in lttng_filter_yy_create_buffer(_IO_FILE*, int, void*) filter/filter-lexer.cpp:1895 #3 0x55acb067ea44 in yyrestart(_IO_FILE*, void*) filter/filter-lexer.cpp:1824 #4 0x55acb0649a43 in filter_parser_ctx_alloc(_IO_FILE*) filter/filter-parser.ypp:271 #5 0x55acb0649e7f in filter_parser_ctx_create_from_filter_expression(char const*, filter_parser_ctx**) filter/filter-parser.ypp:332 #6 0x55acb058ee89 in parse_event_rule commands/add_trigger.cpp:783 #7 0x55acb05920c0 in handle_condition_event commands/add_trigger.cpp:1361 #8 0x55acb0592739 in parse_condition commands/add_trigger.cpp:1457 lttng#9 0x55acb0596b56 in cmd_add_trigger(int, char const**) commands/add_trigger.cpp:2304 lttng#10 0x55acb05a5b80 in handle_command /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng/lttng.cpp:238 lttng#11 0x55acb05a6643 in parse_args /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng/lttng.cpp:427 lttng#12 0x55acb05a694a in main /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng/lttng.cpp:476 lttng#13 0x7f95ef28730f in __libc_start_call_main (/usr/lib/libc.so.6+0x2d30f) Signed-off-by: Jérémie Galarneau <[email protected]> Change-Id: I6fa21e7d066e0cf48afc3f91ceefbfd19c6b86fd
jgalar
added a commit
that referenced
this pull request
May 25, 2022
==1676099==ERROR: LeakSanitizer: detected memory leaks Direct leak of 5 byte(s) in 1 object(s) allocated from: #0 0x7f19429d9dd9 in __interceptor_malloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:145 #1 0x7f19425342ad in poptGetNextOpt (/usr/lib/libpopt.so.0+0x82ad) Signed-off-by: Jérémie Galarneau <[email protected]> Change-Id: Ibafcaf42ad4f842b3fa74cf91dc5ecc8acb3487d
jgalar
added a commit
that referenced
this pull request
May 25, 2022
==1759491==ERROR: LeakSanitizer: detected memory leaks Direct leak of 6 byte(s) in 1 object(s) allocated from: #0 0x7fdbdc94add9 in __interceptor_malloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:145 #1 0x7fdbdc4a0c09 (/usr/lib/libpopt.so.0+0x3c09) Signed-off-by: Jérémie Galarneau <[email protected]> Change-Id: I29cc6ec4390e71829107f309f162247b9be2868c
jgalar
added a commit
that referenced
this pull request
May 25, 2022
==1769573==ERROR: LeakSanitizer: detected memory leaks Direct leak of 24 byte(s) in 1 object(s) allocated from: #0 0x7fef37a29fb9 in __interceptor_calloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:154 #1 0x7fef37792f2f in zmalloc_internal ../../../src/common/macros.hpp:60 #2 0x7fef3779573a in lttng_rotation_schedules* zmalloc<lttng_rotation_schedules>() ../../../src/common/macros.hpp:89 #3 0x7fef377947cc in lttng_rotation_schedules_create /home/jgalar/EfficiOS/src/lttng-tools/src/lib/lttng-ctl/rotate.cpp:353 #4 0x7fef37794aa0 in get_schedules /home/jgalar/EfficiOS/src/lttng-tools/src/lib/lttng-ctl/rotate.cpp:392 #5 0x7fef377956dc in lttng_session_list_rotation_schedules /home/jgalar/EfficiOS/src/lttng-tools/src/lib/lttng-ctl/rotate.cpp:665 #6 0x5646131713f2 in test_add_list_remove_schedule /home/jgalar/EfficiOS/src/lttng-tools/tests/regression/tools/rotation/schedule_api.c:252 #7 0x56461317157b in test_add_list_remove_size_schedule /home/jgalar/EfficiOS/src/lttng-tools/tests/regression/tools/rotation/schedule_api.c:270 #8 0x564613171680 in main /home/jgalar/EfficiOS/src/lttng-tools/tests/regression/tools/rotation/schedule_api.c:307 lttng#9 0x7fef373ae30f in __libc_start_call_main (/usr/lib/libc.so.6+0x2d30f) Signed-off-by: Jérémie Galarneau <[email protected]> Change-Id: I9b7eb537d158791db76f9a7676ffeb5d4a1f2203
jgalar
added a commit
that referenced
this pull request
May 25, 2022
==1801304==ERROR: LeakSanitizer: detected memory leaks Direct leak of 224 byte(s) in 2 object(s) allocated from: #0 0x7fe0f4e73fb9 in __interceptor_calloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:154 #1 0x559fbeb64175 in zmalloc_internal ../../src/common/macros.hpp:60 #2 0x559fbeb6a291 in lttng_trigger* zmalloc<lttng_trigger>() ../../src/common/macros.hpp:89 #3 0x559fbeb64aa6 in lttng_trigger_create /home/jgalar/EfficiOS/src/lttng-tools/src/common/trigger.cpp:58 #4 0x559fbe9dc417 in subscribe_session_consumed_size_rotation(ltt_session*, unsigned long, notification_thread_handle*) /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/rotate.cpp:87 #5 0x559fbe995d6f in cmd_rotation_set_schedule(ltt_session*, bool, lttng_rotation_schedule_type, unsigned long, notification_thread_handle*) /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/cmd.cpp:5993 #6 0x559fbe9fe559 in process_client_msg /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/client.cpp:2246 #7 0x559fbea01378 in thread_manage_clients /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/client.cpp:2624 #8 0x559fbe9ea642 in launch_thread /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/thread.cpp:68 lttng#9 0x7fe0f44935c1 in start_thread (/usr/lib/libc.so.6+0x8d5c1) Indirect leak of 208 byte(s) in 2 object(s) allocated from: #0 0x7fe0f4e73fb9 in __interceptor_calloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:154 #1 0x559fbeb16e21 in zmalloc_internal ../../src/common/macros.hpp:60 #2 0x559fbeb16e31 in lttng_action_notify* zmalloc<lttng_action_notify>() ../../src/common/macros.hpp:89 #3 0x559fbeb168a0 in lttng_action_notify_create actions/notify.cpp:135 #4 0x559fbe9dc34b in subscribe_session_consumed_size_rotation(ltt_session*, unsigned long, notification_thread_handle*) /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/rotate.cpp:80 #5 0x559fbe995d6f in cmd_rotation_set_schedule(ltt_session*, bool, lttng_rotation_schedule_type, unsigned long, notification_thread_handle*) /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/cmd.cpp:5993 #6 0x559fbe9fe559 in process_client_msg /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/client.cpp:2246 #7 0x559fbea01378 in thread_manage_clients /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/client.cpp:2624 #8 0x559fbe9ea642 in launch_thread /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/thread.cpp:68 lttng#9 0x7fe0f44935c1 in start_thread (/usr/lib/libc.so.6+0x8d5c1) Indirect leak of 160 byte(s) in 2 object(s) allocated from: #0 0x7fe0f4e73fb9 in __interceptor_calloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:154 #1 0x559fbeb3d7a1 in zmalloc_internal ../../src/common/macros.hpp:60 #2 0x559fbeb3fa35 in lttng_condition_session_consumed_size* zmalloc<lttng_condition_session_consumed_size>() ../../src/common/macros.hpp:89 #3 0x559fbeb3e6fd in lttng_condition_session_consumed_size_create conditions/session-consumed-size.cpp:206 #4 0x559fbe9dc0f1 in subscribe_session_consumed_size_rotation(ltt_session*, unsigned long, notification_thread_handle*) /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/rotate.cpp:54 #5 0x559fbe995d6f in cmd_rotation_set_schedule(ltt_session*, bool, lttng_rotation_schedule_type, unsigned long, notification_thread_handle*) /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/cmd.cpp:5993 #6 0x559fbe9fe559 in process_client_msg /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/client.cpp:2246 #7 0x559fbea01378 in thread_manage_clients /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/client.cpp:2624 #8 0x559fbe9ea642 in launch_thread /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/thread.cpp:68 lttng#9 0x7fe0f44935c1 in start_thread (/usr/lib/libc.so.6+0x8d5c1) Indirect leak of 112 byte(s) in 2 object(s) allocated from: #0 0x7fe0f4e73fb9 in __interceptor_calloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:154 #1 0x559fbeb242ad in zmalloc_internal ../../src/common/macros.hpp:60 #2 0x559fbeb27062 in zmalloc<(anonymous namespace)::lttng_rate_policy_every_n> ../../src/common/macros.hpp:89 #3 0x559fbeb25e9f in lttng_rate_policy_every_n_create actions/rate-policy.cpp:492 #4 0x559fbeb168b9 in lttng_action_notify_create actions/notify.cpp:141 #5 0x559fbe9dc34b in subscribe_session_consumed_size_rotation(ltt_session*, unsigned long, notification_thread_handle*) /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/rotate.cpp:80 #6 0x559fbe995d6f in cmd_rotation_set_schedule(ltt_session*, bool, lttng_rotation_schedule_type, unsigned long, notification_thread_handle*) /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/cmd.cpp:5993 #7 0x559fbe9fe559 in process_client_msg /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/client.cpp:2246 #8 0x559fbea01378 in thread_manage_clients /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/client.cpp:2624 lttng#9 0x559fbe9ea642 in launch_thread /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/thread.cpp:68 lttng#10 0x7fe0f44935c1 in start_thread (/usr/lib/libc.so.6+0x8d5c1) Indirect leak of 34 byte(s) in 2 object(s) allocated from: #0 0x7fe0f4e19319 in __interceptor_strdup /usr/src/debug/gcc/libsanitizer/asan/asan_interceptors.cpp:454 #1 0x559fbeb3f603 in lttng_condition_session_consumed_size_set_session_name conditions/session-consumed-size.cpp:442 #2 0x559fbe9dc2c4 in subscribe_session_consumed_size_rotation(ltt_session*, unsigned long, notification_thread_handle*) /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/rotate.cpp:71 #3 0x559fbe995d6f in cmd_rotation_set_schedule(ltt_session*, bool, lttng_rotation_schedule_type, unsigned long, notification_thread_handle*) /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/cmd.cpp:5993 #4 0x559fbe9fe559 in process_client_msg /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/client.cpp:2246 #5 0x559fbea01378 in thread_manage_clients /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/client.cpp:2624 #6 0x559fbe9ea642 in launch_thread /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng-sessiond/thread.cpp:68 #7 0x7fe0f44935c1 in start_thread (/usr/lib/libc.so.6+0x8d5c1) The rotation trigger of a session (used for size-based rotations) is never cleaned-up. It is now cleaned up every time its condition is hit and whenever the session is destroyed. Signed-off-by: Jérémie Galarneau <[email protected]> Change-Id: I5a89341535f87b7851b548ded9838c18bd1ccb95
jgalar
added a commit
that referenced
this pull request
May 25, 2022
==1853705==ERROR: LeakSanitizer: detected memory leaks Direct leak of 3 byte(s) in 1 object(s) allocated from: #0 0x7fb67ee0edd9 in __interceptor_malloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:145 #1 0x7fb67e964c09 (/usr/lib/libpopt.so.0+0x3c09) Signed-off-by: Jérémie Galarneau <[email protected]> Change-Id: I2f3346157cb26de6712c6e6ebd5fafa6b51fac08
jgalar
added a commit
that referenced
this pull request
May 25, 2022
==1920281==ERROR: LeakSanitizer: detected memory leaks Direct leak of 6 byte(s) in 1 object(s) allocated from: #0 0x7fa95633add9 in __interceptor_malloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:145 #1 0x7fa955e90c09 (/usr/lib/libpopt.so.0+0x3c09) Signed-off-by: Jérémie Galarneau <[email protected]> Change-Id: I87ce90a77d9624add0cab5d3090a7e83734da7f4
jgalar
pushed a commit
that referenced
this pull request
Apr 19, 2023
Observed issue ============== While a snapshot is being taken, the containing folder can disappear unexpectedly. This can lead to the following errors, which are expected and mostly handled fine: PERROR - 14:47:32.002564464 [2922498/2922507]: Failed to open file relative to trace chunk file_path = "channel0_0", flags = 577, mode = 432: No such file or directory (in _lttng_trace_chunk_open_fs_handle_locked() at trace-chunk.cpp:1411) Error: Failed to open stream file "channel0_0" Error: Snapshot channel failed The problem happens on the subsequent snapshot for the session: #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 #1 0x00007fbbdadb3859 in __GI_abort () at abort.c:79 #2 0x00007fbbdadb3729 in __assert_fail_base (fmt=0x7fbbdaf49588 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x55c4212cfbb5 "!stream->trace_chunk", file=0x55c4212cf820 "kernel-co #3 0x00007fbbdadc5006 in __GI___assert_fail (assertion=0x55c4212cfbb5 "!stream->trace_chunk", file=0x55c4212cf820 "kernel-consumer/kernel-consumer.cpp", line=188, function=0x55c4212cfb00 " #4 0x000055c421268cc6 in lttng_kconsumer_snapshot_channel (channel=0x7fbbc4000b60, key=1, path=0x7fbbd37f8fd4 "", relayd_id=18446744073709551615, nb_packets_per_stream=0) at kernel-consume #5 0x000055c42126b39d in lttng_kconsumer_recv_cmd (ctx=0x55c421b80a90, sock=31, consumer_sockpoll=0x7fbbd37fd280) at kernel-consumer/kernel-consumer.cpp:986 #6 0x000055c4212546d1 in lttng_consumer_recv_cmd (ctx=0x55c421b80a90, sock=31, consumer_sockpoll=0x7fbbd37fd280) at consumer/consumer.cpp:2090 #7 0x000055c421259963 in consumer_thread_sessiond_poll (data=0x55c421b80a90) at consumer/consumer.cpp:3281 #8 0x00007fbbdaf8b609 in start_thread (arg=<optimized out>) at pthread_create.c:477 lttng#9 0x00007fbbdaeb0163 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 How to reproduce: 1. Setting a breakpoint on snapshot_channel() inside src/common/ust-consumer/ust-consumer.cpp 2. When the breakpoint hits, remove the the complete lttng directory containing the session data. 3. Continue the lttng_consumerd process from gdb. 4. In that case you see a negative return value -1 from consumer_stream_create_output_files() inside snapshot_channel(). 5. Take another snapshot and lttng_consumerd crashes because of the `assert(!stream->trace_chunk)` in snapshot_channel(). This last action does not require any breakpoint intervention. Cause ===== During the snapshot, the stream is assigned the channel current chunk. It is expected that the stream does not have a chunk at this point. The error handling is faulty here, the stream chunk must be invalidated/reset on error to allow its reuse later on. The problem exists for both consumer domains (user/kernel). Solution ======== For the ust consumer, we can directly use the `error_close_stream` label. For the kernel consumer, the code path is slightly different since it does not uses `consumer_stream_close`. Note that `consumer_stream_close` cannot be used as is for the kernel consumer. The current implementation partially resembles `consumer_stream_close` at the end of the iteration. It is extracted to its own function for easier reuse from the new `error_finalize_stream` label. Known drawbacks ========= None. Fixes: #1352 Signed-off-by: Marcel Hamer <[email protected]> Signed-off-by: Jonathan Rajotte <[email protected]> Signed-off-by: Jérémie Galarneau <[email protected]> Change-Id: I9fc81917b19aa436ed8e8679672648f2d5baf41a
jgalar
added a commit
that referenced
this pull request
Apr 19, 2023
Issue observed -------------- When running in verbose mode, the session daemon calls abort() when it receives an unknown client command: #1 0x00007f66ffd69958 in raise () from /usr/lib/libc.so.6 #2 0x00007f66ffd5353d in abort () from /usr/lib/libc.so.6 #3 0x000055a671a6f6bd in lttcomm_sessiond_command_str (cmd=1633771873) at ../../../src/common/sessiond-comm/sessiond-comm.hpp:199 #4 0x000055a671a73897 in process_client_msg (cmd_ctx=0x7f66f5ff6d10, sock=0x7f66f5ff6c34, sock_error=0x7f66f5ff6c38) at client.cpp:1006 #5 0x000055a671a777fc in thread_manage_clients (data=0x55a673956100) at client.cpp:2622 #6 0x000055a671a6d290 in launch_thread (data=0x55a673956170) at thread.cpp:68 Cause ----- process_client_msg() logs the client command on entry. While it previously logged the numerical value, it now provides the string-ified version of the command id (since 19f912d). The lttcomm_sessiond_command_str() function aborts when it encounters an unknown command id. This is reasonable (in so far that it is how we handle these situations, typically). However, the validity of the command must be checked beforehand as it comes from an external (untrusted) source. Solution -------- Add lttcomm_sessiond_command_is_valid and tombstone command IDs to lttcomm_sessiond_command to ensure only valid command ids are passed to lttcomm_sessiond_command_str when logging. Drawbacks --------- None Reported-by: Michael Jeanson <[email protected]> Signed-off-by: Jérémie Galarneau <[email protected]> Change-Id: Ibd36f1e69da984c7f27b55ec68e5e3fe06d7ac91
jgalar
added a commit
that referenced
this pull request
Apr 19, 2023
To make debugging easier, log uts information on launch of the daemons. Produces an output of the following form when the daemons are launched in verbose mode: DBG1 - 14:30:14.997217006 [Main]: System information: (in log_system_information() at logging-utils.cpp:23) DBG1 - 14:30:14.997221139 [Main]: sysname: `Linux` (in log_system_information() at logging-utils.cpp:24) DBG1 - 14:30:14.997227199 [Main]: nodename: `carbonara` (in log_system_information() at logging-utils.cpp:25) DBG1 - 14:30:14.997231284 [Main]: release: `6.1.1-arch1-1` (in log_system_information() at logging-utils.cpp:26) DBG1 - 14:30:14.997235261 [Main]: version: `#1 SMP PREEMPT_DYNAMIC Wed, 21 Dec 2022 22:27:55 +0000` (in log_system_information() at logging-utils.cpp:27) DBG1 - 14:30:14.997240214 [Main]: machine: `x86_64` (in log_system_information() at logging-utils.cpp:28) Signed-off-by: Jérémie Galarneau <[email protected]> Change-Id: I399fe8a88da8480e617cc33dd6b1dc2723c300c7
jgalar
added a commit
that referenced
this pull request
Dec 18, 2023
Issue observed -------------- When using the CLI to list the configuration of a session that has an event rule which makes use of multiple exclusions, the session daemon crashes with the following stack trace: (gdb) bt #0 0x00007fa9ed401445 in ?? () from /usr/lib/libc.so.6 #1 0x0000560cd5fc5199 in lttng_strnlen (str=0x615f6f6c6c6568 <error: Cannot access memory at address 0x615f6f6c6c6568>, max=256) at ../../src/common/compat/string.h:19 #2 0x0000560cd5fc6b39 in lttng_event_serialize (event=0x7fa9cc01d8b0, exclusion_count=2, exclusion_list=0x7fa9cc011794, filter_expression=0x0, bytecode_len=0, bytecode=0x0, payload=0x7fa9d3ffda88) at event.c:767 #3 0x0000560cd5f380b5 in list_lttng_ust_global_events (nb_events=<synthetic pointer>, reply_payload=0x7fa9d3ffda88, ust_global=<optimized out>, channel_name=<optimized out>) at cmd.c:472 #4 cmd_list_events (domain=<optimized out>, session=<optimized out>, channel_name=<optimized out>, reply_payload=0x7fa9d3ffda88) at cmd.c:3860 #5 0x0000560cd5f6d76a in process_client_msg (cmd_ctx=0x7fa9d3ffa710, sock=0x7fa9d3ffa5b0, sock_error=0x7fa9d3ffa5b4) at client.c:1890 #6 0x0000560cd5f6f876 in thread_manage_clients (data=0x560cd7879490) at client.c:2629 #7 0x0000560cd5f65a54 in launch_thread (data=0x560cd7879500) at thread.c:66 #8 0x00007fa9ed32d44b in ?? () from /usr/lib/libc.so.6 lttng#9 0x00007fa9ed3b0e40 in ?? () from /usr/lib/libc.so.6 Cause ----- lttng_event_serialize expects a `char **` list of exclusion names, as provided by the other callsite in liblttng-ctl. However, the callsite in list_lttng_ust_global_events passes pointer to the exclusions as stored in lttng_event_exclusion. lttng_event_exclusion contains an array of fixed-length strings (with a stride of 256 bytes) which isn't an expected layout for lttng_event_serialize. Solution -------- A temporary array of pointers is constructed before invoking lttng_event_serialize to construct a list of exclusions with the layout that lttng_event_serialize expects. The array itself is reused for all events, limiting the number of allocations. Note ---- None. Change-Id: I266a1cc9e9f18e0476177a0047b1d8f468110575 Signed-off-by: Jérémie Galarneau <[email protected]>
jgalar
added a commit
that referenced
this pull request
Dec 18, 2023
Coverity warns that some container operations used by random_access_container_wrapper can throw even though methods are marked as noexcept. CID 1512893 (#1-2 of 2): Uncaught exception (UNCAUGHT_EXCEPT) exn_spec_violation: An exception of type lttng::invalid_argument_error is thrown but the exception specification noexcept doesn't allow it to be thrown. This will result in a call to terminate(). The noexcept specifier is remvoved from operator* and end() of random_access_container_wrapper's iterator implementation. To make this a bit clearer, a bounds check is performed in operator[] directly which will make errors easier to catch. Reported-by: Coverity Scan Signed-off-by: Jérémie Galarneau <[email protected]> Change-Id: I31d51e8709d33b3c80d64c8c05a23e519e3a93e7
jgalar
added a commit
that referenced
this pull request
Dec 18, 2023
Issue observed -------------- When running the session daemon under ASAN, the following report is produced: Direct leak of 104 byte(s) in 1 object(s) allocated from: #0 0x7f93866e0cd1 in __interceptor_calloc /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_malloc_linux.cpp:77 #1 0x55c55a7c4963 in zmalloc_internal /home/simark/src/lttng-tools/src/common/macros.hpp:60 #2 0x55c55a7c4973 in lttng_pipe* zmalloc<lttng_pipe>() /home/simark/src/lttng-tools/src/common/macros.hpp:88 #3 0x55c55a7c26eb in _pipe_create /home/simark/src/lttng-tools/src/common/pipe.cpp:111 #4 0x55c55a7c351d in lttng_pipe_open(int) /home/simark/src/lttng-tools/src/common/pipe.cpp:185 #5 0x55c55a586dd6 in operator() /home/simark/src/lttng-tools/src/bin/lttng-sessiond/rotation-thread.cpp:403 #6 0x55c55a58744a in lttng::sessiond::rotation_thread::rotation_thread(lttng::sessiond::rotation_thread_timer_queue&, notification_thread_handle&) /home/simark/src/lttng-tools/src/bin/lttng-sessiond/rotation-thread.cpp:402 #7 0x55c55a46377f in std::unique_ptr<lttng::sessiond::rotation_thread, std::default_delete<lttng::sessiond::rotation_thread> > lttng::make_unique<lttng::sessiond::rotation_thread, lttng::sessiond::rotation_thread_timer_queue&, notification_thread_handle&>(lttng::sessiond::rotation_thread_timer_queue&, notification_thread_handle&) /home/simark/src/lttng-tools/src/common/make-unique.hpp:18 #8 0x55c55a455024 in _main /home/simark/src/lttng-tools/src/bin/lttng-sessiond/main.cpp:1773 lttng#9 0x55c55a455c2e in main /home/simark/src/lttng-tools/src/bin/lttng-sessiond/main.cpp:1982 lttng#10 0x7f9385c1484f (/usr/lib/libc.so.6+0x2384f) (BuildId: 2f005a79cd1a8e385972f5a102f16adba414d75e) Cause ----- On destruction, the std::unique_ptr wrapper of lttng_pipe (lttng_pipe::uptr) invokes `lttng_pipe_close` (which only closes the file descriptors of the underlying pipe) rather than `lttng_pipe_destroy` which closes the file descriptors _and_ frees the memory allocated by lttng_open. Currently, the rotation thread is the only user of this wrapper (through its quit_pipe). Solution -------- The deleter of lttng_pipe::uptr is replaced to invoke lttng_pipe_destroy. Fixes #1380 Signed-off-by: Jérémie Galarneau <[email protected]> Change-Id: I5715ac6131c5aa134cfd18d8b677f31aabed36f0
jgalar
added a commit
that referenced
this pull request
Dec 18, 2023
Issue observed -------------- ASAN reports the following leak when running the tests/regression/tools/context/test_ust.py test suite: Direct leak of 8 byte(s) in 1 object(s) allocated from: #0 0x7f32e5ae0cd1 in __interceptor_calloc /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_malloc_linux.cpp:77 #1 0x5653e1092088 in zmalloc_internal ../../../src/common/macros.hpp:60 #2 0x5653e10922b3 in char* calloc<char>(unsigned long) string-utils/../macros.hpp:113 #3 0x5653e119d68f in get_context_type commands/add_context.cpp:1012 #4 0x5653e119ddf5 in cmd_add_context(int, char const**) commands/add_context.cpp:1059 #5 0x5653e11e12e7 in handle_command /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng/lttng.cpp:237 #6 0x5653e11e2027 in parse_args /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng/lttng.cpp:427 #7 0x5653e11e24e1 in _main /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng/lttng.cpp:474 #8 0x5653e11e25bd in main /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng/lttng.cpp:485 lttng#9 0x7f32e3e3984f (/usr/lib/libc.so.6+0x2384f) (BuildId: 2f005a79cd1a8e385972f5a102f16adba414d75e) Direct leak of 5 byte(s) in 1 object(s) allocated from: #0 0x7f32e5ae0cd1 in __interceptor_calloc /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_malloc_linux.cpp:77 #1 0x5653e1092088 in zmalloc_internal ../../../src/common/macros.hpp:60 #2 0x5653e10922b3 in char* calloc<char>(unsigned long) string-utils/../macros.hpp:113 #3 0x5653e119d2ae in get_context_type commands/add_context.cpp:1003 #4 0x5653e119ddf5 in cmd_add_context(int, char const**) commands/add_context.cpp:1059 #5 0x5653e11e12e7 in handle_command /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng/lttng.cpp:237 #6 0x5653e11e2027 in parse_args /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng/lttng.cpp:427 #7 0x5653e11e24e1 in _main /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng/lttng.cpp:474 #8 0x5653e11e25bd in main /home/jgalar/EfficiOS/src/lttng-tools/src/bin/lttng/lttng.cpp:485 lttng#9 0x7f32e3e3984f (/usr/lib/libc.so.6+0x2384f) (BuildId: 2f005a79cd1a8e385972f5a102f16adba414d75e) Cause ----- The context and provider names are dynamically allocated by get_context_type() and stored in ctx_type. However, destroy_ctx_type() never frees those members when the structure is of type CONTEXT_APP_CONTEXT. Solution -------- Free both names when an application context type is destroyed. Signed-off-by: Jérémie Galarneau <[email protected]> Change-Id: I86dde1eed9f0cc63499c936cf373b094168035e2
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See 71214b6.