Staging test fixes v1

doc/Makefile.am

@@ -1,7 +1,8 @@
 SUBDIRS = man
 EXTRA_DIST = quickstart.txt streaming-howto.txt python-howto.txt \
 	snapshot-howto.txt calibrate.txt kernel-CodingStyle.txt \
-	live-reading-howto.txt live-reading-protocol.txt
+	live-reading-howto.txt live-reading-protocol.txt \
+	relayd-architecture.txt
 
 dist_doc_DATA = quickstart.txt streaming-howto.txt python-howto.txt \
 	snapshot-howto.txt calibrate.txt live-reading-howto.txt \

doc/relayd-architecture.txt

@@ -0,0 +1,95 @@
+LTTng Relay Daemon Architecture
+Mathieu Desnoyers, August 2015
+
+This document describes the object model and architecture of the relay
+daemon, after the refactoring done within the commit "Fix: Relay daemon
+ownership and reference counting".
+
+We have the following object composition hierarchy:
+
+relay connection (main.c, for sessiond/consumer)
+  |
+  \-> 0 or 1 session
+       |
+       \-> 0 or many ctf-trace
+                 |
+                 \-> 0 or many stream
+                      |   |
+                      |   \-> 0 or many index
+                      |
+                      \-------> 0 or 1 viewer stream
+
+live connection (live.c, for client)
+  |
+  \-> 1 viewer session
+       |
+       \-> 0 or many session (actually a reference to session as created
+             |                by the relay connection)
+             |
+             \-> ..... (ctf-trace, stream, index, viewer stream)
+
+There are global tables declared in lttng-relayd.h for sessions
+(sessions_ht, indexed by session id), streams (relay_streams_ht, indexed
+by stream handle), and viewer streams (viewer_streams_ht, indexed by
+stream handle). The purpose of those tables is to allow fast lookup of
+those objects using the IDs received in the communication protocols.
+
+There is also one connection hash table per worker thread. There is one
+worker thread to receive data (main.c), and one worker thread to
+interact with viewer clients (live.c). Those tables are indexed by
+socket file descriptor.
+
+A RCU lookup+refcounting scheme has been introduced for all objects
+(except viewer session which is still an exception at the moment). This
+scheme allows looking up the objects or doing a traversal on the RCU
+linked list or hash table in combination with a getter on the object.
+This getter validates that there is still at least one reference to the
+object, else the lookup acts just as if the object does not exist. This
+scheme is protected by a "reflock" mutex in each object. "reflock"
+mutexes can be nested from the innermost object to the outermost object.
+IOW, the session reflock can nest within the ctf-trace reflock.
+
+There is also a "lock" mutex in each object. Those are used to
+synchronize between threads (currently the main.c relay thread and
+live.c client thread) when objects are shared. Locks can be nested form
+the outermost object to the innermost object. IOW, the ctf-trace lock can
+nest within the session lock.
+
+A "lock" should never nest within a "reflock".
+
+RCU linked lists are used to iterate using RCU, and protected by
+their own mutex for modifications. Iterations should be confirmed using
+the object "getter" to ensure its refcount is not 0 (exept in cases
+where the caller actually owns the objects and therefore can assume its
+refcount is not 0).
+
+RCU hash tables are used to iterate using RCU. Iteration should be
+confirmed using the object "getter" to ensure its refcount is not 0
+(except again if we have ownership and can assume the object refcount is
+not 0).
+
+Object creation has a refcount of 1. Each getter increments the
+refcount, and need to be paired with a "put" to decrement it. A final
+put on "self" (ownership) will allow refcount to reach 0, therefore
+triggering release, and thus free through call_rcu.
+
+In the composition scheme, we find back references from each composite
+to its container. Therefore, each composite holds a reference (refcount)
+on its container. This allows following pointers from e.g. viewer stream
+to stream to ctf-trace to session without performing any validation,
+due to transitive refcounting of those back-references.
+
+In addition to those back references, there are a few key ownership
+references held. The connection in the relay worker thread (main.c)
+holds ownership on the session, and on each stream it contains. The
+connection in the live worker thread (live.c) holds ownership on each
+viewer stream it creates. The rest is ensured by back references from
+composite to container objects. When a connection is closed, it puts all
+the ownership references it is holding. This will then eventually
+trigger destruction of the session, streams, and viewer streams
+associated with the connection when all the back references reach 0.
+
+RCU read-side locks are now only held during iteration on RCU lists and
+hash tables, and within the internals of the get (lookup) and put
+functions. Those functions then use refcounting to ensure existence of
+the object when returned to their caller.

src/bin/lttng-relayd/Makefile.am

@@ -17,7 +17,9 @@ lttng_relayd_SOURCES = main.c lttng-relayd.h utils.h utils.c cmd.h \
                        viewer-stream.h viewer-stream.c \
                        session.c session.h \
                        stream.c stream.h \
-                       connection.c connection.h
+                       stream-fd.c stream-fd.h \
+                       connection.c connection.h \
+                       viewer-session.c viewer-session.h
 
 # link on liblttngctl for check if relayd is already alive.
 lttng_relayd_LDADD = -lrt -lurcu-common -lurcu \

src/bin/lttng-relayd/cmd-2-1.c

@@ -1,6 +1,7 @@
 /*
  * Copyright (C) 2013 - Julien Desfossez <[email protected]>
  *                      David Goulet <[email protected]>
+ *               2015 - Mathieu Desnoyers <[email protected]>
  *
  * This program is free software; you can redistribute it and/or modify it
  * under the terms of the GNU General Public License, version 2 only, as
@@ -28,30 +29,30 @@
 #include "cmd-2-1.h"
 #include "utils.h"
 
+/*
+ * cmd_recv_stream_2_1 allocates path_name and channel_name.
+ */
 int cmd_recv_stream_2_1(struct relay_connection *conn,
-		struct relay_stream *stream)
+		char **path_name, char **channel_name)
 {
 	int ret;
 	struct lttcomm_relayd_add_stream stream_info;
 
-	assert(conn);
-	assert(stream);
-
 	ret = cmd_recv(conn->sock, &stream_info, sizeof(stream_info));
 	if (ret < 0) {
 		ERR("Unable to recv stream version 2.1");
 		goto error;
 	}
 
-	stream->path_name = create_output_path(stream_info.pathname);
-	if (stream->path_name == NULL) {
+	*path_name = create_output_path(stream_info.pathname);
+	if (*path_name == NULL) {
 		PERROR("Path name allocation");
 		ret = -ENOMEM;
 		goto error;
 	}
 
-	stream->channel_name = strdup(stream_info.channel_name);
-	if (stream->channel_name == NULL) {
+	*channel_name = strdup(stream_info.channel_name);
+	if (*channel_name == NULL) {
 		ret = -errno;
 		PERROR("Path name allocation");
 		goto error;

src/bin/lttng-relayd/cmd-2-1.h

@@ -1,6 +1,10 @@
+#ifndef RELAYD_CMD_2_1_H
+#define RELAYD_CMD_2_1_H
+
 /*
  * Copyright (C) 2013 - Julien Desfossez <[email protected]>
  *                      David Goulet <[email protected]>
+ *               2015 - Mathieu Desnoyers <[email protected]>
  *
  * This program is free software; you can redistribute it and/or modify it
  * under the terms of the GNU General Public License, version 2 only, as
@@ -16,13 +20,9 @@
  * Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
  */
 
-#ifndef RELAYD_CMD_2_1_H
-#define RELAYD_CMD_2_1_H
-
 #include "lttng-relayd.h"
-#include "stream.h"
 
 int cmd_recv_stream_2_1(struct relay_connection *conn,
-		struct relay_stream *stream);
+		char **path_name, char **channel_name);
 
 #endif /* RELAYD_CMD_2_1_H */

src/bin/lttng-relayd/cmd-2-2.c

@@ -1,6 +1,7 @@
 /*
  * Copyright (C) 2013 - Julien Desfossez <[email protected]>
  *                      David Goulet <[email protected]>
+ *               2015 - Mathieu Desnoyers <[email protected]>
  *
  * This program is free software; you can redistribute it and/or modify it
  * under the terms of the GNU General Public License, version 2 only, as
@@ -30,37 +31,38 @@
 #include "cmd-2-1.h"
 #include "utils.h"
 
+/*
+ * cmd_recv_stream_2_2 allocates path_name and channel_name.
+ */
 int cmd_recv_stream_2_2(struct relay_connection *conn,
-		struct relay_stream *stream)
+		char **path_name, char **channel_name,
+		uint64_t *tracefile_size, uint64_t *tracefile_count)
 {
 	int ret;
 	struct lttcomm_relayd_add_stream_2_2 stream_info;
 
-	assert(conn);
-	assert(stream);
-
 	ret = cmd_recv(conn->sock, &stream_info, sizeof(stream_info));
 	if (ret < 0) {
 		ERR("Unable to recv stream version 2.2");
 		goto error;
 	}
 
-	stream->path_name = create_output_path(stream_info.pathname);
-	if (stream->path_name == NULL) {
+	*path_name = create_output_path(stream_info.pathname);
+	if (*path_name == NULL) {
 		PERROR("Path name allocation");
 		ret = -ENOMEM;
 		goto error;
 	}
 
-	stream->channel_name = strdup(stream_info.channel_name);
-	if (stream->channel_name == NULL) {
+	*channel_name = strdup(stream_info.channel_name);
+	if (*channel_name == NULL) {
 		ret = -errno;
 		PERROR("Path name allocation");
 		goto error;
 	}
 
-	stream->tracefile_size = be64toh(stream_info.tracefile_size);
-	stream->tracefile_count = be64toh(stream_info.tracefile_count);
+	*tracefile_size = be64toh(stream_info.tracefile_size);
+	*tracefile_count = be64toh(stream_info.tracefile_count);
 	ret = 0;
 
 error:

src/bin/lttng-relayd/cmd-2-2.h

@@ -1,6 +1,10 @@
+#ifndef RELAYD_CMD_2_2_H
+#define RELAYD_CMD_2_2_H
+
 /*
  * Copyright (C) 2013 - Julien Desfossez <[email protected]>
  *                      David Goulet <[email protected]>
+ *               2015 - Mathieu Desnoyers <[email protected]>
  *
  * This program is free software; you can redistribute it and/or modify it
  * under the terms of the GNU General Public License, version 2 only, as
@@ -16,12 +20,10 @@
  * Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
  */
 
-#ifndef RELAYD_CMD_2_2_H
-#define RELAYD_CMD_2_2_H
-
 #include "lttng-relayd.h"
 
 int cmd_recv_stream_2_2(struct relay_connection *conn,
-		struct relay_stream *stream);
+		char **path_name, char **channel_name,
+		uint64_t *tracefile_size, uint64_t *tracefile_count);
 
 #endif /* RELAYD_CMD_2_2_H */

src/bin/lttng-relayd/cmd-2-4.c

@@ -1,6 +1,7 @@
 /*
  * Copyright (C) 2013 - Julien Desfossez <[email protected]>
  *                      David Goulet <[email protected]>
+ *               2015 - Mathieu Desnoyers <[email protected]>
  *
  * This program is free software; you can redistribute it and/or modify it
  * under the terms of the GNU General Public License, version 2 only, as
@@ -30,26 +31,24 @@
 #include "lttng-relayd.h"
 
 int cmd_create_session_2_4(struct relay_connection *conn,
-		struct relay_session *session)
+		char *session_name, char *hostname,
+		uint32_t *live_timer, bool *snapshot)
 {
 	int ret;
 	struct lttcomm_relayd_create_session_2_4 session_info;
 
-	assert(conn);
-	assert(session);
-
 	ret = cmd_recv(conn->sock, &session_info, sizeof(session_info));
 	if (ret < 0) {
 		ERR("Unable to recv session info version 2.4");
 		goto error;
 	}
 
-	strncpy(session->session_name, session_info.session_name,
-			sizeof(session->session_name));
-	strncpy(session->hostname, session_info.hostname,
-			sizeof(session->hostname));
-	session->live_timer = be32toh(session_info.live_timer);
-	session->snapshot = be32toh(session_info.snapshot);
+	strncpy(session_name, session_info.session_name,
+			sizeof(session_info.session_name));
+	strncpy(hostname, session_info.hostname,
+			sizeof(session_info.hostname));
+	*live_timer = be32toh(session_info.live_timer);
+	*snapshot = be32toh(session_info.snapshot);
 
 	ret = 0;
 

src/bin/lttng-relayd/cmd-2-4.h

@@ -1,6 +1,10 @@
+#ifndef RELAYD_CMD_2_4_H
+#define RELAYD_CMD_2_4_H
+
 /*
  * Copyright (C) 2013 - Julien Desfossez <[email protected]>
  *                      David Goulet <[email protected]>
+ *               2015 - Mathieu Desnoyers <[email protected]>
  *
  * This program is free software; you can redistribute it and/or modify it
  * under the terms of the GNU General Public License, version 2 only, as
@@ -16,12 +20,10 @@
  * Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
  */
 
-#ifndef RELAYD_CMD_2_4_H
-#define RELAYD_CMD_2_4_H
-
 #include "lttng-relayd.h"
 
 int cmd_create_session_2_4(struct relay_connection *conn,
-		struct relay_session *session);
+		char *session_name, char *hostname,
+		uint32_t *live_timer, bool *snapshot);
 
 #endif /* RELAYD_CMD_2_4_H */

src/bin/lttng-relayd/cmd-generic.c

@@ -1,6 +1,7 @@
 /*
  * Copyright (C) 2013 - Julien Desfossez <[email protected]>
  *                      David Goulet <[email protected]>
+ *               2015 - Mathieu Desnoyers <[email protected]>
  *
  * This program is free software; you can redistribute it and/or modify it
  * under the terms of the GNU General Public License, version 2 only, as

src/bin/lttng-relayd/cmd-generic.h

@@ -1,6 +1,10 @@
+#ifndef RELAYD_CMD_GENERIC_H
+#define RELAYD_CMD_GENERIC_H
+
 /*
  * Copyright (C) 2013 - Julien Desfossez <[email protected]>
  *                      David Goulet <[email protected]>
+ *               2015 - Mathieu Desnoyers <[email protected]>
  *
  * This program is free software; you can redistribute it and/or modify it
  * under the terms of the GNU General Public License, version 2 only, as
@@ -16,9 +20,6 @@
  * Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
  */
 
-#ifndef RELAYD_CMD_GENERIC_H
-#define RELAYD_CMD_GENERIC_H
-
 #include <common/sessiond-comm/sessiond-comm.h>
 
 #include "connection.h"

src/bin/lttng-relayd/cmd.h

@@ -1,6 +1,10 @@
+#ifndef RELAYD_CMD_H
+#define RELAYD_CMD_H
+
 /*
  * Copyright (C) 2013 - Julien Desfossez <[email protected]>
  *                      David Goulet <[email protected]>
+ *               2015 - Mathieu Desnoyers <[email protected]>
  *
  * This program is free software; you can redistribute it and/or modify it
  * under the terms of the GNU General Public License, version 2 only, as
@@ -16,9 +20,6 @@
  * Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
  */
 
-#ifndef RELAYD_CMD_H
-#define RELAYD_CMD_H
-
 #include "cmd-generic.h"
 #include "cmd-2-1.h"
 #include "cmd-2-2.h"