Overhaul of the Logging Infrastructure (#1596)

* Pulled ostream operators for IPAddress and MacAddress into the namespace to satisfy ADL requirements. * Refactor of Logger. - LogLevel is now a top level enum class. - Added a new log level Off to disable output from a specific module. - Logger::LogLevel is a deprecated alias to LogLevel. - Logger::Info, Debug, Error are deprecated aliases to LogLevel::... - Removed public "internal*" functions from Logger.Logger - Added LogSource struct to encapsulate source information. - Added shouldLog method to check if a log should be emitted for a given level and module. - Removed nonfunctional artifacts "m_LogStream" and "Logger::operator<<" - Added templated "log" functions that are friends to Logger. - Reworked PCPP_LOG macros to no longer utilize the now removed internal functions. - Added PCPP_LOG_INFO macro level. - Changed PCPP_LOG_ERROR to now check if the log should be emitted. - Fixed NetworkUtils log module name overlapping with NetworkUtils class. - Fixed missing enum value for PacketLogModuleSll2Layer. * Cleanup and fixes. * Added the new Off log level to the string conversion. * Fixed wrong variable name. * Added documentation to log source. * Lint. * Fixed docstring for LogSource. * Fixed extra / * Fixed explicit warning. * Moved log functions inside logger. * Revert "Moved log functions inside logger." This reverts commit be95ab8. * Moved the log functions to the Logger class. - Added optional compile time elimination of log calls below set level. * Fixed typo in macro names. * Changed value param to const-ref. * Added "venv" and "./out" to ignored directories by codespell. * Reverted to previous optimizations to keep executable binary size low. - Renamed printLogMessage to emit and changed visibility to public. - Added new class LogContext to encapsulate a single emittable log message. - Added new methods createLogContext which is practically a rework of internalCreateLogStream but returns a LogContext. - Added optional use of object pooling optimization for reusing log contexts. (Enabled via preprocessor flag PCPP_LOG_USE_OBJECT_POOL) * Fixed warnings about unreferenced local variables if the compile time minimum log level set to too high severity.. * Removed useless variable. * Fixed friend class definition. * Fixed variable assignment. * Added method useContextPooling to control if the logger should use context pooling. - Removed preprocessor variable PCPP_LOG_USE_OBJECT_POOL. - Disabled context pooling for the unit tests as it interferes with the memory leak checker. * Fixed more warnings about unreferenced local variables if the compile time minimum log level set to too high severity.. * Addressed warnings and documentation. - Added documentation to new methods and classes. - Addressed explicit constructor warnings. - Moved disable of context pooling for unit tests to the main.cpp files of the respective tests. * Fixed include. * Fixed pointer dereference. * Fixed memory checker issues with logger. - Added 2 preallocated log contexts to the object pool. * Lint * Added mutex lock on the default log printer to support proper multi-threading and eliminate possibility of data races during log emission. * Fixed typos in documentation. * Changed level variable to private. * Changed LogPrinter definition to use the metaprogramming construct std::add_pointer. * Replaced C library includes with C++ equivalents. * Updated documentation format. * Added full namespace qualifier for marco code. * Changed C-array to std::array. * Added a mutex lock when writing or reading to last error string to prevent tearing. * Fixed object pool member variables. * Added mutators to change the max size of an object pool at runtime. * Added exception if pool preallocation size is larger then the maximum allowed pool size. * Added tests for ObjectPool<T>. - Changed infinite pool size to be maximum value of size_t instead of 0, to fix an issue if max size is set to 0. - Added size getter to the pool. - Marked the pool mutex as mutable. * Lint * Added option to set the max pool size for the Logger context pool. * Disabled logger context pooling in the tests as it is detected as a false positive memory leak. * Lint * Lint * Updated logger tests. - Changed `pcpp::Logger::getInstance()` to use a cached `logger` variable. - Removed the need for fully qualified names in the logger test. - C-style casts to Cpp casts. * Added tests for `shouldLog`. * Moved iostream and iomanip to Logger.cpp as they are unnessesary in the header. * Lint * Added iostream to examples that depended on transitively including iostream from Logger.h * Added <iomanip> to examples that depended on transitively including from Logger.h * Moved ObjectPool to internal namespace. * Renamed ObjectPool to DynamicObjectPool. * Renamed preallocate ctor parameter to initialSize. * Removed unused logging functions. * Fixed missed include <cstring>.
seladb · Jan 29, 2025 · 3af7383 · 3af7383
1 parent 2d0fc5a
commit 3af7383
Show file tree

Hide file tree

Showing 20 changed files with 607 additions and 103 deletions.
diff --git a/.codespellrc b/.codespellrc
@@ -1,4 +1,4 @@
 [codespell]
-skip = *.dat,typos-config.toml,.git,.venv,./ci,./Dist,./mk,./Tests/ExamplesTest/expected_output,./Tests/ExamplesTest/pcap_examples,./Tests/Packet++Test/PacketExamples,./Tests/Pcap++Test/PcapExamples,./3rdParty,./Examples/PcapSearch/dirent-for-Visual-Studio
+skip = *.dat,typos-config.toml,.git,.venv,venv,./out,./ci,./Dist,./mk,./Tests/ExamplesTest/expected_output,./Tests/ExamplesTest/pcap_examples,./Tests/Packet++Test/PacketExamples,./Tests/Pcap++Test/PcapExamples,./3rdParty,./Examples/PcapSearch/dirent-for-Visual-Studio
 ignore-words = codespell-ignore-list.txt
 count =
diff --git a/Common++/CMakeLists.txt b/Common++/CMakeLists.txt
@@ -22,6 +22,7 @@ set(
   header/Logger.h
   header/LRUList.h
   header/MacAddress.h
+  header/ObjectPool.h
   header/OUILookup.h
   header/PcapPlusPlusVersion.h
   header/PointerVector.h

diff --git a/Common++/header/IpAddress.h b/Common++/header/IpAddress.h
@@ -876,5 +876,4 @@ namespace pcpp
 		oss << network.toString();
 		return oss;
 	}
-
 }  // namespace pcpp
diff --git a/Common++/header/Logger.h b/Common++/header/Logger.h
@@ -1,10 +1,14 @@
 #pragma once
 
 #include <cstdio>
-#include <iostream>
-#include <sstream>
-#include <iomanip>
 #include <cstdint>
+#include <memory>
+#include <array>
+#include <mutex>
+#include <ostream>
+#include <sstream>
+#include "DeprecationUtils.h"
+#include "ObjectPool.h"
 
 #ifndef LOG_MODULE
 #	define LOG_MODULE UndefinedLogModule
@@ -17,35 +21,29 @@
 #	define PCAPPP_FILENAME __FILE__
 #endif
 
-#define PCPP_LOG(level, message)                                                                                       \
-	do                                                                                                                 \
-	{                                                                                                                  \
-		std::ostringstream* sstream = pcpp::Logger::getInstance().internalCreateLogStream();                           \
-		(*sstream) << message;                                                                                         \
-		pcpp::Logger::getInstance().internalPrintLogMessage(sstream, level, PCAPPP_FILENAME, __FUNCTION__, __LINE__);  \
-	} while (0)
-
-#define PCPP_LOG_DEBUG(message)                                                                                        \
-	do                                                                                                                 \
-	{                                                                                                                  \
-		if (pcpp::Logger::getInstance().logsEnabled() && pcpp::Logger::getInstance().isDebugEnabled(LOG_MODULE))       \
-		{                                                                                                              \
-			PCPP_LOG(pcpp::Logger::Debug, message);                                                                    \
-		}                                                                                                              \
-	} while (0)
+/// @file
 
-#define PCPP_LOG_ERROR(message)                                                                                        \
-	do                                                                                                                 \
-	{                                                                                                                  \
-		PCPP_LOG(pcpp::Logger::Error, message);                                                                        \
-	} while (0)
+// Compile time log levels.
+// Allows for conditional removal of unwanted log calls at compile time.
+#define PCPP_LOG_LEVEL_OFF 0
+#define PCPP_LOG_LEVEL_ERROR 1
+#define PCPP_LOG_LEVEL_INFO 2
+#define PCPP_LOG_LEVEL_DEBUG 3
 
-/// @file
+// All log messages built via a PCPP_LOG_* macro below the PCPP_ACTIVE_LOG_LEVEL will be removed at compile time.
+// Uses the PCPP_ACTIVE_LOG_LEVEL if it is defined, otherwise defaults to PCAP_LOG_LEVEL_DEBUG
+#ifndef PCPP_ACTIVE_LOG_LEVEL
+#	define PCPP_ACTIVE_LOG_LEVEL PCPP_LOG_LEVEL_DEBUG
+#endif  // !PCPP_ACTIVE_LOG_LEVEL
 
 /// @namespace pcpp
 /// @brief The main namespace for the PcapPlusPlus lib
 namespace pcpp
 {
+	/// Cross-platform and thread-safe version of strerror
+	/// @param errnum Value of errno
+	/// @return String representation of the error number
+	std::string getErrorString(int errnum);
 
 	/// An enum representing all PcapPlusPlus modules
 	enum LogModule : uint8_t
@@ -75,6 +73,7 @@ namespace pcpp
 		PacketLogModuleGreLayer,         ///< GreLayer module (Packet++)
 		PacketLogModuleSSLLayer,         ///< SSLLayer module (Packet++)
 		PacketLogModuleSllLayer,         ///< SllLayer module (Packet++)
+		PacketLogModuleSll2Layer,        ///< Sll2Layer module (Packet++)
 		PacketLogModuleNflogLayer,       ///< NflogLayer module (Packet++)
 		PacketLogModuleDhcpLayer,        ///< DhcpLayer module (Packet++)
 		PacketLogModuleDhcpV6Layer,      ///< DhcpV6Layer module (Packet++)
@@ -109,14 +108,100 @@ namespace pcpp
 		PcapLogModuleDpdkDevice,         ///< DpdkDevice module (Pcap++)
 		PcapLogModuleKniDevice,          ///< KniDevice module (Pcap++)
 		PcapLogModuleXdpDevice,          ///< XdpDevice module (Pcap++)
-		NetworkUtils,                    ///< NetworkUtils module (Pcap++)
+		PcapLogModuleNetworkUtils,       ///< Network Utils module (Pcap++)
 		NumOfLogModules
 	};
 
-	/// Cross-platform and thread-safe version of strerror
-	/// @param errnum Value of errno
-	/// @return String representation of the error number
-	std::string getErrorString(int errnum);
+	/// @struct LogSource
+	/// Represents the source of a log message.
+	/// Contains information about the source file, function, line number, and the log module.
+	struct LogSource
+	{
+		/// Default constructor for LogSource.
+		constexpr LogSource() = default;
+
+		/// Constructor for LogSource with only the log module.
+		/// @param logModule The log module.
+		explicit constexpr LogSource(LogModule logModule) : logModule(logModule)
+		{}
+
+		/// Constructor for LogSource with all parameters.
+		/// @param logModule The log module.
+		/// @param file The source file.
+		/// @param function The source function.
+		/// @param line The line number.
+		constexpr LogSource(LogModule logModule, const char* file, const char* function, int line)
+		    : file(file), function(function), line(line), logModule(logModule)
+		{}
+
+		const char* file = nullptr;               /**< The source file. */
+		const char* function = nullptr;           /**< The source function. */
+		int line = 0;                             /**< The line number. */
+		LogModule logModule = UndefinedLogModule; /**< The log module. */
+	};
+
+	/// An enum representing the log level. Currently 4 log levels are supported: Off, Error, Info and Debug. Info is
+	/// the default log level
+	enum class LogLevel
+	{
+		Off = PCPP_LOG_LEVEL_OFF,      ///< No log messages are emitted.
+		Error = PCPP_LOG_LEVEL_ERROR,  ///< Error level logs are emitted.
+		Info = PCPP_LOG_LEVEL_INFO,    ///< Info level logs and above are emitted.
+		Debug = PCPP_LOG_LEVEL_DEBUG   ///< Debug level logs and above are emitted.
+	};
+
+	inline std::ostream& operator<<(std::ostream& s, LogLevel v)
+	{
+		return s << static_cast<std::underlying_type<LogLevel>::type>(v);
+	}
+
+	// Forward declaration
+	class Logger;
+
+	namespace internal
+	{
+		/// @class LogContext
+		/// @brief A context encapsulating the details of a single log message to be passed to the Logger.
+		class LogContext
+		{
+		public:
+			friend class pcpp::Logger;
+
+			/// @brief Creates a context with an empty message with Info level and no source.
+			LogContext() = default;
+
+			/// @brief Creates a context with an empty message with the given level and source.
+			/// @param level The log level for this message.
+			/// @param source The log source.
+			explicit LogContext(LogLevel level, LogSource const& source = {}) : m_Source(source), m_Level(level)
+			{}
+
+			/// @brief Initializes the context with an empty message and the given level and source.
+			/// @param level The log level for this message.
+			/// @param source The log source.
+			void init(LogLevel level, LogSource const& source)
+			{
+				m_Source = source;
+				m_Level = level;
+				m_Stream.clear();
+				m_Stream.str({});
+			}
+
+			/// @brief Appends to the message.
+			/// @param value The value to append.
+			/// @return A reference to this context.
+			template <class T> inline LogContext& operator<<(T const& value)
+			{
+				m_Stream << value;
+				return *this;
+			}
+
+		private:
+			std::ostringstream m_Stream;
+			LogSource m_Source;
+			LogLevel m_Level = LogLevel::Info;
+		};
+	}  // namespace internal
 
 	/// @class Logger
 	/// PcapPlusPlus logger manager.
@@ -138,14 +223,14 @@ namespace pcpp
 	class Logger
 	{
 	public:
-		/// An enum representing the log level. Currently 3 log levels are supported: Error, Info and Debug. Info is the
-		/// default log level
-		enum LogLevel : uint8_t
-		{
-			Error,  ///< Error log level
-			Info,   ///< Info log level
-			Debug   ///< Debug log level
-		};
+		// Deprecated, Use the LogLevel in the pcpp namespace instead.
+		using LogLevel = pcpp::LogLevel;
+		PCPP_DEPRECATED("Use the LogLevel in the pcpp namespace instead.")
+		static const LogLevel Error = LogLevel::Error;
+		PCPP_DEPRECATED("Use the LogLevel in the pcpp namespace instead.")
+		static const LogLevel Info = LogLevel::Info;
+		PCPP_DEPRECATED("Use the LogLevel in the pcpp namespace instead.")
+		static const LogLevel Debug = LogLevel::Debug;
 
 		/// @typedef LogPrinter
 		/// Log printer callback. Used for printing the logs in a custom way.
@@ -154,7 +239,10 @@ namespace pcpp
 		/// @param[in] file The source file in PcapPlusPlus code the log message is coming from
 		/// @param[in] method The method in PcapPlusPlus code the log message is coming from
 		/// @param[in] line The line in PcapPlusPlus code the log message is coming from
-		using LogPrinter = void (*)(LogLevel, const std::string&, const std::string&, const std::string&, const int);
+		/// @remarks The printer callback should support being called from multiple threads simultaneously.
+		using LogPrinter =
+		    std::add_pointer<void(LogLevel logLevel, const std::string& logMessage, const std::string& file,
+		                          const std::string& method, const int line)>::type;
 
 		/// A static method for converting the log level enum to a string.
 		/// @param[in] logLevel A log level enum
@@ -182,7 +270,16 @@ namespace pcpp
 		/// @return True if this module log level is "debug". False otherwise
 		bool isDebugEnabled(LogModule module) const
 		{
-			return m_LogModulesArray[module] == Debug;
+			return m_LogModulesArray[module] == LogLevel::Debug;
+		}
+
+		/// @brief Check whether a log level should be emitted by the logger.
+		/// @param level The level of the log message.
+		/// @param module PcapPlusPlus module
+		/// @return True if the message should be emitted. False otherwise.
+		bool shouldLog(LogLevel level, LogModule module) const
+		{
+			return level != LogLevel::Off && m_LogModulesArray[module] >= level;
 		}
 
 		/// Set all PcapPlusPlus modules to a certain log level
@@ -209,8 +306,9 @@ namespace pcpp
 		}
 
 		/// @return Get the last error message
-		std::string getLastError()
+		std::string getLastError() const
 		{
+			std::lock_guard<std::mutex> lock(m_LastErrorMtx);
 			return m_LastError;
 		}
 
@@ -233,17 +331,32 @@ namespace pcpp
 			return m_LogsEnabled;
 		}
 
-		template <class T> Logger& operator<<(const T& msg)
+		/// @brief Controls if the logger should use a pool of LogContext objects.
+		///
+		/// If enabled is set to false, preallocate and maxPoolSize are ignored.
+		/// @param enabled True to enable context pooling, false to disable.
+		/// @param preallocate The number of LogContext objects to preallocate in the pool.
+		/// @param maxPoolSize The maximum number of LogContext objects to keep in the pool.
+		/// @remarks Disabling the pooling clears the pool.
+		void useContextPooling(bool enabled, std::size_t preallocate = 2, std::size_t maxPoolSize = 10)
 		{
-			(*m_LogStream) << msg;
-			return *this;
-		}
+			m_UseContextPooling = enabled;
 
-		static std::ostringstream* internalCreateLogStream();
+			if (m_UseContextPooling)
+			{
+				m_LogContextPool.setMaxSize(maxPoolSize);
 
-		/// An internal method to print log messages. Shouldn't be used externally.
-		void internalPrintLogMessage(std::ostringstream* logStream, Logger::LogLevel logLevel, const char* file,
-		                             const char* method, int line);
+				if (preallocate > 0)
+				{
+					m_LogContextPool.preallocate(preallocate);
+				}
+			}
+			else
+			{
+				// Clear the pool if we're disabling pooling.
+				m_LogContextPool.clear();
+			}
+		}
 
 		/// Get access to Logger singleton
 		/// @todo: make this singleton thread-safe/
@@ -254,17 +367,74 @@ namespace pcpp
 			return instance;
 		}
 
+		/// @brief Creates a new LogContext with Info level and no source.
+		/// @return A new LogContext.
+		std::unique_ptr<internal::LogContext> createLogContext();
+
+		/// @brief Creates a new LogContext with the given level and source.
+		/// @param level The log level for this message.
+		/// @param source The log source.
+		/// @return A new LogContext.
+		std::unique_ptr<internal::LogContext> createLogContext(LogLevel level, LogSource const& source = {});
+
+		/// @brief Directly emits a log message bypassing all level checks.
+		/// @param source The log source.
+		/// @param level The log level for this message. This is only used for the log printer.
+		/// @param message The log message.
+		void emit(LogSource const& source, LogLevel level, std::string const& message);
+
+		/// @brief Directly emits a log message bypassing all level checks.
+		/// @param message The log message.
+		void emit(std::unique_ptr<internal::LogContext> message);
+
 	private:
 		bool m_LogsEnabled;
-		Logger::LogLevel m_LogModulesArray[NumOfLogModules]{};
+		std::array<LogLevel, NumOfLogModules> m_LogModulesArray;
 		LogPrinter m_LogPrinter;
+
+		mutable std::mutex m_LastErrorMtx;
 		std::string m_LastError;
-		std::ostringstream* m_LogStream{};
+
+		bool m_UseContextPooling = true;
+		// Keep a maximum of 10 LogContext objects in the pool.
+		internal::DynamicObjectPool<internal::LogContext> m_LogContextPool{ 10, 2 };
 
 		// private c'tor - this class is a singleton
 		Logger();
 
 		static void defaultLogPrinter(LogLevel logLevel, const std::string& logMessage, const std::string& file,
 		                              const std::string& method, int line);
 	};
+
 }  // namespace pcpp
+
+#define PCPP_LOG(level, message)                                                                                       \
+	do                                                                                                                 \
+	{                                                                                                                  \
+		auto& logger = pcpp::Logger::getInstance();                                                                    \
+		if (logger.shouldLog(level, LOG_MODULE))                                                                       \
+		{                                                                                                              \
+			auto ctx =                                                                                                 \
+			    logger.createLogContext(level, pcpp::LogSource(LOG_MODULE, PCAPPP_FILENAME, __FUNCTION__, __LINE__));  \
+			(*ctx) << message;                                                                                         \
+			logger.emit(std::move(ctx));                                                                               \
+		}                                                                                                              \
+	} while (0)
+
+#if PCPP_ACTIVE_LOG_LEVEL >= PCPP_LOG_LEVEL_DEBUG
+#	define PCPP_LOG_DEBUG(message) PCPP_LOG(pcpp::LogLevel::Debug, message)
+#else
+#	define PCPP_LOG_DEBUG(message) (void)0
+#endif
+
+#if PCPP_ACTIVE_LOG_LEVEL >= PCPP_LOG_LEVEL_INFO
+#	define PCPP_LOG_INFO(message) PCPP_LOG(pcpp::LogLevel::Info, message)
+#else
+#	define PCPP_LOG_INFO(message) (void)0
+#endif
+
+#if PCPP_ACTIVE_LOG_LEVEL >= PCPP_LOG_LEVEL_ERROR
+#	define PCPP_LOG_ERROR(message) PCPP_LOG(pcpp::LogLevel::Error, message)
+#else
+#	define PCPP_LOG_ERROR(message) (void)0
+#endif