Merge pull request #10 from rfoxkendo/upstream-merge-test

Provide fribdaq-readout to upstream repository.

Author: flueke

extras/CMakeLists.txt

@@ -8,6 +8,7 @@ add_subdirectory(mini-daq)
 add_subdirectory(mvlc-ctrl-tests)
 add_subdirectory(dev-tools)
 add_subdirectory(mvlc-cli)
+add_subdirectory(fribdaq)
 
 if (UNIX AND NOT APPLE)
     add_executable(netlink-test netlink-socket-mem-monitoring/netlink-test.cc)

extras/fribdaq/CMakeLists.txt

@@ -0,0 +1,8 @@
+#cmake_minimum_required(VERSION 3.13)
+#project(fribdaq-readout)
+
+# NSCLDAQ_ROOT  - defines where the NSCLDAQ software is installed
+# MVLC_ROOT     - Defines where the Mesytec mvlc software is installed.
+
+
+add_subdirectory(src)

extras/fribdaq/Readme.md

@@ -0,0 +1,129 @@
+### Introduction
+
+This directory supplies an optional program, fribdaq-readout that is based on minidaq, but can write data to an FRIB/NSCLDAQ ring buffer.  To enable building and installing the program you must add
+
+```
+ -DNSCLDAQ_ROOT=nscldaqroot
+```
+
+where ```nscldaqroot``` is the top level directory of the FRIB/NSCLDAQ installation you want this to link against.  The softare was tested with FRIB/NSCLDAQ version ```12.1-007```, however it should operate with any version at ```11.0``` or above.
+
+when the full mvlc support software is built, the installation ```bin``` directory will include the program ```fribdaq-readout```
+
+
+### Using fribdaq-readout
+
+```fribdaq-readout``` supports the options supported by minidaq
+(use --help to list the full option set and short form documentation)  with the added options:
+
+* ```--ring ringname```  selects the name of the ring buffer to which the data should be written.  This is a ring name not a URI, as only local ringbuffers can be written to.  If the option is not provided, this defaults to the name of the logged in user.
+* ```--sourceid id``` When used in a larger system that builds events from several sources using the FRIB/NSCLDAQ event builder, this specifies the unique integer source id  that will be used to identify fragments from this data source.   If not provided, this defaults to 0.
+* ```--timestamp-library``` When used with a larger system that builds events from several sources using the FRIB/NSCLDAQ event builder, this specifies a shared library file which includes code to extract timestamps from the raw event data.   If not provided, the body headers required to build events will not be included in the ```PHYSCIS_EVENT``` items.
+
+Following all options on the command line, a single positional parameter provides the name of a .yaml configuration file that was either exported from ```mdaq``` or produced using the FRIBDA/NSCLDAQ configuration tools (to be included in versions 12.2 of FRIB/NSCLDAQ).
+
+####  fribdaq-readout commands
+
+Once the program has been started successfully, it will enter a command processing loop.  This loop recognizes the following, very simple commands, which can be provided in either lower case or upper case (but not [yet] mixed  case):
+
+*  ```SETRUN run-number``` sets the number of the next run. An positive integer *run-number* is the sole parameter of this command.
+This command is only accepted if the run is halted.
+* ```TITLE The title text``` sets the title to be used for the next run.  The remainder of the command line will be used as the title. This command is only accepted if the run is halted.
+* ```BEGIN``` starts taking data (begins a new run).  A data format item and a BEGIN state change item are emitted to the ringbuffer prior to any other data.  This command is only accepte if the run is halted.
+* ```PAUSE``` Pauses an active run.  After  pausing, a format item and a PAUSE state change item are emitted to the ringbuffer.
+* ```RESUME``` Resumes a paused run.  Before resuming, a format item
+and a RESUME state change item are emitted to the ringbuffer.  This command is only accepted if the run is paused.
+* ```END``` Ends a run.  This command is only acceeptd if the run is active or paused.  When the run is ended, a format item and END state change item will be emitted to the ringbuffer.
+
+
+#### Creating setups with mdaq.
+
+The fribdaq-readout program makes some assumptions about the configuration it has been given:
+
+* Stack 0 in mdaq is assumed to be PHYSICS_EVENT data.  It will generate PHYSICS_EVENT ring items.
+* Stack 1 is assumed to be data from periodic scalers.  It will generate PERIODIC_SCALER ring items.
+
+FRIB/NSCLDAQ setup software will generate appropriate stack configurations with the event trigger a pulse in NIM1.
+
+Note that if you use mdaq you should remove the MCST sections of the configurations as multiple instances will cause start errors at this time.
+
+#### Using fribdaq-readout with the FRIB/NSCLDAQ event builder.
+
+To use the fribdaq-reaout with the FRIB/NSCLDAQ event builder, you must supply code to extract timestamps from the data.   This code is dynamically loaded from a shared object.  The shared library must have a  C bindings entry point named ```extract_timestamp```
+that returns a uint64 timestamp.  Input parameters are:
+*  unsigned nmodules The number of module data structs in event below,
+*  const mesytec::mvlc::readout_parser::ModuleData* event a pointer to the first module data in the event.
+
+Here is a sample timestamp extractor that just returns an incrementing trigger number:
+
+```c++
+#include <fribdaq/parser_callbacks.h>
+#include <stdint.h>
+// Timestamp extractors must have C linkage:
+extern "C" {
+    /** Sample timestamp extractor
+        normal extractors need to look at the data which are
+        in the module structs
+     */
+    uint64_t
+    extract_timestamp(unsigned numModules, const mesytec::mvlc::readout_parser::ModuleData* event) {
+        static uint64_t timestamp = 0;
+
+        return timestamp++;
+    }
+}
+```
+
+*  Note the use of extern "C" to force the bindings to C rather than C++ bindings.  If you actually compile your extractor in C you should not do this.
+*  The parser_callbacks.h provides definitions need by the parser callbacks and also pulls in the Mesytec headers that define the ModulData structure.  Module data supplies the data for each module in the mdaq configuration in order from first to last.  It has a few fields but the ones you usually care about are:
+    * size - which is th number of uint32_t items in the data for this module
+    * data - a void pointer to the data read from the module.
+
+In a typical actual extractor the first 64 bits of the first module will typically be the timestamp. so the last line of the example might be:
+```c++
+...
+    const uint64_t* pTs = reinterpret_cast<const uint64_t*>(event->data);
+    return *pTs;
+...
+```
+
+#### Event format.
+
+Recall that stack 0 is used by fribdaq-readout for event data.  It is packaged in a ringbuffer using a normal CPhysicsEventItem.  If a timestamp extractor is provided, the item will have a body header.  If not it won't.  The body of the event will consist just of the data from each ModuleData packed end-to-end in the payload of the ring item.
+
+Here is the code fragment that creates and submits ring items:
+
+```c++
+
+...
+std::unique_ptr<CPhysicsEventItem> pEvent;
+    if (context->s_tsExtractor) {
+        pEvent.reset(new CPhysicsEventItem(
+            context->s_tsExtractor(moduleCount, pModuleDataList), context->s_sourceid, 0,
+            eventSize + 100
+        ));
+    } else {
+        pEvent.reset(new CPhysicsEventItem(eventSize + 100));
+    }
+    CPhysicsEventItem& event(*pEvent);   
+
+    for ( int i  = 0; i < moduleCount; i++) {
+        uint32_t* pCursor = reinterpret_cast<uint32_t*>(event.getBodyCursor());
+        auto size = pModuleDataList->data.size;
+        memcpy(pCursor, pModuleDataList->data.data, size * sizeof(uint32_t));
+        pCursor += size;
+        event.setBodyCursor(pCursor);
+    }
+    event.updateSize();
+
+    event.commitToRing(*(context->s_pRing));
+...
+```
+
+where context is a struct that contains information like the source id, the ring item, and the timestamp extractor function pointer.
+
+See the FRIB/NSCLDAQ documentation for CPhysicsEventitem for more information.   
+
+The work of putting data into the ring item body  is done in the for loop which iterates over the modules ModuleData items passed in and copies the data associated with each module into the ring item body.  Note that the size of the body can be gotten from a reconstituted object by calling the object's getBodySize method.  
+
+

extras/fribdaq/src/CMakeLists.txt

@@ -0,0 +1,57 @@
+
+set(NSCLDAQ_ROOT "_"  CACHE STRING "_")
+
+
+# Only build any of this stuff if NSCLDAQ_ROOT has been defined.
+# 
+
+if (NSCLDAQ_ROOT STREQUAL "_")    # not defined.
+    
+else ()
+    
+    add_executable(fribdaq-readout 
+    fribdaq_readout.cc
+    command_parse.cc
+    username.cc
+    parser_callbacks.cc
+    )
+    target_include_directories(
+        fribdaq-readout PRIVATE
+        ${CMAKE_CURRENT_SOURCE_DIR}
+        ${NSCLDAQ_ROOT}/include
+    )
+    target_link_libraries(fribdaq-readout
+        PRIVATE mesytec-mvlc
+        PRIVATE BFG::Lyra
+        PRIVATE spdlog::spdlog
+        PRIVATE ${NSCLDAQ_ROOT}/lib/libdataformat.so
+        PRIVATE ${NSCLDAQ_ROOT}/lib/liburl.so
+        PRIVATE ${NSCLDAQ_ROOT}/lib/libdaqshm.so
+        PRIVATE ${NSCLDAQ_ROOT}/lib/libDataFlow.so
+        PRIVATE ${NSCLDAQ_ROOT}/lib/libException.so
+    )  
+    target_link_options(fribdaq-readout PRIVATE
+        -ldl -Wl,-rpath=${NSCLDAQ_ROOT}/lib
+    )  
+    install(TARGETS fribdaq-readout)
+
+    # parser_callbacks.h must be installed.
+    # We'll put it in ${CMAKE_INSTALL_INCLUDEDIR}/fribdaq
+
+    set_target_properties(fribdaq-readout PROPERTIES PUBLIC_HEADER parser_callbacks.h)
+    install(TARGETS fribdaq-readout
+        PUBLIC_HEADER DESTINATION "${CMAKE_INSTALL_INCLUDEDIR}/fribdaq"
+    )
+
+    # Tests:
+
+    add_executable(parse_test parse_test.cc command_parse.cc)
+    target_link_libraries(parse_test
+            PRIVATE gtest
+            PRIVATE gtest_main
+    )
+    add_test(NAME parse_test COMMAND parse_test)
+endif ()
+
+
+

extras/fribdaq/src/command_parse.cc

@@ -0,0 +1,133 @@
+/**
+ * @file command_parse.cc
+ * @brief interface for stupid command parser.
+ * @author Ron Fox <fox @ frib dot msu dot edu>
+ * 
+ *
+ *   This software is Copyright by the Board of Trustees of Michigan
+ *   State University (c) Copyright 2025.
+ *
+ *   You may use this software under the terms of the GNU public license
+ *   (GPL).  The terms of this license are described at:
+ *
+ *    http://www.gnu.org/licenses/gpl.txt
+ * 
+ * @note - this file is private to fribdaq's readout 
+ * @note - this is  extracted from fribdaq_readout.cc to allow unit tests to be run
+ * against it.
+ */
+#include "command_parse.h"
+#include <sstream>
+#include <algorithm>
+#include <ctype.h>
+#include <map>
+
+
+// Map between command words and enum
+static std::map<std::string, StdinCommands> commandMap = {
+    {"BEGIN", BEGIN}, {"begin", BEGIN},
+    {"EXIT", EXIT}, {"exit", EXIT},
+    {"END", END}, {"end", END},
+    {"PAUSE", PAUSE}, {"pause", PAUSE},
+    {"RESUME", RESUME}, {"resume", RESUME},
+    {"TITLE", TITLE},  {"title", TITLE},  // Remainder of line is a title. 
+    {"SETRUN", SETRUN}, {"setrun", SETRUN} // Remainder of line is a + integer run number.
+};
+
+/* trim leading/trailing whitespace from a string: */
+static std::string
+trim(std::string& str) {
+    std::string result(str);  
+    
+    // Trim the front:
+
+    result.erase(result.begin(), std::find_if(result.begin(), result.end(), [](unsigned char ch) {
+        return !std::isspace(ch);
+    }));
+
+    // Trim from end:
+
+    result.erase(std::find_if(result.rbegin(), result.rend(), [](unsigned char ch) {
+        return !std::isspace(ch);
+    }).base(), result.end());
+
+    return result;
+}
+/*  True if the parameter is a string that is only whitespace */
+bool
+isBlank(const std::string& str) {
+    return std::all_of(str.begin(), str.end(), isspace);
+}
+/*
+ *  parseCommand implementation.
+ */
+ParsedCommand
+parseCommand(const std::string& line) {
+    ParsedCommand parsed;
+
+    // Get the command word and map it to a command value:
+
+    std::stringstream words(line); 
+    std::string commandword;
+    std::string remainder;
+    words >> commandword;
+    if (commandMap.find(commandword) == commandMap.end()) {
+        parsed.s_error ="Invalid command keyword";
+        // invalid (empty goes here too):
+        return parsed;   //   Initializes to invalid
+    }
+    parsed.s_command = commandMap[commandword];
+    std::getline(words, remainder);
+    switch (parsed.s_command) {
+    case BEGIN:
+    case END:                           // These should have at most a whitespace remainder:
+    case PAUSE:
+    case RESUME:
+    case EXIT:
+        if (!isBlank(remainder)) {
+            parsed.s_command = INVALID;
+            parsed.s_error = "Unexpected characters following the command keyword";
+        }
+        break;
+    case TITLE:                        // Remainder is the arg.
+        parsed.s_stringarg = remainder;
+        if (isBlank(remainder)) {
+            parsed.s_command = INVALID;
+            parsed.s_error = "TITLE needs a title string tail";
+        } else {
+            parsed.s_stringarg = trim(remainder);
+        }
+        break;
+    case SETRUN:  {                    // Remainder is a postitive integer.
+            
+            std::stringstream tail(remainder);
+            remainder = "";        // Since getline and end won't fill this.
+            tail >> parsed.s_intarg;
+            if (tail.bad() || tail.fail()) {
+                parsed.s_command = INVALID;
+                parsed.s_error = "SETRUN needs a run number";
+            } else {
+                // Run must be > 0:
+
+                if (parsed.s_intarg <= 0) {
+                    parsed.s_command = INVALID;
+                    parsed.s_error = "SETRUN's run number must be > 0";
+                } else {
+                    // The remainder now must be empty:
+
+                    getline(tail, remainder);
+                    
+                    if (!isBlank(remainder)) {
+                        parsed.s_command = INVALID;
+                        parsed.s_error = "Unexpected characters following the command keyword";
+                    }
+                }
+            }
+        }
+        break;
+    default:                                      // Error to land here:
+        parsed.s_command = INVALID;
+        parsed.s_error = "The command parser has an error and took a case it should not have";
+    }
+    return parsed;
+}