node-red-flexdash

Node-RED nodes to interface with FlexDash

FlexDash is a self-contained web dashboard for Node-RED and IoT devices. This package provides Node-RED nodes that form the core of a FlexDash integration. These nodes provide the basis for communicating with FlexDash but they do not make any of the built-in widgets available. For Node-RED nodes corresponding to the built-in widgets use the Node-RED FD CoreWidgets package.

Under the hood the nodes here use Socket.IO and the dashboard(s) can use Node-RED's web server (i.e. the same port) or a different port.

For help, please read the FlexDash Docs and check the Node-RED forum.

Installation

TL;DR: You most likely do not want to explicitly install this package, you want to install the core widgets, which will bring in this package and more and will provide a usable whole.

npm i @flexdash/node-red-fd-corewidgets

If you really want to install node-red-flexdash, e.g., to get a specific version:

npm i @flexdash/node-red-flexdash

You also do not need to install FlexDash, it comes bundled in the node-red-flexdash package.

Pre-release (dev) versions are published on npm using a "dev" tag:

• latest dev version: npm i @flexdash/node-red-flexdash-plugin@dev
• specific dev version: npm i @flexdash/[email protected]

Internals

The main interface with FlexDash is flexdash-dashboard.*, which handles serving up FlexDash and handling messages to/from the dashboard. It also exposes a config node to represent the dashboard connection. There can be multiple dashboards as long as they use different paths or ports.

The Node-RED plugin in flexdash-plugin/* is manages the hierarchy of FlexDash objects, e.g. tabs, grids, panel, widget. It is a Node-RED plugin so node-red widget nodes can call "into FlexDash" without having to figure out how to get a handle onto the appropriate flexdash-dashboard config node first. The plugin is in a subdirectory and forms a separate NPM package due to a bug in Node-RED.

The config nodes in flexdash-tab.* and flexdash-container.* represent containers for widgets in the dashboard. flexdash-container can represent a grid or a panel. A Widget either belongs to a grid or to a panel which itself belongs to a grid. A grid belongs to a tab and there can be multiple tabs per dashboard.

The development server in flexdash-dev-server.* is a node that can be placed anywhere to launch a development server for FlexDash widgets. It runs Vite, which is a web server that automatically pushes source code to the web browser as soon as you save a file (hot module reload). To access the dev server, once launched, point your browser at the dashboard URL plus a -dev suffix, e.g. /flexdash-dev instead of /flexdash for the default set-up. (Note: the dev server feature should be moved into a side-bar eventually.)

The flexdash-in.* and flexdash-out.* nodes are not currently supported. They send/receive raw messages to/from FlexDash which supports advanced usage. However, the current saving of dashboard configuration changes does not really support such advanced usage, so these nodes are not currently exported.

Some relevant Node-RED internals

• The flow editor operates on node configs that it passes to the runtime.
• The runtime has the node configs, but then operates on node objects, i.e. instantiations of the configs.
• The flow config can be traversed in the runtime using RED.nodes.eachNode, eachConfig, eachSubflow, etc. Confusingly RED.nodes.node is to query only active instantiated nodes!
• There are nodes in the config that are not instantiated: nodes in disabled flows and nodes in subflows are two examples.
• The set of active/instantiated flows cannot be traversed (no eachXxx function).
• There are nodes in the active set that are not in the config: nodes in instances of subflows are one example.
• Everything is a node: a flow, a subflow, a subflow instance, a node, a config node, etc.
• Special fields:
  • .z refers to the flow in which a node is located, it doesn't exist in global config nodes
  • ._alias in runtime is found in nodes in subflows and refers to the config (template) that the node was instantiated from
  • .d is true(?) for nodes in disabled flows
• Subflows:
  • The subflow "template" is represented by a type=subflow node.
  • A node type subflow:<subflow node id> is created for each subflow.
  • Subflow instance nodes are of that type
  • For each node in a subflow template a new node is instantiated on deploy, its ._alias has the ID of the node template.
  • The template nodes are not instantiated in the runtime, they remain solely as configs, i.e., no constructor gets called.

Flows

This section describes how Node-RED-FlexDash represents dashboards, tabs,. grids, widgets, etc. and integrates with the Node-RED concepts of flows, config nodes, nodes, etc.

Dashboard and Tab config nodes

• Dashboards and tabs are represented by config nodes in Node-RED.
• A dashboard's fd_children is a list of tab node IDs.
• A tab has the dashboard's config node ID in fd, and fd_children contains a list of grids.
• For tabs the FlexDash ID generated is t followed by the Node-RED node ID.
• Thanks to a hack (see the code in flexdash-tab.html) Node-RED is not aware of the parent/child relationship between dashboards and tabs, which has the effect that exporting a bunch of nodes from a flow does not export the dashboard they are children of. This is good because on import, having a new dashboard created is almost never what the user wants.

Grid and Panel config nodes

• Both are represented by nodes of type "flexdash container" in Node-RED. See below for the rationale.
• A Grid's tab contains the ID of the tab config node to which it belongs.
• A Panel's parent contains the ID of the grid config node to which it belongs.
• Their fd_children contains a list of widget node IDs.
• A grid's fd_children may also contain panel config node IDs.
• For grids the FlexDash ID generated is g followed by the Node-RED node ID, and for panels the prefix is w (because panels are really a special form of widget in FlexDash).

Widgets and Array widgets

• A widget node's fd_container contains the ID of the grid or panel to which it belongs.
• The reason that grids and panels are both of type "flexdash container" (as opposed to having a "flexdash grid" and a "flexdash panel") is that this is required for the "container" drop down in a widget to include both grids and panels. I.e., it is not possible to ask Node-RED to provide a drop-down with two types of config nodes.
• For non-array widgets the generated FlexDash ID is w + Node-RED node ID
• For widgets that are arrays the FlexDash IDs generated are w + Node-RED node ID + | + the array topic.

Widget fields

• dyn_root: node-red/<widget-id> the topic path in FLexDash to which a widget should listen
• output: nr/<node-id> the topic in FlexDash that a widget should use for its output messages
• group: <array-node-id> a grouping of array widgets to allow moving them as a group

Subflows

Node-RED FlexDash supports one level of subflows, that is, widget nodes can be placed in a subflow and that subflow can be instantiated many times in flows, but it cannot be instantiated/nested in another subflow. In addition, all the widgets in a subflow must be placed into a SubflowPanel, i.e., associated with a SubflowPanel config nodes and they will then appear in a panel in FlexDash.

Subflows are quite complicated because in the flow editor (and the "config" saved and passed into the run-time) there are only the subflow "template" nodes and then a subflow instance node for each instance of a subflow. The run-time then expands each subflow instance into a full copy of the subflow template, so new nodes are created (with new IDs at each deploy!) and those are the ones that exist in the run-time as "nodes" while the template nodes only exist as "config".

At a high level, the relationships in the flow editor are as follows:

• in the flow editor the user creates a subflow and places widgets into the subflow
• the user creates a subflow panel and associates it with the subflow (there must be a 1-1 correspondence), and associates all the widgets in the subflow with that panel.
• the user then creates one or more subflow instances in various flows and associates each instance with a grid

In the end, the FlexDash hierarchy is as follows:

• each widget belongs to a subflow panel
• the subflow panel belongs to a subflow
• the subflow is instantiated one or multiple times
• each instance belong to a grid
• the grid belongs to a tab, which belongs to a dashboard

In the runtime all this stuff further gets instantiated: each instance of the subflow results in a new set of widgets nodes and a new subflow panel node. The subflow instance node remains as-is and the subflow itself is only present in the config.

The IDs of FlexDash widgets (incl. subflow panels) is w + the Node-RED ID, which means that on every Node-RED deploy the FlexDash widgets are rebuilt from scratch. This is not ideal, but the alternative would add quite a bit of complexity.

Subflow

• A subflow is represented by a subflow config in the flow editor, this config is not instantiated as a node in the run-time.

Subflow instance

• A subflow instance config refers to the subflow it instantiates by the type, specifically, the type is subflow:<config id> where the ID is that of the subflow config.
• The subflow instance config's z refers to the flow in which it is placed.
• The subflow instance is instantiated in the run-time.
• A subflow instance is associated with a grid via a flexdash_grid env variable.

Subflow panel

• A subflow panel is a Node-RED config node of type "flexdash container".
• It must be associated with a subflow in the flow editor, which is represented in z.
• The panel's fd_children contains the IDs of the widget nodes it contains, however, these are the IDs of the template nodes, which are then instantiated with fresh IDs for each subflow instance...

Subflow panel instance

• When a subflow is instantiated an associated subflow panel is also instantiated as a node in the run-time.
• The subflow panel instance refers to the subflow instance using z and to its template node config using _alias.

Subflow widget

• A config in a subflow refers to a subflow panel using fd_container (like regular widgets).
• It gets instantiated as widget node for every subflow instance and then refers to the subflow instance using z, and to its template node config using _alias.

fd_container / fd_children

A widget config's fd_container refers to the subflow panel config, not to the subflow panel instance node. Similarly, a subflow panel's fd_children contains a list of subflow widget configs.

A subflow panel or a subflow panel instance do not have an fd_container. Instead, the containing grid is found in the subflow instance flexdash_grid env variable. The grid's fd_children contains the subflow instance ID.

Disabled flows

When flows are disabled the nodes simply don't show up in the runtime. This means that fd_children has IDs that are missing and look like they were deleted.

The config coming out of the flow editor needs to ensure that all nodes referenced in the "children" list exist (in the flow editor). Then in the run-time, missing nodes found in the children list are assumed to be disabled and are marked as such, but not removed.

When altering the position of widgets in the dashboard, the missing widgets need to be kept. When the config is pushed back into the flow editor, the missing widgets should be reconnected with the disabled nodes.

• the pruning of deleted nodes in children lists must only happen in the flow editor

Widgets in disabled flows end up with an ID starting with an 'x' to signal to FlexDash that these are to be skipped.

Dirty laundry

Node-red-flexdash uses a bunch of hacks to work around problems in Node-RED. I'm listing them here in case I get asked which specific ones :-) :

• the annoying node-red-flexdash-plugin separation is necessary due to node-red/node-red#3523
• panels and grids use the same config node type because in a drop-down one can only have config nodes of one type
• tabs do some hacky hiding of the dependency to the dashboard so the latter is not part of an export of a couple fo nodes
• the whole dependency tracking of which widgets belong to a panel/grid, which grids belong to a tab, etc. is a total nightmare due to there not being any signal that everything has been loaded, which makes it virtually impossible to prune dead nodes. https://discourse.nodered.org/t/new-editor-event-when-all-nodes-have-been-loaded/60314
• the edit panel for a subflow instance node with a FlexDash SubFlowPanel is monkey patched so one can select the grid/panel to display the node in (fd_container config node), this is necessary because one cannot create an env variable to select a config node
• the "general" tab in the node edit panel is hacked into the (hidden) DOM before a node is created but something like this would be much cleaner if done in oneditprepare, but that's too late to get the current values filled-in.
• the fact that nodes can only depend on config nodes and that config nodes can't send/receive messages (i.e., they can't appear in flows) means a "flexdash ctrl" node is necessary
• the whole notion of dependency tracking between nodes (incl. config nodes) is a mess, more in the flow editor than the run-time, but even there it's pretty murky when a node ID definitely refers to something that no longer exists. Also, the flow editor tracks "users" of a config node, that info is lost in the run-time (and has to be reconstructed by FD).

License

MIT