Add bangs for avoiding retaining the ProtocolInfo and LedgerDbArgs

[jasagredo]

The backing store tracer

As lgrTracer was lazy, and also >$< is lazy, the bsTracer was a thunk that retained the LedgerDbArgs which contain the Genesis ledger state.

The ProtocolInfo

As pInfoConfig is lazy, the codecConfig bind was remaining as a thunk and it was retaining the whole ProtocolInfo which contains the Genesis ledger state.

[bladyjoker]

Wow! I want to see you debug these issues live some day, quite cool you were able to find where the issue was.

[jasagredo]

This comment explains how I found this issue and why I think it has been solved by these changes.

The initial problem statement was that when benchmarking the node in the workbench there was not a big observable difference between the InMemory and LMDB backends. This is counterintuitive because the LMDB backend should flush all UTxOs to the backing store, so they should be gone from the heap.

Thanks to @fmaste, I got a first heap snapshot using ghc-debug, which already narrowed the problem. Thanks to @mgmeier I was able to get a workbench profile in which the nodes run for a long time, with the LMDB backend and doing "nothing" (no transactions in the network). To instrument the node, it is sufficient to add:

+import GHC.Debug.Stub

-main = do
+main = withGhcDebug $ do

and

executable cardano-node
  ...
-  build-depends:
+  build-depends: ghc-debug-stub,

For this to be useful, the libraries to be inspected need to be compiled with info-tables:

package ouroboros-consensus
  ghc-options: -finfo-table-map -fdistinct-constructor-tables

package ouroboros-consensus-diffusion
  ghc-options: -finfo-table-map -fdistinct-constructor-tables

Un-retaining the LedgerDbArgs

Once the node starts, we can connect to it with ghc-debug-brick and pause the process to inspect the heap. Searching for retainers of TxIn (Ctrl+e), the following shows up:

which already points to the LedgerDbArgs being retained. In particular it points to lgrGenesis (because of the entries above the highlighted one).

At first I thought about reworking that field to get it out of the LedgerDbArgs, but actually those args should "dissolve" once we opened the database so the issue is that the arguments themselves are retained.

Looking at the entry just below the highlighted one, we see the following description (see top of the screenshot):

So that already points to the bsTracer. Adding a bang there still didn't fix it, and adding a bang to lgrTracer and bsTracer also didn't fix it, I suspect because >$< might also induce undesired laziness. Forcing lgrTracer with the !tr bind fixes the issue.

Un-retaining the ProtocolInfo

If we run the node again, and search for TxIn retainers now we see the following heap:

which shows another retainer chain, this time ProtocolInfo. We can see that the field that retains the Genesis is pInfoInitLedger as we can see above the highlighted entry.

But once again, the ProtocolInfo should dissolve once we started the node. This time looking at the entries below ProtocolInfo we can see that the CodecConfig is being retained.

This time pInfoConfig is lazy, so adding a bang there and also in the codecConfig binding solves the issue.

Note: if we let the node run until it connects to other peers, it will force the CodecConfig eventually so the ProtocolInfo (and with it the Genesis, and with it all the TxIns) will disappear, as it can be seen in this heap from a node that already established connections:
In any case I think it is worth it forcing this value just in case.