Further incremental improvements to path finding memory usage

[ximinez]

High Level Overview of Change

These changes continue building on top of PR #4099, which in turn builds on the changes from PR #4080. Both of those PRs are approved to merge. Thus, reviewers can skip straight to the commit labelled, "Don't load trust lines that can't participate in path finding".

• Reduce memory requirements for RippleState #4080 reduces the memory requirements per trust line in memory
• Incremental improvements to path finding memory usage #4099 speeds up the removal of trust lines and path finding requests from memory
• This PR significantly reduces the number of trust lines that will be loaded and held in memory in the first place.

Context of Change

As described in the other two PRs, the number of trustlines on the network has grown significantly over the last few months. That in-turn has led to a significant increase in the amount of memory used to load and cache those trustlines.

Type of Change

• [X ] New feature (non-breaking change which adds functionality)
• [X ] Refactor (non-breaking change that only restructures code)

Before / After

These changes take advantage of two facts:

1. "A path is considered invalid if and only if it enters and exits an address node through trust lines where No Ripple has been enabled for that address." (https://xrpl.org/rippling.html#specifics)
2. Most "end-user" accounts / wallets do not have the "default ripple" flag set, and thus have the NoRipple flag set on all of their trustlines.

Before

Currently, during path finding, all trust lines which might link the source account to the destination account are examined and cached. The path finding engine checks for invalid paths ("enters and exits an address node through trust lines where No Ripple has been enabled"), and does not use any of the cached trust lines to look for more paths, but the trust lines remain cached, potentially for a rather long time.

After

After these changes, during path finding, the trust line lookups for a given account consider the state of the no ripple flag on that account's side of the trust line used to find that account. If not set (rippling is allowed), the account is considered "outgoing", and all trust lines are loaded and cached. However, if the no ripple flag is set (rippling is not allowed), the account is considered "incoming", and only trust lines which do not have the no ripple flag set (thus allowing rippling) are loaded and cached.

Example

Consider the case where 1000 wallets all have trustlines to the same 100 issued tokens. Alice and Bob are among those 1000. If Alice tries to find a path to Bob, with the current behavior: Alice's 100 trust lines will be loaded and cached. The first token line will be considered, leading to the token issuer. The token issuer's 1000 trust lines will be loaded and cached. Then for each of those 999 other wallet accounts, the 100 trust lines to each issuer will be loaded and cached. Unfortunately, none of those trust lines will be usable in a path. This effectively leads to 99,900 unusable cached trust lines. In this example, the second token issuer will lead to the same set of 1000 accounts, but on the mainnet, there are more accounts trusting more tokens without necessarily overlapping, leading to even bigger data sets.

With these changes, each time a wallet account is considered, no trust lines will be loaded or cached, preventing those 99,900 unusable trustlines from taking up memory. (Under the hood, the SLE will be loaded regardless, but no PathFindTrustLines will be created. In the case where there are 0 usable trust lines found, the std::vector will not even be cached.)

Additionally, the changes handle the case where an account has some trust lines with the no ripple flag and without, and avoids duplication.

Test Plan

This test plan is similar to the one for #4099

Run an expensive path_find request. Observe that less memory is used by the rippled process overall. Using the get_counts command, observe that fewer instances of PathFindTrustLine are being created than were created for #4099 and older. (Note that the counter for PathFindTrustLine was only added in #4080.)

[seelabs]

Very nice optimization! I like this a lot.

Since we're filtering out trustlines, we should make sure there are tests that have an incoming trustline set to ripple and an outgoing trusline set to no ripple that tests whether the pathfinder still finds that path. If we don't have such a test can we add one?

[seelabs]

Adding the true param makes this line harder to understand. I'd either add an inline comment getRippleLines(account, /*outgoing*/true), make a named variable, or make the parameter an enum. I probably vote for an enum, but whatever you decide is fine with me. (ditto for the other calls to this function).

[ximinez]

Done

[ximinez]

Since we're filtering out trustlines, we should make sure there are tests that have an incoming trustline set to ripple and an outgoing trusline set to no ripple that tests whether the pathfinder still finds that path. If we don't have such a test can we add one?

I just pushed a commit that adds a test case to test all four combinations of the incoming and outgoing flags.

[seelabs]

👍 Great job on this!

[ximinez]

This PR was incorrectly closed by #4127, so I reopened it.