Skip to content

Commit

Permalink
blockchain: Convert to direct single-step reorgs.
Browse files Browse the repository at this point in the history
This modifies the chain reorganization logic to directly perform the
reorg one block at a time with rollback in the case of failure, as
opposed to the existing memory-based two-step approach, so that it is
more optimized for the typical case, better handles large reorgs, gives
the ability to implement better caching strategies, and helps provide a
path to decouple the chain processing and connection code from the
download logic.  It also removes the cached stxos from the view since
the aforementioned changes make them no longer necessary.

A side effect of these changes is that it is no longer possible to know
if a reorg will succeed before actually performing it, so the
NTReorganization notification is now sent after a successful reorg.  The
notification really should have been sent after the reorg before anyway.

Prior to these changes, chain reorganization used a two-step approach
such that the first step involved checking all of the blocks along the
reorg path in memory and then actually performing the reorg in a second
step if those checks succeeded.  While that approach does have some
benefits in terms of avoiding any intermediate mutation to the current
best chain for failed reorgs, and thus not requiring a rollback in that
case, it also has some disadvantages such as not scaling well with large
reorgs, being more difficult to make use of different caching
strategies, and hindering the ability to decouple the connection code
from the download logic.

In a certain sense, the approach this replaces assumed that a reorg
would fail and took measures to detect that condition prior to
performing the reorg, while the new approach assumes the reorg will
succeed and rolls back the changes in the very rare case it doesn't.
This is an acceptable and safe assumption because the proof-of-work
requirements make it exceedingly expensive to create blocks that are
valid enough to trigger a reorg yet ultimately end up failing to
connect, thus miners are heavily disincentivized from creating such
invalid blocks and attackers are also unable to easily create such
blocks either.  Even in the case of attack, the only result would be
nodes performing slightly more database updates than the existing
approach.

The following results show the difference between performing the large
reorg full block tests before and after these changes:

before: 4.3GB memory usage, 2m18.629s to complete
after:  2.8GB memory usage, 2m04.056s to complete

As can be seen, the new approach takes much less memory and is also a
bit faster as well.
  • Loading branch information
davecgh committed Nov 12, 2018
1 parent df1898c commit 9c4569a
Show file tree
Hide file tree
Showing 3 changed files with 180 additions and 289 deletions.
Loading

0 comments on commit 9c4569a

Please sign in to comment.