block_accepted "REORG!" log msg without corresponding "fork?" log msg

[antiochp]

20191004 11:45:58.631 DEBUG grin_servers::common::hooks - block_accepted (head+): 000016855a5e at 375491 (diff: 1135976891692834)
20191004 11:45:58.768 DEBUG grin_servers::common::hooks - block_accepted (fork?): 00000140089b at 375491 (diff: 1135976891692834)
20191004 11:47:28.795 WARN grin_servers::common::hooks - block_accepted (REORG!): 000003ada72b at 375492 (depth: 1, diff: 1135979125113839)
--
20191004 23:16:59.375 DEBUG grin_servers::common::hooks - block_accepted (head+): 0000008e2697 at 376179 (diff: 1137470173506101)
20191004 23:16:59.621 DEBUG grin_servers::common::hooks - block_accepted (fork?): 00000149b113 at 376179 (diff: 1137470173506101)
20191004 23:18:35.900 WARN grin_servers::common::hooks - block_accepted (REORG!): 000003b5df09 at 376180 (depth: 1, diff: 1137472459280012)
--
20191006 05:31:57.794 DEBUG grin_servers::common::hooks - block_accepted (head+): 000001966a85 at 377972 (diff: 1141437607805950)
20191006 05:32:48.651 DEBUG grin_servers::common::hooks - block_accepted (head+): 000030adfb23 at 377973 (diff: 1141439627970597)
20191006 05:33:57.673 WARN grin_servers::common::hooks - block_accepted (REORG!): 000000560bf1 at 377974 (depth: 1, diff: 1141441644516950)
--
20191006 10:20:36.019 DEBUG grin_servers::common::hooks - block_accepted (head+): 0000038bc8b5 at 378273 (diff: 1142013270321342)
20191006 10:20:36.362 DEBUG grin_servers::common::hooks - block_accepted (fork?): 000002f6cecc at 378273 (diff: 1142013270321342)
20191006 10:21:47.338 WARN grin_servers::common::hooks - block_accepted (REORG!): 000003097655 at 378274 (depth: 1, diff: 1142015366661637)
--
20191006 12:55:07.162 DEBUG grin_servers::common::hooks - block_accepted (head+): 00003c5429f1 at 378423 (diff: 1142346012682112)
20191006 12:55:07.207 DEBUG grin_servers::common::hooks - block_accepted (fork?): 000001d7d77d at 378423 (diff: 1142346012682112)
20191006 12:55:47.469 WARN grin_servers::common::hooks - block_accepted (REORG!): 0000045cbfee at 378424 (depth: 1, diff: 1142348224989937)
--
20191007 03:06:01.610 DEBUG grin_servers::common::hooks - block_accepted (head+): 000004924eab at 379264 (diff: 1144161543864200)
20191007 03:07:25.679 DEBUG grin_servers::common::hooks - block_accepted (head+): 00003690bc2b at 379265 (diff: 1144163456992598)
20191007 03:09:15.170 WARN grin_servers::common::hooks - block_accepted (REORG!): 000004d62066 at 379266 (depth: 1, diff: 1144165353988495)

We log a REORG! if we see a chain reorg but we should always have a corresponding fork? in the logs as the reorg causes the chain head to migrate from one fork to another (and by definition we should have processed both blocks at the previous height.

I'm not entirely sure why this is happening and we should investigate further.
Not sure if this is a symptom of a larger issue or potentially just a logging issue.

[antiochp]

We broke this way back in #2349

#2349 makes the assumption that if we requested a block then we are syncing and we should suppress the header broadcast when receiving the block.
This is not correct behavior though because we can also request full blocks when failing to hydrate compact blocks. And in this case we do want to broadcast the header to our peers.

We don't see the "fork?" log msg in some of these examples because we suppressed both the logging callback and the header broadcast due to us incorrectly assuming we were syncing.

We should suppress broadcast (and callbacks) iff -

• we specifically requested this block and
• we requested it due to missing blocks during sync

If we request a block due to a compact block not hydrating successfully then we want to broadcast the header to our peers.

Rather than tracking a true/false "did we request this block?" we need to track the opts (SYNC or NONE) related to the originating block request.
During sync we will track a requested block with opts=SYNC (suppressing the broadcast).
During block hydration we will track a requested block with opts=NONE (broadcasting normally).