Optimise HTTP validator lookups (sigp#3559)

## Issue Addressed

While digging around in some logs I noticed that queries for validators by pubkey were taking 10ms+, which seemed too long. This was due to a loop through the entire validator registry for each lookup.

## Proposed Changes

Rather than using a loop through the register, this PR utilises the pubkey cache which is usually initialised at the head*. In case the cache isn't built, we fall back to the previous loop logic. In the vast majority of cases I expect the cache will be built, as the validator client queries at the `head` where all caches should be built.

## Additional Info

*I had to modify the cache build that runs after fork choice to build the pubkey cache. I think it had been optimised out, perhaps accidentally. I think it's preferable to have the exit cache and the pubkey cache built on the head state, as they are required for verifying deposits and exits respectively, and we may as well build them off the hot path of block processing. Previously they'd get built the first time a deposit or exit needed to be verified.

I've deleted the unused `map_state` function which was obsoleted by `map_state_and_execution_optimistic`.

Author: michaelsproul

beacon_node/beacon_chain/src/canonical_head.rs

@@ -654,11 +654,11 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
                     })
                 })
                 .and_then(|mut snapshot| {
-                    // Regardless of where we got the state from, attempt to build the committee
-                    // caches.
+                    // Regardless of where we got the state from, attempt to build all the
+                    // caches except the tree hash cache.
                     snapshot
                         .beacon_state
-                        .build_all_committee_caches(&self.spec)
+                        .build_all_caches(&self.spec)
                         .map_err(Into::into)
                         .map(|()| snapshot)
                 })?;

beacon_node/http_api/src/lib.rs

@@ -668,17 +668,34 @@ pub fn serve<T: BeaconChainTypes>(
                 "Invalid validator ID".to_string(),
             ))
         }))
+        .and(log_filter.clone())
         .and(warp::path::end())
         .and_then(
-            |state_id: StateId, chain: Arc<BeaconChain<T>>, validator_id: ValidatorId| {
+            |state_id: StateId, chain: Arc<BeaconChain<T>>, validator_id: ValidatorId, log| {
                 blocking_json_task(move || {
                     let (data, execution_optimistic) = state_id
                         .map_state_and_execution_optimistic(
                             &chain,
                             |state, execution_optimistic| {
                                 let index_opt = match &validator_id {
                                     ValidatorId::PublicKey(pubkey) => {
-                                        state.validators().iter().position(|v| v.pubkey == *pubkey)
+                                        // Fast path: use the pubkey cache which is probably
+                                        // initialised at the head.
+                                        match state.get_validator_index_read_only(pubkey) {
+                                            Ok(result) => result,
+                                            Err(e) => {
+                                                // Slow path, fall back to iteration.
+                                                debug!(
+                                                    log,
+                                                    "Validator look-up cache miss";
+                                                    "reason" => ?e,
+                                                );
+                                                state
+                                                    .validators()
+                                                    .iter()
+                                                    .position(|v| v.pubkey == *pubkey)
+                                            }
+                                        }
                                     }
                                     ValidatorId::Index(index) => Some(*index as usize),
                                 };

beacon_node/http_api/src/state_id.rs

@@ -155,33 +155,12 @@ impl StateId {
         Ok((state, execution_optimistic))
     }
 
-    /*
     /// Map a function across the `BeaconState` identified by `self`.
     ///
+    /// The optimistic status of the requested state is also provided to the `func` closure.
+    ///
     /// This function will avoid instantiating/copying a new state when `self` points to the head
     /// of the chain.
-    #[allow(dead_code)]
-    pub fn map_state<T: BeaconChainTypes, F, U>(
-        &self,
-        chain: &BeaconChain<T>,
-        func: F,
-    ) -> Result<U, warp::Rejection>
-    where
-        F: Fn(&BeaconState<T::EthSpec>) -> Result<U, warp::Rejection>,
-    {
-        match &self.0 {
-            CoreStateId::Head => chain
-                .with_head(|snapshot| Ok(func(&snapshot.beacon_state)))
-                .map_err(warp_utils::reject::beacon_chain_error)?,
-            _ => func(&self.state(chain)?),
-        }
-    }
-    */
-
-    /// Functions the same as `map_state` but additionally computes the value of
-    /// `execution_optimistic` of the state identified by `self`.
-    ///
-    /// This is to avoid re-instantiating `state` unnecessarily.
     pub fn map_state_and_execution_optimistic<T: BeaconChainTypes, F, U>(
         &self,
         chain: &BeaconChain<T>,

consensus/state_processing/src/per_block_processing/verify_deposit.rs

@@ -29,9 +29,7 @@ pub fn verify_deposit_signature(deposit_data: &DepositData, spec: &ChainSpec) ->
 /// Returns a `Some(validator index)` if a pubkey already exists in the `validators`,
 /// otherwise returns `None`.
 ///
-/// ## Errors
-///
-/// Errors if the state's `pubkey_cache` is not current.
+/// Builds the pubkey cache if it is not already built.
 pub fn get_existing_validator_index<T: EthSpec>(
     state: &mut BeaconState<T>,
     pub_key: &PublicKeyBytes,

consensus/types/src/beacon_state.rs

@@ -447,6 +447,21 @@ impl<T: EthSpec> BeaconState<T> {
         Ok(self.pubkey_cache().get(pubkey))
     }
 
+    /// Immutable variant of `get_validator_index` which errors if the cache is not up to date.
+    pub fn get_validator_index_read_only(
+        &self,
+        pubkey: &PublicKeyBytes,
+    ) -> Result<Option<usize>, Error> {
+        let pubkey_cache = self.pubkey_cache();
+        if pubkey_cache.len() != self.validators().len() {
+            return Err(Error::PubkeyCacheIncomplete {
+                cache_len: pubkey_cache.len(),
+                registry_len: self.validators().len(),
+            });
+        }
+        Ok(pubkey_cache.get(pubkey))
+    }
+
     /// The epoch corresponding to `self.slot()`.
     pub fn current_epoch(&self) -> Epoch {
         self.slot().epoch(T::slots_per_epoch())