feat(ops): allow disabled entities for storage sets that are not in use

[onewland]

Add a new config that lets us disable entity loading when we want to disable storage sets temporarily.

We want to remove the cdc and sessions storage sets for ops changes, but that causes the validation on EntityFactory loading to fail

 File "/Users/oliver/workplace/snuba/snuba/datasets/schemas/tables.py", line 92, in get_table_name
    if get_cluster(self.__storage_set_key).is_single_node()
  File "/Users/oliver/workplace/snuba/snuba/clusters/cluster.py", line 482, in get_cluster
    raise UndefinedClickhouseCluster(
snuba.clusters.cluster.UndefinedClickhouseCluster: StorageSetKey.CDC is not defined in the CLUSTERS setting for this environment

even if the dataset is in DISABLED_DATASETS. This is because initialize_snuba is loading entities first:

snuba/snuba/core/initialize.py

Lines 32 to 43 in afcfb2a

	def initialize_snuba() -> None:
	logger.info("Initializing Snuba...")
	# The order of the functions matters The reference direction is
	#
	# datasets -> entities -> storages
	#
	# The initialization goes bottom up, starting from the
	# lowest-level concept (storage) to the highest (dataset).
	_load_storages()
	_load_entities()
	_load_datasets()

snuba/snuba/datasets/entities/factory.py

Lines 67 to 72 in a6a41ef

	EntityKey.GROUPASSIGNEE: GroupAssigneeEntity(),
	EntityKey.GROUPEDMESSAGE: GroupedMessageEntity(),
	EntityKey.OUTCOMES: OutcomesEntity(),
	EntityKey.OUTCOMES_RAW: OutcomesRawEntity(),
	EntityKey.SESSIONS: SessionsEntity(),
	EntityKey.ORG_SESSIONS: OrgSessionsEntity(),

I tested locally that with the relevant entities disabled we can remove the storage sets cdc and sessions and snuba api will start without the above error

[codecov-commenter]

Codecov Report

Base: 92.25% // Head: 21.96% // Decreases project coverage by -70.29% ⚠️

Coverage data is based on head (afcfb2a) compared to base (a6a41ef).
Patch coverage: 50.00% of modified lines in pull request are covered.

Additional details and impacted files

@@             Coverage Diff             @@
##           master    #3572       +/-   ##
===========================================
- Coverage   92.25%   21.96%   -70.30%     
===========================================
  Files         725      682       -43     
  Lines       33828    32434     -1394     
===========================================
- Hits        31207     7123    -24084     
- Misses       2621    25311    +22690     

Impacted Files	Coverage Δ
snuba/datasets/entities/factory.py	0.00% <0.00%> (-92.11%)	⬇️
snuba/settings/__init__.py	94.90% <100.00%> (+0.03%)	⬆️
tests/base.py	0.00% <0.00%> (-100.00%)	⬇️
tests/helpers.py	0.00% <0.00%> (-100.00%)	⬇️
tests/conftest.py	0.00% <0.00%> (-100.00%)	⬇️
tests/fixtures.py	0.00% <0.00%> (-100.00%)	⬇️
tests/test_cli.py	0.00% <0.00%> (-100.00%)	⬇️
tests/test_util.py	0.00% <0.00%> (-100.00%)	⬇️
snuba/admin/user.py	0.00% <0.00%> (-100.00%)	⬇️
tests/assertions.py	0.00% <0.00%> (-100.00%)	⬇️
... and 636 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.