Sensor raises Error when using sensor_service.list_values

[Moradf90]

SUMMARY

when the sensor tries to read from the datastore (st2KV) with self.sensor_service.list_values(prefix'something'), it raises this Error:

ERROR [-] Sensor "MySensor" run method raised an exception: no such option in group api: max_page_size.
WARNING [-] Sensor "MySensor" run method raised an exception: no such option in group api: max_page_size.
KeyError: ('api', 'max_page_size')
    limit = limit or cfg.CONF.api.max_page_size
oslo_config.cfg.NoSuchOptError: no such option in group api: max_page_size
ERROR [-] ('api', 'max_page_size')
ERROR [-] limit = limit or cfg.CONF.api.max_page_size
ERROR [-] no such option in group api: max_page_size

STACKSTORM VERSION

v3.5.0

OS, environment, install method

using docker version

Steps to reproduce the problem

1. create a new pack with sensor class :

from st2reactor.sensor.base import PollingSensor

class MySensor(PollingSensor):
    def __init__(self, sensor_service, config=None):
        super(SyslogAlertsSenderSensor, self).__init__(
            sensor_service=sensor_service,
            config=config,
            poll_interval=60
        )
        ...

    def poll(self):
        payloads = self.sensor_service.list_values(local=False, prefix='something')
        ...

2. install the pack
3. enable the sensor and look at the logs of st2sencorcontainer.

Expected Results

 DEBUG [-] Sensor my_pack.MySensor started.

Actual Results

ERROR [-] Sensor "MySensor" run method raised an exception: no such option in group api: max_page_size.

Workaround

add this code to sensor file:

from oslo_config import cfg

try:
    api_group = cfg.OptGroup('api')
    cfg.CONF.register_group(api_group)
    cfg.CONF.register_opts([cfg.IntOpt('max_page_size', default=50)], group=api_group)
except cfg.DuplicateOptError:
    pass

Thanks!

[ytjohn]

Introduced in #5176

[arm4b]

@ytjohn sounds like a regression then since the st2 v3.5.0?

[ytjohn]

@armab Yes. We just ran across this Friday.

This line specifically:
https://github.com/StackStorm/st2/pull/5176/files#diff-e80db3ed79737fd467db4f0046b9a1668a1a2a433fa22bb611da25ffd1642bd7R92

While st2api config.py reads in api.max_page_size, this is not read in by the st2common or st2reactor. The sensor_wrapper in st2reactor appears to pass in a different config context, which doesn't include the same api options that st2common.services.datastore is expecting.

[rush-skills]

This is still an issue (st2 3.8)

Not sure why the config is not read (even when explicitly added to st2.conf).
In our case, it was action_service failing

self.action_service.list_values(local=False, prefix=key_prefix)

fails with

st2.actions.python.CheckOldCeilings: DEBUG    Creating new Client object.
st2.actions.python.CheckOldCeilings: DEBUG    Retrieving all the values from the datastore
Traceback (most recent call last):
  File \"/opt/stackstorm/st2/lib/python3.8/site-packages/oslo_config/cfg.py\", line 2262, in _get
    return self.__cache[key]
KeyError: ('api', 'max_page_size')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File \"/opt/stackstorm/st2/lib/python3.8/site-packages/python_runner/python_action_wrapper.py\", line 395, in <module>
    obj.run()
  File \"/opt/stackstorm/st2/lib/python3.8/site-packages/python_runner/python_action_wrapper.py\", line 214, in run
    output = action.run(**self._parameters)
  File \"/opt/stackstorm/packs/htcondor/actions/check_old_ceilings.py\", line 73, in run
    current_ceilings = self.get_ceilings_from_kv(key_prefix)
  File \"/opt/stackstorm/packs/htcondor/actions/check_old_ceilings.py\", line 43, in get_ceilings_from_kv
    kvp = self.action_service.list_values(local=False, prefix=key_prefix) # use global namespace for keys (local is action specific namespace)
  File \"/opt/stackstorm/st2/lib/python3.8/site-packages/python_runner/python_action_wrapper.py\", line 136, in list_values
    return self.datastore_service.list_values(local=local, prefix=prefix)
  File \"/opt/stackstorm/st2/lib/python3.8/site-packages/st2common/services/datastore.py\", line 92, in list_values
    limit = limit or cfg.CONF.api.max_page_size
  File \"/opt/stackstorm/st2/lib/python3.8/site-packages/oslo_config/cfg.py\", line 2547, in __getattr__
    return self._conf._get(name, self._group)
  File \"/opt/stackstorm/st2/lib/python3.8/site-packages/oslo_config/cfg.py\", line 2264, in _get
    value = self._do_get(name, group, namespace)
  File \"/opt/stackstorm/st2/lib/python3.8/site-packages/oslo_config/cfg.py\", line 2282, in _do_get
    info = self._get_opt_info(name, group)
  File \"/opt/stackstorm/st2/lib/python3.8/site-packages/oslo_config/cfg.py\", line 2415, in _get_opt_info
    raise NoSuchOptError(opt_name, group)
oslo_config.cfg.NoSuchOptError: no such option in group api: max_page_size

Workaround

 kvp = self.action_service.datastore_service.list_values(local=False, prefix=key_prefix, limit=100) 

[glafkosch]

I have upgraded to latest Stackstorm 3.8 version and I am getting also this issue. Any fix?

st2.actions.python.WorkflowCleanupAction: DEBUG Creating new Client object.\n\"auth.api_url\" configuration option is not configured\nst2.actions.python.WorkflowCleanupAction: DEBUG Retrieving all the values from the datastore\nTraceback (most recent call last):\n File \"/opt/stackstorm/st2/lib/python3.8/site-packages/oslo_config/cfg.py\", line 2262, in _get\n return self.__cache[key]\nKeyError: ('api', 'max_page_size')\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/opt/stackstorm/st2/lib/python3.8/site-packages/python_runner/python_action_wrapper.py\", line 395, in <module>\n obj.run()\n File \"/opt/stackstorm/st2/lib/python3.8/site-packages/python_runner/python_action_wrapper.py\", line 214, in run\n output = action.run(**self._parameters)\n File \"/opt/stackstorm/packs/tf/actions/workflow_cleanup.py\", line 14, in run\n items = self.action_service.list_values(local=False, prefix=key_prefix)\n File \"/opt/stackstorm/st2/lib/python3.8/site-packages/python_runner/python_action_wrapper.py\", line 136, in list_values\n return self.datastore_service.list_values(local=local, prefix=prefix)\n File \"/opt/stackstorm/st2/lib/python3.8/site-packages/st2common/services/datastore.py\", line 92, in list_values\n limit = limit or cfg.CONF.api.max_page_size\n File \"/opt/stackstorm/st2/lib/python3.8/site-packages/oslo_config/cfg.py\", line 2547, in __getattr__\n return self._conf._get(name, self._group)\n File \"/opt/stackstorm/st2/lib/python3.8/site-packages/oslo_config/cfg.py\", line 2264, in _get\n value = self._do_get(name, group, namespace)\n File \"/opt/stackstorm/st2/lib/python3.8/site-packages/oslo_config/cfg.py\", line 2282, in _do_get\n info = self._get_opt_info(name, group)\n File \"/opt/stackstorm/st2/lib/python3.8/site-packages/oslo_config/cfg.py\", line 2415, in _get_opt_info\n raise NoSuchOptError(opt_name, group)\noslo_config.cfg.NoSuchOptError: no such option in group api: max_page_size\n"

My workflow_clean.py

from st2common.runners.base_action import Action

class WorkflowCleanupAction(Action):
    def run(self, scan_id):
        key_prefix = f"tf.StoreWorkerResultAction:{scan_id}"
        items = self.action_service.list_values(local=False, prefix=key_prefix)
        for item in items:
            self.action_service.delete_value(item.name, local=False)

        return (True, None)

Temporary Fix

        #items = self.action_service.list_values(local=False, prefix=key_prefix)
        items = self.action_service.datastore_service.list_values(local=False, prefix=key_prefix, limit=100)

[maxfactor1]

Also experiencing this issue. However, I am also getting an issue where if I try to specify a 'limit' inside my sensor, I get this message:

list_values() got an unexpected keyword argument 'limit'

I verified in datastore.py that 'limit' should be a valid argument, but the wrapper for list_values doesn't include the option to pass 'limit' through.

sensor_wrapper.py snippet:

    def list_values(self, local=True, prefix=None):
        return self.datastore_service.list_values(local=local, prefix=prefix)

datastore.py snippet:

    def list_values(self, local=True, prefix=None, limit=None, offset=0):
        """
        Retrieve all the datastores items.

        :param local: List values from a namespace local to this pack/class. Defaults to True.
        :type: local: ``bool``

        :param prefix: Optional key name prefix / startswith filter.
        :type prefix: ``str``

        :param limit: Number of keys to get. Defaults to the configuration set at 'api.max_page_size'.
        :type limit: ``integer``

        :param offset: Number of keys to offset. Defaults to 0.
        :type offset: ``integer``

        :rtype: ``list`` of :class:`KeyValuePair`
        """
        client = self.get_api_client()
        self._logger.debug("Retrieving all the values from the datastore")
        limit = limit or cfg.CONF.api.max_page_size
        key_prefix = self._get_full_key_prefix(local=local, prefix=prefix)
        kvps = client.keys.get_all(prefix=key_prefix, limit=limit, offset=offset)
        return kvps

My workaround was to manually edit the sensor_wrapper.py to include all of the arguments in the upstream method in datastore.py like the following:

139
140     ##################################
141     # Methods for datastore management
142     ##################################
143
144     def list_values(self, local=True, prefix=None, limit=None, offset=0):
145         return self.datastore_service.list_values(local=local, prefix=prefix, limit=limit, offset=offset)
146
147     def get_value(self, name, local=True, scope=SYSTEM_SCOPE, decrypt=False):
148         return self.datastore_service.get_value(
149             name=name, local=local, scope=scope, decrypt=decrypt
150         )
151
...

[maxfactor1]

I'll be submitting a PR to fix this, but in the meantime for anyone who is experiencing this issue, I think the 'correct' fix is to include the max_page_size option in st2common/st2common/config.py, as it seems like it may have been missed when this config option was added.

# Common API options
api_opts = [
    cfg.StrOpt("host", default="127.0.0.1", help="StackStorm API server host"),
    cfg.IntOpt("port", default=9101, help="StackStorm API server port"),
    #Add the below line as a workaround:
    cfg.IntOpt("max_page_size", default=100, help="Number of keys to get for kv list_values operations"),