Run benchmark directly using agent locator

[Adaickalavan]

Runs the benchmark directly using an agent locator string, instead of going through a config file such as SMARTS/baselines/driving_smarts/v0/agent_config.yaml.

Addresses #1838 (comment) .

[Gamenot]

This appears to be a reminder of two things: we should set up a utility to determine the general compatibility of agents.

This is relevant to #381, #1602, and #1603 but we seem to be missing a specific issue.

And to add the validation test pre-benchmark to exclude invalid agents from use in the benchmark.

This is relevant to #1732

[Gamenot]

Since we are using agent_locator now I think we should use smarts.core.utils.class_factory.is_valid_locator to test if the locator is correctly formatted.

We can use smarts.core.utils.class_factory.NAME_CONSTRAINT_REGEX.pattern.

something like:

from smarts.core.utils.class_factory import NAME_CONSTRAINT_REGEX

...
if not NAME_CONSTRAINT_REGEX.search(agent_locator):
    raise ValueError(
        f"Invalid locator={agent_locator}. Syntax must match regular expression: "
        "{NAME_CONSTRAINT_REGEX.pattern}"
    )

[Adaickalavan]

We have _raise_on_invalid_locator(locator) function guarding the ClassRegister.make(). So perhaps we don't have to check locator validity in other parts of the code?

SMARTS/smarts/core/utils/class_factory.py

Lines 130 to 134 in 86d7b4d

	def make(self, locator, **kwargs):
	"""Calls the factory with `locator` name key supplying the keyword arguments as argument
	overrides.
	"""
	factory = self.find_factory(locator)

SMARTS/smarts/core/utils/class_factory.py

Lines 106 to 108 in 86d7b4d

	def find_factory(self, locator):
	"""Locates a factory given a locator."""
	self._raise_on_invalid_locator(locator)

[Gamenot]

Fair enough, I was mentioning this since it is best practice to exit as early as possible. The factory side check waits all the way to the point where we call registry.make. That is after installing dependencies, reading configuration, and uncovering the environment.

The check is probably better to go through the registry interface instead similar to how gym works with the locator:

https://github.com/Farama-Foundation/Gymnasium/blob/76559ed553cffc3e5082227b0a8acea461273e84/gymnasium/envs/registration.py#L115-L136

[Gamenot]

I just thought we could hack it using NAME_CONSTRAINT_REGEX but that is perhaps inappropriate to use the regex object directly in case of changing the underlying implementation.