[Typing] Exports of Cython objects are treated as Any

[eddiebergman]

I'm trying to have some type safety which involves having a variable known to be a type of ConfigurationSpace, however mypy fails completely at this and downcasts it to simply be Any.

My guess is that mypy relies on reading the source files of a library to infer types and does not read .pyx files.

Two possible solutions if we want to support this, while keeping Cython code:

1. We move all .pyx functionality to standalone functions and just call them from plain python functions. My estimate from looking at call traces and generated code from previous work is that this interplay between python calling a Cythonized function is relatively low, especially if the types passed beetween the two layers are simple/array types. However this overhead could be quite high if done constantly, which should be avoided.
2. Release .pyi files, essentially typestubs. This would however add even more maintanence issues as these .pyi files would need to be kept in sync and be correct as they become the source of truth for code linters and mypy, disregarding implementation of the .pyx files. There exists non-perfect automated solutions for regular .py files (e.g. pyright can generate them) but a quick google found nothing for cython code.

[eddiebergman]

Turns out I was wrong!

We can generate .pyi files for ConfigSpace very easily with mypy!

$ stubgen <file>

Heres' the result for configuration_space.pyx. It looks like it needs some editing but this would definitely allow code tools to be much smarter, at least about types with ConfigSpace. However docstrings in editors would still probably be an issue.

This also raises an extra manual step that needs to be done and is likely quite complicated to any contributor if not familiar with ConfigSpace. This would only need to be done on an API change however.

import collections.abc
import numpy as np
from ConfigSpace.conditions import AbstractCondition, ConditionComponent
from ConfigSpace.forbidden import AbstractForbiddenComponent
from ConfigSpace.hyperparameters import Hyperparameter, OrdinalHyperparameter as OrdinalHyperparameter
from _typeshed import Incomplete
from typing import Any, Dict, Iterable, KeysView, List, Optional, Set, Tuple, Union

class ConfigurationSpace(collections.abc.Mapping):
    name: Incomplete
    meta: Incomplete
    forbidden_clauses: Incomplete
    random: Incomplete
    def __init__(self, name: Union[str, Dict, None] = ..., seed: Union[int, None] = ..., meta: Optional[Dict] = ..., *, space: Optional[Dict[str, Union[Tuple[int, int], Tuple[float, float], List[Union[int, float, str]], int, float, str]]] = ...) -> None: ...
    def generate_all_continuous_from_bounds(self, bounds: List[List[Any]]) -> None: ...
    def add_hyperparameters(self, hyperparameters: List[Hyperparameter]) -> List[Hyperparameter]: ...
    def add_hyperparameter(self, hyperparameter: Hyperparameter) -> Hyperparameter: ...
    def add_condition(self, condition: ConditionComponent) -> ConditionComponent: ...
    def add_conditions(self, conditions: List[ConditionComponent]) -> List[ConditionComponent]: ...
    def add_forbidden_clause(self, clause: AbstractForbiddenComponent) -> AbstractForbiddenComponent: ...
    def add_forbidden_clauses(self, clauses: List[AbstractForbiddenComponent]) -> List[AbstractForbiddenComponent]: ...
    def add_configuration_space(self, prefix: str, configuration_space: ConfigurationSpace, delimiter: str = ..., parent_hyperparameter: dict = ...) -> ConfigurationSpace: ...
    def get_hyperparameters(self) -> List[Hyperparameter]: ...
    def get_hyperparameters_dict(self) -> Dict[str, Hyperparameter]: ...
    def get_hyperparameter_names(self) -> List[str]: ...
    def __getitem__(self, key: str) -> Hyperparameter: ...
    def get_hyperparameter(self, name: str) -> Hyperparameter: ...
    def get_hyperparameter_by_idx(self, idx: int) -> str: ...
    def get_idx_by_hyperparameter_name(self, name: str) -> int: ...
    def get_conditions(self) -> List[AbstractCondition]: ...
    def get_forbiddens(self) -> List[AbstractForbiddenComponent]: ...
    def get_children_of(self, name: Union[str, Hyperparameter]) -> List[Hyperparameter]: ...
    def get_child_conditions_of(self, name: Union[str, Hyperparameter]) -> List[AbstractCondition]: ...
    def get_parents_of(self, name: Union[str, Hyperparameter]) -> List[Hyperparameter]: ...
    def get_parent_conditions_of(self, name: Union[str, Hyperparameter]) -> List[AbstractCondition]: ...
    def get_all_unconditional_hyperparameters(self) -> List[str]: ...
    def get_all_conditional_hyperparameters(self) -> List[str]: ...
    def get_default_configuration(self) -> Configuration: ...
    def check_configuration(self, configuration: Configuration) -> None: ...
    def check_configuration_vector_representation(self, vector: np.ndarray) -> None: ...
    def get_active_hyperparameters(self, configuration: Configuration) -> Set: ...
    def __eq__(self, other: Any) -> bool: ...
    def __ne__(self, other: Any) -> bool: ...
    def __hash__(self) -> int: ...
    def __iter__(self) -> Iterable: ...
    def keys(self) -> KeysView[str]: ...
    def __len__(self) -> int: ...
    def sample_configuration(self, size: int = ...) -> Union['Configuration', List['Configuration']]: ...
    def seed(self, seed: int) -> None: ...
    def remove_hyperparameter_priors(self) -> ConfigurationSpace: ...
    def estimate_size(self) -> Union[float, int]: ...
    @staticmethod
    def substitute_hyperparameters_in_conditions(conditions, new_configspace) -> List['ConditionComponent']: ...
    @staticmethod
    def substitute_hyperparameters_in_forbiddens(forbiddens, new_configspace) -> List['ConditionComponent']: ...

class Configuration(collections.abc.Mapping):
    configuration_space: Incomplete
    allow_inactive_with_values: Incomplete
    origin: Incomplete
    config_id: Incomplete
    def __init__(self, configuration_space: ConfigurationSpace, values: Union[None, Dict[str, Union[str, float, int]]] = ..., vector: Union[None, np.ndarray] = ..., allow_inactive_with_values: bool = ..., origin: Any = ..., config_id: Optional[int] = ...) -> None: ...
    def is_valid_configuration(self) -> None: ...
    def __getitem__(self, item: str) -> Any: ...
    def get(self, item: str, default: Union[None, Any] = ...) -> Union[None, Any]: ...
    def __setitem__(self, key, value) -> None: ...
    def __contains__(self, item: str) -> bool: ...
    def __eq__(self, other: Any) -> bool: ...
    def __ne__(self, other: Any) -> bool: ...
    def __hash__(self) -> int: ...
    def __iter__(self) -> Iterable: ...
    def __len__(self) -> int: ...
    def keys(self) -> List[str]: ...
    def get_dictionary(self) -> Dict[str, Union[str, float, int]]: ...
    def get_array(self) -> np.ndarray: ...

[eddiebergman]

Closed by #346