EnvSettingsSource does not respect populate_by_name for aliased fields

[hozn]

It looks like the logic to extract fields from the environment (e.g. os.environ) only respects the alias for aliased Fields vs. also allowing for env vars to be set by name when populate_by_name=True in the model config.

Simple reproduce script:

import os
from pydantic import Field
from pydantic_settings import BaseSettings, SettingsConfigDict


class AppSettings(BaseSettings):
    SETTING_NAME: str = Field("default", alias="ALIAS_NAME")
    model_config = SettingsConfigDict(populate_by_name=True, case_sensitive=True)


if __name__ == "__main__":
    os.environ['SETTING_NAME'] = 'by-name'
    settings = AppSettings()
    print(f"{settings.SETTING_NAME=}")

    os.environ['ALIAS_NAME'] = 'by-alias'
    settings = AppSettings()
    print(f"{settings.SETTING_NAME=}")

# Outputs:
# settings.SETTING_NAME='default'
# settings.SETTING_NAME='by-alias'

Note that changing the env source to use a custom source does appear to address this, though I'm not confident this is the correct approach:

import os
from pydantic import Field
from pydantic.fields import FieldInfo
from pydantic_settings import BaseSettings, SettingsConfigDict, EnvSettingsSource, PydanticBaseSettingsSource, \
    DotEnvSettingsSource


class BetterAliasEnvSettingsSource(EnvSettingsSource):
    """
    Overrides the base env settings source to also respect populate_by_name when settings vars are aliased.
    """

    def _extract_field_info(self, field: FieldInfo, field_name: str) -> list[tuple[str, str, bool]]:
        """
        Overrides super method to add better support for aliased fields.
        """

        field_info = super()._extract_field_info(field, field_name)

        if field.validation_alias:
            # Also add the column name if configured to allow setting by name
            if self.config.get("populate_by_name"):
                field_info.append((field_name, self._apply_case_sensitive(self.env_prefix + field_name), False))

        return field_info


class AppSettings(BaseSettings):
    SETTING_NAME: str = Field("default", alias="ALIAS_NAME")
    model_config = SettingsConfigDict(populate_by_name=True, case_sensitive=True)

    @classmethod
    def settings_customise_sources(
        cls,
        settings_cls: type[BaseSettings],
        init_settings: PydanticBaseSettingsSource,
        env_settings: EnvSettingsSource,
        dotenv_settings: DotEnvSettingsSource,
        file_secret_settings: PydanticBaseSettingsSource,
    ) -> tuple[PydanticBaseSettingsSource, ...]:

        fixed_env_settings = BetterAliasEnvSettingsSource(
            settings_cls,
            case_sensitive=env_settings.case_sensitive,
            env_prefix=env_settings.env_prefix,
            env_nested_delimiter=env_settings.env_nested_delimiter,
            env_ignore_empty=env_settings.env_ignore_empty,
            env_parse_none_str=env_settings.env_parse_none_str,
            env_parse_enums=env_settings.env_parse_enums,
        )

        return (
            init_settings,
            fixed_env_settings,
            dotenv_settings,
            file_secret_settings,
        )
    

if __name__ == "__main__":
    os.environ['SETTING_NAME'] = 'by-name'
    settings = AppSettings()
    print(f"{settings.SETTING_NAME=}")

    os.environ['ALIAS_NAME'] = 'by-alias'
    settings = AppSettings()
    print(f"{settings.SETTING_NAME=}")

# Outputs:
# settings.SETTING_NAME='by-name'
# settings.SETTING_NAME='by-alias'

I suspect this issue may also apply to DotEnvSettingsSource, but I have not experimented as comprehensively with this one.

[hramezani]

Thanks @hozn for reporting this issue.

pydantic-settings doesn't respect to populate_by_name.

It would be great if you could work on a PR that adds populate_by_name support + some tests.

[hozn]

Ok, it would be nice / would have saved me some time stepping through with the debugger 😂 if this were documented as just not being supported, since populate_by_name is (due to inheritance) a valid member of the SettingsConfigDict.

But, yes, I will work on a PR as time allows!

[hozn]

@hramezani -- this proved easy to add and very obvious on how to add tests (if I did it right 😅 ) -- thanks for making that easy. Obviously let me know in PR if additional work needed.

[hramezani]

Great work @hozn