feat: preserve provider output languages

README.md

@@ -43,6 +43,8 @@ uv run --frozen weather-briefing run briefing
 
 `env.example` 将必填项、条件必填项和选填项分别写在注释中，所有凭据和投递标识均为无效占位值。复制 `locations.example.json` 为被 Git 忽略的 `locations.json` 后可配置多个地点；示例使用北京市西城区中南海的公开坐标。每项必须有稳定 `id`，并在 `name` 与成对的 `latitude`、`longitude` 之间至少提供一项：只有名称时程序正向解析并支持降精度回退，只有坐标时通过 Nominatim 反查规范地点名和行政信息，两者都有时不发起定位请求。解析结果缓存到 `state/`。
 
+`CONTEXT_SOURCES_JSON` 可配置由 `id`、`name`、`url` 和可选 `language` 组成的辅助 HTTP 上下文数组。`language` 使用基础 BCP 47-like 标签记录正文实际语种；无法确定时省略并按 `und` 处理，不能默认假定为中文。
+
 LLM 调用由 any-llm SDK 承担。`LLM_PROVIDER` 使用 any-llm 的 provider ID，`LLM_MODEL` 是该 provider 接受的模型 ID；凭据、API Base 和其他认证配置直接使用 any-llm 为对应 provider 定义的环境变量。开发环境安装 `any-llm-sdk[all]` 以验证全部 provider，应用的基础运行依赖只安装 SDK 核心包；官方 OCI 镜像预装 `deepseek`、`openai` 和 `openrouter` provider extras，其他生产安装可按实际选择合并 extras，例如 `uv pip install 'weather-briefing' 'any-llm-sdk[deepseek,openai]>=1.19,<2'`。为兼容已有 DeepSeek 部署，`DEEPSEEK_MODEL` 可作为 `LLM_MODEL` 的后备，`DEEPSEEK_BASE_URL` 可作为 `DEEPSEEK_API_BASE` 的后备。
 
 应用将带时间、级别和 logger 名称的运行日志写入标准错误；INFO 日志记录每个地点的天气 provider 顺序和逻辑降级过程，并为天气、空气质量、地理编码、LLM、RSS、辅助上下文及 Telegram 的每个实际 HTTP 请求记录 provider、operation、方法、成功或失败、耗时和 HTTP 状态或异常类型，因此可从容器日志还原外部 API 调用历史。RSS 重试与 Telegram 分片分别按实际请求次数记录。常规 INFO 日志及仅由 `DEBUG=true` 启用的非敏感诊断不记录坐标、标题、正文、URL、token、chat ID、请求 endpoint 或异常消息；DEBUG 元数据覆盖从 RSS 清洗、权威预报转发和平台渲染到 Telegram 分片接受状态的链路。若仍需排查平台渲染或分片内容，可在不重启 daemon 的情况下临时记录完整渲染正文：

docs/design.md

@@ -50,6 +50,8 @@ service 将最终解析得到的完整地点名作为 `location_scope.full_name`
 
 能力组合边界由 `capabilities.py` 的 `CapabilityProviderSet` 承担。天气、空气质量、结构化过敏原、生活指数、预警和短时预报属于可独立声明的 capability；现有 QWeather/Open-Meteo 完整上下文 adapter 暂时挂在天气槽位，AQICN 挂在空气质量槽位。`ALLERGEN` 表示 adapter 能提供独立 `AllergenSnapshot`，不表示任意与过敏有关的文本；QWeather 类型 7 综合过敏指数仍属于 `LIFESTYLE`，即使它会标记文档包含过敏建议。这样本地气象机构可以只实现预警或 nowcast，而不必伪装为完整天气 provider；后续能力 provider 不应为填充无关字段而发起额外请求。
 
+每个天气或能力 provider 通过 `LanguageSupport` 声明固定或可选的输出语言，规范化快照和 `SourceDocument` 使用 BCP 47 标签记录实际正文语言，并把该标签随当前及历史上下文传入 LLM。可选语言在 provider 请求边界映射为厂商 wire code；固定语言不会被伪装为用户目标语言。LLM 只在来源语言与目标输出语言不同时于最终输出阶段翻译一次，来源语言已经匹配时不得先翻译到中间语言再译回。SQLite 持久化来源语言，旧快照迁移时按当时唯一的中文输出契约标记为 `zh-CN`。
+
 `QWeatherProvider` 的常规预报读取实时空气质量、今明两日天气和当日生活指数；显式目标日期查询改用 3 日生活指数及 3 日空气质量预报，并按天气预报中的目标日期选择同一天的数据。它提供目标日期天气、温度、风、湿度、预期降水及运动、穿衣、旅游、舒适度和交通指数。空气质量请求失败不会丢弃已经有效的天气结果；常规预报把空气质量留空交给补充层，目标日期查询则保留缺失而不使用当前 AQICN 观测冒充预报。
 
 `OpenMeteoProvider` 使用全球 Weather Forecast API，并尝试从其独立 Air Quality API 获取 U.S. AQI 与 PM2.5 浓度。公开 endpoint 适用于非商业免费使用、要求署名且无 SLA；Base URL 和可选 API Key 可配置，以便切换商业 endpoint。

docs/notes.md

@@ -82,6 +82,8 @@ Nominatim 限速使用 `await asyncio.sleep()`，它只挂起当前协程并把
 
 ## LLM 历史输入预算的当前范围
 
+来源语言是输入边界的事实元数据，不是用户目标语种的别名。`SourceDocument` 构造时规范化基础 BCP 47-like 标签，再写入 SQLite 和 LLM payload，避免同一正文因大小写差异产生伪历史变化；旧数据库的语言列只保留迁移时已知的 `zh-CN` 默认值，未声明语言的任意 HTTP context 使用 `und`。`LanguageSupport.select()` 继续用于显式选择 provider 已声明的 wire 语言并严格拒绝未知值，而地点级用户目标语种通过 `match()` 按逐步去除区域/脚本后缀选择最接近的 provider 语言，匹配失败回到 provider 默认值。这样固定只输出 `ja` 的 API 可以服务 `ja-JP` 的用户而不在构造阶段失败；LLM 仍收到实际来源标签，并在同一主语言时避免无意义的往返翻译。当前成立条件是所有内置 provider、地点配置和 context source 只需要语言、脚本与地区子标签，因此解析刻意不承诺完整 BCP 47 extension/private-use 语法。任一 provider wire code、用户配置或外部持久化契约开始要求 extension/private-use 标签，或出现需要按 Unicode locale extension 区分输出的产品需求时，必须改用成熟 BCP 47 解析器，并重新审视规范化、主语言匹配与 wire-code 映射。
+
 `LLM_HISTORY_MAX_DOCUMENTS` 和 `LLM_HISTORY_MAX_CHARACTERS` 只限制逐小时积累的历史 API 上下文快照。历史文章与已发布简报仍由 `HISTORY_HOURS` 控制时间范围，不另设数量或字符预算。当前接受这一边界，是因为已配置 RSS 的发布频率较低，普通 briefing 只有值得打扰时才保存正文，而 API 快照是唯一会在每个调度轮次稳定增长的输入类别。
 
 应用以 DEBUG 元数据记录完整 payload 的序列化字符数，但不记录正文。若 RSS 发布量或地点数显著增加、历史简报开始接近逐小时产生、payload 字符数持续增长，或再次出现与输入规模相关的模型超时或上下文长度错误，应把文章、简报和快照收敛到带类别最低保留量的统一历史输入预算，而不是继续分别增加当前两个快照上限。

env.example

@@ -36,6 +36,7 @@ GEOCODING_CACHE_PATH=state/geocoding.json
 
 # Optional private RSS sources. The file may be absent or contain an empty array.
 RSS_SOURCES_FILE=rss-sources.json
+# Auxiliary HTTP context items accept id, name, url, and optional language; unknown defaults to und.
 CONTEXT_SOURCES_JSON=[]
 RSS_MAX_ATTEMPTS=3
 RSS_RETRY_MIN_SECONDS=3

tests/test_air_quality.py

@@ -220,10 +220,36 @@ def test_air_quality_document_labels_forecast_time() -> None:
     assert "预报时段：2026-07-15T18:00:00+08:00" in document.content
     assert "观测时间" not in document.content
     assert document.history_value is not None
-    assert document.history_value.startswith("时间类型：forecast\n")
+    assert document.history_value.startswith("时间类型：预报\n")
     assert "2026-07-15T18:00:00+08:00" not in document.history_value
 
 
+def test_air_quality_document_scaffold_matches_english_source_language() -> None:
+    snapshot = AirQualitySnapshot(
+        source_id="air-quality:test",
+        source_name="Test",
+        source_url="https://example.invalid/air-quality",
+        effective_at=None,
+        time_kind=AirQualityTimeKind.OBSERVATION,
+        aqi=42,
+        aqi_display="42",
+        aqi_standard="US EPA",
+        pm25_aqi=None,
+        pm25_concentration=None,
+        pm25_unit=None,
+        category="Good",
+        health_guidance="Normal activities are suitable.",
+        output_language="en",
+    )
+
+    document = air_quality_to_document(snapshot)
+
+    assert document.language == "en"
+    assert "Observed at: Unavailable" in document.content
+    assert "Health guidance: Normal activities are suitable." in document.content
+    assert "健康提示" not in document.content
+
+
 def test_health_guidance_unbounded_band_required(monkeypatch) -> None:
     from weather_briefing.air_quality import _guidance_bands
 

tests/test_allergen.py

@@ -76,6 +76,26 @@ def test_allergen_to_document_format() -> None:
     assert "2026-07-13T08:00:00Z" not in document.history_value
 
 
+def test_allergen_document_scaffold_matches_english_source_language() -> None:
+    snapshot = AllergenSnapshot(
+        source_id="allergen:test",
+        source_name="Test pollen",
+        source_url="https://example.invalid/allergen",
+        observed_at=None,
+        levels=(AllergenLevel(name="Birch", category="Moderate", concentration=15),),
+        overall_category="Moderate",
+        health_guidance="Reduce prolonged outdoor activity.",
+        output_language="en",
+    )
+
+    document = allergen_to_document(snapshot)
+
+    assert document.language == "en"
+    assert "Pollen allergens:" in document.content
+    assert "Birch: 15 grains/m³ (Moderate)" in document.content
+    assert "花粉过敏原" not in document.content
+
+
 def test_allergen_to_document_without_observed_at() -> None:
     snapshot = AllergenSnapshot(
         source_id="allergen:test",

tests/test_config.py

@@ -777,6 +777,36 @@ def test_context_source_accepts_and_strips_required_strings(self, monkeypatch) -
         assert source.id == "context"
         assert source.name == "Context"
         assert source.url == "https://example.invalid/context"
+        assert source.language == "und"
+
+    def test_context_source_normalizes_declared_language(self, monkeypatch) -> None:
+        _required_environment(monkeypatch)
+        monkeypatch.setenv(
+            "CONTEXT_SOURCES_JSON",
+            '[{"id":"context","name":"Context","url":"https://example.invalid/context","language":"EN-us"}]',
+        )
+
+        assert Settings.from_env().context_sources[0].language == "en-US"
+
+    @pytest.mark.parametrize("language", ("english", 1, None))
+    def test_context_source_rejects_invalid_language(self, monkeypatch, language: object) -> None:
+        _required_environment(monkeypatch)
+        monkeypatch.setenv(
+            "CONTEXT_SOURCES_JSON",
+            json.dumps(
+                [
+                    {
+                        "id": "context",
+                        "name": "Context",
+                        "url": "https://example.invalid/context",
+                        "language": language,
+                    }
+                ]
+            ),
+        )
+
+        with pytest.raises(ConfigurationError, match=r"CONTEXT_SOURCES_JSON\[0\]\.language"):
+            Settings.from_env()
 
     def test_invalid_timezone_raises_error(self, monkeypatch) -> None:
         _required_environment(monkeypatch)

tests/test_languages.py

@@ -0,0 +1,98 @@
+import pytest
+
+from weather_briefing.languages import LanguageSupport, localized_labels, normalize_language_tag
+from weather_briefing.models import SourceDocument
+from weather_briefing.weather_context import OPEN_METEO_LANGUAGE_SUPPORT, QWEATHER_LANGUAGE_SUPPORT
+
+
+def test_language_tags_are_normalized() -> None:
+    assert normalize_language_tag(" zh-hans ") == "zh-Hans"
+    assert normalize_language_tag("EN-us") == "en-US"
+
+
+@pytest.mark.parametrize("value", ("", "en_US", "english"))
+def test_language_tags_reject_invalid_values(value: str) -> None:
+    with pytest.raises(ValueError, match="BCP 47"):
+        normalize_language_tag(value)
+
+
+def test_language_support_selects_supported_api_language() -> None:
+    support = LanguageSupport(
+        default="zh-CN",
+        supported=("zh-CN", "ja"),
+        api_codes=(("zh-CN", "zh"), ("ja", "ja")),
+    )
+
+    assert support.select(None) == "zh-CN"
+    assert support.select("JA") == "ja"
+    assert support.api_code("ja") == "ja"
+    assert support.selectable is True
+
+
+def test_language_support_matches_region_variant_to_provider_language() -> None:
+    support = LanguageSupport(default="zh-CN", supported=("zh-CN", "ja"))
+
+    assert support.match(None) == "zh-CN"
+    assert support.match("ja-JP") == "ja"
+    assert support.match("fr-FR") == "zh-CN"
+
+
+def test_fixed_language_rejects_other_output_language() -> None:
+    support = LanguageSupport.fixed("en")
+
+    assert support.selectable is False
+    assert support.api_code() == "en"
+    with pytest.raises(ValueError, match="does not support"):
+        support.select("ja")
+
+
+def test_weather_provider_language_metadata_distinguishes_selectable_and_fixed_sources() -> None:
+    assert QWEATHER_LANGUAGE_SUPPORT.select("ja") == "ja"
+    assert QWEATHER_LANGUAGE_SUPPORT.api_code("ja") == "ja"
+    assert OPEN_METEO_LANGUAGE_SUPPORT.selectable is False
+    assert OPEN_METEO_LANGUAGE_SUPPORT.default == "zh-CN"
+
+
+@pytest.mark.parametrize(
+    ("default", "supported"),
+    (("en", ()), ("ja", ("en",)), ("en", ("en", "EN"))),
+)
+def test_language_support_rejects_invalid_supported_sets(default: str, supported: tuple[str, ...]) -> None:
+    with pytest.raises(ValueError, match="unique supported default"):
+        LanguageSupport(default=default, supported=supported)
+
+
+@pytest.mark.parametrize(
+    "api_codes",
+    (
+        (("en", "en"),),
+        (("en", "en"), ("ja", "ja"), ("JA", "jp")),
+        (("en", "en"), ("fr", "fr")),
+        (("en", "en"), ("ja", " ")),
+    ),
+)
+def test_language_support_rejects_incomplete_or_invalid_api_codes(
+    api_codes: tuple[tuple[str, str], ...],
+) -> None:
+    with pytest.raises(ValueError, match="API codes"):
+        LanguageSupport(default="en", supported=("en", "ja"), api_codes=api_codes)
+
+
+def test_localized_labels_match_exact_then_primary_language() -> None:
+    translations = {"en": {"label": "Label"}, "zh-TW": {"label": "標籤"}}
+
+    assert localized_labels("en-US", translations)["label"] == "Label"
+    assert localized_labels("zh-TW", translations)["label"] == "標籤"
+    with pytest.raises(ValueError, match="No document scaffold"):
+        localized_labels("ja", translations)
+
+
+def test_source_document_normalizes_language_at_model_boundary() -> None:
+    document = SourceDocument("source", "Source", "https://example.invalid/source", "内容", language="JA")
+
+    assert document.language == "ja"
+
+
+def test_source_document_rejects_invalid_language() -> None:
+    with pytest.raises(ValueError, match="language tag"):
+        SourceDocument("source", "Source", "https://example.invalid/source", "内容", language="english")

tests/test_service.py

@@ -341,6 +341,7 @@ def test_context_history_enforces_document_and_serialized_character_limits() ->
         "latest",
         oversized.name,
         oversized.url,
+        oversized.language,
         oversized.content,
         "",
     ):
@@ -380,6 +381,7 @@ def test_context_history_uses_deterministic_summary_before_skipping_mandatory_do
         "source_id": "weather",
         "name": "Source weather",
         "url": "https://example.invalid/weather",
+        "language": "zh-CN",
         "content": "weather summary",
         "history_role": "latest",
         "content_compacted": True,

tests/test_sources.py

@@ -263,12 +263,13 @@ async def test_http_context_source_fetches_successfully() -> None:
         transport=httpx.MockTransport(lambda _: httpx.Response(200, text="context data"))
     ) as client:
         result = await HTTPContextSource(client).fetch(
-            ContextSourceConfig(id="ctx", name="Context", url="https://example.invalid/ctx")
+            ContextSourceConfig(id="ctx", name="Context", url="https://example.invalid/ctx", language="en")
         )
 
     assert isinstance(result, SourceDocument)
     assert result.id == "ctx"
     assert result.content == "context data"
+    assert result.language == "en"
 
 
 async def test_http_context_source_raises_on_http_error() -> None:

tests/test_state.py

@@ -379,6 +379,22 @@ def test_context_snapshots_are_available_for_briefing_change_detection(tmp_path:
         assert state.recent_context_documents(now.add(hours=3), 2) == ()
 
 
+def test_context_snapshot_language_is_persisted(tmp_path: Path) -> None:
+    now = pendulum.datetime(2026, 7, 13, 9, tz="Asia/Shanghai")
+    document = SourceDocument(
+        "weather:jma",
+        "JMA",
+        "https://example.invalid/jma",
+        "雨",
+        language="ja",
+    )
+
+    with SQLiteStateStore(tmp_path / "state.db") as state:
+        state.save_context_documents((document,), now)
+
+        assert state.recent_context_documents(now.add(hours=1), 2) == (document,)
+
+
 def test_existing_context_snapshot_schema_adds_history_fields(tmp_path: Path) -> None:
     database_path = tmp_path / "existing-state.db"
     with closing(sqlite3.connect(database_path)) as connection: