Skip to content

feat(dataset-updater): migrate to official OpenAI .NET SDK (#183) - #210

Merged
jsboige merged 2 commits into
masterfrom
feat/issue-183-sdk-upgrade
Apr 22, 2026
Merged

feat(dataset-updater): migrate to official OpenAI .NET SDK (#183)#210
jsboige merged 2 commits into
masterfrom
feat/issue-183-sdk-upgrade

Conversation

@jsboige

@jsboige jsboige commented Apr 19, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Replaces Betalgo.OpenAI v8.1.0 + Betalgo.OpenAI.Utilities v8.0.1 with the official OpenAI v2.10.0 NuGet package
  • Rewrites Prompt.cs to use OpenAIClient/ChatClient/ChatTool/BinaryData APIs
  • Replaces FunctionCallingHelper auto-reflection with manual FunctionToolDef class + JSON schema
  • Updates all model references from deprecated Models.Gpt_* constants to current model strings (gpt-4.1, gpt-4.1-mini)
  • Removes FunctionDescription attribute from RecordsUpdater.UpdateRecord

Test plan

  • dotnet build — 0 errors (warnings are pre-existing)
  • dotnet test — 42 pass, 1 fail (pre-existing SimpleImageGeneratorTests JSON serialization issue), 1 skip
  • Manual test: run DatasetUpdater with Enabled = true on a task config to verify API calls still work
  • Verify function calling still triggers UpdateRecord correctly

Closes #183

🤖 Generated with Claude Code

… .NET SDK v2.10.0 (#183)

Replace Betalgo.OpenAI v8.1.0 + Betalgo.OpenAI.Utilities with the official
OpenAI .NET SDK (v2.10.0). Rewrite Prompt.cs to use OpenAIClient/ChatClient,
replace FunctionCallingHelper with manual FunctionToolDef, and update all
model references from deprecated constants to current model strings.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@jsboige

jsboige commented Apr 22, 2026

Copy link
Copy Markdown
Contributor Author

Review — PR #210 SDK Migration (#183)

Verdict structurel : LGTM ✅ (runtime validation à confirmer par test manuel DatasetUpdater)

Ce qui fonctionne bien

  • Scope minimal : 5 fichiers, +192/-160, zéro changement dans le reste du pipeline
  • Isolation propre : FunctionToolDef encapsule la conversion vers ChatTool — remplacement 1:1 de FunctionCallingHelper
  • Signatures préservées : Prompt.Send(), DatasetUpdaterConfig.DoChatGPTCall(), RecordsUpdater.UpdateRecord() gardent leurs contrats
  • Models cohérents : 7 configs migrées vers gpt-4.1 / gpt-4.1-mini (cohérent avec feat(translation): upgrade SDK from Betalgo to official OpenAI .NET SDK + SOTA models #205)
  • JSON schema manuel correspond aux paramètres réels de UpdateRecord (primaryKey, fieldName, newValue)
  • Build OK, 42/43 tests pass — le seul fail est préexistant (SimpleImageGeneratorTests)

Points d'attention (non-bloquants)

  1. Param matching via reflection (Prompt.cs l.124-139) : method.GetParameters()[].Name dépend de la préservation des noms de paramètres. OK en debug/release .NET standard, mais un ILLink agressif ou AOT pourrait casser. Low risk vu l'usage actuel (build complet sans trimming).

  2. _openAIClient field inutilisé (Prompt.cs l.16) : assigné dans le getter ChatClient mais jamais relu. Pourrait être local. Cosmétique.

  3. Pas d'options client : new OpenAIClient(ApiKey) ne permet pas de customiser base URL / org ID. Si un jour on veut Azure OpenAI ou proxy, faudra un overload. Hors scope feat(translations): Upgrade OpenAI translation pipeline + complete Virtues i18n #183.

  4. JSON raw string avec tabs (DatasetUpdaterConfig.cs l.539-549) : JSON valide, cosmétique uniquement.

  5. Tests unitaires absents pour FunctionToolDef.ToChatTool() et le nouveau CallFunction. chore(tests): expand unit test coverage for recent pipeline improvements #204 peut tracker ça séparément — pas un bloqueur pour merger.

Runtime check manquant

Le test plan mentionne correctement : "Manual test: run DatasetUpdater with Enabled = true". Tant que ce test n'est pas passé (nécessite OpenAI-Key.txt), on ne peut garantir que le round-trip API fonctionne. Structurellement tout est en place, mais je recommande :

  • Faire tourner 1 task config en mode Enabled = true sur un petit chunk (3-5 records) pour valider le function call
  • Idéalement : le CleanupPrompts config (plus simple que les traductions) ou un dry-run sur Fallacies

Impact aval

Cette PR débloque :

Prêt à merger après validation manuelle round-trip. 👍

Review fix on PR #210:
- Add #nullable enable at top of Prompt.cs — nullable annotations (`?`,
  `null!`) were present but produced CS8632 warnings outside a nullable
  context. Intent was already clear; just make it explicit.
- Guard param.Name null check in CallFunction reflection — ParameterInfo.Name
  is declared nullable. Prevents CS8604 and handles the unlikely edge
  case of compiler-generated unnamed parameters defensively.

Build: 55 -> 53 warnings, 0 errors. Tests: 43/44 pass unchanged.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@jsboige
jsboige merged commit 984d40e into master Apr 22, 2026
1 check passed
@jsboige
jsboige deleted the feat/issue-183-sdk-upgrade branch April 22, 2026 06:23
jsboige added a commit that referenced this pull request Apr 24, 2026
Adds 3 DatasetUpdaterConfig entries (Enabled=false) for translating the 76
untranslated scenarios per target language (EN/RU/PT) via OpenAI GPT-5.4-mini,
plus 6 Scenarii-specific prompt files (User + Assistant x 3 languages).

Audit (167 total FR-complete scenarios):
- EN: 85 scenarios missing ≥1 field (category, subcategory, title,
  smoothTalker, drawer, context, issue, suggestion_en)
- RU: 76 scenarios missing _ru fields
- PT: 76 scenarios missing _pt fields

Config pattern mirrors #231 (Virtues): SequentialChunks, ChunkSize=8,
SelectEmptyTargets=true, MaxGroupItemNb=12, UseFunctionCalling=true.
Each config uses a dedicated prompt pair tailored to scenario context:
- Scenarii-specific field mapping (catégorie→category, baratineur→smoothTalker,
  piocheur→drawer, enjeu→issue, suggestion→suggestion_en — only one with suffix)
- Narrative-tone instructions (rhythm, humor, brevity)
- pt-PT (not pt-BR); Russian contemporary register (not archaic)
- Category/subcategory consistency across scenarios
- Measured adaptation of French cultural references

Prerequisites satisfied: SDK migration #183 (PR #210) + GPT-5.x models (PR #222).
Scenario.cs entity already had complete i18n scaffolding (EN without _en suffix
is historical design; RU/PT use suffix).

Build: 0 errors, 17 warnings (pre-existing).
Tests: 88 pass, 1 skip (Freeplane GUI), 0 fail.

Closes #219 (pending successful end-to-end run with OpenAI key).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
jsboige added a commit that referenced this pull request Jun 29, 2026
…sal (#609)

Grounds #141 in the CURRENT taxonomy state (issue text is ~2 years old;
DatasetUpdater + translation PRs have moved the state considerably).

Census finding (read-only script on ba8e4a6, reproducible like #600/#606):
  - Non-card nodes: Fallacies 1232, Virtues 110 (Scenarii/Rules out of scope).
  - TEXT enrichment (desc/example/title x 8 langs) = 100% on both datasets
    => original #141 scope item 3 (descriptions + examples + translations)
    is DONE. No text-enrichment gap to script.
  - The "GPT-4 enrichment script" = already adapted: modern DatasetUpdater
    (PR #210, OpenAI SDK v2.10.0, gpt-5.5). Not a rewrite.

Reframes #141: the genuinely-open residue is the AIF cross-reference graph
(crossLink_* 8 relationship cols + AIF_skos* 4 SKOS mappings). Schema is
READY (columns exist), content ~0% on non-cards:
  - Fallacies: crossLink 0.2%, AIF_skos 0.5% (barely started)
  - Virtues: crossLink_Opposes + skosDirectRef/MappingType 100% via #498
    pilots, but 7 OTHER crossLink verbs = 0%
  - ~16700 empty cross-ref cells where schema is waiting.

Proposal: 4-stage method (ground -> gpt-5.5 candidate via /v1/responses
effort=low -> drift-free sidecar dry-run -> expert/jsboige ratification
gate -> OWL/2sxc export). NOT auto-write: AIF/Walton mappings are
specialised (memory: "Anti-Fab Validator: Walton scheme = WARN"), and
Cards/ is frozen pre-tag. "Adapt the script" = add a crossLink prompt +
task config to DatasetUpdater, GATED post-release on jsboige GO + schema
decision (single decimal_path vs structured {target,note}).

Overlaps #498 (AIF scale-up pilots produced the few filled cells).
Feeds dependencies #141 declares: #130 (OWL/SKOS) + #136 (2sxc).

Scope: docs + read-only script only. 0 write Cards/, 0 AssetConverter code
change (pre-tag safe). Base ba8e4a6.

Relates to #130, #136, #498.

Co-authored-by: Your <your.email@example.com>
Co-authored-by: Claude-Code <noreply@anthropic.com>
jsboige added a commit that referenced this pull request Jun 29, 2026
…or, 0 Cards/) (#610)

* docs(taxonomy): #141 non-card census + AIF cross-ref enrichment proposal

Grounds #141 in the CURRENT taxonomy state (issue text is ~2 years old;
DatasetUpdater + translation PRs have moved the state considerably).

Census finding (read-only script on ba8e4a6, reproducible like #600/#606):
  - Non-card nodes: Fallacies 1232, Virtues 110 (Scenarii/Rules out of scope).
  - TEXT enrichment (desc/example/title x 8 langs) = 100% on both datasets
    => original #141 scope item 3 (descriptions + examples + translations)
    is DONE. No text-enrichment gap to script.
  - The "GPT-4 enrichment script" = already adapted: modern DatasetUpdater
    (PR #210, OpenAI SDK v2.10.0, gpt-5.5). Not a rewrite.

Reframes #141: the genuinely-open residue is the AIF cross-reference graph
(crossLink_* 8 relationship cols + AIF_skos* 4 SKOS mappings). Schema is
READY (columns exist), content ~0% on non-cards:
  - Fallacies: crossLink 0.2%, AIF_skos 0.5% (barely started)
  - Virtues: crossLink_Opposes + skosDirectRef/MappingType 100% via #498
    pilots, but 7 OTHER crossLink verbs = 0%
  - ~16700 empty cross-ref cells where schema is waiting.

Proposal: 4-stage method (ground -> gpt-5.5 candidate via /v1/responses
effort=low -> drift-free sidecar dry-run -> expert/jsboige ratification
gate -> OWL/2sxc export). NOT auto-write: AIF/Walton mappings are
specialised (memory: "Anti-Fab Validator: Walton scheme = WARN"), and
Cards/ is frozen pre-tag. "Adapt the script" = add a crossLink prompt +
task config to DatasetUpdater, GATED post-release on jsboige GO + schema
decision (single decimal_path vs structured {target,note}).

Overlaps #498 (AIF scale-up pilots produced the few filled cells).
Feeds dependencies #141 declares: #130 (OWL/SKOS) + #136 (2sxc).

Scope: docs + read-only script only. 0 write Cards/, 0 AssetConverter code
change (pre-tag safe). Base ba8e4a6.

Relates to #130, #136, #498.

Co-Authored-By: Claude-Code <noreply@anthropic.com>

* tools(i18n): link_* langlinks resolver (sidecar candidate-URL generator, 0 Cards/)

SECONDAIRE of ai-01 deep-queue dispatch. Extends the #600/#606 coverage
track from MEASUREMENT to RESOLUTION.

The existing probe (docs/taxonomy/192-link-coverage-langlinks-probe.py)
measures the resolvable ceiling (~2919 cells, 57%, #600 §5.1) but
discards the target title (returns only lang codes). This tool captures
the target-language title via the MediaWiki langlinks API and emits the
candidate fill URLs as a sidecar report — step 1 of #600 §6 methodology.

For every node with an en.wikipedia.org/wiki/<Title> link_en missing
link_<lang>, queries langlinks, captures the target title, emits:
  dataset,key,link_lang,resolved_url
to stdout or --out <path>. NEVER writes under Cards/.

Verified on a strided sample of 10 fallacies: 0 errors, 37 candidate
fills (~57% rate, consistent with the probe ceiling), URLs correctly
URL-encoded (Cyrillic %D0, PT accented %C3). Sample also surfaced the
homonym risk #600 §6.4 warns about (English "Engagement" for a fallacy
node) -> confirms human spot-validation is non-optional for AR/FA/ZH.

Safety:
  - 0 write Cards/ (sidecar only, pre-tag freeze).
  - Public MediaWiki API, no key, 0.3s throttle, descriptive UA
    (default urllib UA is 403-forbidden).
  - Resolves FROM link_en (Wikipedia only); 433 non-Wikipedia curated
    sources excluded + preserved as-is (#600 §6.2).

Next (post-release, gated): follow-up PR consumes the sidecar -> apply
candidate URLs cell-by-cell (drift-free QUOTE_MINIMAL+CRLF, method #595),
skip non-empty cells, human spot-validate ~5% residue (AR/FA/ZH, ~150).

Relates to #600, #606.

Co-Authored-By: Claude-Code <noreply@anthropic.com>

---------

Co-authored-by: Your <your.email@example.com>
Co-authored-by: Claude-Code <noreply@anthropic.com>
jsboige added a commit that referenced this pull request Jul 22, 2026
…fort (#141) (#852)

#141 re-scope (po-2024, ai-01 coordinator GO msg-q9uhmf). The DatasetUpdater
translation loop called Chat Completions (CompleteChatAsync), which for reasoning
models (gpt-5.x) burns the token budget on HIDDEN reasoning and returns EMPTY
Content. The dashboard API metric documents the intended config
("gpt-5.5 /v1/responses, reasoning:{effort:low}") — the code contradicted it.

Add an opt-in Responses-API (/v1/responses) path alongside CompleteChatAsync
(reversible: default off). Plumbs reasoning effort (low) so gpt-5.x returns usable
output within budget. Function-calling carried over (all gpt-5.5 tasks are
UseFunctionCalling=true).

Why #141 as written was superseded: the 91 non-card taxonomy nodes (depth 1-3) are
100% complete (desc_*+text_* x8 langs, 0 empty) and the SDK was already modernized
(PR #210). The real residual gap was /v1/responses routing — generic across all 48
tasks, not non-card-specific.

Changes:
- Prompt.cs: UseResponsesApi + ReasoningEffort props; lazy ResponsesClient (shares
  the OpenAIClient); SendViaResponses (items via options.InputItems, reasoning
  effort, function tools + tool-choice, FunctionCallResponseItem processing,
  GetOutputText). #pragma disable OPENAI001 (OpenAI.Responses is [Experimental]).
- DatasetUpdaterConfig.cs: UseResponsesApi + ReasoningEffort config (default off/null).
- PromptResponsesApiTests.cs: ParseReasoningEffort mapping (case-insensitive,
  unknown->Low fallback, no throw) + Responses knobs default off (legacy preserved).

Governance: DatasetUpdater stays Enabled=false (0 run live, 0 prod CSV — T&A freeze).
Reversible code plumbing, 637/0/5 tests green. Coordinator decision, not gated jsboige.

Relates #141 #498.

Co-authored-by: Your <your.email@example.com>
Co-authored-by: Claude-Code <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(translations): Upgrade OpenAI translation pipeline + complete Virtues i18n

1 participant