Skip to content
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
5283853
feat(openai): OpenAI-compatible protocol DTOs and configuration
ginccc Jul 27, 2026
2575f0e
Merge origin/main into feat/openai-api-adapter
ginccc Jul 27, 2026
9cf41af
feat(openai): expose deployed agents via GET /v1/models, with authent…
ginccc Jul 27, 2026
653be68
feat(openai): map OpenAI messages and image content to EDDI InputData
ginccc Jul 27, 2026
1e37f1c
feat(openai): stateful chat completions, sync and streaming
ginccc Jul 27, 2026
3b384d9
docs(openai): Open WebUI integration guide, changelog and roadmap entry
ginccc Jul 27, 2026
5b8c471
fix(openai): six defects found in self-review of the adapter
ginccc Jul 27, 2026
8046183
feat(openai): accept PDF/document and audio attachments, not just images
ginccc Jul 28, 2026
16165cd
feat(openai): accept 'stateless' as a request body field, not only a …
ginccc Jul 28, 2026
54d4813
docs(openai): recipe for using EDDI as a thin LLM gateway
ginccc Jul 28, 2026
0ae2939
test(openai): HTTP-level integration test for the stream dispatch
ginccc Jul 28, 2026
2c43db5
fix(docs): stop gitleaks flagging the adapter's curl examples
ginccc Jul 28, 2026
36e7418
chore(security): suppress the adapter docs' placeholder-credential fi…
ginccc Jul 28, 2026
3ff4fa8
chore(security): stop the ignore file's own comment tripping the scanner
ginccc Jul 28, 2026
b0a8f6a
fix(openai): wait for the streaming turn before closing the response
ginccc Jul 28, 2026
a68f9d1
feat(openai): runnable Open WebUI demo, and two defects it caught
ginccc Jul 28, 2026
d72d5ab
fix(openai): make the demo agent demonstrate state, and stop Open Web…
ginccc Jul 28, 2026
a73e577
docs(openai): correct where Open WebUI injects RAG context, and cover…
ginccc Jul 28, 2026
fc43606
feat(openai): document the builtin_tools injection, add an optional L…
ginccc Jul 28, 2026
09a55b3
feat(openai): hold the demo LLM key in the Secrets Vault, not in the …
ginccc Jul 28, 2026
84606fa
docs(openai): explain the client-side tool gap instead of just instru…
ginccc Jul 28, 2026
384bdd1
feat(openai): render structured outputs and report token usage
ginccc Jul 28, 2026
9075010
Merge remote-tracking branch 'origin/main' into feat/openai-api-adapter
ginccc Jul 28, 2026
a8501b1
docs(openai): explain why quick replies round-trip, not just that the…
ginccc Jul 28, 2026
05f893f
fix(openai): sanitize logged intent, drop the regex CodeQL flags as R…
ginccc Jul 28, 2026
34b9ae6
docs(openai): justify the auth-filter CodeQL dismissal, and pin it wi…
ginccc Jul 28, 2026
2c543db
Merge remote-tracking branch 'origin/main' into feat/openai-api-adapter
ginccc Jul 29, 2026
8e35aa0
docs(changelog): restore date order after the main merge
ginccc Jul 29, 2026
0705fdb
Merge remote-tracking branch 'origin/main' into feat/openai-api-adapter
ginccc Jul 29, 2026
66ec461
fix(demo): make the Open WebUI seeder correct on a second run
ginccc Jul 29, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,7 @@ Follow this order unless the user explicitly requests something different.
| — | Test Coverage | 12,000+ tests, >90% instruction / >80% branch coverage, OpenSSF Gold compliance |
| — | Security Hardening v6.0.2 | SSRF prevention, SafeHttpClient, auth guard, vault salt, security headers, CodeQL + Trivy CI |
| 9b | HITL Framework | Two human-approval gates (turn-level `PAUSE_CONVERSATION` + per-tool-call gating), timeout/no-progress policies, audit ledger, Slack + MCP approval surfaces, crash recovery — see [`docs/hitl.md`](docs/hitl.md) |
| — | OpenAI-Compatible API | `/v1` adapter presenting deployed agents as OpenAI models for Open WebUI and OpenAI SDK clients; per-chat conversation isolation, streaming, multimodal, HITL-aware — see [`docs/open-webui-integration.md`](docs/open-webui-integration.md) |

### In Progress / Upcoming

Expand Down
53 changes: 53 additions & 0 deletions docs/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,59 @@
> **Purpose:** Living document tracking all changes, decisions, and reasoning during implementation. Updated as work progresses for easy reference and review.


---

## ✨ feat(openai): OpenAI-compatible API adapter for Open WebUI (2026-07-27)

**Repo:** EDDI (`feat/openai-api-adapter`)

New `/v1` surface presenting deployed agents as OpenAI "models", so Open WebUI, the Python `openai` SDK, LangChain and LiteLLM can drive EDDI conversations. New package `integrations/openai/`, parallel to `integrations/slack/`. **Disabled by default.** Full guide: [`docs/open-webui-integration.md`](open-webui-integration.md); design rationale in [`planning/openai-api-adapter-plan.md`](../planning/openai-api-adapter-plan.md).

**Four earlier drafts of this plan were built on premises that turned out to be false.** Each was verified against source before implementing; the corrections shaped the design:

- **Content negotiation cannot dispatch sync vs streaming.** The plan routed on `Accept` via two `@Produces` methods. But `openai-python` hardcodes `Accept: application/json` and never varies it by `stream`, and Open WebUI sends no `Accept` header at all (it sniffs the *response* content-type). Every streaming request would have landed on the JSON method. → **one method, dispatching on the `stream` field of the body**, which is what the OpenAI spec says and what vLLM/llama.cpp/LiteLLM/Ollama all do.
- **Open WebUI does not map `chat_id` to the `user` field.** [#27174](https://github.com/open-webui/open-webui/pull/27174) is an *issue*, closed as *not planned*; `payload['user']` is set only for pipeline models and is an object, not a chat id. The plan's entire per-chat isolation mechanism did not exist. → **key on `X-OpenWebUI-Chat-Id`** ([#15813](https://github.com/open-webui/open-webui/pull/15813), gated by `ENABLE_FORWARD_USER_INFO_HEADERS`), with `user` as a defensively-typed fallback.
- **`IRestAgentAdministration` is `@RolesAllowed({"eddi-admin","eddi-editor"})` at type level**, and Quarkus enforces it via a CDI interceptor. Injecting it for `/v1/models`, as planned, would have 403'd for every ordinary caller. → read `IAgentFactory` + `IDocumentDescriptorStore` directly. REST facades are the authorization boundary; a public surface must not reach around one.
- **The auth story did not exist.** `quarkus.http.auth.permission.authenticated.paths=/,/*` captures `/v1/*`, so an `sk-…` bearer would be rejected by OIDC before the adapter ran. → explicit `/v1/*` permission entry plus two modes (`permit` + shared key, or `authenticated` + OIDC), constant-time key compare, and a startup guard.
- **`UtilityAgentProvisioner` was cut.** It had an unauthenticated caller creating and deploying an agent, cloning another agent's credential reference and consuming `maxAgentsPerTenant` quota — blocked by the role checks above anyway, and in tension with Pillar 1 (the engine authoring agent config at runtime). → replaced by **`<model>:stateless` variants**: start, say, end. No writes, no roles, and useful beyond title generation.
- **The new-chat heuristic was cut.** `userMessageCount == 1 && hasTurns → endConversation` is unreliable (regenerate and edit-and-resend look identical to a first message), destructive (memory loss is irrecoverable), and directly contradicted the plan's own recommended Open WebUI Filter, which strips history to the last user message — under which *every* turn would have destroyed the conversation. A new chat is a new chat key, which is a new intent. No inference needed.
- **HITL was unaddressed.** `onSkipped` fires for both `AWAITING_HUMAN` and `IN_PROGRESS`; the plan mapped both to "conversation busy", which would make any agent using `PAUSE_CONVERSATION` permanently unusable with a misleading message. → sentinel-snapshot discrimination, mirroring `SlackEventHandler`, and pauses surface as **chat text with `200`**, never as an HTTP error — a 4xx makes clients discard the user's message.

**Components** (`src/main/java/ai/labs/eddi/integrations/openai/`): `RestOpenAiAdapter` (`/v1`), `OpenAiConversationBridge`, `AgentModelResolver`, `OpenAiMessageMapper`, `OpenAiSseWriter`, `OpenAiAuthFilter`, `OpenAiStartupGuard`, `OpenAiCompatConfig`, `OpenAiApiException` + `OpenAiExceptionMapper`, and 11 wire DTOs under `model/`.

**Other design decisions:**
- Model ids are `<slug>-<last 6 of agentId>`. Agent names are not unique, so a bare slug would be non-deterministic with two agents called "Support". Name and slug lookups are accepted but only when unique; an ambiguous match returns 400 listing candidates rather than guessing.
- Slugging folds accents via NFD rather than dropping them — `Übersicht` was slugging to `bersicht`, mangling every non-ASCII agent name. Caught by its own test.
- Images map to `attachment_N` context entries, so they flow through the existing `AttachmentForwarder` with its vision gating, byte caps and SSRF-guarded fetching. Zero core changes. Two parsing fixes: `data:image/png,payload` is legal and has no `;` (scanning to `;` threw), and remote URLs get a concrete MIME from the extension — `image/*` passes the forwarder's `startsWith("image/")` gate but is rejected by providers when handed to `ImageContent.from`.
- `usage` is omitted rather than zero-filled: EDDI does not surface per-request token counts here, and zeros would render as a factual "0 tokens".
- In-flight completions are semaphore-bounded — each holds a worker thread, since the bridge blocks on the turn as the Slack handler does.
- **Reused, not duplicated:** `ConversationOutputExtractor.extractResponse()` already existed in `engine/memory` (added upstream) and handles more output formats than the Slack-local copy the plan proposed extracting. Phase 2 of the plan became unnecessary. Noted separately: `SlackHitlSupport.extractSlackResponseText` still duplicates it and should delegate.

**Self-review pass** (after the feature was complete) found and fixed six defects, each now covered by a test:
- **Semaphore permit leak.** The streaming path handed its permit to the `StreamingOutput` body to release. If that body never runs — a client disconnecting before serialization starts, say — the permit is never reclaimed, and after `max-concurrent-requests` such events the adapter returns 429 *permanently*, until restart. The resource now releases unconditionally and the stream body takes its own permit inside one try/finally.
- **`GET /v1/models/{id}` echoed the caller's string, not the canonical id** — a lookup by agent name returned `{"id":"Customer Support"}`, which is absent from `GET /v1/models`, so a client round-tripping the answer would ask for a model that does not exist. `ResolvedModel` now carries requested and canonical ids separately (plus the descriptor timestamp, which was hardcoded to 0).
- **`InterruptedException` was swallowed** in the turn wait, leaving the worker thread looking healthy with its shutdown signal gone.
- **A null exception message rendered to the user as the literal text "null"** in the stream error path (NPEs carry no message).
- **`hasSentContent()` lied** — it returned "the stream has started", true even after a content-free `finish()`. Renamed `hasStarted()`.
- **The `eddi.openai.requests` counter documented in the plan was never implemented.** Added with `mode` and `outcome` tags, so `paused` (reviewers behind) and `busy` (clients racing) are distinguishable from real errors.

Also verified rather than assumed: `quarkus.rest.jackson.optimization.enable-reflection-free-serializers=false` in this project, so `@JsonProperty` on record components works — `finish_reason` and `owned_by` serialize correctly. Added `OpenAiWireFormatTest` to pin that, since the non-streaming response shape had no coverage at all.

Corrected a documentation claim rather than the code: unimplemented `/v1` paths (`/v1/embeddings` etc.) return Quarkus' plain 404, not an OpenAI error envelope. A catch-all route would risk shadowing the real endpoints for a cosmetic gain.

**Tests:** 143 unit tests, all green. Mutation-checked — reverting the model-ambiguity guard, the identity refusal, and the HITL sentinel distinction each makes the relevant tests fail. `RestOpenAiAdapterTest` (`@QuarkusTest`, binds a socket) is deferred to CI per the local-environment constraint.

**Attachment support** covers all three binary content-part types: `image_url`, `file` (inline PDFs and documents) and `input_audio`. All map to `attachment_N` context entries, so EDDI's existing forwarder does the real work — capability gating, byte caps, PDF text extraction, SSRF-guarded fetching. Three details the wire formats make easy to get wrong, each pinned by a test:
- `input_audio.data` is **raw base64 with no `data:` prefix**, unlike every other binary payload in the protocol, with the type in a separate `format` field. `mp3` maps to `audio/mpeg` — `"audio/" + format` would produce `audio/mp3`, which is not a real media type.
- `file.file_data` **is** a full data URI, and the declared `filename` beats a generic `application/octet-stream` type, since clients that base64 a file without sniffing it send exactly that.
- `file.file_id` references the OpenAI Files API, which EDDI does not implement; those parts are skipped with an actionable warning rather than becoming empty attachments.

**Stateless requests** have two routes to the same behaviour. The `:stateless` model suffix exists because a model name is the only per-request dimension a UI like Open WebUI can express — its title-generation setting is a dropdown, so a query param, header or body field could not be selected there at all. It follows the ecosystem convention for behavioural model variants (OpenRouter's `:nitro`/`:floor`/`:free`, Ollama's `llama3:8b`) and has the side benefit of appearing in `GET /v1/models`, making the capability discoverable. Alongside it, a `stateless` **body field** gives programmatic callers the explicit parameter (`extra_body={"stateless": True}` in the Python SDK). The two are OR-ed rather than letting either win: `model:"x:stateless"` plus `stateless:false` is self-contradictory, and running stateless only loses continuity while running stateful would persist a conversation the caller may not have wanted. `expose-stateless-variants=false` blocks both routes, so the switch cannot be circumvented by moving the request into the body.

**Gateway-agent recipe** documented in §11 of the integration guide, rather than building a passthrough proxy mode. A thin LLM-only agent behind this adapter already provides vaulted API keys, audit, tenant quotas and cost tracking — which is what the passthrough idea was actually after. Building a real proxy would mean entering the LLM-gateway market (LiteLLM, Portkey, Cloudflare AI Gateway) with a worse product, inheriting per-provider streaming/tool/vision passthrough maintenance for zero agent value, and shipping a feature with neither logic nor configuration — which is the opposite of Pillar 1. The recipe's limits are stated up front: single-turn only, one agent per model, no caching/fallbacks/virtual keys.

**Not implemented (v1):** `/v1/embeddings`, `tool_calls` passthrough (would cause double-execution — EDDI's tools do not exist in Open WebUI), `n > 1`, `logprobs`. Quick replies and `inputField` outputs are dropped; only text reaches OpenAI clients.

---

## 🔒 fix(ci): remove accidentally-committed langchain4j-mcp decompiled sources (2026-07-27)
Expand Down
Loading
Loading