Skip to content
Merged
Show file tree
Hide file tree
Changes from 33 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
921871c
chore[bc]: remove BaseInference inheritance from diffusion addon
donriddo Apr 9, 2026
c76b809
fix: restore JSDoc comments in index.js and index.d.ts
donriddo Apr 9, 2026
3b27c36
docs: update SD README for new constructor pattern
donriddo Apr 10, 2026
71d885b
fix: guard SD _runInternal against run-before-load with clear error
donriddo Apr 10, 2026
c7e51a3
docs: align SD architecture.md with new constructor and composition p…
donriddo Apr 10, 2026
f5c9426
chore[bc]: address PR #1496 review findings and bump to 0.2.0
donriddo Apr 10, 2026
7083f7e
refactor: move SD C++ event normalization into addon.js
donriddo Apr 10, 2026
e506204
fix: address PR #1496 second-round review findings
donriddo Apr 10, 2026
4a3a261
Merge remote-tracking branch 'origin/main' into chore/sd-addon-interf…
donriddo Apr 10, 2026
ab71710
fix: remove task-doc reference and refactor-narration comments
donriddo Apr 12, 2026
45d3b19
fix: throw on second load(), log rejected responses, add mapAddonEven…
donriddo Apr 14, 2026
51bb1e3
fix: restore JSDoc on run() that was dropped during BaseInference rem…
donriddo Apr 14, 2026
5063a20
fix: correct CHANGELOG error quote and remove dead files.model fallback
donriddo Apr 14, 2026
f6f424d
fix: make load() idempotent when already loaded
donriddo Apr 15, 2026
0ec9a04
Merge remote-tracking branch 'upstream/main' into chore/sd-addon-inte…
donriddo Apr 15, 2026
8a6e82d
Merge remote-tracking branch 'upstream/main' into chore/sd-addon-inte…
donriddo Apr 15, 2026
943220f
doc: document missing breaking changes from BaseInference removal
donriddo Apr 15, 2026
de7e693
fix: address lifecycle, cleanup, and CI-surface review findings
donriddo Apr 16, 2026
2973126
fix[ci]: run test:unit inside test:integration flow
donriddo Apr 16, 2026
5519dea
fix[ci]: run test:unit via run-lint-and-unit-tests action
donriddo Apr 16, 2026
71d1c83
chore: test script chains test:unit + test:integration
donriddo Apr 16, 2026
1b302cb
doc: fix mermaid classDiagram parsing error in architecture.md
donriddo Apr 16, 2026
c4341e9
Merge remote-tracking branch 'upstream/main' into chore/sd-addon-inte…
donriddo Apr 16, 2026
ff980bd
chore[ci]: rename step to reflect what the action actually runs
donriddo Apr 16, 2026
78c8cd0
fix: doc and type drift around img2img; dead code in SdModel.cpp
donriddo Apr 16, 2026
0170b29
doc: refresh Key Features, migration marker, and img2img JSDoc
donriddo Apr 16, 2026
7250731
doc: restore JSDoc on SD cancel() and unload()
donriddo Apr 16, 2026
ff968f6
doc: trim verbose comments added during the refactor
donriddo Apr 16, 2026
a2710ba
doc: restore pre-refactor createAddon JSDoc and load error log
donriddo Apr 16, 2026
80819f5
fix: release native logger when addon construction throws
donriddo Apr 16, 2026
a0f71ad
chore: drop unused 'test' script, inline into 'test:all'
donriddo Apr 16, 2026
9f68237
doc: fix 0.3.0 CHANGELOG heading depth and queue serialization scope
donriddo Apr 16, 2026
1f4a8b9
doc: fix flowchart mermaid parse errors in data-flows
donriddo Apr 17, 2026
98657e6
doc: note FLUX.2 ignores strength in GenerationParams JSDoc
donriddo Apr 17, 2026
4032040
Merge branch 'main' into chore/sd-addon-interface-refactor
donriddo Apr 17, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .github/workflows/on-pr-lib-infer-diffusion.yml
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,15 @@ jobs:
working-directory: packages/lib-infer-diffusion
run: npm run test:dts

- name: Run lint and unit tests
id: run_lint_and_unit_tests
uses: tetherto/oss-actions/.github/actions/run-lint-and-unit-tests@4c64bed91fc8eba3a201adb1495e61b4c1a2246d
with:
gpr-token: ${{ secrets.GITHUB_TOKEN }}
pat-token: ${{ secrets.GITHUB_TOKEN }}
registry-type: gpr
workdir: packages/lib-infer-diffusion

prebuild:
needs: [authorize, sanity-checks]
if: needs.authorize.outputs.allowed == 'true'
Expand Down
1 change: 1 addition & 0 deletions packages/lib-infer-diffusion/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,4 @@ temp/
*.deb
*.zip
test/integration/all.js
test/unit/all.js
99 changes: 99 additions & 0 deletions packages/lib-infer-diffusion/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,104 @@
# Changelog

## [0.3.0] - 2026-04-15

This release migrates the diffusion addon off `BaseInference` inheritance and onto the composable `createJobHandler` + `exclusiveRunQueue` utilities from `@qvac/infer-base@^0.4.0`. The constructor signature is replaced with a single object whose `files` field carries absolute paths for every model component, mirroring the parallel embed and LLM addon refactors. This is a breaking change β€” every caller must update.

### Breaking Changes

#### Constructor signature: single object with `files` instead of `(args, config)`

`ImgStableDiffusion` now takes a single `{ files, config, logger?, opts? }` object. The old `diskPath` + `modelName` + per-component filename pattern is gone β€” callers pass absolute paths directly via `files`. Companion model fields are renamed (`clipLModel` β†’ `clipL`, `clipGModel` β†’ `clipG`, `t5XxlModel` β†’ `t5Xxl`, `llmModel` β†’ `llm`, `vaeModel` β†’ `vae`).

```js
// BEFORE (≀ 0.2.x)
const model = new ImgStableDiffusion({
diskPath: '/models',
modelName: 'flux-2-klein-4b-Q8_0.gguf',
llmModel: 'Qwen3-4B-Q4_K_M.gguf',
vaeModel: 'flux2-vae.safetensors',
logger: console
}, { threads: 8 })

// AFTER (0.3.0)
const model = new ImgStableDiffusion({
files: {
model: '/models/flux-2-klein-4b-Q8_0.gguf',
llm: '/models/Qwen3-4B-Q4_K_M.gguf',
vae: '/models/flux2-vae.safetensors'
},
config: { threads: 8 },
logger: console,
opts: { stats: true }
})
```

#### `BaseInference` inheritance removed

`ImgStableDiffusion` no longer extends `BaseInference`. The class composes `createJobHandler` and `exclusiveRunQueue` from `@qvac/infer-base@^0.4.0` directly. The public lifecycle (`load` / `run` / `cancel` / `unload` / `getState`) is unchanged in shape; only construction differs. Internal helpers like `_withExclusiveRun` and `_outputCallback` are removed.

#### Caller owns absolute paths β€” addon no longer joins `diskPath` + filename

Callers that previously relied on the addon to resolve `path.join(diskPath, filename)` must now do that resolution themselves before constructing the model.

#### `getState()` returns a narrower shape

`getState()` previously returned `{ configLoaded, weightsLoaded, destroyed }` (the three-field shape from `BaseInference`). It now returns `{ configLoaded }` only. The `weightsLoaded` and `destroyed` fields are gone β€” `weightsLoaded` collapsed into `configLoaded` because the refactored `load()` does both in one step, and `destroyed` is no longer tracked since `unload()` resets `configLoaded` and nulls the addon handle instead. Callers reading `state.weightsLoaded` or `state.destroyed` must switch to `state.configLoaded`.

#### Public methods removed from `ImgStableDiffusion`

`ImgStableDiffusion` previously exposed these methods via `BaseInference` inheritance, all of which are now gone:

- `downloadWeights(onDownloadProgress, opts)` β€” the diffusion addon never used the loader in practice, but the inherited method was still present on the public surface. It is removed along with the base class.
- `pause()` / `unpause()` / `stop()` β€” BaseInference job-lifecycle helpers. The refactor uses `createJobHandler` directly; use `cancel()` to terminate an in-flight generation.
- `status()` β€” replaced by `getState()` for the static readiness flag; per-job state is observed via the `QvacResponse` returned by `run()`.
- `destroy()` β€” folded into `unload()`, which now both releases native resources and nulls `this.addon`.
- `getApiDefinition()` β€” no longer exposed; consumers should import types from `index.d.ts`.

#### `cancel()` no longer accepts a `jobId`

`BaseInference.cancel(jobId)` took an optional `jobId` argument. The refactor's `cancel()` is parameterless β€” there is always at most one active generation per instance, owned by `createJobHandler`. Any caller passing a `jobId` will have it ignored; update call sites to `await model.cancel()`.

### Features

#### Constructor input validation

The constructor now throws `TypeError('files.model must be an absolute path string')` when `files.model` is missing or not a string, or `TypeError('files.model must be an absolute path (got: <value>)')` when supplied as a relative path. This produces a clear error for callers porting old code instead of a confusing `Cannot read properties of undefined`. The same validation applies to optional companion fields (`clipL`, `clipG`, `t5Xxl`, `llm`, `vae`) when supplied.

#### `run()`-before-`load()` guard

Calling `run()` before `load()` now throws `Error('Addon not initialized. Call load() first.')` instead of crashing in native code. Covered by a new regression test in `test/integration/api-behavior.test.js`.

#### `load()` is now idempotent when already loaded

A second `load()` call on an already-loaded instance is now a silent no-op instead of unloading and reloading. This aligns with the ReadyResource pattern used elsewhere in QVAC and prevents accidental double-loads from triggering expensive work. Callers that intentionally want to swap weights must call `unload()` first (which clears `configLoaded`) and then `load()` again.

#### Broader split-layout detection

`isSplitLayout` now also triggers when only `clipL` or `clipG` is supplied. This closes a footgun where a FLUX.1 caller passing `{ model, clipL, clipG, vae }` (without `t5Xxl`) would silently mis-route the diffusion model into the all-in-one `path` parameter and fail to load.

### Bug Fixes

#### `unload()` clears the addon reference

`unload()` now sets `this.addon = null` after `await this.addon.unload()`, so post-unload `cancel()` / `run()` calls hit the explicit `if (!this.addon)` guard rather than dereferencing a disposed native handle.

#### Unknown addon events no longer pollute the output stream

`_addonOutputCallback` previously had a fallthrough that pushed any non-error / non-image / non-stats event into `response.output` (including `null` and `undefined`). It now logs unknown events at debug level and does not feed them into the active response.

#### Crash-safe activation

If `addon.activate()` throws during `_load()` (for example a native init failure or a missing model file discovered late), the partially-initialized addon is now best-effort-unloaded, the native logger is released, and `this.addon` is reset to `null`. A subsequent `load()` call starts cleanly instead of leaking a zombie native instance.

#### `load()` is serialized through the exclusive run queue

`load()` is now routed through the same `exclusiveRunQueue` used by `run()` and `unload()`. Previously two overlapping `load()` calls on the same instance could both pass the `configLoaded` guard before it flipped to `true`, both allocate a native addon, and clobber `this.addon` β€” leaking one native handle. Concurrent `load()` on a single instance is now safe.

### Pull Requests

- [#1496](https://github.com/tetherto/qvac/pull/1496) - chore[bc]: diffusion addon interface refactor β€” remove BaseInference

## [0.2.0] - 2026-04-15

### Added
Expand Down
41 changes: 24 additions & 17 deletions packages/lib-infer-diffusion/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,28 +176,35 @@ const path = require('bare-path')
const MODELS_DIR = path.resolve(__dirname, './models')
const args = {
logger: console,
diskPath: MODELS_DIR,
modelName: 'flux-2-klein-4b-Q8_0.gguf',
llmModel: 'Qwen3-4B-Q4_K_M.gguf', // Qwen3 text encoder for FLUX.2 [klein]
vaeModel: 'flux2-vae.safetensors'
files: {
model: path.join(MODELS_DIR, 'flux-2-klein-4b-Q8_0.gguf'),
llm: path.join(MODELS_DIR, 'Qwen3-4B-Q4_K_M.gguf'), // Qwen3 text encoder for FLUX.2 [klein]
vae: path.join(MODELS_DIR, 'flux2-vae.safetensors')
},
config: { threads: 8 },
opts: { stats: true }
}
```

| Property | Required | Description |
|----------|----------|-------------|
| `diskPath` | βœ… | Local directory where model files are already stored |
| `modelName` | βœ… | Diffusion model file name (all-in-one for SD1.x/2.x; diffusion-only GGUF for FLUX.2) |
| `files` | βœ… | Object of absolute paths to model files (see below) |
| `files.model` | βœ… | Absolute path to diffusion model file (all-in-one for SD1.x/2.x; diffusion-only GGUF for FLUX.2) |
| `files.clipL` | β€” | Absolute path to separate CLIP-L text encoder (SD3) |
| `files.clipG` | β€” | Absolute path to separate CLIP-G text encoder (SDXL / SD3) |
| `files.t5Xxl` | β€” | Absolute path to separate T5-XXL text encoder (SD3) |
| `files.llm` | β€” | Absolute path to Qwen3 LLM text encoder (FLUX.2 [klein]) |
| `files.vae` | β€” | Absolute path to separate VAE file |
| `config` | β€” | Native backend configuration object (see next section) |
| `logger` | β€” | Logger instance (e.g. `console`) |
| `clipLModel` | β€” | Separate CLIP-L text encoder (SD3) |
| `clipGModel` | β€” | Separate CLIP-G text encoder (SDXL / SD3) |
| `t5XxlModel` | β€” | Separate T5-XXL text encoder (SD3) |
| `llmModel` | β€” | Qwen3 LLM text encoder (FLUX.2 [klein]) |
| `vaeModel` | β€” | Separate VAE file |
| `opts` | β€” | Additional options (e.g. `{ stats: true }`) |

### 3. Create the `config` object
### 3. Configure the native backend (`args.config`)

`config` is a field on the `args` object built in step 2 β€” there is no separate constructor argument. The native backend reads it during `load()`.

```js
const config = {
args.config = {
threads: 8 // CPU threads for tensor operations (Metal handles GPU automatically)
}
```
Expand All @@ -216,18 +223,18 @@ Config values are coerced to strings internally. Generation parameters (prompt,
### 4. Create a Model Instance

```js
const model = new ImgStableDiffusion(args, config)
const model = new ImgStableDiffusion(args)
```

The constructor stores configuration only β€” no memory is allocated yet.
The constructor takes a single object containing `files`, `config`, `logger`, and `opts`. It stores configuration only β€” no memory is allocated yet.

### 5. Load the Model

```js
await model.load()
```

This creates the native `sd_ctx_t` and loads all weights into memory. It can take 10–30 seconds depending on disk speed and model size. All model files must already be present on disk at `diskPath`.
This creates the native `sd_ctx_t` and loads all weights into memory. It can take 10–30 seconds depending on disk speed and model size. All model files must be passed as absolute paths via the `files` object.

### 6. Run Inference

Expand Down Expand Up @@ -360,7 +367,7 @@ await model.unload()

### Stable Diffusion 1.x / 2.x

Pass an all-in-one checkpoint directly as `modelName`. No separate encoders needed.
Pass an all-in-one checkpoint absolute path as `files.model`. No separate encoders needed.

---

Expand Down
28 changes: 27 additions & 1 deletion packages/lib-infer-diffusion/addon.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,32 @@

const path = require('bare-path')

/**
* Normalize a raw native event into `Output` (image bytes or progress
* tick), `Error`, or `JobEnded`. Returns `null` for unknown shapes
* (caller logs and skips).
*
* @param {string} rawEvent
* @param {*} rawData
* @param {*} rawError
* @returns {{ type: string, data: *, error: * } | null}
*/
function mapAddonEvent (rawEvent, rawData, rawError) {
if (typeof rawEvent === 'string' && rawEvent.includes('Error')) {
return { type: 'Error', data: rawData, error: rawError }
}

if (rawData instanceof Uint8Array || typeof rawData === 'string') {
return { type: 'Output', data: rawData, error: null }
}

if (rawData && typeof rawData === 'object') {
return { type: 'JobEnded', data: rawData, error: null }
}

return null
}
Comment thread
jesusmb1995 marked this conversation as resolved.

/**
* Extract pixel dimensions from a PNG or JPEG buffer without a full decode.
*
Expand Down Expand Up @@ -151,4 +177,4 @@ class SdInterface {
}
}

module.exports = { SdInterface, readImageDimensions }
module.exports = { SdInterface, mapAddonEvent, readImageDimensions }
Original file line number Diff line number Diff line change
Expand Up @@ -575,7 +575,6 @@ std::any SdModel::process(const std::any& input) {
if (gen.width == 512 && gen.height == 512) {
genParams.width = imgW;
genParams.height = imgH;
genParams.height = imgH;
}
gen.width = genParams.width;
gen.height = genParams.height;
Expand Down
Loading
Loading