Skip to content
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
921871c
chore[bc]: remove BaseInference inheritance from diffusion addon
donriddo Apr 9, 2026
c76b809
fix: restore JSDoc comments in index.js and index.d.ts
donriddo Apr 9, 2026
3b27c36
docs: update SD README for new constructor pattern
donriddo Apr 10, 2026
71d885b
fix: guard SD _runInternal against run-before-load with clear error
donriddo Apr 10, 2026
c7e51a3
docs: align SD architecture.md with new constructor and composition p…
donriddo Apr 10, 2026
f5c9426
chore[bc]: address PR #1496 review findings and bump to 0.2.0
donriddo Apr 10, 2026
7083f7e
refactor: move SD C++ event normalization into addon.js
donriddo Apr 10, 2026
e506204
fix: address PR #1496 second-round review findings
donriddo Apr 10, 2026
4a3a261
Merge remote-tracking branch 'origin/main' into chore/sd-addon-interf…
donriddo Apr 10, 2026
ab71710
fix: remove task-doc reference and refactor-narration comments
donriddo Apr 12, 2026
45d3b19
fix: throw on second load(), log rejected responses, add mapAddonEven…
donriddo Apr 14, 2026
51bb1e3
fix: restore JSDoc on run() that was dropped during BaseInference rem…
donriddo Apr 14, 2026
5063a20
fix: correct CHANGELOG error quote and remove dead files.model fallback
donriddo Apr 14, 2026
f6f424d
fix: make load() idempotent when already loaded
donriddo Apr 15, 2026
0ec9a04
Merge remote-tracking branch 'upstream/main' into chore/sd-addon-inte…
donriddo Apr 15, 2026
8a6e82d
Merge remote-tracking branch 'upstream/main' into chore/sd-addon-inte…
donriddo Apr 15, 2026
943220f
doc: document missing breaking changes from BaseInference removal
donriddo Apr 15, 2026
de7e693
fix: address lifecycle, cleanup, and CI-surface review findings
donriddo Apr 16, 2026
2973126
fix[ci]: run test:unit inside test:integration flow
donriddo Apr 16, 2026
5519dea
fix[ci]: run test:unit via run-lint-and-unit-tests action
donriddo Apr 16, 2026
71d1c83
chore: test script chains test:unit + test:integration
donriddo Apr 16, 2026
1b302cb
doc: fix mermaid classDiagram parsing error in architecture.md
donriddo Apr 16, 2026
c4341e9
Merge remote-tracking branch 'upstream/main' into chore/sd-addon-inte…
donriddo Apr 16, 2026
ff980bd
chore[ci]: rename step to reflect what the action actually runs
donriddo Apr 16, 2026
78c8cd0
fix: doc and type drift around img2img; dead code in SdModel.cpp
donriddo Apr 16, 2026
0170b29
doc: refresh Key Features, migration marker, and img2img JSDoc
donriddo Apr 16, 2026
7250731
doc: restore JSDoc on SD cancel() and unload()
donriddo Apr 16, 2026
ff968f6
doc: trim verbose comments added during the refactor
donriddo Apr 16, 2026
a2710ba
doc: restore pre-refactor createAddon JSDoc and load error log
donriddo Apr 16, 2026
80819f5
fix: release native logger when addon construction throws
donriddo Apr 16, 2026
a0f71ad
chore: drop unused 'test' script, inline into 'test:all'
donriddo Apr 16, 2026
9f68237
doc: fix 0.3.0 CHANGELOG heading depth and queue serialization scope
donriddo Apr 16, 2026
1f4a8b9
doc: fix flowchart mermaid parse errors in data-flows
donriddo Apr 17, 2026
98657e6
doc: note FLUX.2 ignores strength in GenerationParams JSDoc
donriddo Apr 17, 2026
4032040
Merge branch 'main' into chore/sd-addon-interface-refactor
donriddo Apr 17, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 19 additions & 14 deletions packages/lib-infer-diffusion/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -172,23 +172,28 @@ const path = require('bare-path')
const MODELS_DIR = path.resolve(__dirname, './models')
const args = {
logger: console,
diskPath: MODELS_DIR,
modelName: 'flux-2-klein-4b-Q8_0.gguf',
llmModel: 'Qwen3-4B-Q4_K_M.gguf', // Qwen3 text encoder for FLUX.2 [klein]
vaeModel: 'flux2-vae.safetensors'
files: {
model: path.join(MODELS_DIR, 'flux-2-klein-4b-Q8_0.gguf'),
llm: path.join(MODELS_DIR, 'Qwen3-4B-Q4_K_M.gguf'), // Qwen3 text encoder for FLUX.2 [klein]
vae: path.join(MODELS_DIR, 'flux2-vae.safetensors')
},
config: { threads: 8 },
opts: { stats: true }
}
```

| Property | Required | Description |
|----------|----------|-------------|
| `diskPath` | βœ… | Local directory where model files are already stored |
| `modelName` | βœ… | Diffusion model file name (all-in-one for SD1.x/2.x; diffusion-only GGUF for FLUX.2) |
| `files` | βœ… | Object of absolute paths to model files (see below) |
| `files.model` | βœ… | Absolute path to diffusion model file (all-in-one for SD1.x/2.x; diffusion-only GGUF for FLUX.2) |
| `files.clipL` | β€” | Absolute path to separate CLIP-L text encoder (FLUX.1 / SD3) |
| `files.clipG` | β€” | Absolute path to separate CLIP-G text encoder (SDXL / SD3) |
| `files.t5Xxl` | β€” | Absolute path to separate T5-XXL text encoder (FLUX.1 / SD3) |
| `files.llm` | β€” | Absolute path to Qwen3 LLM text encoder (FLUX.2 [klein]) |
| `files.vae` | β€” | Absolute path to separate VAE file |
| `config` | β€” | Native backend configuration object (see next section) |
| `logger` | β€” | Logger instance (e.g. `console`) |
| `clipLModel` | β€” | Separate CLIP-L text encoder (FLUX.1 / SD3) |
| `clipGModel` | β€” | Separate CLIP-G text encoder (SDXL / SD3) |
| `t5XxlModel` | β€” | Separate T5-XXL text encoder (FLUX.1 / SD3) |
| `llmModel` | β€” | Qwen3 LLM text encoder (FLUX.2 [klein]) |
| `vaeModel` | β€” | Separate VAE file |
| `opts` | β€” | Additional options (e.g. `{ stats: true }`) |

### 3. Create the `config` object

Expand All @@ -212,7 +217,7 @@ All config values are coerced to strings internally before being passed to the n
### 4. Create a Model Instance

```js
const model = new ImgStableDiffusion(args, config)
const model = new ImgStableDiffusion(args)
```

The constructor stores configuration only β€” no memory is allocated yet.
Expand All @@ -223,7 +228,7 @@ The constructor stores configuration only β€” no memory is allocated yet.
await model.load()
```

This creates the native `sd_ctx_t` and loads all weights into memory. It can take 10–30 seconds depending on disk speed and model size. All model files must already be present on disk at `diskPath`.
This creates the native `sd_ctx_t` and loads all weights into memory. It can take 10–30 seconds depending on disk speed and model size. All model files must be passed as absolute paths via the `files` object.

### 6. Run Inference

Expand Down Expand Up @@ -316,7 +321,7 @@ await model.unload()

### Stable Diffusion 1.x / 2.x

Pass an all-in-one checkpoint directly as `modelName`. No separate encoders needed.
Pass an all-in-one checkpoint absolute path as `files.model`. No separate encoders needed.

---

Expand Down
95 changes: 53 additions & 42 deletions packages/lib-infer-diffusion/docs/architecture.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Architecture Documentation

**Package:** `@qvac/diffusion-cpp` v0.1.0
**Stack:** JavaScript, C++20, stable-diffusion.cpp, Bare Runtime, CMake, vcpkg
**Package:** `@qvac/diffusion-cpp` v0.1.2
**Stack:** JavaScript, C++20, stable-diffusion.cpp, Bare Runtime, CMake, vcpkg
**License:** Apache-2.0

---
Expand Down Expand Up @@ -49,7 +49,7 @@
## Key Features

- **Cross-platform**: macOS, Linux, Windows, iOS, Android
- **Disk-local models**: Files must be present on disk at `diskPath`
- **Disk-local models**: Files must be present on disk; the caller passes absolute file paths via `files.{model,clipL,clipG,t5Xxl,llm,vae}` to the constructor
- **Progress tracking**: Step-by-step generation progress callbacks
- **GPU acceleration**: Metal, Vulkan, OpenCL
- **Quantized models**: GGUF, safetensors, checkpoint formats
Expand Down Expand Up @@ -122,17 +122,17 @@ graph TB

| Package | Type | Version | Purpose |
|---------|------|---------|---------|
| @qvac/infer-base | Framework | ^0.2.0 | Base classes (BaseInference, QvacResponse) |
| qvac-lib-inference-addon-cpp | Native | β‰₯1.1.1 | C++ addon framework (single-job runner) |
| @qvac/infer-base | Framework | ^0.4.0 | Composition utilities (`createJobHandler`, `exclusiveRunQueue`, `QvacResponse`) |
| qvac-lib-inference-addon-cpp | Native | β‰₯1.1.2 | C++ addon framework (single-job runner) |
| stable-diffusion.cpp | Native | latest | Diffusion inference engine |
| Bare Runtime | Runtime | β‰₯1.24.0 | JavaScript execution |

**Integration Points:**

| From | To | Mechanism | Data Format |
|------|-----|-----------|-------------|
| JavaScript | ImgStableDiffusion | Constructor | args, config objects |
| ImgStableDiffusion | BaseInference | Inheritance | Template method pattern |
| JavaScript | ImgStableDiffusion | Constructor | Single `{ files, config, logger?, opts? }` object |
| ImgStableDiffusion | createJobHandler / exclusiveRunQueue | Composition | Job lifecycle + run-queue helpers from `@qvac/infer-base` |
| ImgStableDiffusion | SdInterface | Composition | Method calls |
| SdInterface | C++ Addon | require.addon() | Native binding |

Expand All @@ -147,20 +147,26 @@ graph TB
```mermaid
classDiagram
class ImgStableDiffusion {
+constructor(args, config)
+constructor(args: {files, config, logger?, opts?})
Comment thread
maxim-smotrov marked this conversation as resolved.
Outdated
+load() Promise~void~
+run(params: GenerationParams) Promise~QvacResponse~
+cancel() Promise~void~
+unload() Promise~void~
+getState() {configLoaded}
}

class BaseInference {
<<abstract>>
+load() Promise~void~
+run() Promise~QvacResponse~
+unload() Promise~void~
#_runInternal() Promise~QvacResponse~
#_withExclusiveRun(fn) Promise~any~
class JobHandler {
<<from @qvac/infer-base>>
+start() QvacResponse
+output(data)
+end(stats?, payload?)
+fail(error)
+active QvacResponse
}

class ExclusiveRunQueue {
<<from @qvac/infer-base>>
+(fn) Promise~T~
}

class QvacResponse {
Expand All @@ -169,8 +175,9 @@ classDiagram
+cancel() Promise~void~
}

ImgStableDiffusion --|> BaseInference
ImgStableDiffusion ..> QvacResponse : creates
ImgStableDiffusion ..> JobHandler : composes via createJobHandler()
ImgStableDiffusion ..> ExclusiveRunQueue : composes via exclusiveRunQueue()
JobHandler ..> QvacResponse : creates per start()
```

<details>
Expand All @@ -180,16 +187,20 @@ classDiagram

| Class | Responsibility | Lifecycle | Dependencies |
|-------|----------------|-----------|--------------|
| ImgStableDiffusion | Orchestrate model lifecycle, manage loading/inference | Created by user, persistent | SdInterface |
| BaseInference | Define standard inference API (template method pattern) | Abstract base class | None |
| QvacResponse | Handle generation progress and result | Created per `run()` call | None |
| ImgStableDiffusion | Orchestrate model lifecycle, manage loading/inference | Created by user, persistent | SdInterface, JobHandler, ExclusiveRunQueue |
| JobHandler (`createJobHandler`) | Start/end/fail a single in-flight job and emit a `QvacResponse` | Per-instance, lives as long as the model | None |
| ExclusiveRunQueue (`exclusiveRunQueue`) | Serialize public API calls so only one job is in flight at a time | Per-instance | None |
| QvacResponse | Handle generation progress and result | Created per `run()` call by the JobHandler | None |

**Key Relationships:**

| From | To | Type | Purpose |
|------|-----|------|---------|
| ImgStableDiffusion | BaseInference | Inheritance | Standard QVAC inference API |
| ImgStableDiffusion | QvacResponse | Creates | Progress/result per generation |
| ImgStableDiffusion | JobHandler | Composition | Lifecycle of the active job (replaces inheriting from `BaseInference`) |
| ImgStableDiffusion | ExclusiveRunQueue | Composition | Serializes `run()` / `cancel()` / `unload()` |
| JobHandler | QvacResponse | Creates | Progress/result per generation |

> **Note:** `ImgStableDiffusion` no longer extends `BaseInference`. It composes the helpers exposed by `@qvac/infer-base` (`createJobHandler`, `exclusiveRunQueue`) directly.

</details>

Expand All @@ -206,7 +217,7 @@ graph TB
subgraph "Layer 1: JavaScript API"
APP["Application Code"]
IMGCLASS["ImgStableDiffusion<br/>(index.js)"]
BASEINF["BaseInference<br/>(@qvac/infer-base)"]
BASE["createJobHandler / exclusiveRunQueue<br/>(@qvac/infer-base)"]
RESPONSE["QvacResponse"]
end

Expand Down Expand Up @@ -234,7 +245,7 @@ graph TB
end

APP --> IMGCLASS
IMGCLASS --> BASEINF
IMGCLASS --> BASE
IMGCLASS --> SDIF
IMGCLASS -.-> RESPONSE

Expand Down Expand Up @@ -267,7 +278,7 @@ graph TB

| Layer | Components | Responsibility | Language | Why This Layer |
|-------|------------|----------------|----------|----------------|
| 1. JavaScript API | ImgStableDiffusion, BaseInference, QvacResponse | High-level API, error handling | JS | Ergonomic API for npm consumers |
| 1. JavaScript API | ImgStableDiffusion, `createJobHandler` / `exclusiveRunQueue` (from `@qvac/infer-base`), QvacResponse | High-level API, error handling | JS | Ergonomic API for npm consumers |
| 2. Bridge | SdInterface, binding.js | JS↔C++ communication | JS wrapper | Lifecycle management, handle safety |
| 3. C++ Addon | JsInterface, AddonCpp/AddonJs | Single-job runner, threading, callbacks | C++ | Performance, native integration |
| 4. Model | SdModel, Contexts | Diffusion logic, sampling | C++ | Direct stable-diffusion.cpp integration |
Expand Down Expand Up @@ -533,43 +544,43 @@ See [qvac-lib-inference-addon-cpp Decision 4: Why Bare Runtime](https://github.c

---

## Decision 3: Disk-Local Model Files
## Decision 3: Disk-Local Model Files (caller-supplied absolute paths)

<details>
<summary>⚑ TL;DR</summary>

**Chose:** Require model files to already exist on disk at `diskPath`
**Why:** Simplicity β€” the addon loads files directly from disk, no streaming/download layer needed
**Cost:** Caller must ensure files are present before calling `load()`
**Chose:** Require model files to already exist on disk; the caller passes absolute paths via `files.{model,clipL,clipG,t5Xxl,llm,vae}`
**Why:** Simplicity β€” the addon loads files directly from disk, no streaming/download layer needed and no loader abstraction
**Cost:** Caller must ensure files are present and supply absolute paths before calling `load()`

</details>

### Context

Diffusion models consist of multiple large files (diffusion model, text encoders, VAE). The addon needs these files to create the native `sd_ctx_t` context.

Unlike the LLM addon which historically used WeightsProvider for streaming weights, diffusion loads files directly from disk paths β€” no loader abstraction is involved.
Unlike the LLM addon which historically used WeightsProvider for streaming weights, diffusion has always loaded files directly from disk. After the addon-loader-abstraction refactor, there is also no `Loader` interface and no `diskPath` / `modelName` joining inside the addon β€” the caller passes absolute paths through the new `files` argument.

### Decision

Require all model files to be present on disk at `diskPath` before `load()` is called. The addon constructs file paths by joining `diskPath` with each model filename and passes them directly to stable-diffusion.cpp.
Require all model files to be present on disk before `load()` is called. The constructor accepts a single `files` object whose entries are absolute paths (`files.model` is required; `files.clipL`, `files.clipG`, `files.t5Xxl`, `files.llm`, `files.vae` are optional companions). `_load()` reads `this._files` and forwards the paths directly to stable-diffusion.cpp.

### Rationale

**Simplicity:**
- No download/streaming abstraction layer needed
- No WeightsProvider, no progress tracking for downloads
- No WeightsProvider, no Loader, no progress tracking for downloads
- Direct file paths to stable-diffusion.cpp

**Split-model support:**
- Diffusion models may have multiple components (diffusion GGUF, CLIP-L, CLIP-G, T5-XXL, LLM encoder, VAE)
- All resolved as `path.join(diskPath, filename)` in `_load()`
- Split vs all-in-one layout detected via heuristic (`isSplitLayout = !!llmModel || !!t5XxlModel`)
- The caller supplies each component as an absolute path on `files`
- Split vs all-in-one layout is detected via heuristic in `_load()` (`isSplitLayout = !!this._files.llm || !!this._files.t5Xxl`)

### Trade-offs
- βœ… Simple, no abstraction overhead
- βœ… No streaming/buffering complexity
- ❌ Caller responsible for ensuring files exist on disk
- ❌ Caller responsible for ensuring files exist on disk and for resolving absolute paths

---

Expand All @@ -578,19 +589,19 @@ Require all model files to be present on disk at `diskPath` before `load()` is c
<details>
<summary>⚑ TL;DR</summary>

**Chose:** Pass file paths directly to stable-diffusion.cpp via `sd_ctx_params_t`
**Chose:** Pass absolute file paths directly to stable-diffusion.cpp via `sd_ctx_params_t`
**Why:** stable-diffusion.cpp natively loads from file paths; no need for buffer intermediary
**Cost:** Files must exist on disk (no streaming from P2P sources)

</details>

### Context

stable-diffusion.cpp accepts model files via file paths in its context parameters (`model_path`, `diffusion_model_path`, `clip_l_path`, `vae_path`, etc.). The addon constructs these paths from `diskPath` + filenames.
stable-diffusion.cpp accepts model files via file paths in its context parameters (`model_path`, `diffusion_model_path`, `clip_l_path`, `vae_path`, etc.). The caller supplies these as absolute paths on the constructor's `files` object; the addon never joins a base directory with a filename.

### Decision

Pass absolute file paths directly to stable-diffusion.cpp rather than using buffer-based loading. The `_load()` method constructs a `configurationParams` object with resolved paths and passes it to the native addon.
Pass absolute file paths directly to stable-diffusion.cpp rather than using buffer-based loading. `_load()` builds a `configurationParams` object from `this._files` and passes it to the native addon as-is.

### Rationale

Expand Down Expand Up @@ -679,8 +690,8 @@ interface GenerationParams {
<details>
<summary>⚑ TL;DR</summary>

**Chose:** Promise-based exclusive run queue using `_withExclusiveRun()` wrapper
**Why:** Ensure generation jobs complete without interruption (long-running operations)
**Chose:** Compose `exclusiveRunQueue()` from `@qvac/infer-base` to serialize public API entrypoints
**Why:** Ensure generation jobs complete without interruption (long-running operations)
**Cost:** One generation at a time per model instance

</details>
Expand All @@ -691,7 +702,7 @@ Diffusion generation takes significant time (seconds to minutes). Without coordi

### Decision

Implement JavaScript-level promise queue ensuring only one generation job runs at a time per model instance.
Use the `exclusiveRunQueue()` helper from `@qvac/infer-base`. The constructor stores the queue as `this._run`, and `run()`, `cancel()`, and `unload()` all wrap their bodies with `this._run(() => …)`. This replaces the previous `BaseInference._withExclusiveRun()` template-method approach with a small composable utility.

### Rationale

Expand Down Expand Up @@ -760,4 +771,4 @@ Provide hand-written TypeScript definitions in `index.d.ts`.

---

**Last Updated:** 2026-03-11
**Last Updated:** 2026-04-10
19 changes: 9 additions & 10 deletions packages/lib-infer-diffusion/examples/generate-image-sd2.js
Original file line number Diff line number Diff line change
Expand Up @@ -49,21 +49,20 @@ async function main () {
console.log('Seed :', SEED)
console.log()

const model = new ImgStableDiffusion(
{
logger: console,
diskPath: MODELS_DIR,
modelName: MODEL_NAME
// No llmModel β€” SD2.1 uses the CLIP text encoder baked into the checkpoint.
// No vaeModel β€” the VAE is baked into the checkpoint.
const model = new ImgStableDiffusion({
files: {
model: path.join(MODELS_DIR, MODEL_NAME)
// No llm β€” SD2.1 uses the CLIP text encoder baked into the checkpoint.
// No vae β€” the VAE is baked into the checkpoint.
},
{
config: {
threads: 8,
// SD2.1 uses v-prediction. This safetensors file has no GGUF metadata so
// auto-detection cannot determine the prediction type; set it explicitly.
prediction: 'v'
}
)
},
logger: console
})

try {
// ── 1. Load weights ───────────────────────────────────────────────────────
Expand Down
19 changes: 9 additions & 10 deletions packages/lib-infer-diffusion/examples/generate-image-sd3.js
Original file line number Diff line number Diff line change
Expand Up @@ -54,24 +54,23 @@ async function main () {
console.log('Seed :', SEED)
console.log()

const model = new ImgStableDiffusion(
{
logger: console,
diskPath: MODELS_DIR,
modelName: MODEL_NAME
// All-in-one safetensors: no clipLModel, clipGModel, t5XxlModel, or vaeModel.
const model = new ImgStableDiffusion({
files: {
model: path.join(MODELS_DIR, MODEL_NAME)
// All-in-one safetensors: no clipL, clipG, t5Xxl, or vae.
//
// To add T5-XXL (better text following) without redownloading the main file:
// t5XxlModel: 't5xxl_fp8_e4m3fn.safetensors' // download via download-model-sd3.sh
// t5Xxl: path.join(MODELS_DIR, 't5xxl_fp8_e4m3fn.safetensors') // download via download-model-sd3.sh
},
{
config: {
threads: 4,
// SD3 uses flow-matching. The safetensors metadata allows auto-detection,
// but we set these explicitly as safety overrides.
prediction: 'flow', // FLOW_PRED β€” SD3 flow-matching
flow_shift: '3.0' // SD3 Medium default; overrides INFINITY sentinel
}
)
},
logger: console
})

try {
// ── 1. Load weights ───────────────────────────────────────────────────────
Expand Down
Loading
Loading