Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion docs/spec/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,13 @@ keys used by model references.

Named model definitions own provider/model identity and metadata. Roles reference definitions,
so changing Main or Fallback does not destroy overrides belonging to the previous model.
Capability discovery does not write `ContextWindow`, `InputModalities`, or `OutputModalities`.
The daemon resolves those dynamic values at startup when the operator leaves them absent.

Older releases can contain discovery snapshots in these override fields. The stored shape

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rationale: I did not add an automatic migration. Older config does not record whether a field came from discovery or an operator. Automatic deletion could remove a real override. The existing clear flags provide a safe, explicit repair.

does not record each field's source. Netclaw preserves these values to protect explicit
operator overrides. Use `--clear-context-window` and `--clear-modalities` once to restore
runtime detection for an affected definition.

```json
{
Expand Down Expand Up @@ -142,7 +149,7 @@ so changing Main or Fallback does not destroy overrides belonging to the previou
|-------|------|---------|-------------|
| `Provider` | string | `"local-ollama"` | Key into the `Providers` dictionary. |
| `ModelId` | string | `"qwen3:30b"` | Model identifier as used by the provider's API. |
| `ContextWindow` | int? | `null` | Effective runtime context window in tokens. When set, it clamps the detected provider value. If not set, Netclaw uses the provider-reported value when available, otherwise defaults to 32,768. |
| `ContextWindow` | int? | `null` | Operator override for the runtime context window. When set, it clamps the detected provider value. If not set, Netclaw uses the provider-reported value when available, otherwise defaults to 32,768. Model selection does not persist a discovered value here. |
| `InputModalities` | string? | `null` | Manual override for input modalities. Comma-separated flags from `Text`, `Image`, `Audio`, `Video` — e.g. `"Text"` or `"Text, Image"`. When set, bypasses automated capability detection. |
| `OutputModalities` | string? | `null` | Manual override for output modalities. Same form as `InputModalities`. |

Expand Down
2 changes: 1 addition & 1 deletion feeds/skills/.system/files/netclaw-operations/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: netclaw-operations
description: "REQUIRED when the user asks about scheduling, reminders, cron jobs, timers, background jobs, diagnostics, troubleshooting, MCP tools, daemon health, identity updates, or Netclaw capabilities and self-maintenance."
metadata:
author: netclaw
version: "2.39.0"
version: "2.40.0"
---

# Netclaw Operations
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -88,19 +88,26 @@ openai` defaults to the ChatGPT OAuth device flow. Use `--auth api-key
`netclaw model set <role> <provider> <model-id>` creates or reuses a named model
definition and assigns it to a role (`main`, `fallback`, `compaction`). Definitions
own provider/model identity and metadata, while roles only reference definitions.
Switching away from a model and back therefore preserves its overrides. Two attributes can be overridden by the
operator and are **operator-owned**: the context window and the input/output
modalities. Provider discovery seeds a new definition but never changes an existing
definition, including adding a property the definition deliberately omits.
Switching away from a model and back therefore preserves its overrides. The operator
owns the context window and modality overrides.

Provider discovery validates the model ID for selection. It does not persist the
discovered context window or modalities. The daemon detects these values at each
startup. Only explicit operator flags or manual configuration create capability
overrides.

Older releases can contain discovery snapshots in the override fields. The config does
not record each field's source. Netclaw preserves these values to protect explicit
operator overrides. Run `netclaw model set` with `--clear-context-window` and
`--clear-modalities` once to restore runtime detection for an affected definition.

- `--context-window <tokens>` clamps the session budget and takes precedence
over provider-reported detection. Supplying it configures the model manually
and skips the metadata probe.
- `--input-modalities <list>` / `--output-modalities <list>` override detected
modalities with a comma-separated list of named flags (`Text`, `Image`,
`Audio`, `Video`). These do **not** skip the probe — the model is still
validated and its context window discovered; the override just wins over the
discovered modalities.
`Audio`, `Video`). These do **not** skip the probe. The probe still validates
the model ID. The override wins over runtime detection.
- `--clear-context-window` and `--clear-modalities` remove the respective
override so runtime capability detection resolves it again (use these after a
provider enlarges a model's window or fixes mis-reported modalities).
Expand Down
120 changes: 63 additions & 57 deletions src/Netclaw.Cli.Tests/Config/ModelEntryWriterTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
using System.Text.Json;
using Netclaw.Cli.Config;
using Netclaw.Configuration;
using Netclaw.Daemon.Configuration;
using Xunit;
using ModalityOverride = Netclaw.Cli.Config.ValueOverride<Netclaw.Configuration.ModelModality>;
using ContextWindowOverride = Netclaw.Cli.Config.ValueOverride<int>;
Expand All @@ -14,10 +15,9 @@ namespace Netclaw.Cli.Tests.Config;

/// <summary>
/// Guards the single persist path shared by `model set`, the init wizard, and the TUI
/// model manager. The rule under test: modalities are written only when the discovery
/// source genuinely reported them, so an unknown is never frozen into config as a
/// permanent "Text" override (#1290), and operator-owned overrides survive re-selection
/// (#1127 / #1610).
/// model manager. The writer persists capability values only after explicit operator input.
/// Runtime detection owns provider capability data (#1756).
/// Operator overrides survive model re-selection (#1127 and #1610).
/// </summary>
public class ModelEntryWriterTests
{
Expand Down Expand Up @@ -74,64 +74,90 @@ public void WriteRole_SameModelWithoutModalities_PreservesHandSetModalities()
// Re-set the same model with only a context-window change; no modality intent supplied.
ModelEntryWriter.WriteRole(
models, "Main", "spark", "qwen-vl", ModelDiscoverySource.Manual,
ContextWindowOverride.Set(131072), ModalityOverride.Unset, ModalityOverride.Unset, discovered: null);
ContextWindowOverride.Set(131072), ModalityOverride.Unset, ModalityOverride.Unset);

var entry = ActiveEntry(models, "Main");
Assert.Equal("Text, Image", entry["InputModalities"]); // preserved (#1127)
Assert.Equal(131072, entry["ContextWindow"]); // explicit override applied
}

[Fact]
public void WriteRole_SameModel_PreservesExistingClampOverDiscoveredWindow()
public void WriteRole_SameModel_PreservesExistingClampWithoutExplicitChange()
{
// Operator clamped ContextWindow below what the provider reports.
var models = Models(
"""
{ "Main": { "Provider": "spark", "ModelId": "qwen-vl", "ContextWindow": 32000 } }
""");

// Re-select the same model (picker path): no explicit --context-window, but the probe
// reports the model's full window. The operator's clamp must win (#1610): ContextWindow
// is documented to take precedence over provider-reported detection.
// Re-select the same model through a picker that has no override input.
ModelEntryWriter.WriteRole(
models, "Main", "spark", "qwen-vl", ModelDiscoverySource.Live,
ContextWindowOverride.Unset, ModalityOverride.Unset, ModalityOverride.Unset,
Discovered(contextWindow: 128000));
ContextWindowOverride.Unset, ModalityOverride.Unset, ModalityOverride.Unset);

var entry = ActiveEntry(models, "Main");
Assert.Equal(32000, entry["ContextWindow"]);
}

[Fact]
public void WriteRole_FirstTimeSet_UsesDiscoveredWindow()
public void WriteRole_FirstTimeSet_OmitsCapabilityOverrides()
{
// No existing entry for the role: discovery is the fallback that seeds the value.
var models = new Dictionary<string, object>();

ModelEntryWriter.WriteRole(
models, "Main", "spark", "qwen-vl", ModelDiscoverySource.Live,
ContextWindowOverride.Unset, ModalityOverride.Unset, ModalityOverride.Unset,
Discovered(contextWindow: 128000));
ContextWindowOverride.Unset, ModalityOverride.Unset, ModalityOverride.Unset);

var entry = ActiveEntry(models, "Main");
Assert.Equal(128000, entry["ContextWindow"]);
Assert.False(entry.ContainsKey("ContextWindow"));
Assert.False(entry.ContainsKey("InputModalities"));
Assert.False(entry.ContainsKey("OutputModalities"));
}

[Fact]
public void WriteRole_ClearContextWindow_DropsStoredClampAndDiscoveredWindow()
public void WriteRole_ProviderCapabilityChange_ReachesRuntimeResolution()
{
// Operator clamped the window; now --clear-context-window removes the clamp so runtime
// detection resolves it. Clear wins over BOTH the stored value and a probe that reports
// a window (#1610) — mirroring --clear-modalities.
var models = new Dictionary<string, object>();
var selectedDuringProbe = new DiscoveredModel
{
ModelId = new ModelId("deepseek-v4-flash-dspark"),
ContextWindowTokens = 327680,
InputModalities = ModelModality.Text,
OutputModalities = ModelModality.Text,
};

// The writer receives the selected identity, but it does not receive dynamic capabilities.
ModelEntryWriter.WriteRole(
models, "Main", "spark", selectedDuringProbe.ModelId.Value, ModelDiscoverySource.Live,
ContextWindowOverride.Unset, ModalityOverride.Unset, ModalityOverride.Unset);

var main = ConfigFileHelper.DeserializeSection<ModelReference>(ActiveEntry(models, "Main"))!;
var detectedAtStartup = new ResolvedModelCapabilities(
main.ModelId,
ModelModality.Text | ModelModality.Image,
ModelModality.Text,
100000);

var resolved = ModelCapabilityResolution.ResolveModelCapabilities(
new ModelSelection { Main = main }, detectedAtStartup);

Assert.Equal(100000, resolved.ContextWindowTokens);

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rationale: This test covers the complete producer-to-consumer contract. Selection sees 327680, startup sees 100000, and the runtime selects 100000. The modality change proves the same rule for all dynamic capability fields.

Assert.Equal(ModelModality.Text | ModelModality.Image, resolved.InputModalities);
Assert.Equal(ModelModality.Text, resolved.OutputModalities);
}

[Fact]
public void WriteRole_ClearContextWindow_DropsStoredClamp()
{
// The operator removes the clamp. Runtime detection resolves the window.
var models = Models(
"""
{ "Main": { "Provider": "spark", "ModelId": "qwen-vl", "ContextWindow": 32000 } }
""");

ModelEntryWriter.WriteRole(
models, "Main", "spark", "qwen-vl", ModelDiscoverySource.Live,
ContextWindowOverride.Clear, ModalityOverride.Unset, ModalityOverride.Unset,
Discovered(contextWindow: 128000));
ContextWindowOverride.Clear, ModalityOverride.Unset, ModalityOverride.Unset);

var entry = ActiveEntry(models, "Main");
Assert.False(entry.ContainsKey("ContextWindow")); // clamp removed → runtime detection
Expand All @@ -147,27 +173,22 @@ public void WriteRole_SameDefinitionWithoutWindow_DiscoveryDoesNotResurrect()

ModelEntryWriter.WriteRole(
models, "Main", "spark", "qwen-vl", ModelDiscoverySource.Live,
ContextWindowOverride.Unset, ModalityOverride.Unset, ModalityOverride.Unset,
Discovered(contextWindow: 128000));
ContextWindowOverride.Unset, ModalityOverride.Unset, ModalityOverride.Unset);

Assert.False(ActiveEntry(models, "Main").ContainsKey("ContextWindow"));
}

[Fact]
public void WriteRole_SameModelFreshProbe_DoesNotOverrideExistingModalities()
public void WriteRole_SameModel_PreservesExistingModalities()
{
// Operator override on disk; a fresh probe reports a coarser Text-only capability.
var models = Models(
"""
{ "Main": { "Provider": "spark", "ModelId": "qwen-vl", "InputModalities": "Text, Image" } }
""");

// The stored override wins — discovery never silently overwrites it (#1610 / #5). This is
// the same rule as ContextWindow: the field is documented to bypass automated detection.
ModelEntryWriter.WriteRole(
models, "Main", "spark", "qwen-vl", ModelDiscoverySource.Live,
ContextWindowOverride.Unset, ModalityOverride.Unset, ModalityOverride.Unset,
Discovered(input: ModelModality.Text, output: ModelModality.Text));
ContextWindowOverride.Unset, ModalityOverride.Unset, ModalityOverride.Unset);

var entry = ActiveEntry(models, "Main");
Assert.Equal("Text, Image", entry["InputModalities"]);
Expand All @@ -187,8 +208,7 @@ public void WriteRole_SameModelEntryClearedModality_DiscoveryDoesNotResurrect()

ModelEntryWriter.WriteRole(
models, "Main", "spark", "qwen-vl", ModelDiscoverySource.Live,
ContextWindowOverride.Unset, ModalityOverride.Unset, ModalityOverride.Unset,
Discovered(input: ModelModality.Text | ModelModality.Image));
ContextWindowOverride.Unset, ModalityOverride.Unset, ModalityOverride.Unset);

var entry = ActiveEntry(models, "Main");
Assert.False(entry.ContainsKey("InputModalities")); // stays cleared, not resurrected
Expand All @@ -207,7 +227,7 @@ public void WriteRole_ExplicitModalityOverride_ReplacesExisting()
ModelEntryWriter.WriteRole(
models, "Main", "spark", "qwen-vl", ModelDiscoverySource.Manual,
ContextWindowOverride.Unset,
ModalityOverride.Set(ModelModality.Text), ModalityOverride.Unset, discovered: null);
ModalityOverride.Set(ModelModality.Text), ModalityOverride.Unset);

var entry = ActiveEntry(models, "Main");
Assert.Equal("Text", entry["InputModalities"]);
Expand All @@ -221,12 +241,10 @@ public void WriteRole_ClearModalities_RemovesOverrideEvenWhenProbeReportsOne()
{ "Main": { "Provider": "spark", "ModelId": "qwen-vl", "InputModalities": "Text, Image", "OutputModalities": "Text" } }
""");

// --clear-modalities removes both overrides so runtime detection resolves them; clear
// wins over BOTH the stored value and a probe that still reports modalities (#1610 / #4).
// The command removes both overrides. Runtime detection resolves the modalities.
ModelEntryWriter.WriteRole(
models, "Main", "spark", "qwen-vl", ModelDiscoverySource.Manual,
ContextWindowOverride.Unset, ModalityOverride.Clear, ModalityOverride.Clear,
Discovered(input: ModelModality.Text | ModelModality.Image));
ContextWindowOverride.Unset, ModalityOverride.Clear, ModalityOverride.Clear);

var entry = ActiveEntry(models, "Main");
Assert.False(entry.ContainsKey("InputModalities"));
Expand All @@ -246,7 +264,7 @@ public void WriteRole_ExistingEntryOmitsModelId_DoesNotFalseMatchDefaultModel()

ModelEntryWriter.WriteRole(
models, "Main", "local-ollama", "qwen3:30b", ModelDiscoverySource.Manual,
ContextWindowOverride.Unset, ModalityOverride.Unset, ModalityOverride.Unset, discovered: null);
ContextWindowOverride.Unset, ModalityOverride.Unset, ModalityOverride.Unset);

var entry = ActiveEntry(models, "Main");
Assert.Equal("qwen3:30b", entry["ModelId"]);
Expand All @@ -269,7 +287,7 @@ public void WriteRole_SameModel_PreservesProvenanceUnlessFreshlyDiscovered(

ModelEntryWriter.WriteRole(
models, "Main", "spark", "qwen-vl", incoming,
ContextWindowOverride.Unset, ModalityOverride.Unset, ModalityOverride.Unset, discovered: null);
ContextWindowOverride.Unset, ModalityOverride.Unset, ModalityOverride.Unset);

var entry = ActiveEntry(models, "Main");
Assert.Equal(expected.ToString(), entry["Provenance"]);
Expand All @@ -288,8 +306,7 @@ public void WriteRole_CorruptExistingEntry_OverwritesInsteadOfThrowing()

var ex = Record.Exception(() => ModelEntryWriter.WriteRole(
models, "Main", "spark", "qwen-vl", ModelDiscoverySource.Live,
ContextWindowOverride.Unset, ModalityOverride.Unset, ModalityOverride.Unset,
Discovered(contextWindow: 128000)));
ContextWindowOverride.Unset, ModalityOverride.Unset, ModalityOverride.Unset));

Assert.Null(ex);
var entry = ActiveEntry(models, "Main");
Expand All @@ -311,8 +328,7 @@ public void WriteRole_CorruptModalityButValidWindow_PreservesWindow()

ModelEntryWriter.WriteRole(
models, "Main", "spark", "qwen-vl", ModelDiscoverySource.Live,
ContextWindowOverride.Unset, ModalityOverride.Unset, ModalityOverride.Unset,
Discovered(contextWindow: 128000));
ContextWindowOverride.Unset, ModalityOverride.Unset, ModalityOverride.Unset);

var entry = ActiveEntry(models, "Main");
Assert.Equal(32768, entry["ContextWindow"]); // operator clamp preserved, not clobbered
Expand All @@ -331,12 +347,11 @@ public void WriteRole_CorruptEntryForDifferentModel_DoesNotLeakWindow()

ModelEntryWriter.WriteRole(
models, "Main", "spark", "qwen-vl", ModelDiscoverySource.Live,
ContextWindowOverride.Unset, ModalityOverride.Unset, ModalityOverride.Unset,
Discovered(contextWindow: 128000));
ContextWindowOverride.Unset, ModalityOverride.Unset, ModalityOverride.Unset);

var entry = ActiveEntry(models, "Main");
Assert.Equal("qwen-vl", entry["ModelId"]);
Assert.Equal(128000, entry["ContextWindow"]); // discovered window, not the other model's 32768
Assert.False(entry.ContainsKey("ContextWindow"));
}

[Fact]
Expand All @@ -350,13 +365,13 @@ public void WriteRole_SwitchAwayAndBack_PreservesPreviousModelModalities()
// Switching roles changes only the role reference. The old definition remains intact.
ModelEntryWriter.WriteRole(
models, "Main", "spark", "other-model", ModelDiscoverySource.Manual,
ContextWindowOverride.Unset, ModalityOverride.Unset, ModalityOverride.Unset, discovered: null);
ContextWindowOverride.Unset, ModalityOverride.Unset, ModalityOverride.Unset);

Assert.Equal("other-model", ActiveEntry(models, "Main")["ModelId"]);

ModelEntryWriter.WriteRole(
models, "Main", "spark", "qwen-vl", ModelDiscoverySource.Manual,
ContextWindowOverride.Unset, ModalityOverride.Unset, ModalityOverride.Unset, discovered: null);
ContextWindowOverride.Unset, ModalityOverride.Unset, ModalityOverride.Unset);

Assert.Equal("Text, Image", ActiveEntry(models, "Main")["InputModalities"]);
}
Expand All @@ -373,13 +388,4 @@ private static Dictionary<string, object> ActiveEntry(
return (Dictionary<string, object>)definitions[definitionName];
}

private static DiscoveredModel Discovered(
int? contextWindow = null, ModelModality? input = null, ModelModality? output = null)
=> new()
{
ModelId = new ModelId("qwen-vl"),
ContextWindowTokens = contextWindow,
InputModalities = input,
OutputModalities = output,
};
}
Loading
Loading