Sync with llama-swap v178

LICENSE.md

@@ -1,6 +1,7 @@
 MIT License
 
 Copyright (c) 2024 Benson Wong
+Copyright (c) 2025 Aleksei Leshikhin
 
 Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
 

README.md

@@ -192,23 +192,26 @@ As a safeguard, llmsnap also sets `X-Accel-Buffering: no` on SSE responses. Howe
 
 ## Monitoring Logs on the CLI
 
-```shell
+```sh
 # sends up to the last 10KB of logs
-curl http://host/logs'
+$ curl http://host/logs
 
 # streams combined logs
-curl -Ns 'http://host/logs/stream'
+curl -Ns http://host/logs/stream
+
+# stream llmsnap's proxy status logs
+curl -Ns http://host/logs/stream/proxy
 
-# just llama-swap's logs
-curl -Ns 'http://host/logs/stream/proxy'
+# stream logs from upstream processes that llmsnap loads
+curl -Ns http://host/logs/stream/upstream
 
-# just upstream's logs
-curl -Ns 'http://host/logs/stream/upstream'
+# stream logs only from a specific model
+curl -Ns http://host/logs/stream/{model_id}
 
 # stream and filter logs with linux pipes
 curl -Ns http://host/logs/stream | grep 'eval time'
 
-# skips history and just streams new log entries
+# appending ?no-history will disable sending buffered history first
 curl -Ns 'http://host/logs/stream?no-history'
 ```
 

ai-plans/2025-12-14-efficient-ring-buffer.md

@@ -0,0 +1,85 @@
+# Replace ring.Ring with Efficient Circular Byte Buffer
+
+## Overview
+
+Replace the inefficient `container/ring.Ring` implementation in `logMonitor.go` with a simple circular byte buffer that uses a single contiguous `[]byte` slice. This eliminates per-write allocations, improves cache locality, and correctly implements a 10KB buffer.
+
+## Current Issues
+
+1. `ring.New(10 * 1024)` creates 10,240 ring **elements**, not 10KB of storage
+2. Every `Write()` call allocates a new `[]byte` slice inside the lock
+3. `GetHistory()` iterates all 10,240 elements and appends repeatedly (geometric reallocs)
+4. Linked list structure has poor cache locality and pointer overhead
+
+## Design Requirements
+
+### New CircularBuffer Type
+
+Create a simple circular byte buffer with:
+- Single pre-allocated `[]byte` of fixed capacity (10KB)
+- `head` and `size` integers to track write position and data length
+- No per-write allocations
+
+### API Requirements
+
+The new buffer must support:
+1. **Write(p []byte)** - Append bytes, overwriting oldest data when full
+2. **GetHistory() []byte** - Return all buffered data in correct order (oldest to newest)
+
+### Implementation Details
+
+```go
+type circularBuffer struct {
+    data []byte  // pre-allocated capacity
+    head int     // next write position
+    size int     // current number of bytes stored (0 to cap)
+}
+```
+
+**Write logic:**
+- If `len(p) >= capacity`: just keep the last `capacity` bytes
+- Otherwise: write bytes at `head`, wrapping around if needed
+- Update `head` and `size` accordingly
+- Data is copied into the internal buffer (not stored by reference)
+
+**GetHistory logic:**
+- Calculate start position: `(head - size + cap) % cap`
+- If not wrapped: single slice copy
+- If wrapped: two copies (end of buffer + beginning)
+- Returns a **new slice** (copy), not a view into internal buffer
+
+### Immutability Guarantees (must preserve)
+
+Per existing tests:
+1. Modifying input `[]byte` after `Write()` must not affect stored data
+2. `GetHistory()` returns independent copy - modifications don't affect buffer
+
+## Files to Modify
+
+- `proxy/logMonitor.go` - Replace `buffer *ring.Ring` with new circular buffer
+
+## Testing Plan
+
+Existing tests in `logMonitor_test.go` should continue to pass:
+- `TestLogMonitor` - Basic write/read and subscriber notification
+- `TestWrite_ImmutableBuffer` - Verify writes don't affect returned history
+- `TestWrite_LogTimeFormat` - Timestamp formatting
+
+Add new tests:
+- Test buffer wrap-around behavior
+- Test large writes that exceed buffer capacity
+- Test exact capacity boundary conditions
+
+## Checklist
+
+- [ ] Create `circularBuffer` struct in `logMonitor.go`
+- [ ] Implement `Write()` method for circular buffer
+- [ ] Implement `GetHistory()` method for circular buffer
+- [ ] Update `LogMonitor` struct to use new buffer
+- [ ] Update `NewLogMonitorWriter()` to initialize new buffer
+- [ ] Update `LogMonitor.Write()` to use new buffer
+- [ ] Update `LogMonitor.GetHistory()` to use new buffer
+- [ ] Remove `"container/ring"` import
+- [ ] Run `make test-dev` to verify existing tests pass
+- [ ] Add wrap-around test case
+- [ ] Run `make test-all` for final validation

config.example.yaml

@@ -46,6 +46,16 @@ logLevel: info
 # - For more info, read: https://pkg.go.dev/time#pkg-constants
 logTimeFormat: ""
 
+# logToStdout: controls what is logged to stdout
+# - optional, default: "proxy"
+# - valid values:
+#   - "proxy": logs generated by llmsnap when swapping models,
+#      handling requests, etc.
+#   - "upstream": a copy of an upstream processes stdout logs
+#   - "both": both the proxy and upstream logs interleaved together
+#   - "none": no logs are ever written to stdout
+logToStdout: "proxy"
+
 # metricsMaxInMemory: maximum number of metrics to keep in memory
 # - optional, default: 1000
 # - controls how many metrics are stored in memory before older ones are discarded

docs/configuration.md

@@ -90,6 +90,9 @@ llmsnap supports many more features to customize how you want to manage your env
 > This is a copy of `config.example.yaml`. Always check that for the most up to date examples.
 
 ```yaml
+# add this modeline for validation in vscode
+# yaml-language-server: $schema=https://raw.githubusercontent.com/napmany/llmsnap/refs/heads/main/config-schema.json
+#
 # llmsnap YAML configuration example
 # -----------------------------------
 #
@@ -115,6 +118,24 @@ healthCheckTimeout: 500
 # - Valid log levels: debug, info, warn, error
 logLevel: info
 
+# logTimeFormat: enables and sets the logging timestamp format
+# - optional, default (disabled): ""
+# - Valid values: "", "ansic", "unixdate", "rubydate", "rfc822", "rfc822z",
+#   "rfc850", "rfc1123", "rfc1123z", "rfc3339", "rfc3339nano", "kitchen",
+#   "stamp", "stampmilli", "stampmicro", and "stampnano".
+# - For more info, read: https://pkg.go.dev/time#pkg-constants
+logTimeFormat: ""
+
+# logToStdout: controls what is logged to stdout
+# - optional, default: "proxy"
+# - valid values:
+#   - "proxy": logs generated by llmsnap when swapping models,
+#      handling requests, etc.
+#   - "upstream": a copy of an upstream processes stdout logs
+#   - "both": both the proxy and upstream logs interleaved together
+#   - "none": no logs are ever written to stdout
+logToStdout: "proxy"
+
 # metricsMaxInMemory: maximum number of metrics to keep in memory
 # - optional, default: 1000
 # - controls how many metrics are stored in memory before older ones are discarded
@@ -139,6 +160,20 @@ wakeRequestTimeout: 20
 # - it is automatically incremented for every model that uses it
 startPort: 10001
 
+# sendLoadingState: inject loading status updates into the reasoning (thinking)
+# field
+# - optional, default: false
+# - when true, a stream of loading messages will be sent to the client in the
+#   reasoning field so chat UIs can show that loading is in progress.
+# - see #366 for more details
+sendLoadingState: true
+
+# includeAliasesInList: present aliases within the /v1/models OpenAI API listing
+# - optional, default: false
+# - when true, model aliases will be output to the API model listing duplicating
+#   all fields except for Id so chat UIs can use the alias equivalent to the original.
+includeAliasesInList: false
+
 # macros: a dictionary of string substitutions
 # - optional, default: empty dictionary
 # - macros are reusable snippets
@@ -287,6 +322,10 @@ models:
     # - recommended to be omitted and the default used
     concurrencyLimit: 0
 
+    # sendLoadingState: overrides the global sendLoadingState setting for this model
+    # - optional, default: undefined (use global setting)
+    sendLoadingState: false
+
   # Unlisted model example:
   "qwen-unlisted":
     # unlisted: boolean, true or false

proxy/config/config.go

@@ -15,6 +15,12 @@ import (
 )
 
 const DEFAULT_GROUP_ID = "(default)"
+const (
+	LogToStdoutProxy    = "proxy"
+	LogToStdoutUpstream = "upstream"
+	LogToStdoutBoth     = "both"
+	LogToStdoutNone     = "none"
+)
 
 type MacroEntry struct {
 	Name  string
@@ -116,6 +122,7 @@ type Config struct {
 	LogRequests         bool                   `yaml:"logRequests"`
 	LogLevel            string                 `yaml:"logLevel"`
 	LogTimeFormat       string                 `yaml:"logTimeFormat"`
+	LogToStdout         string                 `yaml:"logToStdout"`
 	MetricsMaxInMemory  int                    `yaml:"metricsMaxInMemory"`
 	Models              map[string]ModelConfig `yaml:"models"` /* key is model ID */
 	Profiles            map[string][]string    `yaml:"profiles"`
@@ -181,6 +188,7 @@ func LoadConfigFromReader(r io.Reader) (Config, error) {
 		StartPort:           5800,
 		LogLevel:            "info",
 		LogTimeFormat:       "",
+		LogToStdout:         LogToStdoutProxy,
 		MetricsMaxInMemory:  1000,
 	}
 	err = yaml.Unmarshal(data, &config)
@@ -207,6 +215,12 @@ func LoadConfigFromReader(r io.Reader) (Config, error) {
 		return Config{}, fmt.Errorf("startPort must be greater than 1")
 	}
 
+	switch config.LogToStdout {
+	case LogToStdoutProxy, LogToStdoutUpstream, LogToStdoutBoth, LogToStdoutNone:
+	default:
+		return Config{}, fmt.Errorf("logToStdout must be one of: proxy, upstream, both, none")
+	}
+
 	// Populate the aliases map
 	config.aliases = make(map[string]string)
 	for modelName, modelConfig := range config.Models {

proxy/config/config_posix_test.go

@@ -166,6 +166,7 @@ groups:
 	expected := Config{
 		LogLevel:      "info",
 		LogTimeFormat: "",
+		LogToStdout:   LogToStdoutProxy,
 		StartPort:     5800,
 		Macros: MacroList{
 			{"svr-path", "path/to/server"},

proxy/config/config_windows_test.go

@@ -158,6 +158,7 @@ groups:
 	expected := Config{
 		LogLevel:      "info",
 		LogTimeFormat: "",
+		LogToStdout:   LogToStdoutProxy,
 		StartPort:     5800,
 		Macros: MacroList{
 			{"svr-path", "path/to/server"},