Skip to content

common : enable streaming JSON argument values#23173

Merged
aldehir merged 2 commits into
ggml-org:masterfrom
aldehir:remove-atomic-from-json-values
May 17, 2026
Merged

common : enable streaming JSON argument values#23173
aldehir merged 2 commits into
ggml-org:masterfrom
aldehir:remove-atomic-from-json-values

Conversation

@aldehir
Copy link
Copy Markdown
Contributor

@aldehir aldehir commented May 17, 2026 •
edited
Loading

Overview

For JSON argument values, such as Pi's edit tool, llama.cpp is not streaming JSON argument values. This leads to timeout issues such as earendil-works/pi#4519 for long-running edits.

This PR:

  • Removes atomic from JSON argument parsers.
  • Removes the additional JSON parsing logic in the mapper as it is unnecessary. The parser and the grammar are already doing the heavy lifting here. Additionally, the fallback logic leads to invalid diffs on negative numbers.

Requirements

@aldehir aldehir requested a review from a team as a code owner May 17, 2026 00:43
@aldehir aldehir merged commit f4cc787 into ggml-org:master May 17, 2026
39 of 49 checks passed
@aldehir aldehir deleted the remove-atomic-from-json-values branch May 17, 2026 08:44
kgrama pushed a commit to kgrama/llama.cpp that referenced this pull request May 19, 2026
@aldehir @kgrama
common : enable streaming JSON argument values (ggml-org#23173)
85e9386 
* common : remove atomic from json arguments

* common : remove parsing logic on JSON arguments
xxmustafacooTR pushed a commit to xxPlayground/llama-cpp-turboquant that referenced this pull request May 19, 2026
@aldehir @xxmustafacooTR
common : enable streaming JSON argument values (ggml-org#23173)
781c0cf 
* common : remove atomic from json arguments

* common : remove parsing logic on JSON arguments
rsenthilkumar6 pushed a commit to rsenthilkumar6/llama.cpp that referenced this pull request May 19, 2026
@aldehir @rsenthilkumar6
common : enable streaming JSON argument values (ggml-org#23173)
f35850f 
* common : remove atomic from json arguments

* common : remove parsing logic on JSON arguments
ArberSephirotheca pushed a commit to ArberSephirotheca/llama.cpp that referenced this pull request May 19, 2026
@aldehir @ArberSephirotheca
common : enable streaming JSON argument values (ggml-org#23173)
47c3f41 
* common : remove atomic from json arguments

* common : remove parsing logic on JSON arguments
fhnmor21 pushed a commit to fhnmor21/llama-cpp-turboquant that referenced this pull request May 19, 2026
@aldehir @fhnmor21
common : enable streaming JSON argument values (ggml-org#23173)
ddb57dc 
* common : remove atomic from json arguments

* common : remove parsing logic on JSON arguments
dbrain pushed a commit to dbrain/hbd-llama-cpp-turboquant that referenced this pull request May 21, 2026
@aldehir
common : enable streaming JSON argument values (ggml-org#23173)
f0f775a 
* common : remove atomic from json arguments

* common : remove parsing logic on JSON arguments
baramofme pushed a commit to baramofme/llama-cpp-turboquant that referenced this pull request May 23, 2026
@aldehir @baramofme
common : enable streaming JSON argument values (ggml-org#23173)
6d7e366 
* common : remove atomic from json arguments

* common : remove parsing logic on JSON arguments
srossitto79 pushed a commit to srossitto79/llama.cpp that referenced this pull request May 23, 2026
@aldehir @srossitto79
common : enable streaming JSON argument values (ggml-org#23173)
38692d7 
* common : remove atomic from json arguments

* common : remove parsing logic on JSON arguments
carlosfundora pushed a commit to carlosfundora/llama.cpp-1-bit-turbo that referenced this pull request May 24, 2026
@aldehir
common : enable streaming JSON argument values (ggml-org#23173)
b495e7e 
* common : remove atomic from json arguments

* common : remove parsing logic on JSON arguments

(cherry picked from commit f4cc787)
winstonma pushed a commit to winstonma/llama.cpp that referenced this pull request May 27, 2026
@aldehir @winstonma
common : enable streaming JSON argument values (ggml-org#23173)
2dc5aa3 
* common : remove atomic from json arguments

* common : remove parsing logic on JSON arguments
fewtarius pushed a commit to fewtarius/llama.cpp that referenced this pull request May 30, 2026
@aldehir
common : enable streaming JSON argument values (ggml-org#23173)
3e91bf1 
* common : remove atomic from json arguments

* common : remove parsing logic on JSON arguments
@aldehir aldehir mentioned this pull request Jun 2, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ggerganov ggerganov ggerganov approved these changes

@pwilkin pwilkin pwilkin approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@aldehir @ggerganov @pwilkin