Performance issue write/edit long files

[CsBigDataHub]

Messages buffer shows -

error in process filter: eca-process--filter: Args out of range: "ent-Length: 1074

Debugger error -

Debugger entered--entering a function:
* #f(compiled-function () #<bytecode 0x4faff5a36df8bc1>)()
  markdown-match-italic(30000)
  font-lock-fontify-keywords-region(12178 30000 nil)
  font-lock-default-fontify-region(22104 22105 nil)
  font-lock-fontify-region(22104 22105)
  #f(compiled-function (fun) #<bytecode 0x1ba2db84c408d1bf>)(font-lock-fontify-region)
  run-hook-wrapped(#f(compiled-function (fun) #<bytecode 0x1ba2db84c408d1bf>) font-lock-fontify-region)
  jit-lock--run-functions(22104 22105)
  jit-lock-fontify-now(22104 22105)
  jit-lock-function(22104)
  redisplay_internal\ \(C\ function\)()
  accept-process-output(#<process ispell>)
  flyspell-word()
  flyspell-post-command-hook()

to bring emacs back I trigger pkill -SIGUSR2 Emacs

[ericdallo]

@CsBigDataHub I can't see how this could be related to eca-emacs, seems slike some issue with flyspell, I can't repro that

[CsBigDataHub]

I will look in to it further but it happened twice, I do tab on preparing tool: eca_edit_file to see what's happening, emacs immediately freezes and I do see emacs process capping one of the processor core at 99.4%.

So I trigger pkill -SIGUSR2 Emacs from the terminal to kill the process.

I do see bunch of json appear in *Messages* buffer, likely response from llm.

[ericdallo]

Does tab means expanding the tool call? this is an expensive operation for emacs indeed, but should not be that bad.
Also, I improved performance recently about that, so make sure you are on latest eca-emacs version.

[CsBigDataHub]

Does tab means expanding the tool call?

Yes.

make sure you are on latest eca-emacs version.

Will do

[ericdallo]

@CsBigDataHub I did some profiling and you are right, it seems eca-emacs has a performance issue with writing/creating long diffs, this is probably something to improve here, but not sure hoe much of that is emacs's faults, but probably there are hacks we can do to improve that. Any help is welcome

[CsBigDataHub]

@CsBigDataHub I did some profiling and you are right, it seems eca-emacs has a performance issue with writing/creating long diffs, this is probably something to improve here, but not sure hoe much of that is emacs's faults, but probably there are hacks we can do to improve that. Any help is welcome

No worries, can you please share your profiling report for me to compare.

[ericdallo]

What I can see from that is: a lot of GC happens and the biggest time is spend on that function 606 which is the eca-process--filter, which is a function called everytime we receive a message.

For writes/edits, we receives lots of toolCall/prepare messages from server, which is the LLM outputing the text that wanna call the tool, but most of the time, it's irrelevant since user wanna see the full args, which come on toolCall/run once.
So we could:

• improve the performance of the message parsing/receiving: I believe it's hard, have no clue what to improve.
• Improve the performance of toolCall/Prepare
• maybe have a flag on eca-emacs or server to ignore subsequent toolCall/prepare and send only the first one?

[CsBigDataHub]

• Every toolCallPrepare message triggers expensive operations:

  eca-chat--get-expandable-content lookup

  eca-chat--update-expandable-content with content appending

  eca-chat--content-table processing

  Buffer modification and overlay updates

• The current logic already handles duplicate IDs by appending content, but still processes every message

• Performance bottleneck is in the frequent buffer updates and expandable content system during heavy message flow

[CsBigDataHub]

Thinking out loud here -

can we introduce

(defcustom eca-chat-tool-prepare-throttle 'smart
  "How to handle toolCallPrepare messages to improve performance.
- 'all: Process all toolCallPrepare messages (current behavior)
- 'first-only: Only process the first toolCallPrepare per tool call ID
- 'smart: Process first and periodic updates (every 5th message)
- 'none: Skip all toolCallPrepare messages entirely"
  :type '(choice (const :tag "All messages" all)
                 (const :tag "First only" first-only)
                 (const :tag "Smart throttling" smart)
                 (const :tag "None" none))
  :group 'eca)

(defcustom eca-chat-tool-prepare-update-interval 5
  "For smart throttling, process every Nth toolCallPrepare message."
  :type 'integer
  :group 'eca)

;; Add to existing defvar-local declarations
(defvar-local eca-chat--tool-prepare-counters (make-hash-table :test 'equal)
  "Hash table tracking toolCallPrepare message counts per tool call ID.")

• Replace the existing "toolCallPrepare" case in eca-chat-content-received with :

("toolCallPrepare" (let* ((name (plist-get content :name))
                          (origin (plist-get content :origin))
                          (argsText (plist-get content :argumentsText))
                          (id (plist-get content :id))
                          (summary (plist-get content :summary))
                          (current-count (gethash id eca-chat--tool-prepare-counters 0))
                          (should-process-p
                           (pcase eca-chat-tool-prepare-throttle
                             ('all t)
                             ('none nil)
                             ('first-only (= current-count 0))
                             ('smart (or (= current-count 0)
                                        (= (mod current-count eca-chat-tool-prepare-update-interval) 0))))))
                     
                     ;; Always increment counter
                     (puthash id (1+ current-count) eca-chat--tool-prepare-counters)
                     
                     ;; Only process if throttling allows it
                     (when should-process-p
                       (let ((label (concat (propertize (or summary
                                                           (format "Preparing %s tool call: %s"
                                                                   (if (string= "mcp" origin) "MCP" "ECA")
                                                                   name))
                                                       'font-lock-face 'eca-chat-mcp-tool-call-label-face)
                                           " "
                                           eca-chat-mcp-tool-call-loading-symbol)))
                         (if (eca-chat--get-expandable-content id)
                             (eca-chat--update-expandable-content id label argsText t)
                           (eca-chat--add-expandable-content id
                                                            label
                                                            (eca-chat--content-table
                                                             `(("Tool" . ,name)
                                                               ("Arguments" . ,argsText)))))))))

• Add cleanup logic to the "toolCalled" handler to prevent memory leaks.

(remhash id eca-chat--tool-prepare-counters)

• Modify the eca-chat-reset function to clear the tracking hash table.

;; Add this line to eca-chat-reset function
(setq-local eca-chat--tool-prepare-counters (make-hash-table :test 'equal))

[CsBigDataHub]

We could also add configuration for better memory management, similar to LSP mode

(defcustom eca-chat-gc-cons-threshold (* 100 1024 1024)
  "Temporarily increase GC threshold during heavy message processing."
  :type 'integer
  :group 'eca)

;; In eca-process--filter, wrap message processing with:
(let ((gc-cons-threshold eca-chat-gc-cons-threshold))
  ;; Process messages here
  )