Cannot stream results from candidates function

[snyball]

I'm trying to use vertico to search an sqlite database, and I don't want to return all the results at once, because there are 10s of thousands of results initially. This works fine with the default completing read frontend in Emacs, but vertico will not update the candidates. Instead the candidates are only updated once there are no candidates that match any of the initial results (using the completion-style.)

The same problem can be seen in corfu.

(require 'pcomplete)
(require 'pcmpl-git)

(defconst pcmpl-cargo--package-directory (file-name-directory load-file-name))
(defconst pcmpl-cargo--crates-query
  "SELECT name FROM crates AS c WHERE c.name LIKE ? ORDER BY DOWNLOADS DESC LIMIT 20")
(defvar pcmpl-cargo--db nil)

(defun pcmpl-cargo--open-db ()
  "Open the database."
  (unless (sqlite-available-p)
    (error "This version of Emacs was not compiled with sqlite support"))
  (setq pcmpl-cargo--db
        (sqlite-open (concat user-emacs-directory "/pcmpl-cargo/crates.sqlite"))))

(defun pcmpl-cargo--packages (prefix)
  "Return the top 20 Rust crates that begin with PREFIX."
  (mapcar #'car (sqlite-select pcmpl-cargo--db
                               pcmpl-cargo--crates-query
                               `(,(concat prefix "%")))))

(defun cargo-add ()
  "Add a dependency to crate."
  (interactive)
  (completing-read "crate: "
                   #'(lambda (p s a)
                       (complete-with-action a (pcmpl-cargo--packages p) p s))))

How do I get vertico/corfu to update the candidates from the candidates function live. I think they already do this for lsp-capf, but I could be wrong.

[minad]

You can use completion-table-dynamic together with the basic completion style.

[snyball]

Thanks! That does work. However, why does it only work with the basic completion style?

I'm also not able to override the completion style with a category:

(setf (alist-get 'cargo-crates completion-category-overrides) '((styles . (basic))))
(setf (alist-get 'cargo-crates completion-category-defaults) '((styles . (basic))))
(defun cargo-add ()
  "Add a dependency to crate."
  (interactive)
  (completing-read "crate: " (lambda (str pred action)
                               (message (format "%s" completion-styles))
                               (if (eq action 'metadata)
                                   `(metadata
                                     (category . cargo-crates)))
                               (funcall (completion-table-dynamic #'pcmpl-cargo--packages)
                                        str pred action))))

Does vertico not support completion-category-overrides or am I doing something incorrect with the way I'm setting the overrides and/or setting the category of the completion function?

[minad]

Does vertico not support completion-category-overrides

Yes, it does.

[snyball]

@minad Ok, I'll try to see if there's anything else futzing with the completion styles by running this in a clean environment with vertico/corfu. Thanks for the help!