Autocomplete unexpectedly overrides sortText with local heuristics

[fiksen99]

Does this issue occur when all extensions are disabled?: Yes/No

• VS Code Version: 1.58
• OS Version: Mac OS 11.6

In our custom C++ language server extension, we are performing some ML modeling over our codebase to improve the relevancy of autocomplete results. Once the model has determined the "best" ordering of autocomplete results based on what the user has typed etc., we specify the sortText in the CompletionItems response to correctly sort the results in the client.

What we have found is that VS Code "thinks it knows better" and applies its own heuristics on top of our ML model. In particular, we see that VS Code will always boost the autocomplete results which match the casing of the text typed. So in our example (note I've appended the "expected" autocomplete index to the label for easier comparison of expected vs actual result), we see that co autocomplete results push ConfigeratorApi to the bottom of the list, when it should be at index 2. Conversely, autocompleting Co pushes ConfigeratorApi to the top, when again it should be at index 2.

How can we prevent VS Code client from applying it's own heuristics to our autocomplete sort order returned?

[fiksen99]

See the following gist which is the autocomplete request and response LSP values, then observe the final list displayed in VS Code. Observe VS Code has applied its own heuristic to the ordering based on case matching of the prefix.
https://gist.github.com/fiksen99/576acc7140fa1b06d95d75c5113be097

[mxrachel]

Your version of VS Code is outdated. Please try updating the software.

[fiksen99]

We are currently using an internal forked version of VS Code from this version, however I suspect this behavior is intentional. Is there a mechanism we can leverage to avoid this behavior?

[jrieken]

Yes - the sort text looses its relevance when user have a prefix typed. We like to think about that as "user typing selects suggestion". This doc tries do clarify this behavior. We understand that language authors tend to disagree with this, because you spend lots of thoughts on a particular sorting, but users typing prefixes or other selecting characters are always given precedence.

[svsool]

Hi,

I noticed the same problem, did the logic change at some point? Remember it was working as expected before.

In my extension completion items (wiki links) sorted by path (can be seen here and here) which makes sense considering how extension is used.

When I type [[note in the editor it shows the following:

According to sortText in the list bellow I'd expect Features/Note1 to come before How to/Note1, also note and Note items to come in the end of list and not in the beginning.

[
    {
        "label": "Features/Note1",
        "kind": 16,
        "insertText": "Note1",
        "sortText": "19"
    },
    {
        "label": "Features/Opening links in the default app",
        "kind": 16,
        "insertText": "Opening links in the default app",
        "sortText": "20"
    },
    {
        "label": "Features/Short and long links support",
        "kind": 16,
        "insertText": "Short and long links support",
        "sortText": "21"
    },
    {
        "label": "How to/How to",
        "kind": 16,
        "insertText": "How to",
        "sortText": "22"
    },
    {
        "label": "How to/Note1",
        "kind": 16,
        "insertText": "How to/Note1",
        "sortText": "23"
    },
    {
        "label": "How to/Pasting HTML as Markdown",
        "kind": 16,
        "insertText": "Pasting HTML as Markdown",
        "sortText": "24"
    },
    {
        "label": "How to/Pasting images from clipboard",
        "kind": 16,
        "insertText": "Pasting images from clipboard",
        "sortText": "25"
    },
    {
        "label": "Attachments/Automatic link synchronization 2.gif",
        "kind": 16,
        "insertText": "Automatic link synchronization 2.gif",
        "sortText": "26"
    },
    {
        "label": "Attachments/Automatic link synchronization.gif",
        "kind": 16,
        "insertText": "Automatic link synchronization.gif",
        "sortText": "27"
    },
    {
        "label": "Attachments/Backlinks panel.png",
        "kind": 16,
        "insertText": "Backlinks panel.png",
        "sortText": "28"
    },
    {
        "label": "Attachments/Creating links.gif",
        "kind": 16,
        "insertText": "Creating links.gif",
        "sortText": "29"
    },
    {
        "label": "Attachments/Creating notes from links.png",
        "kind": 16,
        "insertText": "Creating notes from links.png",
        "sortText": "30"
    },
    {
        "label": "Attachments/Embed files.gif",
        "kind": 16,
        "insertText": "Embed files.gif",
        "sortText": "31"
    },
    {
        "label": "Attachments/Find all references.png",
        "kind": 16,
        "insertText": "Find all references.png",
        "sortText": "32"
    },
    {
        "label": "Attachments/Links labeling.png",
        "kind": 16,
        "insertText": "Links labeling.png",
        "sortText": "33"
    },
    {
        "label": "Attachments/Links navigation.gif",
        "kind": 16,
        "insertText": "Links navigation.gif",
        "sortText": "34"
    },
    {
        "label": "Attachments/Notes and images preview.gif",
        "kind": 16,
        "insertText": "Notes and images preview.gif",
        "sortText": "35"
    },
    {
        "label": "Attachments/Open daily note.gif",
        "kind": 16,
        "insertText": "Open daily note.gif",
        "sortText": "36"
    },
    {
        "label": "Attachments/Open link command.png",
        "kind": 16,
        "insertText": "Open link command.png",
        "sortText": "37"
    },
    {
        "label": "Attachments/Open link to the side.png",
        "kind": 16,
        "insertText": "Open link to the side.png",
        "sortText": "38"
    },
    {
        "label": "Attachments/Open random note.gif",
        "kind": 16,
        "insertText": "Open random note.gif",
        "sortText": "39"
    },
    {
        "label": "Attachments/Opening common file types in the default app.gif",
        "kind": 16,
        "insertText": "Opening common file types in the default app.gif",
        "sortText": "40"
    },
    {
        "label": "Attachments/Opening links in the default app.gif",
        "kind": 16,
        "insertText": "Opening links in the default app.gif",
        "sortText": "41"
    },
    {
        "label": "Attachments/Pasting HTML as Markdown.gif",
        "kind": 16,
        "insertText": "Pasting HTML as Markdown.gif",
        "sortText": "42"
    },
    {
        "label": "Attachments/Short and long links support 2.png",
        "kind": 16,
        "insertText": "Short and long links support 2.png",
        "sortText": "43"
    },
    {
        "label": "Attachments/Short and long links support.png",
        "kind": 16,
        "insertText": "Short and long links support.png",
        "sortText": "44"
    },
    {
        "label": "Attachments/Structure 1.png",
        "kind": 16,
        "insertText": "Structure 1.png",
        "sortText": "45"
    },
    {
        "label": "Attachments/Structure 2.png",
        "kind": 16,
        "insertText": "Structure 2.png",
        "sortText": "46"
    },
    {
        "label": "Attachments/Dummy.pdf",
        "kind": 16,
        "insertText": "Dummy.pdf",
        "sortText": "47"
    },
    {
        "label": "link",
        "kind": 16,
        "insertText": "link",
        "sortText": "48"
    },
    {
        "label": "note",
        "kind": 16,
        "insertText": "note",
        "sortText": "49"
    },
    {
        "label": "Note",
        "kind": 16,
        "insertText": "Note",
        "sortText": "50"
    }
]

I read the doc that you've shared and a bit struggle to understand why it works the way it is and what's the purpose of the sortText prop now.

Is there a way to instruct VSCode to show completion items the way extension sorted them?

Thank you!