Git diff ignores language configured encoding

[vlopes11]

• VSCode Version: 1.22.2
• OS Version: Windows 10 x64 version 1709

Version 1.22.2
Commit 3aeede7
Date 2018-04-12T16:38:45.278Z
Shell 1.7.12
Renderer 58.0.3029.110
Node 7.9.0
Architecture x64

Steps to Reproduce:

1. Setup user settings according to this

    "[json]": {
        "files.encoding": "windows1252"
    },
    "files.encoding": "utf8",

2. Initialize a git repo
3. Create a test.json file with the following content (win1252 encoding) and commit

{
	"some-test": "Some content with win1252 characters",
	"content": "ãçÇ~'aàá",
	"test": "regular chars"
}

4. Change only line 4 for something (like this)

{
	"some-test": "Some content with win1252 characters",
	"content": "ãçÇ~'aàá",
	"test": "regular chars test"
}

5. Check the git diff. Will report changes on lines 3 and 4 because will convert to utf8 before diff, ignoring the language encoding configuration (win1252)
6. Perform a git diff to see the difference and how it should work properly (only line 4 reported as changed):

$ git diff test.json
diff --git a/test.json b/test.json
index 638f49c..b19527f 100644
--- a/test.json
+++ b/test.json
@@ -1,5 +1,5 @@
 {
        "some-test": "Some content with win1252 characters",
        "content": "<E3><E7><C7>~'a<E0><E1>",
-       "test": "regular chars"
+       "test": "regular chars test"^M
 }

7. Issue won't happen if my user settings are like this:

    "[json]": {
        "files.encoding": "windows1252"
    },
    "files.encoding": "windows1252",

Does this issue occur when all extensions are disabled?: Yes

[vlopes11]

I did some quick research, seems the diff command creates a URI of scheme git and other of type file (for current one)

Editor instance is considering user setting language encoding only for file schemes; therefore git scheme is being rendered with default configuration.

Later I'll try to PR on this.

[mkavidas]

@vlopes11 Are you currently working on this? I am looking to get started with the project and would like to maybe tackle this as my first issue?

[vlopes11]

@mkavidas That would be amazing! I'm quite short on available time so you could fully dive into this.

As a tip, seems like the resourceEditorInput class don't have getEncoding and setEncoding methods implemented.

diffEditorInput will instantiate this class for original EditorInput and fileEditorInput for modified one in this bug example

If these methods exists, seems like standard Editor creation will use them to process the encoding.

[mkavidas]

@vlopes11 thanks for the help. I've been trying to just figure out how the code works/ how to fix it.

I first tried to make resourceEditorInput implement IEncodingSupport then I added the necessary getEncoding and setEncoding methods. It seems that despite the methods being there they are never called.

Did you mean to modify the constructor to add a "preferredEncoding" argument?

It also thought this might be the culprit: https://github.com/Microsoft/vscode/blob/8578f0241e1a64fb5af8ee1b030e42579ef9cd6f/src/vs/workbench/services/editor/common/editorService.ts#L329

And while it is true that resourceInput.encoding is undefined, even if I hard code it to be 'windows1252' it has no effect.

I feel like I might be missing something so any advice you can give me would be greatly appreciated!

I'm mostly curious on how the resourceEditorInput grabs the default files.encoding (while ignoring the language specific). I think if I can find where that happens I could add some logic to grab the correct encoding.

[ArcticLampyrid]

Is there any progress?
I am also troubled by this problem.

[MikolajTwarog]

I would like to take care of this issue, but I'm new to this project, so any help would be appreciated.

[joaomoreno]

Same underlying issue as #36219, now fixed thanks to #55110