Spyder not conform PEP3120

[slawomirmarczynski]

Description

What steps will reproduce the problem?

Spyder 5.3.0 on MS Windows 10. Open a UTF-8 encoded source file (without -*- coding: utf-8 -*- encoding declaration).

Spyder DOES NOT RESPECT PEP3120: the file will be opened with Windows-1250 encoding. (It may/may not depends on BOM.) This is a bug.

The proper way (for Python 3, see PEP3120) is to open each file (by default) as UTF-8 encoding.

Obviously one can enforce UTF-8 encoding (see PEP263). It is a workaround.

BTW, PyCharm and MS Visual Code, the same PC/OS, behave as expected: open .py files with UTF-8 as default encodings.

Versions

• Spyder version: 5.3.0
• Python version: 3.9.12
• Qt version: 5.12.9
• PyQt5 version: 5.12.3
• Operating System: Windows 10

Dependencies

# Mandatory:
atomicwrites >=1.2.0          :  1.4.0 (OK)
chardet >=2.0.0               :  4.0.0 (OK)
cloudpickle >=0.5.0           :  2.0.0 (OK)
cookiecutter >=1.6.0          :  1.7.3 (OK)
diff_match_patch >=20181111   :  20200713 (OK)
intervaltree >=3.0.2          :  3.0.2 (OK)
IPython >=7.31.1;<8.0.0       :  7.33.0 (OK)
jedi >=0.17.2;<0.19.0         :  0.18.1 (OK)
jellyfish >=0.7               :  0.9.0 (OK)
jsonschema >=3.2.0            :  4.4.0 (OK)
keyring >=17.0.0              :  23.4.0 (OK)
nbconvert >=4.0               :  6.5.0 (OK)
numpydoc >=0.6.0              :  1.3.1 (OK)
paramiko >=2.4.0              :  2.10.4 (OK)
parso >=0.7.0;<0.9.0          :  0.8.3 (OK)
pexpect >=4.4.0               :  4.8.0 (OK)
pickleshare >=0.4             :  0.7.5 (OK)
psutil >=5.3                  :  5.9.0 (OK)
pygments >=2.0                :  2.12.0 (OK)
pylint >=2.5.0                :  2.13.7 (OK)
pyls_spyder >=0.4.0           :  0.4.0 (OK)
pylsp >=1.4.1;<1.5.0          :  1.4.1 (OK)
pylsp_black >=1.2.0           :  1.2.1 (OK)
qdarkstyle >=3.0.2;<3.1.0     :  3.0.3 (OK)
qstylizer >=0.1.10            :  0.2.1 (OK)
qtawesome >=1.0.2             :  1.1.1 (OK)
qtconsole >=5.3.0;<5.4.0      :  5.3.0 (OK)
qtpy >=2.0.1                  :  2.0.1 (OK)
rtree >=0.9.7                 :  1.0.0 (OK)
setuptools >=49.6.0           :  62.1.0 (OK)
sphinx >=0.6.6                :  4.5.0 (OK)
spyder_kernels >=2.3.0;<2.4.0 :  2.3.0 (OK)
textdistance >=4.2.0          :  4.2.2 (OK)
three_merge >=0.1.1           :  0.1.1 (OK)
watchdog >=0.10.3             :  2.1.7 (OK)
zmq >=17                      :  22.3.0 (OK)

# Optional:
cython >=0.21                 :  None (NOK)
matplotlib >=3.0.0            :  3.5.1 (OK)
numpy >=1.7                   :  1.22.3 (OK)
pandas >=1.1.1                :  None (NOK)
scipy >=0.17.0                :  1.8.0 (OK)
sympy >=0.7.3                 :  None (NOK)

[dalthviz]

Hi @slawomirmarczynski thank you for the feedback! I think I was able to reproduce this. We will try to fix it in a future release 👍

[azelcer]

@ccordoba12 @dalthviz I have a similar issue opening UTF-8 encoded .md files (no BOM), that are detected as MACROMAN. Can you give me a hint of where the autodetecting code is, so I try to fix this?