Config is resetted after a CONF_VERSION bump

[jnsebgosselin]

Problem Description

After doing a minor bump to the CONF_VERSION , the config are resettled to default.

A configparser.InterpolationMissingOptionError is raised when trying to load the config when the CONF_VERSION after a minor version bump, so Spyder revert to producing a default spyder.ini file.

# Main configuration instance
try:
    CONF = UserConfig('spyder', defaults=DEFAULTS, load=True,
                      version=CONF_VERSION, subfolder=SUBFOLDER, backup=True,
                      raw_mode=True)
except Exception:
    CONF = UserConfig('spyder', defaults=DEFAULTS, load=False,
                      version=CONF_VERSION, subfolder=SUBFOLDER, backup=True,
raw_mode=True) 

What steps reproduce the problem?

1. Remove the try statement here in order to get a traceback
2. Delete the .init files in the .spyder-py3 folder
3. Start Spyder
4. Do a minor version bump to CONF_VERSION here
5. Restart Spyder

Paste Traceback/Error Below (if applicable)

Traceback (most recent call last):
  File "C:\Users\User\spyder\bootstrap.py", line 178, in <module>
    from spyder.app import start  # analysis:ignore
  File "C:\Users\User\spyder\spyder\app\start.py", line 34, in <module>
    from spyder.config.main import CONF
  File "C:\Users\User\spyder\spyder\config\main.py", line 757, in <module>
    raw_mode=True)
  File "C:\Users\User\spyder\spyder\config\user.py", line 218, in __init__
    self._update_defaults(defaults, old_ver)
  File "C:\Users\User\spyder\spyder\config\user.py", line 289, in _update_defaults
    old_value = old_defaults.get(section, option)
  File "C:\Python36\lib\configparser.py", line 800, in get
    d)
  File "C:\Python36\lib\configparser.py", line 394, in before_get
    self._interpolate_some(parser, option, L, value, section, defaults, 1)
  File "C:\Python36\lib\configparser.py", line 434, in _interpolate_some
    option, section, rawval, var) from None
configparser.InterpolationMissingOptionError: Bad value substitution: option 'python' in section 'lsp-server' contains an interpolation key 'host' which is not a valid option name. Raw value: "{'index': 0, 'cmd': 'pyls', 'args': '--host %(host)s --port %(port)s --tcp', 'host': '127.0.0.1', 'port': 2087, 'external': False, 'configurations': {'pyls': {'configurationSources': ['pycodestyle', 'pyflakes'], 'plugins': {'pycodestyle': {'enabled': True, 'exclude': [], 'filename': [], 'select': [], 'ignore': [], 'hangClosing': False, 'maxLineLength': 79}, 'pyflakes': {'enabled': True}, 'yapf': {'enabled': False}, 'pydocstyle': {'enabled': False, 'convention': 'pep257', 'addIgnore': [], 'addSelect': [], 'ignore': [], 'select': [], 'match': '(?!test_).*\\\\.py', 'matchDir': '[^\\\\.].*'}, 'rope': {'extensionModules': None, 'ropeFolder': []}, 'rope_completion': {'enabled': False}, 'jedi_completion': {'enabled': True}, 'jedi_hover': {'enabled': True}, 'jedi_references': {'enabled': True}, 'jedi_signature_help': {'enabled': True}, 'jedi_symbols': {'enabled': True, 'all_scopes': True}, 'mccabe': {'enabled': False, 'threshold': 15}, 'preload': {'enabled': True, 'modules': []}}}}}"

Versions

• Spyder version: Spyder 4.0.0.dev0 at 39245fb
• Python version: Python 3.6.5 64-bit
• Qt version: Qt 5.9.3
• PyQt version: PyQt5 5.9.2
• Operating System name/version: Windows 10

Dependencies

pygments >=2.0    :  2.2.0 (OK)
sphinx >=0.6.6    :  1.7.1 (OK)
pyls >=0.19.0     :  0.19.0 (OK)
nbconvert >=4.0   :  5.3.1 (OK)
pandas >=0.13.1   :  0.22.0 (OK)
numpy >=1.7       :  1.13.0 (OK)
sympy >=0.7.3     :  1.1.1 (OK)
cython >=0.21     :  0.27.3 (OK)
qtconsole >=4.2.0 :  4.3.1 (OK)
IPython >=4.0     :  6.2.1 (OK)
matplotlib >=2.0.0:  2.2.0 (OK)
pylint >=0.25     :  1.8.3 (OK)

[jitseniesen]

I remember that configuration options cannot contain the % symbol; I had to work around in PR #3611.

[jnsebgosselin]

It should be possible to use the % symbol for substitution:
https://docs.python.org/3/library/configparser.html#interpolation-of-values

I think the problem occurs because the host key is not stored at the option level, but within the python option itself, so the parser cannot find host as a valid option for substitution?

[jitseniesen]

From what I remember, we are not consistent in whether we use basic interpolation or no interpolation. As far as I could determine, the reason is Python 2 compatibility. But this was quite a while ago, so I may not remember correctly and our code base may have changed; it is also quite possible I did not misunderstood our implementation as I did not try to get to the bottom of it.

[jitseniesen]

Hmm, looking at the traceback, your explanation makes more sense.

[jnsebgosselin]

As far as I could determine, the reason is Python 2 compatibility

I think you are right with this though. I've read something along this line on stackoverflow.

[jnsebgosselin]

@ccordoba12 We really need to fix this before Beta2 is release, because it causes Spyder config to be wiped every-time the conf version increase.

I think this should be a fairly easy one to fix, but since it is LSP related, I'm not sure I'm the best one to do it.

[ccordoba12]

Yeah, I'll address this next week. My plan is to refactor the LSP gui options added by @andfoy.

[jnsebgosselin]

Ok cool, thanks!

[jnsebgosselin]

In the meanwhile, a workaround to avoid losing our config while developing is to go in spyder/config/user and make the except (cp.NoSectionError, cp.NoOptionError) a bare except instead in the method _update_defaults, so that we have instead:

try:
    old_value = old_defaults.get(section, option)
except Exception:
    old_value = None

@CAM-Gerlach this might interest you

[ccordoba12]

@dalthviz, please also work on this one.