Crash (panic: runtime error) if Poetry is installed on system

[hwalinga]

If run on a system with Poetry installed, the application crashes.

» mllint -o mllint-report.md                                                                                                                                   
Linting project at  /home/hielke/repos/REMLA-project-group-7                                                                                                   
Using configuration from .mllint.yml (default: false)                                                                                                          
---                                                                                                                                                            
                                                                                                                                                               
panic: runtime error: invalid memory address or nil pointer dereference                                                                                        
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x62e632]                                                                                         
                                                                                                                                                               
goroutine 1 [running]:                                                                                                                                         
github.com/pelletier/go-toml.(*Tree).GetPath(0x0, 0xc0011af870, 0x1, 0x1, 0x0, 0x0)                                                                            
        /home/runner/go/pkg/mod/github.com/pelletier/[email protected]/toml.go:118 +0xf2                                                                          
github.com/pelletier/go-toml.(*Tree).HasPath(...)                                                                                                              
        /home/runner/go/pkg/mod/github.com/pelletier/[email protected]/toml.go:66                                                                                 
github.com/pelletier/go-toml.(*Tree).Has(0x0, 0xc5935e, 0x4, 0x0)                                                                                              
        /home/runner/go/pkg/mod/github.com/pelletier/[email protected]/toml.go:61 +0xa5                                                                           
github.com/bvobart/mllint/setools/depmanagers.Poetry.HasDevDependency(...)                                                                                     
        /home/runner/work/mllint/mllint/setools/depmanagers/poetry.go:47                                                                                       
github.com/bvobart/mllint/setools/depmanagers.Poetry.HasDependency(0xc0011af7b0, 0xc5935e, 0x4, 0xc000d334c0)                                                  
        /home/runner/work/mllint/mllint/setools/depmanagers/poetry.go:43 +0x85                                                                                 
github.com/bvobart/mllint/setools/cqlinters.DetectLinter(0x13330b8, 0x1bbb1d8, 0xc00002c034, 0x28, 0xc000d334c0, 0x36, 0xc00110c270, 0x28, 0xc001110080, 0x6, ...)
        /home/runner/work/mllint/mllint/setools/cqlinters/detect.go:32 +0xfd
github.com/bvobart/mllint/setools/cqlinters.Detect(0xc00002c034, 0x28, 0xc000d334c0, 0x36, 0xc00110c270, 0x28, 0xc001110080, 0x6, 0x0, 0xc0010c73e0, ...)
        /home/runner/work/mllint/mllint/setools/cqlinters/detect.go:15 +0x13f
github.com/bvobart/mllint/commands.(*runCommand).runPreAnalysisChecks(0xc00103fbd0, 0x1, 0x1)
        /home/runner/work/mllint/mllint/commands/run.go:57 +0x198
github.com/bvobart/mllint/commands.(*runCommand).RunLint(0xc00103fbd0, 0xc000feea00, 0xc0010b6fe0, 0x0, 0x2, 0x5, 0xc0010b3f80)
        /home/runner/work/mllint/mllint/commands/run.go:98 +0x2c5
github.com/bvobart/mllint/commands.runRoot(0xc000feea00, 0xc0010b6fe0, 0x0, 0x2, 0x0, 0x0)
        /home/runner/work/mllint/mllint/commands/root.go:51 +0x12d
github.com/spf13/cobra.(*Command).execute(0xc000feea00, 0xc00001e0a0, 0x2, 0x2, 0xc000feea00, 0xc00001e0a0)
        /home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:852 +0x472
github.com/spf13/cobra.(*Command).ExecuteC(0xc000feea00, 0x2f7bf60, 0x1b8bd20, 0x7fc5cab55a98)
        /home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:960 +0x375
github.com/spf13/cobra.(*Command).Execute(...)
        /home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:897
github.com/bvobart/mllint/commands.Execute(0xb8c5c0, 0xc0000380b8)
        /home/runner/work/mllint/mllint/commands/root.go:13 +0x5b
main.main()
        /home/runner/work/mllint/mllint/main.go:10 +0x25

OS: Debian 11
Python: 3.9.2
mllint version: 0.12.1
mllint commit: 7e2a29d

[hwalinga]

We did disable the dependency manager, but it still failed.

rules:
  disabled:
    - dependency-management

[bvobart]

Hi Hielke, thanks for the stack trace, I've been able to find the bug causing your issue!

Turns out it was a simple mistake causing a very rare bug that would only trigger if you have Poetry installed, as well as a pyproject.toml in your project that specifies that you're using Poetry (build-backend = "poetry.core.masonry.api"), but don't specify any dependencies ([tool.poetry.dependencies]). The bug occurred during pre-analysis checks while searching for evidence of code quality linters in your project. There is no way to disable that check.

I've fixed the bug and will release the fixed version as v0.12.2 very soon.

[hwalinga]

Thanks Bart, but your description doesn't fit our pyproject.toml file. Here it is

[tool.poetry]
name = "src"
version = "0.1.28"
description = "This project is created to experience setting up an ML environment for a team with different expertieces. This repo is created for the Release Engeneering for Machine Learning Applications taught at the TU Delft."
authors = ['Chiel de Vries <[email protected]>', 'Hielke Walinga <[email protected]>', 'Lars van Koetsveld van Ankeren <[email protected]>', 'Mark Bekooy <[email protected]>', 'Melchior Oudemans <[email protected]>']
license = "MIT"
readme = "README.md"
packages = [{include = "src"}]

[tool.poetry.dependencies]
python = "3.9.*"
dvc = {extras = ["gdrive"], version = "^2.10.2"}
numpy = "^1.21.6"
pandas = "^1.3.5"
scikit-learn = "^1.0.2"
nltk = "^3.7"
scipy = "^1.8.1"
markupsafe = "^2.1.1"
fastapi = "^0.78.0"
uvicorn = {extras = ["standard"], version = "^0.17.6"}
python-multipart = "^0.0.5"

[tool.poetry.group.dev.dependencies]
pytest = "^7.1.2"
Sphinx = "^5.0.0"
coverage = "^6.4"
awscli = "^1.25.0"
flake8 = "^3.9.2"
pytest-cov = "^3.0.0"
tensorflow-data-validation = {extras = ["visualization"], version = "^1.8.0"}
mllint = "^0.12.1"
pylint = "^2.14.1"
mypy = "^0.961"
black = "^22.3.0"
isort = "^5.10.1"
bandit = "^1.7.4"

[build-system]
requires = ["poetry-core", "setuptools", "poetry>=0.12"]
build-backend = "poetry.core.masonry.api"

[tool.coverage.run]
parallel = true

[tool.isort]
profile = "black"

[tool.pylint.'MESSAGES CONTROL']
max-line-length = 88  # 88 is black default
disable = """
invalid-name,C0326
"""
# invalid-name: For ML applications it doesn't matter too much to use one-letter variables
# C0326: White space is sometimes helpfull for aligning

[hwalinga]

Might it be due to the new way dev dependency are described in Poetry version 1.2? It now uses a group named dev to do that. tool.poetry.group.dev.dependencies

[hwalinga]

fixed version as v0.12.2 very soon.

I hope to see the fix very soon as well, the deadline for our course is tomorrow!

[bvobart]

Ah, I see, I didn't know about that new convention. I've updated mllint so it is also able to recognise dev dependencies in [tool.poetry.group.dev.dependencies]

v0.12.2 will be released in about 15 to 20 minutes. See https://github.com/bvobart/mllint/actions/runs/2531849384 to follow the release pipeline.