Swiftlint analyzer fails when the app's Display Name contains a blank

[gereons]

New Issue Checklist

• Updated SwiftLint to the latest version
• I searched for existing GitHub issues

Describe the bug

I'm running swiftlint analyze with rule explicit_self enabled in my project: https://github.com/gereons/SwiftLintBug2 (use the enclosed lint.sh to reproduce).

Expected result: swiftlint reports the violations

Actual result: swiftlint does not analyze any files

$ ./lint.sh
note: Using new build system

** CLEAN SUCCEEDED **

Loading configuration from '.swiftlint.yml'
Analyzing Swift files at paths
Done analyzing! Found 0 violations, 0 serious in 0 files.

Instead of the expected output of

Analyzing Swift files at paths
Analyzing 'ExplicitSelf.swift' (1/4)
Analyzing 'AppDelegate.swift' (2/4)
Analyzing 'ContentView.swift' (3/4)
Analyzing 'SceneDelegate.swift' (4/4)
... offending files
Done analyzing! Found 3 violations, 0 serious in 4 files.

Opening the .xcodeproj and removing the blank from the app's "Display Name" makes swiftlint work again.

Swiftlint version: 0.38.1, installed via homebrew
Xcode version: 11.3

[jpsim]

Thanks for taking the time to create a project to reproduce the issue.

[DimDL]

Hi There. I can confirm I hit the same issue, but in my case changing the BUNDLE_DISPLAY_NAME wasn't enough, I also had to change the target name (which IMHO makes more sense) to make SwiftLint analyze work. Config: SwiftLint 0.38.2.

[stale]

This issue has been automatically marked as stale because it has not had any recent activity. Please comment to prevent this issue from being closed. Thank you for your contributions!

[gereons]

This issue is still present in v0.41.0 and should not be closed, IMO

[stephengroombbc]

The issue is that CompilerArgumentsExtractor.allCompilerInvocations does a .components(separatedBy: " ") and doesn't account for app\ name.

Doing something like

let invocation = String(line[swiftcIndex...])
                    .split(usingRegex: "(?<!\\\\) ")
                    .map({ $0.replacingOccurrences(of: "\\", with: "") })
                    .expandingResponseFiles
                    .joined(separator: " ")

(splitting by spaces not preceded by a "\" and then replacing "\" with "") solves the problem.

EDIT: split(usingRegex) would have to be implemented of course. I have a version of it found on StackOverflow that I'm really not happy with which works but is bad, hence not submitting this as a PR.

[stephengroombbc]

and then replacing "\" with ""

This introduces another bug. It appears that some the \s in compiler arguments is required as removing them all breaks the sourcekitten index query for some files

[stale]

This issue has been automatically marked as stale because it has not had any recent activity. Please comment to prevent this issue from being closed. Thank you for your contributions!

[gereons]

This issue is still present in v0.42.0 and should not be closed, IMO

[jpsim]

This can be worked around by using the --compile-commands option and passing a JSON file in the compilation database format instead of relying on extracting compilation arguments from the Xcode build log. Support for that was added in this PR, which might give more context: #2962

[gereons]

How would I generate such compilation database file using xcodebuild? I've tried using

xcodebuild -project SwiftLintTest.xcodeproj -scheme SwiftLintTest | xcpretty -r json-compilation-database -o compile_commands.json
swiftlint analyze --compile-commands compile_commands.json

in the example project I've linked, but this still outputs Done analyzing! Found 0 violations, 0 serious in 0 files.. I'm using xcpretty v0.3.0.

compile_commands.json only contains an empty JSON array, [].

[jpsim]

If the the compilation database is empty, maybe that's a bug or limitation in xcpretty? Maybe it fails for the same reason SwiftLint is failing here (the space in the scheme name)?

Maybe this tool works? https://github.com/jerrymarino/XcodeCompilationDatabase

[gereons]

Thanks for the pointer. The tool you've linked does indeed produce a compile_commands.json with content that looks reasonable, but feeding this into SwiftLint yields:

Could not read compilation database at path: 'compile_commands.json' Missing or invalid 
(must be an array of strings) 'arguments' key in compile_commands.json at index 0

This seems to be a bug in the parser for the compilation database, if I understand the documentation correctly - arguments is optional when command is present, which it is in this case.

[jpsim]

Is there any chance you could share step-by-step instructions to reproduce this? It’d be easier for me to help you if have a sample project to start debugging with.

[gereons]

Sure. I've updated the https://github.com/gereons/SwiftLintBug2 repo, simply clone it and run the enclosed lint.sh script

[jpsim]

Thanks for doing that, so here's the compilation database format SwiftLint is expecting:

[
  {
    "arguments": [
      "swiftc",
      "..."
    ],
    "file": "Modules/Module1/Sources/Source1.swift"
  },
  {
    "arguments": [
      "swiftc",
      "..."
    ],
    "file": "Modules/Module1/Sources/Source2.swift"
  }
]

And here's what's produced by XcodeCompilationDatabase in your sample project.

[
  {
    "command": "  /Applications/Xcode-12.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift -frontend -c -primary-file ...",
    "file": "/full/path/to/project/SwiftLintBug2/SwiftLintTest/AppDelegate.swift",
    "directory": ""
  },
  {
    "command": "  /Applications/Xcode-12.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift -frontend -c ...",
    "file": "/full/path/to/project/SwiftLintBug2/SwiftLintTest/ExplicitSelf.swift",
    "directory": ""
  }
]

There are lots of differences:

Expected Format	XcodeCompilationDatabase
Entries have arguments	Entries have command
Entries have swiftc invocations	Entries have swift -frontend invocations
Files paths are relative	Files paths are absolute

According to clang's compilation database format, either the command or the arguments value should be specified, so it seems that both are equally valid, however SwiftLint only checks for arguments.

• command: The compile command executed. After JSON unescaping, this must be a valid command to rerun the exact compilation step for the translation unit in the environment the build system uses. Parameters use shell quoting and shell escaping of quotes, with " and \ being the only special characters. Shell expansion is not supported.
• arguments: The compile command executed as list of strings. Either arguments or command is required.

I'll see if parsing the command values and extracting the arguments the same way we parse them from Xcode logs works for your project.

[jpsim]

It seems that SourceKit isn't satisfied if you give it swift frontend arguments, it needs swiftc arguments.

It logs these errors:

error: unknown argument: '-primary-file'
error: unknown argument: '-emit-module-doc-path'
error: unknown argument: '-emit-dependencies-path'
error: unknown argument: '-emit-reference-dependencies-path'
error: unknown argument: '-enable-objc-interop'
error: unknown argument: '-serialize-debugging-options'
error: unknown argument: '-enable-anonymous-context-mangled-names'
error: unknown argument: '-target-sdk-version'
error: unknown argument: '-index-system-modules'
error: option '-serialize-diagnostics-path' is not supported by 'swiftc'; did you mean to use 'swift'?

And if I filter these out:

let unsupportedArguments = [
    "-primary-file",
    "-emit-module-doc-path",
    "-emit-dependencies-path",
    "-emit-reference-dependencies-path",
    "-enable-objc-interop",
    "-serialize-debugging-options",
    "-enable-anonymous-context-mangled-names",
    "-target-sdk-version",
    "-index-system-modules"
]

arguments.removeAll(where: unsupportedArguments.contains)

let unsupportedOptions = ["-serialize-diagnostics-path"]
for option in unsupportedOptions {
    if let optionIndex = arguments.firstIndex(of: option) {
        arguments.remove(at: optionIndex) // Option parameter
        arguments.remove(at: optionIndex) // Option value
    }
}

the analysis appears to run without errors but produces no results:

Analyzing Swift files in current working directory
Analyzing 'AppDelegate.swift' (1/4)
Analyzing 'ExplicitSelf.swift' (2/4)
Analyzing 'ContentView.swift' (3/4)
Analyzing 'SceneDelegate.swift' (4/4)
Done analyzing! Found 0 violations, 0 serious in 4 files.
Program ended with exit code: 0

[jpsim]

I found a fix for the initial issue you reported, I should have a PR up later today.

[jpsim]

@gereons @DimDL @stephengroombbc can you please confirm that #3549 fixes this issue in your projects?

[stephengroombbc]

Works for me 🎉

[jpsim]

Thanks for checking! I've merged the fix to master.