Code analysis misses includes generated at build time

[anatolyburakov]

Issue Type: Bug

We have a project that relies on build-time configuration for some things. This build-time configuration is in the form of a C header file (rte_build_config.h) that gets dropped into the build directory.

When running code analysis on any source files that end up including this file, i get the following error:

#include errors detected. Consider updating your compile_commands.json or includePath. Squiggles are disabled for this translation unit (/usr/local/home/anatoly/DPDK/examples/l3fwd-power/main.c).
cannot open source file "rte_build_config.h" (dependency of "/usr/local/home/anatoly/DPDK/config/rte_config.h")

Looking at my compile_commands.json, i can see the following:

...
  {
    "directory": "/usr/local/home/anatoly/build/DPDK",
    "command": "cc -Iexamples/dpdk-l3fwd-power.p -Iexamples -I../../DPDK/examples -Iexamples/l3fwd-power -I../../DPDK/examples/l3fwd-power -I. -I../../DPDK -Iconfig -I../../DPDK/config -Ilib/librte_eal/include -I../../DPDK/lib/librte_eal/include -Ilib/librte_eal/linux/include -I../../DPDK/lib/librte_eal/linux/include -Ilib/librte_eal/x86/include -I../../DPDK/lib/librte_eal/x86/include -Ilib/librte_eal/common -I../../DPDK/lib/librte_eal/common -Ilib/librte_eal -I../../DPDK/lib/librte_eal -Ilib/librte_kvargs -I../../DPDK/lib/librte_kvargs -Ilib/librte_metrics -I../../DPDK/lib/librte_metrics -Ilib/librte_telemetry -I../../DPDK/lib/librte_telemetry -Ilib/librte_mempool -I../../DPDK/lib/librte_mempool -Ilib/librte_ring -I../../DPDK/lib/librte_ring -Ilib/librte_net -I../../DPDK/lib/librte_net -Ilib/librte_mbuf -I../../DPDK/lib/librte_mbuf -Ilib/librte_ethdev -I../../DPDK/lib/librte_ethdev -Ilib/librte_meter -I../../DPDK/lib/librte_meter -Ilib/librte_cmdline -I../../DPDK/lib/librte_cmdline -Ilib/librte_power -I../../DPDK/lib/librte_power -Ilib/librte_timer -I../../DPDK/lib/librte_timer -Ilib/librte_lpm -I../../DPDK/lib/librte_lpm -Ilib/librte_hash -I../../DPDK/lib/librte_hash -Ilib/librte_rcu -I../../DPDK/lib/librte_rcu -fdiagnostics-color=always -pipe -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -g -include rte_config.h -Wextra -Wcast-qual -Wdeprecated -Wformat -Wformat-nonliteral -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wpointer-arith -Wsign-compare -Wstrict-prototypes -Wundef -Wwrite-strings -Wno-address-of-packed-member -Wno-packed-not-aligned -Wno-missing-field-initializers -D_GNU_SOURCE -march=native -Wno-format-truncation -DALLOW_EXPERIMENTAL_API -MD -MQ examples/dpdk-l3fwd-power.p/l3fwd-power_main.c.o -MF examples/dpdk-l3fwd-power.p/l3fwd-power_main.c.o.d -o examples/dpdk-l3fwd-power.p/l3fwd-power_main.c.o -c ../../DPDK/examples/l3fwd-power/main.c",
    "file": "../../DPDK/examples/l3fwd-power/main.c",
    "output": "examples/dpdk-l3fwd-power.p/l3fwd-power_main.c.o"
  },
...

The rte_build_config.h is not included directly, but is included by rte_common.h which in turn is included somewhere along the line from main.c. The file is located at the build directory, and compile_commands.json appears to include build directory:

...
  {
    "directory": "/usr/local/home/anatoly/build/DPDK",
    "command": "... -I. ...",
    "file": "../../DPDK/examples/l3fwd-power/main.c",
    "output": "examples/dpdk-l3fwd-power.p/l3fwd-power_main.c.o"
  },
...

So, the rte_build_config.h should be found, but isn't for some reason.

It also appears that there is no way to tell the C++ extension to either force-include this file or this directory (as when compile_commands.json is used, other configuration appears to be overridden), or tell it to look for it in the build directory in addition to source directory (despite the fact that the compile command itself contains the directory key which contains the path where the file is located, and the compile_commands.json includes that path).

Extension version: 1.3.0-insiders
VS Code version: Code - Insiders 1.55.0-insider (8437d38d364ba93b53c1e659c20b1c86f710ec28, 2021-03-24T08:16:02.872Z)
OS version: Windows_NT x64 10.0.18363
Remote OS version: Linux x64 5.8.7

A/B Experiments

vsliv695:30137379
vsins829:30139715
vsliv368cf:30146710
vsreu685:30147344
python383:30185418
pythonvspyt602:30263608
vspor879:30202332
vspor708:30202333
vspor363:30204092
vstry914:30244316
pythonvsdeb440:30224570
pythonvsded773:30223139
pythonvspyt875:30259475
pythonvsnew554:30265444
pythontb:30258533
pythonvspyt943:30277000
wslfolderdoc:30278405
vspre833cf:30267465
vshan820cf:30276953

[sean-mcmanus]

@Colengms Do you have enough info to repro/investigate this? Or do we need more logging?

[Colengms]

@anatolyburakov A couple questions to help us investigate this:

1. When you open the source file that is repro'ing that error, has the file yet been generated? After the file has been generated, if you close the file, wait several seconds for the associated cpptools-srv process to shut down, and re-open the file, does the error go away?

2. Does the folder containing rte_build_config.h exist when VS Code is launched? If not, does reloading VS Code after that folder exists avoid the error?

I see the command line includes a forced-include of the file the error is associated with (-include rte_config.h). Prior to 1.3.0-insiders, we had a bug that resulted in forced includes silently failing to be handled, if the file was not directly in the workspace folder or found directly in one of the user-specified include paths. For compile_commands.json, the command line is resolved at startup, and include paths that cannot be found on disk at the time are omitted. If that's what's happening, # 2 above should work around the issue. Can you confirm that is what's happening?

[Colengms]

Hi @anatolyburakov . In addition to my previous two questions, would it be possible to provide a minimal example we could use to investigate further? If the issue persists after the file has been created and VS Code reloaded, we will need to investigate further to identify the cause. Or, if the issue does not persist after the file has been created and VS Code reloaded, I believe I have a fix for our next insiders build.

[anatolyburakov]

1. When you open the source file that is repro'ing that error, has the file yet been generated?  After the file has been generated, if you close the file, wait several seconds for the associated cpptools-srv process to shut down, and re-open the file, does the error go away?

No, the error never goes away. Well, it's not there when the file is first loaded, but by the time code analysis is complete, i get this error always, even if i close/reopen the file.

2. Does the folder containing `rte_build_config.h` exist when VS Code is launched?  If not, does reloading VS Code after that folder exists avoid the error?

Yes, it does. It's the same folder compile_commands.json exists in.

I see the command line includes a forced-include of the file the error is associated with (-include rte_config.h). Prior to 1.3.0-insiders, we had a bug that resulted in forced includes silently failing to be handled, if the file was not directly in the workspace folder or found directly in one of the user-specified include paths. For compile_commands.json, the command line is resolved at startup, and include paths that cannot be found on disk at the time are omitted. If that's what's happening, # 2 above should work around the issue. Can you confirm that is what's happening?

I believe i reported that specific bug too :D no, reloading etc. doesn't help with this error.

Hi @anatolyburakov . In addition to my previous two questions, would it be possible to provide a minimal example we could use to investigate further? If the issue persists after the file has been created and VS Code reloaded, we will need to investigate further to identify the cause. Or, if the issue does not persist after the file has been created and VS Code reloaded, I believe I have a fix for our next insiders build.

I will try to create a minimally reproducible example.

[anatolyburakov]

Here's a minimal example that i can reproduce on my machine:

https://github.com/anatolyburakov/vscode_7221_repro

After building and reloading the file, i see the following:

[sean-mcmanus]

All the compile commands paths are being incorrectly taken from the workspace folder instead of the "directory" of the compile commands. Appears to be a 1.3.0-insiders regression.

[sean-mcmanus]

Fixed with https://github.com/microsoft/vscode-cpptools/releases/tag/1.3.0-insiders2 (at least with the sample repro case you provided) -- let us know if you still notice any other related issues.

[anatolyburakov]

@sean-mcmanus seems to be fixed for the minimal example, but i still see the issue on real codebase.

[anatolyburakov]

@sean-mcmanus it turns out that my errors are due to a different bug. I'll submit a new one, but this issue appears to be fixed. Thanks!