Runfiles libraries don't find runfiles in directories that are themselves runfiles when using manifest

[fmeum]

Description of the problem / feature request:

In the way they are currently implemented, the Bazel runfiles libraries at @bazel_tools//tools/*/runfiles do not always find the identical set of runfiles when using a manifest compared to when using a runfiles directory.

Specificially, if a target has runfiles foo (a directory) and foo/a.txt (a file in that directory), then Runfiles#filterListForObscuringSymlinks will filter out foo/a.txt. As a result:

1. when using a directory-based runfiles lookup, foo/a.txt will be found since the symlink foo resolves to the correct directory containing a.txt;
2. when using a manifest-based runfiles lookup, foo/a.txt is not found since it is not listed in the manifest, only foo itself is.

Note that it is pretty easy to pick up both a directory and a file contained in that directory as runfiles through transitive dependencies and everything will work well on Linux/macOS, only to non-obviously break on Windows. I encountered this issue while working on a ruleset that would allow for compile-time checked rlocation lookups for C++ and Java, but found the current issue to block this effort.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

#!/usr/bin/env bash
touch WORKSPACE

mkdir -p foo
touch foo/a.txt

cat <<EOF >main.cc
int main() {}
EOF

cat <<EOF >BUILD.bazel
cc_binary(
    name = "main",
    data = [
        "foo",
        "foo/a.txt",
    ],
    srcs = ["main.cc"],
)
EOF

bazel build //:main
cat bazel-bin/main.runfiles_manifest

correctly prints

__main__/foo /tmp/repro/foo
__main__/main /home/user/.cache/bazel/_bazel_user/.../execroot/__main__/bazel-out/k8-fastbuild/bin/main

When using @bazel_tools//tools/cpp/runfiles with the runfiles manifest, e.g. on Windows, then Rlocation("foo/a.txt") will return an empty string instead of the correct path for the declared runfile.

What operating system are you running Bazel on?

Any, but this mostly applies to Windows.

What's the output of bazel info release?

Any version is affected, including current master.

 Have you found anything relevant by searching the web?

No.

Any other information, logs, or outputs that you want to share?

I prepared a PR (#14335) that extends the C++, Java, and Python runfiles libraries so that they also find runfiles within runfiles directories when using the runfiles manifest. I could also fix this for the bash implementation, but would like to wait for general feedback before I start working on this.

[fmeum]

@bazel-io fork