[SPARK-43368] Use libnss_wrapper to fake passwd entry

[Yikun]

What changes were proposed in this pull request?

Use libnss_wrapper to fake passwd entry instead of changing passwd to resolve random UID problem. And also we only attempt to setup fake passwd entry for driver/executor, but for cmd like bash, the fake passwd will not be set.

Why are the changes needed?

In the past, we add the entry to /etc/passwd directly for current UID, it's mainly for OpenShift anonymous random uid case (See also in apache-spark-on-k8s/spark#404), but this way bring the pontential security issue about widely permision of /etc/passwd.

According to DOI reviewer suggestion, we'd better to resolve this problem by using libnss_wrapper. It's a library to help set a fake passwd entry by setting LD_PRELOAD, NSS_WRAPPER_PASSWD, NSS_WRAPPER_GROUP. Such as random UID is 1000, the env will be:

spark@6f41b8e5be9b:/opt/spark/work-dir$ id -u
1000
spark@6f41b8e5be9b:/opt/spark/work-dir$ id -g
1000
spark@6f41b8e5be9b:/opt/spark/work-dir$ whoami
spark
spark@6f41b8e5be9b:/opt/spark/work-dir$ echo $LD_PRELOAD
/usr/lib/libnss_wrapper.so
spark@6f41b8e5be9b:/opt/spark/work-dir$ echo $NSS_WRAPPER_PASSWD
/tmp/tmp.r5x4SMX35B
spark@6f41b8e5be9b:/opt/spark/work-dir$ cat /tmp/tmp.r5x4SMX35B
spark:x:1000:1000:${SPARK_USER_NAME:-anonymous uid}:/opt/spark:/bin/false
spark@6f41b8e5be9b:/opt/spark/work-dir$ echo $NSS_WRAPPER_GROUP
/tmp/tmp.XcnnYuD68r
spark@6f41b8e5be9b:/opt/spark/work-dir$ cat /tmp/tmp.XcnnYuD68r
spark:x:1000:

Does this PR introduce any user-facing change?

Yes, setup fake ENV rather than changing /etc/passwd.

How was this patch tested?

1. Without attempt_setup_fake_passwd_entry, the user is I have no name!

# docker run -it --rm --user 1000:1000  spark-test bash
groups: cannot find name for group ID 1000
I have no name!@998110cd5a26:/opt/spark/work-dir$
I have no name!@0fea1d27d67d:/opt/spark/work-dir$ id -u
1000
I have no name!@0fea1d27d67d:/opt/spark/work-dir$ id -g
1000
I have no name!@0fea1d27d67d:/opt/spark/work-dir$ whoami
whoami: cannot find name for user ID 1000

2. Mannual stub the attempt_setup_fake_passwd_entry, the user is spark.

2.1 Apply a tmp change to cmd

diff --git a/entrypoint.sh.template b/entrypoint.sh.template
index 08fc925..77d5b04 100644
--- a/entrypoint.sh.template
+++ b/entrypoint.sh.template
@@ -118,6 +118,7 @@ case "$1" in

   *)
     # Non-spark-on-k8s command provided, proceeding in pass-through mode...
+    attempt_setup_fake_passwd_entry
     exec "$@"
     ;;
 esac

2.2 Build and run the image, specify a random UID/GID 1000

$ docker build . -t spark-test
$ docker run -it --rm --user 1000:1000  spark-test bash
# the user is set to spark rather than unknow user
spark@6f41b8e5be9b:/opt/spark/work-dir$
spark@6f41b8e5be9b:/opt/spark/work-dir$ id -u
1000
spark@6f41b8e5be9b:/opt/spark/work-dir$ id -g
1000
spark@6f41b8e5be9b:/opt/spark/work-dir$ whoami
spark

# NSS env is set right
spark@6f41b8e5be9b:/opt/spark/work-dir$ echo $LD_PRELOAD
/usr/lib/libnss_wrapper.so
spark@6f41b8e5be9b:/opt/spark/work-dir$ echo $NSS_WRAPPER_PASSWD
/tmp/tmp.r5x4SMX35B
spark@6f41b8e5be9b:/opt/spark/work-dir$ cat /tmp/tmp.r5x4SMX35B
spark:x:1000:1000:${SPARK_USER_NAME:-anonymous uid}:/opt/spark:/bin/false
spark@6f41b8e5be9b:/opt/spark/work-dir$ echo $NSS_WRAPPER_GROUP
/tmp/tmp.XcnnYuD68r
spark@6f41b8e5be9b:/opt/spark/work-dir$ cat /tmp/tmp.XcnnYuD68r
spark:x:1000:

3. If specify current exsiting user (such as spark, root), no fake setup

# docker run -it --rm --user 0  spark-test bash
root@e5bf55d4df22:/opt/spark/work-dir# echo $LD_PRELOAD

# docker run -it --rm  spark-test bash
spark@def8d8ca4e7d:/opt/spark/work-dir$ echo $LD_PRELOAD

[Yikun]

cc @yosifkit Thanks for the solution!
cc @pan3793 @dongjoon-hyun @erikerlandson (who involved spark on k8s user related work)
also cc @HyukjinKwon @zhengruifeng

[Yikun]

@HyukjinKwon @pan3793 Thanks, I will merge this soon.

[Yikun]

Merged.

[Reamer]

Hi @Yikun,
This change has also been adopted by the Spark operator. See kubeflow/spark-operator#2312
I am an Openshift user myself and therefore ran into a regression. See: kubeflow/spark-operator#2644

Have you also tested your commands for the home directory?

I tested the home directory with the following small Java class. I don't know how Spark evaluates this.

import javax.swing.filechooser.FileSystemView;
import java.io.File;
class Home {
    public static void main(String... args) {
        FileSystemView view = FileSystemView.getFileSystemView();
        File file = view.getHomeDirectory();
        String desktopPath = file.getPath();
        System.out.println(desktopPath);
    }
}

The output of the small program returns the following, which is obviously incorrect.

java Home
-anonymous uid}

Is there a reason to evaluate the GECOS field later?

[Reamer]

Pull Request with a possible fix. #90