[branch-1.5][SPARK-11821] Propagate Kerberos keytab for all environments

[woj-i]

I prepared a patch for recent bugfix. The scope of the previous bugfix is too narrow- it works only on YARN. I need it on local mode and I think the other modes also need the information (because reflection works the same for each environment having JVM).

[AmplabJenkins]

Can one of the admins verify this patch?

[woj-i]

I also see, that a change of authentication method from a simple to a kerberos was required while renewing credentials. I've made a commit.

[andrewor14]

@tgravescs @harishreedharan

[andrewor14]

@woj-i please open the patch against the master branch instead. Committers will backport it into branch-1.5 when they merge it.

[harishreedharan]

This seems wrong. If you are talking to hadoop cluster that is secure, your client configuration should already have this. Setting this here alone is not going to help, since all other instances used all over Spark will not have this information (where required).

[woj-i]

Hi!
I think it is required because of another context of execution in this part of code. Here is a context of the classloader, which cause an empty environment, with default value of authentication, which is "simple". In case of "simple" a keytab is not used and credentials cannot be granted.

[harishreedharan]

Ah, ok. Did you hit this issue when running an app?

[woj-i]

Yes, I did.

[andrewor14]

@woj-i would you mind tagging this one as [branch-1.5] in the title of the PR so it's clearer?
@harishreedharan please review #9859 instead, which is the same patch opened against master

[woj-i]

Title changed.

[tgravescs]

I made comments on #9859.

[vanzin]

@woj-i could you close this PR?