filter exit code from command output #242

balihb · 2017-10-30T22:11:01Z

Ok, I won't say this is the nicest solution for this, but I have no better idea to prevent things like the docker client jenkins plugin from failing with kubernetes.

marvinthepa · 2017-11-02T09:19:33Z

@balihb

Thanks for the pull request.

I think you are right, it is probably a good idea in general to not pass the "EXITCODE" stuff to the client.
And you are right, there "should" be a nicer solution.

Could you please add more information on "the docker client jenkins plugin [...] failing with kubernetes", i.e. an example pipeline that shows it and the error in the build log?
If at all possible, could you also add a test that fails on master but doesn't fail on this branch?

balihb · 2017-11-02T10:24:47Z

I'm planning a change on this PR. I hope if it only runs the matching part of the code at the point where the plugin reads out the exitcode it won't need to match on all write.

the docker client part:

19:18:26 java.lang.IllegalArgumentException: Expecting 64-char full image ID, but got 3f8539f89064c27dfb723eb2a7972b78b293243a635dea00cf4089a20a7ea92a
19:18:26 EXITCODE   0
19:18:26 	at org.jenkinsci.plugins.docker.commons.fingerprint.DockerFingerprints.getFingerprintHash(DockerFingerprints.java:71)
19:18:26 	at org.jenkinsci.plugins.docker.commons.fingerprint.DockerFingerprints.forDockerInstance(DockerFingerprints.java:148)
19:18:26 	at org.jenkinsci.plugins.docker.commons.fingerprint.DockerFingerprints.forImage(DockerFingerprints.java:115)
19:18:26 	at org.jenkinsci.plugins.docker.commons.fingerprint.DockerFingerprints.forImage(DockerFingerprints.java:100)
19:18:26 	at org.jenkinsci.plugins.docker.commons.fingerprint.DockerFingerprints.addRunFacet(DockerFingerprints.java:226)
19:18:26 	at org.jenkinsci.plugins.docker.workflow.RunFingerprintStep$Execution.run(RunFingerprintStep.java:77)
19:18:26 	at org.jenkinsci.plugins.docker.workflow.RunFingerprintStep$Execution.run(RunFingerprintStep.java:63)
19:18:26 	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
19:18:26 	at hudson.security.ACL.impersonate(ACL.java:260)
19:18:26 	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
19:18:26 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
19:18:26 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
19:18:26 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
19:18:26 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
19:18:26 	at java.lang.Thread.run(Thread.java:748)

The problem here is that the docker plugin expects the ProcStarter to only return the output of the command.

I haven't looked at test code, but I'll add the test (I hope I can :D).

One question:
https://github.com/jenkinsci/kubernetes-plugin/blob/master/src/main/java/org/csanchez/jenkins/plugins/kubernetes/pipeline/ContainerExecDecorator.java#L327
Can these debug messages be passed to the logger and not the output? I can make the change (tried it already, but haven't added to the PR).

marvinthepa · 2017-11-02T15:51:53Z

I'm planning a change on this PR.

Cool. I added "[WIP]" to the title of this PR, please remove once it is ready to review again.

balihb · 2017-11-06T17:29:49Z

I've been testing out the ssh-agent bug and it looks like I found a solution:
in doExec:

                 watch.getInput().write(s.getBytes(StandardCharsets.UTF_8));
             }
+            sb.append(" 2> /dev/null");
             sb.append(NEWLINE);

marvinthepa · 2017-11-07T16:08:22Z

@balihb:

while this might get rid of the error, I guess it has to be considered a change of functionality to just throw away stderr...

balihb · 2017-11-07T16:22:22Z

there is no stderr anyway. at least not from the command.
An example command:

INFO: Executing command: "nohup" "sh" "-c" "echo \$\$ > '/home/jenkins/workspace/boot-react@tmp/durable-4f232592/pid'; jsc=durable-9c1ff7314119e604e5e6da6a39f23b37; JENKINS_SERVER_COOKIE=\$jsc '/home/jenkins/workspace/boot-react@tmp/durable-4f232592/script.sh' > '/home/jenkins/workspace/boot-react@tmp/durable-4f232592/jenkins-log.txt' 2>&1; echo \$? > '/home/jenkins/workspace/boot-react@tmp/durable-4f232592/jenkins-result.txt'"  2> /dev/null
printf "EXITCODE %3d" $?; exit

The output of the command is piped to stdout anyway, there is something else that sneaks something funny on the output and stderr is not collected separately:

                Execable<String, ExecWatch> execable = client.pods().inNamespace(namespace).withName(podName).inContainer(containerName)
                        .redirectingInput().writingOutput(stream).writingError(stream)

IMHO it shouldn't be collected anyway. It can cause problems if it is pushed on stdout (which is happening now) and if another plugin expects it the current solution can't give it anything.

balihb · 2017-11-13T15:18:54Z

this should work now IMHO.
@marvinthepa I've changed a few of the writes to the output to be just log. I hope it is ok.

balihb · 2017-11-14T16:25:25Z

this latest change should also fix the ssh-agent problem.
if I can do anything else to get this PR accepted, please let me know.

balihb · 2017-11-21T09:30:20Z

@marvinthepa or maybe @carlossg ?

MattLud · 2017-12-29T21:39:18Z

Tested locally a few dozen times with a particular smoke test 👍

balihb · 2018-01-07T19:34:40Z

@carlossg
I am contacting you because the merging of my PR would be crucial for my company (and for others, too), which is why I have been wondering if there is any error or problem you have found within the code? I certainly hope you did not find any of my earlier comments rude, since it was not my intention. Should you have any suggestions or requirements concerning the PR, please let me know.

carlossg · 2018-01-31T09:30:14Z

sorry for the delay, I am trying to figure out if there is a better way to fetch the exit code because this is adding a lot of complexity to the code

carlossg

Can you add a summary on how the stream flow works across all classes?

Thanks!

carlossg · 2018-01-31T09:31:39Z

src/main/java/org/csanchez/jenkins/plugins/kubernetes/pipeline/ContainerExecDecorator.java

-
-                //check if the cmd is sourced from Jenkins, rather than another plugin; if so, skip cmdEnvs as we are getting other environment variables
-                for (String cmd : cmdEnvs) {
-                    if (cmd.startsWith(JENKINS_HOME)) {


this is no longer needed ?
it's being reimplemented in
https://github.com/jenkinsci/kubernetes-plugin/pull/245/files#diff-8a1ab34452b01252e89ffeee0b2eda45R201

carlossg · 2018-01-31T09:35:22Z

src/main/java/org/csanchez/jenkins/plugins/kubernetes/pipeline/ContainerExecDecorator.java

@@ -431,4 +437,37 @@ public int getExitCode() {
            return i;
        }
    }
+
+	static class FilterOutExitCodeOutputStream extends OutputStream {


can we move it to a new class org.csanchez.jenkins.plugins.kubernetes.pipeline.exec. FilterOutExitCodeOutputStream with

@Restricted(NoExternalUse.class)

carlossg · 2018-01-31T09:35:41Z

...main/java/org/csanchez/jenkins/plugins/kubernetes/pipeline/ContainerExecCutExitCodeUtil.java

@@ -0,0 +1,69 @@
+package org.csanchez.jenkins.plugins.kubernetes.pipeline;


can we put it under org.csanchez.jenkins.plugins.kubernetes.pipeline.exec

carlossg · 2018-01-31T09:36:08Z

...main/java/org/csanchez/jenkins/plugins/kubernetes/pipeline/ContainerExecCutExitCodeUtil.java

+
+import org.apache.commons.lang.ArrayUtils;
+
+public class ContainerExecCutExitCodeUtil {


@Restricted(NoExternalUse.class)

merge with jenkinsci/kubernetes-plugin master and review modifications

balihb · 2018-03-09T12:44:33Z

I've made a graph about the output stream flow
https://pasteboard.co/Hb6dUnf.png

…efined or not

balihb · 2018-03-20T13:25:20Z

@carlossg
is there any chance to get this pulled?

giannello · 2018-03-25T12:21:06Z

@balihb for how much I'd love to have this merged, there is a better way than filtering streams this way.

This fabric8io/kubernetes-client#1045 provides the base support for keeping the signalling stream separate. Would you mind giving it a look?

carlossg · 2018-03-25T15:18:30Z

Yes, this whole section needs to be rewritten with the improvements in fabric8io/kubernetes-client#1045 JENKINS-50392

carlossg

To be rewritten with the improvements in fabric8io/kubernetes-client#1045

Thanks for the effort though!

carlossg · 2018-03-26T12:14:40Z

I think #300 will sort this out

balihb force-pushed the exitcode2 branch 5 times, most recently from d6d8703 to a97190b Compare October 31, 2017 08:29

marvinthepa mentioned this pull request Nov 2, 2017

Jenkins 47225: Fix ProcStarter #236

Merged

marvinthepa changed the title ~~filter exit code from command output~~ [WIPfilter exit code from command output Nov 2, 2017

marvinthepa changed the title ~~[WIPfilter exit code from command output~~ [WIP] filter exit code from command output Nov 2, 2017

balihb force-pushed the exitcode2 branch 2 times, most recently from 19f408b to 158774b Compare November 13, 2017 15:17

balihb changed the title ~~[WIP] filter exit code from command output~~ filter exit code from command output Nov 13, 2017

balihb force-pushed the exitcode2 branch from 158774b to c9c8db7 Compare November 14, 2017 16:04

balihb force-pushed the exitcode2 branch 4 times, most recently from 7943f8c to fdbd931 Compare November 22, 2017 16:38

balihb force-pushed the exitcode2 branch from f872453 to e738ea2 Compare January 2, 2018 08:35

filter exit code from command output

ba96d27

balihb force-pushed the exitcode2 branch from e738ea2 to ba96d27 Compare January 18, 2018 12:35

carlossg reviewed Jan 31, 2018

View reviewed changes

mgaal86 and others added 7 commits February 16, 2018 14:36

Merge remote-tracking branch 'upstream/exitcode2'

175fe22

review points from carlossg

4f8b28e

Merge remote-tracking branch 'upstream/master'

7c896c9

Merge pull request #1 from mosoka/master

85d4a92

merge with jenkinsci/kubernetes-plugin master and review modifications

removed temporary debug code

20c4fdd

Merge branch 'master' into exitcode2

303e83c

refactor outpout stream management in ContainerExecDecorator

488d8fb

balihb added 5 commits March 9, 2018 13:46

Merge branch 'master' into exitcode2

226381e

don't throw out passed down environment, but check if it is already d…

155dcc8

…efined or not

write out buffer on close

c02fcbc

Merge branch 'master' into exitcode2

f14db93

Merge branch 'master' into exitcode2

f9a5b9e

carlossg requested changes Mar 25, 2018

View reviewed changes

carlossg closed this Mar 26, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

filter exit code from command output #242

filter exit code from command output #242

balihb commented Oct 30, 2017 •

edited

Loading

marvinthepa commented Nov 2, 2017

balihb commented Nov 2, 2017

marvinthepa commented Nov 2, 2017

balihb commented Nov 6, 2017

marvinthepa commented Nov 7, 2017

balihb commented Nov 7, 2017

balihb commented Nov 13, 2017

balihb commented Nov 14, 2017

balihb commented Nov 21, 2017

MattLud commented Dec 29, 2017

balihb commented Jan 7, 2018 •

edited

Loading

carlossg commented Jan 31, 2018

carlossg left a comment

carlossg Jan 31, 2018

carlossg Jan 31, 2018

carlossg Jan 31, 2018

carlossg Jan 31, 2018

balihb commented Mar 9, 2018

balihb commented Mar 20, 2018

giannello commented Mar 25, 2018

carlossg commented Mar 25, 2018

carlossg left a comment

carlossg commented Mar 26, 2018

		@@ -0,0 +1,69 @@
		package org.csanchez.jenkins.plugins.kubernetes.pipeline;


		import org.apache.commons.lang.ArrayUtils;

		public class ContainerExecCutExitCodeUtil {

filter exit code from command output #242

filter exit code from command output #242

Conversation

balihb commented Oct 30, 2017 • edited Loading

marvinthepa commented Nov 2, 2017

balihb commented Nov 2, 2017

marvinthepa commented Nov 2, 2017

balihb commented Nov 6, 2017

marvinthepa commented Nov 7, 2017

balihb commented Nov 7, 2017

balihb commented Nov 13, 2017

balihb commented Nov 14, 2017

balihb commented Nov 21, 2017

MattLud commented Dec 29, 2017

balihb commented Jan 7, 2018 • edited Loading

carlossg commented Jan 31, 2018

carlossg left a comment

Choose a reason for hiding this comment

carlossg Jan 31, 2018

Choose a reason for hiding this comment

carlossg Jan 31, 2018

Choose a reason for hiding this comment

carlossg Jan 31, 2018

Choose a reason for hiding this comment

carlossg Jan 31, 2018

Choose a reason for hiding this comment

balihb commented Mar 9, 2018

balihb commented Mar 20, 2018

giannello commented Mar 25, 2018

carlossg commented Mar 25, 2018

carlossg left a comment

Choose a reason for hiding this comment

carlossg commented Mar 26, 2018

balihb commented Oct 30, 2017 •

edited

Loading

balihb commented Jan 7, 2018 •

edited

Loading