error in step cloudFoundryDeploy: running command 'cf' failed [ref:#3297]

[nuramon9999]

ref:#3297

Hello,
The once vanished error is back and with that I will try to supply the necessary information.

We also recently updated jenkins and are unsure as to if that might be a factor in the unsuccessful runs.
The first error occurs in this step: /piper cloudFoundryDeploy
— Shell Script

info  cloudFoundryDeploy - REQUEST: [2022-06-03T08:20:51Z]

info  cloudFoundryDeploy - GET /login HTTP/1.1

info  cloudFoundryDeploy - Host: login.cf.eu10.hana.ondemand.com

info  cloudFoundryDeploy - Accept: application/json

info  cloudFoundryDeploy - Authorization: [PRIVATE DATA HIDDEN]

info  cloudFoundryDeploy - Connection: close

info  cloudFoundryDeploy - User-Agent: cf/7.4.0+e55633fed.2021-11-15 (go1.16.6; amd64 linux)

info  cloudFoundryDeploy - 

info  cloudFoundryDeploy - RESPONSE: [2022-06-03T08:20:51Z]

info  cloudFoundryDeploy - HTTP/1.1 200 OK

info  cloudFoundryDeploy - Cache-Control: no-store

info  cloudFoundryDeploy - Content-Language: en-US

info  cloudFoundryDeploy - Content-Type: application/json;charset=UTF-8

info  cloudFoundryDeploy - Date: Fri, 03 Jun 2022 08:20:51 GMT

info  cloudFoundryDeploy - Set-Cookie: [PRIVATE DATA HIDDEN]

info  cloudFoundryDeploy - Strict-Transport-Security: max-age=31536000; includeSubDomains; preload;

info  cloudFoundryDeploy - X-Content-Type-Options: nosniff

info  cloudFoundryDeploy - X-Frame-Options: DENY

info  cloudFoundryDeploy - X-Vcap-Request-Id: e5d21b8e-7f4b-4d05-6cf7-2ffa23323bc8

info  cloudFoundryDeploy - X-Xss-Protection: 1; mode=block

info  cloudFoundryDeploy - {

info  cloudFoundryDeploy -   "app": {

info  cloudFoundryDeploy -     "version": "75.19.0"

info  cloudFoundryDeploy -   },

info  cloudFoundryDeploy -   "commit_id": "99f1161",

info  cloudFoundryDeploy -   "defaultIdpName": "sap.ids",

info  cloudFoundryDeploy -   "entityID": "login.cf.eu10.hana.ondemand.com",

info  cloudFoundryDeploy -   "idpDefinitions": {},

info  cloudFoundryDeploy -   "links": {

info  cloudFoundryDeploy -     "login": "https://login.cf.eu10.hana.ondemand.com/",

info  cloudFoundryDeploy -     "uaa": "https://uaa.cf.eu10.hana.ondemand.com/"

info  cloudFoundryDeploy -   },

info  cloudFoundryDeploy -   "prompts": {

info  cloudFoundryDeploy -     "passcode": [

info  cloudFoundryDeploy -       "password",

info  cloudFoundryDeploy -       "Temporary Authentication Code ( Get one at https://login.cf.eu10.hana.ondemand.com/passcode )"

info  cloudFoundryDeploy -     ],

info  cloudFoundryDeploy -     "password": "[PRIVATE DATA HIDDEN]",

info  cloudFoundryDeploy -     "username": [

info  cloudFoundryDeploy -       "text",

info  cloudFoundryDeploy -       "Email"

info  cloudFoundryDeploy -     ]

info  cloudFoundryDeploy -   },

info  cloudFoundryDeploy -   "showLoginLinks": true,

info  cloudFoundryDeploy -   "timestamp": "2022-04-27T13:09:42+0000",

info  cloudFoundryDeploy -   "zone_name": "uaa"

info  cloudFoundryDeploy - }

info  cloudFoundryDeploy - 

info  cloudFoundryDeploy - 

info  cloudFoundryDeploy - ### END OF CF CLI TRACE OUTPUT ###

info  cloudFoundryDeploy - fatal error: errorDetails{"category":"undefined","correlationId":"[http://svr-dev-build.int.korasoft.net/job/ci-kora-suite-backend/40852/","error":"running](http://svr-dev-build.int.korasoft.net/job/ci-kora-suite-backend/40852/%22,%22error%22:%22running) command 'cf' failed: cmd.Run() failed: exit status 1","library":"SAP/jenkins-library","message":"step execution failed: running command 'cf' failed: cmd.Run() failed: exit status 1","result":"failure","stepName":"cloudFoundryDeploy","time":"2022-06-03T08:20:52.098516911Z"}

fatal cloudFoundryDeploy - step execution failed: running command 'cf' failed: cmd.Run() failed: exit status 1 - running command 'cf' failed: cmd.Run() failed: exit status 1

info  cloudFoundryDeploy - Step telemetry data:{"StepStartTime":"2022-06-03 08:20:13.237788763 +0000 UTC","PipelineURLHash":"3f5d6f9a4551c523587616a64c3a7e56c0c988f9","BuildURLHash":"9660754ec0231e4824a54321056c4e0ee5810f89","StageName":"test \u0026 deploy","StepName":"cloudFoundryDeploy","ErrorCode":"1","StepDuration":"38867","ErrorCategory":"undefined","CorrelationID":"[http://svr-dev-build.int.korasoft.net/job/ci-kora-suite-backend/40852/","PiperCommitHash":"903f27301271bf3421752740a566384ff505117a","ErrorDetail":{"category":"undefined","correlationId":"http://svr-dev-build.int.korasoft.net/job/ci-kora-suite-backend/40852/","error":"running](http://svr-dev-build.int.korasoft.net/job/ci-kora-suite-backend/40852/%22,%22PiperCommitHash%22:%22903f27301271bf3421752740a566384ff505117a%22,%22ErrorDetail%22:%7B%22category%22:%22undefined%22,%22correlationId%22:%22http://svr-dev-build.int.korasoft.net/job/ci-kora-suite-backend/40852/%22,%22error%22:%22running) command 'cf' failed: cmd.Run() failed: exit status 1","library":"SAP/jenkins-library","message":"step execution failed: running command 'cf' failed: cmd.Run() failed: exit status 1","result":"failure","stepName":"cloudFoundryDeploy","time":"2022-06-03T08:20:52.098516911Z"}}

script returned exit code 1

Later errors refer to the shown issue:

----------------------------------------------------------

--- An error occurred in the library step: dockerExecute

----------------------------------------------------------



The following parameters were available to the step:

***

*** to show step parameters, set verbose:true in general pipeline configuration

*** WARNING: this may reveal sensitive information. ***

***



The error was:

***

hudson.AbortException: [cloudFoundryDeploy] Step execution failed (category: undefined). Error: running command 'cf' failed: cmd.Run() failed: exit status 1

***



Further information:

* Documentation of library step dockerExecute: https://sap.github.io/jenkins-library/steps/dockerExecute/

* Source code of library step dockerExecute: https://github.com/SAP/jenkins-library/blob/master/vars/dockerExecute.groovy

* Library documentation: https://sap.github.io/jenkins-library/

* Library repository: https://github.com/SAP/jenkins-library/



----------------------------------------------------------

--- End library step of: dockerExecute ---

[cloudFoundryDeploy] Step execution failed (category: undefined). Error: running command 'cf' failed: cmd.Run() failed: exit status 1

----------------------------------------------------------

--- An error occurred in the library step: cloudFoundryDeploy

----------------------------------------------------------



The following parameters were available to the step:

***

*** to show step parameters, set verbose:true in general pipeline configuration

*** WARNING: this may reveal sensitive information. ***

***



The error was:

***

hudson.AbortException: [cloudFoundryDeploy] Step execution failed (category: undefined). Error: running command 'cf' failed: cmd.Run() failed: exit status 1

***



Further information:

* Documentation of library step cloudFoundryDeploy: https://sap.github.io/jenkins-library/steps/cloudFoundryDeploy/

* Source code of library step cloudFoundryDeploy: https://github.com/SAP/jenkins-library/blob/master/vars/cloudFoundryDeploy.groovy

* Library documentation: https://sap.github.io/jenkins-library/

* Library repository: https://github.com/SAP/jenkins-library/



----------------------------------------------------------

--- End library step of: cloudFoundryDeploy ---

[cloudFoundryDeploy] Step execution failed (category: undefined). Error: running command 'cf' failed: cmd.Run() failed: exit status 1

How do we get the deploy stage to pass?

Best of wishes,
Pascal Ehrlich.

[ashlymat]

Hi @nuramon9999 ,

Could you please point me to your pipelines to access the logs? The error here looks too generic

[matripat-simac]

I'm also facing same issue. Any resolution yet?

[nuramon9999]

Hi @nuramon9999 ,

Could you please point me to your pipelines to access the logs? The error here looks too generic

Hello,
that kind of was the issue. We escalated the ticket and found the issue by trying ... a lot. I will document what we found and how we solved it.

[nuramon9999]

I'm also facing same issue. Any resolution yet?

For us yes. It was an issue of not having enough memory for us. For the final deployment step, we temporarily have two instances of our app running, which meant that we needed double the amount of quota for the step. Weird thing was, that after it running fine once we could reduce it back downto the typical parameters that we needed and it worked fine.

Depending on the issue there should be more in your logs. Tomorrow I would be able to show some more info and how we found out about the problem.

[ashlymat]

Hi @nuramon9999 ,
Glad that it is working now.
Will be helpful if you share the info here.
Closing the ticket in the meantime.

[i353590]

Hello,

I am also facing this issue. My deployment shows successful but I get error at the end: 11:14:41 info cloudFoundryDeploy - Step telemetry data:{"StepStartTime":"2023-08-04 05:42:01.981580358 +0000 UTC","PipelineURLHash":"3e75fc5d13dfe22db53dbd84eeab769ba418d39b","BuildURLHash":"45f7332196ecf3ce7bd40af8fec501d1cfe3f12d","StageName":"Deploy Mock","StepName":"cloudFoundryDeploy","ErrorCode":"0","StepDuration":"159831","ErrorCategory":"undefined","CorrelationID":"https://gkerefappscicd.jaas-gcp.cloud.sap.corp/job/s4h-ext-mission/1065/","PiperCommitHash":"ff5eddfb7cbc5fc50e7947ed852de2eae96e12ed","ErrorDetail":null}

There's no specific info about the error to dig into.Please help me here.

Thanks in advance.
Shubham

[nuramon9999]

Hello,

I am also facing this issue. My deployment shows successful but I get error at the end: 11:14:41 info cloudFoundryDeploy - Step telemetry data:{"StepStartTime":"2023-08-04 05:42:01.981580358 +0000 UTC","PipelineURLHash":"3e75fc5d13dfe22db53dbd84eeab769ba418d39b","BuildURLHash":"45f7332196ecf3ce7bd40af8fec501d1cfe3f12d","StageName":"Deploy Mock","StepName":"cloudFoundryDeploy","ErrorCode":"0","StepDuration":"159831","ErrorCategory":"undefined","CorrelationID":"https://gkerefappscicd.jaas-gcp.cloud.sap.corp/job/s4h-ext-mission/1065/","PiperCommitHash":"ff5eddfb7cbc5fc50e7947ed852de2eae96e12ed","ErrorDetail":null}

There's no specific info about the error to dig into.Please help me here.

Thanks in advance. Shubham

Hello Shubham,

I can only refer you to our memory solution that actually solved this for us. We had a certain amount of ram, gave some more to the application and all of a sudden the build went through. We just removed the ram again afterwards. because on paper the application is running fine with the amount of ram we had dedicated to it.
That is, why this was so hard to find for us - it seemed to need more memory during the build process, than it needs running. Also most of the builds still go through with less memory after it is successful once, so we could remove the memory again.

Sorry, that I do not have a clearer solution for you. With our issue, we also found a remark of the memory we used and it not having enough memory, even though we were on 2.4Gb and had 4 available. The only thing that made sense to us seems to be, that during the build, the application already allocates its ram, while the old application still blocks its memory for a short time.

Good luck,
Pascal.