Deploying modules/content (relatively) slow?

[grtjn]

Particularly when deploying remotely one can notice that deploying modules or content can take quite long. 1500 modules can take 5 up to 10 minutes, while locally it runs in mere seconds.

Looking at the access log of the XCC server, it looks as if each file is uploaded individually, and synchronized. Also, I expect deployment is using the Ruby XCC library. It might be worthwhile to consider using MLCP instead for newer MarkLogic versions..

[grtjn]

Maybe worth measuring performance?

[grtjn]

Recent change that allows uploading only new/changed content could be applied to modules, schemas etc as well:

#551 (comment)

[grtjn]

Fix for #777 makes the --incremental=yes option work for modules, schemas, and triggers as well.

[grtjn]

Fixed in MASTER

[rlouapre]

We also have deploy modules taking 8 min for over 900 files.
Have you ever tried to compress it before the upload?

[grtjn]

Compressing would require logic to uncompress on MarkLogic side. That would be custom code.

Did you try with the --incremental=yes flag?

[rlouapre]

I do not believe --incremental=yes will help since it is running from Jenkins with the new modules database.