Task not loaded by Celery (was: "Unable to upload (regression)")

[frafra]

Commit 2675ca3 has been merged into master, but it introduces a regression makes impossible to upload. This regression has been discovered with https://github.com/GeoNode/geonode-selenium. @hishamkaram

Please revert or fix it, thank you,

[t-book]

@frafra Is this with docker? I wonder as travis should have complained otherwise, right?

[frafra]

@t-book Yes, right, geonode-selenium tests are based on the containers generated by SPCGeonode.

[afabiani]

@frafra we can spend some time to add geonode-selenium tests to Travis also.

We can create tasks for both geonode running in dev mode and docker.

I will give a look at this soon.

[hishamkaram]

@frafra it fails even when this commit has been reverted :D

[frafra]

@hishamkaram I tried multiple times, just to be sure that it was a regression related to that specific commit. If the test fails it is probably because of a bug in the test scrip; for example, I am trying to make it work with Travis as @afabiani suggested, and I see that curl fails because is too old to support --retry-connrefused (I just replaced it with a Docker health check).

[afabiani]

@frafra how your test is building the docker images?

Spcgeonode is based on those images

olivierdalang/spcgeonode:django-latest

which I guess are not updated with master.

[hishamkaram]

@afabiani @frafra I think this is a old script, in this test we need to make sure our form/div and it's content have been loaded something like this:

            self.wait_support.until(
                EC.presence_of_element_located((By.CSS_SELECTOR, "#contents")))

then we can start our layer upload operation, I can help you to write this test

[frafra]

@afabiani The images are built every time:

    cd "$GEONODE_REPOSITORY/scripts/spcgeonode/"
    docker-compose -f docker-compose.yml down --volumes
    docker-compose -f docker-compose.yml up -d --build django geoserver postgres nginx

[frafra]

@hishamkaram The problem is not related with the test (mine has a timeout of 60 seconds after the upload starts), because the same thing happens if I try to upload the layer manually: with the current master I get 504 Gateway Time-out nginx/1.13.7, while if I revert the commit I am able to upload it in few seconds.

[afabiani]

@frafra haven't the chance to test it yet on my side, however, could it be related to ASYNC option? It is possible that the task is not loaded by celery somehow.

[frafra]

@afabiani You're right, thank you: using ASYNC_SIGNALS=False fixes the issue, but I suppose that is not ideal for production. It also make possible to delete layers (this functionality is not working on master at the moment).

[afabiani]

@frafra absolutely this does not solve the issue, it is only a workaround for the moment. At least this test sheds some light on the real problem though. That means that the shared task is not bounded to any queue managed by Celery and therefore it doesn't work.

If no fixes will be pushed in the meantime, I will try to give it a look during the week. Unfortunately I am very very busy in these days, so I cannot take care of this before 2/3 days.

[afabiani]

@frafra the task is not bounded to the "update" queue. Any chance to make another try using updated master branch?

If still not working we might envisage a revert for now.

[frafra]

Tests are failing on master.
master+reverted commit allows to upload layers, but not to remove them (testing layers removal is now part of geonode-selenium).
ASYNC_SIGNAL=False is the only workaround that fixes both issues.

[giohappy]

@frafra so the issue is not the commit for the async thumbnails. The problem is with celery workers and/or scheduler. The layer deletion use a celery task too, and you report that it fails too.
Are you sure your celery setup (with scpgeonode) is fine?

@afabiani I'd remove the regression label and create a new label for docker and maybe scpgeonode if it's specific.

[frafra]

@giohappy you are right, it is a celery issue; I will update the tests accordingly. Thank you all.