Line layer uploaded to GeoNode is rendered as point layer

[capooti]

Expected Behavior

A line shapefile is uploaded to GeoNode. The layer is correctly registered in GeoNode, a line style is created in GeoServer and the style is associated to the layer as the default style.

Actual Behavior

The line layer is uploaded in GeoNode. In GeoServer two styles are created for the layer, one named "layername" (with a point symbolizer) and the other "layername_layer" (with a line symbolizer). The incorrect one is associated to the layer.

Steps to Reproduce the Problem

I can't reproduce this on GeoNode master, but I noticed this happening randomly on two different GeoNode instances.

Specifications

• GeoNode version: master
• Installation method (manual, GeoNode Docker, SPCGeoNode Docker): manual
• Platform: Ubuntu
• Additional details:

[capooti]

I believe this is connected with this issue as well: #2932

Under some circumstances GeoNode can't get the right response from GeoServer REST styles endpoint.

[capooti]

Reopening this, as I am experiencing again.

The layer is uploaded and I see this in log:
Step 7. Creating style for [subwaylines_p_odp]
Could not find any suitable style in GeoServer for Layer: "subwaylines_p_odp"
default style set to subwaylines_p_odp

but the layer is associated with the default style set to "point", even if in GeoServer there are two newly created styles (which are correctly with a LineSymbolizer): "subwaylines_p_odp", "subwaylines_p_odp_layer"

[capooti]

@afabiani I debugged this strange behaviour in one instance with many layers (about 1,000)
My best guess is that the code creating the style in step 7 of the upload process is getting always a point style because any cat.get_style is unsuccessful. It appears that this must be related to the time the cat.create_style (in gsconfig) needs to execute (it uses an http_request). Infact using a time.sleep(3) after cat.create_style fix things and finally cat.get_style works.
Do you have any recommendation about how to tackle this problem?
I was thinking that a possible approach could be to return the style in the create_style function in gsconfig, what do you think?

[afabiani]

@capooti yes, we should somehow allow GeoNode to double check if the style has been created or not.
Probably a simple upgrade of the gsconfig library would be sufficient.

FWI, we forked from the Boundless one. We now rely on a new one which runs with Python3 also:

https://github.com/GeoNode/geoserver-restconfig

Feel free to track issues and open PRs there.

[capooti]

Yes, I am aware of the fork and was already looking the source code. Should I close the ticket here and open a new one there?

[capooti]

Also, I am using already the very latest version (gsconfig==1.0.10) as I created the virtualenv using requirements.txt from master

[afabiani]

No, let's keep this issue open too. Let me know if you need some help.

[capooti]

@afabiani I can work with this but I am not sure what would be the best approach.
How about using requests.packages.urllib3.util.retry.Retry as suggested here? https://www.peterbe.com/plog/best-practice-with-retries-with-requests

This could be done in geoserver-restconfig in catalog.create_style. Right now create_style doesn't return anything. We could return the style itself or None in case even with Retry the style can't be retrieved successfully

[afabiani]

Yep, it looks to me a good approach.

[capooti]

@afabiani I realized my virtualenv had both gsconfig==1.0.10 and geoserver-restconfig==1.0.3 and GeoNode was using gsconfig in place of geoserver-restconfig. Just after uninstalling gsconfig GeoNode started using geoserver-restconfig and this issue seems now gone :)
For now I am closing it, hopefully we won't need to re-open it.
Thanks again for helping!

[capooti]

A user reported that now layers are correctly styled, but thumbnails still are created using a point style. Not sure if this is connected.

[capooti]

Unfortunately I have to reopen this. The problem arise intermittently both with line and polygon layers, which are styled with lines...

[capooti]

@afabiani I debugged the issue with a script using geoserver-config and it definitely look to be a GeoServer scalability problem.

Basically in the instance where I am experiencing the problem I can successfully read a newly created style only after 4/5 seconds (!!!) from the creation of the style itself. Looking at the GeoServer log it appears that, during the style read operation, GeoFence is checking every single layer in the instance - that is why it takes a while.

For each layer this is what I see in the log:

2019-09-20 02:10:21,501 DEBUG [org.geoserver.geofence] - Admin auth for User: Workspace:geonode: false
2019-09-20 02:10:21,501 DEBUG [org.geoserver.geofence] - Getting access limits for Layer a__5a0
2019-09-20 02:10:21,501 DEBUG [org.geoserver.geofence] - Getting access limits for Resource a__5a0
2019-09-20 02:10:21,501 DEBUG [org.geoserver.geofence] - ResourceInfo filter: RuleFilter[user:DEFAULT role:ANY inst:name+:default-gs ip:"127.0.0.1"+ serv:"WMS"+ req:"GETCAPABILITIES"+ ws:"geonode"+ layer:"a__5a0"+]
2019-09-20 02:10:21,501 DEBUG [org.geoserver.geofence.cache] - Request for RuleFilter[user:DEFAULT role:ANY inst:name+:default-gs ip:"127.0.0.1"+ serv:"WMS"+ req:"GETCAPABILITIES"+ ws:"geonode"+ layer:"a__5a0"+]
2019-09-20 02:10:21,501 DEBUG [org.geoserver.geofence] - Returning mode HIDE for resource FeatureTypeInfoImpl[a__5a0]
2019-09-20 02:10:21,501 DEBUG [org.geoserver.geofence] - Returning VectorAccessLimits [readAttributes=null, writeAttributes=null, writeFilter=Filter.EXCLUDE, readFilter=Filter.EXCLUDE, mode=HIDE] for layer geonode:a__5a0 and user null
2019-09-20 02:10:21,501 DEBUG [org.geoserver.geofence] - Getting access limits for workspace geonode
2019-09-20 02:10:21,501 DEBUG [org.geoserver.geofence] - Getting admin auth for Workspace geonode
2019-09-20 02:10:21,501 DEBUG [org.geoserver.geofence] - AdminAuth filter: RuleFilter[user:DEFAULT role:ANY inst:name+:default-gs ip:"127.0.0.1"+ serv:ANY req:ANY ws:"geonode"+ layer:ANY]

The big latency between the the style creation and its successful read (4/5 seconds in this case) explains why now GeoNode defaults to a point style.

Any suggestion about the approach for tackling this problem?

[simboss]

Ciao @capooti,
can you share this script so that we can reproduce?

[capooti]

@afabiani unfortunately as soon as I started using this in our instances the issues started happening again.
In the last 7/8 days we used this [1], and we didn't notice anymore #4827 and #5049 happening

[1] https://github.com/capooti/geonode/tree/camp-before-2.10.1-merge

[afabiani]

@capooti I'll compare the branches and do some tests tomorrow.

[capooti]

thanks @afabiani ;)

[capooti]

Unfortunately this is happening again, so I have to re-open it.
Everything works correctly with this branch:
https://github.com/capooti/geonode/tree/camp-before-2.10.1-merge

[capooti]

@afabiani here is a fix
https://github.com/capooti/geonode/pull/new/fixes-4827-geonode-2.10.1

the two lines which I removed are not anymore needed
cat.create_style will raise an error as the style is already there and in any case it is useless as we don't read the style again after creating it, so the next "if style" block won't be execute.

Not sure where to send the PR now that we released 2.10.1...

[afabiani]

@capooti this patch will go into 2.10.2

[afabiani]

@capooti I'm going to create a new issue referring to this one and assigned to 2.10.2 milestone.

[afabiani]

Scheduled for 2.10.2 through #5272