Random crashes in initial.go

[dennisjenkins75]

Upon re-run, the mapserver does not recrash at the same spot, but will crash again sometime later. I'll run it with -debug to capture more detail and update this bug when I have it.

~$ ~/bin/mapserver-linux-x86_64 -version
Mapserver version: git-3ee8477d621a07e6ea4f6ddf06fd98d87cca794a
OS: linux
Architecture: amd64


INFO[19186] Initial rendering                             elapsed=3.848852989s mapblocks=4480 progress%=22 tiles=245
ERRO[19194] Tile render job tile                          LayerId=0 X=23 Y=3 Zoom=12 err="metadata too short: 96171, valuelength: 1684364322" prefix=tilerenderjob
INFO[19203] Initial rendering                             elapsed=17.032469164s mapblocks=5592 progress%=22 tiles=1518
ERRO[19213] Tile render job tile                          LayerId=0 X=23 Y=3 Zoom=12 err="metadata too short: 96171, valuelength: 1684364322" prefix=tilerenderjob
INFO[19221] Initial rendering                             elapsed=17.576638551s mapblocks=8775 progress%=22 tiles=1534
INFO[19221] Initial rendering                             elapsed=78.339187ms mapblocks=0 progress%=22 tiles=0
INFO[19226] Initial rendering                             elapsed=5.000932902s mapblocks=2902 progress%=22 tiles=360
INFO[19238] Initial rendering                             elapsed=12.349995172s mapblocks=7770 progress%=22 tiles=630
INFO[19258] Initial rendering                             elapsed=19.747092087s mapblocks=3179 progress%=22 tiles=185
INFO[19261] Initial rendering                             elapsed=3.091403663s mapblocks=3751 progress%=22 tiles=175
INFO[19267] Initial rendering                             elapsed=6.696210709s mapblocks=10000 progress%=23 tiles=400
INFO[19273] Initial rendering                             elapsed=5.360291985s mapblocks=10000 progress%=23 tiles=270
INFO[19275] Initial rendering                             elapsed=2.453625717s mapblocks=394 progress%=23 tiles=216
ERRO[19276] mapblock-pars                                 err="metadata too short: 96171, valuelength: 1684364322" x=46 y=0 z=-7
panic: metadata too short: 96171, valuelength: 1684364322

goroutine 54 [running]:
mapserver/tilerendererjob.initialRender(0xc000266990)
        /app/tilerendererjob/initial.go:24 +0x64c
mapserver/tilerendererjob.Job(0xc000266990)
        /app/tilerendererjob/job.go:23 +0x12b
created by main.main
        /app/main.go:66 +0x3f2

[BuckarooBanzay]

Are you sure you have the latest version?
This is actually a "feature" that skips the bad or not parseable map-parts:

mapserver/tilerendererjob/initial.go

Lines 21 to 26 in 98279ee

	result, err := ctx.MapBlockAccessor.FindNextLegacyBlocks(ctx.Settings, ctx.Config.Layers, ctx.Config.RenderingFetchLimit)
	if err != nil {
	logrus.Error("Error in initial rendering run, trying to continue: " + err.Error())
	continue
	}

In any case: you have a few very old mapblocks in there and i would be interested why my parser fails 🤔 any chance you could dump the mapblock in question?

[dennisjenkins75]

I can share the mapblock. I need to translate the coords to the block id though. x=46 y=0 z=-7.

[BuckarooBanzay]

I can share the mapblock. I need to translate the coords to the block id though. x=46 y=0 z=-7.

That would be the (sqlite) blockid: -117440466 (see: https://go.dev/play/p/sg8UsOSY7Qq)

Best to export it in the following format:

sqlite3 map.sqlite "select quote(data) from blocks where pos = -117440466" | cut -d\' -f2 | xxd -r -p

[dennisjenkins75]

Hmm, that use of cut and xxd did not produce what I think that you wanted... I'll get the block data out and attach it somehow in a few minutes.

1hit@EdgyNet:~$ ./bin/sqlite3 ./worlds/world/map.sqlite "select quote(data) from blocks where pos = -117440466" | cut -d\' -f2 | xxd -r -p
??x^?Zmz? 
          ?????O?z??7Xo??`???@??Z
                                 "??"??Bm?E9,7??c}?????C#J*?y{?ί??7?>֛?+?+?>?[?_???٥???c????????Ŀ???????K?]?????`?K????7?>ֻg~??|???????F<?䎿t?!??q????;B?E???}?>?3g|??+?y??z?#?K???µޟ??G?????_?n??]???
                                         ??`/??
?b<x??V????2^?/?G?c??f???/??j_?_*>??4ς?w?x??]?݊_%??_?k??}?ς????n8J??g?(r?/7???V)?O85??0P&~u
*?8?>??
       B?????#?Q??=??:?P????????????o]??????|艬2?W??w??~B?????ۢ????p????
                                                                       ?VW/??Vů;???I????ObT?A1?{???Ǥd?=x??????oG?
                                                                                                                 Z?bx#1??(Q??AR???t5?|
                                                                                                                                      j????K????aL?Ӟ?D?¸5???7!x^?]?n???nN
(rest truncated)

[dennisjenkins75]

Ah, I needed to redirect the output of tr, as it was intended to be binary.

1hit@EdgyNet:~$ ./bin/sqlite3 ./worlds/world/map.sqlite "select quote(data) from blocks where pos = -117440466" | cut -d\' -f2 > mapserver-issue-278.bin

1hit@EdgyNet:~$ ls -l ./mapserver-issue-278.bin 
-rw-r--r-- 1 1hit 1hit 10037 Oct 20 15:59 ./mapserver-issue-278.bin

# b/c github won't let me attach a ".bin" file....
$ zip -9u mapserver-issue-278.zip mapserver-issue-278.bin 

mapserver-issue-278.zip

[dennisjenkins75]

The mapblock (46, 0, -7):

http://el.edgy1.net:18080/#!/map/0/12/758/-116