Linksys E4200 v2 can't autoupdate due to missing signatures #415

grische · 2024-06-03T19:28:16Z

The Autoupdater on a Linksys E4200 v2 is unable to validate (valid) signatures of the firmware manifest.

gluon: gluon-v2023.2.2+
ffmuc: v2024.4.2-next
Patch: https://github.com/freifunkMUC/site-ffm/blob/next/patches/targets-kirkwood.patch

https://map.ffmuc.net/#!/de/map/586d8ff5af6f

darkdragon-001 · 2024-06-03T19:33:04Z

# uci show autoupdater.next
autoupdater.next=branch
autoupdater.next.mirror='http://firmware.ffmuc.net/next/sysupgrade' 'http://5.1.66.255/next/sysupgrade' 'http://185.150.99.255/next/sysupgrade' 'http://[2001:678:e68:f000::]/next/sysupgrade' 'http://[2001:678:ed0:f000::]/next/sysupgrade'
autoupdater.next.good_signatures='1'
autoupdater.next.name='next'
autoupdater.next.pubkey='6dcfc670a4150e16962c1852066669d9b337f168d0f6a863ed930968c2f047eb' 'dc44c9810a0470b2de63990128dbae392a836f4385d23e57eb72880ea8fbcf16' '5700c7a266d80aa1c6c33f29835a7b200bdd500e88ee86c0a63e24a0023364f6' '216a34d34a15688e127d8d90e6c57587aa8c682b0648322f68338487657fd5ff' '2a74ed02120a7d48bb2dc9be988b3480ed99844054b3d7f3e5d3df27d19d814b' '56c4201f6ce2994678b0142e19099dd28d6ed17775d35ca9a7f12d9235890ffc' '3cc6dc6d0d7499d4d17cacad5f16336634e0b26700807f29cc241eaabbde481f' 'e7a50df4bc8875113eaf75dff5204e8753a9189baeed40c6db0509398a428820' '50bafd8216cab2ee1c11c215b528dd7c6396f3edfdab689c70ca04a9f284b931' 'cc7e3a6a4788d4246075923b298634f14a4fd179020ef6369fb74d601f55cc84' '18fecc717a237ea5702fb97fa4689f9307d0267e3ee8f09377dbed99397b77b9'

GoliathLabs · 2024-06-21T20:23:56Z

@darkdragon-001 did that change with the new next update? What happens if you execute autoupdater -f

darkdragon-001 · 2024-06-23T14:34:56Z

Nope, everything still the same.

# autoupdater -f
Retrieving manifest from http://firmware.ffmuc.net/next/sysupgrade/next.manifest ...
autoupdater: warning: manifest http://firmware.ffmuc.net/next/sysupgrade/next.manifest only carried 0 valid signatures, 1 are required
Retrieving manifest from http://185.150.99.255/next/sysupgrade/next.manifest ...
autoupdater: warning: manifest http://185.150.99.255/next/sysupgrade/next.manifest only carried 0 valid signatures, 1 are required
Retrieving manifest from http://[2001:678:e68:f000::]/next/sysupgrade/next.manifest ...
autoupdater: warning: manifest http://[2001:678:e68:f000::]/next/sysupgrade/next.manifest only carried 0 valid signatures, 1 are required
Retrieving manifest from http://[2001:678:ed0:f000::]/next/sysupgrade/next.manifest ...
autoupdater: warning: manifest http://[2001:678:ed0:f000::]/next/sysupgrade/next.manifest only carried 0 valid signatures, 1 are required
Retrieving manifest from http://5.1.66.255/next/sysupgrade/next.manifest ...
autoupdater: warning: manifest http://5.1.66.255/next/sysupgrade/next.manifest only carried 0 valid signatures, 1 are required
autoupdater: error: no usable mirror found

neocturne · 2024-06-24T16:37:27Z

What does lua -e 'print(require("platform_info").get_image_name())' print on the device?

(should be unrelated to signature verification, but I currently have no idea what might be going wrong for a single device)

grische · 2024-06-24T17:05:28Z

@neocturne isn't this just going to be this?

site-ffm/patches/targets-kirkwood.patch

Line 18 in e1c1b74

+device('linksys-e4200-v2', 'linksys_e4200-v2', {

@T0biii edited the initial comment with some more info a few hours ago.

neocturne · 2024-06-24T17:13:32Z

@neocturne isn't this just going to be this?

site-ffm/patches/targets-kirkwood.patch

Line 18 in e1c1b74

+device('linksys-e4200-v2', 'linksys_e4200-v2', {

It should be that if the device entry in the target file is correct, but if I'm looking at the right Device Tree, the device name might actually be set incorrectly and not match the image name.

darkdragon-001 · 2024-06-24T18:21:52Z

What does lua -e 'print(require("platform_info").get_image_name())' print on the device?

(should be unrelated to signature verification, but I currently have no idea what might be going wrong for a single device)

# lua -e 'print(require("platform_info").get_image_name())'
linksys-e4200-v2-viper

Indeed, this has the additional -viper suffix.

neocturne · 2024-06-24T22:55:45Z

Okay, regardless of the naming error, something very weird is going on.

As far as I can tell, there is nothing wrong with the autoupdater in the v2024.4.2-next firmware for the Linksys E4200 v2. To verify, I extracted the rootfs from the sysupgrade image, unpacked that in an armsr-armv7 Gluon system running in qemu, and used chroot to run the autoupdater binary + libraries from the extracted rootfs.

Depending on the model name I set in /tmp/sysinfo/model, this either resulted in the expected error (device not found due to the incorrect name) or an attempt to run the autoupdate. The signature was always verified correctly.

grische · 2024-06-25T10:38:18Z

@darkdragon-001 you can find an updated build for the device with the fixed name around 14:00 CEST here: https://github.com/freifunkMUC/site-ffm/actions/runs/9660907085?pr=446

neocturne · 2024-06-25T16:02:48Z

There's one more thing I'd like to check: Please provide the full /etc/config/autoupdater of the affected device.

darkdragon-001 · 2024-06-25T22:02:25Z

There's one more thing I'd like to check: Please provide the full /etc/config/autoupdater of the affected device.

# cat /etc/config/autoupdater

config autoupdater 'settings'
	option enabled '1'
	option branch 'next'
	option version_file '/lib/gluon/release'

config branch 'experimental'
	list mirror 'http://firmware.ffmuc.net/experimental/sysupgrade'
	list mirror 'http://5.1.66.255/experimental/sysupgrade'
	list mirror 'http://185.150.99.255/experimental/sysupgrade'
	list mirror 'http://[2001:678:e68:f000::]/experimental/sysupgrade'
	list mirror 'http://[2001:678:ed0:f000::]/experimental/sysupgrade'
	option good_signatures '1'
	option name 'experimental'
	list pubkey '6dcfc670a4150e16962c1852066669d9b337f168d0f6a863ed930968c2f047eb'
	list pubkey 'dc44c9810a0470b2de63990128dbae392a836f4385d23e57eb72880ea8fbcf16'
	list pubkey '5700c7a266d80aa1c6c33f29835a7b200bdd500e88ee86c0a63e24a0023364f6'
	list pubkey '216a34d34a15688e127d8d90e6c57587aa8c682b0648322f68338487657fd5ff'
	list pubkey '2a74ed02120a7d48bb2dc9be988b3480ed99844054b3d7f3e5d3df27d19d814b'
	list pubkey '56c4201f6ce2994678b0142e19099dd28d6ed17775d35ca9a7f12d9235890ffc'
	list pubkey '3cc6dc6d0d7499d4d17cacad5f16336634e0b26700807f29cc241eaabbde481f'
	list pubkey 'e7a50df4bc8875113eaf75dff5204e8753a9189baeed40c6db0509398a428820'
	list pubkey '50bafd8216cab2ee1c11c215b528dd7c6396f3edfdab689c70ca04a9f284b931'
	list pubkey 'cc7e3a6a4788d4246075923b298634f14a4fd179020ef6369fb74d601f55cc84'
	list pubkey '18fecc717a237ea5702fb97fa4689f9307d0267e3ee8f09377dbed99397b77b9'

config branch 'next'
	list mirror 'http://firmware.ffmuc.net/next/sysupgrade'
	list mirror 'http://5.1.66.255/next/sysupgrade'
	list mirror 'http://185.150.99.255/next/sysupgrade'
	list mirror 'http://[2001:678:e68:f000::]/next/sysupgrade'
	list mirror 'http://[2001:678:ed0:f000::]/next/sysupgrade'
	option good_signatures '1'
	option name 'next'
	list pubkey '6dcfc670a4150e16962c1852066669d9b337f168d0f6a863ed930968c2f047eb'
	list pubkey 'dc44c9810a0470b2de63990128dbae392a836f4385d23e57eb72880ea8fbcf16'
	list pubkey '5700c7a266d80aa1c6c33f29835a7b200bdd500e88ee86c0a63e24a0023364f6'
	list pubkey '216a34d34a15688e127d8d90e6c57587aa8c682b0648322f68338487657fd5ff'
	list pubkey '2a74ed02120a7d48bb2dc9be988b3480ed99844054b3d7f3e5d3df27d19d814b'
	list pubkey '56c4201f6ce2994678b0142e19099dd28d6ed17775d35ca9a7f12d9235890ffc'
	list pubkey '3cc6dc6d0d7499d4d17cacad5f16336634e0b26700807f29cc241eaabbde481f'
	list pubkey 'e7a50df4bc8875113eaf75dff5204e8753a9189baeed40c6db0509398a428820'
	list pubkey '50bafd8216cab2ee1c11c215b528dd7c6396f3edfdab689c70ca04a9f284b931'
	list pubkey 'cc7e3a6a4788d4246075923b298634f14a4fd179020ef6369fb74d601f55cc84'
	list pubkey '18fecc717a237ea5702fb97fa4689f9307d0267e3ee8f09377dbed99397b77b9'

config branch 'stable'
	list mirror 'http://firmware.ffmuc.net/stable/sysupgrade'
	list mirror 'http://5.1.66.255/stable/sysupgrade'
	list mirror 'http://185.150.99.255/stable/sysupgrade'
	list mirror 'http://[2001:678:e68:f000::]/stable/sysupgrade'
	list mirror 'http://[2001:678:ed0:f000::]/stable/sysupgrade'
	option good_signatures '3'
	option name 'stable'
	list pubkey '6dcfc670a4150e16962c1852066669d9b337f168d0f6a863ed930968c2f047eb'
	list pubkey 'dc44c9810a0470b2de63990128dbae392a836f4385d23e57eb72880ea8fbcf16'
	list pubkey '5700c7a266d80aa1c6c33f29835a7b200bdd500e88ee86c0a63e24a0023364f6'
	list pubkey '216a34d34a15688e127d8d90e6c57587aa8c682b0648322f68338487657fd5ff'
	list pubkey '2a74ed02120a7d48bb2dc9be988b3480ed99844054b3d7f3e5d3df27d19d814b'
	list pubkey '56c4201f6ce2994678b0142e19099dd28d6ed17775d35ca9a7f12d9235890ffc'
	list pubkey '3cc6dc6d0d7499d4d17cacad5f16336634e0b26700807f29cc241eaabbde481f'
	list pubkey 'e7a50df4bc8875113eaf75dff5204e8753a9189baeed40c6db0509398a428820'
	list pubkey '50bafd8216cab2ee1c11c215b528dd7c6396f3edfdab689c70ca04a9f284b931'
	list pubkey 'cc7e3a6a4788d4246075923b298634f14a4fd179020ef6369fb74d601f55cc84'
	list pubkey '18fecc717a237ea5702fb97fa4689f9307d0267e3ee8f09377dbed99397b77b9'

config branch 'testing'
	list mirror 'http://firmware.ffmuc.net/testing/sysupgrade'
	list mirror 'http://5.1.66.255/testing/sysupgrade'
	list mirror 'http://185.150.99.255/testing/sysupgrade'
	list mirror 'http://[2001:678:e68:f000::]/testing/sysupgrade'
	list mirror 'http://[2001:678:ed0:f000::]/testing/sysupgrade'
	option good_signatures '2'
	option name 'testing'
	list pubkey '6dcfc670a4150e16962c1852066669d9b337f168d0f6a863ed930968c2f047eb'
	list pubkey 'dc44c9810a0470b2de63990128dbae392a836f4385d23e57eb72880ea8fbcf16'
	list pubkey '5700c7a266d80aa1c6c33f29835a7b200bdd500e88ee86c0a63e24a0023364f6'
	list pubkey '216a34d34a15688e127d8d90e6c57587aa8c682b0648322f68338487657fd5ff'
	list pubkey '2a74ed02120a7d48bb2dc9be988b3480ed99844054b3d7f3e5d3df27d19d814b'
	list pubkey '56c4201f6ce2994678b0142e19099dd28d6ed17775d35ca9a7f12d9235890ffc'
	list pubkey '3cc6dc6d0d7499d4d17cacad5f16336634e0b26700807f29cc241eaabbde481f'
	list pubkey 'e7a50df4bc8875113eaf75dff5204e8753a9189baeed40c6db0509398a428820'
	list pubkey '50bafd8216cab2ee1c11c215b528dd7c6396f3edfdab689c70ca04a9f284b931'
	list pubkey 'cc7e3a6a4788d4246075923b298634f14a4fd179020ef6369fb74d601f55cc84'
	list pubkey '18fecc717a237ea5702fb97fa4689f9307d0267e3ee8f09377dbed99397b77b9'

neocturne · 2024-06-26T16:24:06Z

Okay, still no idea what is going on...

darkdragon-001 · 2024-06-26T19:36:40Z

This seems very farfetched but could it be that the signature verification instructions don't work on that chip correctly for some reason?
How difficult would it be to verify such claim? Basically creating an executable taking a file, signature and public key as arguments and printing the result of the check?

neocturne · 2024-07-04T17:08:32Z

I guess the first step would be to add some debug logging to the autoupdater (printing the downloaded manifest, SHA256 hash, individual verification inputs and results, maybe some other things I'm forgetting) to narrow down the cause. Once we've done that it might make sense to write a test program for the specific thing that goes wrong.

grische · 2024-07-04T20:01:38Z

@neocturne I can add a bunch of print statements all over the place. Do you mean in the upstream package or as a (temporary) patch in the firmware?

neocturne · 2024-07-05T12:39:08Z

I have pushed a Gluon branch that includes a debug patch for the autoupdater: https://github.com/neocturne/gluon/tree/autoupdater-debug

Run autoupdater with the additional argument -d to dump the whole downloaded manifest, as well as a few values I'm interested in.

As the patch is rather small, it might also make sense to include it in the upstream autoupdater in the future.

grische · 2024-07-05T12:54:14Z

@darkdragon-001 you will find a new firmware with the above patch by 16:30 CEST: https://github.com/freifunkMUC/site-ffm/actions/runs/9808595276

@neocturne I added this on top of Gluon v2023.2.3 and it seemed to apply cleanly: 67821f7

darkdragon-001 · 2024-08-25T13:45:46Z

Thanks for your help!

@neocturne I extracted the autoupdater binary from @grische's image, transferred it to my router via scp and ran it there. You can find the output in the following file: autoupdater-debug-output.md

neocturne · 2024-08-26T18:03:11Z

Huh. Calculating 4 different incorrect SHA256 hashes for the same manifest from 4 mirrors - even though the manifest is transferred correctly, and fed into ecdsa_sha256_update line by line deterministically - is wild. I guess we'll need to have a closer look at the SHA256 implementation next.

neocturne · 2024-08-27T20:09:18Z

I've updated my autoupdater-debug branch to add tracing around the ecdsa_sha256_update calls when passing -d to the autoupdater.

neocturne · 2024-08-29T19:38:43Z

I also have a new idea what might be cause of the issue (and a possible fix), but let's wait for the log with the current version first.

T0biii · 2024-09-01T17:02:11Z

I added the updated autoupdater-debug branch on top of Gluon v2023.2.3: 1797fe7

@darkdragon-001 the build for the new firmware can be found here: https://github.com/freifunkMUC/site-ffm/actions/runs/10655791524

neocturne · 2024-09-17T16:26:02Z

I got a Linksys EA4500 here now and could reproduce the SHA256 issue, so I should be able find the root cause soon.

neocturne · 2024-09-21T15:09:34Z

Will be fixed by freifunk-gluon/ecdsautils#1

grische added the next label Jun 3, 2024

grische added the bug label Jun 24, 2024

neocturne mentioned this issue Jun 24, 2024

kirkwood-generic: fix device name of Linksys E4200 v2 freifunk-gluon/gluon#3292

Merged

neocturne mentioned this issue Sep 21, 2024

SHA256 fixes freifunk-gluon/ecdsautils#1

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Linksys E4200 v2 can't autoupdate due to missing signatures #415

Linksys E4200 v2 can't autoupdate due to missing signatures #415

grische commented Jun 3, 2024 •

edited by T0biii

Loading

darkdragon-001 commented Jun 3, 2024

GoliathLabs commented Jun 21, 2024

darkdragon-001 commented Jun 23, 2024

neocturne commented Jun 24, 2024 •

edited

Loading

grische commented Jun 24, 2024 •

edited

Loading

neocturne commented Jun 24, 2024

darkdragon-001 commented Jun 24, 2024 •

edited

Loading

neocturne commented Jun 24, 2024

grische commented Jun 25, 2024

neocturne commented Jun 25, 2024

darkdragon-001 commented Jun 25, 2024

neocturne commented Jun 26, 2024

darkdragon-001 commented Jun 26, 2024 •

edited

Loading

neocturne commented Jul 4, 2024

grische commented Jul 4, 2024

neocturne commented Jul 5, 2024

grische commented Jul 5, 2024 •

edited

Loading

darkdragon-001 commented Aug 25, 2024 •

edited

Loading

neocturne commented Aug 26, 2024

neocturne commented Aug 27, 2024

neocturne commented Aug 29, 2024

T0biii commented Sep 1, 2024 •

edited

Loading

neocturne commented Sep 17, 2024

neocturne commented Sep 21, 2024

Linksys E4200 v2 can't autoupdate due to missing signatures #415

Linksys E4200 v2 can't autoupdate due to missing signatures #415

Comments

grische commented Jun 3, 2024 • edited by T0biii Loading

darkdragon-001 commented Jun 3, 2024

GoliathLabs commented Jun 21, 2024

darkdragon-001 commented Jun 23, 2024

neocturne commented Jun 24, 2024 • edited Loading

grische commented Jun 24, 2024 • edited Loading

neocturne commented Jun 24, 2024

darkdragon-001 commented Jun 24, 2024 • edited Loading

neocturne commented Jun 24, 2024

grische commented Jun 25, 2024

neocturne commented Jun 25, 2024

darkdragon-001 commented Jun 25, 2024

neocturne commented Jun 26, 2024

darkdragon-001 commented Jun 26, 2024 • edited Loading

neocturne commented Jul 4, 2024

grische commented Jul 4, 2024

neocturne commented Jul 5, 2024

grische commented Jul 5, 2024 • edited Loading

darkdragon-001 commented Aug 25, 2024 • edited Loading

neocturne commented Aug 26, 2024

neocturne commented Aug 27, 2024

neocturne commented Aug 29, 2024

T0biii commented Sep 1, 2024 • edited Loading

neocturne commented Sep 17, 2024

neocturne commented Sep 21, 2024

grische commented Jun 3, 2024 •

edited by T0biii

Loading

neocturne commented Jun 24, 2024 •

edited

Loading

grische commented Jun 24, 2024 •

edited

Loading

darkdragon-001 commented Jun 24, 2024 •

edited

Loading

darkdragon-001 commented Jun 26, 2024 •

edited

Loading

grische commented Jul 5, 2024 •

edited

Loading

darkdragon-001 commented Aug 25, 2024 •

edited

Loading

T0biii commented Sep 1, 2024 •

edited

Loading