Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pi5 failed to contact RP1 firmware #6642

Open
llllllllqq opened this issue Feb 1, 2025 · 12 comments
Open

Pi5 failed to contact RP1 firmware #6642

llllllllqq opened this issue Feb 1, 2025 · 12 comments

Comments

@llllllllqq
Copy link

llllllllqq commented Feb 1, 2025

Describe the bug

dmesg says:

...
rp1-pio 1f00178000.pio: error -ENOENT: failed to contact RP1 firmware [ 3.957885] rp1-pio: probe of 1f00178000.pio failed with error -2
...

Although nothing strange occurred.
The OS still works fine without reboot.

Steps to reproduce the behaviour

It randomly occurs after sudo reboot --halt and then manually press the on-board power button to power on (with default eeprom config).

It also randomly occurs when use raspi-config to change power off behavior to "full power off" and then reboot/poweroff. (Sorry I cannot remember whether it is reboot or poweroff)

Update: It also occurs after a suddenly plug-out the usb-c power cable and then plug-in back.
It seems only poweroff-related behaviors will trigger this bug. I tested 30 times reboot, no bugs occurred.
Update2: reboot can indeed also trigger this bug, only less frequent.

Device (s)

Raspberry Pi 5

System

Raspberry Pi reference 2024-11-19 Generated using pi-gen, https://github.com/RPi-Distro/pi-gen, 891df1e21ed2b6099a2e6a13e26c91dea44b34d4, stage4

2025/01/22 00:16:51
Copyright (c) 2012 Broadcom version a7753063 (release) (embedded)

Linux rpi5 6.6.74+rpt-rpi-2712 #1 SMP PREEMPT Debian 1:6.6.74-1+rpt1 (2025-01-27) aarch64 GNU/Linux

Logs

1.txt

Additional context

Can it be safely ignored?

@pelwell
Copy link
Contributor

pelwell commented Feb 3, 2025

That's interesting - it looks like a dependency/ordering issue, and not one I've seen before:

[    3.922665] rp1-pio 1f00178000.pio: error -ENOENT: failed to contact RP1 firmware
[    3.922671] rp1-pio: probe of 1f00178000.pio failed with error -2
...
[    3.932284] rp1-firmware rp1_firmware: RP1 Firmware version eb39cfd516f8c90628aa9d91f52370aade5d0a55

I get:

[   13.445656] rp1-firmware rp1_firmware: RP1 Firmware version eb39cfd516f8c90628aa9d91f52370aade5d0a55
[   13.550428] rp1-pio 1f00178000.pio: Created instance as pio0

The code for communicating with the firmware exists in the firmware module, so I thought that would enforce an ordering. Some investigation is required.

@llllllllqq
Copy link
Author

That's interesting - it looks like a dependency/ordering issue, and not one I've seen before:

[    3.922665] rp1-pio 1f00178000.pio: error -ENOENT: failed to contact RP1 firmware
[    3.922671] rp1-pio: probe of 1f00178000.pio failed with error -2
...
[    3.932284] rp1-firmware rp1_firmware: RP1 Firmware version eb39cfd516f8c90628aa9d91f52370aade5d0a55

I get:

[   13.445656] rp1-firmware rp1_firmware: RP1 Firmware version eb39cfd516f8c90628aa9d91f52370aade5d0a55
[   13.550428] rp1-pio 1f00178000.pio: Created instance as pio0

The code for communicating with the firmware exists in the firmware module, so I thought that would enforce an ordering. Some investigation is required.

Thanks for your reply.
Here is my dmesg when the rp1&pio works fine.
I hope it can help to investigate this issue.

normal.txt

@pelwell
Copy link
Contributor

pelwell commented Feb 3, 2025

Yes - in the working case the order is reversed:

[    3.955929] rp1-firmware rp1_firmware: RP1 Firmware version eb39cfd516f8c90628aa9d91f52370aade5d0a55
[    3.961971] vc4-drm axi:gpu: bound 107c580000.hvs (ops vc4_hvs_ops [vc4])
[    3.967397] rp1-pio 1f00178000.pio: Created instance as pio0

@aBUGSworstnightmare-rpi
Copy link
Contributor

[ 5.557565] rp1-pio 1f00178000.pio: error -ENOENT: failed to contact RP1 firmware [ 5.557571] rp1-pio: probe of 1f00178000.pio failed with error -2 [ 5.558534] rp1-firmware rp1_firmware: RP1 Firmware version eb39cfd516f8c90628aa9d91f52370aade5d0a55

Seeing that on reboot as well /custom kernel on a Pi5 DUT, trying to get an overlay to work)

Linux Pi5uSDDUT 6.6.74-v8-16k+ #1 SMP PREEMPT Sun Feb 2 10:41:23 CET 2025 aarch64 GNU/Linux Revision : c04170 Serial : cb30eb1fd7e7e672 Model : Raspberry Pi 5 Model B Rev 1.0

@pelwell
Copy link
Contributor

pelwell commented Feb 3, 2025

It's really strange. I can't get it to fail in that way without hacks - even with a massive delay in the firmware driver, the pio driver waits. Anyway, I'm just testing a fix.

@aBUGSworstnightmare-rpi
Copy link
Contributor

seems quite random. Seen it frequently on reboots (running 'sudo reboot' on CLI)

pelwell added a commit to pelwell/linux that referenced this issue Feb 3, 2025
If the RP1 firmware has reported an error then return that from the PIO
probe function, otherwise defer the probing.

Link: raspberrypi#6642

Signed-off-by: Phil Elwell <[email protected]>
pelwell added a commit to pelwell/linux that referenced this issue Feb 3, 2025
To avoid pointless retries, let the probe function succeed if the
firmware interface is configured correctly but the firmware is
incompatible. The value of the private drvdata field holds the outcome.

Link: raspberrypi#6642

Signed-off-by: Phil Elwell <[email protected]>
@mayo228
Copy link

mayo228 commented Feb 3, 2025

is there a way to fix this? I keep getting this error and I can't boot the OS, should I be worried that I will loose data on my raspberry?

@pelwell
Copy link
Contributor

pelwell commented Feb 3, 2025

I don't understand why this affecting some people while I can't reproduce it without hacking the driver, but regardless, #6645 should fix the issue. After about 40 minutes you should be able to install a trial kernel with sudo rpi-update pulls/6645, after first backing up any important data.

@pelwell
Copy link
Contributor

pelwell commented Feb 3, 2025

And no, you shouldn't worry - it would only affect you if you wanted to use piolib to drive the GPIOs.

@aBUGSworstnightmare-rpi
Copy link
Contributor

aBUGSworstnightmare-rpi commented Feb 3, 2025

I have no idea if and how below is related but I just wanted to state that I'm not making use of piolib for accessing GPIO's.

[   13.678330] ieee80211 phy0: brcmf_p2p_send_action_frame: Unknown Frame: category 0xa, action 0x8
[   15.374753] Bluetooth: RFCOMM TTY layer initialized
[   15.374765] Bluetooth: RFCOMM socket layer initialized
[   15.374772] Bluetooth: RFCOMM ver 1.11
[   27.660651] brcmfmac: brcmf_set_channel: set chanspec 0xd022 fail, reason -52
[   27.768681] brcmfmac: brcmf_set_channel: set chanspec 0xd026 fail, reason -52
[   27.872656] brcmfmac: brcmf_set_channel: set chanspec 0xd02a fail, reason -52
[   27.976661] brcmfmac: brcmf_set_channel: set chanspec 0xd02e fail, reason -52
[   29.636663] brcmfmac: brcmf_set_channel: set chanspec 0xd090 fail, reason -52
[   29.636875] brcmfmac: brcmf_set_channel: set chanspec 0xd095 fail, reason -52
[   29.637196] brcmfmac: brcmf_set_channel: set chanspec 0xd099 fail, reason -52
[   29.637400] brcmfmac: brcmf_set_channel: set chanspec 0xd09d fail, reason -52
[   29.637594] brcmfmac: brcmf_set_channel: set chanspec 0xd0a1 fail, reason -52
[   29.637791] brcmfmac: brcmf_set_channel: set chanspec 0xd0a5 fail, reason -52

@pelwell
Copy link
Contributor

pelwell commented Feb 3, 2025

I have no idea if and how below is related

You should know that WiFi and Bluetooth information and warning messages are unrelated.

pelwell added a commit to pelwell/linux that referenced this issue Feb 3, 2025
To avoid pointless retries, let the probe function succeed if the
firmware interface is configured correctly but the firmware is
incompatible. The value of the private drvdata field holds the outcome.

Link: raspberrypi#6642

Signed-off-by: Phil Elwell <[email protected]>
@llllllllqq
Copy link
Author

It seems after applying sudo rpi-update pulls/6645, this issue disappeared. (I tested about 10 times reboot/halt/suddenly poweroff). Thanks for your work! If anything strange found, I'll post here.

BTW, can I safely stay at this firmware and wait for the next stable release offered by apt? Or do I need a sudo apt install --reinstall raspi-firmware to restore first before the next stable release?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants