Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IRQ trigger rising edge doesnt work with 5.10.4 #4096

Closed
Makurisan opened this issue Jan 25, 2021 · 45 comments
Closed

IRQ trigger rising edge doesnt work with 5.10.4 #4096

Makurisan opened this issue Jan 25, 2021 · 45 comments

Comments

@Makurisan
Copy link

Makurisan commented Jan 25, 2021

I called this function in the probe and the handler is immeadiately called and the value on the GPIO is 0. It seems the irq handler is configured with POLLing enabled but I want only rising edge.

rc = devm_request_irq(&vhub->spi->dev, vhub->irq, ast_vhub_irq, IRQF_TRIGGER_RISING | IRQF_SHARED,
					  KBUILD_MODNAME, vhub);

Under /proc/interrupts the interrupt appears with the "Edge" attribute
50: 3744 0 0 0 pinctrl-bcm2835 17 Edge v_hub

And the pin is defined:
GPIO 17: level=0 fsel=0 func=INPUT pull=DOWN

Is this a bug or what leads to this?

Tested with PI4 4GB and kernel
Linux pi 5.10.4-v8+ #1389 SMP PREEMPT Wed Jan 6 13:52:18 GMT 2021 aarch64 GNU/Linux

@pelwell
Copy link
Contributor

pelwell commented Jan 25, 2021

How are you declaring the GPIO interrupt? It is normal (and perhaps necessary) to declare the trigger type using the IRQ flags in Device Tree, e.g. (from https://github.com/raspberrypi/linux/blob/rpi-5.10.y/arch/arm/boot/dts/overlays/sc16is750-i2c-overlay.dts):

			sc16is750: sc16is750@48 {
				compatible = "nxp,sc16is750";
				reg = <0x48>; /* i2c address */
				clocks = <&sc16is750_clk>;
				interrupt-parent = <&gpio>;
				interrupts = <24 2>; /* IRQ_TYPE_EDGE_FALLING */
				gpio-controller;
				#gpio-cells = <2>;
				i2c-max-frequency = <400000>;
			};

IRQ_TYPE_EDGE_RISING has the value 1, so your interrupts declaration would be:

				interrupt-parent = <&gpio>;
				interrupts = <0 1>; /* IRQ_TYPE_EDGE_RISING */

@Makurisan
Copy link
Author

Ok, at the moment we use the config.txt with
dtoverlay=anyspi:spi0-0,dev="hubshield,v-hub",speed=1000000
In the linux documentation it is said that request_irq with parameter flag equal zero use the device tree otherwise the flags which are specified.

Is it possible to set the parameter in the config.txt?

@pelwell
Copy link
Contributor

pelwell commented Jan 25, 2021

The anyspi overlay doesn't support interrupts. You will have to make a custom overlay to go along with your out-of-tree driver.

@Makurisan
Copy link
Author

Makurisan commented Jan 26, 2021

  1. I set the gpio nbr already in code. Why is this not possible for request_irq. Can you explain the reason for that?
  2. Can you give me an example which show this overlay stuff with spi. I tried the following without success. Do you see any problem because it is not loaded.
/dts-v1/;
/plugin/;

/*
#include <include/dt-bindings/interrupt-controller/irq.h>
 */

/ {
	compatible = "brcm,bcm2711";

	fragment@0 {
		target = <&spidev0>;
		__overlay__ {
			status = "disabled";
		};
	};

	fragment@1 {
		target = <&spi0>;
        __overlay__ {
			/* needed to avoid dtc warning */
    		#address-cells = <1>;
			#size-cells = <0>;
 			status = "okay";
            vhub: vhub@48 {
				compatible = "hubshield,v-hub";
				reg = <0x0>;
				interrupt-parent = <&gpio>;
				interrupts = <0 2>; /* IRQ_TYPE_EDGE_FALLING */
                reset-gpios = <&gpio 25 1>; // GPIO_ACTIVE_HIGH
				gpio-controller;
				#gpio-cells = <2>;
				spi-max-frequency = <4000000>;
			};
		};
	};

	__overrides__ {
		int_pin = <&vhub>,"interrupts:0";
        cs =  <&vhub>,"reg:0";
		speed =  <&vhub>,"spi-max-frequency:0";
    };

};

};


@pelwell
Copy link
Contributor

pelwell commented Jan 26, 2021

Post the code you are using to configure the GPIO IRQ.

@Makurisan
Copy link
Author

Makurisan commented Jan 26, 2021

I have changed it to the above overlay and it is the same, means polling.

	vhub->irq = gpio_to_irq(vhub->gpio_irq);
	if (vhub->irq < 0) {
		dev_info(&vhub->spi->dev, "Failed to request spi irq:%d from device tree\n",
					vhub->irq);
	}
	else {
		dev_info(&vhub->spi->dev, "Succeeded to request gpio:%d with irq:%d\n", vhub->gpio_irq, vhub->irq);
	}
	// irq_set_irq_type(vhub->irq, IRQ_TYPE_LEVEL_HIGH);
	// irq_set_irq_type(vhub->irq, IRQ_TYPE_EDGE_FALLING );
	rc = devm_request_threaded_irq(&vhub->spi->dev, vhub->irq, NULL, ast_vhub_irq,
					IRQF_ONESHOT | IIRQF_TRIGGER_FALLING, KBUILD_MODNAME, vhub);
	if (rc)
	{
		dev_err(&vhub->spi->dev, "Failed to request interrupt\n");
		goto err;
	}

I also changed the interrupts to 0 in the overlay because the gpio irq comes from "gpio_to_irq(vhub->gpio_irq);".

interrupts = <0 2>; /* IRQ_TYPE_EDGE_FALLING */

@pelwell
Copy link
Contributor

pelwell commented Jan 26, 2021

How can you tell that it is polling?

@Makurisan
Copy link
Author

Makurisan commented Jan 26, 2021

If I load the kernel driver the irq handler is called immeadiately again and again. (and btw I return IRQ_HANDLED which means the irq is cleared)

Inside the irq handler I print a debug message with the value from the gpio "gpio_get_value(g_gpio_ip_irq)". The function gets the value "1" if I set the gpio to "1". Do I set the gpio to low the gpio in the handler has the value "0".

You see it is polling regardless what I set in the request_irq function.

@pelwell
Copy link
Contributor

pelwell commented Jan 26, 2021

There's a typo in your code (IIRQF_TRIGGER_FALLING) - are you sure you are building and running it?

Also, are you sure there is no glitching on the IRQ line?

Have a look at my gpio-fsm driver - drivers/gpio/gpio-fsm.c and arch/arm/boot/dts/overlays/fsm-demo-overlay.dts - for an example of code that uses edge triggering. In your situation I would add some debug code to that driver to confirm that the edge triggering is working as expected, then see how your code differs.

@Makurisan
Copy link
Author

Makurisan commented Jan 26, 2021

Yes, this was an copy,paste,rewrite problem.
I now changed to

rc = devm_request_irq(&vhub->spi->dev, vhub->irq, ast_vhub_irq, 0, KBUILD_MODNAME, vhub);
and in the overlay

interrupts = <17 2>; /* IRQ_TYPE_EDGE_FALLING */

The gpio is now from the overlay. The same result. always polling.

It is not the problem with device tree overlay. Any other idea?

@pelwell
Copy link
Contributor

pelwell commented Jan 26, 2021

As I said above, start with the known-good gpio-fsm driver, perhaps changing the configuration to use GPIO 0, and add enough debug to prove to yourself that edge triggering works. If it doesn't, I can test that configuration here.

@Makurisan
Copy link
Author

No, I dont do that. I have compiled our kernel with 5.4.83-v7+ on an rpi 3 and it works as expected.

It is a problem in your changes. Do it like I and you will see the problem.

@pelwell
Copy link
Contributor

pelwell commented Jan 27, 2021

You are asking me to investigate a problem that so far only you are seeing, with a non-standard driver and overlay. I've requested that you test with an (RPi-)standard driver and overlay - is that too much to ask?

@pelwell pelwell added the Waiting for external input Waiting for a comment from the originator of the issue, or a collaborator. label Jan 27, 2021
@Makurisan
Copy link
Author

Makurisan commented Jan 27, 2021

Ok, write exactly what I should do and I test it. This means write how to load the gpio-fsm driver with gpio 5 as INPUT irq.

I searched for the overlay and don*t know which one and how to use it.
Tried:
sudo modprobe gpio-fsm
sudo dtoverlay fsm-demo

The following error appears:
Of: /fsm-demo: could not get #gpio-cells' for /soc/interrupt controller...

on:
Linux pi 5.10.4-v8+ #1389 SMP PREEMPT Wed Jan 6 13:52:18 GMT 2021 aarch64 GNU/Linux

@Makurisan
Copy link
Author

Makurisan commented Feb 4, 2021

Is it possible that someone gives hints on this?

@pelwell
Copy link
Contributor

pelwell commented Feb 4, 2021

It's still on the list to look at. If you can provide a minimal test-case driver to demonstrate the problem that simply watches a GPIO line, where I can run a patch cable between it and another GPIO and trigger it with raspi-gpio (to avoid bounces) then it will get higher priority.

@Makurisan
Copy link
Author

The problem still exist but I made further investigation. The problem comes with the gpio irq function "devm_request_irq". For other reasons I switched to "devm_request_threaded_irq". This version works fine. The trigger comes only once.

@pelwell
Copy link
Contributor

pelwell commented Feb 18, 2021

If you are calling a function that sleeps in the IRQ handler then all sorts of things might go wrong - the threaded IRQ handler would avoid that.

@Makurisan
Copy link
Author

Yes, I read about this. An other point regarding the above DTs. How can I set pull down to the gpio 25 in the dts?

fragment@1 {
	target = <&spi0>;
    __overlay__ {
		/* needed to avoid dtc warning */
		#address-cells = <1>;
		#size-cells = <0>;
		status = "okay";
        vhub: vhub@48 {
			compatible = "hubshield,v-hub";
			reg = <0x0>;
			interrupt-parent = <&gpio>;
			interrupts = <0 2>; /* IRQ_TYPE_EDGE_FALLING */
            reset-gpios = <**&gpio 25 1**>; // GPIO_ACTIVE_HIGH
			gpio-controller;
			#gpio-cells = <2>;
			spi-max-frequency = <4000000>;
		};
	};
};

Thanks for your help.

@pelwell
Copy link
Contributor

pelwell commented Feb 18, 2021

Do you mean pull? Pull is normally redundant on an output GPIO - the drive of the pin overrides the pull. What do you think it will achieve?

That reset-gpios declaration looks wrong - the data doesn't match the comment; from the dt-bindings:

#define GPIO_ACTIVE_HIGH 0
#define GPIO_ACTIVE_LOW 1

If you do really want to set the pull for some reason then it will need a separate pinctrl declaration under the gpio node - I'll show you how once I know that you need it.

@Makurisan
Copy link
Author

Makurisan commented Feb 18, 2021

This is my actual DTS.

I want GPIO 5 as INPUT PullDown // this is per default OUT

Thanks

fragment@1 {
	target = <&spi0>;
    __overlay__ {
		/* needed to avoid dtc warning */
		#address-cells = <1>;
		#size-cells = <0>;
		status = "okay";
        vhub: vhub@48 {
			compatible = "hubshield,v-hub";
			reg = <0x0>;
			interrupt-parent = <&gpio>;
			interrupts = <5 2>; /* IRQ_TYPE_EDGE_FALLING */
    reset-gpios = <&gpio 25 1>; // GPIO_ACTIVE_HIGH
			max-ports = <5>; // count of virtual hub ports
			endpoints = <15>; // count of hub generic endpoints
			gpio-controller;
			#gpio-cells = <2>;
			spi-max-frequency = <1000000>;
		};
	};
};

__overrides__ {
	ports = <&vhub>,"max-ports:0";
	spi_irq = <&vhub>,"interrupts:0";
    spi_cs =  <&vhub>,"reg:0";
	speed =  <&vhub>,"spi-max-frequency:0";
};

@pelwell
Copy link
Contributor

pelwell commented Feb 18, 2021

You want a pull down on a falling edge IRQ on GPIO 5 (not 25, as you first said)? You'd normally use a pull up with a falling edge IRQ.

Do you have nothing to say about the reset GPIO?

@Makurisan
Copy link
Author

Makurisan commented Feb 19, 2021

The reset-pin is named for our devboard which is connected to the pi. If I know how it can be done for GPIO 5, I know it for GPIO 25.
Btw the GPIO5 is working. I set the pull down at the moment with raspi-gpio.

Please show how this can be configured with DTS.

@pelwell pelwell removed the Waiting for external input Waiting for a comment from the originator of the issue, or a collaborator. label Feb 19, 2021
@pelwell
Copy link
Contributor

pelwell commented Feb 19, 2021

This should work - we add a new fragment to add the pins definition under the gpio controller, then add pinctrl properties to the main driver node so that the pins are activated when the driver is probed:

fragment@1 {
	target = <&spi0>;
    __overlay__ {
		/* needed to avoid dtc warning */
		#address-cells = <1>;
		#size-cells = <0>;
		status = "okay";
        vhub: vhub@48 {
			compatible = "hubshield,v-hub";
			reg = <0x0>;
			interrupt-parent = <&gpio>;
			interrupts = <5 2>; /* IRQ_TYPE_EDGE_FALLING */
    reset-gpios = <&gpio 25 1>; // GPIO_ACTIVE_HIGH
			max-ports = <5>; // count of virtual hub ports
			endpoints = <15>; // count of hub generic endpoints
			gpio-controller;
			#gpio-cells = <2>;
			spi-max-frequency = <1000000>;
			pinctrl-names = "default";
			pinctrl-0 = <&vhub_pins>;
		};
	};
};

fragment@2 {
	target = <&gpio>;
	__overlay__ {
		vhub_pins: vhub_pins {
			brcm,pins = <5>;
			brcm,function = <0>; // input
			brcm,pull = <1>;     // down
		};
	};
};

You could have worked this out for yourself by reading the other overlays - all the source code is available to browse: https://github.com/raspberrypi/linux/tree/rpi-5.10.y/arch/arm/boot/dts/overlays

@Makurisan
Copy link
Author

Great! Sorry, can you show me how the pin 25 output, pullup must be referenced in fragment@2?

@pelwell
Copy link
Contributor

pelwell commented Feb 19, 2021

Just add a second pin, function and pull to the vhub_pins node. I'll do the first one for you:

brcm,pins = <5 25>;

An output is function 1, and a pull 2 is up.

@pelwell
Copy link
Contributor

pelwell commented Feb 19, 2021

But I still think setting a pull on an output is strange - I think you will find that making it an output early is enough so you can set the pull to 0 (none).

@Makurisan
Copy link
Author

Makurisan commented Feb 19, 2021

I do the following but the gpio 25 is IN. How can I check if something goes wrong?

__overlay__ {
		vhub_pins: vhub_pins {
			brcm,pins = <5 25>;	   		
			brcm,function = <0 1>; // input, output
			brcm,pull = <1 0>;     // down, none
		};
	};

@pelwell
Copy link
Contributor

pelwell commented Feb 19, 2021

Yes - that looks good.

@Makurisan
Copy link
Author

Makurisan commented Feb 19, 2021

Oh, you answered. How can I check if something goes wrong? dtc compile is correct.

pi@pi3:~/vusb » raspi-gpio get 25
GPIO 25: level=0 fsel=0 func=INPUT

@pelwell
Copy link
Contributor

pelwell commented Feb 19, 2021

Oh, you asked another question. Use sudo vcdbg log msg to look for overlay errors, and raspi-gpio get 0-27 to check the pin state.

@Makurisan
Copy link
Author

GPIO 25: level=0 fsel=0 func=INPUT

pi@pi3:~/vusb » sudo vcdbg log msg
001408.866: brfs: File read: /mfs/sd/config.txt
001409.898: brfs: File read: 1784 bytes
001479.408: brfs: File read: /mfs/sd/config.txt
001480.342: gpioman: gpioman_get_pin_num: pin LEDS_PWR_OK not defined
001499.999: brfs: File read: 1784 bytes
001691.071: gpioman: gpioman_get_pin_num: pin DISPLAY_DSI_PORT not defined
001692.399: gpioman: gpioman_get_pin_num: pin LEDS_PWR_OK not defined
001692.462: *** Restart logging
001711.771: HDMI0: hdmi_pixel_encoding: 162000000
001717.115: dtb_file 'bcm2710-rpi-3-b.dtb'
001723.196: brfs: File read: /mfs/sd/bcm2710-rpi-3-b.dtb
001723.219: Loading 'bcm2710-rpi-3-b.dtb' to 0x100 size 0x6e14
001736.617: brfs: File read: 28180 bytes
001749.257: brfs: File read: /mfs/sd/overlays/overlay_map.dtb
001814.796: brfs: File read: 1523 bytes
001819.589: brfs: File read: /mfs/sd/config.txt
001820.209: dtparam: audio=on
001836.414: brfs: File read: 1784 bytes
001838.813: brfs: File read: /mfs/sd/cmdline.txt
001838.874: Read command line from file 'cmdline.txt':
001838.898: 'console=serial0,115200 console=tty1 root=PARTUUID=4e17d452-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait quiet splash plymouth.ignore-serial-consoles'
003548.976: gpioman: gpioman_get_pin_num: pin EMMC_ENABLE not defined
003603.744: brfs: File read: 166 bytes
004035.373: brfs: File read: /mfs/sd/kernel7.img
004035.398: Loading 'kernel7.img' to 0x8000 size 0x6034a8
004035.430: Device tree loaded to 0x2eff8d00 (size 0x7288)
004040.062: gpioman: gpioman_get_pin_num: pin SDCARD_CONTROL_POWER not defined
007662.740: vchiq_core: vchiq_init_state: slot_zero = 0xf7580000, is_master = 1
007667.391: hdmi: HDMI:hdmi_get_state is deprecated, use hdmi_get_display_state instead
007673.228: TV service:host side not connected, dropping notification 0x00000002, 0x00000002, 0x

@Makurisan
Copy link
Author

Makurisan commented Feb 19, 2021

Modinfo has with some versions this plus sign in the name:
vermagic: 5.10.11-v7+
What does this mean?

@pelwell
Copy link
Contributor

pelwell commented Feb 19, 2021

The overlay doesn't appear in the firmware log, so you must be loading it with the dtoverlay command under Linux? You might find some messages in the kernel log - dmesg - so if you're loading the overlay from the command line then:

$ sudo dmesg -C                # to clear out the old messages
$ sudo dtoverlay your-overlay
$ dmesg                        # to show any new messages

The + just means there are downstream changes. As long as it matches uname -r it's all good.

@Makurisan
Copy link
Author

Makurisan commented Feb 19, 2021

I know, because I print some messages:

  • hub port, spi clock

But now I get this overlay: WARNING. This is new. It seems there is a problem with fragment@2?

Is the order in the dts correct? this second overrides?

[Feb19 16:03] v_hub: loading out-of-tree module taints kernel.
[Feb19 16:04] OF: overlay: WARNING: memory leak will occur if overlay removed, property: /soc/spi@7e204000/spidev@0/status
[  +0.000047] OF: overlay: WARNING: memory leak will occur if overlay removed, property: /soc/spi@7e204000/status
[  +0.038799] v_hub spi0.0: Hub device has initiated 5 hub ports.
[  +0.000055] v_hub spi0.0: Spi clock set at 1000 KHz.
/*
 * Universal device tree overlay for SPI devices
 */

/dts-v1/;
/plugin/;

/*
#include <include/dt-bindings/interrupt-controller/irq.h>
 */

/ {
	compatible = "brcm,bcm2711";

	fragment@0 {
		target = <&spidev0>;
		__overlay__ {
			status = "disabled";
		};
	};

	fragment@1 {
		target = <&spi0>;
        __overlay__ {
			/* needed to avoid dtc warning */
    		#address-cells = <1>;
			#size-cells = <0>;
 			status = "okay";
      vhub: vhub@48 {
				compatible = "hubshield,v-hub";
				reg = <0x0>;
				interrupt-parent = <&gpio>;
				interrupts = <5 2>; /* IRQ_TYPE_EDGE_FALLING */
        reset-gpios = <&gpio 25 1>; // GPIO_ACTIVE_HIGH
				max-ports = <5>; // count of virtual hub ports
				endpoints = <15>; // count of hub generic endpoints
				gpio-controller;
				#gpio-cells = <2>;
				spi-max-frequency = <1000000>;
			};
		};
	};
	fragment@2 {
		target = <&gpio>;
		__overlay__ {
			vhub_pins: vhub_pins {
				brcm,pins = <5 25>;	   		
				brcm,function = <0 1>; // input, output
				brcm,pull = <1 0>;     // down, none
			};
		};
	};
	__overrides__ {
		ports = <&vhub>,"max-ports:0";
		spi_irq = <&vhub>,"interrupts:0";
    spi_cs =  <&vhub>,"reg:0";
		speed =  <&vhub>,"spi-max-frequency:0";
    };

};

@Makurisan
Copy link
Author

I see something. The chip is bcm2711 but the pi is Pi3 B. Can this be the cause?
If I want the dts for 3, 4 pi is BCM2837 correct?

@pelwell
Copy link
Contributor

pelwell commented Feb 19, 2021

For a generic overlay (not BCM2711-specific) use brcm,bcm2835.

The warnings about leaks are harmless and unavoidable.

@Makurisan
Copy link
Author

I checked some dts. Is a line like this
pinctrl-0 = <&vhub_pins>;
necessary because this is not in the dts?

@pelwell
Copy link
Contributor

pelwell commented Feb 19, 2021

Fragment 2 puts vhub_pins in the dts. The pinctrl-0 line is necessary if you want the GPIOs configured automatically with pulls etc.

@Makurisan
Copy link
Author

Makurisan commented Feb 19, 2021

Thanks, it works now.

@Makurisan
Copy link
Author

Makurisan commented Feb 23, 2021

I dont know what happens but spi isn't working and it is not clear why.
If I list the active spi devices with "ls -l /dev/spi*":
crw-rw---- 1 root spi 153, 0 Feb 23 08:18 /dev/spidev0.1
The device with which I will working is "spi0.0".
What must appear in the dev list, if I overload the spidev0 with the above DTS?

@pelwell
Copy link
Contributor

pelwell commented Feb 23, 2021

The /dev/spidev* devices are created by the spidev driver - with spidev on 0.0 disabled there will be no /dev/spidev0.0. You might find an entry for your device in /sys/class/spi_master or /sys/bus/spi, otherwise you need to look for device-specific evidence; for example, an enc28j60 will appear as a network interface in ifconfig (probably eth1).

@Makurisan
Copy link
Author

Makurisan commented Feb 23, 2021

Ah, yes my device is under:
ls /sys/class/spi_master/spi0/spi0.0

pi@pi3:~/spidev-test(master○) » ls /sys/class/spi_master/spi0/spi0.0
driver modalias power spi0.0:p2 spi0.0:p4 statistics uevent
driver_override of_node spi0.0:p1 spi0.0:p3 spi0.0:p5 subsystem

I create a udc and I have 5 spi0.0:p1 ... p5
Do you know what here is now the device because I want to loopback miso and mosi but the spidev-test needs the device?

@pelwell
Copy link
Contributor

pelwell commented Feb 23, 2021

Do you know what here is now the device because I want to loopback miso and mosi but the spidev-test needs the device?

I'm not sure that I understand your question - can you reword it? - but it isn't possible to use spidev and a dedicated driver with the same SPI device/CS. If you want to use spidev you'll have to disable your device node and create an spidev node instead.

N.B. This "Issues" section is for reporting bugs in our kernels. I think you've got past that stage now.

@pelwell pelwell closed this as completed Feb 23, 2021
@Makurisan
Copy link
Author

thanks for your help.

popcornmix pushed a commit that referenced this issue Oct 4, 2024
[ Upstream commit 89a906d ]

Floating point instructions in userspace can crash some arm kernels
built with clang/LLD 17.0.6:

    BUG: unsupported FP instruction in kernel mode
    FPEXC == 0xc0000780
    Internal error: Oops - undefined instruction: 0 [#1] ARM
    CPU: 0 PID: 196 Comm: vfp-reproducer Not tainted 6.10.0 #1
    Hardware name: BCM2835
    PC is at vfp_support_entry+0xc8/0x2cc
    LR is at do_undefinstr+0xa8/0x250
    pc : [<c0101d50>]    lr : [<c010a80c>]    psr: a0000013
    sp : dc8d1f68  ip : 60000013  fp : bedea19c
    r10: ec532b17  r9 : 00000010  r8 : 0044766c
    r7 : c0000780  r6 : ec532b17  r5 : c1c13800  r4 : dc8d1fb0
    r3 : c10072c4  r2 : c0101c88  r1 : ec532b17  r0 : 0044766c
    Flags: NzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
    Control: 00c5387d  Table: 0251c008  DAC: 00000051
    Register r0 information: non-paged memory
    Register r1 information: vmalloc memory
    Register r2 information: non-slab/vmalloc memory
    Register r3 information: non-slab/vmalloc memory
    Register r4 information: 2-page vmalloc region
    Register r5 information: slab kmalloc-cg-2k
    Register r6 information: vmalloc memory
    Register r7 information: non-slab/vmalloc memory
    Register r8 information: non-paged memory
    Register r9 information: zero-size pointer
    Register r10 information: vmalloc memory
    Register r11 information: non-paged memory
    Register r12 information: non-paged memory
    Process vfp-reproducer (pid: 196, stack limit = 0x61aaaf8b)
    Stack: (0xdc8d1f68 to 0xdc8d2000)
    1f60:                   0000081f b6f69300 0000000f c10073f4 c10072c4 dc8d1fb0
    1f80: ec532b17 0c532b17 0044766c b6f9ccd8 00000000 c010a80c 00447670 60000010
    1fa0: ffffffff c1c13800 00c5387d c0100f10 b6f68af8 00448fc0 00000000 bedea188
    1fc0: bedea314 00000001 00448ebc b6f9d000 00447608 b6f9ccd8 00000000 bedea19c
    1fe0: bede9198 bedea188 b6e1061c 0044766c 60000010 ffffffff 00000000 00000000
    Call trace:
    [<c0101d50>] (vfp_support_entry) from [<c010a80c>] (do_undefinstr+0xa8/0x250)
    [<c010a80c>] (do_undefinstr) from [<c0100f10>] (__und_usr+0x70/0x80)
    Exception stack(0xdc8d1fb0 to 0xdc8d1ff8)
    1fa0:                                     b6f68af8 00448fc0 00000000 bedea188
    1fc0: bedea314 00000001 00448ebc b6f9d000 00447608 b6f9ccd8 00000000 bedea19c
    1fe0: bede9198 bedea188 b6e1061c 0044766c 60000010 ffffffff
    Code: 0a000061 e3877202 e594003c e3a09010 (eef16a10)
    ---[ end trace 0000000000000000 ]---
    Kernel panic - not syncing: Fatal exception in interrupt
    ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---

This is a minimal userspace reproducer on a Raspberry Pi Zero W:

    #include <stdio.h>
    #include <math.h>

    int main(void)
    {
            double v = 1.0;
            printf("%fn", NAN + *(volatile double *)&v);
            return 0;
    }

Another way to consistently trigger the oops is:

    calvin@raspberry-pi-zero-w ~$ python -c "import json"

The bug reproduces only when the kernel is built with DYNAMIC_DEBUG=n,
because the pr_debug() calls act as barriers even when not activated.

This is the output from the same kernel source built with the same
compiler and DYNAMIC_DEBUG=y, where the userspace reproducer works as
expected:

    VFP: bounce: trigger ec532b17 fpexc c0000780
    VFP: emulate: INST=0xee377b06 SCR=0x00000000
    VFP: bounce: trigger eef1fa10 fpexc c0000780
    VFP: emulate: INST=0xeeb40b40 SCR=0x00000000
    VFP: raising exceptions 30000000

    calvin@raspberry-pi-zero-w ~$ ./vfp-reproducer
    nan

Crudely grepping for vmsr/vmrs instructions in the otherwise nearly
idential text for vfp_support_entry() makes the problem obvious:

    vmlinux.llvm.good [0xc0101cb8] <+48>:  vmrs   r7, fpexc
    vmlinux.llvm.good [0xc0101cd8] <+80>:  vmsr   fpexc, r0
    vmlinux.llvm.good [0xc0101d20] <+152>: vmsr   fpexc, r7
    vmlinux.llvm.good [0xc0101d38] <+176>: vmrs   r4, fpexc
    vmlinux.llvm.good [0xc0101d6c] <+228>: vmrs   r0, fpscr
    vmlinux.llvm.good [0xc0101dc4] <+316>: vmsr   fpexc, r0
    vmlinux.llvm.good [0xc0101dc8] <+320>: vmrs   r0, fpsid
    vmlinux.llvm.good [0xc0101dcc] <+324>: vmrs   r6, fpscr
    vmlinux.llvm.good [0xc0101e10] <+392>: vmrs   r10, fpinst
    vmlinux.llvm.good [0xc0101eb8] <+560>: vmrs   r10, fpinst2

    vmlinux.llvm.bad  [0xc0101cb8] <+48>:  vmrs   r7, fpexc
    vmlinux.llvm.bad  [0xc0101cd8] <+80>:  vmsr   fpexc, r0
    vmlinux.llvm.bad  [0xc0101d20] <+152>: vmsr   fpexc, r7
    vmlinux.llvm.bad  [0xc0101d30] <+168>: vmrs   r0, fpscr
    vmlinux.llvm.bad  [0xc0101d50] <+200>: vmrs   r6, fpscr  <== BOOM!
    vmlinux.llvm.bad  [0xc0101d6c] <+228>: vmsr   fpexc, r0
    vmlinux.llvm.bad  [0xc0101d70] <+232>: vmrs   r0, fpsid
    vmlinux.llvm.bad  [0xc0101da4] <+284>: vmrs   r10, fpinst
    vmlinux.llvm.bad  [0xc0101df8] <+368>: vmrs   r4, fpexc
    vmlinux.llvm.bad  [0xc0101e5c] <+468>: vmrs   r10, fpinst2

I think LLVM's reordering is valid as the code is currently written: the
compiler doesn't know the instructions have side effects in hardware.

Fix by using "asm volatile" in fmxr() and fmrx(), so they cannot be
reordered with respect to each other. The original compiler now produces
working kernels on my hardware with DYNAMIC_DEBUG=n.

This is the relevant piece of the diff of the vfp_support_entry() text,
from the original oopsing kernel to a working kernel with this patch:

         vmrs r0, fpscr
         tst r0, #4096
         bne 0xc0101d48
         tst r0, #458752
         beq 0xc0101ecc
         orr r7, r7, #536870912
         ldr r0, [r4, #0x3c]
         mov r9, #16
        -vmrs r6, fpscr
         orr r9, r9, #251658240
         add r0, r0, #4
         str r0, [r4, #0x3c]
         mvn r0, #159
         sub r0, r0, #-1207959552
         and r0, r7, r0
         vmsr fpexc, r0
         vmrs r0, fpsid
        +vmrs r6, fpscr
         and r0, r0, #983040
         cmp r0, #65536
         bne 0xc0101d88

Fixes: 4708fb0 ("ARM: vfp: Reimplement VFP exception entry in C code")
Signed-off-by: Calvin Owens <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>
Signed-off-by: Sasha Levin <[email protected]>
popcornmix pushed a commit that referenced this issue Oct 10, 2024
[ Upstream commit 89a906d ]

Floating point instructions in userspace can crash some arm kernels
built with clang/LLD 17.0.6:

    BUG: unsupported FP instruction in kernel mode
    FPEXC == 0xc0000780
    Internal error: Oops - undefined instruction: 0 [#1] ARM
    CPU: 0 PID: 196 Comm: vfp-reproducer Not tainted 6.10.0 #1
    Hardware name: BCM2835
    PC is at vfp_support_entry+0xc8/0x2cc
    LR is at do_undefinstr+0xa8/0x250
    pc : [<c0101d50>]    lr : [<c010a80c>]    psr: a0000013
    sp : dc8d1f68  ip : 60000013  fp : bedea19c
    r10: ec532b17  r9 : 00000010  r8 : 0044766c
    r7 : c0000780  r6 : ec532b17  r5 : c1c13800  r4 : dc8d1fb0
    r3 : c10072c4  r2 : c0101c88  r1 : ec532b17  r0 : 0044766c
    Flags: NzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
    Control: 00c5387d  Table: 0251c008  DAC: 00000051
    Register r0 information: non-paged memory
    Register r1 information: vmalloc memory
    Register r2 information: non-slab/vmalloc memory
    Register r3 information: non-slab/vmalloc memory
    Register r4 information: 2-page vmalloc region
    Register r5 information: slab kmalloc-cg-2k
    Register r6 information: vmalloc memory
    Register r7 information: non-slab/vmalloc memory
    Register r8 information: non-paged memory
    Register r9 information: zero-size pointer
    Register r10 information: vmalloc memory
    Register r11 information: non-paged memory
    Register r12 information: non-paged memory
    Process vfp-reproducer (pid: 196, stack limit = 0x61aaaf8b)
    Stack: (0xdc8d1f68 to 0xdc8d2000)
    1f60:                   0000081f b6f69300 0000000f c10073f4 c10072c4 dc8d1fb0
    1f80: ec532b17 0c532b17 0044766c b6f9ccd8 00000000 c010a80c 00447670 60000010
    1fa0: ffffffff c1c13800 00c5387d c0100f10 b6f68af8 00448fc0 00000000 bedea188
    1fc0: bedea314 00000001 00448ebc b6f9d000 00447608 b6f9ccd8 00000000 bedea19c
    1fe0: bede9198 bedea188 b6e1061c 0044766c 60000010 ffffffff 00000000 00000000
    Call trace:
    [<c0101d50>] (vfp_support_entry) from [<c010a80c>] (do_undefinstr+0xa8/0x250)
    [<c010a80c>] (do_undefinstr) from [<c0100f10>] (__und_usr+0x70/0x80)
    Exception stack(0xdc8d1fb0 to 0xdc8d1ff8)
    1fa0:                                     b6f68af8 00448fc0 00000000 bedea188
    1fc0: bedea314 00000001 00448ebc b6f9d000 00447608 b6f9ccd8 00000000 bedea19c
    1fe0: bede9198 bedea188 b6e1061c 0044766c 60000010 ffffffff
    Code: 0a000061 e3877202 e594003c e3a09010 (eef16a10)
    ---[ end trace 0000000000000000 ]---
    Kernel panic - not syncing: Fatal exception in interrupt
    ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---

This is a minimal userspace reproducer on a Raspberry Pi Zero W:

    #include <stdio.h>
    #include <math.h>

    int main(void)
    {
            double v = 1.0;
            printf("%fn", NAN + *(volatile double *)&v);
            return 0;
    }

Another way to consistently trigger the oops is:

    calvin@raspberry-pi-zero-w ~$ python -c "import json"

The bug reproduces only when the kernel is built with DYNAMIC_DEBUG=n,
because the pr_debug() calls act as barriers even when not activated.

This is the output from the same kernel source built with the same
compiler and DYNAMIC_DEBUG=y, where the userspace reproducer works as
expected:

    VFP: bounce: trigger ec532b17 fpexc c0000780
    VFP: emulate: INST=0xee377b06 SCR=0x00000000
    VFP: bounce: trigger eef1fa10 fpexc c0000780
    VFP: emulate: INST=0xeeb40b40 SCR=0x00000000
    VFP: raising exceptions 30000000

    calvin@raspberry-pi-zero-w ~$ ./vfp-reproducer
    nan

Crudely grepping for vmsr/vmrs instructions in the otherwise nearly
idential text for vfp_support_entry() makes the problem obvious:

    vmlinux.llvm.good [0xc0101cb8] <+48>:  vmrs   r7, fpexc
    vmlinux.llvm.good [0xc0101cd8] <+80>:  vmsr   fpexc, r0
    vmlinux.llvm.good [0xc0101d20] <+152>: vmsr   fpexc, r7
    vmlinux.llvm.good [0xc0101d38] <+176>: vmrs   r4, fpexc
    vmlinux.llvm.good [0xc0101d6c] <+228>: vmrs   r0, fpscr
    vmlinux.llvm.good [0xc0101dc4] <+316>: vmsr   fpexc, r0
    vmlinux.llvm.good [0xc0101dc8] <+320>: vmrs   r0, fpsid
    vmlinux.llvm.good [0xc0101dcc] <+324>: vmrs   r6, fpscr
    vmlinux.llvm.good [0xc0101e10] <+392>: vmrs   r10, fpinst
    vmlinux.llvm.good [0xc0101eb8] <+560>: vmrs   r10, fpinst2

    vmlinux.llvm.bad  [0xc0101cb8] <+48>:  vmrs   r7, fpexc
    vmlinux.llvm.bad  [0xc0101cd8] <+80>:  vmsr   fpexc, r0
    vmlinux.llvm.bad  [0xc0101d20] <+152>: vmsr   fpexc, r7
    vmlinux.llvm.bad  [0xc0101d30] <+168>: vmrs   r0, fpscr
    vmlinux.llvm.bad  [0xc0101d50] <+200>: vmrs   r6, fpscr  <== BOOM!
    vmlinux.llvm.bad  [0xc0101d6c] <+228>: vmsr   fpexc, r0
    vmlinux.llvm.bad  [0xc0101d70] <+232>: vmrs   r0, fpsid
    vmlinux.llvm.bad  [0xc0101da4] <+284>: vmrs   r10, fpinst
    vmlinux.llvm.bad  [0xc0101df8] <+368>: vmrs   r4, fpexc
    vmlinux.llvm.bad  [0xc0101e5c] <+468>: vmrs   r10, fpinst2

I think LLVM's reordering is valid as the code is currently written: the
compiler doesn't know the instructions have side effects in hardware.

Fix by using "asm volatile" in fmxr() and fmrx(), so they cannot be
reordered with respect to each other. The original compiler now produces
working kernels on my hardware with DYNAMIC_DEBUG=n.

This is the relevant piece of the diff of the vfp_support_entry() text,
from the original oopsing kernel to a working kernel with this patch:

         vmrs r0, fpscr
         tst r0, #4096
         bne 0xc0101d48
         tst r0, #458752
         beq 0xc0101ecc
         orr r7, r7, #536870912
         ldr r0, [r4, #0x3c]
         mov r9, #16
        -vmrs r6, fpscr
         orr r9, r9, #251658240
         add r0, r0, #4
         str r0, [r4, #0x3c]
         mvn r0, #159
         sub r0, r0, #-1207959552
         and r0, r7, r0
         vmsr fpexc, r0
         vmrs r0, fpsid
        +vmrs r6, fpscr
         and r0, r0, #983040
         cmp r0, #65536
         bne 0xc0101d88

Fixes: 4708fb0 ("ARM: vfp: Reimplement VFP exception entry in C code")
Signed-off-by: Calvin Owens <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>
Signed-off-by: Sasha Levin <[email protected]>
popcornmix pushed a commit that referenced this issue Oct 10, 2024
[ Upstream commit 89a906d ]

Floating point instructions in userspace can crash some arm kernels
built with clang/LLD 17.0.6:

    BUG: unsupported FP instruction in kernel mode
    FPEXC == 0xc0000780
    Internal error: Oops - undefined instruction: 0 [#1] ARM
    CPU: 0 PID: 196 Comm: vfp-reproducer Not tainted 6.10.0 #1
    Hardware name: BCM2835
    PC is at vfp_support_entry+0xc8/0x2cc
    LR is at do_undefinstr+0xa8/0x250
    pc : [<c0101d50>]    lr : [<c010a80c>]    psr: a0000013
    sp : dc8d1f68  ip : 60000013  fp : bedea19c
    r10: ec532b17  r9 : 00000010  r8 : 0044766c
    r7 : c0000780  r6 : ec532b17  r5 : c1c13800  r4 : dc8d1fb0
    r3 : c10072c4  r2 : c0101c88  r1 : ec532b17  r0 : 0044766c
    Flags: NzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
    Control: 00c5387d  Table: 0251c008  DAC: 00000051
    Register r0 information: non-paged memory
    Register r1 information: vmalloc memory
    Register r2 information: non-slab/vmalloc memory
    Register r3 information: non-slab/vmalloc memory
    Register r4 information: 2-page vmalloc region
    Register r5 information: slab kmalloc-cg-2k
    Register r6 information: vmalloc memory
    Register r7 information: non-slab/vmalloc memory
    Register r8 information: non-paged memory
    Register r9 information: zero-size pointer
    Register r10 information: vmalloc memory
    Register r11 information: non-paged memory
    Register r12 information: non-paged memory
    Process vfp-reproducer (pid: 196, stack limit = 0x61aaaf8b)
    Stack: (0xdc8d1f68 to 0xdc8d2000)
    1f60:                   0000081f b6f69300 0000000f c10073f4 c10072c4 dc8d1fb0
    1f80: ec532b17 0c532b17 0044766c b6f9ccd8 00000000 c010a80c 00447670 60000010
    1fa0: ffffffff c1c13800 00c5387d c0100f10 b6f68af8 00448fc0 00000000 bedea188
    1fc0: bedea314 00000001 00448ebc b6f9d000 00447608 b6f9ccd8 00000000 bedea19c
    1fe0: bede9198 bedea188 b6e1061c 0044766c 60000010 ffffffff 00000000 00000000
    Call trace:
    [<c0101d50>] (vfp_support_entry) from [<c010a80c>] (do_undefinstr+0xa8/0x250)
    [<c010a80c>] (do_undefinstr) from [<c0100f10>] (__und_usr+0x70/0x80)
    Exception stack(0xdc8d1fb0 to 0xdc8d1ff8)
    1fa0:                                     b6f68af8 00448fc0 00000000 bedea188
    1fc0: bedea314 00000001 00448ebc b6f9d000 00447608 b6f9ccd8 00000000 bedea19c
    1fe0: bede9198 bedea188 b6e1061c 0044766c 60000010 ffffffff
    Code: 0a000061 e3877202 e594003c e3a09010 (eef16a10)
    ---[ end trace 0000000000000000 ]---
    Kernel panic - not syncing: Fatal exception in interrupt
    ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---

This is a minimal userspace reproducer on a Raspberry Pi Zero W:

    #include <stdio.h>
    #include <math.h>

    int main(void)
    {
            double v = 1.0;
            printf("%fn", NAN + *(volatile double *)&v);
            return 0;
    }

Another way to consistently trigger the oops is:

    calvin@raspberry-pi-zero-w ~$ python -c "import json"

The bug reproduces only when the kernel is built with DYNAMIC_DEBUG=n,
because the pr_debug() calls act as barriers even when not activated.

This is the output from the same kernel source built with the same
compiler and DYNAMIC_DEBUG=y, where the userspace reproducer works as
expected:

    VFP: bounce: trigger ec532b17 fpexc c0000780
    VFP: emulate: INST=0xee377b06 SCR=0x00000000
    VFP: bounce: trigger eef1fa10 fpexc c0000780
    VFP: emulate: INST=0xeeb40b40 SCR=0x00000000
    VFP: raising exceptions 30000000

    calvin@raspberry-pi-zero-w ~$ ./vfp-reproducer
    nan

Crudely grepping for vmsr/vmrs instructions in the otherwise nearly
idential text for vfp_support_entry() makes the problem obvious:

    vmlinux.llvm.good [0xc0101cb8] <+48>:  vmrs   r7, fpexc
    vmlinux.llvm.good [0xc0101cd8] <+80>:  vmsr   fpexc, r0
    vmlinux.llvm.good [0xc0101d20] <+152>: vmsr   fpexc, r7
    vmlinux.llvm.good [0xc0101d38] <+176>: vmrs   r4, fpexc
    vmlinux.llvm.good [0xc0101d6c] <+228>: vmrs   r0, fpscr
    vmlinux.llvm.good [0xc0101dc4] <+316>: vmsr   fpexc, r0
    vmlinux.llvm.good [0xc0101dc8] <+320>: vmrs   r0, fpsid
    vmlinux.llvm.good [0xc0101dcc] <+324>: vmrs   r6, fpscr
    vmlinux.llvm.good [0xc0101e10] <+392>: vmrs   r10, fpinst
    vmlinux.llvm.good [0xc0101eb8] <+560>: vmrs   r10, fpinst2

    vmlinux.llvm.bad  [0xc0101cb8] <+48>:  vmrs   r7, fpexc
    vmlinux.llvm.bad  [0xc0101cd8] <+80>:  vmsr   fpexc, r0
    vmlinux.llvm.bad  [0xc0101d20] <+152>: vmsr   fpexc, r7
    vmlinux.llvm.bad  [0xc0101d30] <+168>: vmrs   r0, fpscr
    vmlinux.llvm.bad  [0xc0101d50] <+200>: vmrs   r6, fpscr  <== BOOM!
    vmlinux.llvm.bad  [0xc0101d6c] <+228>: vmsr   fpexc, r0
    vmlinux.llvm.bad  [0xc0101d70] <+232>: vmrs   r0, fpsid
    vmlinux.llvm.bad  [0xc0101da4] <+284>: vmrs   r10, fpinst
    vmlinux.llvm.bad  [0xc0101df8] <+368>: vmrs   r4, fpexc
    vmlinux.llvm.bad  [0xc0101e5c] <+468>: vmrs   r10, fpinst2

I think LLVM's reordering is valid as the code is currently written: the
compiler doesn't know the instructions have side effects in hardware.

Fix by using "asm volatile" in fmxr() and fmrx(), so they cannot be
reordered with respect to each other. The original compiler now produces
working kernels on my hardware with DYNAMIC_DEBUG=n.

This is the relevant piece of the diff of the vfp_support_entry() text,
from the original oopsing kernel to a working kernel with this patch:

         vmrs r0, fpscr
         tst r0, #4096
         bne 0xc0101d48
         tst r0, #458752
         beq 0xc0101ecc
         orr r7, r7, #536870912
         ldr r0, [r4, #0x3c]
         mov r9, #16
        -vmrs r6, fpscr
         orr r9, r9, #251658240
         add r0, r0, #4
         str r0, [r4, #0x3c]
         mvn r0, #159
         sub r0, r0, #-1207959552
         and r0, r7, r0
         vmsr fpexc, r0
         vmrs r0, fpsid
        +vmrs r6, fpscr
         and r0, r0, #983040
         cmp r0, #65536
         bne 0xc0101d88

Fixes: 4708fb0 ("ARM: vfp: Reimplement VFP exception entry in C code")
Signed-off-by: Calvin Owens <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>
Signed-off-by: Sasha Levin <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants