[gs]etsockopt per subflow: BPF #76

matttbe · 2020-08-07T20:58:31Z

@nrybowski is working on it

A short documentation (in the wiki) could be useful!

[BUG] Unmounting a btrfs filesystem with quota disabled will cause the following NULL pointer dereference: BTRFS info (device dm-5): has skinny extents BUG: kernel NULL pointer dereference, address: 0000000000000018 #PF: supervisor read access in kernel mode #PF: error_code(0x0000) - not-present page CPU: 7 PID: 637 Comm: umount Not tainted 5.8.0-rc7-next-20200731-custom #76 RIP: 0010:kobject_del+0x6/0x20 Call Trace: btrfs_sysfs_del_qgroups+0xac/0xf0 [btrfs] btrfs_free_qgroup_config+0x63/0x70 [btrfs] close_ctree+0x1f5/0x323 [btrfs] btrfs_put_super+0x15/0x17 [btrfs] generic_shutdown_super+0x72/0x110 kill_anon_super+0x18/0x30 btrfs_kill_super+0x17/0x30 [btrfs] deactivate_locked_super+0x3b/0xa0 deactivate_super+0x40/0x50 cleanup_mnt+0x135/0x190 __cleanup_mnt+0x12/0x20 task_work_run+0x64/0xb0 exit_to_user_mode_prepare+0x18a/0x190 syscall_exit_to_user_mode+0x4f/0x270 do_syscall_64+0x45/0x50 entry_SYSCALL_64_after_hwframe+0x44/0xa9 ---[ end trace 37b7adca5c1d5c5d ]--- [CAUSE] Commit 079ad2f ("kobject: Avoid premature parent object freeing in kobject_cleanup()") changed kobject_del() that it no longer accepts NULL pointer. Before that commit, kobject_del() and kobject_put() all accept NULL pointers and just ignore such NULL pointers. But that mentioned commit needs to access the parent node, killing the old NULL pointer behavior. Unfortunately btrfs is relying on that hidden feature thus we will trigger such NULL pointer dereference. [FIX] Instead of just saving several lines, do proper fs_info->qgroups_kobj check before calling kobject_del() and kobject_put(). Reviewed-by: Nikolay Borisov <[email protected]> Signed-off-by: Qu Wenruo <[email protected]> Reviewed-by: David Sterba <[email protected]> Signed-off-by: David Sterba <[email protected]>

matttbe · 2021-02-25T09:17:39Z

Unassigning myself (in fact more Nicolas) just in case someone else wants to look at it. Nobody here is actively looking at that one for the moment.

In the case of MDIO bus registration failure due to no external PHY devices is connected to the MAC, clk_disable_unprepare() is called in stmmac_bus_clk_config() and intel_eth_pci_probe() respectively. The second call in intel_eth_pci_probe() will caused the following:- [ 16.578605] intel-eth-pci 0000:00:1e.5: No PHY found [ 16.583778] intel-eth-pci 0000:00:1e.5: stmmac_dvr_probe: MDIO bus (id: 2) registration failed [ 16.680181] ------------[ cut here ]------------ [ 16.684861] stmmac-0000:00:1e.5 already disabled [ 16.689547] WARNING: CPU: 13 PID: 2053 at drivers/clk/clk.c:952 clk_core_disable+0x96/0x1b0 [ 16.697963] Modules linked in: dwc3 iTCO_wdt mei_hdcp iTCO_vendor_support udc_core x86_pkg_temp_thermal kvm_intel marvell10g kvm sch_fq_codel nfsd irqbypass dwmac_intel(+) stmmac uio ax88179_178a pcs_xpcs phylink uhid spi_pxa2xx_platform usbnet mei_me pcspkr tpm_crb mii i2c_i801 dw_dmac dwc3_pci thermal dw_dmac_core intel_rapl_msr libphy i2c_smbus mei tpm_tis intel_th_gth tpm_tis_core tpm intel_th_acpi intel_pmc_core intel_th i915 fuse configfs snd_hda_intel snd_intel_dspcfg snd_intel_sdw_acpi snd_hda_codec snd_hda_core snd_pcm snd_timer snd soundcore [ 16.746785] CPU: 13 PID: 2053 Comm: systemd-udevd Tainted: G U 5.13.0-rc3-intel-lts #76 [ 16.756134] Hardware name: Intel Corporation Alder Lake Client Platform/AlderLake-S ADP-S DRR4 CRB, BIOS ADLIFSI1.R00.1494.B00.2012031421 12/03/2020 [ 16.769465] RIP: 0010:clk_core_disable+0x96/0x1b0 [ 16.774222] Code: 00 8b 05 45 96 17 01 85 c0 7f 24 48 8b 5b 30 48 85 db 74 a5 8b 43 7c 85 c0 75 93 48 8b 33 48 c7 c7 6e 32 cc b7 e8 b2 5d 52 00 <0f> 0b 5b 5d c3 65 8b 05 76 31 18 49 89 c0 48 0f a3 05 bc 92 1a 01 [ 16.793016] RSP: 0018:ffffa44580523aa0 EFLAGS: 00010086 [ 16.798287] RAX: 0000000000000000 RBX: ffff8d7d0eb70a00 RCX: 0000000000000000 [ 16.805435] RDX: 0000000000000002 RSI: ffffffffb7c62d5f RDI: 00000000ffffffff [ 16.812610] RBP: 0000000000000287 R08: 0000000000000000 R09: ffffa445805238d0 [ 16.819759] R10: 0000000000000001 R11: 0000000000000001 R12: ffff8d7d0eb70a00 [ 16.826904] R13: ffff8d7d027370c8 R14: 0000000000000006 R15: ffffa44580523ad0 [ 16.834047] FS: 00007f9882fa2600(0000) GS:ffff8d80a0940000(0000) knlGS:0000000000000000 [ 16.842177] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 16.847966] CR2: 00007f9882bea3d8 CR3: 000000010b126001 CR4: 0000000000370ee0 [ 16.855144] Call Trace: [ 16.857614] clk_core_disable_lock+0x1b/0x30 [ 16.861941] intel_eth_pci_probe.cold+0x11d/0x136 [dwmac_intel] [ 16.867913] pci_device_probe+0xcf/0x150 [ 16.871890] really_probe+0xf5/0x3e0 [ 16.875526] driver_probe_device+0x64/0x150 [ 16.879763] device_driver_attach+0x53/0x60 [ 16.883998] __driver_attach+0x9f/0x150 [ 16.887883] ? device_driver_attach+0x60/0x60 [ 16.892288] ? device_driver_attach+0x60/0x60 [ 16.896698] bus_for_each_dev+0x77/0xc0 [ 16.900583] bus_add_driver+0x184/0x1f0 [ 16.904469] driver_register+0x6c/0xc0 [ 16.908268] ? 0xffffffffc07ae000 [ 16.911598] do_one_initcall+0x4a/0x210 [ 16.915489] ? kmem_cache_alloc_trace+0x305/0x4e0 [ 16.920247] do_init_module+0x5c/0x230 [ 16.924057] load_module+0x2894/0x2b70 [ 16.927857] ? __do_sys_finit_module+0xb5/0x120 [ 16.932441] __do_sys_finit_module+0xb5/0x120 [ 16.936845] do_syscall_64+0x42/0x80 [ 16.940476] entry_SYSCALL_64_after_hwframe+0x44/0xae [ 16.945586] RIP: 0033:0x7f98830e5ccd [ 16.949177] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 93 31 0c 00 f7 d8 64 89 01 48 [ 16.967970] RSP: 002b:00007ffc66b60168 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 16.975583] RAX: ffffffffffffffda RBX: 000055885de35ef0 RCX: 00007f98830e5ccd [ 16.982725] RDX: 0000000000000000 RSI: 00007f98832541e3 RDI: 0000000000000012 [ 16.989868] RBP: 0000000000020000 R08: 0000000000000000 R09: 0000000000000000 [ 16.997042] R10: 0000000000000012 R11: 0000000000000246 R12: 00007f98832541e3 [ 17.004222] R13: 0000000000000000 R14: 0000000000000000 R15: 00007ffc66b60328 [ 17.011369] ---[ end trace df06a3dab26b988c ]--- [ 17.016062] ------------[ cut here ]------------ [ 17.020701] stmmac-0000:00:1e.5 already unprepared Removing the stmmac_bus_clks_config() call in stmmac_dvr_probe and let dwmac-intel to handle the unprepare and disable of the clk device. Fixes: 5ec5582 ("net: stmmac: add clocks management for gmac driver") Cc: Joakim Zhang <[email protected]> Signed-off-by: Wong Vee Khee <[email protected]> Reviewed-by: Joakim Zhang <[email protected]> Signed-off-by: David S. Miller <[email protected]>

Parsing of USDT arguments is architecture-specific; on arm it is relatively easy since registers used are r[0-10], fp, ip, sp, lr, pc. Format is slightly different compared to aarch64; forms are - "size @ [ reg, #offset ]" for dereferences, for example "-8 @ [ sp, #76 ]" ; " -4 @ [ sp ]" - "size @ reg" for register values; for example "-4@r0" - "size @ #value" for raw values; for example "-8@#1" Add support for parsing USDT arguments for ARM architecture. To test the above changes QEMU's virt[1] board with cortex-a15 CPU was used. libbpf-bootstrap's usdt example[2] was modified to attach to a test program with DTRACE_PROBE1/2/3/4... probes to test different combinations. [1] https://www.qemu.org/docs/master/system/arm/virt.html [2] https://github.com/libbpf/libbpf-bootstrap/blob/master/examples/c/usdt.bpf.c Signed-off-by: Puranjay Mohan <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]

With latest clang18, I hit test_progs failures for the following test: #13/2 bpf_cookie/multi_kprobe_link_api:FAIL #13/3 bpf_cookie/multi_kprobe_attach_api:FAIL #13 bpf_cookie:FAIL #75 fentry_fexit:FAIL #76/1 fentry_test/fentry:FAIL #76 fentry_test:FAIL #80/1 fexit_test/fexit:FAIL #80 fexit_test:FAIL #110/1 kprobe_multi_test/skel_api:FAIL #110/2 kprobe_multi_test/link_api_addrs:FAIL #110/3 kprobe_multi_test/link_api_syms:FAIL #110/4 kprobe_multi_test/attach_api_pattern:FAIL #110/5 kprobe_multi_test/attach_api_addrs:FAIL #110/6 kprobe_multi_test/attach_api_syms:FAIL #110 kprobe_multi_test:FAIL For example, for #13/2, the error messages are: [...] kprobe_multi_test_run:FAIL:kprobe_test7_result unexpected kprobe_test7_result: actual 0 != expected 1 [...] kprobe_multi_test_run:FAIL:kretprobe_test7_result unexpected kretprobe_test7_result: actual 0 != expected 1 clang17 does not have this issue. Further investigation shows that kernel func bpf_fentry_test7(), used in the above tests, is inlined by the compiler although it is marked as noinline. int noinline bpf_fentry_test7(struct bpf_fentry_test_t *arg) { return (long)arg; } It is known that for simple functions like the above (e.g. just returning a constant or an input argument), the clang compiler may still do inlining for a noinline function. Adding 'asm volatile ("")' in the beginning of the bpf_fentry_test7() can prevent inlining. Signed-off-by: Yonghong Song <[email protected]> Signed-off-by: Daniel Borkmann <[email protected]> Tested-by: Eduard Zingerman <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]

geliangtang · 2024-03-18T01:12:38Z

@matttbe Matt, please give me more details about this issue. I'll try to implement it. Thanks.

matttbe · 2024-03-18T10:26:59Z

@geliangtang I think being able to set/get socket options per subflow from BPF is already possible. What is missing is the documentation and maybe a test.

Back then, Nicolas added a test that was adding a different mark (SO_MARK) on each subflow, and changing the TCP CC only on the first subflow: https://github.com/multipath-tcp/mptcp_net-next/blob/scripts/bpf/examples/mptcp_set_sf_sockopt_kern.c
I don't know if Nicolas' test is still OK, as his kernel code had to be adapted.

I think it would be good to have such test in the kernel, so it can be used as an example (and maybe linked to a doc), and be useful to track regressions.

Closes: multipath-tcp#76 Signed-off-by: Geliang Tang <[email protected]>

Move Nicolas's patch into bpf selftests directory. This prog added a test that was adding a different mark (SO_MARK) on each subflow, and changing the TCP CC only on the first subflow. Closes: multipath-tcp/mptcp_net-next#76 Signed-off-by: Geliang Tang <[email protected]>

This patch adds a subtest named test_subflow to load and verify the newly added mptcp subflow example in test_mptcp. Add a helper endpoint_init() to add a new subflow endpoint. Add another helper ss_search() to verify the fwmark and congestion values set by mptcp_subflow prog using setsockopts. Closes: #76 Signed-off-by: Geliang Tang <[email protected]> Reviewed-by: Mat Martineau <[email protected]>

Move Nicolas' patch into bpf selftests directory. This example adds a different mark (SO_MARK) on each subflow, and changes the TCP CC only on the first subflow. From the userspace, an application can do a setsockopt() on an MPTCP socket, and typically the same value will be propagated to all subflows (paths). If someone wants to have different values per subflow, the recommended way is to use BPF. So it is good to add such example here, and make sure there is no regressions. This example shows how it is possible to: Identify the parent msk of an MPTCP subflow. Put different sockopt for each subflow of a same MPTCP connection. Here especially, two different behaviours are implemented: A socket mark (SOL_SOCKET SO_MARK) is put on each subflow of a same MPTCP connection. The order of creation of the current subflow defines its mark. The TCP CC algorithm of the very first subflow of an MPTCP connection is set to "reno". This is just to show it is possible to identify an MPTCP connection, and set socket options, from different SOL levels, per subflow. It is easy to verify with 'ss' that these modifications have been applied correctly. That's what the next patch is going to do. Nicolas' code comes from: commit 4d12018 ("bpf:examples: update mptcp_set_mark_kern.c") from the MPTCP repo https://github.com/multipath-tcp/mptcp_net-next (the "scripts" branch), and it has been adapted by Geliang. Closes: #76 Co-developed-by: Geliang Tang <[email protected]> Signed-off-by: Geliang Tang <[email protected]> Signed-off-by: Nicolas Rybowski <[email protected]> Reviewed-by: Mat Martineau <[email protected]>