emmc failures on 4.4.9+ / 4.4.21+ #103

floion · 2016-09-24T18:51:24Z

Hi, on the above 2 kernels, and I assume that also on kernel versions in between these 2, we get emmc failures when using docker on a partition with a btrfs filesystem. Here is the relevant dmesg log:

733.665535] mmcqd/1: page allocation failure: order:2, mode:0x208c020
[ 733.680082] CPU: 0 PID: 80 Comm: mmcqd/1 Not tainted 4.4.21+ #1
[ 733.680087] Hardware name: Generic AM33XX (Flattened Device Tree)
[ 733.680128] from
[ 733.680148] from
[ 733.680170] from
[ 733.680185] from
[ 733.680196] from
[ 733.680211] from
[ 733.680223] from
[ 733.680235] from
[ 733.680256] from
[ 733.680270] from
[ 733.680280] from
[ 733.680290] from
[ 733.680299] from
[ 733.680309] from
[ 733.680323] from
[ 733.680341] from
[ 733.692342] Mem-Info:
[ 734.087941] active_anon:45429 inactive_anon:2175 isolated_anon:0
active_file:32045 inactive_file:31560 isolated_file:0
unevictable:0 dirty:4275 writeback:1003 unstable:0
slab_reclaimable:7336 slab_unreclaimable:3555
mapped:12949 shmem:2204 pagetables:425 bounce:0
free:1012 free_pcp:139 free_cma:0
[ 734.191726] Normal free:3800kB min:2764kB low:3452kB high:4144kB active_anon:181716kB inactive_anon:8700kB active_file:128208kB inactive_file:126324kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:522240kB managed:503092kB mlocked:0kB dirty:17180kB writeback:4012kB mapped:51796kB shmem:8816kB slab_reclaimable:29344kB slab_unreclaimable:14220kB kernel_stack:1568kB pagetables:1700kB unstable:0kB bounce:0kB free_pcp:716kB local_pcp:716kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
[ 734.348853] lowmem_reserve[]: 0 0 0
[ 734.355748] Normal: 1113_4kB (UME) 159_8kB (UME) 33_16kB (UM) 8_32kB (UM) 2_64kB (UM) 2_128kB (U) 1_256kB (U) 0_512kB 0_1024kB 0_2048kB 0_4096kB 0_8192kB = 7148kB
[ 734.391678] 65030 total pagecache pages
[ 734.411660] 0 pages in swap cache
[ 734.423080] Swap cache stats: add 0, delete 0, find 0/0
[ 734.444296] Free swap = 0kB
[ 734.447204] Total swap = 0kB
[ 734.459255] 130560 pages RAM
[ 734.466300] 0 pages HighMem/MovableOnly
[ 734.478228] 4787 pages reserved
[ 734.489486] 6144 pages cma reserved
[ 734.504504] edma 49000000.edma: edma_prep_slave_sg: Failed to allocate a descriptor
[ 734.532734] omap_hsmmc 481d8000.mmc: prep_slave_sg() failed
[ 734.548775] omap_hsmmc 481d8000.mmc: MMC start dma failure
[ 734.568595] mmcblk1: unknown error -1 sending read/write command, card status 0x900
[ 734.587773] blk_update_request: I/O error, dev mmcblk1, sector 2243096
[ 734.614544] blk_update_request: I/O error, dev mmcblk1, sector 2243104
[ 734.633186] blk_update_request: I/O error, dev mmcblk1, sector 2243112
[ 734.650927] blk_update_request: I/O error, dev mmcblk1, sector 2243120
[ 734.669731] blk_update_request: I/O error, dev mmcblk1, sector 2243128
[ 734.685783] blk_update_request: I/O error, dev mmcblk1, sector 2243136
[ 734.704844] blk_update_request: I/O error, dev mmcblk1, sector 2243144
[ 734.718002] blk_update_request: I/O error, dev mmcblk1, sector 2243152
[ 734.736844] blk_update_request: I/O error, dev mmcblk1, sector 2243160
[ 734.753321] blk_update_request: I/O error, dev mmcblk1, sector 2243168
[ 734.784384] BTRFS error (device mmcblk1p6): bdev /dev/mmcblk1p6 errs: wr 4, rd 0, flush 0, corrupt 0, gen 0

Is this a known issue? I have looked at the existing issues and could not find something related. I see some other people have stumbled upon this before: https://groups.google.com/forum/#!topic/beagleboard/5ZwTUQwGSx4

As in their case, our use case involves a lot of I/O.

Thank you

floion · 2016-09-27T12:47:11Z

hi @RobertCNelson is this something that has affected other users too? Are you relying on the upstream mmc driver in this kernel?

floion · 2016-10-27T10:56:43Z

ping on this @RobertCNelson

locomundo · 2016-10-31T08:39:50Z

We are having this problem too, quite persistently.

pirvanc · 2016-10-31T08:46:25Z

We would be very interested in a fix for this issue as well.

FRVisser · 2016-10-31T11:04:22Z

Any updates?

RobertCNelson · 2016-10-31T14:07:49Z

Any improvement with linux-image-4.8.5-bone2 ?

jaapcrezee · 2016-10-31T14:28:41Z

Could it somehow be related to https://cdn.kernel.org/pub/linux/kernel/v4.x/ChangeLog-4.1.29

commit 74225a4
Author: Steven Rostedt [email protected]
Date: Thu Jul 14 17:55:21 2016 -0400

I can recall seeing this issue before.

Regards,

Jaap Crezee

On 10/31/16 15:07, Robert Nelson wrote:

Any improvement with linux-image-4.8.5-bone2 ?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#103 (comment), or mute the thread
https://github.com/notifications/unsubscribe-auth/AFRTlzSMXs3PrEZ31x2CujdOKcANtX6Vks5q5fY3gaJpZM4KFt2j.

adnanoner · 2016-11-01T10:22:09Z

Following

floion · 2016-11-02T11:39:47Z

@RobertCNelson there are no 4.8 tags. And the 4.9 tags are rc. Do you plan to do a new release?

RobertCNelson · 2016-11-02T15:53:40Z

@floion

the v4.9.x branch will be replacing our v4.4.x in the next quarter or so..

Here's my current progress of getting it ready..
https://groups.google.com/forum/#!category-topic/beagleboard/_TdTH7oPEXE

right now working on eqep..

Regards,

floion · 2017-03-27T13:00:20Z

hi @RobertCNelson is the 4.9 branch now stable so that it can be used in a production environment?
Thank you

floion · 2017-03-27T13:01:08Z

Or the 4.10 one? Which is the latest stable branch/tag available for use in production?

RobertCNelson · 2017-03-28T02:07:45Z

@floion feel free to use the 4.4/4.4-rt/4.4-xenomai or 4.9/4.9-rt branches

floion · 2017-03-28T02:25:36Z

Thank you

willnewton · 2017-06-16T10:03:07Z

I saw a similar failure recently with 4.4.21:

[ 1325.957185] mmcqd/1: page allocation failure: order:3, mode:0x208c020
[ 1325.964604] CPU: 0 PID: 79 Comm: mmcqd/1 Not tainted 4.4.21+ #1
[ 1325.964609] Hardware name: Generic AM33XX (Flattened Device Tree)
[ 1325.964649] [<c0015e09>] (unwind_backtrace) from [<c001259d>] (show_stack+0x11/0x14)
[ 1325.964667] [<c001259d>] (show_stack) from [<c03dba65>] (dump_stack+0x69/0x78)
[ 1325.964689] [<c03dba65>] (dump_stack) from [<c0100a4d>] (warn_alloc_failed+0xb5/0xf0)
[ 1325.964704] [<c0100a4d>] (warn_alloc_failed) from [<c0103393>] (__alloc_pages_nodemask+0x693/0x7d4)
[ 1325.964714] [<c0103393>] (__alloc_pages_nodemask) from [<c0103629>] (alloc_kmem_pages+0x15/0x78)
[ 1325.964730] [<c0103629>] (alloc_kmem_pages) from [<c011782f>] (kmalloc_order+0x13/0x20)
[ 1325.964741] [<c011782f>] (kmalloc_order) from [<c0117855>] (kmalloc_order_trace+0x19/0xbc)
[ 1325.964752] [<c0117855>] (kmalloc_order_trace) from [<c042e2e5>] (edma_prep_slave_sg+0x79/0x248)
[ 1325.964773] [<c042e2e5>] (edma_prep_slave_sg) from [<c05c2cef>] (omap_hsmmc_request+0x353/0x4a4)
[ 1325.964786] [<c05c2cef>] (omap_hsmmc_request) from [<c05b1317>] (mmc_start_request+0xcb/0x1ac)
[ 1325.964796] [<c05b1317>] (mmc_start_request) from [<c05b22ef>] (mmc_start_req+0x1eb/0x2e0)
[ 1325.964805] [<c05b22ef>] (mmc_start_req) from [<c05bd6ff>] (mmc_blk_issue_rw_rq+0x97/0x7fc)
[ 1325.964815] [<c05bd6ff>] (mmc_blk_issue_rw_rq) from [<c05be00f>] (mmc_blk_issue_rq+0x1ab/0x370)
[ 1325.964824] [<c05be00f>] (mmc_blk_issue_rq) from [<c05bf3d9>] (mmc_queue_thread+0x95/0x13c)
[ 1325.964839] [<c05bf3d9>] (mmc_queue_thread) from [<c00493f7>] (kthread+0xaf/0xc4)
[ 1325.964856] [<c00493f7>] (kthread) from [<c000ee15>] (ret_from_fork+0x11/0x1c)
[ 1325.964860] Mem-Info:
[ 1326.145282] active_anon:12699 inactive_anon:1067 isolated_anon:0
                active_file:44015 inactive_file:54031 isolated_file:0
                unevictable:0 dirty:2392 writeback:3374 unstable:0
                slab_reclaimable:6953 slab_unreclaimable:1743
                mapped:1795 shmem:1103 pagetables:227 bounce:0
                free:3166 free_pcp:183 free_cma:0
[ 1326.187989] Normal free:12652kB min:8192kB low:10240kB high:12288kB active_anon:50796kB inactive_anon:4268kB active_file:176060kB inactive_file:216124kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:522240kB managed:503092kB mlocked:0kB dirty:9568kB writeback:13496kB mapped:7180kB shmem:4412kB slab_reclaimable:27812kB slab_unreclaimable:6972kB kernel_stack:1192kB pagetables:908kB unstable:0kB bounce:0kB free_pcp:732kB local_pcp:732kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
[ 1326.244948] lowmem_reserve[]: 0 0 0
[ 1326.248511] Normal: 2986*4kB (UME) 84*8kB (UME) 3*16kB (UE) 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB 0*8192kB = 12664kB
[ 1326.263731] 99149 total pagecache pages
[ 1326.271061] 0 pages in swap cache
[ 1326.276348] Swap cache stats: add 0, delete 0, find 0/0
[ 1326.283657] Free swap  = 0kB
[ 1326.288427] Total swap = 0kB
[ 1326.293281] 130560 pages RAM
[ 1326.298250] 0 pages HighMem/MovableOnly
[ 1326.303974] 4787 pages reserved
[ 1326.309196] 6144 pages cma reserved
[ 1326.314707] edma 49000000.edma: edma_prep_slave_sg: Failed to allocate a descriptor

What strikes me as odd is this is doing an order 3 allocation for DMA descriptors. I estimate DMA descriptors are 40 bytes so in an order 3 allocation we could get over 800 sg entries which seems like a very large (and not very useful) number.

I wonder if possibly the cause of this is the relatively high value of max_segs (1024) in the omaphsmmc driver?

I notice there have been a couple of patches over the years that have attempted to reduce this number:

https://groups.google.com/d/msg/linux.kernel/-JRzNvTytyA/TglXstxMyusJ
https://github.com/beagleboard/meta-beagleboard/blob/master/common-bsp/recipes-kernel/linux/linux-mainline-3.8/mmc/0002-omap-hsmmc-Correct-usage-of-of_find_node_by_name.patch

willnewton · 2017-06-20T15:12:26Z

Patch submitted upstream here:

http://marc.info/?l=linux-omap&m=149786501429968

RobertCNelson · 2017-06-26T15:33:30Z

@willnewton 's patch is now part of 4.4.x-ti-r111 and 4.9.x-ti-r43* (r43 is still build testing, till i push out to github.com/beagleboard/linux)

[ Upstream commit 1151f83 ] When running lscpu on an AArch64 system that has SMBIOS version 2.0 tables, it will segfault in the following way: Unable to handle kernel paging request at virtual address ffff8000bfff0000 pgd = ffff8000f9615000 [ffff8000bfff0000] *pgd=0000000000000000 Internal error: Oops: 96000007 [#1] PREEMPT SMP Modules linked in: CPU: 0 PID: 1284 Comm: lscpu Not tainted 4.11.0-rc3+ #103 Hardware name: QEMU QEMU Virtual Machine, BIOS 0.0.0 02/06/2015 task: ffff8000fa78e800 task.stack: ffff8000f9780000 PC is at __arch_copy_to_user+0x90/0x220 LR is at read_mem+0xcc/0x140 This is caused by the fact that lspci issues a read() on /dev/mem at the offset where it expects to find the SMBIOS structure array. However, this region is classified as EFI_RUNTIME_SERVICE_DATA (as per the UEFI spec), and so it is omitted from the linear mapping. So let's restrict /dev/mem read/write access to those areas that are covered by the linear region. Reported-by: Alexander Graf <[email protected]> Fixes: 4dffbfc ("arm64/efi: mark UEFI reserved regions as MEMBLOCK_NOMAP") Signed-off-by: Ard Biesheuvel <[email protected]> Signed-off-by: Will Deacon <[email protected]> Signed-off-by: Sasha Levin <[email protected]> Signed-off-by: Greg Kroah-Hartman <[email protected]>

[ Upstream commit 5b5e7a0 ] Before using nla_get_u32(), better make sure the attribute is of the proper size. Code recently was changed, but bug has been there from beginning of git. BUG: KMSAN: uninit-value in rtnetlink_put_metrics+0x553/0x960 net/core/rtnetlink.c:746 CPU: 1 PID: 14139 Comm: syz-executor6 Not tainted 4.17.0-rc5+ #103 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0x185/0x1d0 lib/dump_stack.c:113 kmsan_report+0x149/0x260 mm/kmsan/kmsan.c:1084 __msan_warning_32+0x6e/0xc0 mm/kmsan/kmsan_instr.c:686 rtnetlink_put_metrics+0x553/0x960 net/core/rtnetlink.c:746 fib_dump_info+0xc42/0x2190 net/ipv4/fib_semantics.c:1361 rtmsg_fib+0x65f/0x8c0 net/ipv4/fib_semantics.c:419 fib_table_insert+0x2314/0x2b50 net/ipv4/fib_trie.c:1287 inet_rtm_newroute+0x210/0x340 net/ipv4/fib_frontend.c:779 rtnetlink_rcv_msg+0xa32/0x1560 net/core/rtnetlink.c:4646 netlink_rcv_skb+0x378/0x600 net/netlink/af_netlink.c:2448 rtnetlink_rcv+0x50/0x60 net/core/rtnetlink.c:4664 netlink_unicast_kernel net/netlink/af_netlink.c:1310 [inline] netlink_unicast+0x1678/0x1750 net/netlink/af_netlink.c:1336 netlink_sendmsg+0x104f/0x1350 net/netlink/af_netlink.c:1901 sock_sendmsg_nosec net/socket.c:629 [inline] sock_sendmsg net/socket.c:639 [inline] ___sys_sendmsg+0xec0/0x1310 net/socket.c:2117 __sys_sendmsg net/socket.c:2155 [inline] __do_sys_sendmsg net/socket.c:2164 [inline] __se_sys_sendmsg net/socket.c:2162 [inline] __x64_sys_sendmsg+0x331/0x460 net/socket.c:2162 do_syscall_64+0x152/0x230 arch/x86/entry/common.c:287 entry_SYSCALL_64_after_hwframe+0x44/0xa9 RIP: 0033:0x455a09 RSP: 002b:00007faae5fd8c68 EFLAGS: 00000246 ORIG_RAX: 000000000000002e RAX: ffffffffffffffda RBX: 00007faae5fd96d4 RCX: 0000000000455a09 RDX: 0000000000000000 RSI: 0000000020000000 RDI: 0000000000000013 RBP: 000000000072bea0 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000246 R12: 00000000ffffffff R13: 00000000000005d0 R14: 00000000006fdc20 R15: 0000000000000000 Uninit was stored to memory at: kmsan_save_stack_with_flags mm/kmsan/kmsan.c:279 [inline] kmsan_save_stack mm/kmsan/kmsan.c:294 [inline] kmsan_internal_chain_origin+0x12b/0x210 mm/kmsan/kmsan.c:685 __msan_chain_origin+0x69/0xc0 mm/kmsan/kmsan_instr.c:529 fib_convert_metrics net/ipv4/fib_semantics.c:1056 [inline] fib_create_info+0x2d46/0x9dc0 net/ipv4/fib_semantics.c:1150 fib_table_insert+0x3e4/0x2b50 net/ipv4/fib_trie.c:1146 inet_rtm_newroute+0x210/0x340 net/ipv4/fib_frontend.c:779 rtnetlink_rcv_msg+0xa32/0x1560 net/core/rtnetlink.c:4646 netlink_rcv_skb+0x378/0x600 net/netlink/af_netlink.c:2448 rtnetlink_rcv+0x50/0x60 net/core/rtnetlink.c:4664 netlink_unicast_kernel net/netlink/af_netlink.c:1310 [inline] netlink_unicast+0x1678/0x1750 net/netlink/af_netlink.c:1336 netlink_sendmsg+0x104f/0x1350 net/netlink/af_netlink.c:1901 sock_sendmsg_nosec net/socket.c:629 [inline] sock_sendmsg net/socket.c:639 [inline] ___sys_sendmsg+0xec0/0x1310 net/socket.c:2117 __sys_sendmsg net/socket.c:2155 [inline] __do_sys_sendmsg net/socket.c:2164 [inline] __se_sys_sendmsg net/socket.c:2162 [inline] __x64_sys_sendmsg+0x331/0x460 net/socket.c:2162 do_syscall_64+0x152/0x230 arch/x86/entry/common.c:287 entry_SYSCALL_64_after_hwframe+0x44/0xa9 Uninit was created at: kmsan_save_stack_with_flags mm/kmsan/kmsan.c:279 [inline] kmsan_internal_poison_shadow+0xb8/0x1b0 mm/kmsan/kmsan.c:189 kmsan_kmalloc+0x94/0x100 mm/kmsan/kmsan.c:315 kmsan_slab_alloc+0x10/0x20 mm/kmsan/kmsan.c:322 slab_post_alloc_hook mm/slab.h:446 [inline] slab_alloc_node mm/slub.c:2753 [inline] __kmalloc_node_track_caller+0xb32/0x11b0 mm/slub.c:4395 __kmalloc_reserve net/core/skbuff.c:138 [inline] __alloc_skb+0x2cb/0x9e0 net/core/skbuff.c:206 alloc_skb include/linux/skbuff.h:988 [inline] netlink_alloc_large_skb net/netlink/af_netlink.c:1182 [inline] netlink_sendmsg+0x76e/0x1350 net/netlink/af_netlink.c:1876 sock_sendmsg_nosec net/socket.c:629 [inline] sock_sendmsg net/socket.c:639 [inline] ___sys_sendmsg+0xec0/0x1310 net/socket.c:2117 __sys_sendmsg net/socket.c:2155 [inline] __do_sys_sendmsg net/socket.c:2164 [inline] __se_sys_sendmsg net/socket.c:2162 [inline] __x64_sys_sendmsg+0x331/0x460 net/socket.c:2162 do_syscall_64+0x152/0x230 arch/x86/entry/common.c:287 entry_SYSCALL_64_after_hwframe+0x44/0xa9 Fixes: a919525 ("net: Move fib_convert_metrics to metrics file") Fixes: 1da177e ("Linux-2.6.12-rc2") Signed-off-by: Eric Dumazet <[email protected]> Reported-by: syzbot <[email protected]> Cc: David Ahern <[email protected]> Signed-off-by: David S. Miller <[email protected]> Signed-off-by: Greg Kroah-Hartman <[email protected]>

[ Upstream commit 644c7ee ] It seems that rtnl_group_changelink() can call do_setlink while a prior call to validate_linkmsg(dev = NULL, ...) could not validate IFLA_ADDRESS / IFLA_BROADCAST Make sure do_setlink() calls validate_linkmsg() instead of letting its callers having this responsibility. With help from Dmitry Vyukov, thanks a lot ! BUG: KMSAN: uninit-value in is_valid_ether_addr include/linux/etherdevice.h:199 [inline] BUG: KMSAN: uninit-value in eth_prepare_mac_addr_change net/ethernet/eth.c:275 [inline] BUG: KMSAN: uninit-value in eth_mac_addr+0x203/0x2b0 net/ethernet/eth.c:308 CPU: 1 PID: 8695 Comm: syz-executor3 Not tainted 4.17.0-rc5+ #103 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0x185/0x1d0 lib/dump_stack.c:113 kmsan_report+0x149/0x260 mm/kmsan/kmsan.c:1084 __msan_warning_32+0x6e/0xc0 mm/kmsan/kmsan_instr.c:686 is_valid_ether_addr include/linux/etherdevice.h:199 [inline] eth_prepare_mac_addr_change net/ethernet/eth.c:275 [inline] eth_mac_addr+0x203/0x2b0 net/ethernet/eth.c:308 dev_set_mac_address+0x261/0x530 net/core/dev.c:7157 do_setlink+0xbc3/0x5fc0 net/core/rtnetlink.c:2317 rtnl_group_changelink net/core/rtnetlink.c:2824 [inline] rtnl_newlink+0x1fe9/0x37a0 net/core/rtnetlink.c:2976 rtnetlink_rcv_msg+0xa32/0x1560 net/core/rtnetlink.c:4646 netlink_rcv_skb+0x378/0x600 net/netlink/af_netlink.c:2448 rtnetlink_rcv+0x50/0x60 net/core/rtnetlink.c:4664 netlink_unicast_kernel net/netlink/af_netlink.c:1310 [inline] netlink_unicast+0x1678/0x1750 net/netlink/af_netlink.c:1336 netlink_sendmsg+0x104f/0x1350 net/netlink/af_netlink.c:1901 sock_sendmsg_nosec net/socket.c:629 [inline] sock_sendmsg net/socket.c:639 [inline] ___sys_sendmsg+0xec0/0x1310 net/socket.c:2117 __sys_sendmsg net/socket.c:2155 [inline] __do_sys_sendmsg net/socket.c:2164 [inline] __se_sys_sendmsg net/socket.c:2162 [inline] __x64_sys_sendmsg+0x331/0x460 net/socket.c:2162 do_syscall_64+0x152/0x230 arch/x86/entry/common.c:287 entry_SYSCALL_64_after_hwframe+0x44/0xa9 RIP: 0033:0x455a09 RSP: 002b:00007fc07480ec68 EFLAGS: 00000246 ORIG_RAX: 000000000000002e RAX: ffffffffffffffda RBX: 00007fc07480f6d4 RCX: 0000000000455a09 RDX: 0000000000000000 RSI: 00000000200003c0 RDI: 0000000000000014 RBP: 000000000072bea0 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000246 R12: 00000000ffffffff R13: 00000000000005d0 R14: 00000000006fdc20 R15: 0000000000000000 Uninit was stored to memory at: kmsan_save_stack_with_flags mm/kmsan/kmsan.c:279 [inline] kmsan_save_stack mm/kmsan/kmsan.c:294 [inline] kmsan_internal_chain_origin+0x12b/0x210 mm/kmsan/kmsan.c:685 kmsan_memcpy_origins+0x11d/0x170 mm/kmsan/kmsan.c:527 __msan_memcpy+0x109/0x160 mm/kmsan/kmsan_instr.c:478 do_setlink+0xb84/0x5fc0 net/core/rtnetlink.c:2315 rtnl_group_changelink net/core/rtnetlink.c:2824 [inline] rtnl_newlink+0x1fe9/0x37a0 net/core/rtnetlink.c:2976 rtnetlink_rcv_msg+0xa32/0x1560 net/core/rtnetlink.c:4646 netlink_rcv_skb+0x378/0x600 net/netlink/af_netlink.c:2448 rtnetlink_rcv+0x50/0x60 net/core/rtnetlink.c:4664 netlink_unicast_kernel net/netlink/af_netlink.c:1310 [inline] netlink_unicast+0x1678/0x1750 net/netlink/af_netlink.c:1336 netlink_sendmsg+0x104f/0x1350 net/netlink/af_netlink.c:1901 sock_sendmsg_nosec net/socket.c:629 [inline] sock_sendmsg net/socket.c:639 [inline] ___sys_sendmsg+0xec0/0x1310 net/socket.c:2117 __sys_sendmsg net/socket.c:2155 [inline] __do_sys_sendmsg net/socket.c:2164 [inline] __se_sys_sendmsg net/socket.c:2162 [inline] __x64_sys_sendmsg+0x331/0x460 net/socket.c:2162 do_syscall_64+0x152/0x230 arch/x86/entry/common.c:287 entry_SYSCALL_64_after_hwframe+0x44/0xa9 Uninit was created at: kmsan_save_stack_with_flags mm/kmsan/kmsan.c:279 [inline] kmsan_internal_poison_shadow+0xb8/0x1b0 mm/kmsan/kmsan.c:189 kmsan_kmalloc+0x94/0x100 mm/kmsan/kmsan.c:315 kmsan_slab_alloc+0x10/0x20 mm/kmsan/kmsan.c:322 slab_post_alloc_hook mm/slab.h:446 [inline] slab_alloc_node mm/slub.c:2753 [inline] __kmalloc_node_track_caller+0xb32/0x11b0 mm/slub.c:4395 __kmalloc_reserve net/core/skbuff.c:138 [inline] __alloc_skb+0x2cb/0x9e0 net/core/skbuff.c:206 alloc_skb include/linux/skbuff.h:988 [inline] netlink_alloc_large_skb net/netlink/af_netlink.c:1182 [inline] netlink_sendmsg+0x76e/0x1350 net/netlink/af_netlink.c:1876 sock_sendmsg_nosec net/socket.c:629 [inline] sock_sendmsg net/socket.c:639 [inline] ___sys_sendmsg+0xec0/0x1310 net/socket.c:2117 __sys_sendmsg net/socket.c:2155 [inline] __do_sys_sendmsg net/socket.c:2164 [inline] __se_sys_sendmsg net/socket.c:2162 [inline] __x64_sys_sendmsg+0x331/0x460 net/socket.c:2162 do_syscall_64+0x152/0x230 arch/x86/entry/common.c:287 entry_SYSCALL_64_after_hwframe+0x44/0xa9 Fixes: e7ed828 ("netlink: support setting devgroup parameters") Signed-off-by: Eric Dumazet <[email protected]> Reported-by: syzbot <[email protected]> Cc: Dmitry Vyukov <[email protected]> Signed-off-by: David S. Miller <[email protected]> Signed-off-by: Greg Kroah-Hartman <[email protected]>

[ Upstream commit 5b5e7a0 ] Before using nla_get_u32(), better make sure the attribute is of the proper size. Code recently was changed, but bug has been there from beginning of git. BUG: KMSAN: uninit-value in rtnetlink_put_metrics+0x553/0x960 net/core/rtnetlink.c:746 CPU: 1 PID: 14139 Comm: syz-executor6 Not tainted 4.17.0-rc5+ #103 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0x185/0x1d0 lib/dump_stack.c:113 kmsan_report+0x149/0x260 mm/kmsan/kmsan.c:1084 __msan_warning_32+0x6e/0xc0 mm/kmsan/kmsan_instr.c:686 rtnetlink_put_metrics+0x553/0x960 net/core/rtnetlink.c:746 fib_dump_info+0xc42/0x2190 net/ipv4/fib_semantics.c:1361 rtmsg_fib+0x65f/0x8c0 net/ipv4/fib_semantics.c:419 fib_table_insert+0x2314/0x2b50 net/ipv4/fib_trie.c:1287 inet_rtm_newroute+0x210/0x340 net/ipv4/fib_frontend.c:779 rtnetlink_rcv_msg+0xa32/0x1560 net/core/rtnetlink.c:4646 netlink_rcv_skb+0x378/0x600 net/netlink/af_netlink.c:2448 rtnetlink_rcv+0x50/0x60 net/core/rtnetlink.c:4664 netlink_unicast_kernel net/netlink/af_netlink.c:1310 [inline] netlink_unicast+0x1678/0x1750 net/netlink/af_netlink.c:1336 netlink_sendmsg+0x104f/0x1350 net/netlink/af_netlink.c:1901 sock_sendmsg_nosec net/socket.c:629 [inline] sock_sendmsg net/socket.c:639 [inline] ___sys_sendmsg+0xec0/0x1310 net/socket.c:2117 __sys_sendmsg net/socket.c:2155 [inline] __do_sys_sendmsg net/socket.c:2164 [inline] __se_sys_sendmsg net/socket.c:2162 [inline] __x64_sys_sendmsg+0x331/0x460 net/socket.c:2162 do_syscall_64+0x152/0x230 arch/x86/entry/common.c:287 entry_SYSCALL_64_after_hwframe+0x44/0xa9 RIP: 0033:0x455a09 RSP: 002b:00007faae5fd8c68 EFLAGS: 00000246 ORIG_RAX: 000000000000002e RAX: ffffffffffffffda RBX: 00007faae5fd96d4 RCX: 0000000000455a09 RDX: 0000000000000000 RSI: 0000000020000000 RDI: 0000000000000013 RBP: 000000000072bea0 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000246 R12: 00000000ffffffff R13: 00000000000005d0 R14: 00000000006fdc20 R15: 0000000000000000 Uninit was stored to memory at: kmsan_save_stack_with_flags mm/kmsan/kmsan.c:279 [inline] kmsan_save_stack mm/kmsan/kmsan.c:294 [inline] kmsan_internal_chain_origin+0x12b/0x210 mm/kmsan/kmsan.c:685 __msan_chain_origin+0x69/0xc0 mm/kmsan/kmsan_instr.c:529 fib_convert_metrics net/ipv4/fib_semantics.c:1056 [inline] fib_create_info+0x2d46/0x9dc0 net/ipv4/fib_semantics.c:1150 fib_table_insert+0x3e4/0x2b50 net/ipv4/fib_trie.c:1146 inet_rtm_newroute+0x210/0x340 net/ipv4/fib_frontend.c:779 rtnetlink_rcv_msg+0xa32/0x1560 net/core/rtnetlink.c:4646 netlink_rcv_skb+0x378/0x600 net/netlink/af_netlink.c:2448 rtnetlink_rcv+0x50/0x60 net/core/rtnetlink.c:4664 netlink_unicast_kernel net/netlink/af_netlink.c:1310 [inline] netlink_unicast+0x1678/0x1750 net/netlink/af_netlink.c:1336 netlink_sendmsg+0x104f/0x1350 net/netlink/af_netlink.c:1901 sock_sendmsg_nosec net/socket.c:629 [inline] sock_sendmsg net/socket.c:639 [inline] ___sys_sendmsg+0xec0/0x1310 net/socket.c:2117 __sys_sendmsg net/socket.c:2155 [inline] __do_sys_sendmsg net/socket.c:2164 [inline] __se_sys_sendmsg net/socket.c:2162 [inline] __x64_sys_sendmsg+0x331/0x460 net/socket.c:2162 do_syscall_64+0x152/0x230 arch/x86/entry/common.c:287 entry_SYSCALL_64_after_hwframe+0x44/0xa9 Uninit was created at: kmsan_save_stack_with_flags mm/kmsan/kmsan.c:279 [inline] kmsan_internal_poison_shadow+0xb8/0x1b0 mm/kmsan/kmsan.c:189 kmsan_kmalloc+0x94/0x100 mm/kmsan/kmsan.c:315 kmsan_slab_alloc+0x10/0x20 mm/kmsan/kmsan.c:322 slab_post_alloc_hook mm/slab.h:446 [inline] slab_alloc_node mm/slub.c:2753 [inline] __kmalloc_node_track_caller+0xb32/0x11b0 mm/slub.c:4395 __kmalloc_reserve net/core/skbuff.c:138 [inline] __alloc_skb+0x2cb/0x9e0 net/core/skbuff.c:206 alloc_skb include/linux/skbuff.h:988 [inline] netlink_alloc_large_skb net/netlink/af_netlink.c:1182 [inline] netlink_sendmsg+0x76e/0x1350 net/netlink/af_netlink.c:1876 sock_sendmsg_nosec net/socket.c:629 [inline] sock_sendmsg net/socket.c:639 [inline] ___sys_sendmsg+0xec0/0x1310 net/socket.c:2117 __sys_sendmsg net/socket.c:2155 [inline] __do_sys_sendmsg net/socket.c:2164 [inline] __se_sys_sendmsg net/socket.c:2162 [inline] __x64_sys_sendmsg+0x331/0x460 net/socket.c:2162 do_syscall_64+0x152/0x230 arch/x86/entry/common.c:287 entry_SYSCALL_64_after_hwframe+0x44/0xa9 Fixes: a919525 ("net: Move fib_convert_metrics to metrics file") Fixes: 1da177e ("Linux-2.6.12-rc2") Signed-off-by: Eric Dumazet <[email protected]> Reported-by: syzbot <[email protected]> Cc: David Ahern <[email protected]> Signed-off-by: David S. Miller <[email protected]> Signed-off-by: Greg Kroah-Hartman <[email protected]>

pdp7 · 2020-06-10T18:26:42Z

Please re-open if still an issue with our current Debian images:
https://github.com/beagleboard/Latest-Images

… lwt hook [ Upstream commit b02d196 ] xmit_check_hhlen() observes the dst for getting the device hard header length to make sure a modified packet can fit. When a helper which changes the dst - such as bpf_skb_set_tunnel_key() - is called as part of the xmit program the accessed dst is no longer valid. This leads to the following splat: BUG: kernel NULL pointer dereference, address: 00000000000000de #PF: supervisor read access in kernel mode #PF: error_code(0x0000) - not-present page PGD 0 P4D 0 Oops: 0000 [#1] PREEMPT SMP PTI CPU: 0 PID: 798 Comm: ping Not tainted 5.18.0-rc2+ #103 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.14.0-2 04/01/2014 RIP: 0010:bpf_xmit+0xfb/0x17f Code: c6 c0 4d cd 8e 48 c7 c7 7d 33 f0 8e e8 42 09 fb ff 48 8b 45 58 48 8b 95 c8 00 00 00 48 2b 95 c0 00 00 00 48 83 e0 fe 48 8b 00 <0f> b7 80 de 00 00 00 39 c2 73 22 29 d0 b9 20 0a 00 00 31 d2 48 89 RSP: 0018:ffffb148c0bc7b98 EFLAGS: 00010282 RAX: 0000000000000000 RBX: 0000000000240008 RCX: 0000000000000000 RDX: 0000000000000010 RSI: 00000000ffffffea RDI: 00000000ffffffff RBP: ffff922a828a4e00 R08: ffffffff8f1350e8 R09: 00000000ffffdfff R10: ffffffff8f055100 R11: ffffffff8f105100 R12: 0000000000000000 R13: ffff922a828a4e00 R14: 0000000000000040 R15: 0000000000000000 FS: 00007f414e8f0080(0000) GS:ffff922afdc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00000000000000de CR3: 0000000002d80006 CR4: 0000000000370ef0 Call Trace: <TASK> lwtunnel_xmit.cold+0x71/0xc8 ip_finish_output2+0x279/0x520 ? __ip_finish_output.part.0+0x21/0x130 Fix by fetching the device hard header length before running the BPF code. Fixes: 3a0af8f ("bpf: BPF for lightweight tunnel infrastructure") Signed-off-by: Eyal Birger <[email protected]> Signed-off-by: Daniel Borkmann <[email protected]> Link: https://lore.kernel.org/bpf/[email protected] Signed-off-by: Sasha Levin <[email protected]>

pdp7 closed this as completed Jun 10, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

emmc failures on 4.4.9+ / 4.4.21+ #103

emmc failures on 4.4.9+ / 4.4.21+ #103

floion commented Sep 24, 2016

floion commented Sep 27, 2016

floion commented Oct 27, 2016

locomundo commented Oct 31, 2016

pirvanc commented Oct 31, 2016

FRVisser commented Oct 31, 2016

RobertCNelson commented Oct 31, 2016

jaapcrezee commented Oct 31, 2016

adnanoner commented Nov 1, 2016

floion commented Nov 2, 2016

RobertCNelson commented Nov 2, 2016

floion commented Mar 27, 2017

floion commented Mar 27, 2017

RobertCNelson commented Mar 28, 2017

floion commented Mar 28, 2017

willnewton commented Jun 16, 2017

willnewton commented Jun 20, 2017

RobertCNelson commented Jun 26, 2017

pdp7 commented Jun 10, 2020

emmc failures on 4.4.9+ / 4.4.21+ #103

emmc failures on 4.4.9+ / 4.4.21+ #103

Comments

floion commented Sep 24, 2016

floion commented Sep 27, 2016

floion commented Oct 27, 2016

locomundo commented Oct 31, 2016

pirvanc commented Oct 31, 2016

FRVisser commented Oct 31, 2016

RobertCNelson commented Oct 31, 2016

jaapcrezee commented Oct 31, 2016

adnanoner commented Nov 1, 2016

floion commented Nov 2, 2016

RobertCNelson commented Nov 2, 2016

floion commented Mar 27, 2017

floion commented Mar 27, 2017

RobertCNelson commented Mar 28, 2017

floion commented Mar 28, 2017

willnewton commented Jun 16, 2017

willnewton commented Jun 20, 2017

RobertCNelson commented Jun 26, 2017

pdp7 commented Jun 10, 2020