Bug 5740 - [ANCK-5.10-15][rc2][Alinux3][x86_64] 执行 stress-ng --io-uring 2 -t 20s 指令发生crash,Kernel panic - not syncing: Fatal exception
Summary: [ANCK-5.10-15][rc2][Alinux3][x86_64] 执行 stress-ng --io-uring 2 -t 20s 指令发生cra...
Status: NEW
Alias: None
Product: ANCK 5.10 Dev
Classification: ANCK
Component: block/storage (show other bugs) block/storage
Version: 5.10.y-15
Hardware: All Linux
: P3-Medium S3-normal
Target Milestone: ---
Assignee: Ferry Meng
QA Contact: shuming
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-07-04 19:42 UTC by yy1023
Modified: 2023-07-17 17:38 UTC (History)
7 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description yy1023 2023-07-04 19:42:23 UTC
[缺陷描述]:
执行 stress-ng --io-uring 2 -t 20s 指令发生crash

[重现环境]:
# cat /etc/os-release
NAME="Alibaba Cloud Linux"
VERSION="3 (Soaring Falcon)"
ID="alinux"
ID_LIKE="rhel fedora centos anolis"
VERSION_ID="3"
PLATFORM_ID="platform:al8"
PRETTY_NAME="Alibaba Cloud Linux 3 (Soaring Falcon)"
ANSI_COLOR="0;31"
HOME_URL="https://www.aliyun.com/"
How reproducible:

# uname -r
5.10.134-15_rc2.al8.x86_64

# free -h
              total        used        free      shared  buff/cache   available
Mem:          7.3Gi       758Mi       4.3Gi       1.0Mi       2.2Gi       6.3Gi
Swap:            0B          0B          0B

# free -h
              total        used        free      shared  buff/cache   available
Mem:          7.3Gi       758Mi       4.3Gi       1.0Mi       2.2Gi       6.3Gi
Swap:            0B          0B          0B
[root@iZbp1esyiqa4v7lvgcycxtZ kernels]# lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              4
On-line CPU(s) list: 0-3
Thread(s) per core:  2
Core(s) per socket:  2
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
BIOS Vendor ID:      Alibaba Cloud
CPU family:          6
Model:               85
Model name:          Intel(R) Xeon(R) Platinum 8269CY CPU @ 2.50GHz
BIOS Model name:     pc-i440fx-2.1
Stepping:            7
CPU MHz:             2499.998
BogoMIPS:            4999.99
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           32K
L1i cache:           32K
L2 cache:            1024K
L3 cache:            36608K
NUMA node0 CPU(s):   0-3
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves arat avx512_vnni

# rpm -qa | grep -E "crash|kexec-tools"
kexec-tools-2.0.24-6.1.al8.x86_64
crash-7.3.1-5.al8.x86_64

[复现步骤]:
# 下载最新stress-ng工具
git clone https://github.com/ColinIanKing/stress-ng.git  
cd stress-ng
make && make install
Actual results:
# 执行测试
nohup stress-ng --io-uring 2 -t 20s


[预期结果]:
stress-ng --io-uring 2 执行正常,不会发生crash

[实际结果]:
执行 stress-ng --io-uring 2 -t 20s 指令发生crash,vmcore解析如下:
[root@iZbp1esyiqa4v7lvgcycxtZ kernels]# crash /lib/debug/lib/modules/5.10.134-15_rc2.al8.x86_64/vmlinux  /home/qiaoyun/FIT/127.0.0.1-2023-07-04-16\:37\:15/vmcore

crash 7.3.1-5.al8
Copyright (C) 2002-2021  Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
Copyright (C) 1999-2006  Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011, 2020-2021  NEC Corporation
Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions.  Enter "help copying" to see the conditions.
This program has absolutely no warranty.  Enter "help warranty" for details.
 
GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...

WARNING: kernel relocated [848MB]: patching 126601 gdb minimal_symbol values

      KERNEL: /lib/debug/lib/modules/5.10.134-15_rc2.al8.x86_64/vmlinux  [TAINTED]
    DUMPFILE: /home/qiaoyun/FIT/127.0.0.1-2023-07-04-16:37:15/vmcore  [PARTIAL DUMP]
        CPUS: 4
        DATE: Tue Jul  4 16:38:35 CST 2023
      UPTIME: 00:19:58
LOAD AVERAGE: 42732.92, 31053.23, 14370.27
       TASKS: 25859
    NODENAME: iZbp1esyiqa4v7lvgcycxtZ
     RELEASE: 5.10.134-15_rc2.al8.x86_64
     VERSION: #1 SMP Fri Jun 30 17:17:09 CST 2023
     MACHINE: x86_64  (2499 Mhz)
      MEMORY: 7.7 GB
       PANIC: "Kernel panic - not syncing: Fatal exception"
         PID: 1839
     COMMAND: "AliYunDunMonito"
        TASK: ffff9dd185460000  [THREAD_INFO: ffff9dd185460000]
         CPU: 3
       STATE: TASK_RUNNING (PANIC)

crash> bt
PID: 1839   TASK: ffff9dd185460000  CPU: 3   COMMAND: "AliYunDunMonito"
 #0 [ffffad35c0dbbb78] machine_kexec at ffffffffb605fb30
 #1 [ffffad35c0dbbbc0] __crash_kexec at ffffffffb61a4cda
 #2 [ffffad35c0dbbc80] panic at ffffffffb6a8a506
 #3 [ffffad35c0dbbd18] exc_general_protection at ffffffffb6acf869
 #4 [ffffad35c0dbbdb0] asm_exc_general_protection at ffffffffb6c00a3e
    [exception RIP: proc_readfd_common+166]
    RIP: ffffffffb6438a96  RSP: ffffad35c0dbbe60  RFLAGS: 00010206
    RAX: 4900004500080000  RBX: 0000000000000021  RCX: 0000000000000021
    RDX: ffff9dd183a70000  RSI: 00000000029647f0  RDI: ffff9dd227fdd2f8
    RBP: ffffad35c0dbbef0   R8: 0000000000000018   R9: 0000000000000000
    R10: 0000000000000000  R11: 0000000000000000  R12: ffff9dd227fdc480
    R13: ffff9dd186012c00  R14: ffff9dd0a66bdb00  R15: ffffffffb6438360
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018
 #5 [ffffad35c0dbbeb0] iterate_dir at ffffffffb63a3f8c
 #6 [ffffad35c0dbbee8] __x64_sys_getdents64 at ffffffffb63a4c18
 #7 [ffffad35c0dbbf40] do_syscall_64 at ffffffffb6acee73
 #8 [ffffad35c0dbbf50] entry_SYSCALL_64_after_hwframe at ffffffffb6c00099
    RIP: 00007f1df5cee09b  RSP: 00007f1df19fe9b8  RFLAGS: 00000293
    RAX: ffffffffffffffda  RBX: 0000000002964790  RCX: 00007f1df5cee09b
    RDX: 0000000000008000  RSI: 00000000029647c0  RDI: 000000000000001f
    RBP: 00000000029647c0   R8: 0000000000000030   R9: 00007f1df5df9a60
    R10: 0000000000400000  R11: 0000000000000293  R12: ffffffffffffff38
    R13: 0000000002964794  R14: 0000000000000002  R15: 00007f1df6e5a980
    ORIG_RAX: 00000000000000d9  CS: 0033  SS: 002b
crash> 

vmcore-dmesg的信息:
[    0.000000] Linux version 5.10.134-15_rc2.al8.x86_64 (mockbuild@kojid011139182114.na61) (gcc (GCC) 10.2.1 20200825 (Alibaba 10.2.1-3.5 2.32), GNU ld version 2.35-12.2.al8) #1 SMP Fri Jun 30 17:17:09 CST 2023
[    0.000000] Command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.10.134-15_rc2.al8.x86_64 root=UUID=3bda0135-1a08-4c3e-8216-e07c05dd073a ro cryptomgr.notests rcupdate.rcu_cpu_stall_timeout=300 vring_force_dma_api rhgb quiet biosdevname=0 net.ifnames=0 console=tty0 console=ttyS0,115200n8 noibrs nvme_core.io_timeout=4294967295 nvme_core.admin_timeout=4294967295 cgroup.memory=nokmem crashkernel=0M-2G:0M,2G-8G:192M,8G-:256M
[    0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x008: 'MPX bounds registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x010: 'MPX CSR'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'
[    0.000000] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
[    0.000000] x86/fpu: xstate_offset[3]:  832, xstate_sizes[3]:   64
[    0.000000] x86/fpu: xstate_offset[4]:  896, xstate_sizes[4]:   64
[    0.000000] x86/fpu: xstate_offset[5]:  960, xstate_sizes[5]:   64
[    0.000000] x86/fpu: xstate_offset[6]: 1024, xstate_sizes[6]:  512
[    0.000000] x86/fpu: xstate_offset[7]: 1536, xstate_sizes[7]: 1024
[    0.000000] x86/fpu: Enabled xstate features 0xff, context size is 2560 bytes, using 'compacted' format.
[    0.000000] signal: max sigframe size: 3632
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x0000000013ffffff] usable
[    0.000000] BIOS-e820: [mem 0x0000000014000000-0x000000001511ffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000015120000-0x00000000bffcdfff] usable
[    0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000022fffffff] usable
[    0.000000] BIOS-e820: [mem 0x0000000230000000-0x000000023fffffff] reserved
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] SMBIOS 2.8 present.
[    0.000000] DMI: Alibaba Cloud Alibaba Cloud ECS, BIOS 449e491 04/01/2014
[    0.000000] Hypervisor detected: KVM
[    0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
[    0.000000] kvm-clock: cpu 0, msr 25001001, primary cpu clock
[    0.000000] kvm-clock: using sched offset of 518899930217894 cycles
[    0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
[    0.000004] tsc: Detected 2499.998 MHz processor
[    0.001031] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
[    0.001034] e820: remove [mem 0x000a0000-0x000fffff] usable
[    0.001040] last_pfn = 0x230000 max_arch_pfn = 0x400000000
[    0.001061] MTRR default type: write-back
[    0.001062] MTRR fixed ranges enabled:
[    0.001063]   00000-9FFFF write-back
[    0.001064]   A0000-BFFFF uncachable
[    0.001065]   C0000-FFFFF write-protect
[    0.001065] MTRR variable ranges enabled:
[    0.001067]   0 base 0000C0000000 mask 3FFFC0000000 uncachable
[    0.001067]   1 disabled
[    0.001068]   2 disabled
[    0.001068]   3 disabled
[    0.001069]   4 disabled
[    0.001069]   5 disabled
[    0.001070]   6 disabled
[    0.001070]   7 disabled
[    0.001077] x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB  WP  UC- WT  
[    0.001083] last_pfn = 0xbffce max_arch_pfn = 0x400000000
[    0.004814] found SMP MP-table at [mem 0x000f5a10-0x000f5a1f]
[    0.004896] kexec: Reserving the low 1M of memory for crashkernel
[    0.004926] Using GB pages for direct mapping
[    0.005163] RAMDISK: [mem 0x34783000-0x363b9fff]
[    0.005166] ACPI: Early table checksum verification disabled
[    0.005168] ACPI: RSDP 0x00000000000F57F0 000014 (v00 BOCHS )
[    0.005172] ACPI: RSDT 0x00000000BFFE1B12 000034 (v01 BOCHS  BXPCRSDT 00000001 BXPC 00000001)
[    0.005177] ACPI: FACP 0x00000000BFFE0BFE 000074 (v01 BOCHS  BXPCFACP 00000001 BXPC 00000001)
[    0.005182] ACPI: DSDT 0x00000000BFFE0040 000BBE (v01 BOCHS  BXPCDSDT 00000001 BXPC 00000001)
[    0.005184] ACPI: FACS 0x00000000BFFE0000 000040
[    0.005187] ACPI: SSDT 0x00000000BFFE0C72 000D28 (v01 BOCHS  BXPCSSDT 00000001 BXPC 00000001)
[    0.005189] ACPI: APIC 0x00000000BFFE199A 000090 (v01 BOCHS  BXPCAPIC 00000001 BXPC 00000001)
[    0.005192] ACPI: SRAT 0x00000000BFFE1A2A 0000E8 (v01 BOCHS  BXPCSRAT 00000001 BXPC 00000001)
[    0.005194] ACPI: Reserving FACP table memory at [mem 0xbffe0bfe-0xbffe0c71]
[    0.005195] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe0bfd]
[    0.005196] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f]
[    0.005196] ACPI: Reserving SSDT table memory at [mem 0xbffe0c72-0xbffe1999]
[    0.005197] ACPI: Reserving APIC table memory at [mem 0xbffe199a-0xbffe1a29]
[    0.005198] ACPI: Reserving SRAT table memory at [mem 0xbffe1a2a-0xbffe1b11]
[    0.005214] ACPI: Local APIC address 0xfee00000
[    0.005245] SRAT: PXM 0 -> APIC 0x00 -> Node 0
[    0.005246] SRAT: PXM 0 -> APIC 0x01 -> Node 0
[    0.005247] SRAT: PXM 0 -> APIC 0x02 -> Node 0
[    0.005249] SRAT: PXM 0 -> APIC 0x03 -> Node 0
[    0.005252] ACPI: SRAT: Node 0 PXM 0 [mem 0x00000000-0x0009ffff]
[    0.005253] ACPI: SRAT: Node 0 PXM 0 [mem 0x00100000-0xbfffffff]
[    0.005255] ACPI: SRAT: Node 0 PXM 0 [mem 0x100000000-0x23fffffff]
[    0.005257] NUMA: Node 0 [mem 0x00000000-0x0009ffff] + [mem 0x00100000-0xbfffffff] -> [mem 0x00000000-0xbfffffff]
[    0.005260] NUMA: Node 0 [mem 0x00000000-0xbfffffff] + [mem 0x100000000-0x22fffffff] -> [mem 0x00000000-0x22fffffff]
[    0.005275] NODE_DATA(0) allocated [mem 0x22ffd2000-0x22fffcfff]
[    0.005653] Reserving 0MB of extra memory for crashkernel
[    0.005654] Reserving 192MB of memory at 2864MB for crashkernel (System RAM: 7918MB)
[    0.005671] Zone ranges:
[    0.005672]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
[    0.005674]   DMA32    [mem 0x0000000001000000-0x00000000ffffffff]
[    0.005675]   Normal   [mem 0x0000000100000000-0x000000022fffffff]
[    0.005676]   Device   empty
[    0.005677] Movable zone start for each node
[    0.005679] Early memory node ranges
[    0.005680]   node   0: [mem 0x0000000000001000-0x000000000009efff]
[    0.005681]   node   0: [mem 0x0000000000100000-0x0000000013ffffff]
[    0.005682]   node   0: [mem 0x0000000015120000-0x00000000bffcdfff]
[    0.005683]   node   0: [mem 0x0000000100000000-0x000000022fffffff]
[    0.005685] Initmem setup node 0 [mem 0x0000000000001000-0x000000022fffffff]
[    0.005686] On node 0 totalpages: 2027084
[    0.005687]   DMA zone: 64 pages used for memmap
[    0.005687]   DMA zone: 158 pages reserved
[    0.005688]   DMA zone: 3998 pages, LIFO batch:0
[    0.005689]   DMA32 zone: 12155 pages used for memmap
[    0.005690]   DMA32 zone: 777902 pages, LIFO batch:63
[    0.005691]   Normal zone: 19456 pages used for memmap
[    0.005691]   Normal zone: 1245184 pages, LIFO batch:63
[    0.005697] On node 0, zone DMA: 1 pages in unavailable ranges
[    0.005766] On node 0, zone DMA: 97 pages in unavailable ranges
[    0.013458] On node 0, zone DMA32: 4384 pages in unavailable ranges
[    0.025619] On node 0, zone Normal: 50 pages in unavailable ranges
[    0.025978] ACPI: PM-Timer IO Port: 0x608
[    0.025980] ACPI: Local APIC address 0xfee00000
[    0.025988] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[    0.026009] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
[    0.026011] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.026013] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
[    0.026014] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.026015] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
[    0.026016] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
[    0.026017] ACPI: IRQ0 used by override.
[    0.026018] ACPI: IRQ5 used by override.
[    0.026018] ACPI: IRQ9 used by override.
[    0.026019] ACPI: IRQ10 used by override.
[    0.026020] ACPI: IRQ11 used by override.
[    0.026022] Using ACPI (MADT) for SMP configuration information
[    0.026024] TSC deadline timer available
[    0.026025] smpboot: Allowing 4 CPUs, 0 hotplug CPUs
[    0.026048] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]
[    0.026050] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x0009ffff]
[    0.026050] PM: hibernation: Registered nosave memory: [mem 0x000a0000-0x000effff]
[    0.026051] PM: hibernation: Registered nosave memory: [mem 0x000f0000-0x000fffff]
[    0.026052] PM: hibernation: Registered nosave memory: [mem 0x14000000-0x1511ffff]
[    0.026054] PM: hibernation: Registered nosave memory: [mem 0xbffce000-0xbfffffff]
[    0.026054] PM: hibernation: Registered nosave memory: [mem 0xc0000000-0xfeffbfff]
[    0.026055] PM: hibernation: Registered nosave memory: [mem 0xfeffc000-0xfeffffff]
[    0.026056] PM: hibernation: Registered nosave memory: [mem 0xff000000-0xfffbffff]
[    0.026056] PM: hibernation: Registered nosave memory: [mem 0xfffc0000-0xffffffff]
[    0.026058] [mem 0xc0000000-0xfeffbfff] available for PCI devices
[    0.026059] Booting paravirtualized kernel on KVM
[    0.026062] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns
[    0.030822] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1
[    0.031189] percpu: Embedded 65 pages/cpu s229376 r8192 d28672 u524288
[    0.031194] pcpu-alloc: s229376 r8192 d28672 u524288 alloc=1*2097152
[    0.031196] pcpu-alloc: [0] 0 1 2 3 
[    0.031219] kvm-guest: stealtime: cpu 0, msr 228037080
[    0.031225] Built 1 zonelists, mobility grouping on.  Total pages: 1995251
[    0.031226] Policy zone: Normal
[    0.031228] Kernel command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.10.134-15_rc2.al8.x86_64 root=UUID=3bda0135-1a08-4c3e-8216-e07c05dd073a ro cryptomgr.notests rcupdate.rcu_cpu_stall_timeout=300 vring_force_dma_api rhgb quiet biosdevname=0 net.ifnames=0 console=tty0 console=ttyS0,115200n8 noibrs nvme_core.io_timeout=4294967295 nvme_core.admin_timeout=4294967295 cgroup.memory=nokmem crashkernel=0M-2G:0M,2G-8G:192M,8G-:256M
[    0.031279] Force vring dma api enabled
[    0.032581] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)
[    0.033175] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[    0.033234] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.042698] software IO TLB: area num 4.
[    0.066536] Memory: 7624072K/8108336K available (14346K kernel code, 9592K rwdata, 9324K rodata, 2684K init, 7440K bss, 484004K reserved, 0K cma-reserved)
[    0.066695] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.066707] Kernel/User page tables isolation: enabled
[    0.066722] ftrace: allocating 45264 entries in 177 pages
[    0.075159] ftrace: allocated 177 pages with 4 groups
[    0.075270] rcu: Hierarchical RCU implementation.
[    0.075271] rcu:     RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4.
[    0.075273]  RCU CPU stall warnings timeout set to 300 (rcu_cpu_stall_timeout).
[    0.075273]  Rude variant of Tasks RCU enabled.
[    0.075273]  Tracing variant of Tasks RCU enabled.
[    0.075274] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.
[    0.075275] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
[    0.079226] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16
[    0.088069] Console: colour VGA+ 80x25
[    0.088082] printk: console [tty0] enabled
[    0.088125] printk: console [ttyS0] enabled
[    0.088148] ACPI: Core revision 20200925
[    0.088203] APIC: Switch to symmetric I/O mode setup
[    0.088311] try_to_enable_x2apic: remap_mode: -38
[    0.088323] x2apic enabled
[    0.088489] Switched APIC routing to physical x2apic.
[    0.089045] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x240937b9988, max_idle_ns: 440795218083 ns
[    0.089050] Calibrating delay loop (skipped) preset value.. 4999.99 BogoMIPS (lpj=2499998)
[    0.089052] pid_max: default: 32768 minimum: 301
[    0.089094] LSM: Security Framework initializing
[    0.089115] SELinux:  Initializing.
[    0.089178] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)
[    0.089193] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)
[    0.089555] Last level iTLB entries: 4KB 64, 2MB 8, 4MB 8
[    0.089556] Last level dTLB entries: 4KB 64, 2MB 0, 4MB 0, 1GB 4
[    0.089559] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
[    0.089561] Spectre V2 : Mitigation: Retpolines
[    0.089561] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
[    0.089562] Spectre V2 : Spectre v2 / SpectreRSB : Filling RSB on VMEXIT
[    0.089563] RETBleed: WARNING: Spectre v2 mitigation leaves CPU vulnerable to RETBleed attacks, data leaks possible!
[    0.090046] RETBleed: Vulnerable
[    0.090046] Speculative Store Bypass: Vulnerable
[    0.090046] MDS: Vulnerable: Clear CPU buffers attempted, no microcode
[    0.090046] TAA: Vulnerable: Clear CPU buffers attempted, no microcode
[    0.090046] MMIO Stale Data: Vulnerable: Clear CPU buffers attempted, no microcode
[    0.090046] Freeing SMP alternatives memory: 36K
[    0.090046] smpboot: CPU0: Intel(R) Xeon(R) Platinum 8269CY CPU @ 2.50GHz (family: 0x6, model: 0x55, stepping: 0x7)
[    0.090046] Performance Events: unsupported p6 CPU model 85 no PMU driver, software events only.
[    0.090046] rcu: Hierarchical SRCU implementation.
[    0.090046] smp: Bringing up secondary CPUs ...
[    0.090046] x86: Booting SMP configuration:
[    0.090046] .... node  #0, CPUs:      #1
[    0.010662] kvm-clock: cpu 1, msr 25001041, secondary cpu clock
[    0.090046] kvm-guest: stealtime: cpu 1, msr 2280b7080
[    0.090053] MDS CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/mds.html for more details.
[    0.090053] TAA CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/tsx_async_abort.html for more details.
[    0.090053] MMIO Stale Data CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/processor_mmio_stale_data.html for more details.
[    0.090158]  #2
[    0.010662] kvm-clock: cpu 2, msr 25001081, secondary cpu clock
[    0.090364] kvm-guest: stealtime: cpu 2, msr 228137080
[    0.090364]  #3
[    0.010662] kvm-clock: cpu 3, msr 250010c1, secondary cpu clock
[    0.090364] kvm-guest: stealtime: cpu 3, msr 2281b7080
[    0.090364] smp: Brought up 1 node, 4 CPUs
[    0.090364] smpboot: Max logical packages: 1
[    0.090364] smpboot: Total of 4 processors activated (19999.98 BogoMIPS)
[    0.091361] devtmpfs: initialized
[    0.091361] x86/mm: Memory block size: 128MB
[    0.092320] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns
[    0.092320] futex hash table entries: 1024 (order: 4, 65536 bytes, linear)
[    0.092320] pinctrl core: initialized pinctrl subsystem
[    0.092461] NET: Registered protocol family 16
[    0.092572] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations
[    0.092576] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[    0.092579] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[    0.092585] audit: initializing netlink subsys (disabled)
[    0.092592] audit: type=2000 audit(1688458716.569:1): state=initialized audit_enabled=0 res=1
[    0.093053] thermal_sys: Registered thermal governor 'fair_share'
[    0.093054] thermal_sys: Registered thermal governor 'bang_bang'
[    0.093055] thermal_sys: Registered thermal governor 'step_wise'
[    0.093055] thermal_sys: Registered thermal governor 'user_space'
[    0.093074] cpuidle: using governor menu
[    0.093247] HugeTLB: can optimize 4095 vmemmap pages for hugepages-1048576kB
[    0.093251] ACPI: bus type PCI registered
[    0.093252] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
[    0.093351] PCI: Using configuration type 1 for base access
[    0.094270] Kprobes globally optimized
[    0.094270] HugeTLB: can optimize 7 vmemmap pages for hugepages-2048kB
[    0.094270] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[    0.094270] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    0.095095] alg: self-tests disabled
[    0.096051] cryptd: max_cpu_qlen set to 1000
[    0.098106] ACPI: Added _OSI(Module Device)
[    0.098107] ACPI: Added _OSI(Processor Device)
[    0.098107] ACPI: Added _OSI(3.0 _SCP Extensions)
[    0.098108] ACPI: Added _OSI(Processor Aggregator Device)
[    0.098109] ACPI: Added _OSI(Linux-Dell-Video)
[    0.098110] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio)
[    0.098110] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics)
[    0.098524] ACPI: 2 ACPI AML tables successfully acquired and loaded
[    0.099094] ACPI: Interpreter enabled
[    0.099101] ACPI: (supports S0 S5)
[    0.099102] ACPI: Using IOAPIC for interrupt routing
[    0.099112] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[    0.099251] ACPI: Enabled 16 GPEs in block 00 to 0F
[    0.101066] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[    0.101071] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI EDR HPX-Type3]
[    0.101077] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
[    0.101299] acpiphp: Slot [3] registered
[    0.101311] acpiphp: Slot [4] registered
[    0.101323] acpiphp: Slot [5] registered
[    0.101337] acpiphp: Slot [6] registered
[    0.101348] acpiphp: Slot [7] registered
[    0.101362] acpiphp: Slot [8] registered
[    0.101373] acpiphp: Slot [9] registered
[    0.101384] acpiphp: Slot [10] registered
[    0.101396] acpiphp: Slot [11] registered
[    0.101411] acpiphp: Slot [12] registered
[    0.101424] acpiphp: Slot [13] registered
[    0.101442] acpiphp: Slot [14] registered
[    0.101453] acpiphp: Slot [15] registered
[    0.101464] acpiphp: Slot [16] registered
[    0.101475] acpiphp: Slot [17] registered
[    0.101486] acpiphp: Slot [18] registered
[    0.101496] acpiphp: Slot [19] registered
[    0.101510] acpiphp: Slot [20] registered
[    0.101521] acpiphp: Slot [21] registered
[    0.101532] acpiphp: Slot [22] registered
[    0.101543] acpiphp: Slot [23] registered
[    0.101554] acpiphp: Slot [24] registered
[    0.101565] acpiphp: Slot [25] registered
[    0.101575] acpiphp: Slot [26] registered
[    0.101589] acpiphp: Slot [27] registered
[    0.101600] acpiphp: Slot [28] registered
[    0.101611] acpiphp: Slot [29] registered
[    0.101621] acpiphp: Slot [30] registered
[    0.101634] acpiphp: Slot [31] registered
[    0.101642] PCI host bridge to bus 0000:00
[    0.101643] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7 window]
[    0.101644] pci_bus 0000:00: root bus resource [io  0x0d00-0xadff window]
[    0.101645] pci_bus 0000:00: root bus resource [io  0xae0f-0xaeff window]
[    0.101646] pci_bus 0000:00: root bus resource [io  0xaf20-0xafdf window]
[    0.101647] pci_bus 0000:00: root bus resource [io  0xafe4-0xffff window]
[    0.101648] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]
[    0.101648] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]
[    0.101649] pci_bus 0000:00: root bus resource [bus 00-ff]
[    0.101691] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000
[    0.102025] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100
[    0.102398] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180
[    0.103611] pci 0000:00:01.1: reg 0x20: [io  0xc060-0xc06f]
[    0.104055] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io  0x01f0-0x01f7]
[    0.104056] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io  0x03f6]
[    0.104057] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io  0x0170-0x0177]
[    0.104057] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io  0x0376]
[    0.104185] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300
[    0.105259] pci 0000:00:01.2: reg 0x20: [io  0xc000-0xc01f]
[    0.105855] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000
[    0.106131] pci 0000:00:01.3: quirk: [io  0x0600-0x063f] claimed by PIIX4 ACPI
[    0.106137] pci 0000:00:01.3: quirk: [io  0x0700-0x070f] claimed by PIIX4 SMB
[    0.106305] pci 0000:00:02.0: [1013:00b8] type 00 class 0x030000
[    0.107356] pci 0000:00:02.0: reg 0x10: [mem 0xfc000000-0xfdffffff pref]
[    0.107899] pci 0000:00:02.0: reg 0x14: [mem 0xfebf0000-0xfebf0fff]
[    0.111031] pci 0000:00:02.0: reg 0x30: [mem 0xfebe0000-0xfebeffff pref]
[    0.111223] pci 0000:00:03.0: [1af4:1003] type 00 class 0x078000
[    0.111699] pci 0000:00:03.0: reg 0x10: [io  0xc020-0xc03f]
[    0.112051] pci 0000:00:03.0: reg 0x14: [mem 0xfebf1000-0xfebf1fff]
[    0.114342] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000
[    0.114981] pci 0000:00:04.0: reg 0x10: [mem 0xfe000000-0xfe000fff 64bit pref]
[    0.115323] pci 0000:00:04.0: reg 0x18: [mem 0xfe001000-0xfe001fff 64bit pref]
[    0.117178] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000
[    0.117776] pci 0000:00:05.0: reg 0x10: [mem 0xfe002000-0xfe002fff 64bit pref]
[    0.118310] pci 0000:00:05.0: reg 0x18: [mem 0xfe003000-0xfe003fff 64bit pref]
[    0.120128] pci 0000:00:06.0: [1af4:1000] type 00 class 0x020000
[    0.120728] pci 0000:00:06.0: reg 0x10: [mem 0xfe004000-0xfe004fff 64bit pref]
[    0.121058] pci 0000:00:06.0: reg 0x18: [mem 0xfe005000-0xfe005fff 64bit pref]
[    0.122840] pci 0000:00:07.0: [1af4:1002] type 00 class 0x00ff00
[    0.123051] pci 0000:00:07.0: reg 0x10: [io  0xc040-0xc05f]
[    0.124905] ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11)
[    0.124969] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
[    0.125022] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
[    0.125078] ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11)
[    0.125107] ACPI: PCI Interrupt Link [LNKS] (IRQs *9)
[    0.126084] iommu: Default domain type: Passthrough 
[    0.126084] pci 0000:00:02.0: vgaarb: setting as boot VGA device
[    0.126084] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none
[    0.126084] pci 0000:00:02.0: vgaarb: bridge control possible
[    0.126084] vgaarb: loaded
[    0.126184] SCSI subsystem initialized
[    0.126210] ACPI: bus type USB registered
[    0.126219] usbcore: registered new interface driver usbfs
[    0.126230] usbcore: registered new interface driver hub
[    0.126235] usbcore: registered new device driver usb
[    0.126253] pps_core: LinuxPPS API ver. 1 registered
[    0.126254] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.126255] PTP clock support registered
[    0.126271] EDAC MC: Ver: 3.0.0
[    0.126271] NetLabel: Initializing
[    0.126271] NetLabel:  domain hash size = 128
[    0.126271] NetLabel:  protocols = UNLABELED CIPSOv4 CALIPSO
[    0.126271] NetLabel:  unlabeled traffic allowed by default
[    0.126271] PCI: Using ACPI for IRQ routing
[    0.126271] PCI: pci_cache_line_size set to 64 bytes
[    0.126278] e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff]
[    0.126279] e820: reserve RAM buffer [mem 0xbffce000-0xbfffffff]
[    0.127057] clocksource: Switched to clocksource kvm-clock
[    0.142023] VFS: Disk quotas dquot_6.6.0
[    0.142040] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.142110] pnp: PnP ACPI init
[    0.142166] pnp 00:00: Plug and Play ACPI device, IDs PNP0b00 (active)
[    0.142190] pnp 00:01: Plug and Play ACPI device, IDs PNP0303 (active)
[    0.142205] pnp 00:02: Plug and Play ACPI device, IDs PNP0f13 (active)
[    0.142222] pnp 00:03: [dma 2]
[    0.142229] pnp 00:03: Plug and Play ACPI device, IDs PNP0700 (active)
[    0.142285] pnp 00:04: Plug and Play ACPI device, IDs PNP0501 (active)
[    0.142420] pnp: PnP ACPI: found 5 devices
[    0.149166] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[    0.149234] NET: Registered protocol family 2
[    0.149481] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)
[    0.150603] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)
[    0.150617] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)
[    0.150688] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes, linear)
[    0.150793] TCP: Hash tables configured (established 65536 bind 65536)
[    0.150908] MPTCP token hash table entries: 8192 (order: 5, 196608 bytes, linear)
[    0.150929] UDP hash table entries: 4096 (order: 5, 131072 bytes, linear)
[    0.150950] UDP-Lite hash table entries: 4096 (order: 5, 131072 bytes, linear)
[    0.151003] NET: Registered protocol family 1
[    0.151009] NET: Registered protocol family 44
[    0.151020] pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7 window]
[    0.151022] pci_bus 0000:00: resource 5 [io  0x0d00-0xadff window]
[    0.151022] pci_bus 0000:00: resource 6 [io  0xae0f-0xaeff window]
[    0.151023] pci_bus 0000:00: resource 7 [io  0xaf20-0xafdf window]
[    0.151024] pci_bus 0000:00: resource 8 [io  0xafe4-0xffff window]
[    0.151025] pci_bus 0000:00: resource 9 [mem 0x000a0000-0x000bffff window]
[    0.151026] pci_bus 0000:00: resource 10 [mem 0xc0000000-0xfebfffff window]
[    0.151087] pci 0000:00:01.0: PIIX3: Enabling Passive Release
[    0.151096] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
[    0.151104] pci 0000:00:01.0: Activating ISA DMA hang workarounds
[    0.166457] PCI Interrupt Link [LNKD] enabled at IRQ 11
[    0.181992] pci 0000:00:01.2: quirk_usb_early_handoff+0x0/0x100 took 30142 usecs
[    0.182043] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
[    0.182113] PCI: CLS 0 bytes, default 64
[    0.202131] Trying to unpack rootfs image as initramfs...
[    0.583371] Freeing initrd memory: 28892K
[    0.583381] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[    0.583383] software IO TLB: mapped [mem 0x00000000af000000-0x00000000b3000000] (64MB)
[    0.583462] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x240937b9988, max_idle_ns: 440795218083 ns
[    0.583558] clocksource: Switched to clocksource tsc
[    0.586705] Initialise system trusted keyrings
[    0.586717] Key type blacklist registered
[    0.586776] workingset: timestamp_bits=36 max_order=21 bucket_order=0
[    0.588077] zbud: loaded
[    0.588818] integrity: Platform Keyring initialized
[    0.588819] Allocating IMA blacklist keyring.
[    0.595953] NET: Registered protocol family 38
[    0.595956] Key type asymmetric registered
[    0.595957] Asymmetric key parser 'x509' registered
[    0.595967] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 245)
[    0.596023] io scheduler mq-deadline registered
[    0.596024] io scheduler kyber registered
[    0.596067] io scheduler bfq registered
[    0.596480] atomic64_test: passed for x86-64 platform with CX8 and with SSE
[    0.596552] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[    0.596675] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[    0.596702] ACPI: Power Button [PWRF]
[    0.596757] input: Sleep Button as /devices/LNXSYSTM:00/LNXSLPBN:00/input/input1
[    0.596772] ACPI: Sleep Button [SLPF]
[    0.612281] PCI Interrupt Link [LNKC] enabled at IRQ 10
[    0.612323] virtio-pci 0000:00:03.0: virtio_pci: leaving for legacy driver
[    0.612613] virtio-pci 0000:00:04.0: virtio_pci: leaving for legacy driver
[    0.612753] virtio-pci 0000:00:05.0: virtio_pci: leaving for legacy driver
[    0.612873] virtio-pci 0000:00:06.0: virtio_pci: leaving for legacy driver
[    0.628279] virtio-pci 0000:00:07.0: virtio_pci: leaving for legacy driver
[    0.628636] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    0.628733] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
[    0.629238] Non-volatile memory driver v1.3
[    0.631548] virtio_blk virtio1: [vda] 83886080 512-byte logical blocks (42.9 GB/40.0 GiB)
[    0.631551] vda: detected capacity change from 0 to 42949672960
[    0.789431]  vda: vda1
[    0.797274] virtio_blk virtio2: [vdb] 83886080 512-byte logical blocks (42.9 GB/40.0 GiB)
[    0.797277] vdb: detected capacity change from 0 to 42949672960
[    0.925269] rdac: device handler registered
[    0.925371] hp_sw: device handler registered
[    0.925373] emc: device handler registered
[    0.925429] alua: device handler registered
[    0.925601] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    0.925603] ehci-pci: EHCI PCI platform driver
[    0.925630] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    0.925635] ohci-pci: OHCI PCI platform driver
[    0.925674] usbcore: registered new interface driver usbserial_generic
[    0.925677] usbserial: USB Serial support registered for generic
[    0.925716] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
[    0.926290] serio: i8042 KBD port at 0x60,0x64 irq 1
[    0.926294] serio: i8042 AUX port at 0x60,0x64 irq 12
[    0.926467] rtc_cmos 00:00: RTC can wake from S4
[    0.926968] rtc_cmos 00:00: registered as rtc0
[    0.927061] rtc_cmos 00:00: setting system clock to 2023-07-04T08:17:11 UTC (1688458631)
[    0.927077] rtc_cmos 00:00: alarms up to one day, 114 bytes nvram
[    0.927086] intel_pstate: CPU model not supported
[    0.927114] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input2
[    0.927180] cpuidle: using governor haltpoll
[    0.927213] kvm-guest: host does not support poll control
[    0.928017] kvm-guest: host upgrade recommended
[    0.928978] hid: raw HID events driver (C) Jiri Kosina
[    0.929027] usbcore: registered new interface driver usbhid
[    0.929027] usbhid: USB HID core driver
[    0.929062] drop_monitor: Initializing network drop monitor service
[    0.929122] Initializing XFRM netlink socket
[    0.929234] NET: Registered protocol family 10
[    0.929489] Segment Routing with IPv6
[    0.929503] NET: Registered protocol family 17
[    0.929558] mpls_gso: MPLS GSO support
[    0.929840] No MBM correction factor available
[    0.929843] IPI shorthand broadcast: enabled
[    0.929849] AVX2 version of gcm_enc/dec engaged.
[    0.929850] AES CTR mode by8 optimization enabled
[    0.930227] sched_clock: Marking stable (920559980, 9662927)->(945309974, -15087067)
[    0.930303] registered taskstats version 1
[    0.930385] Loading compiled-in X.509 certificates
[    0.930649] Loaded X.509 cert 'Alibaba Cloud: Aliyun Linux Driver update signing key: 9c4b08a61ce9d8b631bb4f728b187a13dc95fa34'
[    0.930673] Loaded X.509 cert 'Alibaba Cloud: Aliyun Linux Driver update signing key: 0b44eb91d8412f625d42a32c9a6d4efb83040852'
[    0.930691] zswap: loaded using pool lzo/zbud
[    0.933328] Key type encrypted registered
[    0.933336] ima: No TPM chip found, activating TPM-bypass!
[    0.933346] ima: Allocated hash algorithm: sha256
[    0.933357] ima: No architecture policies found
[    0.933366] evm: Initialising EVM extended attributes:
[    0.933367] evm: security.selinux
[    0.933368] evm: security.SMACK64
[    0.933368] evm: security.SMACK64EXEC
[    0.933368] evm: security.SMACK64TRANSMUTE
[    0.933369] evm: security.SMACK64MMAP
[    0.933369] evm: security.ima
[    0.933370] evm: security.capability
[    0.933371] evm: HMAC attrs: 0x1
[    0.933687] RAS: Correctable Errors collector initialized.
[    0.933798] NMI watchdog: Perf NMI watchdog permanently disabled
[    0.934069] integrity: Unable to open file: /etc/keys/x509_ima.der (-2)
[    0.934072] integrity: Unable to open file: /etc/keys/x509_evm.der (-2)
[    0.935903] Freeing unused decrypted memory: 2036K
[    0.937228] Freeing unused kernel image (initmem) memory: 2684K
[    0.942064] Write protecting the kernel read-only data: 26624k
[    0.942645] Freeing unused kernel image (text/rodata gap) memory: 2036K
[    0.942911] Freeing unused kernel image (rodata/data gap) memory: 916K
[    0.942935] Run /init as init process
[    0.942936]   with arguments:
[    0.942937]     /init
[    0.942937]     vring_force_dma_api
[    0.942938]     rhgb
[    0.942938]     noibrs
[    0.942939]   with environment:
[    0.942940]     HOME=/
[    0.942940]     TERM=linux
[    0.942941]     BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.10.134-15_rc2.al8.x86_64
[    0.942941]     biosdevname=0
[    0.942942]     crashkernel=0M-2G:0M,2G-8G:192M,8G-:256M
[    0.951509] systemd[1]: systemd 239 (239-68.0.2.al8.1) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy)
[    0.951553] systemd[1]: Detected virtualization kvm.
[    0.951556] systemd[1]: Detected architecture x86-64.
[    0.951557] systemd[1]: Running in initial RAM disk.
[    0.962927] systemd[1]: Set hostname to <iZbp1esyiqa4v7lvgcycxtZ>.
[    1.003462] random: systemd: uninitialized urandom read (16 bytes read)
[    1.003475] systemd[1]: Reached target Local File Systems.
[    1.003517] random: systemd: uninitialized urandom read (16 bytes read)
[    1.003593] systemd[1]: Listening on Journal Socket (/dev/log).
[    1.003612] random: systemd: uninitialized urandom read (16 bytes read)
[    1.003701] systemd[1]: Listening on Journal Socket.
[    1.004544] systemd[1]: Started Hardware RNG Entropy Gatherer Daemon.
[    1.005243] systemd[1]: Started Memstrack Anylazing Service.
[    2.010306] random: systemd: uninitialized urandom read (16 bytes read)
[    2.010972] random: systemd: uninitialized urandom read (16 bytes read)
[    2.014097] random: systemd-journal: uninitialized urandom read (16 bytes read)
[    2.033184] libata version 3.00 loaded.
[    2.035509] ata_piix 0000:00:01.1: version 2.13
[    2.053113] scsi host0: ata_piix
[    2.064642] scsi host1: ata_piix
[    2.064679] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc060 irq 14
[    2.064680] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc068 irq 15
[    2.204645] random: crng init done
[    2.204647] random: 82 urandom warning(s) missed due to ratelimiting
[    2.234160] cirrus 0000:00:02.0: vgaarb: deactivate vga console
[    2.238874] Console: switching to colour dummy device 80x25
[    2.239338] EXT4-fs (vda1): mounted filesystem with ordered data mode. Opts: (null)
[    2.246105] [drm] Initialized cirrus 2.0.0 2019 for 0000:00:02.0 on minor 0
[    2.246736] fbcon: cirrusdrmfb (fb0) is primary device
[    2.252530] Console: switching to colour frame buffer device 128x48
[    2.253346] cirrus 0000:00:02.0: [drm] fb0: cirrusdrmfb frame buffer device
[    3.011140] printk: systemd: 17 output lines suppressed due to ratelimiting
[    3.107848] SELinux:  Runtime disable is deprecated, use selinux=0 on the kernel cmdline.
[    3.108756] SELinux:  Disabled at runtime.
[    3.137121] audit: type=1404 audit(1688458633.710:2): enforcing=0 old_enforcing=0 auid=4294967295 ses=4294967295 enabled=0 old-enabled=1 lsm=selinux res=1
[    3.148199] systemd[1]: systemd 239 (239-68.0.2.al8.1) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy)
[    3.148237] systemd[1]: Detected virtualization kvm.
[    3.148241] systemd[1]: Detected architecture x86-64.
[    3.148869] systemd[1]: Set hostname to <iZbp1esyiqa4v7lvgcycxtZ>.
[    3.274438] systemd[1]: Configuration file /etc/systemd/system/cloudmonitor.service is marked executable. Please remove executable permission bits. Proceeding anyway.
[    3.321232] systemd[1]: systemd-journald.service: Succeeded.
[    3.322559] systemd[1]: initrd-switch-root.service: Succeeded.
[    3.323084] systemd[1]: Stopped Switch Root.
[    3.323345] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart.
[    3.323418] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.
[    3.338320] EXT4-fs (vda1): re-mounted. Opts: (null)
[    3.406073] RPC: Registered named UNIX socket transport module.
[    3.406075] RPC: Registered udp transport module.
[    3.406076] RPC: Registered tcp transport module.
[    3.406077] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    3.440708] Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
[    4.043680] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0
[    4.054360] input: PC Speaker as /devices/platform/pcspkr/input/input4
[    4.062589] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6
[    4.062733] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5
[    4.120075] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer
[    4.229881] mousedev: PS/2 mouse device common for all mice
[    7.772334] process 'local/aegis/aegis_update/AliYunDunUpdate' started with executable stack
[    9.478910] NFSD: Using nfsdcld client tracking operations.
[    9.478913] NFSD: no clients to reclaim, skipping NFSv4 grace period (net f000009a)
[ 1060.983677] watchdog: BUG: soft lockup - CPU#1 stuck for 112s! [stress-ng-io-ur:3315]
[ 1060.984324] Modules linked in: tcp_diag inet_diag rfkill intel_rapl_msr intel_rapl_common joydev mousedev isst_if_common nfit crct10dif_pclmul crc32_pclmul ghash_clmulni_intel rapl psmouse pcspkr virtio_balloon i2c_piix4 nf_tables libcrc32c nfnetlink nfsd auth_rpcgss nfs_acl lockd grace sunrpc cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ata_generic crc32c_intel ata_piix drm libata i2c_core virtio_net net_failover failover serio_raw virtio_console
[ 1060.984349] CPU: 1 PID: 3315 Comm: stress-ng-io-ur Kdump: loaded Not tainted 5.10.134-15_rc2.al8.x86_64 #1
[ 1060.984350] Hardware name: Alibaba Cloud Alibaba Cloud ECS, BIOS 449e491 04/01/2014
[ 1060.984353] RIP: 0010:_raw_spin_unlock_irqrestore+0xa/0x10
[ 1060.984354] Code: 28 64 ff 4c 29 e0 4c 39 f0 76 d8 80 4d 00 04 eb 8e cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc 0f 1f 44 00 00 c6 07 00 56 9d <c3> cc cc cc cc 90 0f 1f 44 00 00 b8 00 fe ff ff f0 0f c1 07 56 9d
[ 1060.984355] RSP: 0018:ffffad35c0a7bc68 EFLAGS: 00000202
[ 1060.984357] RAX: ffff9dd2312411d0 RBX: ffff9dd231241ad0 RCX: 0000000000020100
[ 1060.984358] RDX: ffff9dd186012c00 RSI: 0000000000000202 RDI: ffff9dd185576000
[ 1060.984359] RBP: 0000000000000202 R08: 0000000000000000 R09: ffffffffb63fb000
[ 1060.984359] R10: ffff9dd231241800 R11: 0000000000000001 R12: ffff9dd231241bd0
[ 1060.984360] R13: ffffad35c0a7bca0 R14: ffff9dd185576000 R15: ffff9dd182603800
[ 1060.984361] FS:  00007f63f032e740(0000) GS:ffff9dd2a8080000(0000) knlGS:0000000000000000
[ 1060.984362] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1060.984363] CR2: 00007f8b08696078 CR3: 0000000109ece003 CR4: 00000000003706e0
[ 1060.984365] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 1060.984366] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 1060.984366] Call Trace:
[ 1060.984385]  io_wqe_cancel_pending_work+0x94/0x130
[ 1060.984388]  io_wq_cancel_cb+0x72/0x170
[ 1060.984390]  ? io_match_task+0xd0/0xd0
[ 1060.984392]  io_uring_cancel_files+0x108/0x230
[ 1060.984395]  ? add_wait_queue_exclusive+0x70/0x70
[ 1060.984397]  io_uring_cancel_task_requests+0xb7/0x130
[ 1060.984398]  __io_uring_files_cancel+0x58/0xa0
[ 1060.984400]  do_exit+0xb4/0x420
[ 1060.984401]  do_group_exit+0x33/0x90
[ 1060.984404]  get_signal+0x152/0x620
[ 1060.984407]  ? try_to_wake_up+0x1de/0x440
[ 1060.984409]  ? rwsem_mark_wake+0x1b4/0x2b0
[ 1060.984414]  arch_do_signal+0x2a/0x1d0
[ 1060.984416]  ? mntput_no_expire+0x35/0x240
[ 1060.984419]  ? do_unlinkat+0x84/0x2c0
[ 1060.984421]  ? do_unlinkat+0x84/0x2c0
[ 1060.984424]  exit_to_user_mode_loop+0x8d/0xe0
[ 1060.984425]  exit_to_user_mode_prepare+0x6e/0x80
[ 1060.984428]  syscall_exit_to_user_mode+0x12/0x40
[ 1060.984430]  entry_SYSCALL_64_after_hwframe+0x61/0xc6
[ 1060.984432] RIP: 0033:0x7f63f048792b
[ 1060.984434] Code: f0 ff ff 73 01 c3 48 8b 0d 42 05 0e 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 15 05 0e 00 f7 d8 64 89 01 48
[ 1060.984435] RSP: 002b:00007ffd72ce24e8 EFLAGS: 00000206 ORIG_RAX: 0000000000000057
[ 1060.984436] RAX: 0000000000000000 RBX: 00000000000678ff RCX: 00007f63f048792b
[ 1060.984437] RDX: 0000000000535f61 RSI: 0000000000004000 RDI: 00007ffd72ce2710
[ 1060.984438] RBP: 00007ffd72ce2510 R08: 00007f63edaf1000 R09: 00007ffd72ce2310
[ 1060.984438] R10: 00007f63f06e7da0 R11: 0000000000000206 R12: 0000000000000000
[ 1060.984439] R13: 00007ffd72ce3840 R14: 00000000000678f1 R15: 0000000000865b80
[ 1199.010195] general protection fault, probably for non-canonical address 0x4900004500080000: 0000 [#1] SMP PTI
[ 1199.011024] CPU: 3 PID: 1839 Comm: AliYunDunMonito Kdump: loaded Tainted: G             L    5.10.134-15_rc2.al8.x86_64 #1
[ 1199.011831] Hardware name: Alibaba Cloud Alibaba Cloud ECS, BIOS 449e491 04/01/2014
[ 1199.012414] RIP: 0010:proc_readfd_common+0xa6/0x230
[ 1199.012774] Code: 0f 83 88 00 00 00 49 8b 55 20 8b 02 39 d8 76 6a 89 d9 48 39 c1 48 19 c0 48 8b 52 08 21 d8 48 8d 04 c2 48 8b 00 48 85 c0 74 50 <8b> 40 44 89 44 24 04 e8 8e 5e d3 ff 89 d9 be 0b 00 00 00 48 c7 c2
[ 1199.014175] RSP: 0018:ffffad35c0dbbe60 EFLAGS: 00010206
[ 1199.014572] RAX: 4900004500080000 RBX: 0000000000000021 RCX: 0000000000000021
[ 1199.015101] RDX: ffff9dd183a70000 RSI: 00000000029647f0 RDI: ffff9dd227fdd2f8
[ 1199.015625] RBP: ffffad35c0dbbef0 R08: 0000000000000018 R09: 0000000000000000
[ 1199.016150] R10: 0000000000000000 R11: 0000000000000000 R12: ffff9dd227fdc480
[ 1199.016710] R13: ffff9dd186012c00 R14: ffff9dd0a66bdb00 R15: ffffffffb6438360
[ 1199.017232] FS:  00007f1df19ff640(0000) GS:ffff9dd2a8180000(0000) knlGS:0000000000000000
[ 1199.017834] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1199.018252] CR2: 000035ce97ce1000 CR3: 0000000106768005 CR4: 00000000003706e0
[ 1199.018784] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 1199.019329] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 1199.019856] Call Trace:
[ 1199.020048]  iterate_dir+0x16c/0x1b0
[ 1199.020322]  __x64_sys_getdents64+0x68/0x100
[ 1199.020645]  ? __ia32_sys_getdents+0x100/0x100
[ 1199.020977]  do_syscall_64+0x33/0x40
[ 1199.021243]  entry_SYSCALL_64_after_hwframe+0x61/0xc6
[ 1199.021627] RIP: 0033:0x7f1df5cee09b
[ 1199.021898] Code: eb b7 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 f3 0f 1e fa 48 81 fa ff ff ff 7f b8 ff ff ff 7f 48 0f 47 d0 b8 d9 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 05 c3 0f 1f 40 00 48 8b 15 a1 ad 10 00 f7 d8
[ 1199.023246] RSP: 002b:00007f1df19fe9b8 EFLAGS: 00000293 ORIG_RAX: 00000000000000d9
[ 1199.023814] RAX: ffffffffffffffda RBX: 0000000002964790 RCX: 00007f1df5cee09b
[ 1199.024332] RDX: 0000000000008000 RSI: 00000000029647c0 RDI: 000000000000001f
[ 1199.024866] RBP: 00000000029647c0 R08: 0000000000000030 R09: 00007f1df5df9a60
[ 1199.025392] R10: 0000000000400000 R11: 0000000000000293 R12: ffffffffffffff38
[ 1199.025923] R13: 0000000002964794 R14: 0000000000000002 R15: 00007f1df6e5a980
[ 1199.026441] Modules linked in: tcp_diag inet_diag rfkill intel_rapl_msr intel_rapl_common joydev mousedev isst_if_common nfit crct10dif_pclmul crc32_pclmul ghash_clmulni_intel rapl psmouse pcspkr virtio_balloon i2c_piix4 nf_tables libcrc32c nfnetlink nfsd auth_rpcgss nfs_acl lockd grace sunrpc cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ata_generic crc32c_intel ata_piix drm libata i2c_core virtio_net net_failover failover serio_raw virtio_console
[ 1199.029490] ---[ end trace 8c84679450540b3a ]---
[ 1199.029837] RIP: 0010:proc_readfd_common+0xa6/0x230
[ 1199.030210] Code: 0f 83 88 00 00 00 49 8b 55 20 8b 02 39 d8 76 6a 89 d9 48 39 c1 48 19 c0 48 8b 52 08 21 d8 48 8d 04 c2 48 8b 00 48 85 c0 74 50 <8b> 40 44 89 44 24 04 e8 8e 5e d3 ff 89 d9 be 0b 00 00 00 48 c7 c2
[ 1199.031590] RSP: 0018:ffffad35c0dbbe60 EFLAGS: 00010206
[ 1199.031977] RAX: 4900004500080000 RBX: 0000000000000021 RCX: 0000000000000021
[ 1199.032511] RDX: ffff9dd183a70000 RSI: 00000000029647f0 RDI: ffff9dd227fdd2f8
[ 1199.033471] RBP: ffffad35c0dbbef0 R08: 0000000000000018 R09: 0000000000000000
[ 1199.034428] R10: 0000000000000000 R11: 0000000000000000 R12: ffff9dd227fdc480
[ 1199.035348] R13: ffff9dd186012c00 R14: ffff9dd0a66bdb00 R15: ffffffffb6438360
[ 1199.036274] FS:  00007f1df19ff640(0000) GS:ffff9dd2a8180000(0000) knlGS:0000000000000000
[ 1199.037234] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1199.038035] CR2: 000035ce97ce1000 CR3: 0000000106768005 CR4: 00000000003706e0
[ 1199.038974] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 1199.039911] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 1199.040880] Kernel panic - not syncing: Fatal exception
[ 1199.042777] Kernel Offset: 0x35000000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[root@iZbp1esyiqa4v7lvgcycxtZ kernels]#
Comment 1 wanqian alibaba_cloud_group 2023-07-07 18:13:32 UTC
在rc3上,

echo 0  > /proc/sys/kernel/softlockup_panic
设置后,再测试,通过

[root@anck5 ~]# stress-ng --io-uring 2 -t 20s
stress-ng: info:  [5343] setting to a 20 second run per stressor
stress-ng: info:  [5343] dispatching hogs: 2 io-uring
stress-ng: info:  [5343] passed: 2: io-uring (2)
stress-ng: info:  [5343] failed: 0
stress-ng: info:  [5343] skipped: 0
stress-ng: info:  [5343] successful run completed in 46.03s
Comment 2 Ferry Meng alibaba_cloud_group 2023-07-07 19:05:32 UTC
单从vmcore信息看,现场为fs/proc/fd.c中
            struct file *f
253         f = fcheck_files(files, fd);
254         if (!f)
255             continue;
256         data.mode = f->f_mode;

f的值为(%rax) 0x4900004500080000 , 是个非法值(不在内核空间)。
在对应物理机上重复测试,在ecs上重复测试。无法复现问题。
Comment 3 shuancue alibaba_cloud_group 2023-07-11 19:13:39 UTC
关掉soft_lockup_panic当然不会因此触发panic了,需要确认下softlockup的原因以及是否是可接受的?
Comment 5 maqiao alibaba_cloud_group 2023-07-13 18:09:24 UTC
需要测试继续复现现场,目前无法进一步分析