884 – [Anolis OS 8.4][5.10.112_rc1][x86_64][ecs.ebmg7a.64xlarge][debug-kernel]开机自启动后，dmesg日志中有大量kmemleak: Cannot allocate a kmemleak_object structure

Bug 884 - [Anolis OS 8.4][5.10.112_rc1][x86_64][ecs.ebmg7a.64xlarge][debug-kernel]开机自启动后，dmesg日志中有大量kmemleak: Cannot allocate a kmemleak_object structure

Summary: [Anolis OS 8.4][5.10.112_rc1][x86_64][ecs.ebmg7a.64xlarge][debug-kernel]开机自启动...

Status:	CONFIRMED

Alias:	None

Product:	Anolis OS 8
Classification:	Anolis OS
Component:	kernel - anck-5.10 (show other bugs)	kernel - anck-5.10
Sub Component:
Version:	8.4
Hardware:	x86_64 Linux

Importance:	P3-Medium S3-normal
Target Milestone:	---
Assignee:	kun(llfl)
QA Contact:	shuming

URL:
Whiteboard:
Keywords:

Depends on:
Blocks:

Reported:	2022-04-22 17:15 UTC by wangpingping
Modified:	2023-03-02 15:57 UTC (History)
CC List:	5 users (show)

See Also:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.

Description wangpingping alibaba_cloud_group

2022-04-22 17:15:03 UTC

[缺陷描述]
安装5.10.112-11_rc1.an8.x86_64+debug内核后，开机自启动，dmesg日志中有大量kmemleak: Cannot allocate a kmemleak_object structure

错误如下：

]# dmesg -l warn -T
[Fri Apr 22 16:42:02 2022] kmemleak: Memory pool empty, consider increasing CONFIG_DEBUG_KMEMLEAK_MEM_POOL_SIZE
[Fri Apr 22 16:42:02 2022] kmemleak: Cannot allocate a kmemleak_object structure
[Fri Apr 22 16:42:02 2022] kmemleak: Cannot allocate a kmemleak_object structure
[Fri Apr 22 16:42:02 2022] kmemleak: Cannot allocate a kmemleak_object structure
[Fri Apr 22 16:42:02 2022] kmemleak: Cannot allocate a kmemleak_object structure
[Fri Apr 22 16:42:02 2022] kmemleak: Cannot allocate a kmemleak_object structure
[Fri Apr 22 16:42:02 2022] kmemleak: Cannot allocate a kmemleak_object structure
[Fri Apr 22 16:42:02 2022] kmemleak: Cannot allocate a kmemleak_object structure
[Fri Apr 22 16:42:02 2022] kmemleak: Cannot allocate a kmemleak_object structure
[Fri Apr 22 16:42:02 2022] kmemleak: Cannot allocate a kmemleak_object structure
[Fri Apr 22 16:42:02 2022] kmemleak: Cannot allocate a kmemleak_object structure
[Fri Apr 22 16:42:02 2022] kmemleak: Cannot allocate a kmemleak_object structure
[Fri Apr 22 16:42:02 2022] kmemleak: Cannot allocate a kmemleak_object structure
[Fri Apr 22 16:42:02 2022] kmemleak: Cannot allocate a kmemleak_object structure
[Fri Apr 22 16:42:02 2022] kmemleak: Cannot allocate a kmemleak_object structure
[Fri Apr 22 16:42:02 2022] kmemleak: Cannot allocate a kmemleak_object structure
[Fri Apr 22 16:42:02 2022] kmemleak: Cannot allocate a kmemleak_object structure
[Fri Apr 22 16:42:02 2022] kmemleak: Cannot allocate a kmemleak_object structure
[Fri Apr 22 16:42:02 2022] kmemleak: Cannot allocate a kmemleak_object structure
[Fri Apr 22 16:42:02 2022] kmemleak: Cannot allocate a kmemleak_object structure
[Fri Apr 22 16:42:02 2022] kmemleak: Cannot allocate a kmemleak_object structure
[Fri Apr 22 16:42:02 2022] kmemleak: Cannot allocate a kmemleak_object structure
[Fri Apr 22 16:42:02 2022] kmemleak: Cannot allocate a kmemleak_object structure
[Fri Apr 22 16:42:02 2022] kmemleak: Cannot allocate a kmemleak_object structure
[Fri Apr 22 16:42:02 2022] kmemleak: Cannot allocate a kmemleak_object structure


[重现概率]
必现

[重现环境]
内核：
# uname -r
5.10.112-11_rc1.an8.x86_64+debug

# cat /etc/os-release
NAME="Anolis OS"
VERSION="8.4"
ID="anolis"
ID_LIKE="rhel fedora centos"
VERSION_ID="8.4"
PLATFORM_ID="platform:an8"
PRETTY_NAME="Anolis OS 8.4"
ANSI_COLOR="0;31"
HOME_URL="https://openanolis.cn/"

# lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              256
On-line CPU(s) list: 0-255
Thread(s) per core:  2
Core(s) per socket:  64
Socket(s):           2
NUMA node(s):        8
Vendor ID:           AuthenticAMD
BIOS Vendor ID:      Advanced Micro Devices, Inc.
CPU family:          25
Model:               1
Model name:          AMD EPYC 7T83 64-Core Processor
BIOS Model name:     AMD EPYC 7T83 64-Core Processor
Stepping:            1
CPU MHz:             1920.052
CPU max MHz:         3673.0950
CPU min MHz:         1500.0000
BogoMIPS:            5090.51
Virtualization:      AMD-V
L1d cache:           32K
L1i cache:           32K
L2 cache:            512K
L3 cache:            32768K
NUMA node0 CPU(s):   0-15,128-143
NUMA node1 CPU(s):   16-31,144-159
NUMA node2 CPU(s):   32-47,160-175
NUMA node3 CPU(s):   48-63,176-191
NUMA node4 CPU(s):   64-79,192-207
NUMA node5 CPU(s):   80-95,208-223
NUMA node6 CPU(s):   96-111,224-239
NUMA node7 CPU(s):   112-127,240-255
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 invpcid_single hw_pstate sme ssbd mba sev ibrs ibpb stibp vmmcall sev_es fsgsbase bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd amd_ppin arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload vgif umip pku ospke vaes vpclmulqdq rdpid overflow_recov succor smca fsrm


# free -h
              total        used        free      shared  buff/cache   available
Mem:          876Gi        32Gi       840Gi       2.0Mi       4.3Gi       839Gi
Swap:            0B          0B          0B

[问题重现步骤]	
1、安装内核
2、reboot
3、dmesg -l warn -T

Comment 1 wangpingping alibaba_cloud_group

2022-04-22 17:17:02 UTC

# cat /usr/src/kernels/5.10.112-11_rc1.an8.x86_64+debug/.config |grep CONFIG_DEBUG_KMEMLEAK_MEM_POOL_SIZE
CONFIG_DEBUG_KMEMLEAK_MEM_POOL_SIZE=16000

Comment 2 Shiloong admin

2022-05-11 14:47:58 UTC

感觉是内核配置不足导致的，看似符合预期。
是否可以调整一下配置看看。

Comment 3 kun(llfl) alibaba_cloud_group

2022-06-29 12:51:50 UTC

建议调大CONFIG_DEBUG_KMEMLEAK_MEM_POOL_SIZE到320000试试

Comment 4 meil_wei alibaba_cloud_group

2023-03-02 15:57:57 UTC

5.10.134-14_rc1.an8.x86_64+debug内核安装后有同样问题

# dmesg -l warn -T
[Thu Mar  2 15:21:51 2023] kmemleak: Memory pool empty, consider increasing CONFIG_DEBUG_KMEMLEAK_MEM_POOL_SIZE
[Thu Mar  2 15:21:51 2023] kmemleak: Cannot allocate a kmemleak_object structure
[Thu Mar  2 15:21:51 2023] kmemleak: Cannot allocate a kmemleak_object structure
[Thu Mar  2 15:21:51 2023] kmemleak: Cannot allocate a kmemleak_object structure
[Thu Mar  2 15:21:51 2023] kmemleak: Cannot allocate a kmemleak_object structure
[Thu Mar  2 15:21:51 2023] kmemleak: Cannot allocate a kmemleak_object structure
[Thu Mar  2 15:21:51 2023] kmemleak: Cannot allocate a kmemleak_object structure
[Thu Mar  2 15:21:51 2023] kmemleak: Cannot allocate a kmemleak_object structure
[Thu Mar  2 15:21:51 2023] kmemleak: Cannot allocate a kmemleak_object structure
[Thu Mar  2 15:21:51 2023] kmemleak: Cannot allocate a kmemleak_object structure
[Thu Mar  2 15:21:51 2023] kmemleak: Cannot allocate a kmemleak_object structure
[Thu Mar  2 15:21:51 2023] kmemleak: Cannot allocate a kmemleak_object structure


# uname -r
5.10.134-14_rc1.an8.x86_64+debug

# free -g
              total        used        free      shared  buff/cache   available
Mem:            878          32         845           0           0         843
Swap:             0           0           0

# lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              256
On-line CPU(s) list: 0-255
Thread(s) per core:  2
Core(s) per socket:  64
Socket(s):           2
NUMA node(s):        8
Vendor ID:           AuthenticAMD
BIOS Vendor ID:      Advanced Micro Devices, Inc.
CPU family:          23
Model:               49
Model name:          AMD EPYC 7H12 64-Core Processor
BIOS Model name:     AMD EPYC 7H12 64-Core Processor
Stepping:            0
CPU MHz:             1646.882
CPU max MHz:         2600.0000
CPU min MHz:         1500.0000
BogoMIPS:            5190.17
Virtualization:      AMD-V
L1d cache:           32K
L1i cache:           32K
L2 cache:            512K
L3 cache:            16384K
NUMA node0 CPU(s):   0-15,128-143
NUMA node1 CPU(s):   16-31,144-159
NUMA node2 CPU(s):   32-47,160-175
NUMA node3 CPU(s):   48-63,176-191
NUMA node4 CPU(s):   64-79,192-207
NUMA node5 CPU(s):   80-95,208-223
NUMA node6 CPU(s):   96-111,224-239
NUMA node7 CPU(s):   112-127,240-255
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd amd_ppin arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif umip rdpid overflow_recov succor smca sme sev sev_es

# cat /proc/cmdline
BOOT_IMAGE=(hd8,gpt2)/boot/vmlinuz-5.10.134-14_rc1.an8.x86_64+debug root=UUID=2b0efc6a-16e6-497e-b9f8-c284a4b43b10 ro cryptomgr.notests rcupdate.rcu_cpu_stall_timeout=300 vring_force_dma_api rhgb quiet biosdevname=0 net.ifnames=0 console=tty0 console=ttyS0,115200n8 noibrs nvme_core.io_timeout=4294967295 nvme_core.admin_timeout=4294967295 intel_idle.max_cstate=1 cgroup.memory=nokmem crashkernel=0M-2G:0M,2G-8G:192M,8G-:256M