Bug 19052 - [ANCK6.6.71-3 rc1][aarch64][倚天710机器]执行bpftrace --info命令产生coredump
Summary: [ANCK6.6.71-3 rc1][aarch64][倚天710机器]执行bpftrace --info命令产生coredump
Status: NEW
Alias: None
Product: Antest
Classification: Infrastructures
Component: 测试用例 (show other bugs) 测试用例
Version: unspecified
Hardware: aarch64 Linux
: P3-Medium S3-normal
Target Milestone: ---
Assignee: lv0322
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2025-02-25 15:36 UTC by zhixin01
Modified: 2025-03-06 17:19 UTC (History)
6 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description zhixin01 alibaba_cloud_group 2025-02-25 15:36:06 UTC
[缺陷描述]:
执行bpftrace --info命令产生coredump

测试日志:
#bpftrace --info
System
  OS: Linux 6.6.71-3_rc1.al8.aarch64 #1 SMP PREEMPT_DYNAMIC Fri Feb 21 11:47:20 CST 2025
  Arch: aarch64

Build
  version: v0.16.0
  LLVM: 15.0.7
  ORC: v2
  foreach_sym: yes
  unsafe uprobe: no
  bfd: yes
  bcc_usdt_addsem: yes
  bcc bpf_attach_uprobe refcount: yes
  bcc library path resolution: yes
  libbpf btf dump: yes
  libbpf btf dump type decl: yes
  libbpf bpf_prog_load: no
  libbpf bpf_map_create: no
  libdw (DWARF support): no

Segmentation fault (core dumped)

coredump信息:
       Boot ID: 732a9152dccf4c9396a19b43ed142fbc
    Machine ID: 3e5bdf9822c74974ac3da67daff89a22
      Hostname: t50a07416.sqa.eu95
       Storage: /var/lib/systemd/coredump/core.bpftrace.0.732a9152dccf4c9396a19b43ed142fbc.1305472.1740467979000000.lz4
       Message: Process 1305472 (bpftrace) of user 0 dumped core.

                Stack trace of thread 1305472:
                #0  0x0000ffff8068fbd4 btf__find_by_name_kind (libbpf.so.0)
                #1  0x0000aaaac93ad358 _ZN8bpftrace10BPFfeature8try_loadEN6libbpf13bpf_prog_typeEP8bpf_insnmPKcSt8optionalINS1_15bpf_attach_typeEEPi (bpftrace)
                #2  0x0000aaaac93ad7ac _ZN8bpftrace10BPFfeature10has_d_pathEv.part.0 (bpftrace)
                #3  0x0000aaaac93aee5c _ZN8bpftrace10BPFfeature6reportB5cxx11Ev (bpftrace)
                #4  0x0000aaaac9377020 _Z10parse_argsiPPc (bpftrace)
                #5  0x0000aaaac934cb18 main (bpftrace)
                #6  0x0000ffff78699ad8 __libc_start_main (libc.so.6)
                #7  0x0000aaaac9373438 _start (bpftrace)

           PID: 1316931 (bpftrace)
           UID: 0 (root)
           GID: 0 (root)
        Signal: 11 (SEGV)
     Timestamp: Tue 2025-02-25 15:32:49 CST (34s ago)
  Command Line: bpftrace --info
    Executable: /usr/bin/bpftrace
 Control Group: /user.slice/user-1377975.slice/session-269.scope
          Unit: session-269.scope
         Slice: user-1377975.slice
       Session: 269
     Owner UID: 1377975 (zx01468051)
       Boot ID: 732a9152dccf4c9396a19b43ed142fbc
    Machine ID: 3e5bdf9822c74974ac3da67daff89a22
      Hostname: t50a07416.sqa.eu95
       Storage: /var/lib/systemd/coredump/core.bpftrace.0.732a9152dccf4c9396a19b43ed142fbc.1316931.1740468769000000.lz4
       Message: Process 1316931 (bpftrace) of user 0 dumped core.

                Stack trace of thread 1316931:
                #0  0x0000ffff9a1dfbd4 btf__find_by_name_kind (libbpf.so.0)
                #1  0x0000aaaada6bd358 _ZN8bpftrace10BPFfeature8try_loadEN6libbpf13bpf_prog_typeEP8bpf_insnmPKcSt8optionalINS1_15bpf_attach_typeEEPi (bpftrace)
                #2  0x0000aaaada6bd7ac _ZN8bpftrace10BPFfeature10has_d_pathEv.part.0 (bpftrace)
                #3  0x0000aaaada6bee5c _ZN8bpftrace10BPFfeature6reportB5cxx11Ev (bpftrace)
                #4  0x0000aaaada687020 _Z10parse_argsiPPc (bpftrace)
                #5  0x0000aaaada65cb18 main (bpftrace)
                #6  0x0000ffff92499ad8 __libc_start_main (libc.so.6)
                #7  0x0000aaaada683438 _start (bpftrace)

[重现概率]:
必现

[重现环境]:
环境信息:倚天710机器
11.163.178.238

#uname -r
6.6.71-3_rc1.al8.aarch64

#cat /etc/os-release
NAME="Alibaba Cloud Linux"
VERSION="3 (Soaring Falcon)"
ID="alinux"
ID_LIKE="rhel fedora centos anolis"
VERSION_ID="3"
UPDATE_ID="10"
PLATFORM_ID="platform:al8"
PRETTY_NAME="Alibaba Cloud Linux 3 (Soaring Falcon)"
ANSI_COLOR="0;31"
HOME_URL="https://www.aliyun.com/"

#lscpu
Architecture:        aarch64
Byte Order:          Little Endian
CPU(s):              128
On-line CPU(s) list: 0-127
Thread(s) per core:  1
Core(s) per socket:  128
Socket(s):           1
NUMA node(s):        2
Vendor ID:           ARM
BIOS Vendor ID:      T-HEAD
Model:               0
Model name:          Neoverse-N2
BIOS Model name:     Yitian710-128
Stepping:            r0p0
CPU MHz:             2750.000
BogoMIPS:            100.00
Hypervisor vendor:   Alibaba
Virtualization type: full
L1d cache:           64K
L1i cache:           64K
L2 cache:            1024K
L3 cache:            65536K
NUMA node0 CPU(s):   0-63
NUMA node1 CPU(s):   64-127
Flags:               fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh

#free -h
              total        used        free      shared  buff/cache   available
Mem:          125Gi       3.6Gi       121Gi        12Mi       1.0Gi       122Gi
Swap:         2.0Gi          0B       2.0Gi

#cat /proc/cmdline
BOOT_IMAGE=(hd0,gpt2)/boot/vmlinuz-6.6.71-3_rc1.al8.aarch64 root=UUID=d0af582f-7147-41de-85e3-deb2e14cde99 ro biosdevname=0 rd.driver.pre=ahci iommu.passthrough=1 iommu.strict=0 nospectre_bhb ssbd=force-off systemd.unified_cgroup_hierarchy=0 cgroup.memory=nokmem console=ttyS0,115200 fsck.repair=yes crashkernel=0M-2G:0M,2G-256G:256M,256G-1024G:320M,1024G-:384M

#yum install bpftrace
Last metadata expiration check: 23:34:05 ago on Mon 24 Feb 2025 04:00:44 PM CST.
Package bpftrace-0.16.0-8.al8.aarch64 is already installed.
Dependencies resolved.
Nothing to do.
Complete!

[重现步骤]:
1. yum install bpftrace
2. bpftrace --info

[期望结果]:
可以正常输出结果

[实际结果]:
产生coredump
Comment 1 zhixin01 alibaba_cloud_group 2025-03-05 10:30:16 UTC
1.上述失败影响bpftrace runtime_test测试集执行,报错如下
./runtime-tests.sh: line 18: 3449293 Segmentation fault      (core dumped) "${BPFTRACE_RUNTIME_TEST_EXECUTABLE}/bpftrace" --info
[2025-03-05 10:24:18] ./runtime-tests.sh
====================
bpftrace --info:
====================

2.通过pahole -J --skip_encoding_btf_enum64 /usr/lib/debug/lib/modules/$(uname -r)/vmlinux 获取BPT信息,可以执行bpftrace runtime_test测试集,但是有5个失败用例,麻烦确认下是否有BTF需求,失败用例是否需要关注

失败用例如下
[  FAILED  ] basic.piped script
        Command: /usr/bin//bpftrace - < runtime/scripts/hello_world.bt
        Unclean exit code: 1
        Output: BTF: (2) libbpf: Unsupported BTF_KIND:0\nLog: cannot resolve location before calling set_source().\nERROR: unexpected end of file, expected {\n

[  FAILED  ] builtin.log size too small
        Command: /usr/bin//bpftrace -ve 'BEGIN { if (str($1) == str($2)) { printf("%s\n", str($1)); exit() } }' "hello" "hello"
        Expected: Error loading program: BEGIN
        Found: BTF: (2) libbpf: Unsupported BTF_KIND:0\nINFO: node count: 16\nAttaching 1 probe...\n\nError log: \n\nERROR: Error: Failed to load program, veri             fication log buffer not big enough, try increasing the BPFTRACE_LOG_SIZE environment variable beyond the current value of 1 bytes\n

[  FAILED  ] call.uaddr
        Command: /usr/bin//bpftrace -e 'uprobe:testprogs/uprobe_test:function1 { printf("0x%lx -- 0x%lx\n", *uaddr("GLOBAL_A"), *uaddr("GLOBAL_C")); exit(); }'              -c ./testprogs/uprobe_test
        Expected: 0x55555555 -- 0x33333333
        Found: BTF: (2) libbpf: Unsupported BTF_KIND:0\nAttaching 1 probe...\n__BPFTRACE_NOTIFY_PROBES_ATTACHED\n0x0 -- 0x0\n\n\n

[  FAILED  ] probe.kfunc
        Command: /usr/bin//bpftrace -e 'kfunc:vfs_read { printf("SUCCESS %d\n", pid); exit(); }'
        Unclean exit code: 1
        Output: BTF: (2) libbpf: Unsupported BTF_KIND:0\nstdin:1:1-15: ERROR: kfunc/kretfunc not available for your kernel version.\nkfunc:vfs_read { printf("S             UCCESS %d\\n", pid); exit(); }\n~~~~~~~~~~~~~~\n

[  FAILED  ] probe.kretfunc
        Command: /usr/bin//bpftrace -e 'kretfunc:vfs_read { printf("SUCCESS %d\n", pid); exit(); }'
        Unclean exit code: 1
        Output: BTF: (2) libbpf: Unsupported BTF_KIND:0\nstdin:1:1-18: ERROR: kfunc/kretfunc not available for your kernel version.\nkretfunc:vfs_read { printf             ("SUCCESS %d\\n", pid); exit(); }\n~~~~~~~~~~~~~~~~~\n
Comment 2 lv0322 alibaba_cloud_group 2025-03-06 17:19:02 UTC
该问提是一个已知问题,目前yum源安装的bpftrace和6.6内核存在不适配的问题,考虑到bpftrace与内核有较强的关联性,且现在alinux3/8u环境还存在大量5.10内核,因此不会直接升级bpftrace来做适配。