Created attachment 877 [details] kernelselftest下bpf编译失败日志 [缺陷描述]： llvm软件升级到llvm-15.0.7-1.0.2.al8.aarch64版本后，kernel-selftests下bpf目录编译失败； llvm原来版本为13.0.1时可以正常编译； llvm-13.0.1-1.0.3.al8.aarch64 llvm升级后的版本，如下： #rpm -qa |grep llvm llvm-devel-15.0.7-1.0.2.al8.aarch64 llvm-15.0.7-1.0.2.al8.aarch64 llvm-libs-15.0.7-1.0.2.al8.aarch64 llvm-test-15.0.7-1.0.2.al8.aarch64 llvm-static-15.0.7-1.0.2.al8.aarch64 编译报错日志请见附件 [复现概率]： 必现 [复现环境]： 机器信息：线下vm 内核信息： 5.10.134-1035.git.3513726cd1.al8.aarch64 # cat /etc/os-release NAME="Alibaba Cloud Linux" VERSION="3 (Soaring Falcon)" ID="alinux" ID_LIKE="rhel fedora centos" VERSION_ID="3" PLATFORM_ID="platform:al8" PRETTY_NAME="Alibaba Cloud Linux 3 (Soaring Falcon)" ANSI_COLOR="0;31" HOME_URL="https://www.aliyun.com/" cpu信息： # lscpu Architecture: aarch64 Byte Order: Little Endian CPU(s): 4 On-line CPU(s) list: 0-3 Thread(s) per core: 1 Core(s) per cluster: 4 Socket(s): 4 Cluster(s): 1 NUMA node(s): 1 Vendor ID: HiSilicon BIOS Vendor ID: Alibaba Cloud Model: 0 Model name: Kunpeng-920 BIOS Model name: virt-rhel7.6.0 Stepping: 0x1 BogoMIPS: 200.00 NUMA node0 CPU(s): 0-3 Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma dcpop asimddp asimdfhm 内存信息： # free -h total used free shared buff/cache available Mem: 15Gi 517Mi 13Gi 4.0Mi 1.4Gi 14Gi Swap: 0B 0B 0B [复现步骤]： 下载kernel-5.10.134-1035.git.3513726cd1.al8.src.rpm源码包 rpm -ivh kernel*.src.rpm which yum-builddep || yum install -y yum-utils which rpmbuild || yum install -y rpm-build yum-builddep -y $rpmbuild_dir/SPECS/kernel.spec rpmbuild -bp $rpmbuild_dir/SPECS/kernel.spec cd /root/rpmbuild/BUILD/kernel*/linux*/tools/testing/selftests cd bpf make [预期结果]： bpf目录编译成功 [实际结果]： 升级llvm软件包后，bpf目录编译失败
anolis8和anolis23均存在这个问题 upstream存在同样的问题： https://github.com/llvm/llvm-project/issues/55669 并且提供了修复补丁，请参考： https://lore.kernel.org/bpf/20220419043230.2928530-1-yhs@fb.com/ 手动在anolis8 5.10 x86试了下，将这个补丁合入后，是可以成功编译bpf的，请开发同学参考。
(In reply to shanxifanshi from comment #1) > anolis8和anolis23均存在这个问题 > > upstream存在同样的问题： > https://github.com/llvm/llvm-project/issues/55669 > > 并且提供了修复补丁，请参考： > https://lore.kernel.org/bpf/20220419043230.2928530-1-yhs@fb.com/ > > 手动在anolis8 5.10 x86试了下，将这个补丁合入后，是可以成功编译bpf的，请开发同学参考。 看起来这个问题在an8 5.10 x86内核上已经解决了，最新nightly测试，bpf是可以成功编译的 # uname -r 5.10.134-111.git.3f29353eb6.an8.x86_64 kernel_selftests部分编译日志，如下图所示，bpf整个目录已经编译成功 /tmp/tone/build/kernel-selftests/anolis-source/tools/testing/selftests/bpf/prog_tests/bloom_filter_map.c: In function ‘test_bloom_filter_map’: TEST-OBJ [test_progs-no_alu32] test_profiler.test.o TEST-OBJ [test_progs-no_alu32] global_data_init.test.o TEST-OBJ [test_progs-no_alu32] connect_force_port.test.o TEST-OBJ [test_progs-no_alu32] btf_map_in_map.test.o TEST-OBJ [test_progs-no_alu32] flow_dissector_load_bytes.test.o TEST-OBJ [test_progs-no_alu32] global_data.test.o TEST-OBJ [test_progs-no_alu32] send_signal_sched_switch.test.o TEST-OBJ [test_progs-no_alu32] map_lock.test.o TEST-OBJ [test_progs-no_alu32] btf_endian.test.o TEST-OBJ [test_progs-no_alu32] xdp_link.test.o TEST-OBJ [test_progs-no_alu32] pkt_access.test.o TEST-OBJ [test_progs-no_alu32] sockmap_basic.test.o TEST-OBJ [test_progs-no_alu32] sockmap_listen.test.o BINARY test_progs BINARY test_progs-no_alu32 make[1]: Leaving directory '/tmp/tone/build/kernel-selftests/anolis-source/tools/testing/selftests/bpf'
在多云机器，15内核测试时，该问题仍然存在 # make MKDIR include MKDIR libbpf HOSTCC /root/rpmbuild/BUILD/kernel-5.10.134-15.al8/linux-5.10.134-15.al8.x86_64/tools/testing/selftests/bpf/tools/build/libbpf/fixdep.o ------省略若干行------------------------------- linux-5.10.134-15.al8.x86_64/tools/testing/selftests/bpf/tools/build/bpftool/Documentation/bpf-helpers.7 INSTALL eBPF_helpers-manpage INSTALL Documentation-man GEN vmlinux.h CLNG-LLC [test_maps] test_sk_lookup_kern.o CLNG-LLC [test_maps] test_tc_tunnel.o CLNG-LLC [test_maps] test_tc_neigh.o CLNG-LLC [test_maps] test_xdp_meta.o CLNG-LLC [test_maps] pyperf_subprogs.o CLNG-LLC [test_maps] pyperf180.o CLNG-LLC [test_maps] btf__core_reloc_bitfields___bitfield_vs_int.o CLNG-LLC [test_maps] sample_map_ret0.o CLNG-LLC [test_maps] connect4_prog.o CLNG-LLC [test_maps] btf__core_reloc_nesting___extra_nesting.o CLNG-LLC [test_maps] btf__core_reloc_primitives___err_non_ptr.o CLNG-LLC [test_maps] test_varlen.o CLNG-LLC [test_maps] test_global_func8.o CLNG-LLC [test_maps] btf__core_reloc_nesting___err_missing_container.o CLNG-LLC [test_maps] btf_dump_test_case_bitfields.o CLNG-LLC [test_maps] btf__core_reloc_primitives___err_non_int.o CLNG-LLC [test_maps] btf__core_reloc_ints___reverse_sign.o CLNG-LLC [test_maps] test_cls_redirect_subprogs.o CLNG-LLC [test_maps] bpf_iter_task_stack.o CLNG-LLC [test_maps] tailcall_bpf2bpf1.o CLNG-LLC [test_maps] test_cls_redirect.o CLNG-LLC [test_maps] btf__core_reloc_type_based___incompat.o CLNG-LLC [test_maps] test_btf_newkv.o CLNG-LLC [test_maps] pyperf600.o LLVM ERROR: Branch target out of insn range PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace. Stack dump: 0. Program arguments: llc -mattr=dwarfris -march=bpf -mcpu=v3 -mattr=+alu32 -filetype=obj -o /root/rpmbuild/BUILD/kernel-5.10.134-15.al8/linux-5.10.134-15.al8.x86_64/tools/testing/selftests/bpf/pyperf600.o #0 0x00007f37d4c12c91 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/lib64/libLLVM-15.so+0xbf1c91) #1 0x00007f37d4c10a74 llvm::sys::RunSignalHandlers() (/lib64/libLLVM-15.so+0xbefa74) #2 0x00007f37d4c10c1e (/lib64/libLLVM-15.so+0xbefc1e) #3 0x00007f37d40131d0 __restore_rt sigaction.c:0:0 #4 0x00007f37d3b26005 raise (/lib64/libc.so.6+0x54005) #5 0x00007f37d3af8894 abort (/lib64/libc.so.6+0x26894) #6 0x00007f37d49d734c (/lib64/libLLVM-15.so+0x9b634c) #7 0x00007f37d4b37dae (/lib64/libLLVM-15.so+0xb16dae) #8 0x00007f37d790aad3 (/lib64/libLLVM-15.so+0x38e9ad3) #9 0x00007f37d63541f2 llvm::MCAssembler::layout(llvm::MCAsmLayout&) (/lib64/libLLVM-15.so+0x23331f2) #10 0x00007f37d635437d llvm::MCAssembler::Finish() (/lib64/libLLVM-15.so+0x233337d) #11 0x00007f37d54fc3ac llvm::AsmPrinter::doFinalization(llvm::Module&) (/lib64/libLLVM-15.so+0x14db3ac) #12 0x00007f37d4d43695 llvm::FPPassManager::doFinalization(llvm::Module&) (/lib64/libLLVM-15.so+0xd22695) #13 0x00007f37d4d4e800 llvm::legacy::PassManagerImpl::run(llvm::Module&) (/lib64/libLLVM-15.so+0xd2d800) #14 0x000056118ad57956 (/usr/bin/llc+0x17956) #15 0x000056118ad4eed2 main (/usr/bin/llc+0xeed2) #16 0x00007f37d3b10a72 __libc_start_main (/lib64/libc.so.6+0x3ea72) #17 0x000056118ad4f55e _start (/usr/bin/llc+0xf55e) /bin/sh: line 1: 121618 Done ( clang -g -D__TARGET_ARCH_x86 -mlittle-endian -I/root/rpmbuild/BUILD/kernel-5.10.134-15.al8/linux-5.10.134-15.al8.x86_64/tools/testing/selftests/bpf/tools/include -I/root/rpmbuild/BUILD/kernel-5.10.134-15.al8/linux-5.10.134-15.al8.x86_64/tools/testing/selftests/bpf -I/root/rpmbuild/BUILD/kernel-5.10.134-15.al8/linux-5.10.134-15.al8.x86_64/tools/include/uapi -I/root/rpmbuild/BUILD/kernel-5.10.134-15.al8/linux-5.10.134-15.al8.x86_64/tools/testing/selftests/usr/include -idirafter /usr/lib64/clang/15.0.7/include -idirafter /usr/local/include -idirafter /usr/include -Wno-compare-distinct-pointer-types -O2 -target bpf -emit-llvm -c progs/pyperf600.c -o - || echo "BPF obj compilation failed" ) 121619 Aborted (core dumped) | llc -mattr=dwarfris -march=bpf -mcpu=v3 -mattr=+alu32 -filetype=obj -o /root/rpmbuild/BUILD/kernel-5.10.134-15.al8/linux-5.10.134-15.al8.x86_64/tools/testing/selftests/bpf/pyperf600.o make: *** [Makefile:402: /root/rpmbuild/BUILD/kernel-5.10.134-15.al8/linux-5.10.134-15.al8.x86_64/tools/testing/selftests/bpf/pyperf600.o] Error 134 [root@aidc-kernel-perf-test02 bpf]# yum list llvm Last metadata expiration check: 0:50:54 ago on Sun 07 Apr 2024 01:10:11 PM CST. Installed Packages llvm.x86_64 15.0.7-1.0.2.al8 @alinux3-updates