Bug 4337 - [ANCK-5.10-14-rc1][Anolis8][x86_64]fio从3.22升级到最新的3.27版本,缺失了包含libaio在内的多个引擎能力
Summary: [ANCK-5.10-14-rc1][Anolis8][x86_64]fio从3.22升级到最新的3.27版本,缺失了包含libaio在内的多个引擎能力
Status: CLOSED FIXED
Alias: None
Product: Antest
Classification: Infrastructures
Component: 测试用例 (show other bugs) 测试用例
Version: unspecified
Hardware: x86_64 Linux
: P3-Medium S3-normal
Target Milestone: ---
Assignee: gaochang
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-03-03 14:49 UTC by shanxifanshi
Modified: 2023-07-25 15:24 UTC (History)
12 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description shanxifanshi alibaba_cloud_group 2023-03-03 14:49:03 UTC
[缺陷描述]:
fio从3.22升级到最新的3.27版本,缺失了包含libaio在内的多个引擎能力;对比3.22版本缺失的引擎能力有:
libaio
rdma
rados
rbd
pmemblk
dev-dax
libpmem


测试日志:
# yum list fio
Last metadata expiration check: 1:42:54 ago on Fri 03 Mar 2023 01:01:00 PM CST.
Installed Packages
fio.x86_64                                                           3.27-8.0.1.an8                                                            @AppStream

# fio --enghelp
Available IO engines:
        cpuio
        mmap
        sync
        psync
        vsync
        pvsync
        pvsync2
        null
        net
        netsplice
        ftruncate
        filecreate
        filestat
        filedelete
        posixaio
        falloc
        e4defrag
        splice
        mtd
        sg
        io_uring

内核信息:
# uname -r
5.10.134-14_rc1.an8.x86_64

操作系统信息:
# cat /etc/os-release
NAME="Anolis OS"
VERSION="8.8"
ID="anolis"
ID_LIKE="rhel fedora centos"
VERSION_ID="8.8"
PLATFORM_ID="platform:an8"
PRETTY_NAME="Anolis OS 8.8"
ANSI_COLOR="0;31"
HOME_URL="https://openanolis.cn/"

cpu信息:
# lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              8
On-line CPU(s) list: 0-7
Thread(s) per core:  2
Core(s) per socket:  4
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
BIOS Vendor ID:      Alibaba Cloud
CPU family:          6
Model:               143
Model name:          Intel(R) Xeon(R) Platinum 8475BL
BIOS Model name:     pc-i440fx-2.1
Stepping:            7
CPU MHz:             3038.155
BogoMIPS:            5400.00
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           48K
L1i cache:           32K
L2 cache:            2048K
L3 cache:            99840K
NUMA node0 CPU(s):   0-7
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq monitor ssse3 fma cx16 pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single ibrs_enhanced fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves avx_vnni avx512_bf16 wbnoinvd ida arat avx512vbmi pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid bus_lock_detect cldemote movdir64b fsrm uintr serialize tsxldtrk amx_bf16 avx512_fp16 amx_tile amx_int8 arch_capabilities


内存信息:
# free -h
              total        used        free      shared  buff/cache   available
Mem:           30Gi       372Mi        29Gi       2.0Mi       529Mi        29Gi
Swap:            0B          0B          0B

[期望结果]:
fio支持libaio引擎能力

[实际结果]:
fio不支持libaio引擎能力

[问题发生概率]:必现

[复现步骤]:	
1. yum install安装最新的3.27版本fio
2. fio --enghelp查询当前支持的引擎能力


[原因分析]:
Comment 1 shanxifanshi alibaba_cloud_group 2023-03-03 14:50:16 UTC
补充下fio 3.22版本的引擎能力作为参考:

# rpm -q fio
fio-3.22-1.0.1.an8.x86_64

# fio --enghelp
Available IO engines:
        cpuio
        mmap
        sync
        psync
        vsync
        pvsync
        pvsync2
        null
        net
        netsplice
        ftruncate
        filecreate
        filestat
        posixaio
        falloc
        e4defrag
        splice
        mtd
        sg
        io_uring
        libaio
        rdma
        rados
        rbd
        pmemblk
        dev-dax
        libpmem
Comment 2 yunmeng365524 2023-03-04 21:47:02 UTC
请帮忙确认一下新升级后的fio的相关bug,升级后的fio会导致数个case执行失败。见bug:
https://bugzilla.openanolis.cn/show_bug.cgi?id=4312
Comment 3 shanxifanshi alibaba_cloud_group 2023-03-08 11:35:53 UTC
补充一下,机器是有安装libaio的,但是fio --enghelp查询机器无libaio引擎

# rpm -q fio
fio-3.27-8.0.1.an8.x86_64


# yum list libaio
Last metadata expiration check: 3:00:56 ago on Wed 08 Mar 2023 08:33:15 AM CST.
Installed Packages
libaio.x86_64                                                          0.3.112-1.0.1.an8                                                          @System

# yum list libaio-devel
Last metadata expiration check: 3:01:04 ago on Wed 08 Mar 2023 08:33:15 AM CST.
Installed Packages
libaio-devel.x86_64                                                       0.3.112-1.0.1.an8                                                       @BaseOS

# fio --enghelp
Available IO engines:
        cpuio
        mmap
        sync
        psync
        vsync
        pvsync
        pvsync2
        null
        net
        netsplice
        ftruncate
        filecreate
        filestat
        filedelete
        posixaio
        falloc
        e4defrag
        splice
        mtd
        sg
        io_uring
Comment 4 gaochang alibaba_cloud_group 2023-03-08 11:38:42 UTC
fio 3.27 版本新增了一些engine 子包,安装上这个包后即可获取对应能力。这些包没有正确地上传至源中
问题确认,修复中
Comment 5 ZiyangZhang alibaba_cloud_group 2023-03-08 12:29:23 UTC
https://build.openanolis.cn/buildinfo?buildID=27347

安装 fio-engine-libaio
Comment 6 shanxifanshi alibaba_cloud_group 2023-03-08 14:13:29 UTC
(In reply to ZiyangZhang from comment #5)
> https://build.openanolis.cn/buildinfo?buildID=27347
> 
> 安装 fio-engine-libaio

---这个地址下的包无法下载
# wget http://build.openanolis.cn/kojifiles/packages/fio/3.27/8.0.1.an8/x86_64/fio-engine-libaio-3.27-8.0.1.an8.x86_64.rpm
--2023-03-08 14:13:06--  http://build.openanolis.cn/kojifiles/packages/fio/3.27/8.0.1.an8/x86_64/fio-engine-libaio-3.27-8.0.1.an8.x86_64.rpm
Resolving build.openanolis.cn (build.openanolis.cn)... 220.181.158.231, 220.181.158.226, 220.181.158.230, ...
Connecting to build.openanolis.cn (build.openanolis.cn)|220.181.158.231|:80... connected.
HTTP request sent, awaiting response... 403 Forbidden
2023-03-08 14:13:06 ERROR 403: Forbidden.
Comment 7 ZiyangZhang alibaba_cloud_group 2023-03-08 14:26:02 UTC
(In reply to shanxifanshi from comment #6)
> (In reply to ZiyangZhang from comment #5)
> > https://build.openanolis.cn/buildinfo?buildID=27347
> > 
> > 安装 fio-engine-libaio
> 
> ---这个地址下的包无法下载
> # wget
> http://build.openanolis.cn/kojifiles/packages/fio/3.27/8.0.1.an8/x86_64/fio-
> engine-libaio-3.27-8.0.1.an8.x86_64.rpm
> --2023-03-08 14:13:06-- 
> http://build.openanolis.cn/kojifiles/packages/fio/3.27/8.0.1.an8/x86_64/fio-
> engine-libaio-3.27-8.0.1.an8.x86_64.rpm
> Resolving build.openanolis.cn (build.openanolis.cn)... 220.181.158.231,
> 220.181.158.226, 220.181.158.230, ...
> Connecting to build.openanolis.cn
> (build.openanolis.cn)|220.181.158.231|:80... connected.
> HTTP request sent, awaiting response... 403 Forbidden
> 2023-03-08 14:13:06 ERROR 403: Forbidden.

这个地址不是让你下载的,只是表明fio-engine-libaio包缺失。请等待rpm源修复,使用yum正常安装fio
Comment 8 shanxifanshi alibaba_cloud_group 2023-03-08 14:27:16 UTC
(In reply to ZiyangZhang from comment #7)
> (In reply to shanxifanshi from comment #6)
> > (In reply to ZiyangZhang from comment #5)
> > > https://build.openanolis.cn/buildinfo?buildID=27347
> > > 
> > > 安装 fio-engine-libaio
> > 
> > ---这个地址下的包无法下载
> > # wget
> > http://build.openanolis.cn/kojifiles/packages/fio/3.27/8.0.1.an8/x86_64/fio-
> > engine-libaio-3.27-8.0.1.an8.x86_64.rpm
> > --2023-03-08 14:13:06-- 
> > http://build.openanolis.cn/kojifiles/packages/fio/3.27/8.0.1.an8/x86_64/fio-
> > engine-libaio-3.27-8.0.1.an8.x86_64.rpm
> > Resolving build.openanolis.cn (build.openanolis.cn)... 220.181.158.231,
> > 220.181.158.226, 220.181.158.230, ...
> > Connecting to build.openanolis.cn
> > (build.openanolis.cn)|220.181.158.231|:80... connected.
> > HTTP request sent, awaiting response... 403 Forbidden
> > 2023-03-08 14:13:06 ERROR 403: Forbidden.
> 
> 这个地址不是让你下载的,只是表明fio-engine-libaio包缺失。请等待rpm源修复,使用yum正常安装fio

---OK,那就推到正式源中,我在验证
Comment 9 gaochang alibaba_cloud_group 2023-03-09 11:08:50 UTC
相关 engine 包已更新至 mirror 源中,请尝试安装 fio-engine-xxx 包来获取对应能力。如果无法找到对应包,请尝试清理 yum 缓存后重试
Comment 10 shanxifanshi alibaba_cloud_group 2023-03-09 11:23:46 UTC
(In reply to gaochang from comment #9)
> 相关 engine 包已更新至 mirror 源中,请尝试安装 fio-engine-xxx 包来获取对应能力。如果无法找到对应包,请尝试清理 yum
> 缓存后重试

目前只是推送到了openanolis这个repo源
http://mirrors.openanolis.cn/anolis/8/AppStream/x86_64/os/Packages/

但ecs机器用的是aliyuncs repo源,辛苦往这个repo源也推送一下包,我刚点进去看了下这个repo源,目前还没有
http://mirrors.cloud.aliyuncs.com/anolis/8/AppStream/x86_64/os/Packages/
Comment 11 shanxifanshi alibaba_cloud_group 2023-03-09 14:10:37 UTC
安装缺失的fio-engine-xxx包之后,fio --enghelp查询,缺失的engine能力已包含,问题解决,bug关闭。

测试日志:

# fio --enghelp
Available IO engines:
        cpuio
        mmap
        sync
        psync
        vsync
        pvsync
        pvsync2
        null
        net
        netsplice
        ftruncate
        filecreate
        filestat
        filedelete
        posixaio
        falloc
        e4defrag
        splice
        mtd
        sg
        io_uring
        dev-dax
        libpmem
        rdma
        pmemblk
        libaio
        rbd
        rados

# rpm -qa | grep engine
fio-engine-rdma-3.27-8.0.1.an8.x86_64
fio-engine-libaio-3.27-8.0.1.an8.x86_64
fio-engine-rbd-3.27-8.0.1.an8.x86_64
fio-engine-pmemblk-3.27-8.0.1.an8.x86_64
fio-engine-rados-3.27-8.0.1.an8.x86_64
fio-engine-dev-dax-3.27-8.0.1.an8.x86_64
fio-engine-libpmem-3.27-8.0.1.an8.x86_64

# rpm -qa fio
fio-3.27-8.0.1.an8.x86_64