Bug 2606 - [Anolis23 Preview][ECS][vhd]使用远程ssh登录实例测试时,概率性会SSH connection failed
Summary: [Anolis23 Preview][ECS][vhd]使用远程ssh登录实例测试时,概率性会SSH connection failed
Status: RESOLVED BYDESIGN
Alias: None
Product: Anolis OS 23
Classification: Anolis OS
Component: Images&Installations (show other bugs) Images&Installations
Version: 23.0
Hardware: All Linux
: P3-Medium S3-normal
Target Milestone: ---
Assignee: xuchunmei
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-10-27 14:35 UTC by chuyang_94
Modified: 2022-10-27 19:22 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description chuyang_94 alibaba_cloud_group 2022-10-27 14:35:04 UTC
Description of problem:
安装vhd虚拟云上镜像后,使用远程ssh登录实例进行测试时,概率性会出现SSH connection failed
通过查看对应时间点的系统日志可以看到在Oct 25 20:18:57确实有连接失败的日志:
Oct 25 20:18:56 iZbp14pox71mwfc9iyd4frZ kernel: audit: type=1400 audit(1666700336.962:2735): avc:  denied  { read } for  pid=23209 comm="sshd" name="password-auth" dev="vda2" ino=34889231 scontext=system_u:system_r:sshd_t:s0-s0:c0.c1023 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file permissive=1
Oct 25 20:18:56 iZbp14pox71mwfc9iyd4frZ kernel: audit: type=1400 audit(1666700336.962:2735): avc:  denied  { open } for  pid=23209 comm="sshd" path="/etc/pam.d/password-auth" dev="vda2" ino=34889231 scontext=system_u:system_r:sshd_t:s0-s0:c0.c1023 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file permissive=1
Oct 25 20:18:56 iZbp14pox71mwfc9iyd4frZ kernel: audit: type=1300 audit(1666700336.962:2735): arch=c00000b7 syscall=56 success=yes exit=8 a0=ffffffffffffff9c a1=aaaae665d130 a2=0 a3=0 items=0 ppid=1631 pid=23209 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="sshd" exe="/usr/sbin/sshd" subj=system_u:syste>
Oct 25 20:18:56 iZbp14pox71mwfc9iyd4frZ kernel: audit: type=1327 audit(1666700336.962:2735): proctitle=737368643A20726F6F74205B707269765D
Oct 25 20:18:56 iZbp14pox71mwfc9iyd4frZ kernel: audit: type=1400 audit(1666700336.962:2736): avc:  denied  { getattr } for  pid=23209 comm="sshd" path="/etc/pam.d/password-auth" dev="vda2" ino=34889231 scontext=system_u:system_r:sshd_t:s0-s0:c0.c1023 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file permissive=1
Oct 25 20:18:56 iZbp14pox71mwfc9iyd4frZ kernel: audit: type=1300 audit(1666700336.962:2736): arch=c00000b7 syscall=79 success=yes exit=0 a0=8 a1=ffffaba95938 a2=ffffe6428c68 a3=1000 items=0 ppid=1631 pid=23209 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="sshd" exe="/usr/sbin/sshd" subj=system_u:system>
Oct 25 20:18:56 iZbp14pox71mwfc9iyd4frZ kernel: audit: type=1327 audit(1666700336.962:2736): proctitle=737368643A20726F6F74205B707269765D
Oct 25 20:18:56 iZbp14pox71mwfc9iyd4frZ audit[23209]: USER_AUTH pid=23209 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=pubkey acct="root" exe="/usr/sbin/sshd" hostname=? addr=140.205.118.26 terminal=ssh res=failed'
Oct 25 20:18:57 iZbp14pox71mwfc9iyd4frZ sshd[23209]: Failed password for root from 140.205.118.26 port 52656 ssh2
Oct 25 20:18:57 iZbp14pox71mwfc9iyd4frZ sshd[23209]: Failed password for root from 140.205.118.26 port 52656 ssh2
Oct 25 20:18:57 iZbp14pox71mwfc9iyd4frZ audit[23209]: CRYPTO_KEY_USER pid=23209 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=destroy kind=session fp=? direction=both spid=23210 suid=74 rport=52656 laddr=192.168.1.211 lport=22  exe="/usr/sbin/sshd" hostname=? addr=140.205.118.26 terminal=? res=success'
Oct 25 20:18:57 iZbp14pox71mwfc9iyd4frZ audit[23209]: CRYPTO_KEY_USER pid=23209 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=destroy kind=server fp=SHA256:aa:50:f9:51:82:34:ac:a6:75:fa:5e:1b:56:e9:cd:f8:c7:a3:41:f9:f7:51:24:be:4f:62:a5:4f:5b:09:f1:9d direction=? spid=23210 suid=74  exe="/usr/sbin/sshd" hostname=? a>
Oct 25 20:18:57 iZbp14pox71mwfc9iyd4frZ sshd[23209]: Connection closed by authenticating user root 140.205.118.26 port 52656 [preauth]
Oct 25 20:18:57 iZbp14pox71mwfc9iyd4frZ audit[23209]: USER_ERR pid=23209 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=PAM:bad_ident grantors=? acct="?" exe="/usr/sbin/sshd" hostname=140.205.118.26 addr=140.205.118.26 terminal=ssh res=failed'
Oct 25 20:18:57 iZbp14pox71mwfc9iyd4frZ audit[23209]: CRYPTO_KEY_USER pid=23209 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=destroy kind=server fp=SHA256:aa:50:f9:51:82:34:ac:a6:75:fa:5e:1b:56:e9:cd:f8:c7:a3:41:f9:f7:51:24:be:4f:62:a5:4f:5b:09:f1:9d direction=? spid=23209 suid=0  exe="/usr/sbin/sshd" hostname=? ad>
Oct 25 20:18:57 iZbp14pox71mwfc9iyd4frZ audit[23209]: USER_LOGIN pid=23209 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=login acct="root" exe="/usr/sbin/sshd" hostname=? addr=140.205.118.26 terminal=ssh res=failed'

Version-Release number of selected component (if applicable):
# cat /etc/os-release
NAME="Anolis OS"
VERSION="23"
ID="anolis"
VERSION_ID="23"
PLATFORM_ID="platform:an23"
PRETTY_NAME="Anolis OS 23"
ANSI_COLOR="0;31"
HOME_URL="https://openanolis.cn/"
BUG_REPORT_URL="https://bugzilla.openanolis.cn/"

# cat /etc/image-id
image_name="Anolis OS 23.  64 bit"
image_id="anolis_23__x64_20G_alibase_20221019.vhd"
release_date="20221019203103"

# uname -a
Linux iZ2zeim2hatc8gk7h1ih02Z 5.19.0-2_rc1.an23.x86_64 #1 SMP PREEMPT_DYNAMIC Sun Oct 9 11:02:36 CST 2022 x86_64 GNU/Linux


How reproducible:


Steps to Reproduce:
1.安装anolis23 preview版本的vhd镜像(配置成密码登录)
2.使用自动化框架运行镜像冒烟测试
3.查看测试结果,观测是否有因ssh连接失败而引起测试中断

Actual results:
ssh连接异常,导致测试失败

Expected results:
ssh连接无异常,测试正常运行结束

Additional info:
在连接失败的日志上面,都会出现audit[23209]: USER_AUTH pid=23209 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=pubkey acct="root" exe="/usr/sbin/sshd" hostname=? addr=140.205.118.26 terminal=ssh res=failed'日志,不知是否有直接相关性
Comment 1 xuchunmei alibaba_cloud_group 2022-10-27 14:38:04 UTC
云上只是云下测试,如果是云下,建议把cloud-init相关的组件删掉。
Comment 3 chuyang_94 alibaba_cloud_group 2022-10-27 15:27:04 UTC
(In reply to xuchunmei from comment #1)
> 云上只是云下测试,如果是云下,建议把cloud-init相关的组件删掉。

都是云上机器测试哈
Comment 4 xuchunmei alibaba_cloud_group 2022-10-27 16:28:17 UTC
(In reply to chuyang_94 from comment #3)
> (In reply to xuchunmei from comment #1)
> > 云上只是云下测试,如果是云下,建议把cloud-init相关的组件删掉。
> 
> 都是云上机器测试哈

云上机器不需要处理cloud-init。
Comment 5 xuchunmei alibaba_cloud_group 2022-10-27 17:43:47 UTC
从sshd日志记录看,一直有不同的ip进行攻击,ecs实例有公网ip。建议不要开启root密码登陆的方式,这样会促使攻击不停尝试,影响正常的root密码登陆。
日志信息:
Oct 25 20:15:24 iZ2zeim2hatc8gk7h1ih02Z sshd[30548]: error: kex_exchange_identification: banner line contains invalid characters
Oct 25 20:15:24 iZ2zeim2hatc8gk7h1ih02Z sshd[30548]: banner exchange: Connection from 8.219.119.144 port 54922: invalid format
Oct 25 20:15:24 iZ2zeim2hatc8gk7h1ih02Z sshd[30550]: error: kex_exchange_identification: client sent invalid protocol identifier "GET http://passport.baidu.com/ HTTP/1.1"
Oct 25 20:15:24 iZ2zeim2hatc8gk7h1ih02Z sshd[30550]: banner exchange: Connection from 8.219.119.144 port 56832: invalid format
Oct 25 20:15:33 iZ2zeim2hatc8gk7h1ih02Z sshd[30551]: error: kex_exchange_identification: banner line contains invalid characters
Oct 25 20:15:33 iZ2zeim2hatc8gk7h1ih02Z sshd[30551]: banner exchange: Connection from 8.219.119.144 port 49922: invalid format
Oct 25 20:17:27 iZ2zeim2hatc8gk7h1ih02Z sshd[30562]: Connection closed by 167.248.133.119 port 50588 [preauth]
Oct 25 20:18:56 iZ2zeim2hatc8gk7h1ih02Z sshd[30572]: Failed password for root from 140.205.118.26 port 52643 ssh2
Oct 25 20:18:56 iZ2zeim2hatc8gk7h1ih02Z sshd[30572]: Failed password for root from 140.205.118.26 port 52643 ssh2
建议关闭密码登陆方式,改为使用密钥对的方式。
Comment 6 chuyang_94 alibaba_cloud_group 2022-10-27 19:22:32 UTC
(In reply to xuchunmei from comment #5)
> 从sshd日志记录看,一直有不同的ip进行攻击,ecs实例有公网ip。
> 建议不要开启root密码登陆的方式,这样会促使攻击不停尝试,影响正常的root密码登陆。
> 日志信息:
> Oct 25 20:15:24 iZ2zeim2hatc8gk7h1ih02Z sshd[30548]: error:
> kex_exchange_identification: banner line contains invalid characters
> Oct 25 20:15:24 iZ2zeim2hatc8gk7h1ih02Z sshd[30548]: banner exchange:
> Connection from 8.219.119.144 port 54922: invalid format
> Oct 25 20:15:24 iZ2zeim2hatc8gk7h1ih02Z sshd[30550]: error:
> kex_exchange_identification: client sent invalid protocol identifier "GET
> http://passport.baidu.com/ HTTP/1.1"
> Oct 25 20:15:24 iZ2zeim2hatc8gk7h1ih02Z sshd[30550]: banner exchange:
> Connection from 8.219.119.144 port 56832: invalid format
> Oct 25 20:15:33 iZ2zeim2hatc8gk7h1ih02Z sshd[30551]: error:
> kex_exchange_identification: banner line contains invalid characters
> Oct 25 20:15:33 iZ2zeim2hatc8gk7h1ih02Z sshd[30551]: banner exchange:
> Connection from 8.219.119.144 port 49922: invalid format
> Oct 25 20:17:27 iZ2zeim2hatc8gk7h1ih02Z sshd[30562]: Connection closed by
> 167.248.133.119 port 50588 [preauth]
> Oct 25 20:18:56 iZ2zeim2hatc8gk7h1ih02Z sshd[30572]: Failed password for
> root from 140.205.118.26 port 52643 ssh2
> Oct 25 20:18:56 iZ2zeim2hatc8gk7h1ih02Z sshd[30572]: Failed password for
> root from 140.205.118.26 port 52643 ssh2
> 建议关闭密码登陆方式,改为使用密钥对的方式。

之前anolis7/8版本没有出现ssh断连问题,麻烦看下这块是不是跟云助手直接相关