Bug 308 - [centos8.4迁移anolis8.4]系统迁移后mysql启动异常
Summary: [centos8.4迁移anolis8.4]系统迁移后mysql启动异常
Status: RESOLVED FIXED
Alias: None
Product: Anolis OS 8
Classification: Anolis OS
Component: OsMigration (show other bugs) OsMigration
Version: 8.4
Hardware: All Linux
: P2-High S2-major
Target Milestone: ---
Assignee: xuchunmei
QA Contact: shuming
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-01-10 16:53 UTC by anolislw
Modified: 2022-03-24 04:36 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description anolislw alibaba_cloud_group 2022-01-10 16:53:12 UTC
[问题简述]
centos8.4系统先部署了mysqld服务(加入开机自启动),随后系统迁移至anolis8.4环境reboot后mysqld启动异常
目前存在该问题的迁移对象有如下
--------------------------------
CentOS 8.0 ->Anolis OS 8.2(x86)
CentOS 8.1 ->Anolis OS 8.2(x86)
CentOS 8.2 ->Anolis OS 8.2(x86)
CentOS 8.3 ->Anolis OS 8.4(x86)
CentOS 8.4 ->Anolis OS 8.4(x86/aarch64)
----------------------------------


[机器环境]
------ 系统迁移前以centos8.4环境为例 ------
[root@iZ2zedci5zyx93mnf6sz5vZ ~]# cat /etc/redhat-release
CentOS Linux release 8.4.2105

[root@iZ2zedci5zyx93mnf6sz5vZ ~]# cat /etc/os-release
NAME="CentOS Linux"
VERSION="8"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="8"
PLATFORM_ID="platform:el8"
PRETTY_NAME="CentOS Linux 8"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:8"
HOME_URL="https://centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"
CENTOS_MANTISBT_PROJECT="CentOS-8"
CENTOS_MANTISBT_PROJECT_VERSION="8"

[root@iZ2zedci5zyx93mnf6sz5vZ ~]# uname -a
Linux iZ2zedci5zyx93mnf6sz5vZ 4.18.0-305.19.1.el8_4.x86_64 #1 SMP Wed Sep 15 15:39:39 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

[root@iZ2zedci5zyx93mnf6sz5vZ ~]# rpm -qa | grep kernel
kernel-modules-4.18.0-305.3.1.el8.x86_64
kernel-tools-libs-4.18.0-305.19.1.el8_4.x86_64
kernel-core-4.18.0-305.3.1.el8.x86_64
kernel-core-4.18.0-305.19.1.el8_4.x86_64
kernel-4.18.0-305.19.1.el8_4.x86_64
kernel-tools-4.18.0-305.19.1.el8_4.x86_64
kernel-4.18.0-305.3.1.el8.x86_64
kernel-headers-4.18.0-305.19.1.el8_4.x86_64
kernel-modules-4.18.0-305.19.1.el8_4.x86_64
kernel-devel-4.18.0-305.19.1.el8_4.x86_64

[root@iZ2zedci5zyx93mnf6sz5vZ ~]# yum repolist 
repo id               repo name
appstream             CentOS Linux 8 - AppStream
baseos                CentOS Linux 8 - BaseOS
docker-ce             docker-ce
epel                  Extra Packages for Enterprise Linux 8 - x86_64
extras                CentOS Linux 8 - Extras

[root@iZ2zedci5zyx93mnf6sz5vZ ~]# systemctl --failed
0 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.

[root@iZ2zedci5zyx93mnf6sz5vZ ~]# systemctl list-unit-files | grep enabled | grep mysqld
mysqld.service                             enabled     

-------  系统迁移后机器环境 -------------
[root@iZ2zedci5zyx93mnf6sz5vZ ~]# cat /etc/redhat-release
Anolis OS release 8.4
[root@iZ2zedci5zyx93mnf6sz5vZ ~]# cat /etc/os-release
NAME="Anolis OS"
VERSION="8.4"
ID="anolis"
ID_LIKE="rhel fedora centos"
VERSION_ID="8.4"
PLATFORM_ID="platform:an8"
PRETTY_NAME="Anolis OS 8.4"
ANSI_COLOR="0;31"
HOME_URL="https://openanolis.cn/"

[root@iZ2zedci5zyx93mnf6sz5vZ ~]# uname -r
4.18.0-348.2.1.an8_4.x86_64
[root@iZ2zedci5zyx93mnf6sz5vZ ~]#
[root@iZ2zedci5zyx93mnf6sz5vZ ~]# systemctl --failed
  UNIT           LOAD   ACTIVE SUB    DESCRIPTION
● mysqld.service loaded failed failed MySQL 8.0 database server

LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.

1 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.
[root@iZ2zedci5zyx93mnf6sz5vZ ~]#
[root@iZ2zedci5zyx93mnf6sz5vZ ~]# systemctl list-unit-files | grep enabled | grep mysqld
mysqld.service                             enabled
[root@iZ2zedci5zyx93mnf6sz5vZ ~]# yum repolist
Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/CentOS-Linux-epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist
repo id                                       repo name
AppStream                                     AnolisOS-8.4 - AppStream
BaseOS                                        AnolisOS-8.4 - BaseOS
Extras                                        AnolisOS-8.4 - Extras
PowerTools                                    AnolisOS-8.4 - PowerTools
docker-ce                                     docker-ce
epel                                          Extra Packages for Enterprise Linux 8 - x86_64


[期望结果]
centos8.x环境迁移至anolis8.x环境mysqld服务正常

[实际结果]
[root@iZ2zedci5zyx93mnf6sz5vZ ~]# cat /etc/redhat-release
Anolis OS release 8.4
[root@iZ2zedci5zyx93mnf6sz5vZ ~]# systemctl --failed
  UNIT           LOAD   ACTIVE SUB    DESCRIPTION
● mysqld.service loaded failed failed MySQL 8.0 database server

LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.

1 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.
[root@iZ2zedci5zyx93mnf6sz5vZ ~]#
[root@iZ2zedci5zyx93mnf6sz5vZ ~]# systemctl list-unit-files | grep enabled | grep mysqld
mysqld.service                             enabled

[root@iZ2zedci5zyx93mnf6sz5vZ ~]# systemctl status mysqld
● mysqld.service - MySQL 8.0 database server
   Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Mon 2022-01-10 14:42:16 CST; 2h 0min ago
  Process: 1518 ExecStopPost=/usr/libexec/mysql-wait-stop (code=exited, status=0/SUCCESS)
  Process: 1191 ExecStart=/usr/libexec/mysqld --basedir=/usr (code=exited, status=1/FAILURE)
  Process: 947 ExecStartPre=/usr/libexec/mysql-prepare-db-dir mysqld.service (code=exited, status=0/SUCCESS)
 Main PID: 1191 (code=exited, status=1/FAILURE)
   Status: "Server startup in progress"

Jan 10 14:42:09 iZ2zedci5zyx93mnf6sz5vZ systemd[1]: Starting MySQL 8.0 database server...
Jan 10 14:42:16 iZ2zedci5zyx93mnf6sz5vZ systemd[1]: mysqld.service: Main process exited, code=exited, status=1/FAILURE
Jan 10 14:42:16 iZ2zedci5zyx93mnf6sz5vZ systemd[1]: mysqld.service: Failed with result 'exit-code'.
Jan 10 14:42:16 iZ2zedci5zyx93mnf6sz5vZ systemd[1]: Failed to start MySQL 8.0 database server.
[root@iZ2zedci5zyx93mnf6sz5vZ ~]#
[root@iZ2zedci5zyx93mnf6sz5vZ ~]# systemctl restart mysqld
Job for mysqld.service failed because the control process exited with error code.
See "systemctl status mysqld.service" and "journalctl -xe" for details.
[root@iZ2zedci5zyx93mnf6sz5vZ ~]#
[root@iZ2zedci5zyx93mnf6sz5vZ ~]# journalctl -xe
-- Support: https://access.redhat.com/support
--
-- The unit sysstat-collect.service has successfully entered the 'dead' state.
Jan 10 16:40:36 iZ2zedci5zyx93mnf6sz5vZ systemd[1]: Started system activity accounting tool.
-- Subject: Unit sysstat-collect.service has finished start-up
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- Unit sysstat-collect.service has finished starting up.
--
-- The start-up result is done.
Jan 10 16:40:47 iZ2zedci5zyx93mnf6sz5vZ systemd[1]: Configuration file /etc/systemd/system/cloudmonitor.service is marked execu>
Jan 10 16:43:10 iZ2zedci5zyx93mnf6sz5vZ systemd[1]: Starting MySQL 8.0 database server...
-- Subject: Unit mysqld.service has begun start-up
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- Unit mysqld.service has begun starting up.
Jan 10 16:43:15 iZ2zedci5zyx93mnf6sz5vZ systemd[1]: mysqld.service: Main process exited, code=exited, status=1/FAILURE
Jan 10 16:43:15 iZ2zedci5zyx93mnf6sz5vZ systemd[1]: mysqld.service: Failed with result 'exit-code'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- The unit mysqld.service has entered the 'failed' state with result 'exit-code'.
Jan 10 16:43:15 iZ2zedci5zyx93mnf6sz5vZ systemd[1]: Failed to start MySQL 8.0 database server.
-- Subject: Unit mysqld.service has failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- Unit mysqld.service has failed.
--
-- The result is failed.
lines 2713-2745/2745 (END)

[复现步骤]
#迁移前部署mysql服务:
在centos8.4环境下不是mysqld服务
dnf install -y mysql-server
systemctl start mysqld
systemctl enable mysqld
简单做创库建表操作,确保迁移前mysql功能正常
#开始迁移操作
wget https://gitee.com/anolis/centos2anolis/raw/master/centos2anolis.py
yum -y install python3
python3 centos2anolis.py -v 8.4 -V
#上述迁移成功后reboot环境
systemctl status mysqld

[出现概率]
目前迁移了,以下版本,都存在该问题
CentOS 8.0 ->Anolis OS 8.2(x86)
CentOS 8.1 ->Anolis OS 8.2(x86)
CentOS 8.2 ->Anolis OS 8.2(x86)
CentOS 8.3 ->Anolis OS 8.4(x86)
CentOS 8.4 ->Anolis OS 8.4(x86/aarch64)
Comment 1 xuchunmei alibaba_cloud_group 2022-01-12 19:35:16 UTC
mysql服务启动报错信息:
2022-01-12T11:29:24.971892Z 1 [ERROR] [MY-013171] [InnoDB] Cannot boot server version 80021 on data directory built by version 80026. Downgrade is not supported
mysqld: Can't open file: 'mysql.ibd' (errno: 0 - )

应该是8.4上mysql版本比当前centos最新的(8.5)版本低导致的问题。
Comment 2 yunqi-zwt alibaba_cloud_group 2022-03-04 10:37:37 UTC
当前 mysql-8.0.26 已经发布。该问题应该已经修复了, 麻烦再验证下。