frm ibd恢复 – Database SOS

2022-07-08T04:11:27.413455Z 0 [Note] /www/server/mysql/bin/mysqld (mysqld 5.7.34-log) starting as process 2144 ... 2022-07-08T04:11:27.495536Z 0 [Note] InnoDB: PUNCH HOLE support available 2022-07-08T04:11:27.495559Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2022-07-08T04:11:27.495562Z 0 [Note] InnoDB: Uses event mutexes 2022-07-08T04:11:27.495565Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier 2022-07-08T04:11:27.495568Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.11 2022-07-08T04:11:27.495571Z 0 [Note] InnoDB: Using Linux native AIO 2022-07-08T04:11:27.496130Z 0 [Note] InnoDB: Number of pools: 1 2022-07-08T04:11:27.496227Z 0 [Note] InnoDB: Using CPU crc32 instructions 2022-07-08T04:11:27.510618Z 0 [Note] InnoDB: Initializing buffer pool, total size=256M,instances=1,chunk size=128M 2022-07-08T04:11:27.520144Z 0 [Note] InnoDB: Completed initialization of buffer pool 2022-07-08T04:11:27.522095Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2022-07-08T04:11:27.532135Z 0 [Note] InnoDB: The first innodb_system data file 'ibdata1' did not exist. A new tablespace will be created! 2022-07-08T04:11:27.532259Z 0 [Note] InnoDB: Setting file '/www/server/data/ibdata1' size to 10 MB. Physically writing the file full; Please wait ... 2022-07-08T04:11:27.760116Z 0 [Note] InnoDB: File '/www/server/data/ibdata1' size is now 10 MB. 2022-07-08T04:11:27.760338Z 0 [Note] InnoDB: Setting log file /www/server/data/ib_logfile101 size to 128MB 2022-07-08T04:11:27.760414Z 0 [Note] InnoDB: Progress in MB: 100 2022-07-08T04:11:28.940355Z 0 [Note] InnoDB: Setting log file /www/server/data/ib_logfile1 size to 128 MB 2022-07-08T04:11:28.940442Z 0 [Note] InnoDB: Progress in MB: 100 2022-07-08T04:11:30.517357Z 0 [Note] InnoDB: Renaming log file /www/server/data/ib_logfile101 to /www/server/data/ib_logfile0 2022-07-08T04:11:30.517394Z 0 [Warning] InnoDB: New log files created, LSN=45790 2022-07-08T04:11:30.517401Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables 2022-07-08T04:11:30.517425Z 0 [Note] InnoDB: Setting file '/www/server/data/ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... 2022-07-08T04:11:30.609146Z 0 [Note] InnoDB: File '/www/server/data/ibtmp1' size is now 12 MB. 2022-07-08T04:11:30.609236Z 0 [Note] InnoDB: Doublewrite buffer not found: creating new 2022-07-08T04:11:30.631133Z 0 [Note] InnoDB: Doublewrite buffer created 2022-07-08T04:11:31.160847Z 0 [Note] InnoDB: 96 redo rollback segment(s) found.96 redo rollback segment(s) are active. 2022-07-08T04:11:31.160860Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active. 2022-07-08T04:11:31.160970Z 0 [Warning] InnoDB: Creating foreign key constraint system tables. 2022-07-08T04:11:31.194147Z 0 [Note] InnoDB: Foreign key constraint system tables created 2022-07-08T04:11:31.194175Z 0 [Note] InnoDB: Creating tablespace and datafile system tables. 2022-07-08T04:11:31.195079Z 0 [Note] InnoDB: Tablespace and datafile system tables created. 2022-07-08T04:11:31.195098Z 0 [Note] InnoDB: Creating sys_virtual system tables. 2022-07-08T04:11:31.195974Z 0 [Note] InnoDB: sys_virtual table created 2022-07-08T04:11:31.196099Z 0 [Note] InnoDB: Waiting for purge to start 2022-07-08T04:11:31.246167Z 0 [Note] InnoDB: 5.7.34 started; log sequence number 0 2022-07-08T04:11:31.246379Z 0 [Note] Plugin 'FEDERATED' is disabled. 2022-07-08T04:11:31.248996Z 0 [Warning] InnoDB: Cannot open table mysql/plugin from the internal data dictionary of InnoDB though the .frm file for the table exists. Please refer to http://dev.mysql.com/doc/refman/5.7/en/innodb-troubleshooting.html for how to resolve the issue. mysqld: Table 'mysql.plugin' doesn't exist

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：frm和ibd文件数据库恢复

这次客户rm -rf /var/lib/mysql删除文件,删除一半及时终止,但是已经有很多mysql相关文件被删除,重要的ibdata文件已经被删除,并且客户尝试了大量的恢复工作,对该分区进行了大量的写入操作,导致后面通过对xfs文件系统进行分析，确认无法恢复对应的ibdata文件.比较幸运客户需要的核心的mysql库都还在(frm和ibd文件还存在)

对于这种情况,可以参考以前类似的处理方法:[MySQL异常恢复]mysql ibd文件恢复
由于客户无法提供创建表语句需要通过对frm进行解析获取语句,利用mysqlfrm获取表创建语句

E:\3>mysqlfrm --server=root:oracle@192.168.222.79:3306 --diagnostic T_XIFENFEI.frm
WARNING: Using a password on the command line interface can be insecure.
# Source on 192.168.222.79: ... connected.
# CAUTION: The diagnostic mode is a best-effort parse of the .frm file. As such, it may not identify all of 
  the components of the table correctly. This is especially true for damaged files. 
  It will also not read the default values for the columns and the resulting statement may not be syntactically correct.
# Reading .frm file for EVALUATOR_T.frm:
# The .frm file is a TABLE.
# CREATE TABLE Statement:

CREATE TABLE `T_XIFENFEI` (
  `ID` varchar(32) COLLATE `utf8_general_ci` DEFAULT NULL comment '主键',
  `BO_TYPE_DEFINE_ID` varchar(32) COLLATE `utf8_general_ci` DEFAULT NULL comment '业务对象类型ID',
  `MAIN_ID` varchar(32) COLLATE `utf8_general_ci` DEFAULT NULL comment '业务对象主表记录ID',
  `PARENT_ID` varchar(32) COLLATE `utf8_general_ci` DEFAULT NULL comment '父ID',
  `ROW_NUM` decimal(32,0) DEFAULT NULL comment '行号',
  `VERSION` decimal(32,6) DEFAULT NULL comment '版本',
  `CREATE_DATE` datetime DEFAULT NULL comment '创建时间',
  `UPDATE_DATE` datetime DEFAULT NULL comment '更新时间',
  `BO_SOURCE_ROW_ID` varchar(32) COLLATE `utf8_general_ci` DEFAULT NULL comment '来源明细行ID',
  `EVALUATORS` text COLLATE `utf8_general_ci` DEFAULT NULL,
  `IMPORTANCE` decimal(32,6) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8, COMMENT '评分人';
#...done.

对于有些获取语句失败,比如类似这样错误

E:\TEMP\10000246_1108\db\ync2_fssc_2000003>mysqlfrm --server=root:oracle@192.168.222.79:3306 --diagnostic T_XFF.frm
Traceback (most recent call last):
  File "G:\ade\build\Python-2.7.6-windows-x86-64bit\lib\site-packages\cx_Freeze\initscripts\Console.py",
    line 27, in <module>
  File "scripts\mysqlfrm.py", line 419, in <module>
  File ".\mysql\utilities\command\read_frm.py", line 396, in read_frm_files_diagnostic
  File ".\mysql\utilities\common\frm_reader.py", line 1538, in show_create_table_statement
  File ".\mysql\utilities\common\frm_reader.py", line 1385, in _build_create_statement
  File ".\mysql\utilities\common\frm_reader.py", line 1273, in _get_key_columns
IndexError: list index out of range

使用专门的工具对其进行解析

然后利用这些创建表语句在库中创建表,并利用以下方法进行操作

mysql> alter table  `t_xifenfei` discard tablespace;        
Query OK, 0 rows affected (0.00 sec)

--上传老的t_xifenfei.ibd文件,并修改所有者和属组

mysql> alter table  `t_xifenfei` import tablespace;                
Query OK, 0 rows affected, 2 warnings (0.01 sec)

mysql> select count(1) from   `t_xifenfei` ;              
+----------+
| count(1) |
+----------+
|       78 |
+----------+
1 row in set (0.00 sec)

使用类似的方法对于数据进行批量处理,然后使用mysqldump进行导出.在这个ibd的discard和import的过程中,有些异常情况这三种错误的处理

mysql> alter table T_LOG_XIFENFEI                   import tablespace;
ERROR 1808 (HY000): Schema mismatch (Table has ROW_TYPE_DYNAMIC row format, .ibd file has ROW_TYPE_COMPACT row format.)
mysql> alter table     `T_LOG_XIFENFEI` import tablespace;
ERROR 1817 (HY000): Index corrupt: Externally stored column(4) has a reference length of 4 in the cluster index PRIMARY
mysql> alter table       `T_LOG_XIFENFEI` import tablespace;
ERROR 1815 (HY000): Internal error: Cannot reset LSNs in table `XFF`.`T_LOG_XIFENFEI` : Data structure corruption

Schema mismatch (Table has ROW_TYPE_DYNAMIC row format, .ibd file has ROW_TYPE_COMPACT row format.) 这种错误是由于row_format设置不正确导致,重新创建表使用正确的row_format然后执行discard和import操作.
Index corrupt: Externally stored column(4) has a reference length of 4 in the cluster index PRIMARY 这种错误是由于表的创建语句和ibd中记录数据不匹配,主要是由于创建表语句不完全正确导致,重新获取正确语句进行恢复
Internal error: Cannot reset LSNs in table `XFF`.`T_LOG_XIFENFEI` : Data structure corruption 这种错误是由于ibd文件本身不一致无法使用该方法恢复,对于这类情况使用我们专业的工具进行处理

分类目录归档：frm ibd恢复