InnoDB: Cannot open table db/tab from the internal data dictionary of InnoDB though the .frm file for the table exists

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:InnoDB: Cannot open table db/tab from the internal data dictionary of InnoDB though the .frm file for the table exists

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

有客户找到我说mysql无法正常使用报错如下(库名.表名 doesn’t exist):
mysql-doesnot-exist


客户确认表的frm和ibd文件均存在.通过检查mysql日志,发现大量类似异常
mysql-1

而且情况一样ibd和frm文件均存在,系统日志中提示:
2022-07-08T08:37:57.935514Z 1423 [Warning] InnoDB: Cannot open table 库名/表名 from the internal data dictionary of InnoDB though the .frm file for the table exists. Please refer to http://dev.mysql.com/doc/refman/5.7/en/innodb-troubleshooting.html for how to resolve the issue.
根据对mysql的认知,出现此类问题,很可能是mysql的ibdata文件出了问题,对日志进行分析,发现类似记录

2022-07-08T04:11:27.413455Z 0 [Note] /www/server/mysql/bin/mysqld (mysqld 5.7.34-log) starting as process 2144 ...
2022-07-08T04:11:27.495536Z 0 [Note] InnoDB: PUNCH HOLE support available
2022-07-08T04:11:27.495559Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2022-07-08T04:11:27.495562Z 0 [Note] InnoDB: Uses event mutexes
2022-07-08T04:11:27.495565Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2022-07-08T04:11:27.495568Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2022-07-08T04:11:27.495571Z 0 [Note] InnoDB: Using Linux native AIO
2022-07-08T04:11:27.496130Z 0 [Note] InnoDB: Number of pools: 1
2022-07-08T04:11:27.496227Z 0 [Note] InnoDB: Using CPU crc32 instructions
2022-07-08T04:11:27.510618Z 0 [Note] InnoDB: Initializing buffer pool, total size=256M,instances=1,chunk size=128M
2022-07-08T04:11:27.520144Z 0 [Note] InnoDB: Completed initialization of buffer pool
2022-07-08T04:11:27.522095Z 0 [Note] InnoDB: If the mysqld execution user is authorized, 
        page cleaner thread priority can be changed.  See the man page of setpriority().
2022-07-08T04:11:27.532135Z 0 [Note] InnoDB: The first innodb_system data file 'ibdata1' did not exist. 
       A new tablespace will be created!
2022-07-08T04:11:27.532259Z 0 [Note] InnoDB: Setting file '/www/server/data/ibdata1' size to 10 MB. 
        Physically writing the file full; Please wait ...
2022-07-08T04:11:27.760116Z 0 [Note] InnoDB: File '/www/server/data/ibdata1' size is now 10 MB.
2022-07-08T04:11:27.760338Z 0 [Note] InnoDB: Setting log file /www/server/data/ib_logfile101 size to 128MB
2022-07-08T04:11:27.760414Z 0 [Note] InnoDB: Progress in MB:
 100
2022-07-08T04:11:28.940355Z 0 [Note] InnoDB: Setting log file /www/server/data/ib_logfile1 size to 128 MB
2022-07-08T04:11:28.940442Z 0 [Note] InnoDB: Progress in MB:
 100
2022-07-08T04:11:30.517357Z 0 [Note] InnoDB: Renaming log file /www/server/data/ib_logfile101 
      to /www/server/data/ib_logfile0
2022-07-08T04:11:30.517394Z 0 [Warning] InnoDB: New log files created, LSN=45790
2022-07-08T04:11:30.517401Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2022-07-08T04:11:30.517425Z 0 [Note] InnoDB: Setting file '/www/server/data/ibtmp1' size to 12 MB. 
       Physically writing the file full; Please wait ...
2022-07-08T04:11:30.609146Z 0 [Note] InnoDB: File '/www/server/data/ibtmp1' size is now 12 MB.
2022-07-08T04:11:30.609236Z 0 [Note] InnoDB: Doublewrite buffer not found: creating new
2022-07-08T04:11:30.631133Z 0 [Note] InnoDB: Doublewrite buffer created
2022-07-08T04:11:31.160847Z 0 [Note] InnoDB: 96 redo rollback segment(s) found.96 redo rollback segment(s) are active.
2022-07-08T04:11:31.160860Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active.
2022-07-08T04:11:31.160970Z 0 [Warning] InnoDB: Creating foreign key constraint system tables.
2022-07-08T04:11:31.194147Z 0 [Note] InnoDB: Foreign key constraint system tables created
2022-07-08T04:11:31.194175Z 0 [Note] InnoDB: Creating tablespace and datafile system tables.
2022-07-08T04:11:31.195079Z 0 [Note] InnoDB: Tablespace and datafile system tables created.
2022-07-08T04:11:31.195098Z 0 [Note] InnoDB: Creating sys_virtual system tables.
2022-07-08T04:11:31.195974Z 0 [Note] InnoDB: sys_virtual table created
2022-07-08T04:11:31.196099Z 0 [Note] InnoDB: Waiting for purge to start
2022-07-08T04:11:31.246167Z 0 [Note] InnoDB: 5.7.34 started; log sequence number 0
2022-07-08T04:11:31.246379Z 0 [Note] Plugin 'FEDERATED' is disabled.
2022-07-08T04:11:31.248996Z 0 [Warning] InnoDB: Cannot open table mysql/plugin from the internal data dictionary 
        of InnoDB  though the .frm file for the table exists. 
Please refer to http://dev.mysql.com/doc/refman/5.7/en/innodb-troubleshooting.html for how to resolve the issue.
mysqld: Table 'mysql.plugin' doesn't exist

通过上述日志可以确认,数据库应该被重新初始化了,导致以前库的ibd和frm文件无法被正常访问.对于此类情况,可以参考以前类似恢复案例:frm和ibd文件数据库恢复

frm和ibd文件数据库恢复

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:frm和ibd文件数据库恢复

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

这次客户rm -rf /var/lib/mysql删除文件,删除一半及时终止,但是已经有很多mysql相关文件被删除,重要的ibdata文件已经被删除,并且客户尝试了大量的恢复工作,对该分区进行了大量的写入操作,导致后面通过对xfs文件系统进行分析,确认无法恢复对应的ibdata文件.比较幸运客户需要的核心的mysql库都还在(frm和ibd文件还存在)
20211224105004


对于这种情况,可以参考以前类似的处理方法:[MySQL异常恢复]mysql ibd文件恢复
由于客户无法提供创建表语句需要通过对frm进行解析获取语句,利用mysqlfrm获取表创建语句

E:\3>mysqlfrm --server=root:oracle@192.168.222.79:3306 --diagnostic T_XIFENFEI.frm
WARNING: Using a password on the command line interface can be insecure.
# Source on 192.168.222.79: ... connected.
# CAUTION: The diagnostic mode is a best-effort parse of the .frm file. As such, it may not identify all of 
  the components of the table correctly. This is especially true for damaged files. 
  It will also not read the default values for the columns and the resulting statement may not be syntactically correct.
# Reading .frm file for EVALUATOR_T.frm:
# The .frm file is a TABLE.
# CREATE TABLE Statement:

CREATE TABLE `T_XIFENFEI` (
  `ID` varchar(32) COLLATE `utf8_general_ci` DEFAULT NULL comment '主键',
  `BO_TYPE_DEFINE_ID` varchar(32) COLLATE `utf8_general_ci` DEFAULT NULL comment '业务对象类型ID',
  `MAIN_ID` varchar(32) COLLATE `utf8_general_ci` DEFAULT NULL comment '业务对象主表记录ID',
  `PARENT_ID` varchar(32) COLLATE `utf8_general_ci` DEFAULT NULL comment '父ID',
  `ROW_NUM` decimal(32,0) DEFAULT NULL comment '行号',
  `VERSION` decimal(32,6) DEFAULT NULL comment '版本',
  `CREATE_DATE` datetime DEFAULT NULL comment '创建时间',
  `UPDATE_DATE` datetime DEFAULT NULL comment '更新时间',
  `BO_SOURCE_ROW_ID` varchar(32) COLLATE `utf8_general_ci` DEFAULT NULL comment '来源明细行ID',
  `EVALUATORS` text COLLATE `utf8_general_ci` DEFAULT NULL,
  `IMPORTANCE` decimal(32,6) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8, COMMENT '评分人';
#...done.

对于有些获取语句失败,比如类似这样错误

E:\TEMP\10000246_1108\db\ync2_fssc_2000003>mysqlfrm --server=root:oracle@192.168.222.79:3306 --diagnostic T_XFF.frm
Traceback (most recent call last):
  File "G:\ade\build\Python-2.7.6-windows-x86-64bit\lib\site-packages\cx_Freeze\initscripts\Console.py",
    line 27, in <module>
  File "scripts\mysqlfrm.py", line 419, in <module>
  File ".\mysql\utilities\command\read_frm.py", line 396, in read_frm_files_diagnostic
  File ".\mysql\utilities\common\frm_reader.py", line 1538, in show_create_table_statement
  File ".\mysql\utilities\common\frm_reader.py", line 1385, in _build_create_statement
  File ".\mysql\utilities\common\frm_reader.py", line 1273, in _get_key_columns
IndexError: list index out of range

使用专门的工具对其进行解析
20211224105004


然后利用这些创建表语句在库中创建表,并利用以下方法进行操作

mysql> alter table  `t_xifenfei` discard tablespace;        
Query OK, 0 rows affected (0.00 sec)

--上传老的t_xifenfei.ibd文件,并修改所有者和属组

mysql> alter table  `t_xifenfei` import tablespace;                
Query OK, 0 rows affected, 2 warnings (0.01 sec)

mysql> select count(1) from   `t_xifenfei` ;              
+----------+
| count(1) |
+----------+
|       78 |
+----------+
1 row in set (0.00 sec)

使用类似的方法对于数据进行批量处理,然后使用mysqldump进行导出.在这个ibd的discard和import的过程中,有些异常情况这三种错误的处理

mysql> alter table T_LOG_XIFENFEI                   import tablespace;
ERROR 1808 (HY000): Schema mismatch (Table has ROW_TYPE_DYNAMIC row format, .ibd file has ROW_TYPE_COMPACT row format.)
mysql> alter table     `T_LOG_XIFENFEI` import tablespace;
ERROR 1817 (HY000): Index corrupt: Externally stored column(4) has a reference length of 4 in the cluster index PRIMARY
mysql> alter table       `T_LOG_XIFENFEI` import tablespace;
ERROR 1815 (HY000): Internal error: Cannot reset LSNs in table `XFF`.`T_LOG_XIFENFEI` : Data structure corruption

Schema mismatch (Table has ROW_TYPE_DYNAMIC row format, .ibd file has ROW_TYPE_COMPACT row format.) 这种错误是由于row_format设置不正确导致,重新创建表使用正确的row_format然后执行discard和import操作.
Index corrupt: Externally stored column(4) has a reference length of 4 in the cluster index PRIMARY 这种错误是由于表的创建语句和ibd中记录数据不匹配,主要是由于创建表语句不完全正确导致,重新获取正确语句进行恢复
Internal error: Cannot reset LSNs in table `XFF`.`T_LOG_XIFENFEI` : Data structure corruption 这种错误是由于ibd文件本身不一致无法使用该方法恢复,对于这类情况使用我们专业的工具进行处理
20211224142855


.sql文件被加密恢复

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:.sql文件被加密恢复

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

接到客户请求,有win系统文件被加密勒索,其中有一个mysql的.sql备份最为重要,咨询我们是否可以恢复.通过底层技术分析,确认该文件绝大部分数据可以恢复.
20210123225720


通过winhex分析,发现该文件主要对部分数据进行了加密
20210123231319

通过底层技术处理,可以实现绝大部分数据恢复
20210123231421

如果此类的Sql文件被加密,需要专业恢复技术支持,请联系我们:
电话/微信:17813235971    Q Q:107644445QQ咨询惜分飞    E-Mail:dba@xifenfei.com

xfs文件系统mysql删库恢复

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:xfs文件系统mysql删库恢复

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

有客户在centos 8操作系统上运行的mysql库,本来想删除一个测试表,结果反选了表,导致除该表之外表均被删除,大概有几百张表被删除.客户误操作之后,又使用一个月之前备份导致了十几张表然后终止,关闭机器,保护现场,请求我们给予支持.通过分析,发现该数据库放在/分区下面,第一时间和客户协商,对该分区进行镜像,防止由于系统运行引起的进一步覆盖.对于这类故障大概恢复思路:
1. 通过对xfs文件系统反删除操作,恢复可以恢复的被删除的mysql相关表文件
20210111185625


2. 对于该恢复出来的文件(包含ibd,myd),使用专业mysql工具进行恢复
20210111203401

3.对于异常的表,比如ibd部分损坏,缺少frm等,通过人工修复和单个对象扫描进行恢复
1)使用恢复方法:[MySQL异常恢复]mysql ibd文件恢复
2)使用4的方法进行恢复
4.对于xfs文件系统没有恢复出来的文件的表,尝试底层扫描尽可能恢复数据
恢复参考:MySQL drop database恢复(恢复方法同样适用MySQL drop table,delete,truncate table)
如果您遇到MySQL恢复问题无法自行解决,请联系我们提供专业服务,最大程度减小您的损失:
Phone:17813235971    Q Q:107644445    E-Mail:dba@xifenfei.com
我们数据库恢复原则:最大限度恢复数据,最大限度减少业务影响时间,最大程度让客户满意

又一起mysql rm删除数据库目录事故

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:又一起mysql rm删除数据库目录事故

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

有mysql用户直接使用rm命令删除了所在数据库目录文件夹(删除途中发现删除错误,直接强制终止使得部分删除,部分文件得以保留),如果数据库没有crash,这个问题不大,可以直接通过句柄进行恢复出来对应的ibd文件,然后进行一些特殊处理,数据理论上不会有丢失,参见:mysql 数据库目录被删除恢复.对于这次的情况,只能通过文件系统级别进行恢复,不过该数据库表比较多,大概删除了几千个文件,而且文件系统是ext4,导致大量文件目录被置空,而且该库还运行了一段时间
20200507231749


看来文件恢复删除这条路,基本上很难走得通,可以通过底层碎片技术恢复,通过底层扫描,找到不少碎片
20200510173725

由于涉及的库表多,一个个处理太费时间,通过批量脚本直接解析扫描出来的碎片,并且对其进行入库处理
DBLOGIN=”-uroot ”
DICT_DBNAME=test
RECOVER_DBNAME=xifenfei
WORK_PATH=”/nfs_share/mysql-xff”
cd $WORK_PATH
>/tmp/recover_table_list.$RECOVER_DBNAME
>/tmp/recover_table_parser.$RECOVER_DBNAME
>/tmp/recover_table_unload.$RECOVER_DBNAME
rm -rf /tmp/recover_table_unload_$RECOVER_DBNAME.tmp
echo “tee /tmp/recover_table_unload_$RECOVER_DBNAME.tmp” >/tmp/recover_table_unload.$RECOVER_DBNAME
echo “SET @enable_trigger = 0;”>>/tmp/recover_table_unload.$RECOVER_DBNAME
echo “SET FOREIGN_KEY_CHECKS=0;” >>/tmp/recover_table_unload.$RECOVER_DBNAME
for TABLE_NAME_ID in `mysql $DBLOGIN $DICT_DBNAME -NBe “SELECT SYS_TABLES.NAME FROM SYS_TABLES LEFT JOIN SYS_INDEXES ON SYS_TABLES.ID = SYS_INDEXES.TABLE_ID WHERE SYS_INDEXES.NAME IN (‘PRIMARY’, ‘GENERAL_CLUSTERED_INDEX’) AND SYS_TABLES.NAME LIKE ‘${RECOVER_DBNAME}/%’”`
do
PKEY=`mysql $DBLOGIN $DICT_DBNAME -NBe “SELECT SYS_INDEXES.ID FROM SYS_TABLES LEFT JOIN SYS_INDEXES ON SYS_TABLES.ID = SYS_INDEXES.TABLE_ID WHERE SYS_INDEXES.NAME IN (‘PRIMARY’, ‘GENERAL_CLUSTERED_INDEX’) AND SYS_TABLES.NAME=’$TABLE_NAME_ID’”`
echo $TABLE_NAME_ID’,’$PKEY >>/tmp/recover_table_list.$RECOVER_DBNAME
TABLE_NAME=$(echo `echo $TABLE_NAME_ID`|awk -F”/” ‘{print $2}’ )
#echo $TABLE_NAME
PAGE=”pages-ibdata1/FIL_PAGE_INDEX/`printf ‘%016u’ ${PKEY}`.page”
echo “./c_parser -5f $PAGE -b pages-ibdata1/FIL_PAGE_TYPE_BLOB -t dictionary/$TABLE_NAME.sql > dumps/default/$TABLE_NAME 2> dumps/default/$TABLE_NAME.sql” >>/tmp/recover_table_parser.$RECOVER_DBNAME
echo “source dumps/default/$TABLE_NAME.sql” >>/tmp/recover_table_unload.$RECOVER_DBNAME
echo “select ‘$TABLE_NAME.sql……done’;”>>/tmp/recover_table_unload.$RECOVER_DBNAME
done
echo /tmp/recover_table_parser.$RECOVER_DBNAME
echo /tmp/recover_table_unload.$RECOVER_DBNAME
恢复之后数据整体入库,查询部分没有覆盖表,效果尚可,最大限度抢救了数据
20200511225518

mysql ibd文件被加密恢复

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:mysql ibd文件被加密恢复

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

又一起mysql的ibd文件被加密勒索的恢复请求
20200504203408


通过分析,该文件只有部分被加密,理论上通过底层分析,可以恢复没有被破坏的部分数据
20200504203850

通过处理恢复数据
20200504212047

又一次证明,我们对于mysql文件被加密的勒索病毒,也可以实现较好恢复
如果是Innodb_file_per_table参数为false(5.6之前版本默认为false),需要通过ibdata文件进行恢复,Innodb_file_per_table如果为true,需要通过每个单独的ibd进行恢复.
如果您的数据库(oracle,mysql sql server)不幸被比特币加密,可以联系我们
Tel/微信:17813235971    Q Q:107644445 QQ咨询惜分飞    E-Mail:dba@xifenfei.com提供专业的解密恢复服务.

MySQL勒索恢复

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:MySQL勒索恢复

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

最近遇到几个mysql数据库被黑客删除库,并且留下比特币勒索信息在每个库的WARNING表中

mysql> desc WARNING
    -> ;
+-----------------+----------+------+-----+---------+-------+
| Field           | Type     | Null | Key | Default | Extra |
+-----------------+----------+------+-----+---------+-------+
| id              | int(11)  | YES  |     | NULL    |       |
| warning         | longtext | YES  |     | NULL    |       |
| Bitcoin_Address | longtext | YES  |     | NULL    |       |
| Email           | longtext | YES  |     | NULL    |       |
+-----------------+----------+------+-----+---------+-------+
4 rows in set (0.00 sec)

mysql> select * from WARNING;
+------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------+------------------+
| id   | warning                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        | Bitcoin_Address                    | Email            |
+------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------+------------------+
|    1 | To recover your lost Database and avoid leaking it: Send us 0.06 Bitcoin (BTC) to our Bitcoin address 1BLYhUDmnmVPVjcTWgc6gFT6DCYwbVieUD and contact us by Email with your Server IP or Domain name and a Proof of Payment. If you are unsure if we have your data, contact us and we will send you a proof. Your Database is downloaded and backed up on our servers. Backups that we have right now: xxxx,xxxxxx,xxxxxxxx,xxxxxxx . If we dont receive your payment in the next 10 Days, we will make your database public or use them otherwise. | 1BLYhUDmnmVPVjcTWgc6gFT6DCYwbVieUD | contact@sqldb.to |
+------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------+------------------+
1 row in set (0.00 sec)

大概的意思就是:我们已经把你的数据库备份,您交给我们0.06个比特币,我们把数据给你,如果10天之内我们没有收到款,即将把数据库给公开或者作为其他用途.根据我们以往接触的朋友经验,付款之后数据库也不会给你(很可能黑客根本就没有备份数据库,只是删除了数据库然后勒索比特币.

对于这类情况,通过分析,确认黑客是删除了数据库,在没有覆盖的情况下,我们可以对其数据进行恢复,处理类似:MySQL drop database恢复(恢复方法同样适用MySQL drop table,delete,truncate table)最大限度缓解因为数据库被破坏带来的损失.
20200303125417
如果您也遭遇到该问题,请保护现场,不要导入备份数据库,不要对数据所在分区进行写操作(现场保护的越好,数据恢复效果越好),对相关磁盘进行镜像,防止二次破坏.我们可以提供专业的mysql恢复服务,为您减少损失.

mysql 数据库目录被删除恢复

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:mysql 数据库目录被删除恢复

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

接到朋友请求,把mysql数据库的datadir目录给删除了,数据库目前还处于运行状态,但是很多操作已经无法正常进行
数据库可以登录,但是已经看不到任何业务数据库,可以结合表名查询

[root@hy-db-xff-s-110 mysql3306]# mysql -uroot -ptSQghoV^J1GE^U8*wPElImv5
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 443214
Server version: 5.7.21-log MySQL Community Server (GPL)

Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
+--------------------+
1 row in set (0.00 sec)

mysql> select count(1) from xifenfei.orders;
+----------+
| count(1) |
+----------+
| 16451326 |
+----------+
1 row in set (4.17 sec)

数据无法导出(into outfile不行是由于secure-file-priv参数默认导致)

mysql> select * from xifenfei.orders into outfile '/bakcup/orders_new.sql' 
   FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' LINES TERMINATED BY '\n';
ERROR 1290 (HY000): The MySQL server is running with the --secure-file-priv option so it cannot execute this statement

[root@hy-db-cps-s-110 fd]# mysqldump  -uroot -pwww.xifenfei.com xifenfei orders >/linshi/1.sql
mysqldump: [Warning] Using a password on the command line interface can be insecure.
mysqldump: Got error: 1049: Unknown database 'xifenfei' when selecting the database

因为mysql没有crash,因此相关文件已经存在(没有被真正删除)
rm_mysql_ibd


通过此类方法恢复相关数据文件到新服务器上,然后尝试启动数据库,发现无法正常启动,有部分文件丢失,最后对单独ibd单独处理进行恢复,具体参考:[MySQL异常恢复]mysql ibd文件恢复,实现绝大多数数据的恢复,对于部分无法通过此类方法恢复出来的数据,在磁盘级别没有覆盖的情况下,可以先按照os层面方法恢复,参考:extundelete恢复Linux被删除文件,如果此类方法也无法正常恢复,可以尝试数据库磁盘碎片级别恢复:MySQL drop database恢复(恢复方法同样适用MySQL drop table,delete,truncate table)

mysql数据库被加密恢复

联系:手机/微信(+86 17813235971) QQ(107644445)

标题:mysql数据库被加密恢复

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

有客户mysql数据库被加密,加密信息如下
20200129162256


由于是mysql 5.6的版本,默认情况innodb引擎,Innodb_file_per_table参数默认为true,因此数据为每个表存在为一个单独的ibd文件中,让客户提供需要恢复的表的ibd被加密文件
20200129162048


通过一系列底层操作,实现数据完美恢复
1
2


如果是Innodb_file_per_table参数为false(5.6之前版本默认为false),需要通过ibdata文件进行恢复
如果您的数据库(oracle,mysql sql server)不幸被比特币加密,可以联系我们
Tel/微信:17813235971    Q Q:107644445 QQ咨询惜分飞    E-Mail:dba@xifenfei.com提供专业的解密恢复服务.

[MySQL异常恢复]mysql ibd文件恢复

联系:手机/微信(+86 17813235971) QQ(107644445)

标题:[MySQL异常恢复]mysql ibd文件恢复

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

在mysql中由于某种原因保存有ibd文件,但是表已经被删除或者frm文件损坏亦或者ibdata文件损坏/丢失等。本文模拟在这种情况下,通过mysql自身技术即可完成ibd文件恢复.
测试环境mysql版本

mysql> select version();
+-----------+
| version() |
+-----------+
| 5.6.25    |
+-----------+
1 row in set (0.00 sec)

mysql主要参数

mysql> show variables like 'innodb_file_per_table';
+-----------------------+-------+
| Variable_name         | Value |
+-----------------------+-------+
| innodb_file_per_table | ON    |
+-----------------------+-------+
1 row in set (0.00 sec)
mysql> show variables like 'innodb_force_recovery';
+-----------------------+-------+
| Variable_name         | Value |
+-----------------------+-------+
| innodb_force_recovery | 0     |
+-----------------------+-------+
1 row in set (0.00 sec)

innodb_file_per_table这个参数为on才能够实现每个表存储单独的ibd文件.innodb_force_recovery参数默认范围0

测试表情况

mysql> use xifenfei;
Database changed
mysql> show tables;
+-----------------------------+
| Tables_in_xifenfei          |
+-----------------------------+
| user_login                  |
+-----------------------------+
1 rows in set (0.00 sec)
mysql> select count(*) from user_login;
+----------+
| count(*) |
+----------+
|       48 |
+----------+
1 row in set (0.02 sec)
mysql> desc user_login;
+------------+--------------+------+-----+---------+-------+
| Field      | Type         | Null | Key | Default | Extra |
+------------+--------------+------+-----+---------+-------+
| ID         | varchar(255) | NO   | PRI | NULL    |       |
| ACCOUNT    | varchar(255) | YES  |     | NULL    |       |
| LifeCycle  | int(11)      | YES  |     | NULL    |       |
| Name       | varchar(255) | YES  |     | NULL    |       |
| Password   | varchar(255) | YES  |     | NULL    |       |
| Role       | varchar(255) | YES  |     | NULL    |       |
| UTime      | varchar(255) | YES  |     | NULL    |       |
| UserID     | varchar(255) | YES  |     | NULL    |       |
| UserName   | varchar(255) | YES  |     | NULL    |       |
| UserStatus | int(11)      | YES  |     | NULL    |       |
+------------+--------------+------+-----+---------+-------+
10 rows in set (0.05 sec)
mysql> select * from user_login limit 1;
+----------------------------------+---------+-----------+-----------+----------
------------------------+------+---------------------+--------------------------
--------+----------+------------+
| ID                               | ACCOUNT | LifeCycle | Name      | Password
                        | Role | UTime               | UserID
        | UserName | UserStatus |
+----------------------------------+---------+-----------+-----------+----------
------------------------+------+---------------------+--------------------------
--------+----------+------------+
| 010d6c85a76c44cba80d07cbd8590bb2 | hyh     |         0 | 胡元会    | 698d51a19
d8a121ce581499d7b701668 | |6|  | 2016-08-30 06:04:32 | 0fe3bc4dd9654687a4b85065e
d5cfee8 | NULL     |          1 |
+----------------------------------+---------+-----------+-----------+----------
------------------------+------+---------------------+--------------------------
--------+----------+------------+
1 row in set (0.00 sec)
mysql> show create table user_login \G;
*************************** 1. row *************
       Table: user_login
Create Table: CREATE TABLE `user_login` (
  `ID` varchar(255) NOT NULL,
  `ACCOUNT` varchar(255) DEFAULT NULL,
  `LifeCycle` int(11) DEFAULT NULL,
  `Name` varchar(255) DEFAULT NULL,
  `Password` varchar(255) DEFAULT NULL,
  `Role` varchar(255) DEFAULT NULL,
  `UTime` varchar(255) DEFAULT NULL,
  `UserID` varchar(255) DEFAULT NULL,
  `UserName` varchar(255) DEFAULT NULL,
  `UserStatus` int(11) DEFAULT NULL,
  PRIMARY KEY (`ID`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
1 row in set (0.00 sec)
mysql> show variables like 'datadir';
+---------------+-----------------------------------------------+
| Variable_name | Value                                         |
+---------------+-----------------------------------------------+
| datadir       | D:\xifenfei\mysql-5.6.25-winx64\data\ |
+---------------+-----------------------------------------------+
1 row in set (0.00 sec)

备份ibd文件

C:\Users\XIFENFEI>dir D:\xifenfei\mysql-5.6.25-winx64\data\xifenfei\user_login.ibd
 驱动器 D 中的卷没有标签。
 卷的序列号是 4215-1F18
 D:\xifenfei\mysql-5.6.25-winx64\data\xifenfei 的目录
2016-12-02  20:07            98,304 user_login.ibd
               1 个文件         98,304 字节
               0 个目录 78,789,591,040 可用字节
C:\Users\XIFENFEI>cp D:\xifenfei\mysql-5.6.25-winx64\data\xifenfei\user_login.ibd d:/
C:\Users\XIFENFEI>dir d:\user_login.ibd
 驱动器 D 中的卷没有标签。
 卷的序列号是 4215-1F18
 d:\ 的目录
2016-12-25  23:15            98,304 user_login.ibd
               1 个文件         98,304 字节
               0 个目录 78,789,591,040 可用字节

模拟删除表(ibd文件也被删除)

mysql> drop table xifenfei.user_login;
Query OK, 0 rows affected (0.03 sec)
C:\Users\XIFENFEI>dir D:\xifenfei\mysql-5.6.25-winx64\data\xifenfei\user_login.ibd
 驱动器 D 中的卷没有标签。
 卷的序列号是 4215-1F18
 D:\xifenfei\mysql-5.6.25-winx64\data\xifenfei 的目录
找不到文件

创建新表

mysql> CREATE TABLE `user_login` (
    ->   `ID` varchar(255) NOT NULL,
    ->   `ACCOUNT` varchar(255) DEFAULT NULL,
    ->   `LifeCycle` int(11) DEFAULT NULL,
    ->   `Name` varchar(255) DEFAULT NULL,
    ->   `Password` varchar(255) DEFAULT NULL,
    ->   `Role` varchar(255) DEFAULT NULL,
    ->   `UTime` varchar(255) DEFAULT NULL,
    ->   `UserID` varchar(255) DEFAULT NULL,
    ->   `UserName` varchar(255) DEFAULT NULL,
    ->   `UserStatus` int(11) DEFAULT NULL,
    ->   PRIMARY KEY (`ID`)
    -> ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Query OK, 0 rows affected (0.03 sec)
C:\Users\XIFENFEI>dir D:\xifenfei\mysql-5.6.25-winx64\data\xifenfei\user_login.ibd
 驱动器 D 中的卷没有标签。
 卷的序列号是 4215-1F18
 D:\xifenfei\mysql-5.6.25-winx64\data\xifenfei 的目录
2016-12-25  23:19            98,304 user_login.ibd
               1 个文件         98,304 字节
               0 个目录 78,789,591,040 可用字节
mysql> select count(*) from xifenfei.user_login;
+----------+
| count(*) |
+----------+
|        0 |
+----------+
1 row in set (0.00 sec)

停掉mysql,替换user_login.ibd

C:\Users\XIFENFEI>dir D:\xifenfei\mysql-5.6.25-winx64\data\xifenfei\user_login.ibd
 驱动器 D 中的卷没有标签。
 卷的序列号是 4215-1F18
 D:\xifenfei\mysql-5.6.25-winx64\data\xifenfei 的目录
2016-12-25  23:22            98,304 user_login.ibd
               1 个文件         98,304 字节
               0 个目录 78,787,141,632 可用字节
C:\Users\XIFENFEI>cp d:\user_login.ibd D:\xifenfei\mysql-5.6.25-winx64\data\xifenfei\user_login.ibd
C:\Users\XIFENFEI>dir D:\xifenfei\mysql-5.6.25-winx64\data\xifenfei\user_login.ibd
 驱动器 D 中的卷没有标签。
 卷的序列号是 4215-1F18
 D:\xifenfei\mysql-5.6.25-winx64\data\xifenfei 的目录
2016-12-02  20:07            98,304 user_login.ibd
               1 个文件         98,304 字节
               0 个目录 78,787,141,632 可用字节

启动mysql 服务,查询数据库

mysql> select count(*) from xifenfei.user_login;
ERROR 2013 (HY000): Lost connection to MySQL server during query
mysql> exit
Bye
C:\Users\XIFENFEI>mysql -uroot
ERROR 2003 (HY000): Can't connect to MySQL server on 'localhost' (10061)

mysql 日志报错

2016-12-25 23:31:07 11632 [Note] MySQL: ready for connections.
Version: '5.6.25'  socket: ''  port: 3306  MySQL Community Server (GPL)
InnoDB: Error: tablespace id is 56 in the data dictionary
InnoDB: but in file .\xifenfei\user_login.ibd it is 47!
2016-12-25 23:31:31 2eb8  InnoDB: Assertion failure in thread 11960 in file fil0fil.cc line 796
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be

很明显由于替换的ibd文件和现在数据库记录的ibd文件的page的字典信息不匹配,因为数据库无法正常查询该数据,而且mysql为了安全直接把实例给crash了.

恢复操作

mysql> show variables like 'innodb_force_recovery';
+-----------------------+-------+
| Variable_name         | Value |
+-----------------------+-------+
| innodb_force_recovery | 1     |
+-----------------------+-------+
1 row in set (0.00 sec)
mysql> alter table xifenfei.user_login discard tablespace;
Query OK, 0 rows affected, 2 warnings (0.02 sec)
mysql> alter table xifenfei.user_login import tablespace;
Query OK, 0 rows affected, 1 warning (0.06 sec)
mysql> select count(*) from xifenfei.user_login;
+----------+
| count(*) |
+----------+
|       48 |
+----------+
1 row in set (0.00 sec)
mysql> select * from xifenfei.user_login limit 1;
+----------------------------------+---------+-----------+-----------+----------
------------------------+------+---------------------+--------------------------
--------+----------+------------+
| ID                               | ACCOUNT | LifeCycle | Name      | Password
                        | Role | UTime               | UserID
        | UserName | UserStatus |
+----------------------------------+---------+-----------+-----------+----------
------------------------+------+---------------------+--------------------------
--------+----------+------------+
| 010d6c85a76c44cba80d07cbd8590bb2 | hyh     |         0 | 胡元会    | 698d51a19
d8a121ce581499d7b701668 | |6|  | 2016-08-30 06:04:32 | 0fe3bc4dd9654687a4b85065e
d5cfee8 | NULL     |          1 |
+----------------------------------+---------+-----------+-----------+----------
------------------------+------+---------------------+--------------------------
--------+----------+------------+
1 row in set (0.00 sec)

通过mysql自带的discard tablespace和import tablespace操作后,表数据已经可以完成查询了.
mysql日志

2016-12-25 23:34:08 10464 [ERROR] InnoDB: Failed to find tablespace for table '"xifenfei"."user_login"' in the cache. Attempting to load the tablespace with space id 56.
2016-12-25 23:34:08 10464 [ERROR] InnoDB: In file '.\xifenfei\user_login.ibd', tablespace id and flags are 47 and 0, but in the InnoDB data dictionary they are 56 and 0. Have you moved InnoDB .ibd files around without using the commands DISCARD TABLESPACE and IMPORT TABLESPACE? Please refer to http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting-datadict.html for how to resolve the issue.
2016-12-25 23:34:08 10464 [ERROR] InnoDB: Could not find a valid tablespace file for 'xifenfei/user_login'. See http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting-datadict.html for how to resolve the issue.
2016-12-25 23:34:08 30e8 InnoDB: cannot calculate statistics for table "xifenfei"."user_login" because the .ibd file is missing. For help, please refer to http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting.html
2016-12-25 23:34:08 10464 [ERROR] InnoDB: Cannot delete tablespace 56 because it is not found in the tablespace memory cache.
2016-12-25 23:34:08 10464 [Warning] InnoDB: Cannot delete tablespace 56 in DISCARD TABLESPACE. Tablespace not found
2016-12-25 23:34:41 10464 [Note] InnoDB: Sync to disk
2016-12-25 23:34:41 10464 [Note] InnoDB: Sync to disk - done!
2016-12-25 23:34:41 10464 [Note] InnoDB: Phase I - Update all pages
2016-12-25 23:34:41 10464 [Note] InnoDB: Sync to disk
2016-12-25 23:34:41 10464 [Note] InnoDB: Sync to disk - done!
2016-12-25 23:34:41 10464 [Warning] InnoDB: Tablespace 'xifenfei/user_login' exists in the cache with id 47 != 56
2016-12-25 23:34:41 10464 [Warning] InnoDB: Freeing existing tablespace 'xifenfei/user_login' entry from the cache with id 56
2016-12-25 23:34:41 10464 [Note] InnoDB: Phase III - Flush changes to disk
2016-12-25 23:34:41 10464 [Note] InnoDB: Phase IV - Flush complete

mysql日志依旧报了page字典信息不匹配.但是数据已经可以访问,通过mysqldump导出重新创建表即可.如果由于ibd损坏使用该方法无法恢复,请参考:MySQL drop database恢复(恢复方法同样适用MySQL drop table,delete,truncate table)