2023年2月 – Database SOS

win强制修改盘符导致oracle异常恢复

有客户反馈,他们在系统没有关闭数据库的情况下,强制修改了win系统盘符,然后导致数据库异常,启动报错

Sat Feb 25 12:50:40 2023
Recovery of Online Redo Log: Thread 1 Group 2 Seq 10440 Reading mem 0
  Mem# 0 errs 0: G:\ORACLE\PRODUCT\10.2.0\ORADATA\ORCL\REDO02.LOG
Sat Feb 25 12:50:40 2023
Completed redo application
Sat Feb 25 12:50:40 2023
Errors in file g:\oracle\product\10.2.0\admin\orcl\bdump\orcl_p001_5604.trc:
ORA-00600: 内部错误代码, 参数: [2037], [25801018], [2973409798], [6], [255], [25], [1198764346], [100796692]

Sat Feb 25 12:50:40 2023
Errors in file g:\oracle\product\10.2.0\admin\orcl\udump\orcl_ora_7648.trc:
ORA-07445: 出现异常错误: 核心转储 [ACCESS_VIOLATION][_kslwlmod+166][PC:0x469742][ADDR:0x54F8][UNABLE_TO_WRITE][]
ORA-04096: 触发器 '' 的 WHEN 子句过大, 限量为 2K

Sat Feb 25 12:50:40 2023
Errors in file g:\oracle\product\10.2.0\admin\orcl\udump\orcl_ora_7252.trc:
ORA-00600: internal error code, arguments: [ksuapc2], [258], [0], [2], [1], [2], [], []

Sat Feb 25 12:50:43 2023
Errors in file g:\oracle\product\10.2.0\admin\orcl\bdump\orcl_p001_5604.trc:
ORA-00081: 地址范围 [0x77240440, 0x77240444) 不可读
ORA-07445: 出现异常错误: 核心转储 [ACCESS_VIOLATION][_ksl_cleanup+723][PC:0x46E373][ADDR:0x1C][UNABLE_TO_READ][]
ORA-00081: 地址范围 [0x77240440, 0x77240444) 不可读
ORA-00600: 内部错误代码, 参数: [2037], [25801018], [2973409798], [6], [255], [25], [1198764346], [100796692]

Sat Feb 25 12:50:45 2023
Errors in file g:\oracle\product\10.2.0\admin\orcl\bdump\orcl_dbw0_6332.trc:
ORA-07445: ??????: ???? [ACCESS_VIOLATION][_kews_idle_wait+378][PC:0x604AE6][ADDR:0xED30C470][UNABLE_TO_WRITE][]

Sat Feb 25 12:50:48 2023
Errors in file g:\oracle\product\10.2.0\admin\orcl\udump\orcl_ora_7648.trc:
ORA-00600: 内部错误代码, 参数: [kslwlflux:1], [0xAB805400], [0x549C], [2], [], [], [], []
ORA-00081: 地址范围 [0x74480443, 0x74480447) 不可读
ORA-00600: 内部错误代码, 参数: [kslwlflux:1], [0xAB805400], [0x549C], [2], [], [], [], []
ORA-00081: 地址范围 [0x74480443, 0x74480447) 不可读
ORA-00600: 内部错误代码, 参数: [kslwlflux:1], [0xAB805400], [0x549C], [2], [], [], [], []
ORA-00081: 地址范围 [0x74480443, 0x74480447) 不可读
ORA-00600: 内部错误代码, 参数: [kslwlflux:1], [0xAB805400], [0x549C], [2], [], [], [], []
ORA-00081: 地址范围 [0x74480443, 0x74480447) 不可读
ORA-00600: 内部错误代码, 参数: [kslwlflux:1], [0xAB805400], [0x549C], [2], [], [], [], []
ORA-00081: 地址范围 [0x74480443, 0x74480447) 不可读
ORA-00600: 内部错误代码, 参数: [kslwlflux:1], [0xAB805400], [0x549C], [2], [], [], [], []
ORA-00081: 地址范围 [0x74480443, 0x74480447) 不可读
ORA-00600: 内部错误代码, 参数: [kslwlflux:1], [0xAB805400], [0x549C], [2], [], [], [], []
ORA-00081: 地址范围 [0x74480443, 0x74480447) 不可读
ORA-00600: 内部错误代码, 参

Sat Feb 25 12:51:34 2023
Errors in file g:\oracle\product\10.2.0\admin\orcl\bdump\orcl_d000_8116.trc:
ORA-07445: ??????: ???? [ACCESS_VIOLATION][_kmcgms+121][PC:0x5D6C71][ADDR:0x50][UNABLE_TO_WRITE][]

Sat Feb 25 12:52:04 2023
USER: terminating instance due to error 472
Sat Feb 25 12:52:48 2023
USER: terminating instance due to error 472
Sat Feb 25 12:52:48 2023
Errors in file g:\oracle\product\10.2.0\admin\orcl\udump\orcl_ora_6252.trc:
ORA-07445: exception encountered: core dump [ACCESS_VIOLATION][_ksuitm+631][PC:0x410C07][ADDR:0x1][UNABLE_TO_READ][]

Sat Feb 25 12:55:35 2023
Errors in file g:\oracle\product\10.2.0\admin\orcl\bdump\orcl_smon_7656.trc:
ORA-07445: ??????: ???? [ACCESS_VIOLATION][_kews_idle_wait+378][PC:0x604AE6][ADDR:0xE530C470][UNABLE_TO_WRITE][]

通过恢复一些恢复之后,数据库open之后又挂掉

Sat Feb 25 15:05:49 2023
SMON: enabling tx recovery
Sat Feb 25 15:05:49 2023
Database Characterset is ZHS16GBK
Sat Feb 25 15:05:50 2023
Errors in file g:\oracle\product\10.2.0\admin\orcl\udump\orcl_ora_5308.trc:
ORA-00600: 内部错误代码, 参数: [4194], [34], [31], [], [], [], [], []

Sat Feb 25 15:05:50 2023
Errors in file g:\oracle\product\10.2.0\admin\orcl\bdump\orcl_smon_7568.trc:
ORA-00600: 内部错误代码, 参数: [kcbgtcr_13], [], [], [], [], [], [], []

Sat Feb 25 15:05:51 2023
Non-fatal internal error happenned while SMON was doing logging scn->time mapping.
SMON encountered 1 out of maximum 100 non-fatal internal errors.
Sat Feb 25 15:05:51 2023
Doing block recovery for file 2 block 2951
Block recovery from logseq 10441, block 78 to scn 109906860017
Sat Feb 25 15:05:51 2023
Recovery of Online Redo Log: Thread 1 Group 3 Seq 10441 Reading mem 0
  Mem# 0 errs 0: G:\ORACLE\PRODUCT\10.2.0\ORADATA\ORCL\REDO03.LOG
Block recovery stopped at EOT rba 10441.81.16
Block recovery completed at rba 10441.81.16, scn 25.2532677517
Doing block recovery for file 2 block 113
Block recovery from logseq 10441, block 78 to scn 109906859718
Sat Feb 25 15:05:52 2023
Recovery of Online Redo Log: Thread 1 Group 3 Seq 10441 Reading mem 0
  Mem# 0 errs 0: G:\ORACLE\PRODUCT\10.2.0\ORADATA\ORCL\REDO03.LOG
Block recovery completed at rba 10441.80.16, scn 25.2532677516
replication_dependency_tracking turned off (no async multimaster replication found)
Starting background process QMNC
QMNC started with pid=30, OS id=6904
Sat Feb 25 15:05:53 2023
db_recovery_file_dest_size of 2048 MB is 0.00% used. This is a
user-specified limit on the amount of space that will be used by this
database for recovery-related files, and does not reflect the amount of
space available in the underlying filesystem or ASM diskgroup.
Sat Feb 25 15:05:53 2023
Completed: alter database open
Sat Feb 25 15:05:53 2023
Errors in file g:\oracle\product\10.2.0\admin\orcl\bdump\orcl_j000_5400.trc:
ORA-00600: 内部错误代码, 参数: [4194], [6], [4], [], [], [], [], []

Sat Feb 25 15:05:54 2023
DEBUG: Replaying xcb 0xac458698, pmd 0x8d7e9b9c for failed op 8
Doing block recovery for file 2 block 1515
No block recovery was needed
Sat Feb 25 15:05:55 2023
Errors in file g:\oracle\product\10.2.0\admin\orcl\bdump\orcl_j000_5400.trc:
ORA-00600: 内部错误代码, 参数: [4194], [6], [4], [], [], [], [], []
ORA-00600: 内部错误代码, 参数: [4194], [6], [4], [], [], [], [], []

Sat Feb 25 15:05:56 2023
Errors in file g:\oracle\product\10.2.0\admin\orcl\bdump\orcl_j000_5400.trc:
ORA-00603: ORACLE server session terminated by fatal error
ORA-00600: internal error code, arguments: [4194], [6], [4], [], [], [], [], []
ORA-00600: internal error code, arguments: [4194], [6], [4], [], [], [], [], []

Sat Feb 25 15:05:57 2023
Doing block recovery for file 2 block 2951
Block recovery from logseq 10441, block 78 to scn 109906860017
Sat Feb 25 15:05:57 2023
Recovery of Online Redo Log: Thread 1 Group 3 Seq 10441 Reading mem 0
  Mem# 0 errs 0: G:\ORACLE\PRODUCT\10.2.0\ORADATA\ORCL\REDO03.LOG
Block recovery completed at rba 10441.81.16, scn 25.2532677620
Doing block recovery for file 2 block 113
Block recovery from logseq 10441, block 78 to scn 109906860083
Sat Feb 25 15:05:57 2023
Recovery of Online Redo Log: Thread 1 Group 3 Seq 10441 Reading mem 0
  Mem# 0 errs 0: G:\ORACLE\PRODUCT\10.2.0\ORADATA\ORCL\REDO03.LOG
Block recovery completed at rba 10441.138.16, scn 25.2532677684
Sat Feb 25 15:05:57 2023
Errors in file g:\oracle\product\10.2.0\admin\orcl\bdump\orcl_j003_1716.trc:
ORA-12012: 自动执行作业 42568 出错
ORA-00607: 当更改数据块时出现内部错误
ORA-00607: 当更改数据块时出现内部错误

Sat Feb 25 15:05:59 2023
Flush retried for xcb 0xac4c5a80, pmd 0x8c0cec74
Doing block recovery for file 2 block 2951
Block recovery from logseq 10441, block 78 to scn 109906860017
Sat Feb 25 15:05:59 2023
Recovery of Online Redo Log: Thread 1 Group 3 Seq 10441 Reading mem 0
  Mem# 0 errs 0: G:\ORACLE\PRODUCT\10.2.0\ORADATA\ORCL\REDO03.LOG
Sat Feb 25 15:05:59 2023
DEBUG: Replaying xcb 0xac458698, pmd 0x8d7e9b9c for failed op 8
Doing block recovery for file 2 block 1515
No block recovery was needed
Sat Feb 25 15:05:59 2023
Block recovery completed at rba 10441.81.16, scn 25.2532677620
Sat Feb 25 15:06:00 2023
Errors in file g:\oracle\product\10.2.0\admin\orcl\bdump\orcl_j002_6400.trc:
ORA-00600: internal error code, arguments: [4194], [6], [4], [], [], [], [], []

Sat Feb 25 15:06:02 2023
DEBUG: Replaying xcb 0xac458698, pmd 0x8d7e9b9c for failed op 8
Doing block recovery for file 2 block 1515
No block recovery was needed
Sat Feb 25 15:06:02 2023
Errors in file g:\oracle\product\10.2.0\admin\orcl\bdump\orcl_pmon_1076.trc:
ORA-00600: 内部错误代码, 参数: [4194], [6], [4], [], [], [], [], []

Sat Feb 25 15:06:03 2023
PMON: terminating instance due to error 472
Sat Feb 25 15:06:03 2023
Errors in file g:\oracle\product\10.2.0\admin\orcl\bdump\orcl_j007_7188.trc:
ORA-00472: PMON 进程因错误而终止

Sat Feb 25 15:06:03 2023
Errors in file g:\oracle\product\10.2.0\admin\orcl\bdump\orcl_j006_7624.trc:
ORA-00472: PMON 进程因错误而终止

Sat Feb 25 15:06:03 2023
Errors in file g:\oracle\product\10.2.0\admin\orcl\bdump\orcl_j005_5688.trc:
ORA-00472: PMON  process terminated with error

Sat Feb 25 15:06:10 2023
Errors in file g:\oracle\product\10.2.0\admin\orcl\bdump\orcl_smon_7568.trc:
ORA-00472: PMON 进程因错误而终止

Instance terminated by PMON, pid = 1076

这个ORA-600 4194错误主要是由于undo异常,从而引起pmon异常,报ORA-00472错误.对undo进行处理,数据库稳定open,逻辑导出数据,完成恢复工作,完美帮忙客户恢复数据.

再一例asm disk被误加入vg并且扩容lv恢复

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：再一例asm disk被误加入vg并且扩容lv恢复

又一客户把三块asm disk磁盘加入到vg里面(两个节点的root vg中),并且还进行了扩容

加入之后,还对该lv里面进行了expdp数据导出(导出一半失败了,数据库挂了),进而引起了大量的asm磁盘中数据块被文件系统中的expdp导出的dmp复写

通过对损坏磁盘进行kfed分析大概判断损坏到90GB位置

对于这种asm磁盘损坏比较多的情况,常规kfed无法进行修复,只能采用底层基于block层面的扫描恢复,参考:
asm disk header 彻底损坏恢复
通过底层处理恢复出来数据文件:

然后通过可以有的2天之前的备份,结合dul工具,恢复出来最近数据,业务层面进行整合,完成本次数据恢复
有过类似的恢复案例:
asm disk被加入vg恢复
又一例asm disk 加入vg故障

重建control遗漏数据文件,reseltogs报ORA-1555错误处理

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：重建control遗漏数据文件,reseltogs报ORA-1555错误处理

又一客户,误删除oracle redo导致数据库无法正常启动,自己尝试重建ctl,结果遗漏部分oracle数据文件并且尝试过resetlogs,导致部分文件resetlogs scn不一致.导致重建ctl失败

Fri Feb 10 12:41:20 2023
CREATE CONTROLFILE REUSE DATABASE "orcl" RESETLOGS  NOARCHIVELOG
    MAXLOGFILES 50
    MAXLOGMEMBERS 5
    MAXDATAFILES 100
    MAXINSTANCES 8
    MAXLOGHISTORY 226
LOGFILE
  GROUP 1 'H:\BaiduNetdisk\couga/redo01.log'  SIZE 50M,
  GROUP 2 'H:\BaiduNetdisk\couga/redo02.log'  SIZE 50M,
  GROUP 3 'H:\BaiduNetdisk\couga/redo03.log'  SIZE 50M
DATAFILE
'H:\BaiduNetdisk\couga\system01.dbf',
'H:\BaiduNetdisk\couga\cougaerp.DBF',
'H:\BaiduNetdisk\couga\cougajh.DBF',
'H:\BaiduNetdisk\couga\example01.dbf',
'H:\BaiduNetdisk\couga\sysaux01.dbf',
'H:\BaiduNetdisk\couga\undotbs01.dbf',
'H:\BaiduNetdisk\couga\users01.dbf'
CHARACTER SET zhs16gbk
WARNING: Default Temporary Tablespace not specified in CREATE DATABASE command
Default Temporary Tablespace will be necessary for a locally managed database in future release
Errors in file c:\app\xff\diag\rdbms\orcl\orcl11\trace\orcl11_ora_4132.trc:
ORA-01189: ????????????? RESETLOGS
ORA-01110: ???? 6: 'H:\BaiduNetdisk\couga\cougajh.DBF'

通过OraRecovery工具修改相关异常文件头resetlogs scn之后,重建ctl成功

尝试resetlogs 数据库报ORA-00704 ORA-00604 ORA-01555错误

Fri Feb 10 12:46:04 2023
SMON: enabling cache recovery
ORA-01555 caused by SQL statement below (SQL ID: 4krwuz0ctqxdt, SCN: 0x0000.09dab82d):
select ctime, mtime, stime from obj$ where obj# = :1
Errors in file c:\app\xff\diag\rdbms\orcl\orcl11\trace\orcl11_ora_7088.trc:
ORA-00704: 引导程序进程失败
ORA-00704: 引导程序进程失败
ORA-00604: 递归 SQL 级别 1 出现错误
ORA-01555: 快照过旧: 回退段号 5 (名称为 "_SYSSMU5_1527469038$") 过小
Errors in file c:\app\xff\diag\rdbms\orcl\orcl11\trace\orcl11_ora_7088.trc:
ORA-00704: 引导程序进程失败
ORA-00704: 引导程序进程失败
ORA-00604: 递归 SQL 级别 1 出现错误
ORA-01555: 快照过旧: 回退段号 5 (名称为 "_SYSSMU5_1527469038$") 过小
Error 704 happened during db open, shutting down database
USER (ospid: 7088): terminating the instance due to error 704

通过类似方法处理:
在数据库open过程中常遇到ORA-01555汇总
 数据库open过程遭遇ORA-1555对应sql语句补充
顺利open数据库，并且逻辑方式导出数据,完成恢复

InnoDB: Database page corruption on disk or a failed file read of page恢复

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：InnoDB: Database page corruption on disk or a failed file read of page恢复

由于文件系统或者硬件故障导致mysql启动报错InnoDB: Database page corruption on disk or a failed file read of page

使用innodb_force_recovery参数,数据库启动成功,但是部分表查询报错,对于这种情况,是由于ibd文件本身有损坏,通过DISCARD TABLESPACE和IMPORT TABLESPACE也无法解决,只能对ibd文件进行恢复，通过工具直接解析ibd文件恢复其中数据

_locked加密数据库恢复

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：_locked加密数据库恢复

最近比较多的客户被._locked勒索病毒加密

这种加密和以往的常见加密不太一样,它把文件名都给修改了,无法类似以前那样通过文件名判断出来对应的用途(比如哪些是oracle数据文件,哪些是sql server文件等),所有文件被加密成随机名字._locked,通过对其底层进行分析,可以确认该文件损坏很少,oracle和sql server基本上可以实现完美恢复(特别是11G及其以后版本,恢复效果和数据库直接运行状态下expdp/exp导出效果一样)

通过oracle数据文件加密勒索工具即可实现快速恢复oracle数据文件实现数据库open,然后导出数据

预防远比救援重要，所以为了避免出现此类事件，强烈建议大家日常做好以下防护措施：
①及时给办公终端和服务器打补丁，修复漏洞，包括操作系统以及第三方应用的补丁，防止攻击者通过漏洞入侵系统。
②尽量关闭不必要的端口，如139、445、3389等端口。如果不使用，可直接关闭高危端口，降低被漏洞攻击的风险。
③不对外提供服务的设备不要暴露于公网之上，对外提供服务的系统，应保持较低权限。
④企业用户应采用高强度且无规律的密码来登录办公系统或服务器，要求包括数字、大小写字母、符号，且长度至少为8位的密码，并定期更换口令。
⑤数据备份保护，对关键数据和业务系统做备份，如离线备份，异地备份，云备份等，避免因为数据丢失、被加密等造成业务停摆，甚至被迫向攻击者妥协。
⑥敏感数据隔离，对敏感业务及其相关数据做好网络隔离。避免双重勒索病毒在入侵后轻易窃取到敏感数据，对公司业务和机密信息造成重大威胁。
⑦尽量关闭不必要的文件共享。
⑧提高安全运维人员职业素养，定期进行木马病毒查杀。

部分oracle数据文件被加密完美恢复

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：部分oracle数据文件被加密完美恢复

客户oracle数据文件部分被加密

!!! ALL YOUR FILES ARE ENCRYPTED !!!.TXT中内容为:

!!! ALL YOUR FILES ARE ENCRYPTED !!!

All your files, documents, photos, databases and other important files are encrypted.

You are not able to decrypt it by yourself! The only method of recovering files is to purchase an unique private key.
Only we can give you this key and only we can recover your files.

To be sure we have the decryptor and it works you can send an email: leonardoboss@onionmail.org and decrypt one file for free.
But this file should be of not valuable!

Do you really want to restore your files?
Write to email: leonardoboss@onionmail.org
Reserved email: sunhuyvchay@messagesafe.io

Your personal ID: 205-37B-A6A

Attention!
 * Do not rename encrypted files.
 * Do not try to decrypt your data using third party software, it may cause permanent data loss.
 * Decryption of your files with the help of third parties may cause increased price (they add their fee to our) or you can become a victim of a scam.

通过坏块工具进行分析确认(每个被破坏的文件损坏block 46个)

对于这种情况,通过开发的oracle数据文件勒索加密恢复工具,可以快速open库并且导出数据

对于类似这种被加密的勒索的数据文件,我们可以实现比较好的恢复效果,如果此类的数据库(oracle,mysql,sql server)等被加密,需要专业恢复技术支持，请联系我们:
电话/微信:17813235971 Q Q:107644445 E-Mail:dba@xifenfei.com
系统安全防护措施建议：
1.多台机器，不要使用相同的账号和口令
2.登录口令要有足够的长度和复杂性，并定期更换登录口令
3.重要资料的共享文件夹应设置访问权限控制，并进行定期备份
4.定期检测系统和软件中的安全漏洞，及时打上补丁。
5.定期到服务器检查是否存在异常。
6.安装安全防护软件，并确保其正常运行。
7.从正规渠道下载安装软件。
8.对不熟悉的软件，如果已经被杀毒软件拦截查杀，不要添加信任继续运行。
9.保存良好的备份习惯，尽量做到每日备份，异地备份。

月度归档：2月 2023