Oracle备份恢复 – 第4页

ORA-600 3020/ORA-600 2662故障

数据库重启之后报ORA-600 3020错误

Tue Dec 12 00:37:44 2023
ALTER DATABASE RECOVER    CONTINUE DEFAULT  
Media Recovery Log /data/oracle/flash_recovery_area/ORCL/archivelog/2023_11_30/o1_mf_1_3867_lpj3krsn_.arc
ORA-279 signalled during: ALTER DATABASE RECOVER    CONTINUE DEFAULT  ...
ALTER DATABASE RECOVER    CONTINUE DEFAULT  
Media Recovery Log /data/oracle/flash_recovery_area/ORCL/archivelog/2023_12_02/o1_mf_1_3868_lppkmyhf_.arc
Tue Dec 12 00:37:45 2023
Errors in file /data/oracle/diag/rdbms/orcl/orcl/trace/orcl_pr01_57992.trc  (incident=31482):
ORA-00600: internal error code, arguments: [3020], [1], [87806], [4282110], [], [], [], [], [], [], [], []
ORA-10567: Redo is inconsistent with data block (file# 1, block# 87806, file offset is 719306752 bytes)
ORA-10564: tablespace SYSTEM
ORA-01110: data file 1: '/data/oracle/oradata/orcl/system01.dbf'
ORA-10561: block type 'TRANSACTION MANAGED INDEX BLOCK', data object# 5843
Incident details in: /data/oracle/diag/rdbms/orcl/orcl/incident/incdir_31482/orcl_pr01_57992_i31482.trc
Tue Dec 12 00:37:45 2023
Errors in file /data/oracle/diag/rdbms/orcl/orcl/trace/orcl_pr08_58007.trc  (incident=31538):
ORA-00600: internal error code, arguments: [3020], [1], [61031], [4255335], [], [], [], [], []
ORA-10567:Redo is inconsistent with data block (file# 1, block# 61031, file offset is 499965952 bytes)
ORA-10564: tablespace SYSTEM
ORA-01110: data file 1: '/data/oracle/oradata/orcl/system01.dbf'
ORA-10561: block type 'TRANSACTION MANAGED INDEX BLOCK', data object# 39
Incident details in: /data/oracle/diag/rdbms/orcl/orcl/incident/incdir_31538/orcl_pr08_58007_i31538.trc

客户不太了解上述故障，直接使用隐含参数，强制拉库报ORA-600 2662错误

Tue Dec 12 00:45:11 2023
SMON: enabling cache recovery
Errors in file /data/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_58731.trc  (incident=32668):
ORA-00600: internal error code, arguments: [2662], [0], [120784398], [0], [121441601], [12583040]
Incident details in: /data/oracle/diag/rdbms/orcl/orcl/incident/incdir_32668/orcl_ora_58731_i32668.trc
Errors in file /data/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_58731.trc:
ORA-00600: internal error code, arguments: [2662], [0], [120784398], [0], [121441601], [12583040]
Errors in file /data/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_58731.trc:
ORA-00600: internal error code, arguments: [2662], [0], [120784398], [0], [121441601], [12583040]
Error 600 happened during db open, shutting down database
USER (ospid: 58731): terminating the instance due to error 600
Instance terminated by USER, pid = 58731

接手该故障之后,数据库已经被resetlogs,只能基于该现场进一步恢复,修改数据库scn,open数据库成功,导出数据,完成本次恢复.

Patch SCN工具一键恢复ORA-600 kcbzib_kcrsds_1

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：Patch SCN工具一键恢复ORA-600 kcbzib_kcrsds_1

一个19c库由于某种原因redo损坏强制打开库报ORA-600 kcbzib_kcrsds_1错误

SQL> startup mount pfile='?/database/pfile.txt';
ORACLE instance started.

Total System Global Area  859830696 bytes
Fixed Size                  9034152 bytes
Variable Size             566231040 bytes
Database Buffers          276824064 bytes
Redo Buffers                7741440 bytes
Database mounted.
SQL>
SQL>
SQL> alter database open resetlogs ;
alter database open resetlogs
*
ERROR at line 1:
ORA-00603: ORACLE server session terminated by fatal error
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-00600: internal error code, arguments: [kcbzib_kcrsds_1], [], [], [], [],
[], [], [], [], [], [], []
Process ID: 7236
Session ID: 252 Serial number: 14807

alert日志报错

2023-12-07T21:27:40.176923+08:00
alter database open resetlogs upgrade
2023-12-07T21:27:40.223341+08:00
RESETLOGS is being done without consistancy checks. This may result
in a corrupted database. The database should be recreated.
RESETLOGS after incomplete recovery UNTIL CHANGE 70591877 time 
.... (PID:7236): Clearing online redo logfile 1 C:\USERS\ADMINISTRATOR\DESKTOP\OCRL\REDO01.LOG
.... (PID:7236): Clearing online redo logfile 2 C:\USERS\ADMINISTRATOR\DESKTOP\OCRL\REDO02.LOG
.... (PID:7236): Clearing online redo logfile 3 C:\USERS\ADMINISTRATOR\DESKTOP\OCRL\REDO03.LOG
Clearing online log 1 of thread 1 sequence number 0
Clearing online log 2 of thread 1 sequence number 0
Clearing online log 3 of thread 1 sequence number 0
.... (PID:7236): Clearing online redo logfile 1 complete
.... (PID:7236): Clearing online redo logfile 2 complete
.... (PID:7236): Clearing online redo logfile 3 complete
Resetting resetlogs activation ID 1658522235 (0x62db0a7b)
Online log C:\USERS\ADMINISTRATOR\DESKTOP\OCRL\REDO01.LOG: Thread 1 Group 1 was previously cleared
Online log C:\USERS\ADMINISTRATOR\DESKTOP\OCRL\REDO02.LOG: Thread 1 Group 2 was previously cleared
Online log C:\USERS\ADMINISTRATOR\DESKTOP\OCRL\REDO03.LOG: Thread 1 Group 3 was previously cleared
2023-12-07T21:27:40.896383+08:00
Setting recovery target incarnation to 3
2023-12-07T21:27:40.927396+08:00
Ping without log force is disabled:
  instance mounted in exclusive mode.
Endian type of dictionary set to little
2023-12-07T21:27:40.943147+08:00
Assigning activation ID 1682724028 (0x644c54bc)
2023-12-07T21:27:40.943147+08:00
TT00 (PID:7180): Gap Manager starting
2023-12-07T21:27:40.958429+08:00
Redo log for group 1, sequence 1 is not located on DAX storage
Thread 1 opened at log sequence 1
  Current log# 1 seq# 1 mem# 0: C:\USERS\ADMINISTRATOR\DESKTOP\OCRL\REDO01.LOG
Successful open of redo thread 1
2023-12-07T21:27:40.958429+08:00
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
stopping change tracking
2023-12-07T21:27:40.990460+08:00
TT03 (PID:1292): Sleep 5 seconds and then try to clear SRLs in 2 time(s)
Errors in file C:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\trace\orcl_ora_7236.trc  (incident=12218):
ORA-00600: internal error code, arguments: [kcbzib_kcrsds_1], [], [], [], [], [], [], [], [], [], [], []
Incident details in: C:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\incident\incdir_12218\orcl_ora_7236_i12218.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
2023-12-07T21:27:41.021109+08:00
2023-12-07T21:27:41.848773+08:00
*****************************************************************
An internal routine has requested a dump of selected redo.
This usually happens following a specific internal error, when
analysis of the redo logs will help Oracle Support with the
diagnosis.
It is recommended that you retain all the redo logs generated (by
all the instances) during the past 12 hours, in case additional
redo dumps are required to help with the diagnosis.
*****************************************************************
2023-12-07T21:27:42.082978+08:00
Undo initialization recovery: err:600 start: 3673703 end: 3674796 diff: 1093 ms (1.1 seconds)
2023-12-07T21:27:42.082978+08:00
Errors in file C:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\trace\orcl_ora_7236.trc:
ORA-00600: internal error code, arguments: [kcbzib_kcrsds_1], [], [], [], [], [], [], [], [], [], [], []
2023-12-07T21:27:42.082978+08:00
Errors in file C:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\trace\orcl_ora_7236.trc:
ORA-00600: internal error code, arguments: [kcbzib_kcrsds_1], [], [], [], [], [], [], [], [], [], [], []
Error 600 happened during db open, shutting down database
Errors in file C:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\trace\orcl_ora_7236.trc  (incident=12219):
ORA-00603: ORACLE server session terminated by fatal error
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-00600: internal error code, arguments: [kcbzib_kcrsds_1], [], [], [], [], [], [], [], [], [], [], []
Incident details in: C:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\incident\incdir_12219\orcl_ora_7236_i12219.trc
2023-12-07T21:27:42.881204+08:00
opiodr aborting process unknown ospid (7236) as a result of ORA-603
2023-12-07T21:27:42.881204+08:00
ORA-603 : opitsk aborting process
License high water mark = 4
USER (ospid: (prelim)): terminating the instance due to ORA error 
2023-12-07T21:27:46.052894+08:00
Instance terminated by USER(prelim), pid = 7236

通过patch scn工具快速修改数据库scn

实现数据库顺利打开

DBV-00107: Unknown header format 故障处理

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：DBV-00107: Unknown header format 故障处理

客户linux平台被勒索病毒加密,其中有oracle数据库.客户联系黑客进行解密【勒索解密oracle失败】,但是数据库无法正常启动,dbv检查数据库文件报错

[oracle@hisdb ~]$ dbv file=system01.dbf

DBVERIFY: Release 11.2.0.1.0 - Production on 星期一 11月 27 21:49:17 2023

Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.


DBV-00107: 未知标头格式 (31) (287942924)

对应的英文为:DBV-00107: Unknown header format (31) (287942924)，检查数据文件信息发现提示为 FILE NOT FOUND,使用脚本为:Oracle数据库异常恢复检查脚本(Oracle Database Recovery Check)检测结果

通过分区确认是文件头损坏

修复正确的文件头

再次dbv检查数据文件

[oracle@hisdb ~]$ dbv file=system01.dbf

DBVERIFY: Release 11.2.0.1.0 - Production on 星期一 11月 27 22:05:41 2023

Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.

DBVERIFY - 开始验证: FILE = /u01/app/oracle/oradata/system01.dbf
页 12800 标记为损坏
Corrupt block relative dba: 0x00403200 (file 1, block 12800)
Bad header found during dbv:
Data in bad block:
 type: 88 format: 1 rdba: 0x33877808
 last change scn: 0x257a.7b3a44e3 seq: 0xe8 flg: 0xe6
 spare1: 0x4e spare2: 0x73 spare3: 0x0
 consistency value in tail: 0x65251001
 check value in block header: 0xc3b4
 computed block checksum: 0x4ca7



DBVERIFY - 验证完成

检查的页总数: 13440
处理的页总数 (数据): 3297
失败的页总数 (数据): 0
处理的页总数 (索引): 2097
失败的页总数 (索引): 0
处理的页总数 (其他): 1441
处理的总页数 (段)  : 1
失败的总页数 (段)  : 0
空的页总数: 6604
标记为损坏的总页数: 1
流入的页总数: 0
加密的总页数        : 0
最高块 SCN            : 1667927064 (12.1667927064)

修复其他文件头,并dbv检查,发现均在12800位置损坏.尝试recover database恢复数据库,报ORA-00742 ORA-00312之类错误.【由于redo损坏,报出来数据文件更多坏块】

Sat Nov 25 17:03:39 2023
ALTER DATABASE RECOVER  database  
Media Recovery Start
 started logmerger process
Parallel Media Recovery started with 40 slaves
Sat Nov 25 17:03:40 2023
Recovery of Online Redo Log: Thread 1 Group 7 Seq 27220 Reading mem 0
  Mem# 0: /u01/app/oracle/oradata/redo07.log
Sat Nov 25 17:03:41 2023
Hex dump of (file 3, block 7) in trace file /u01/app/oracle/diag/rdbms/his/his/trace/his_pr0l_52669.trc
Corrupt block relative dba: 0x00c00007 (file 3, block 7)
Bad header found during media recovery
Data in bad block:
 type: 124 format: 7 rdba: 0x1698b845
 last change scn: 0x4fa1.3eaa638f seq: 0x6 flg: 0x24
 spare1: 0x26 spare2: 0x42 spare3: 0x0
 consistency value in tail: 0xa39e1e01
 check value in block header: 0x2ca4
 computed block checksum: 0x3b25
Reading datafile '/u01/app/oracle/oradata/undotbs01.dbf' for corruption at rdba: 0x00c00007 (file 3, block 7)
Reread (file 3, block 7) found same corrupt data (no logical check)
Sat Nov 25 17:03:41 2023
Hex dump of (file 46, block 3) in trace file /u01/app/oracle/diag/rdbms/his/his/trace/his_pr0w_52691.trc
Corrupt block relative dba: 0x0b800003 (file 46, block 3)
Bad header found during media recovery
Data in bad block:
 type: 7 format: 7 rdba: 0x77922022
 last change scn: 0xdff3.c40df5b6 seq: 0x6f flg: 0xe5
 spare1: 0xcd spare2: 0x6d spare3: 0x83d7
 consistency value in tail: 0x63c63d2c
 check value in block header: 0xf662
 computed block checksum: 0xec49
Data in bad block:
 type: 135 format: 4 rdba: 0x45ad2864
 last change scn: 0x9d7e.34949c73 seq: 0x32 flg: 0x3e
 spare1: 0x89 spare2: 0x0 spare3: 0x9f9f
 consistency value in tail: 0xa5807800
 check value in block header: 0xb2c9
 computed block checksum: 0x3aea
Reread (file 5, block 11259) found same corrupt data (no logical check)
 type: 214 format: 1 rdba: 0x0228dbe9
Bad header found during media recovery
 last change scn: 0xed57.ca4f7559 seq: 0x9b flg: 0x4a
Data in bad block:
 spare1: 0x97 spare2: 0x77 spare3: 0x2bab
 type: 33 format: 6 rdba: 0x018d584a
 consistency value in tail: 0x359f90d6
 last change scn: 0xaeb8.2fa361eb seq: 0x60 flg: 0x92
 check value in block header: 0x6b26
 spare1: 0xea spare2: 0xe spare3: 0xb405 block checksum disabled
Reread (file 3, block 4) found same corrupt data (no logical check)
Corrupt block relative dba: 0x0b800e61 (file 46, block 3681)
Bad header found during media recovery
Data in bad block:
 type: 131 format: 6 rdba: 0xc7edd0fc
 last change scn: 0xd319.d0e54941 seq: 0x6f flg: 0x6d
 spare1: 0xe7 spare2: 0x82 spare3: 0x439f
 consistency value in tail: 0x18dc47b6
 check value in block header: 0xe9c8
 computed block checksum: 0x204d
Reread (file 46, block 3681) found same corrupt data (no logical check)
Hex dump of (file 1, block 2017) in trace file /u01/app/oracle/diag/rdbms/his/his/trace/his_pr10_52699.trc
Corrupt block relative dba: 0x004007e1 (file 1, block 2017)
Bad header found during media recovery
Data in bad block:
 type: 159 format: 2 rdba: 0x52c5b2b0
 last change scn: 0x2ed8.e0bc5af9 seq: 0x62 flg: 0xe9
 spare1: 0x81 spare2: 0x1e spare3: 0xda98
 consistency value in tail: 0xc5753dd3
 check value in block header: 0x2bba
 block checksum disabled
Reading datafile '/u01/app/oracle/oradata/system01.dbf' for corruption at rdba: 0x004007e1 (file 1, block 2017)
Reread (file 1, block 2017) found same corrupt data (no logical check)
Media Recovery failed with error 742
Errors in file /u01/app/oracle/diag/rdbms/his/his/trace/his_pr00_52622.trc:
ORA-00283: recovery session canceled due to errors
ORA-00742: Log read detects lost write in thread %d sequence %d block %d
ORA-00312: online log 7 thread 1: '/u01/app/oracle/oradata/redo07.log'
ORA-10877 signalled during: ALTER DATABASE RECOVER  database  ...

尝试强制打开数据库报ORA-600 krsi_al_hdr_update.15,参考:Oracle断电故障处理中有类似报错

SQL> alter database open resetlogs;
alter database open resetlogs
*
ERROR at line 1:
ORA-00600: internal error code, arguments: [krsi_al_hdr_update.15],
[4294967295], [], [], [], [], [], [], [], [], [], []

由于redo问题无法resetlogs成功,解决异常redo,再次尝试open库,由于undo坏块无法open成功,报ORA-01092 ORA-01578等错误

SQL> alter database open resetlogs;
alter database open resetlogs
*
ERROR at line 1:
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-01578: ORACLE data block corrupted (file # 3, block # 1848)
ORA-01110: data file 3: '/u01/app/oracle/oradata/undotbs01.dbf'
Process ID: 55655
Session ID: 2623 Serial number: 5

解决undo异常,数据库open成功.导出客户需要数据,完成此次恢复工作

ORA-1200/ORA-1207数据库恢复

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：ORA-1200/ORA-1207数据库恢复

由于系统性能问题或者底层io问题,数据库alert日志报一下控制文件损坏错误然后crash掉

Mon Nov 13 08:06:44 2023
Thread 1 advanced to log sequence 12100 (LGWR switch)
  Current log# 1 seq# 12100 mem# 0: /u01/oracle/oradata/xifenfei/redo01.log
Mon Nov 13 09:23:59 2023
********************* ATTENTION: ******************** 
 The controlfile header block returned by the OS
 has a sequence number that is too old. 
 The controlfile might be corrupted.
 PLEASE DO NOT ATTEMPT TO START UP THE INSTANCE 
 without following the steps below.
 RE-STARTING THE INSTANCE CAN CAUSE SERIOUS DAMAGE 
 TO THE DATABASE, if the controlfile is truly corrupted.
 In order to re-start the instance safely, 
 please do the following:
 (1) Save all copies of the controlfile for later 
     analysis and contact your OS vendor and Oracle support.
 (2) Mount the instance and issue: 
     ALTER DATABASE BACKUP CONTROLFILE TO TRACE;
 (3) Unmount the instance. 
 (4) Use the script in the trace file to
     RE-CREATE THE CONTROLFILE and open the database. 
*****************************************************
USER (ospid: 17064): terminating the instance
Mon Nov 13 09:24:00 2023
System state dump requested by (instance=1, osid=17064), summary=[abnormal instance termination].

重启数据库报ORA-01122 ORA-01110 ORA-01207错误

Mon Nov 13 10:11:21 2023
ALTER DATABASE OPEN
Errors in file /u01/oracle/diag/rdbms/xifenfei/xifenfei/trace/xifenfei_ora_25824.trc:
ORA-01122: database file 1 failed verification check
ORA-01110: data file 1: '/u01/oracle/oradata/xifenfei/system01.dbf'
ORA-01207: file is more recent than control file - old control file
ORA-1122 signalled during: ALTER DATABASE OPEN...

处理好上述错误之后遭遇ORA-01122 ORA-01110 ORA-01200,类似文章:
bbed处理ORA-01200故障
 ORA-01122 ORA-01200故障处理

Mon Nov 13 10:51:48 2023
alter database open
Read of datafile '/u01/oracle/oradata/xifenfei/sysaux01.dbf' (fno 2) header failed with ORA-01200
Rereading datafile 2 header failed with ORA-01200
Errors in file /u01/oracle/diag/rdbms/xifenfei/xifenfei/trace/xifenfei_ora_24148.trc:
ORA-01122: database file 2 failed verification check
ORA-01110: data file 2: '/u01/oracle/oradata/xifenfei/sysaux01.dbf'
ORA-01200: actual file size of 2860800 is smaller than correct size of 2867200 blocks
ORA-1122 signalled during: alter database open...

解决上述错误之后,尝试open库报ORA-00314 ORA-00312之类错误

Mon Nov 13 18:00:43 2023
alter database open
Beginning crash recovery of 1 threads
 parallel recovery started with 15 processes
Started redo scan
Completed redo scan
 read 61894 KB redo, 589 data blocks need recovery
Started redo application at
 Thread 1: logseq 12100, block 112760
Recovery of Online Redo Log: Thread 1 Group 1 Seq 12100 Reading mem 0
  Mem# 0: /u01/oracle/oradata/xifenfei/redo01.log
Completed redo application of 1.20MB
Mon Nov 13 18:00:44 2023
Hex dump of (file 2, block 39078) in trace file /u01/oracle/diag/rdbms/xifenfei/xifenfei/trace/xifenfei_p011_27469.trc
Reading datafile '/u01/oracle/oradata/xifenfei/sysaux01.dbf' for corruption at rdba: 0x008098a6 (file 2, block 39078)
Reread (file 2, block 39078) found same corrupt data (logically corrupt)
RECOVERY OF THREAD 1 STUCK AT BLOCK 39078 OF FILE 2
Mon Nov 13 18:00:44 2023
Exception [type: SIGSEGV, Address not mapped to object][ADDR:0xC][PC:0x95FB838, kdxlin()+4946][flags: 0x0, count: 1]
Mon Nov 13 18:00:44 2023
Exception [type: SIGSEGV, Address not mapped to object][ADDR:0xC][PC:0x95FB4DE, kdxlin()+4088][flags: 0x0, count: 1]
Errors in file /u01/oracle/diag/rdbms/xifenfei/xifenfei/trace/xifenfei_p011_27469.trc:
ORA-00314: log 2 of thread 1, expected sequence# 12093 doesn't match 12085
ORA-00312: online log 2 thread 1: '/u01/oracle/oradata/xifenfei/redo02.log'
Errors in file /u01/oracle/diag/rdbms/xifenfei/xifenfei/trace/xifenfei_p011_27469.trc:
ORA-00314: log 3 of thread 1, expected sequence# 12096 doesn't match 12080
ORA-00312: online log 3 thread 1: '/u01/oracle/oradata/xifenfei/redo03.log'
ORA-00314: log 2 of thread 1, expected sequence# 12093 doesn't match 12085
ORA-00312: online log 2 thread 1: '/u01/oracle/oradata/xifenfei/redo02.log'

后面继续处理遇到类似这样错误

ALTER DATABASE RECOVER    CANCEL  
Errors in file /u01/oracle/diag/rdbms/xifenfei/xifenfei/trace/xifenfei_pr00_31110.trc:
ORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error below
ORA-01194: file 1 needs more recovery to be consistent
ORA-01110: data file 1: '/u01/oracle/oradata/xifenfei/system01.dbf'
ORA-1547 signalled during: ALTER DATABASE RECOVER    CANCEL  ...
ALTER DATABASE RECOVER CANCEL 
ORA-1112 signalled during: ALTER DATABASE RECOVER CANCEL ...
Mon Nov 13 19:06:28 2023
alter database open resetlogs
ORA-1194 signalled during: alter database open resetlogs...

最后确认其他数据文件均可recover 成功,只有file 2 无法正常recover

SQL> recover datafile 1;
Media recovery complete.
SQL> recover datafile 2;
ORA-00283: recovery session canceled due to errors
ORA-00600: internal error code, arguments: [3020], [2], [950840], [9339448],
[], [], [], [], [], [], [], []
ORA-10567: Redo is inconsistent with data block (file# 2, block# 950840, file
offset is 3494313984 bytes)
ORA-10564: tablespace SYSAUX
ORA-01110: data file 2: '/u01/oracle/oradata/xifenfei/sysaux01.dbf'
ORA-10560: block type 'FIRST LEVEL BITMAP BLOCK'
SQL> recover datafile 3;
Media recovery complete.
SQL> recover datafile 4;
Media recovery complete.
SQL> recover datafile 5;
Media recovery complete.
SQL> recover datafile 6;
Media recovery complete.
SQL> recover datafile 7;
Media recovery complete.

SQL> recover  datafile 2 allow 1 corruption;
ORA-00283: recovery session canceled due to errors
ORA-00600: internal error code, arguments: [3020], [2], [2410240], [10798848],
[], [], [], [], [], [], [], []
ORA-10567: Redo is inconsistent with data block (file# 2, block# 2410240, file
offset is 2564816896 bytes)
ORA-10564: tablespace SYSAUX
ORA-01110: data file 2: '/u01/oracle/oradata/xifenfei/sysaux01.dbf'
ORA-10560: block type 'FIRST LEVEL BITMAP BLOCK'

通过bbed修改文件头,直接open数据库成功,并协助客户顺利导出数据
参考类似文章:
使用bbed修复损坏datafile header
使用bbed让rac中的sysaux数据文件online

存储断电导致ORA-00325故障恢复

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：存储断电导致ORA-00325故障恢复

由于存储断电,数据库启动最初报错

Reconfiguration complete
 parallel recovery started with 32 processes
Started redo scan
Completed redo scan
 read 22887 KB redo, 2025 data blocks need recovery
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl11/trace/orcl11_ora_25449.trc  (incident=25522):
ORA-00600: internal error code, arguments: [kcratr_nab_less_than_odr], [1], [203596], [1537], [3046], []
Incident details in: /u01/app/oracle/diag/rdbms/orcl/orcl11/incident/incdir_25522/orcl11_ora_25449_i25522.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Abort recovery for domain 0
Aborting crash recovery due to error 600
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl11/trace/orcl11_ora_25449.trc:
ORA-00600: internal error code, arguments: [kcratr_nab_less_than_odr], [1], [203596], [1537], [3046], []
Abort recovery for domain 0
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl11/trace/orcl11_ora_25449.trc:
ORA-00600: internal error code, arguments: [kcratr_nab_less_than_odr], [1], [203596], [1537], [3046], []
ORA-600 signalled during: ALTER DATABASE OPEN...

尝试重建ctl之后,恢复数据库报错ORA-00325错误

[oracle@xifenfei ~]$ oerr ora 325
00325, 00000, "archived log for thread %s, wrong thread # %s in header"
// *Cause:  The archived log is corrupted or for another thread. Can not
//          use the log for applying redo.
// *Action: Find correct archived log.

归档日志文件的头部中的线程编号和实际使用的线程编号不匹配的时候报出的错误。该错误的常见情形是由于线程 ID (thread ID) 没有被正确分配或更新造成的，如从主控制档（control file）恢复用户表空间后更改线程 ID 但系统没有知道，导致系统在访问归档日志文件时，由于归档日志文件的头部中的线程编号与实际使用的线程编号不匹配，而出现该错误。

尝试强制拉库,报ORA-600 2662错误

尝试修改scn继续启动库报ORA-600 kclchkblk_4

继续调整scn,数据库open成功.后续就是各种收尾工作(比如undo回滚段,tempfile,各种ORA-600错误等)

重建ctl遭遇ORA-01163 ORA-01110和ORA-600 kccccfl_1错误

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：重建ctl遭遇ORA-01163 ORA-01110和ORA-600 kccccfl_1错误

有客户联系我,数据库强制关闭之后无法正常启动,通过他们一系列尝试,数据库报open 报ORA-01555错误

通过分析确认是由于客户屏蔽一致性强制拉库失败导致

Wed Nov 01 19:49:03 2023
alter database open resetlogs
RESETLOGS is being done without consistancy checks. This may result
in a corrupted database. The database should be recreated.
RESETLOGS after incomplete recovery UNTIL CHANGE 1199697777
Archived Log entry 68106 added for thread 1 sequence 3829 ID 0x761a769a dest 1:
Archived Log entry 68107 added for thread 1 sequence 3830 ID 0x761a769a dest 1:
Archived Log entry 68108 added for thread 1 sequence 3828 ID 0x761a769a dest 1:
Clearing online redo logfile 1 /u01/app/oracle/oradata/ORACLE/redo01.log
Clearing online log 1 of thread 1 sequence number 3829
Clearing online redo logfile 1 complete
Clearing online redo logfile 2 /u01/app/oracle/oradata/ORACLE/redo02.log
Clearing online log 2 of thread 1 sequence number 3830
Clearing online redo logfile 2 complete
Clearing online redo logfile 3 /u01/app/oracle/oradata/ORACLE/redo03.log
Clearing online log 3 of thread 1 sequence number 3828
Clearing online redo logfile 3 complete
Resetting resetlogs activation ID 1981445786 (0x761a769a)
Online log /u01/app/oracle/oradata/ORACLE/redo01.log: Thread 1 Group 1 was previously cleared
Online log /u01/app/oracle/oradata/ORACLE/redo02.log: Thread 1 Group 2 was previously cleared
Online log /u01/app/oracle/oradata/ORACLE/redo03.log: Thread 1 Group 3 was previously cleared
Wed Nov 01 19:49:09 2023
Setting recovery target incarnation to 3
Wed Nov 01 19:49:09 2023
Assigning activation ID 2024817472 (0x78b04340)
LGWR: STARTING ARCH PROCESSES
Wed Nov 01 19:49:09 2023
ARC0 started with pid=21, OS id=2078 
ARC0: Archival started
LGWR: STARTING ARCH PROCESSES COMPLETE
ARC0: STARTING ARCH PROCESSES
Wed Nov 01 19:49:10 2023
ARC1 started with pid=22, OS id=2080 
Thread 1 advanced to log sequence 2 (thread open)
Wed Nov 01 19:49:10 2023
ARC2 started with pid=23, OS id=2082 
Wed Nov 01 19:49:10 2023
ARC3 started with pid=24, OS id=2084 
ARC1: Archival started
ARC2: Archival started
ARC1: Becoming the 'no FAL' ARCH
ARC1: Becoming the 'no SRL' ARCH
ARC2: Becoming the heartbeat ARCH
Thread 1 opened at log sequence 2
  Current log# 2 seq# 2 mem# 0: /u01/app/oracle/oradata/ORACLE/redo02.log
Successful open of redo thread 1
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Wed Nov 01 19:49:10 2023
SMON: enabling cache recovery
ARC2: LGWR is scheduled to archive destination LOG_ARCHIVE_DEST_2 after log switch
Wed Nov 01 19:49:10 2023
NSA2 started with pid=25, OS id=2086 
ORA-01555 caused by SQL statement below (SQL ID: 4krwuz0ctqxdt, SCN: 0x0000.4781ef78):
select ctime, mtime, stime from obj$ where obj# = :1
Errors in file /u01/app/oracle/diag/rdbms/oracle_pd/ORACLE/trace/ORACLE_ora_2076.trc:
ORA-00704: bootstrap process failure
ORA-00704: bootstrap process failure
ORA-00604: error occurred at recursive SQL level 1
ORA-01555: snapshot too old: rollback segment number 11 with name "_SYSSMU11_467363169$" too small
Errors in file /u01/app/oracle/diag/rdbms/oracle_pd/ORACLE/trace/ORACLE_ora_2076.trc:
ORA-00704: bootstrap process failure
ORA-00704: bootstrap process failure
ORA-00604: error occurred at recursive SQL level 1
ORA-01555: snapshot too old: rollback segment number 11 with name "_SYSSMU11_467363169$" too small
Error 704 happened during db open, shutting down database
USER (ospid: 2076): terminating the instance due to error 704
Instance terminated by USER, pid = 2076
ORA-1092 signalled during: alter database open resetlogs...
opiodr aborting process unknown ospid (2076) as a result of ORA-1092

这种错误比较常见恢复简单,以前有过类似的文章:
在数据库open过程中常遇到ORA-01555汇总
 数据库open过程遭遇ORA-1555对应sql语句补充
 重建control遗漏数据文件,reseltogs报ORA-1555错误处理
 Oracle Recovery Tools恢复—ORA-00704 ORA-01555故障
这个错误恢复本身不难,只是在数据库的恢复过程中需要重建文件,在这次的重建文件中由于ctl语句错误【语句是通过sql查询拼接而成】,导致重建文件报了两个错误,供参考:
ORA-01163 ORA-01110错误

SQL> CREATE CONTROLFILE REUSE DATABASE "ORACLE" NORESETLOGS  NOARCHIVELOG
        MAXLOGFILES 50
       MAXLOGMEMBERS 5
       MAXDATAFILES 1000
       MAXINSTANCES 8
       MAXLOGHISTORY 2920
LOGFILE
       group 1   '/u01/app/oracle/oradata/ORACLE/redo01.log' size 200M,
  2         group 3   '/u01/app/oracle/oradata/ORACLE/redo03.log' size 200M,
       group 2   '/u01/app/oracle/oradata/ORACLE/redo02.log' size 200M,
DATAFILE
        '/u01/app/oracle/oradata/ORACLE/system01.dbf',
        …………
        '/u01/app/oracle/oradata/ORACLE/xifenfei06.dbf',
CHARACTER SET  ZHS16GBK ;
 23   24   25   26   27  CREATE CONTROLFILE REUSE DATABASE "ORACLE" NORESETLOGS	NOARCHIVELOG
*
ERROR at line 1:
ORA-01503: CREATE CONTROLFILE failed
ORA-01163: SIZE clause indicates 12800 (blocks), but should match header
1564672
ORA-01110: data file 15: '/u01/app/oracle/oradata/ORACLE/xifenfei06.dbf'

出现该错误是由于我写路径的时候在最后一个数据文件路径后面多了一个【,】导致

ORA-600 kccccfl_1错误

SQL> CREATE CONTROLFILE REUSE DATABASE "ORACLE" NORESETLOGS  NOARCHIVELOG
        MAXLOGFILES 50
       MAXLOGMEMBERS 5
       MAXDATAFILES 1000
       MAXINSTANCES 8
       MAXLOGHISTORY 2920
LOGFILE
       group 1   '/u01/app/oracle/oradata/ORACLE/redo01.log'  size 200M,
       group 3   '/u01/app/oracle/oradata/ORACLE/redo03.log' size 200M,
       group 2   '/u01/app/oracle/oradata/ORACLE/redo02.log' size 200M,
DATAFILE
        '/u01/app/oracle/oradata/ORACLE/system01.dbf',
        ………… 
        '/u01/app/oracle/oradata/ORACLE/xifenfei06.dbf'
CHARACTER SET  ZHS16GBK ;
 23   24   25   26   27  CREATE CONTROLFILE REUSE DATABASE "ORACLE" NORESETLOGS	NOARCHIVELOG
*
ERROR at line 1:
ORA-01503: CREATE CONTROLFILE failed
ORA-00600: internal error code, arguments: [kccccfl_1], [], [], [], [], [], [],
[], [], [], [], []

出现该错误是由于在最后redo文件路径后面多了【,】导致

再现ORA-600 4000故障处理

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：再现ORA-600 4000故障处理

有一个10g的库，由于redo损坏导致无法正常recover成功
20231018111911

正常途径无法open成功,尝试强制打开库

Wed Oct 18 11:23:25 2023
alter database open resetlogs
Wed Oct 18 11:23:25 2023
RESETLOGS is being done without consistancy checks. This may result
in a corrupted database. The database should be recreated.
RESETLOGS after incomplete recovery UNTIL CHANGE 236715370
Resetting resetlogs activation ID 4009817354 (0xef00f50a)
Wed Oct 18 11:23:26 2023
Setting recovery target incarnation to 3
Wed Oct 18 11:23:26 2023
Assigning activation ID 4271120987 (0xfe94225b)
Thread 1 opened at log sequence 1
  Current log# 1 seq# 1 mem# 0: D:\ORADATA\EAIS\REDO01.LOG
Successful open of redo thread 1
Wed Oct 18 11:23:26 2023
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Wed Oct 18 11:23:26 2023
SMON: enabling cache recovery
Wed Oct 18 11:23:27 2023
Errors in file d:\hys_db\admin\eais\udump\eais_ora_10916.trc:
ORA-00600: 内部错误代码, 参数: [4000], [9], [], [], [], [], [], []

Wed Oct 18 11:23:28 2023
Errors in file d:\hys_db\admin\eais\udump\eais_ora_10916.trc:
ORA-00704: 引导程序进程失败
ORA-00704: 引导程序进程失败
ORA-00600: 内部错误代码, 参数: [4000], [9], [], [], [], [], [], []

Wed Oct 18 11:23:28 2023
Error 704 happened during db open, shutting down database
USER: terminating instance due to error 704
Wed Oct 18 11:23:28 2023
Errors in file d:\hys_db\admin\eais\bdump\eais_pmon_1764.trc:
ORA-00704: bootstrap process failure

Wed Oct 18 11:23:29 2023
Errors in file d:\hys_db\admin\eais\bdump\eais_reco_9628.trc:
ORA-00704: bootstrap process failure

Wed Oct 18 11:23:29 2023
Errors in file d:\hys_db\admin\eais\bdump\eais_smon_9884.trc:
ORA-00704: bootstrap process failure

Wed Oct 18 11:23:29 2023
Errors in file d:\hys_db\admin\eais\bdump\eais_ckpt_10096.trc:
ORA-00704: bootstrap process failure

Wed Oct 18 11:23:29 2023
Errors in file d:\hys_db\admin\eais\bdump\eais_lgwr_8932.trc:
ORA-00704: bootstrap process failure

Wed Oct 18 11:23:29 2023
Errors in file d:\hys_db\admin\eais\bdump\eais_dbw0_9892.trc:
ORA-00704: bootstrap process failure

Wed Oct 18 11:23:29 2023
Errors in file d:\hys_db\admin\eais\bdump\eais_mman_9472.trc:
ORA-00704: bootstrap process failure

Wed Oct 18 11:23:29 2023
Errors in file d:\hys_db\admin\eais\bdump\eais_psp0_7692.trc:
ORA-00704: bootstrap process failure

Instance terminated by USER, pid = 10916
ORA-1092 signalled during: alter database open resetlogs...

resetlogs失败,报ora-600 4000错误,查看相关trace文件

*** 2023-10-18 11:23:27.103
ksedmp: internal or fatal error
ORA-00600: 内部错误代码, 参数: [4000], [9], [], [], [], [], [], []
Current SQL statement for this session:
select ctime, mtime, stime from obj$ where obj# = :1
----- Call Stack Trace -----
calling              call     entry                argument values in hex      
location             type     point                (? means dubious value)     
-------------------- -------- -------------------- ----------------------------
_ksedst+38           CALLrel  _ksedst1+0           0 1
_ksedmp+898          CALLrel  _ksedst+0            0
_ksfdmp+14           CALLrel  _ksedmp+0            3
603A816A             CALLreg  00000000             6AE8258 3
603A8550             CALLrel  603A80D8             6AE8258 4FB1090 FA0 1 8C886B0
__VInfreq__ktudba+1  CALLrel  _kgeasi+0            6AE8258 4FB1090 FA0 2 1 0 9 0
26                                                 
_ktrgcm+851          CALLrel  _ktudba+0            9 8C88B68 0 0
_ktrgtc+526          CALLrel  _ktrgcm+0            4FC753C
_kdsgrp+937          CALLrel  _ktrgtc+0            4FC753C 4FC74E0 1F1FB00
                                                   8C88CB4 240 2AED19C 1F1F71C
_kdsfbrcb+432        CALLrel  _kdsgrp+0            4FC7538 0 4FC7538
_qertbFetchByRowID+  CALLrel  _kdsfbrcb+0          4FC7538 4FDF004 0 1 0 0
4344                                               4FC74E0 22E8078 8C88E14 0
_opifch2+3104        CALL???  00000000             8DA05838 20135FC 8C890AC 1
_opifch+51           CALLrel  _opifch2+0           89 5 8C891F0
_opiodr+1099         CALLreg  00000000             5 2 8C89938
_rpidrus+178         CALLrel  _opiodr+0            5 2 8C89938 5
_rpidru+88           CALLrel  _rpidrus+0           8C894C4
_rpiswu2+426         CALLreg  00000000             8C8988C
_rpidrv+1461         CALLrel  _rpiswu2+0           90F3CCB4 0 8C89860 2 8C89880
                                                   0 8C89860 0 84DF58 84E018
                                                   8C8988C 8
_rpifch+44           CALLrel  _rpidrv+0            5 5 8C89938 8
_kqdpts+166          CALLrel  _rpifch+0            5 5 5 3 90AE2383 7 C 0 0 0 0
                                                   0 0 0 5 2 90AE237C 7 C 0 0 0
                                                   0 0 0 0
_kqrlfc+478          CALLrel  _kqdpts+0            90AE21F4
_kqlbplc+102         CALLrel  _kqrlfc+0            
_kqlblfc+240         CALLrel  _kqlbplc+0           0
_adbdrv+12911        CALLrel  _kqlblfc+0           0 8C8D034
_opiexe+11778        CALLrel  _adbdrv+0            
_opiosq0+6088        CALLrel  _opiexe+0            4 0 8C8D894
_kpooprx+232         CALLrel  _opiosq0+0           3 E 8C8D9AC A4
_kpoal8+775          CALLrel  _kpooprx+0           8C8F6CC 8C8E248 1D 1 0 A4
_opiodr+1099         CALLreg  00000000             5E 17 8C8F6C8
60FEFF8D             CALLreg  00000000             5E 17 8C8F6C8 0
_opitsk+1017         CALL???  00000000             
_opiino+1087         CALLrel  _opitsk+0            0 0
_opiodr+1099         CALLreg  00000000             3C 4 8C8FC60
_opidrv+819          CALLrel  _opiodr+0            3C 4 8C8FC60 0
_sou2o+45            CALLrel  _opidrv+0            3C 4 8C8FC60
_opimai_real+112     CALLrel  _sou2o+0             8C8FC54 3C 4 8C8FC60
_opimai+92           CALLrel  _opimai_real+0       2 8C8FC8C
_OracleThreadStart@  CALLrel  _opimai+0            
4+708                                              
74A93438             CALLptr  00000000             
76F09830             CALLreg  00000000             
76F09800             CALLrel  76F0980B             
 
--------------------- Binary Stack Dump ---------------------

通过一些技巧处理数据库正常open,导出数据完成本次恢复，参考以前恢复类似case:

重现ORA-600 4000异常
 通过bbed解决ORA-00600[4000]案例
 记录一次ORA-600 4000数据库故障恢复
 ORACLE 8.1.7 数据库ORA-600 4000故障恢复

ORA-07445: exception encountered: core dump [kdxlin()+4088]处理

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：ORA-07445: exception encountered: core dump [kdxlin()+4088]处理

abort方式关闭数据库，启动报错

Tue Sep 19 21:52:56 2023
NOTE: dependency between database orcl and diskgroup resource ora.DATA.dg is established
Tue Sep 19 21:52:57 2023
Reconfiguration started (old inc 4, new inc 6)
List of instances:
 1 (myinst: 1) 
 Global Resource Directory frozen
 * dead instance detected - domain 0 invalid = TRUE 
 Communication channels reestablished
 Master broadcasted resource hash value bitmaps
 Non-local Process blocks cleaned out
Tue Sep 19 21:52:57 2023
Tue Sep 19 21:52:57 2023
 LMS 3: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
Tue Sep 19 21:52:57 2023
 LMS 0: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
 LMS 1: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
Tue Sep 19 21:52:57 2023
 LMS 2: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
 Set master node info 
 Submitted all remote-enqueue requests
 Dwn-cvts replayed, VALBLKs dubious
 All grantable enqueues granted
 Post SMON to start 1st pass IR
 Submitted all GCS remote-cache requests
 Post SMON to start 1st pass IR
 Fix write in gcs resources
Reconfiguration complete
Tue Sep 19 21:53:05 2023
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl1/trace/orcl1_ora_28917.trc  (incident=492333):
ORA-00600: internal error code, arguments: [2131], [33], [32], [], [], [], [], [], [], [], [], []
Incident details in:/u01/app/oracle/diag/rdbms/orcl/orcl1/incident/incdir_492333/orcl1_ora_28917_i492333.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
ORA-600 signalled during: ALTER DATABASE MOUNT /* db agent *//* {1:34652:2} */...

重建ctl之后，尝试recover数据库报错ORA-600 3020和ORA-07445 kdxlin等错误

SQL> recover database;
ORA-00600: internal error code, arguments: [3020], [41], [3142531],
[175108995], [], [], [], [], [], [], [], []
ORA-10567: Redo is inconsistent with data block (file# 41, block# 3142531, file
offset is 4268777472 bytes)
ORA-10564: tablespace XIFENFEI
ORA-01110: data file 41: '+DATA/orcl/datafile/xifenfei07.dbf'
ORA-10560: block type 'FIRST LEVEL BITMAP BLOCK'

Wed Sep 20 00:15:00 2023
ALTER DATABASE RECOVER  database  
Media Recovery Start
 started logmerger process
Parallel Media Recovery started with 64 slaves
Wed Sep 20 00:15:02 2023
Recovery of Online Redo Log: Thread 2 Group 6 Seq 67008 Reading mem 0
  Mem# 0: +DATA/orcl/onlinelog/group_6.268.942097791
Recovery of Online Redo Log: Thread 1 Group 2 Seq 81767 Reading mem 0
  Mem# 0: +DATA/orcl/onlinelog/group_2.262.942097651
Recovery of Online Redo Log: Thread 1 Group 5 Seq 81768 Reading mem 0
  Mem# 0: +DATA/orcl/onlinelog/group_5.263.942097651
Wed Sep 20 00:15:08 2023
Hex dump of (file 41, block 3142531) in trace file /u01/app/oracle/diag/rdbms/orcl/orcl1/trace/orcl1_pr1m_45463.trc
Reading datafile '+DATA/orcl/datafile/ts_his3bz07.dbf' for corruption at rdba: 0x0a6ff383 (file 41, block 3142531)
Reread (file 41, block 3142531) found different corrupt data (logically corrupt)
Hex dump of (file 41, block 3142531) in trace file /u01/app/oracle/diag/rdbms/orcl/orcl1/trace/orcl1_pr1m_45463.trc
Wed Sep 20 00:15:08 2023
Exception [type: SIGSEGV, Address not mapped to object][ADDR:0xC] [PC:0x95FB582, kdxlin()+4088][flags: 0x0,count:1]
Wed Sep 20 00:15:08 2023
Exception [type: SIGSEGV, Address not mapped to object][ADDR:0xC] [PC:0x95FB582, kdxlin()+4088][flags: 0x0,count:1]
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl1/trace/orcl1_pr10_45419.trc  (incident=564584):
ORA-07445: exception encountered:core dump [kdxlin()+4088][SIGSEGV][ADDR:0xC][PC:0x95FB582][Address not mapped to object]
Incident details in: /u01/app/oracle/diag/rdbms/orcl/orcl1/incident/incdir_564640/orcl1_pr17_45433_i564640.trc

尝试随机恢复文件，也遭遇ORA-07445 kdxlin异常

SQL> recover datafile 34;
ORA-00283: recovery session canceled due to errors
ORA-10562: Error occurred while applying redo to data block (file# 34, block#
1999809)
ORA-10564: tablespace XIFENFEI
ORA-01110: data file 34: '+DATA/orcl/datafile/xifeifenfei06'
ORA-10561: block type 'TRANSACTION MANAGED INDEX BLOCK', data object# 97961
ORA-00607: Internal error occurred while making a change to a data block
ORA-00602: internal programming exception
ORA-07445: exception encountered: core dump [kdxlin()+4088] [SIGSEGV]
[ADDR:0xC] [PC:0x95FB582] [Address not mapped to object] []

出现这种情况是由于redo和数据文件块不一致导致无法正常应用日志,人工对于异常的block进行处理,数据库open成功,然后遭遇undo回滚段异常,对其进行规避,数据库open并且稳定运行

bbed解决ORA-01578

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：bbed解决ORA-01578

业务报ORA-01578坏块,无法正常使用,alert日志报错如下

Reading datafile '/data/u01/ZLDOCXML01.DBF' for corruption at rdba: 0x02efdc97 (file 11, block 3136663)
Reread (file 11, block 3136663) found same corrupt data
Wed Sep 13 19:02:04 2023
Corrupt Block Found
         TSN = 10, TSNAME = ZLDOCXML
         RFN = 11, BLK = 3136663, RDBA = 49274007
         OBJN = 73646, OBJD = 73646, OBJECT = SYS_LOB0000073645C00029$$, SUBOBJECT =
         SEGMENT OWNER = ZLDOC, SEGMENT TYPE = Lob Segment
DDE: Problem Key 'ORA 1578' was completely flood controlled (0x6)
Further messages for this problem key will be suppressed for up to 10 minutes

通过dbv检查数据文件,发现只有这一个坏块

[oracle@zlemr ~]$ dbv file=/data/u01/ZLDOCXML01.DBF

DBVERIFY: Release 11.2.0.1.0 - Production on Wed Sep 13 17:51:03 2023

Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.


DBVERIFY - Verification starting : FILE = /data/u01/ZLDOCXML01.DBF
Page 3136663 is influx - most likely media corrupt
Corrupt block relative dba: 0x02efdc97 (file 11, block 3136663)
Fractured block found during dbv: 
Data in bad block:
 type: 40 format: 2 rdba: 0x02efdc97
 last change scn: 0x0002.1065d622 seq: 0x2 flg: 0x04
 spare1: 0x0 spare2: 0x0 spare3: 0x0
 consistency value in tail: 0x00000001
 check value in block header: 0x48cf
 computed block checksum: 0xfe21


DBVERIFY - Verification complete

Total Pages Examined         : 3289600
Total Pages Processed (Data) : 0
Total Pages Failing   (Data) : 0
Total Pages Processed (Index): 21037
Total Pages Failing   (Index): 0
Total Pages Processed (Lob)  : 2051270
Total Pages Failing   (Lob)  : 0
Total Pages Processed (Other): 1068900
Total Pages Processed (Seg)  : 0
Total Pages Failing   (Seg)  : 0
Total Pages Empty            : 148392
Total Pages Marked Corrupt   : 1
Total Pages Influx           : 1
Total Pages Encrypted        : 0
Highest block SCN            : 278716397 (2.278716397)

通过bbed进入进行修复

[oracle@zlemr ~]$ bbed 
Password: 

BBED: Release 2.0.0.0.0 - Limited Production on Wed Sep 13 19:05:53 2023

Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.

************* !!! For Oracle Internal Use only !!! ***************

BBED> set filename '/data/u01/ZLDOCXML01.DBF'
        FILENAME        /data/u01/ZLDOCXML01.DBF

BBED> set block 3136663
        BLOCK#          3136663

BBED> verify
DBVERIFY - Verification starting
FILE = /data/u01/ZLDOCXML01.DBF
BLOCK = 3136663

Block 3136663 is corrupt
Corrupt block relative dba: 0x02efdc97 (file 0, block 3136663)
Fractured block found during verification
Data in bad block:
 type: 40 format: 2 rdba: 0x02efdc97
 last change scn: 0x0002.1065d622 seq: 0x2 flg: 0x04
 spare1: 0x0 spare2: 0x0 spare3: 0x0
 consistency value in tail: 0x00000001
 check value in block header: 0x48cf
 computed block checksum: 0xfe21


DBVERIFY - Verification complete

Total Blocks Examined         : 1
Total Blocks Processed (Data) : 0
Total Blocks Failing   (Data) : 0
Total Blocks Processed (Index): 0
Total Blocks Failing   (Index): 0
Total Blocks Empty            : 0
Total Blocks Marked Corrupt   : 1
Total Blocks Influx           : 2
Message 531 not found;  product=RDBMS; facility=BBED


BBED> map
 File: /data/u01/ZLDOCXML01.DBF (0)
 Block: 3136663                                Dba:0x00000000
------------------------------------------------------------
BBED-00400: invalid blocktype (40)


BBED> d
 File: /data/u01/ZLDOCXML01.DBF (0)
 Block: 3136663           Offsets:    0 to  511           Dba:0x00000000
------------------------------------------------------------------------
 28a20000 97dcef02 22d66510 02000204 cf480000 ae1f0100 00000001 00000280 
 93840000 00000000 00000000 00000000 80dcef02 00000000 000d000a 005b4e3b 
 6cbb533b 5e0867e5 623f8bb0 5f55005d 000d000a 4e3b6cbb 533b5e08 67e5623f 
 8bb05f55 00320030 00320033 002d0030 0039002d 00310033 00200030 0039003a 
 00310036 000d000a 4eca65e5 968f9ec4 4e1d51e4 4e3b6cbb 533b5e08 67e5623f 
 002c60a3 80058bc9 65e08179 75dbff0c 65e053d1 70ed3001 754f5bd2 ff0c65e0 
 59346655 30015934 75db3001 773c82b1 ff0c65e0 9f3b585e 30016d41 6d953001 
 54bd75db 300154b3 55fd3001 54b375f0 ff0c65e0 60765fc3 30015455 5410ff0c 
 65e080f8 95f73001 5fc360b8 3001547c 543856f0 96beff0c 65e05c3f 98913001 
 5c3f6025 30015c3f 75dbff0c 65e08179 6cfb7b49 4e0d9002 ff0c7cbe 795e3001 
 98df6b32 30017761 772053ef ff0c5927 5c0f4fbf 6b635e38 300267e5 4f53ff1a 
 751f547d 5f816b63 5e38ff0c 54bd65e0 51458840 ff0c6241 68434f53 65e080bf 
 5927ff0c 53cc4fa7 4e73623f 4e0d80c0 ff0c4e73 6c415206 6ccc591a ff0c672a 
 89e653ca 786c7ed3 ff0c5fc3 80ba672a 89c1660e 663e5f02 5e38ff0c 81798f6f 
 ff0c8179 90e8538b 75dbff0c 65e053cd 8df375db ff0c80a0 9e2397f3 6b635e38 
 30028f85 52a968c0 67e5ff1a 767d5e26 5e3889c4 ff1a767d 7ec680de 0020002b 

 <32 bytes per line>


BBED> set offset 8188
        OFFSET          8188

BBED> d
 File: /data/u01/ZLDOCXML01.DBF (0)
 Block: 3136663           Offsets: 8188 to 8191           Dba:0x00000000
------------------------------------------------------------------------
 01000000 

 <32 bytes per line>

BBED> set count 32
        COUNT           32

BBED> set mode edit
        MODE            Edit

BBED> d
 File: /data/u01/ZLDOCXML01.DBF (0)
 Block: 3136663           Offsets:   14 to   45           Dba:0x00000000
------------------------------------------------------------------------
 0204cf48 0000ae1f 01000000 00010000 02809384 00000000 00000000 00000000 

 <32 bytes per line>

BBED> set offset 8188
        OFFSET          8188

BBED> m /x 022822d6
 File: /data/u01/ZLDOCXML01.DBF (0)
 Block: 3136663           Offsets: 8188 to 8191           Dba:0x00000000
------------------------------------------------------------------------
 022822d6 

 <32 bytes per line>

BBED> sum apply
Check value for File 0, Block 3136663:
current = 0x48cf, required = 0x48cf

BBED> verify
DBVERIFY - Verification starting
FILE = /data/u01/ZLDOCXML01.DBF
BLOCK = 3136663


DBVERIFY - Verification complete

Total Blocks Examined         : 1
Total Blocks Processed (Data) : 0
Total Blocks Failing   (Data) : 0
Total Blocks Processed (Index): 0
Total Blocks Failing   (Index): 0
Total Blocks Empty            : 0
Total Blocks Marked Corrupt   : 0
Total Blocks Influx           : 0
Message 531 not found;  product=RDBMS; facility=BBED

dbv检测文件正常

[oracle@zlemr ~]$ dbv file=/data/u01/ZLDOCXML01.DBF

DBVERIFY: Release 11.2.0.1.0 - Production on Wed Sep 13 19:17:21 2023

Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.

DBVERIFY - Verification starting : FILE = /data/u01/ZLDOCXML01.DBF


DBVERIFY - Verification complete

Total Pages Examined         : 3289600
Total Pages Processed (Data) : 0
Total Pages Failing   (Data) : 0
Total Pages Processed (Index): 21037
Total Pages Failing   (Index): 0
Total Pages Processed (Lob)  : 2051586
Total Pages Failing   (Lob)  : 0
Total Pages Processed (Other): 1069031
Total Pages Processed (Seg)  : 0
Total Pages Failing   (Seg)  : 0
Total Pages Empty            : 147946
Total Pages Marked Corrupt   : 0
Total Pages Influx           : 0
Total Pages Encrypted        : 0
Highest block SCN            : 278849105 (2.278849105)

业务测试正常,数据完美恢复

ORA-600 ksuloget2 恢复

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：ORA-600 ksuloget2 恢复

客户在win 32位的操作系统上调至sga超过2G,数据库运行过程中报ORA-600 ksuloget2错误

Thread 1 cannot allocate new log, sequence 43586
Checkpoint not complete
  Current log# 1 seq# 43585 mem# 0: D:\ORACLE\ORADATA\ORCL\REDO01.LOG
Fri Aug 04 14:57:02 2023
Errors in file d:\oracle\diag\rdbms\orcl\orcl\trace\orcl_ora_42996.trc  (incident=67481):
ORA-00600: 内部错误代码, 参数: [ksuloget2], [0xFEBA6208], [0xFEBA3B08], [500], [0xFEBA622C], [], [], [], [], []
Thread 1 advanced to log sequence 43586 (LGWR switch)
  Current log# 2 seq# 43586 mem# 0: D:\ORACLE\ORADATA\ORCL\REDO02.LOG

重启数据库,进行尝试恢复继续报ORA-600 ksuloget2

Thu Aug 17 17:38:27 2023
ALTER DATABASE RECOVER  database using backup controlfile  
Media Recovery Start
 started logmerger process
Parallel Media Recovery started with 24 slaves
ORA-279 signalled during: ALTER DATABASE RECOVER  database using backup controlfile  ...
Thu Aug 17 17:39:01 2023
ALTER DATABASE RECOVER LOGFILE 'D:\oracle\flash_recovery_area\orcl\ARCHIVELOG\2023_08_04\REDO03.LOG'  
Media Recovery Log D:\oracle\flash_recovery_area\orcl\ARCHIVELOG\2023_08_04\REDO03.LOG
Thu Aug 17 17:39:01 2023
Errors in file d:\oracle\diag\rdbms\orcl\orcl\trace\orcl_pr00_5528.trc  (incident=110724):
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
ORA-00342: archived log does not have expected resetlogs SCN 685171428
ORA-00334: archived log: 'D:\ORACLE\FLASH_RECOVERY_AREA\ORCL\ARCHIVELOG\2023_08_04\REDO03.LOG'
Errors in file d:\oracle\diag\rdbms\orcl\orcl\trace\orcl_pr00_5528.trc:
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
ORA-00342: archived log does not have expected resetlogs SCN 685171428
ORA-00334: archived log: 'D:\ORACLE\FLASH_RECOVERY_AREA\ORCL\ARCHIVELOG\2023_08_04\REDO03.LOG'
Errors in file d:\oracle\diag\rdbms\orcl\orcl\trace\orcl_ora_5604.trc  (incident=110709):
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
ORA-00342: archived log does not have expected resetlogs SCN 685171428
ORA-00334: archived log: 'D:\ORACLE\FLASH_RECOVERY_AREA\ORCL\ARCHIVELOG\2023_08_04\REDO03.LOG'
Incident details in: d:\oracle\diag\rdbms\orcl\orcl\incident\incdir_110709\orcl_ora_5604_i110709.trc
ORA-600 signalled during:ALTER DATABASE RECOVER LOGFILE 'D:\oracle\flash_recovery_area\orcl\2023_08_04\REDO03.LOG'
ALTER DATABASE RECOVER CANCEL 
Errors in file d:\oracle\diag\rdbms\orcl\orcl\trace\orcl_pr00_5528.trc  (incident=110725):
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
Errors in file d:\oracle\diag\rdbms\orcl\orcl\trace\orcl_pr00_5528.trc  (incident=110726):
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
Errors in file d:\oracle\diag\rdbms\orcl\orcl\trace\orcl_pr00_5528.trc:
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
Errors in file d:\oracle\diag\rdbms\orcl\orcl\trace\orcl_pr00_5528.trc  (incident=110727):
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
Errors in file d:\oracle\diag\rdbms\orcl\orcl\trace\orcl_pr00_5528.trc  (incident=110728):
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
Errors in file d:\oracle\diag\rdbms\orcl\orcl\trace\orcl_ora_5604.trc  (incident=110710):
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [
Incident details in: d:\oracle\diag\rdbms\orcl\orcl\incident\incdir_110710\orcl_ora_5604_i110710.trc

由于是应用日志失败,屏蔽日志一致性,强制打开数据库,检查数据ok,业务可以直接使用,对于这类问题,官方建议:ORA-600: [Ksuloget2] Hit on Windows When SGA Greater Than 1G (Doc ID 836109.1)
20230819105750

分类目录归档：Oracle备份恢复