Oracle备份恢复 – Database SOS

接到客户反馈,一套运行在虚拟化平台中的Oracle数据库,由于机房断电,导致数据库无法启动,最初启动报错

2025-04-22T16:59:48.922227+08:00
Completed: alter database mount exclusive
alter database open
2025-04-22T16:59:52.609726+08:00
Ping without log force is disabled:
  instance mounted in exclusive mode.
2025-04-22T16:59:52.937852+08:00
Beginning crash recovery of 1 threads
 parallel recovery started with 15 processes
 Thread 1: Recovery starting at checkpoint rba (logseq 11031 block 139863), scn 0
2025-04-22T16:59:53.094081+08:00
Started redo scan
2025-04-22T16:59:53.203464+08:00
Completed redo scan
 read 15533 KB redo, 1140 data blocks need recovery
2025-04-22T16:59:53.515972+08:00
Hex dump of (file 3,block 82642) in trace file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p001_6612.trc

Corrupt block relative dba: 0x00c142d2 (file 3,block 82642)
Fractured block found during crash/instance recovery
Data in bad block:
 type: 6 format: 2 rdba: 0x00c142d2
 last change scn: 0x0000.0001.7a0219e2 seq: 0x1 flg: 0x06
 spare3: 0x0
 consistency value in tail: 0x0fcb0601
 check value in block header: 0x82eb
 computed block checksum: 0x172a

2025-04-22T16:59:53.563004+08:00
Reading datafile 'D:\APP\ADMINISTRATOR\ORADATA\HIS\SYSAUX01.DBF' for corrupt data at rdba: 0x00c142d2 (file 3,block 82642)
Reread (file 3,block 82642) found same corrupt data (no logical check)
2025-04-22T16:59:54.578467+08:00
Hex dump of (file 3,block 207498) in trace file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p000_6064.trc
2025-04-22T16:59:54.578467+08:00

Corrupt block relative dba: 0x00c32a8a (file 3,block 207498)
Fractured block found during crash/instance recovery
Data in bad block:
 type: 6 format: 2 rdba: 0x00c32a8a
 last change scn: 0x0000.0001.731ec74f seq: 0x2 flg: 0x06
 spare3: 0x0
 consistency value in tail: 0x10520601
 check value in block header: 0x61c3
 computed block checksum: 0xe27a

Reading datafile 'D:\APP\ADMINISTRATOR\ORADATA\HIS\SYSAUX01.DBF' for corrupt data at rdba: 0x00c32a8a (file 3,block 207498)
Reread (file 3,block 207498) found same corrupt data (no logical check)
Hex dump of (file 4, block 3959) in trace file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p000_6064.trc

Corrupt block relative dba: 0x01000f77 (file 4, block 3959)
Fractured block found during crash/instance recovery
Data in bad block:
 type: 2 format: 2 rdba: 0x01000f77
 last change scn: 0x0000.0001.79fa3621 seq: 0x4 flg: 0x04
 spare3: 0x0
 consistency value in tail: 0xcf6c0201
 check value in block header: 0x99d2
 computed block checksum: 0x79e0

Reading datafile 'D:\APP\ADMINISTRATOR\ORADATA\HIS\UNDOTBS01.DBF' for corrupt data at rdba: 0x01000f77 (file 4,block 3959)
Reread (file 4, block 3959) found same corrupt data (no logical check)
2025-04-22T16:59:54.640991+08:00
Started redo application at
 Thread 1: logseq 11031, block 139863, offset 0
2025-04-22T16:59:54.672246+08:00
Recovery of Online Redo Log: Thread 1 Group 3 Seq 11031 Reading mem 0
  Mem# 0: D:\APP\ADMINISTRATOR\ORADATA\HIS\REDO03.LOG
2025-04-22T16:59:54.703494+08:00
Hex dump of (file 3,block 89738) in trace file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p006_5236.trc

Corrupt block relative dba: 0x00c15e8a (file 3,block 89738)
Fractured block found during crash/instance recovery
Data in bad block:
 type: 6 format: 2 rdba: 0x00c15e8a
 last change scn: 0x0000.0001.7a09108a seq: 0x2 flg: 0x06
 spare3: 0x0
 consistency value in tail: 0x278e0602
 check value in block header: 0xd721
 computed block checksum: 0x2705

Reading datafile 'D:\APP\ADMINISTRATOR\ORADATA\HIS\SYSAUX01.DBF' for corrupt data at rdba: 0x00c15e8a (file 3,block 89738)
Reread (file 3,block 89738) found same corrupt data (no logical check)
2025-04-22T16:59:54.734756+08:00
Hex dump of (file 3,block 9504) in trace file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p005_3660.trc

Reading datafile 'D:\APP\ADMINISTRATOR\ORADATA\HIS\SYSAUX01.DBF' for corrupt data at rdba: 0x00c02520 (file 3,block 9504)
Reread (file 3,block 9504) found same corrupt data (logically corrupt)
*****************************************************************
An internal routine has requested a dump of selected redo.
This usually happens following a specific internal error, when
analysis of the redo logs will help Oracle Support with the
diagnosis.
It is recommended that you retain all the redo logs generated (by
all the instances) during the past 12 hours, in case additional
redo dumps are required to help with the diagnosis.
*****************************************************************
2025-04-22T16:59:54.750364+08:00
Exception [type: ACCESS_VIOLATION, UNABLE_TO_READ] [ADDR:0xC] [PC:0x7FF64221F2F8, kdxlin()+4824]
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p008_3964.trc  (incident=521460):
ORA-07445: 出现异常错误: 核心转储 [kdxlin()+4824][ACCESS_VIOLATION][ADDR:0xC][PC:0x7FF64221F2F8][UNABLE_TO_READ][]
Incident details in: D:\APP\ADMINISTRATOR\diag\rdbms\his\his\incident\incdir_521460\his_p008_3964_i521460.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
2025-04-22T16:59:55.156605+08:00
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p005_3660.trc:
ORA-00742: 日志读取在线程 1 序列 11029 块 51218 中检测到写入丢失情况
ORA-00312: 联机日志 1 线程 1: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\REDO01.LOG'
2025-04-22T16:59:55.828516+08:00
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p00a_3428.trc  (incident=521476):
ORA-00600: 内部错误代码, 参数: [ktbair2: illegal  inheritance], [], [], [], [], [], [], [], [], [], [], []
Incident details in: D:\APP\ADMINISTRATOR\diag\rdbms\his\his\incident\incdir_521476\his_p00a_3428_i521476.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
2025-04-22T16:59:56.844113+08:00
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p006_5236.trc  (incident=521444):
ORA-00600: 内部错误代码, 参数: [ktbair2: illegal  inheritance], [], [], [], [], [], [], [], [], [], [], []
Incident details in: D:\APP\ADMINISTRATOR\diag\rdbms\his\his\incident\incdir_521444\his_p006_5236_i521444.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
2025-04-22T16:59:59.062944+08:00
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p005_3660.trc:
ORA-00742: 日志读取在线程 1 序列 11029 块 51218 中检测到写入丢失情况
ORA-00334: 归档日志: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\REDO01.LOG'
2025-04-22T16:59:59.156645+08:00
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p00a_3428.trc:
ORA-00742: 日志读取在线程 1 序列 11029 块 51218 中检测到写入丢失情况
ORA-00312: 联机日志 1 线程 1: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\REDO01.LOG'
ORA-00600: 内部错误代码, 参数: [ktbair2: illegal  inheritance], [], [], [], [], [], [], [], [], [], [], []
Incident details in: D:\APP\ADMINISTRATOR\diag\rdbms\his\his\incident\incdir_521477\his_p00a_3428_i521477.trc
2025-04-22T17:00:04.250414+08:00
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p006_5236.trc:
ORA-00742: 日志读取在线程 1 序列 11029 块 51218 中检测到写入丢失情况
ORA-00334: 归档日志: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\REDO01.LOG'
ORA-00600: 内部错误代码, 参数: [ktbair2: illegal  inheritance], [], [], [], [], [], [], [], [], [], [], []
2025-04-22T17:00:04.344200+08:00
Slave encountered ORA-10388 exception during crash recovery
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p006_5236.trc  (incident=521445):
ORA-01578: ORACLE 数据块损坏 (文件号 3, 块号 299732)
ORA-01110: 数据文件 3: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\SYSAUX01.DBF'
ORA-10564: tablespace SYSAUX
ORA-01110: 数据文件 3: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\SYSAUX01.DBF'
ORA-10561: block type 'TRANSACTION MANAGED INDEX BLOCK', data object# 11351
ORA-00600: 内部错误代码, 参数: [ktbair2: illegal  inheritance], [], [], [], [], [], [], [], [], [], [], []
Incident details in: D:\APP\ADMINISTRATOR\diag\rdbms\his\his\incident\incdir_521445\his_p006_5236_i521445.trc
2025-04-22T17:00:05.312967+08:00
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p00a_3428.trc:
ORA-01578: ORACLE 数据块损坏 (文件号 3, 块号 216445)
ORA-01110: 数据文件 3: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\SYSAUX01.DBF'
ORA-10564: tablespace SYSAUX
ORA-01110: 数据文件 3: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\SYSAUX01.DBF'
ORA-10561: block type 'TRANSACTION MANAGED DATA BLOCK', data object# 11171
ORA-00600: 内部错误代码, 参数: [ktbair2: illegal  inheritance], [], [], [], [], [], [], [], [], [], [], []
2025-04-22T17:00:05.359805+08:00
Slave encountered ORA-10388 exception during crash recovery
2025-04-22T17:00:06.172286+08:00
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p006_5236.trc:
ORA-01578: ORACLE 数据块损坏 (文件号 3, 块号 299732)
ORA-01110: 数据文件 3: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\SYSAUX01.DBF'
ORA-10564: tablespace SYSAUX
ORA-01110: 数据文件 3: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\SYSAUX01.DBF'
ORA-10561: block type 'TRANSACTION MANAGED INDEX BLOCK', data object# 11351
ORA-00600: 内部错误代码, 参数: [ktbair2: illegal  inheritance], [], [], [], [], [], [], [], [], [], [], []
2025-04-22T17:00:06.172286+08:00
Slave encountered ORA-1578 exception during crash recovery
Slave exiting with ORA-1578 exception
2025-04-22T17:00:06.172286+08:00
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_p006_5236.trc:
ORA-01578: ORACLE 数据块损坏 (文件号 3, 块号 299732)
ORA-01110: 数据文件 3: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\SYSAUX01.DBF'
ORA-10564: tablespace SYSAUX
ORA-01110: 数据文件 3: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\SYSAUX01.DBF'
ORA-10561: block type 'TRANSACTION MANAGED INDEX BLOCK', data object# 11351
ORA-00600: 内部错误代码, 参数: [ktbair2: illegal  inheritance], [], [], [], [], [], [], [], [], [], [], []
ORA-1578 signalled during: alter database open...

这里面主要报错有几大类:
1.在需要实例恢复的block中有坏块,而且主要集中在3号文件(sysaux)上
2. ORA-00742: 日志读取在线程 1 序列 11029 块 51218 中检测到写入丢失情况
3. ORA-600 ktbair2: illegal inheritance错误
对于这些问题的本质都是数据库无法正常实例恢复,尝试按照数据文件级别进行恢复操作,结果只有datafile 3 不成功

2025-04-22T20:11:36.724880+08:00
ALTER DATABASE RECOVER  datafile 3  
2025-04-22T20:11:36.740523+08:00
Media Recovery Start
2025-04-22T20:11:36.771771+08:00
Serial Media Recovery started
2025-04-22T20:11:36.896781+08:00
Recovery of Online Redo Log: Thread 1 Group 3 Seq 11031 Reading mem 0
  Mem# 0: D:\APP\ADMINISTRATOR\ORADATA\HIS\REDO03.LOG
Exception [type: ACCESS_VIOLATION, UNABLE_TO_READ] [ADDR:0xC] [PC:0x7FF64221F2F8, kdxlin()+4824]
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_ora_2512.trc  (incident=721216):
ORA-07445: ??????: ???? [kdxlin()+4824] [ACCESS_VIOLATION] [ADDR:0xC] [PC:0x7FF64221F2F8] [UNABLE_TO_READ] []
Incident details in: D:\APP\ADMINISTRATOR\diag\rdbms\his\his\incident\incdir_721216\his_ora_2512_i721216.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
2025-04-22T20:12:56.585813+08:00
ALTER DATABASE RECOVER  datafile 3  
2025-04-22T20:12:56.601434+08:00
Media Recovery Start
2025-04-22T20:12:56.601434+08:00
Serial Media Recovery started
2025-04-22T20:12:57.351435+08:00
Recovery of Online Redo Log: Thread 1 Group 3 Seq 11031 Reading mem 0
  Mem# 0: D:\APP\ADMINISTRATOR\ORADATA\HIS\REDO03.LOG
2025-04-22T20:12:58.492071+08:00
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_ora_5332.trc  (incident=721217):
ORA-00600: 内部错误代码, 参数: [ktbair2: illegal  inheritance], [], [], [], [], [], [], [], [], [], [], []
Incident details in: D:\APP\ADMINISTRATOR\diag\rdbms\his\his\incident\incdir_721217\his_ora_5332_i721217.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
2025-04-22T20:13:00.710878+08:00
*****************************************************************
An internal routine has requested a dump of selected redo.
This usually happens following a specific internal error, when
analysis of the redo logs will help Oracle Support with the
diagnosis.
It is recommended that you retain all the redo logs generated (by
all the instances) during the past 12 hours, in case additional
redo dumps are required to help with the diagnosis.
*****************************************************************
2025-04-22T20:13:01.398399+08:00
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_ora_5332.trc:
ORA-00742: 日志读取在线程 1 序列 11029 块 51218 中检测到写入丢失情况
ORA-00312: 联机日志 1 线程 1: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\REDO01.LOG'
ORA-00600: 内部错误代码, 参数: [ktbair2: illegal  inheritance], [], [], [], [], [], [], [], [], [], [], []
2025-04-22T20:13:03.820304+08:00
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_mz00_2200.trc:
ORA-01110: 数据文件 3: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\SYSAUX01.DBF'
2025-04-22T20:13:06.070340+08:00
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\his\his\trace\his_ora_5332.trc:
ORA-00742: 日志读取在线程 1 序列 11029 块 51218 中检测到写入丢失情况
ORA-00334: 归档日志: 'D:\APP\ADMINISTRATOR\ORADATA\HIS\REDO01.LOG'
ORA-00600: 内部错误代码, 参数: [ktbair2: illegal  inheritance], [], [], [], [], [], [], [], [], [], [], []
2025-04-22T20:13:06.085990+08:00
Media Recovery failed with error 10562
ORA-283 signalled during: ALTER DATABASE RECOVER  datafile 3  ...

对于这样的故障,前段时间通过Oracle Recovery Tools工具处理过类似异常:Oracle Recovery Tools修复ORA-600 6101/kdxlin:psno out of range故障

然后尝试recover datafile 3,并打开数据库成功

至此数据库open成功,由于该库有不少坏块(包括system的一些字典),后续使用逻辑方式迁移数据到新库中(其中由于C_OBJ#簇中有坏块,导致迁移过程中大量index丢失,通过一些方法使用obj$,inde$等基表人工生成index.sql对于丢失的index进行弥补),最终客户业务正常运行

11.2.0.4库中遇到ORA-600 kcratr_nab_less_than_odr报错

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：11.2.0.4库中遇到ORA-600 kcratr_nab_less_than_odr报错

今天有一个客户11.2.0.4的库由于断电报ORA-600 kcratr_nab_less_than_odr的故障

C:\Users\Administrator>sqlplus / as sysdba

SQL*Plus: Release 11.2.0.4.0 Production on 星期二 4月 22 10:56:36 2025

Copyright (c) 1982, 2013, Oracle.  All rights reserved.


连接到:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> select * from v$version;

BANNER
--------------------------------------------------------------------------------

Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
PL/SQL Release 11.2.0.4.0 - Production
CORE    11.2.0.4.0      Production
TNS for 64-bit Windows: Version 11.2.0.4.0 - Production
NLSRTL Version 11.2.0.4.0 - Production

SQL> startup mount
ORACLE 例程已经启动。

Total System Global Area 1.9174E+10 bytes
Fixed Size                  2289320 bytes
Variable Size            3355443544 bytes
Database Buffers         1.5771E+10 bytes
Redo Buffers               45932544 bytes
数据库装载完毕。
SQL> alter database open;
alter database open
*
第 1 行出现错误:
ORA-00600: 内部错误代码, 参数: [kcratr_nab_less_than_odr], [1], [4653],
[50575], [50922], [], [], [], [], [], [], []

alert日志报错信息

Tue Apr 22 05:08:12 2025
ALTER DATABASE   MOUNT
Successful mount of redo thread 1, with mount id 3880792636
Database mounted in Exclusive Mode
Lost write protection disabled
Completed: ALTER DATABASE   MOUNT
Tue Apr 22 05:08:17 2025
ALTER DATABASE OPEN
Beginning crash recovery of 1 threads
 parallel recovery started with 15 processes
Started redo scan
Completed redo scan
 read 2451 KB redo, 1442 data blocks need recovery
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\ydhl\ydhl\trace\ydhl_ora_2164.trc  (incident=240429):
ORA-00600: 内部错误代码, 参数: [kcratr_nab_less_than_odr], [1], [4653], [50575], [50922],[],[],[],[],[],[],[]
Incident details in: D:\APP\ADMINISTRATOR\diag\rdbms\ydhl\ydhl\incident\incdir_240429\ydhl_ora_2164_i240429.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Aborting crash recovery due to error 600
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\ydhl\ydhl\trace\ydhl_ora_2164.trc:
ORA-00600: 内部错误代码, 参数: [kcratr_nab_less_than_odr], [1], [4653], [50575], [50922],[],[],[],[],[],[],[]
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\ydhl\ydhl\trace\ydhl_ora_2164.trc:
ORA-00600: 内部错误代码, 参数: [kcratr_nab_less_than_odr], [1], [4653], [50575], [50922],[],[],[],[],[],[],[]
ORA-600 signalled during: ALTER DATABASE OPEN...

这个故障在一般在11.2.0.1版本中最为常见,处理起来比较简单,一般数据也会0丢失
12c启动报kcratr_nab_less_than_odr
又一例ORA-600 kcratr_nab_less_than_odr
ORA-600 kcratr_nab_less_than_odr故障解决
 差点被误操作的ORA-600 kcratr_nab_less_than_odr故障
这里处理比较简单,重建控制文件就完成了恢复工作

Oracle Recovery Tools修复ORA-600 6101/kdxlin:psno out of range故障

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：Oracle Recovery Tools修复ORA-600 6101/kdxlin:psno out of range故障

数据库异常断电,然后启动异常,我接手该库,尝试recover恢复

SQL> recover database;
ORA-10562: Error occurred while applying redo to data block (file# 2, block#
63710)
ORA-10564: tablespace SYSAUX
ORA-01110: ???????? 2: 'H:\TEMP\GDLISNET\SYSAUX01.DBF'
ORA-10561: block type 'TRANSACTION MANAGED INDEX BLOCK', data object# 179030
ORA-00600: ????????????, ????: [6101], [0], [700], [32], [], [], [], [], [],
[], [], []

SQL> recover datafile 1;
完成介质恢复。
SQL> recover datafile 3;
完成介质恢复。
SQL> recover datafile 4;
完成介质恢复。
SQL> recover datafile 5;
完成介质恢复。
SQL> recover datafile 6;
完成介质恢复。
SQL> recover datafile 2;
ORA-00283: ??????????
ORA-10562: Error occurred while applying redo to data block (file# 2, block#
63744)
ORA-10564: tablespace SYSAUX
ORA-01110: ???? 2: 'H:\TEMP\GDLISNET\SYSAUX01.DBF'
ORA-10561: block type 'TRANSACTION MANAGED INDEX BLOCK', data object# 67482
ORA-00600: ??????, ??: [kdxlin:psno out of range], [], [], [], [], [], [], [],
[], [], [], []

alert日志信息

Mon Apr 14 12:31:55 2025
ALTER DATABASE RECOVER  database  
Media Recovery Start
 started logmerger process
Parallel Media Recovery started with 20 slaves
Mon Apr 14 12:31:56 2025
Recovery of Online Redo Log: Thread 1 Group 2 Seq 6845 Reading mem 0
  Mem# 0: H:\TEMP\GDLISNET\REDO02.LOG
Mon Apr 14 12:31:56 2025
Errors in file c:\app\xff\diag\rdbms\gdlisnet\gdlisnet\trace\gdlisnet_pr0e_9840.trc  (incident=1489):
ORA-00600: 内部错误代码, 参数: [6101], [0], [700], [32], [], [], [], [], [], [], [], []
Incident details in: c:\app\xff\diag\rdbms\gdlisnet\gdlisnet\incident\incdir_1489\gdlisnet_pr0e_9840_i1489.trc
Slave exiting with ORA-10562 exception
Errors in file c:\app\xff\diag\rdbms\gdlisnet\gdlisnet\trace\gdlisnet_pr0e_9840.trc:
ORA-10562: Error occurred while applying redo to data block (file# 2, block# 63710)
ORA-10564: tablespace SYSAUX
ORA-01110: 数据文件 2: 'H:\TEMP\GDLISNET\SYSAUX01.DBF'
ORA-10561: block type 'TRANSACTION MANAGED INDEX BLOCK', data object# 179030
ORA-00600: 内部错误代码, 参数: [6101], [0], [700], [32], [], [], [], [], [], [], [], []
Mon Apr 14 12:31:57 2025
Checker run found 6 new persistent data failures
Recovery Slave PR0E previously exited with exception 10562
Media Recovery failed with error 448
Errors in file c:\app\xff\diag\rdbms\gdlisnet\gdlisnet\trace\gdlisnet_pr00_14232.trc:
ORA-00283: 恢复会话因错误而取消
ORA-00448: 后台进程正常结束
Slave exiting with ORA-283 exception
Errors in file c:\app\xff\diag\rdbms\gdlisnet\gdlisnet\trace\gdlisnet_pr00_14232.trc:
ORA-00283: 恢复会话因错误而取消
ORA-00448: 后台进程正常结束
ORA-10562 signalled during: ALTER DATABASE RECOVER  database  ...

ALTER DATABASE RECOVER  datafile 2  
Media Recovery Start
Serial Media Recovery started
Recovery of Online Redo Log: Thread 1 Group 2 Seq 6845 Reading mem 0
  Mem# 0: H:\TEMP\GDLISNET\REDO02.LOG
Errors in file c:\app\xff\diag\rdbms\gdlisnet\gdlisnet\trace\gdlisnet_ora_22400.trc  (incident=1369):
ORA-00600: ??????, ??: [kdxlin:psno out of range], [], [], [], [], [], [], [], [], [], [], []
Incident details in: c:\app\xff\diag\rdbms\gdlisnet\gdlisnet\incident\incdir_1369\gdlisnet_ora_22400_i1369.trc
Media Recovery failed with error 10562
ORA-283 signalled during: ALTER DATABASE RECOVER  datafile 2  ...

发现2号数据文件异常,报ORA-600 6101和ORA-600 kdxlin:psno out of range错误,出现该错误的原因主要是由于数据文件的bock和redo中信息不匹配导致,对于这种情况,我这边选择使用Oracle Recovery Tools工具进行快速恢复

再次recover并且直接打开库

SQL> set numw 16
SQL> col CHECKPOINT_TIME for a40
SQL> set lines 150
SQL> set pages 1000
SQL> SELECT status,
  2  to_char(checkpoint_time,'yyyy-mm-dd hh24:mi:ss') checkpoint_time,FUZZY,checkpoint_change#,
  3  count(*) ROW_NUM
  4  FROM v$datafile_header
  5  GROUP BY status, checkpoint_change#, to_char(checkpoint_time,'yyyy-mm-dd hh24:mi:ss'),fuzzy
  6  ORDER BY status, checkpoint_change#, checkpoint_time;

STATUS  CHECKPOINT_TIME                          FUZ CHECKPOINT_CHANGE#          ROW_NUM
------- ---------------------------------------- --- ------------------ ----------------
ONLINE  2025-03-28 22:00:09                      NO           118575875                6

SQL>
SQL>
SQL> alter database open;
alter database open
*
第 1 行出现错误:
ORA-01113: 文件 2 需要介质恢复
ORA-01110: 数据文件 2: 'H:\TEMP\GDLISNET\SYSAUX01.DBF'


SQL> recover datafile 2;
完成介质恢复。
SQL> alter database open;

数据库已更改。

SQL>

逻辑方法导出该库,完成该库的恢复工作

虚拟机故障引起ORA-00310 ORA-00334故障处理

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：虚拟机故障引起ORA-00310 ORA-00334故障处理

有客户由于硬件底层问题,导致运行在虚拟机环境中的oracle数据库突然爆大量错误

Reread (file 5, block 2371528) found same corrupt data (no logical check)
Errors in file /home/oracle/app/diag/rdbms/orcl/orcl/trace/orcl_j000_10927.trc  (incident=397049):
ORA-01578: ORACLE data block corrupted (file # 5, block # 2371528)
ORA-01110: data file 5: '/home/oracle/app/oradata/users01.dbf'

Wed Apr 02 23:10:24 2025
Errors in file /home/oracle/app/diag/rdbms/orcl/orcl/trace/orcl_j000_10927.trc  (incident=397050):
ORA-00600: internal error code, arguments: [5400], [], [], [], [], [], [], [], [], [], [], []

Wed Apr 02 23:15:29 2025
Errors in file /home/oracle/app/diag/rdbms/orcl/orcl/trace/orcl_ora_11605.trc  (incident=397075):
ORA-00600: internal error code, arguments: [ktbdchk1: bad dscn], [], [], [], [], [], [], [], [], [], [], []

Wed Apr 02 23:20:32 2025
Errors in file /home/oracle/app/diag/rdbms/orcl/orcl/trace/orcl_ora_11530.trc  (incident=397034):
ORA-00600: internal error code, arguments: [25027], [6], [196610], [], [], [], [], [], [], [], [], []

Wed Apr 02 23:20:52 2025
Errors in file /home/oracle/app/diag/rdbms/orcl/orcl/trace/orcl_ora_11528.trc  (incident=397027):
ORA-00600: internal error code, arguments: [ktspfpblk:kcbz_objdchk], [0], [0], [1], [], [], [], [], [], [], [], []

Wed Apr 02 23:22:53 2025
Errors in file /home/oracle/app/diag/rdbms/orcl/orcl/trace/orcl_ora_11609.trc  (incident=397082):
ORA-00600: internal error code, arguments: [6002], [6], [189], [1], [0], [], [], [], [], [], [], []

Wed Apr 02 23:26:41 2025
Errors in file /home/oracle/app/diag/rdbms/orcl/orcl/trace/orcl_m000_11966.trc  (incident=397035):
ORA-00600: internal error code, arguments: [dbgrmblur_update_range_1], [11], [6], [], [], [], [], [], [], [], [], []

Wed Apr 02 23:31:47 2025
Errors in file /home/oracle/app/diag/rdbms/orcl/orcl/trace/orcl_j000_10927.trc:
ORA-12012: error on auto execute of job "SYS"."ORA$AT_SA_SPC_SY_49685"
ORA-08102: index key not found, obj# 39, file 1, block 55190 (2)

Thu Apr 03 00:15:18 2025
Exception [type: SIGSEGV, Address not mapped to object] [ADDR:0x8] [PC:0xB9EC41, ksuloget()+421] [flags: 0x0, count: 1]
Errors in file /home/oracle/app/diag/rdbms/orcl/orcl/trace/orcl_m000_12633.trc  (incident=400879):
ORA-07445: exception encountered:core dump [ksuloget()+421][SIGSEGV][ADDR:0x8][PC:0xB9EC41][Address not mapped to object]

Thu Apr 03 00:15:23 2025
Errors in file /home/oracle/app/diag/rdbms/orcl/orcl/trace/orcl_pmon_4097.trc  (incident=396817):
ORA-00600: internal error code, arguments: [1100], [0x2E3947E78], [0x2E3947E78], [], [], [], [], [], [], [], [], []

数据库crash掉之后,处理好硬件环境和虚拟机启动之后,数据库直接启动失败,报ORA-01172 ORA-01151

Beginning crash recovery of 1 threads
Started redo scan
Completed redo scan
 read 29239 KB redo, 4020 data blocks need recovery
Started redo application at
 Thread 1: logseq 211603, block 9107
Recovery of Online Redo Log: Thread 1 Group 4 Seq 211603 Reading mem 0
  Mem# 0: /home/oracle/app/oradata/orcl/redo04.log
  Mem# 1: /home/oracle/app/oradata/orcl/redo041.log
Hex dump of (file 2, block 4835) in trace file /home/oracle/app/diag/rdbms/orcl/orcl/trace/orcl_ora_19174.trc
Reading datafile '/home/oracle/app/oradata/orcl/sysaux01.dbf' for corruption at rdba: 0x008012e3 (file 2, block 4835)
Reread (file 2, block 4835) found same corrupt data (logically corrupt)
RECOVERY OF THREAD 1 STUCK AT BLOCK 4835 OF FILE 2
Aborting crash recovery due to error 1172
Errors in file /home/oracle/app/diag/rdbms/orcl/orcl/trace/orcl_ora_19174.trc:
ORA-01172: recovery of thread 1 stuck at block 4835 of file 2
ORA-01151: use media recovery to recover block, restore backup if needed
Errors in file /home/oracle/app/diag/rdbms/orcl/orcl/trace/orcl_ora_19174.trc:
ORA-01172: recovery of thread 1 stuck at block 4835 of file 2
ORA-01151: use media recovery to recover block, restore backup if needed
ORA-1172 signalled during: ALTER DATABASE OPEN...

然后再次尝试重启提示ORA-01113 ORA-01110

Fri Apr 04 09:34:36 2025
ALTER DATABASE OPEN
Errors in file /home/oracle/app/diag/rdbms/orcl/orcl/trace/orcl_ora_4076.trc:
ORA-01113: file 5 needs media recovery
ORA-01110: data file 5: '/home/oracle/app/oradata/users01.dbf'
ORA-1113 signalled during: ALTER DATABASE OPEN...

可以自行尝试了各种恢复,比如using backup controlfile,until cancel,rectl等操作,数据库均为open成功,基本上都是卡在类似如下报ORA-00310 ORA-00334错

Sat Apr 05 10:17:34 2025
ALTER DATABASE RECOVER database using backup controlfile
Media Recovery Start
started logmerger process
Sat Apr 05 10:17:34 2025
WARNING! Recovering data file 1 from a fuzzy file. If not the current file
it might be an online backup taken without entering the begin backup command.
WARNING! Recovering data file 2 from a fuzzy file. If not the current file
it might be an online backup taken without entering the begin backup command.
WARNING! Recovering data file 3 from a fuzzy file. If not the current file
it might be an online backup taken without entering the begin backup command.
WARNING! Recovering data file 4 from a fuzzy file. If not the current file
it might be an online backup taken without entering the begin backup command.
WARNING! Recovering data file 5 from a fuzzy file. If not the current file
it might be an online backup taken without entering the begin backup command.
WARNING! Recovering data file 6 from a fuzzy file. If not the current file
it might be an online backup taken without entering the begin backup command.
WARNING! Recovering data file 7 from a fuzzy file. If not the current file
it might be an online backup taken without entering the begin backup command.
WARNING! Recovering data file 8 from a fuzzy file. If not the current file
it might be an online backup taken without entering the begin backup command.
WARNING! Recovering data file 9 from a fuzzy file. If not the current file
it might be an online backup taken without entering the begin backup command.
WARNING! Recovering data file 10 from a fuzzy file. If not the current file
it might be an online backup taken without entering the begin backup command.
WARNING! Recovering data file 11 from a fuzzy file. If not the current file
it might be an online backup taken without entering the begin backup command.
WARNING! Recovering data file 12 from a fuzzy file. If not the current file
it might be an online backup taken without entering the begin backup command.
WARNING! Recovering data file 13 from a fuzzy file. If not the current file
it might be an online backup taken without entering the begin backup command.
WARNING! Recovering data file 14 from a fuzzy file. If not the current file
it might be an online backup taken without entering the begin backup command.
WARNING! Recovering data file 15 from a fuzzy file. If not the current file
it might be an online backup taken without entering the begin backup command.
Parallel Media Recovery started with 28 slaves
ORA-279 signalled during: ALTER DATABASE RECOVER database using backup controlfile ...
Sat Apr 05 10:17:59 2025
ALTER DATABASE RECOVER LOGFILE '/home/oradata/redo02.log'
Media Recovery Log /home/oradata/redo02.log
Sat Apr 05 10:17:59 2025
Errors with log /home/oradata/redo02.log
Errors in file /u01/app/oracle/diag/rdbms/oorcl/oorcl/trace/oorcl_pr00_12141.trc:
ORA-00310: archived log contains sequence 211550; sequence 211603 required
ORA-00334: archived log: '/home/oradata/redo02.log'
ORA-310 signalled during: ALTER DATABASE RECOVER LOGFILE '/home/oradata/redo02.log' ...
ALTER DATABASE RECOVER CANCEL
Media Recovery Canceled
Completed: ALTER DATABASE RECOVER CANCEL

基于上述情况,数据库由于底层异常,导致所需要的redo和实际存在的redo文件内容不匹配,只能屏蔽一致性强制打开库

SQL> alter database open resetlogs ;
alter database open resetlogs 
*
ERROR at line 1:
ORA-00603: ORACLE server session terminated by fatal error
ORA-00600: internal error code, arguments: [2662], [0], [1685409503], [0], [1685415469], [12583040], []
ORA-00600: internal error code, arguments: [2662], [0], [1685409502], [0], [1685415469], [12583040], []
ORA-01092: ORACLE instance terminated. Disconnection force
ORA-00600: internal error code, arguments: [2662], [0], [1685409498], [0], [1685415469], [12583040], []
Process ID: 10637
Session ID: 645 Serial number: 7

ORA-600 2662这个错误比较常见,通过修改数据库scn,进行规避然后尝试打开库

Sat Apr 05 10:31:45 2025
alter database open resetlogs
RESETLOGS is being done without consistancy checks. This may result
in a corrupted database. The database should be recreated.
RESETLOGS after incomplete recovery UNTIL CHANGE 1685409495
Resetting resetlogs activation ID 1725417463 (0x66d7c7f7)
Sat Apr 05 10:31:46 2025
Setting recovery target incarnation to 2
Initializing SCN for created control file
Database SCN compatibility initialized to 3
Warning - High Database SCN: Current SCN value is 1685409498, threshold SCN value is 0
Sat Apr 05 10:31:46 2025
Assigning activation ID 1725412798 (0x66d7b5be)
Thread 1 opened at log sequence 1
  Current log# 2 seq# 1 mem# 0: /home/oradata/redo02.log
Successful open of redo thread 1
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Sat Apr 05 10:31:46 2025
SMON: enabling cache recovery
Undo initialization finished serial:0 start:61632504 end:61632514 diff:10 (0 seconds)
Dictionary check beginning
Tablespace 'TEMP' #3 found in data dictionary,
but not in the controlfile. Adding to controlfile.
Dictionary check complete
Verifying file header compatibility for 11g tablespace encryption..
Verifying 11g file header compatibility for tablespace encryption completed
*********************************************************************
WARNING: The following temporary tablespaces contain no files.
         This condition can occur when a backup controlfile has
SMON: enabling tx recovery
         been restored.  It may be necessary to add files to these
         tablespaces.  That can be done using the SQL statement:
 
         ALTER TABLESPACE <tablespace_name> ADD TEMPFILE
 
         Alternatively, if these temporary tablespaces are no longer
         needed, then they can be dropped.
           Empty temporary tablespace: TEMP
*********************************************************************
Database Characterset is AL32UTF8
Errors in file /u01/app/oracle/diag/rdbms/oorcl/oorcl/trace/oorcl_smon_22927.trc  (incident=8145):
ORA-00600: internal error code, arguments: [4137], [9.1.436887], [0], [0], [], [], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/oorcl/oorcl/incident/incdir_8145/oorcl_smon_22927_i8145.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Stopping background process MMNL
ORACLE Instance oorcl (pid = 17) - Error 600 encountered while recovering transaction (9, 1).
Errors in file /u01/app/oracle/diag/rdbms/oorcl/oorcl/trace/oorcl_smon_22927.trc:
ORA-00600: internal error code, arguments: [4137], [9.1.436887], [0], [0], [], [], [], [], [], [], [], []
Sat Apr 05 10:31:46 2025
Sweep [inc][8145]: completed
Errors in file /u01/app/oracle/diag/rdbms/oorcl/oorcl/trace/oorcl_smon_22927.trc  (incident=8146):
ORA-00600: internal error code, arguments: [4137], [9.1.436887], [0], [0], [], [], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/oorcl/oorcl/incident/incdir_8146/oorcl_smon_22927_i8146.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Sat Apr 05 10:31:46 2025
Errors in file /u01/app/oracle/diag/rdbms/oorcl/oorcl/trace/oorcl_p054_2643.trc  (incident=8625):
ORA-00600: internal error code, arguments: [kturbleurec1], [], [], [], [], [], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/oorcl/oorcl/incident/incdir_8625/oorcl_p054_2643_i8625.trc
Sat Apr 05 10:31:46 2025
Errors in file /u01/app/oracle/diag/rdbms/oorcl/oorcl/trace/oorcl_p034_2603.trc  (incident=8465):
ORA-00600: internal error code, arguments: [kturbleurec1], [], [], [], [], [], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/oorcl/oorcl/incident/incdir_8465/oorcl_p034_2603_i8465.trc
replication_dependency_tracking turned off (no async multimaster replication found)
LOGSTDBY: Validating controlfile with logical metadata
LOGSTDBY: Validation complete
Completed: alter database open resetlogs
Sat Apr 05 10:31:48 2025
Starting background process CJQ0
Sat Apr 05 10:31:48 2025
CJQ0 started with pid=80, OS id=2852 
SMON: Restarting fast_start parallel rollback
Errors in file /u01/app/oracle/diag/rdbms/oorcl/oorcl/trace/oorcl_smon_22927.trc  (incident=8147):
ORA-00600: internal error code, arguments: [4137], [9.1.436887], [0], [0], [], [], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/oorcl/oorcl/incident/incdir_8147/oorcl_smon_22927_i8147.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Sat Apr 05 10:31:50 2025
Errors in file /u01/app/oracle/diag/rdbms/oorcl/oorcl/trace/oorcl_p000_2535.trc  (incident=8169):
ORA-00600: internal error code, arguments: [4198], [], [], [], [], [], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/oorcl/oorcl/incident/incdir_8169/oorcl_p000_2535_i8169.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
ORACLE Instance oorcl (pid = 17) - Error 600 encountered while recovering transaction (9, 1).
Block recovery from logseq 1, block 19 to scn 2147483682
Recovery of Online Redo Log: Thread 1 Group 2 Seq 1 Reading mem 0
  Mem# 0: /home/oradata/redo02.log
Block recovery completed at rba 1.734.16, scn 0.2147483683
Block recovery from logseq 1, block 404 to scn 2147483682
Recovery of Online Redo Log: Thread 1 Group 2 Seq 1 Reading mem 0
  Mem# 0: /home/oradata/redo02.log
Block recovery completed at rba 1.734.16, scn 0.2147483683
Errors in file /u01/app/oracle/diag/rdbms/oorcl/oorcl/trace/oorcl_smon_22927.trc  (incident=8148):
ORA-00600: internal error code, arguments: [4198], [], [], [], [], [], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/oorcl/oorcl/incident/incdir_8148/oorcl_smon_22927_i8148.trc
Sat Apr 05 10:31:50 2025
Sweep [inc][8147]: completed
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
SMON: Parallel transaction recovery slave got internal error
SMON: Downgrading transaction recovery to serial
Errors in file /u01/app/oracle/diag/rdbms/oorcl/oorcl/trace/oorcl_smon_22927.trc  (incident=8149):
ORA-00600: internal error code, arguments: [4137], [10.28.1201778], [0], [0], [], [], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/oorcl/oorcl/incident/incdir_8149/oorcl_smon_22927_i8149.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
ORACLE Instance oorcl (pid = 17) - Error 600 encountered while recovering transaction (10, 28).
Errors in file /u01/app/oracle/diag/rdbms/oorcl/oorcl/trace/oorcl_smon_22927.trc:
ORA-00600: internal error code, arguments: [4137], [10.28.1201778], [0], [0], [], [], [], [], [], [], [], []
Sat Apr 05 10:31:50 2025
Sweep [inc][8149]: completed
Checker run found 1 new persistent data failures
Errors in file /u01/app/oracle/diag/rdbms/oorcl/oorcl/trace/oorcl_smon_22927.trc  (incident=8150):
ORA-00600: internal error code, arguments: [4137], [10.28.1201778], [0], [0], [], [], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/oorcl/oorcl/incident/incdir_8150/oorcl_smon_22927_i8150.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
ORACLE Instance oorcl (pid = 17) - Error 600 encountered while recovering transaction (10, 28).
Errors in file /u01/app/oracle/diag/rdbms/oorcl/oorcl/trace/oorcl_smon_22927.trc  (incident=8151):
ORA-00600: internal error code, arguments: [4137], [10.28.1201778], [0], [0], [], [], [], [], [], [], [], []
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Sat Apr 05 10:31:51 2025
Sweep [inc][8150]: completed
ORACLE Instance oorcl (pid = 17) - Error 600 encountered while recovering transaction (10, 28).

虽然数据库open成功，但是有ORA-600 4137/ORA-600 kturbleurec1/ORA-600 4198等错误,但是这里比较明显的undo有问题,对于异常undo进行处理,然后逻辑导出数据,导入新库完成本次恢复任务

差点被误操作的ORA-600 kcratr_nab_less_than_odr故障

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：差点被误操作的ORA-600 kcratr_nab_less_than_odr故障

晚上接到一个客户电话,数据库无法启动,咨询我的Oracle Recovery Tools工具的授权问题,我远程登录客户的环境进行查看故障,发现客户进行了一下操作导致最后open报ORA-01152错误

C:\Users\Administrator>sqlplus sys/sys as sysdba

SQL*Plus: Release 11.2.0.1.0 Production on Tue Mar 18 22:43:24 2025

Copyright (c) 1982, 2010, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> startup mount
ORACLE instance started.

Total System Global Area 1603411968 bytes
Fixed Size                  2176168 bytes
Variable Size             973081432 bytes
Database Buffers          620756992 bytes
Redo Buffers                7397376 bytes
Database mounted.
SQL> recover database;
Media recovery complete.
SQL> alter database open;
alter database open
*
ERROR at line 1:
ORA-00600: internal error code, arguments: [kcratr_nab_less_than_odr], [1],
[27890], [1589], [1649], [], [], [], [], [], [], []

SQL> recover database;
ORA-00283: 恢复会话因错误而取消
ORA-00264: 不要求恢复


SQL> alter database open;
alter database open
*
ERROR at line 1:
ORA-00600: internal error code, arguments: [kcratr_nab_less_than_odr], [1],
[27890], [1589], [1649], [], [], [], [], [], [], []


SQL> recover database until cancel;
ORA-10879: error signaled in parallel recovery slave
ORA-01547: 警告: RECOVER 成功但 OPEN RESETLOGS 将出现如下错误
ORA-01152: 文件 1 没有从过旧的备份中还原
ORA-01110: 数据文件 1: 'D:\APP\ADMINISTRATOR\ORADATA\orcl\SYSTEM01.DBF'

SQL> alter database open resetlogs;
alter database open resetlogs
*
ERROR at line 1:
ORA-01152: file 1 was not restored from a sufficiently old backup
ORA-01110: data file 1: 'D:\APP\ADMINISTRATOR\ORADATA\orcl\SYSTEM01.DBF'


SQL> RECOVER DATABASE USING BACKUP CONTROLFILE;
ORA-00279: change 17384974762395 generated at 03/18/2025 18:30:34 needed for
thread 1
ORA-00289: suggestion :
D:\APP\ADMINISTRATOR\FLASH_RECOVERY_AREA\orcl\ARCHIVELOG\2025_03_19\O1_MF_1_27
890_%U_.ARC
ORA-00280: change 17384974762395 for thread 1 is in sequence #27890


Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
D\APP\ADMINISTRATOR\ORADATA\orcl\RED001.LOG
ORA-00308: 无法打开归档日志 'D\APP\ADMINISTRATOR\ORADATA\orcl\RED001.LOG'
ORA-27041: 无法打开文件
OSD-04002: 无法打开文件
O/S-Error: (OS 3) 系统找不到指定的路径。


Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
D:\app\Administrator\oradata\orcl\RED001.LOG
ORA-00308: 无法打开归档日志 'D:\app\Administrator\oradata\orcl\RED001.LOG'
ORA-27041: 无法打开文件
OSD-04002: 无法打开文件
O/S-Error: (OS 2) 系统找不到指定的文件。

最初数据库启动报ORA-600 kcratr_nab_less_than_odr错误(这个是一个非常典型的错误,早期写过处理方法:ORA-600 kcratr_nab_less_than_odr故障解决)，客户处理故障思路不太清晰,使用了recover database until cancel和alter database open resetlogs等不当操作,导致数据库没有open成功.然后希望使用我的Oracle Recovery Tools小工具进行修复,但是根据我的判断,这个故障还用不上该工具,直接可以open库.对其进行ctl重建并open库

SQL> alter database backup controlfile to trace as 'd:/ctl.txt';

Database altered.

SQL> create pfile='d:/pfile.txt' from spfile;

File created.

SQL> shutdown immediate;
ORA-01109: database not open


Database dismounted.
ORACLE instance shut down.
SQL> startup nomount;
ORACLE instance started.

Total System Global Area 1603411968 bytes
Fixed Size                  2176168 bytes
Variable Size             973081432 bytes
Database Buffers          620756992 bytes
Redo Buffers                7397376 bytes
SQL> @d:/xifenfei/rectl.sql

Control file created.

SQL>
SQL>
SQL> recover database;
Media recovery complete.

SQL> alter database open;

Database altered.

在open之后,数据库报ORA-600 4194错误

Wed Mar 19 01:51:12 2025
alter database open
Beginning crash recovery of 1 threads
 parallel recovery started with 23 processes
Started redo scan
Completed redo scan
 read 793 KB redo, 0 data blocks need recovery
Started redo application at
 Thread 1: logseq 27890, block 2, scn 17384974741793
Recovery of Online Redo Log: Thread 1 Group 4 Seq 27890 Reading mem 0
  Mem# 0: D:\APP\ADMINISTRATOR\ORADATA\orcl\REDO04.LOG
Completed redo application of 0.00MB
Completed crash recovery at
 Thread 1: logseq 27890, block 1589, scn 17384974762395
 0 data blocks read, 0 data blocks written, 793 redo k-bytes read
Wed Mar 19 01:51:14 2025
Thread 1 advanced to log sequence 27891 (thread open)
Thread 1 opened at log sequence 27891
  Current log# 2 seq# 27891 mem# 0: D:\APP\ADMINISTRATOR\ORADATA\orcl\REDO02.LOG
Successful open of redo thread 1
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Wed Mar 19 01:51:14 2025
SMON: enabling cache recovery
Successfully onlined Undo Tablespace 2.
Dictionary check beginning
Tablespace 'TEMP' #3 found in data dictionary,
but not in the controlfile. Adding to controlfile.
Tablespace 'TEMP1' #11 found in data dictionary,
but not in the controlfile. Adding to controlfile.
Dictionary check complete
Verifying file header compatibility for 11g tablespace encryption..
Verifying 11g file header compatibility for tablespace encryption completed
SMON: enabling tx recovery
*********************************************************************
WARNING: The following temporary tablespaces contain no files.
         This condition can occur when a backup controlfile has
         been restored.  It may be necessary to add files to these
         tablespaces.  That can be done using the SQL statement:
 
         ALTER TABLESPACE <tablespace_name> ADD TEMPFILE
 
         Alternatively, if these temporary tablespaces are no longer
         needed, then they can be dropped.
           Empty temporary tablespace: TEMP
           Empty temporary tablespace: TEMP1
*********************************************************************
Database Characterset is ZHS16GBK
No Resource Manager plan active
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_smon_4084.trc  (incident=158542):
ORA-00600: 内部错误代码, 参数: [4194], [], [], [], [], [], [], [], [], [], [], []
Incident details in: d:\app\administrator\diag\rdbms\orcl\orcl\incident\incdir_158542\orcl_smon_4084_i158542.trc
Wed Mar 19 01:51:26 2025
Trace dumping is performing id=[cdmp_20250319015126]
Wed Mar 19 01:51:31 2025
Sweep [inc][158542]: completed
Sweep [inc2][158542]: completed
Wed Mar 19 01:51:35 2025
Doing block recovery for file 3 block 1415
Resuming block recovery (PMON) for file 3 block 1415
Block recovery from logseq 27891, block 86 to scn 17384974782720
Recovery of Online Redo Log: Thread 1 Group 2 Seq 27891 Reading mem 0
  Mem# 0: D:\APP\ADMINISTRATOR\ORADATA\orcl\REDO02.LOG
Block recovery stopped at EOT rba 27891.87.16
Block recovery completed at rba 27891.87.16, scn 4047.3242135803
Doing block recovery for file 3 block 264
Resuming block recovery (PMON) for file 3 block 264
Block recovery from logseq 27891, block 86 to scn 17384974782714
Recovery of Online Redo Log: Thread 1 Group 2 Seq 27891 Reading mem 0
  Mem# 0: D:\APP\ADMINISTRATOR\ORADATA\orcl\REDO02.LOG
Block recovery completed at rba 27891.87.16, scn 4047.3242135803
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_smon_4084.trc:
ORA-01595: 释放区 (2) 回退段 (12) 时出错
ORA-00600: 内部错误代码, 参数: [4194], [], [], [], [], [], [], [], [], [], [], []
Wed Mar 19 01:51:36 2025
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_ora_5680.trc  (incident=158590):
ORA-00600: internal error code, arguments: [4194], [], [], [], [], [], [], [], [], [], [], []
Incident details in: d:\app\administrator\diag\rdbms\orcl\orcl\incident\incdir_158590\orcl_ora_5680_i158590.trc

数据库open成功,但是后台报ORA-01595/ORA-600 4194错误,这个问题比较常见,直接处理异常undo即可恢复.

21c数据库恢复遭遇ora-600 ktugct: corruption detected

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：21c数据库恢复遭遇ora-600 ktugct: corruption detected

接手一个oracle 21c的库恢复请求,通过Oracle数据库异常恢复检查脚本(Oracle Database Recovery Check)脚本检测之后,发现undo文件offline之后,做了resetlogs操作,导致该文件目前处于WRONG RESETLOGS状态
21c
wrong-resetlogs

尝试恢复数据库ORA-16433错误

SQL> recover datafile 1;
ORA-00283: recovery session canceled due to errors
ORA-16433: The database must be opened in read/write mode.

2025-03-11T12:07:43.381296+08:00
ALTER DATABASE RECOVER  datafile 1  
2025-03-11T12:07:43.381296+08:00
Media Recovery Start
2025-03-11T12:07:43.381296+08:00
Media Recovery failed with error 16433
ORA-283 signalled during: ALTER DATABASE RECOVER  datafile 1  ...

官方对于该错误的解释

[oracle@iZbp11c0qyuuo1gr7j98upZ ~]$ oerr ora 16433
16433, 00000, "The database must be opened in read/write mode."
// *Cause:  An attempt was made to open the database in read-only mode after an
//          operation that requires that the database be opened in read/write
//          mode.
// *Action: Open the database in read/write mode. The database can then be
//          opened in read-only mode.

对于该错误,可以通过rectl进行解决,但是由于undo文件的resetlog 信息不对,无法直接重建ctl,这种情况下,通过Oracle Recovery Tools小工具对undo文件的resetlogs信息今星期修改

然后重建ctl成功

C:\Users\XFF>sqlplus / as sysdba

SQL*Plus: Release 21.0.0.0.0 - Production on 星期二 3月 11 12:15:00 2025
Version 21.3.0.0.0

Copyright (c) 1982, 2021, Oracle.  All rights reserved.


连接到:
Oracle Database 21c Enterprise Edition Release 21.0.0.0.0 - Production
Version 21.3.0.0.0

SQL> CREATE CONTROLFILE REUSE DATABASE "ORCLCDB" NORESETLOGS  NOARCHIVELOG
  2         MAXLOGFILES 50
  3         MAXLOGMEMBERS 5
  4         MAXDATAFILES 5000
  5         MAXINSTANCES 8
  6         MAXLOGHISTORY 2920
  7  LOGFILE
  8         group 1   'H:\TEMP\ORCLCDB\REDO01.LOG' size 200M,
  9         group 3   'H:\TEMP\ORCLCDB\REDO03.LOG' size 200M,
 10         group 2   'H:\TEMP\ORCLCDB\REDO02.LOG' size 200M
 11  DATAFILE
 12          'H:\TEMP\ORCLCDB\SYSTEM01.DBF',
 13          'H:\TEMP\ORCLCDB\SYSAUX01.DBF',
 14          'H:\TEMP\ORCLCDB\UNDOTBS01.DBF',
 15          'H:\TEMP\ORCLCDB\PDBSEED\SYSTEM01.DBF',
 16          'H:\TEMP\ORCLCDB\PDBSEED\SYSAUX01.DBF',
 17          'H:\TEMP\ORCLCDB\USERS01.DBF',
 18          'H:\TEMP\ORCLCDB\PDBSEED\UNDOTBS01.DBF',
 19          'H:\TEMP\ORCLCDB\ORCLPDB1\SYSTEM01.DBF',
 20          'H:\TEMP\ORCLCDB\ORCLPDB1\UNDOTBS01.DBF',
 21          'H:\TEMP\ORCLCDB\ORCLPDB1\USERS01.DBF'
 22  CHARACTER SET  AL32UTF8 ;

控制文件已创建。

尝试open库报ORA-600 ktugct: corruption detected错误

SQL> recover database;
完成介质恢复。
SQL> alter database open;
alter database open
*
第 1 行出现错误:
ORA-00603: ORACLE server session terminated by fatal error
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-00600: internal error code, arguments: [ktugct: corruption detected], [],
[], [], [], [], [], [], [], [], [], []
进程 ID: 8372
会话 ID: 978 序列号: 21772

通过对启动过程进行分析,确认是在seq$表访问的时候报错

PARSING IN CURSOR #2238117495280 len=102 dep=1 uid=0 oct=3 lid=0 tim=821317050254 
 hv=3967354608 ad='7ff9048c41b8' sqlid='axmdf8vq7k1rh'
select increment$,minvalue,maxvalue,cycle#,order$,cache,highwater,audit$,flags from seq$ where obj#=:1
END OF STMT
PARSE #2238117495280:c=6253,e=6253,p=5,cr=90,cu=0,mis=1,r=0,dep=1,og=4,plh=0,tim=821317050254
BINDS #2238117156016:

 Bind#0
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=00 fl2=1000001 frm=00 csi=00 siz=48 off=0
  kxsbbbfp=1a33f610  bln=22  avl=02  flg=05
  value=100
 Bind#1
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=00 fl2=1000001 frm=00 csi=00 siz=0 off=24
  kxsbbbfp=1a33f628  bln=22  avl=02  flg=01
  value=1
EXEC #2238117156016:c=95,e=95,p=0,cr=0,cu=0,mis=0,r=0,dep=2,og=3,plh=2239883476,tim=821317050580
FETCH #2238117156016:c=8,e=8,p=0,cr=3,cu=0,mis=0,r=1,dep=2,og=3,plh=2239883476,tim=821317050600
CLOSE #2238117156016:c=3,e=3,dep=2,type=3,tim=821317050619
BINDS #2238117156016:

 Bind#0
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=00 fl2=1000001 frm=00 csi=00 siz=48 off=0
  kxsbbbfp=1a33f610  bln=22  avl=02  flg=05
  value=100
 Bind#1
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=00 fl2=1000001 frm=00 csi=00 siz=0 off=24
  kxsbbbfp=1a33f628  bln=22  avl=02  flg=01
  value=2
EXEC #2238117156016:c=87,e=87,p=0,cr=0,cu=0,mis=0,r=0,dep=2,og=3,plh=2239883476,tim=821317050746
FETCH #2238117156016:c=7,e=7,p=0,cr=3,cu=0,mis=0,r=1,dep=2,og=3,plh=2239883476,tim=821317050764
CLOSE #2238117156016:c=3,e=3,dep=2,type=3,tim=821317050781
BINDS #2238117156016:

 Bind#0
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=00 fl2=1000001 frm=00 csi=00 siz=48 off=0
  kxsbbbfp=1a33f610  bln=22  avl=02  flg=05
  value=100
 Bind#1
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=00 fl2=1000001 frm=00 csi=00 siz=0 off=24
  kxsbbbfp=1a33f628  bln=22  avl=02  flg=01
  value=3
EXEC #2238117156016:c=84,e=84,p=0,cr=0,cu=0,mis=0,r=0,dep=2,og=3,plh=2239883476,tim=821317050885
FETCH #2238117156016:c=6,e=6,p=0,cr=3,cu=0,mis=0,r=1,dep=2,og=3,plh=2239883476,tim=821317050902
CLOSE #2238117156016:c=2,e=2,dep=2,type=3,tim=821317050917
BINDS #2238117156016:

 Bind#0
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=00 fl2=1000001 frm=00 csi=00 siz=48 off=0
  kxsbbbfp=1a33f610  bln=22  avl=02  flg=05
  value=100
 Bind#1
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=00 fl2=1000001 frm=00 csi=00 siz=0 off=24
  kxsbbbfp=1a33f628  bln=22  avl=02  flg=01
  value=4
EXEC #2238117156016:c=83,e=83,p=0,cr=0,cu=0,mis=0,r=0,dep=2,og=3,plh=2239883476,tim=821317051019
FETCH #2238117156016:c=5,e=5,p=0,cr=3,cu=0,mis=0,r=1,dep=2,og=3,plh=2239883476,tim=821317051035
CLOSE #2238117156016:c=3,e=2,dep=2,type=3,tim=821317051050
BINDS #2238117156016:

 Bind#0
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=00 fl2=1000001 frm=00 csi=00 siz=48 off=0
  kxsbbbfp=1a33f610  bln=22  avl=02  flg=05
  value=100
 Bind#1
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=00 fl2=1000001 frm=00 csi=00 siz=0 off=24
  kxsbbbfp=1a33f628  bln=22  avl=02  flg=01
  value=5
EXEC #2238117156016:c=88,e=88,p=0,cr=0,cu=0,mis=0,r=0,dep=2,og=3,plh=2239883476,tim=821317051158
FETCH #2238117156016:c=5,e=5,p=0,cr=3,cu=0,mis=0,r=1,dep=2,og=3,plh=2239883476,tim=821317051174
CLOSE #2238117156016:c=2,e=2,dep=2,type=3,tim=821317051190
BINDS #2238117156016:

 Bind#0
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=00 fl2=1000001 frm=00 csi=00 siz=48 off=0
  kxsbbbfp=1a33f610  bln=22  avl=02  flg=05
  value=100
 Bind#1
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=00 fl2=1000001 frm=00 csi=00 siz=0 off=24
  kxsbbbfp=1a33f628  bln=22  avl=02  flg=01
  value=6
EXEC #2238117156016:c=83,e=83,p=0,cr=0,cu=0,mis=0,r=0,dep=2,og=3,plh=2239883476,tim=821317051292
FETCH #2238117156016:c=5,e=6,p=0,cr=3,cu=0,mis=0,r=1,dep=2,og=3,plh=2239883476,tim=821317051308
CLOSE #2238117156016:c=2,e=2,dep=2,type=3,tim=821317051323
BINDS #2238117156016:

 Bind#0
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=00 fl2=1000001 frm=00 csi=00 siz=48 off=0
  kxsbbbfp=1a33f610  bln=22  avl=02  flg=05
  value=100
 Bind#1
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=00 fl2=1000001 frm=00 csi=00 siz=0 off=24
  kxsbbbfp=1a33f628  bln=22  avl=02  flg=01
  value=7
EXEC #2238117156016:c=83,e=83,p=0,cr=0,cu=0,mis=0,r=0,dep=2,og=3,plh=2239883476,tim=821317051425
FETCH #2238117156016:c=5,e=5,p=0,cr=3,cu=0,mis=0,r=1,dep=2,og=3,plh=2239883476,tim=821317051440
CLOSE #2238117156016:c=3,e=3,dep=2,type=3,tim=821317051456
BINDS #2238117156016:

 Bind#0
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=00 fl2=1000001 frm=00 csi=00 siz=48 off=0
  kxsbbbfp=1a33f610  bln=22  avl=02  flg=05
  value=100
 Bind#1
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=00 fl2=1000001 frm=00 csi=00 siz=0 off=24
  kxsbbbfp=1a33f628  bln=22  avl=02  flg=01
  value=8
EXEC #2238117156016:c=83,e=83,p=0,cr=0,cu=0,mis=0,r=0,dep=2,og=3,plh=2239883476,tim=821317051558
FETCH #2238117156016:c=5,e=5,p=0,cr=3,cu=0,mis=0,r=1,dep=2,og=3,plh=2239883476,tim=821317051573
CLOSE #2238117156016:c=2,e=2,dep=2,type=3,tim=821317051588
BINDS #2238117156016:

 Bind#0
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=00 fl2=1000001 frm=00 csi=00 siz=48 off=0
  kxsbbbfp=1a33f610  bln=22  avl=02  flg=05
  value=100
 Bind#1
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=00 fl2=1000001 frm=00 csi=00 siz=0 off=24
  kxsbbbfp=1a33f628  bln=22  avl=02  flg=01
  value=9
EXEC #2238117156016:c=82,e=82,p=0,cr=0,cu=0,mis=0,r=0,dep=2,og=3,plh=2239883476,tim=821317051688
FETCH #2238117156016:c=5,e=5,p=0,cr=3,cu=0,mis=0,r=1,dep=2,og=3,plh=2239883476,tim=821317051704
CLOSE #2238117156016:c=2,e=2,dep=2,type=3,tim=821317051719
BINDS #2238117156016:

 Bind#0
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=00 fl2=1000001 frm=00 csi=00 siz=48 off=0
  kxsbbbfp=1a33f610  bln=22  avl=02  flg=05
  value=100
 Bind#1
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=00 fl2=1000001 frm=00 csi=00 siz=0 off=24
  kxsbbbfp=1a33f628  bln=22  avl=02  flg=01
  value=10
EXEC #2238117156016:c=82,e=82,p=0,cr=0,cu=0,mis=0,r=0,dep=2,og=3,plh=2239883476,tim=821317051821
FETCH #2238117156016:c=5,e=5,p=0,cr=3,cu=0,mis=0,r=1,dep=2,og=3,plh=2239883476,tim=821317051837
CLOSE #2238117156016:c=2,e=2,dep=2,type=3,tim=821317051852
BINDS #2238117508144:

 Bind#0
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=00 fl2=1000001 frm=00 csi=00 siz=72 off=0
  kxsbbbfp=1a33f5f8  bln=22  avl=02  flg=05
  value=100
 Bind#1
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=00 fl2=1000001 frm=00 csi=00 siz=0 off=24
  kxsbbbfp=1a33f610  bln=22  avl=02  flg=01
  value=10
 Bind#2
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=00 fl2=1000001 frm=00 csi=00 siz=0 off=48
  kxsbbbfp=1a33f628  bln=22  avl=01  flg=01
  value=0
EXEC #2238117508144:c=144,e=144,p=0,cr=0,cu=0,mis=0,r=0,dep=2,og=3,plh=3312420081,tim=821317052013
FETCH #2238117508144:c=164,e=164,p=1,cr=3,cu=0,mis=0,r=3,dep=2,og=3,plh=3312420081,tim=821317052189
CLOSE #2238117508144:c=4,e=4,dep=2,type=3,tim=821317052208
BINDS #2238117495280:

 Bind#0
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=00 fl2=1000001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=1a33f628  bln=22  avl=03  flg=05
  value=1749
EXEC #2238117495280:c=2211,e=2211,p=1,cr=33,cu=0,mis=1,r=0,dep=1,og=4,plh=2203911306,tim=821317052521
WAIT #2238117495280: nam='db file sequential read' ela= 82 file#=1 block#=1201 blocks=1 obj#=105 tim=821317052620
WAIT #2238117495280: nam='db file sequential read' ela= 78 file#=1 block#=1202 blocks=1 obj#=105 tim=821317052723
WAIT #2238117495280: nam='db file sequential read' ela= 77 file#=1 block#=1161 blocks=1 obj#=100 tim=821317052824
ktuisc detected lost changes to transaction table:
 xid of interest:  xid: 0x0003.00c.0006e788 xid from block :  xid: 0x0003.00c.0006e785
2025-03-11T12:18:43.210707+08:00
Incident 25673 created, dump file: C:\APP\XFF\diag\rdbms\orclcdb\o21c\incident\incdir_25673\o21c_ora_12976_i25673.trc
ORA-00600: 内部错误代码, 参数: [ktugct: corruption detected], [], [], [], [], [], [], [], [], [], [], []

FETCH #2238117495280:c=1350284,e=1359602,p=3,cr=4,cu=0,mis=0,r=0,dep=1,og=4,plh=2203911306,tim=821318412135
STAT #2238117495280 id=1 cnt=0 pid=0 pos=1 obj=100 
  op='TABLE ACCESS BY INDEX ROWID SEQ$ (cr=0 pr=0 pw=0 str=1 time=1 us cost=2 size=73 card=1)'
STAT #2238117495280 id=2 cnt=1 pid=1 pos=1 obj=105 
  op='INDEX UNIQUE SCAN I_SEQ1 (cr=2 pr=2 pw=0 str=1 time=209 us cost=1 size=0 card=1)'
<error barrier> at 0x0000003197FF21D8 placed dbsdrv.c@5035
ORA-00600: 内部错误代码, 参数: [ktugct: corruption detected], [], [], [], [], [], [], [], [], [], [], []
<error barrier> at 0x0000003197FF21D8 placed dbsdrv.c@5035
ORA-00600: 内部错误代码, 参数: [ktugct: corruption detected], [], [], [], [], [], [], [], [], [], [], []
2025-03-11T12:18:44.572180+08:00
Incident 25674 created, dump file: C:\APP\XFF\diag\rdbms\orclcdb\o21c\incident\incdir_25674\o21c_ora_12976_i25674.trc
ORA-00603: ORACLE 服务器会话因致命错误而终止
ORA-01092: ORACLE 实例终止。强制断开连接
ORA-00600: 内部错误代码, 参数: [ktugct: corruption detected], [], [], [], [], [], [], [], [], [], [], []

进一步分析trace信息

[TOC00001]
ORA-00600: 内部错误代码, 参数: [ktugct: corruption detected], [], [], [], [], [], [], [], [], [], [], []

[TOC00001-END]
[TOC00002]
========= Dump for incident 25673 (ORA 600 [ktugct: corruption detected]) ========
[TOC00003]
----- Beginning of Customized Incident Dump(s) -----
Corruption or lost changes detected in transaction table
Cleanout struct - flg: 0x00000000 ne = 1 cu: 0 cv: 1 cec:0 actcnt:0
xids being resolved:
 xid: 0x0003.00c.0006e788 sit: 2
inverse itl idx to cleanout entry mapping:
Dump of memory from 0x000002091036B32A to 0x000002091036B429

Block cleanout record, scn: 0xffffffffffffffff ver: 0x01 opt: 0x01
, entries follow...
-- Start of buffer-cache dump for undo segment header --
Dump of buffer cache at level 10 for pdb=1 tsn=2 rdba=16777376
BH (0x7ffaccf5e9e0) file#: 4 rdba: 0x010000a0 (4/160) class: 21 ba: 0x7ffacc17c000
  set: 57 pool: 3 bsz: 8192 bsi: 0 sflg: 2 pwc: 0,0
  dbwrid: 0 obj: -1 objn: 0 tsn: [1/2] afn: 4 hint: f
  hash: [0x7ff91759e230,0x7ff91759e230] lru: [0x7ffaccf5ec30,0x7ffaccf5e960]
  ckptq: [NULL] fileq: [NULL]
  objq: [0x7ff9041e1708,0x7ff9041e1708] objaq: [0x7ff9041e16f8,0x7ff9041e16f8] qhead: 0x7ff9041e16e8
  st: XCURRENT md: NULL fpin: 'ktuwh117: ktugus:kturec' fscn: 0x1a321837 tch: 1
  flags:
  LRBA: [0x0.0.0] LSCN: [0x0] HSCN: [0x0] HSUB: [65535] 
  Printing buffer operation history (latest change first): (cnt 10)
  01. sid:14 L353:gcur:set:MEXCL      02. sid:14 L145:zib:mk:EXCL       
  03. sid:14 L212:zib:bic:FSQ         04. sid:14 L122:zgb:set:st        
  05. sid:14 L830:olq1:clr:WRT+CKT    06. sid:14 L951:zgb:lnk:objq      
  07. sid:14 L372:zgb:set:MEXCL       08. sid:14 L123:zgb:no:FEN        
  09. sid:14 L083:zgb:ent:fn          10. sid:00 L203:w_ini_dc:bic:FVB  
  buffer tsn: 2 rdba: 0x010000a0 (4/160)
  scn: 0x192ea609 seq: 0x01 flg: 0x04 tail: 0xa6092601
  frmt: 0x02 chkval: 0x3395 type: 0x26=KTU SMU HEADER BLOCK
Hex dump of block: st=0, typ_found=1
Dump of memory from 0x00007FFACC17C000 to 0x00007FFACC17E000
7FFACC17C000 0000A226 010000A0 192EA609 04010000  [&...............]
7FFACC17C010 00003395 00000000 00000000 00000000  [.3..............]
7FFACC17C020 00000000 00000004 0000010F 00000FF0  [................]
7FFACC17C030 00000000 00000000 00000007 010000A1  [................]
7FFACC17C040 00000000 00000000 00000000 00000000  [................]
7FFACC17C050 00000000 00000000 00000000 00000004  [................]
7FFACC17C060 00000000 00000000 40000000 010000A1  [...........@....]
7FFACC17C070 00000007 010000B8 00000008 01000780  [................]
7FFACC17C080 00000080 01000180 00000080 00000000  [................]
7FFACC17C090 00000000 00000000 00000000 00000000  [................]
        Repeat 246 times
7FFACC17D000 00000000 67770DDD 6776D1BB 6776EEC1  [......wg..vg..vg]
7FFACC17D010 67770DDD 6775F337 6775F337 6775F34B  [..wg7.ug7.ugK.ug]
7FFACC17D020 67734C0B 676F7532 676CB861 6734082B  [.Lsg2uoga.lg+.4g]
7FFACC17D030 6734082B 671AD100 671AD100 671AD100  [+.4g...g...g...g]
7FFACC17D040 671AD100 671AA6DC 67014F31 67014F31  [...g...g1O.g1O.g]
7FFACC17D050 67014F31 67014F31 67014F31 62BB38EE  [1O.g1O.g1O.g.8.b]
7FFACC17D060 62BB38EE 62BB38EE 62BB38EE 62BB38EE  [.8.b.8.b.8.b.8.b]
        Repeat 1 times
7FFACC17D080 62BB38EE 62BB38EE 62BB0EE3 62BB0EE3  [.8.b.8.b...b...b]
7FFACC17D090 62BB0EE3 62BB0EE3 62BB0EE3 62BB0EE3  [...b...b...b...b]
        Repeat 14 times
7FFACC17D180 62BB0EE3 00000000 00000000 00000000  [...b............]
7FFACC17D190 00000000 00000000 00000000 00000000  [................]
        Repeat 99 times
7FFACC17D7D0 192EA31E 00000000 010000A1 001E3150  [............P1..]
7FFACC17D7E0 31500001 00000001 00000000 00080004  [..P1............]
7FFACC17D7F0 0001B000 00000068 7FFFFFFE 010000A1  [....h...........]
7FFACC17D800 001E3150 055E0000 00000000 0007314E  [P1....^.....N1..]
7FFACC17D810 0CEC0002 00000000 000B3146 1A180002  [........F1......]
7FFACC17D820 00000000 00173103 12B40002 00000000  [.....1..........]
7FFACC17D830 00000000 00000000 0006E7A0 010000A1  [................]
7FFACC17D840 192EA53E 00000000 00100009 00000000  [>...............]
7FFACC17D850 00000000 00000000 00000001 67770DEE  [..............wg]
7FFACC17D860 0006E7AF 010000A1 192EA4E4 00000000  [................]
7FFACC17D870 000F0009 00000000 00000000 00000000  [................]
7FFACC17D880 00000001 67770DE8 0006E790 010000A1  [......wg........]
7FFACC17D890 192EA58D 00000000 001F0009 00000000  [................]
7FFACC17D8A0 00000000 00000000 00000001 67770DF3  [..............wg]
7FFACC17D8B0 0006E797 010001FF 192EA431 00000000  [........1.......]
7FFACC17D8C0 000B0009 00000000 00000000 00000000  [................]
7FFACC17D8D0 00000001 67770DDC 0006E795 010001FF  [......wg........]
7FFACC17D8E0 192EA33D 00000000 000D0009 00000000  [=...............]
7FFACC17D8F0 00000000 00000000 00000001 67770DCC  [..............wg]
7FFACC17D900 0006E79A 010000A1 192EA4D1 00000000  [................]
7FFACC17D910 00160009 00000000 00000000 00000000  [................]
7FFACC17D920 00000001 67770DE6 0006E792 010000A1  [......wg........]
7FFACC17D930 192EA472 00000000 000A0009 00000000  [r...............]
7FFACC17D940 00000000 00000000 00000001 67770DE0  [..............wg]
7FFACC17D950 0006E7A6 010001FF 192EA3FB 00000000  [................]
7FFACC17D960 00090009 00000000 00000000 00000000  [................]
7FFACC17D970 00000001 67770DD8 0006E7A7 010000A1  [......wg........]
7FFACC17D980 192EA609 00000000 FFFF0009 00000000  [................]
7FFACC17D990 00000000 00000000 00000001 67770DFB  [..............wg]
7FFACC17D9A0 0006E79C 010001FF 192EA41A 00000000  [................]
7FFACC17D9B0 00030009 00000000 00000000 00000000  [................]
7FFACC17D9C0 00000001 67770DDA 0006E79E 010000A1  [......wg........]
7FFACC17D9D0 192EA481 00000000 00150009 00000000  [................]
7FFACC17D9E0 00000000 00000000 00000001 67770DE1  [..............wg]
7FFACC17D9F0 0006E7A3 010000A1 192EA44C 00000000  [........L.......]
7FFACC17DA00 00190009 00000000 00000000 00000000  [................]
7FFACC17DA10 00000002 67770DDD 0006E785 010001FF  [......wg........]
7FFACC17DA20 192EA391 00000000 00180009 00000000  [................]
7FFACC17DA30 00000000 00000000 00000001 67770DD1  [..............wg]
7FFACC17DA40 0006E7AB 010001FF 192EA355 00000000  [........U.......]
7FFACC17DA50 001A0009 00000000 00000000 00000000  [................]
7FFACC17DA60 00000001 67770DCD 0006E793 010000A1  [......wg........]
7FFACC17DA70 192EA51F 00000000 00000009 00000000  [................]
7FFACC17DA80 00000000 00000000 00000001 67770DEC  [..............wg]
7FFACC17DA90 0006E7AD 010000A1 192EA4FA 00000000  [................]
7FFACC17DAA0 000E0009 00000000 00000000 00000000  [................]
7FFACC17DAB0 00000001 67770DE9 0006E794 010000A1  [......wg........]
7FFACC17DAC0 192EA550 00000000 00130009 00000000  [P...............]
7FFACC17DAD0 00000000 00000000 00000001 67770DEF  [..............wg]
7FFACC17DAE0 0006E792 010000A1 192EA5DF 00000000  [................]
7FFACC17DAF0 00210009 00000000 00000000 00000000  [..!.............]
7FFACC17DB00 00000001 67770DF9 0006E766 010000A1  [......wgf.......]
7FFACC17DB10 192EA5C2 00000000 001E0009 00000000  [................]
7FFACC17DB20 00000000 00000000 00000001 67770DF7  [..............wg]
7FFACC17DB30 0006E787 010000A1 192EA561 00000000  [........a.......]
7FFACC17DB40 001C0009 00000000 00000000 00000000  [................]
7FFACC17DB50 00000001 67770DF0 0006E7A3 010001FF  [......wg........]
7FFACC17DB60 192EA3D2 00000000 00070009 00000000  [................]
7FFACC17DB70 00000000 00000000 00000001 67770DD6  [..............wg]
7FFACC17DB80 0006E7A5 010000A1 192EA498 00000000  [................]
7FFACC17DB90 00200009 00000000 00000000 00000000  [.. .............]
7FFACC17DBA0 00000001 67770DE2 0006E791 010000A1  [......wg........]
7FFACC17DBB0 192EA4DD 00000000 00010009 00000000  [................]
7FFACC17DBC0 00000000 00000000 00000001 67770DE7  [..............wg]
7FFACC17DBD0 0006E79F 010000A1 192EA5F6 00000000  [................]
7FFACC17DBE0 00080009 00000000 00000000 00000000  [................]
7FFACC17DBF0 00000001 67770DFA 0006E78A 010001FF  [......wg........]
7FFACC17DC00 192EA3A7 00000000 001D0009 00000000  [................]
7FFACC17DC10 00000000 00000000 00000001 67770DD3  [..............wg]
7FFACC17DC20 0006E797 010000A1 192EA45D 00000000  [........].......]
7FFACC17DC30 00060009 00000000 00000000 00000000  [................]
7FFACC17DC40 00000001 67770DDF 0006E796 010001FF  [......wg........]
7FFACC17DC50 192EA368 00000000 001B0009 00000000  [h...............]
7FFACC17DC60 00000000 00000000 00000001 67770DCF  [..............wg]
7FFACC17DC70 0006E78E 010001FF 192EA37D 00000000  [........}.......]
7FFACC17DC80 000C0009 00000000 00000000 00000000  [................]
7FFACC17DC90 00000001 67770DD0 0006E798 010000A1  [......wg........]
7FFACC17DCA0 192EA56F 00000000 00020009 00000000  [o...............]
7FFACC17DCB0 00000000 00000000 00000001 67770DF1  [..............wg]
7FFACC17DCC0 0006E7A1 010001FF 192EA3B5 00000000  [................]
7FFACC17DCD0 00140009 00000000 00000000 00000000  [................]
7FFACC17DCE0 00000001 67770DD4 0006E7A2 010000A1  [......wg........]
7FFACC17DCF0 192EA5CE 00000000 00110009 00000000  [................]
7FFACC17DD00 00000000 00000000 00000001 67770DF7  [..............wg]
7FFACC17DD10 0006E794 010000A1 192EA5A0 00000000  [................]
7FFACC17DD20 00120009 00000000 00000000 00000000  [................]
7FFACC17DD30 00000001 67770DF4 0006E7A2 010000A1  [......wg........]
7FFACC17DD40 192EA4A5 00000000 00050009 00000000  [................]
7FFACC17DD50 00000000 00000000 00000001 67770DE3  [..............wg]
7FFACC17DD60 0006E7A5 010000A1 192EA5E9 00000000  [................]
7FFACC17DD70 00170009 00000000 00000000 00000000  [................]
7FFACC17DD80 00000001 67770DF9 00000003 00000000  [......wg........]
7FFACC17DD90 00000000 00000000 00000000 00000000  [................]
        Repeat 1 times
7FFACC17DDB0 00000001 00000000 00000000 00000000  [................]
        Repeat 33 times
7FFACC17DFD0 00000000 00000000 00000000 00000000  [................]
        Repeat 1 times
7FFACC17DFF0 00000000 00000000 00000000 A6092601  [.............&..]
  Extent Control Header
  -----------------------------------------------------------------
  Extent Header:: spare1: 0      spare2: 0      #extents: 4      #blocks: 271   
                  last map  0x00000000  #maps: 0      offset: 4080  
      Highwater::  0x010000a1  ext#: 0      blk#: 0      ext size: 7     
  #blocks in seg. hdr's freelists: 0     
  #blocks below: 0     
  mapblk  0x00000000  offset: 0     
                   Unlocked
     Map Header:: next  0x00000000  #extents: 4    obj#: 0      flag: 0x40000000
  Extent Map
  -----------------------------------------------------------------
   0x010000a1  length: 7     
   0x010000b8  length: 8     
   0x01000780  length: 128   
   0x01000180  length: 128   
  
 Retention Table 
  -----------------------------------------------------------
 Extent Number:0  Commit Time: 1735855581
 Extent Number:1  Commit Time: 1735840187
 Extent Number:2  Commit Time: 1735847617
 Extent Number:3  Commit Time: 1735855581
  
  TRN CTL:: seq: 0x3150 chd: 0x0004 ctl: 0x0008 inc: 0x00000000 nfb: 0x0001
            mgc: 0xb000 xts: 0x0068 flg: 0x0001 opt: 2147483646 (0x7ffffffe)
            uba: 0x010000a1.3150.1e scn: 0x00000000192ea31e
Version: 0x01
  FREE BLOCK POOL::
    uba: 0x010000a1.3150.1e ext: 0x0  spc: 0x55e   
    uba: 0x00000000.314e.07 ext: 0x2  spc: 0xcec   
    uba: 0x00000000.3146.0b ext: 0x2  spc: 0x1a18  
    uba: 0x00000000.3103.17 ext: 0x2  spc: 0x12b4  
    uba: 0x00000000.0000.00 ext: 0x0  spc: 0x0     
  TRN TBL::
 
  index  state cflags  wrap#    uel         scn            dba            parent-xid    nub     stmt_num    cmt
  -----------------------------------------------------------------------------------------
   0x00  9 0x00  0x6e7a0  0x0010  0x00000000192ea53e  0x010000a1 0x0000.000.00000000  0x00000001 0x00000000 1735855598
   0x01  9 0x00  0x6e7af  0x000f  0x00000000192ea4e4  0x010000a1 0x0000.000.00000000  0x00000001 0x00000000 1735855592
   0x02  9 0x00  0x6e790  0x001f  0x00000000192ea58d  0x010000a1 0x0000.000.00000000  0x00000001 0x00000000 1735855603
   0x03  9 0x00  0x6e797  0x000b  0x00000000192ea431  0x010001ff 0x0000.000.00000000  0x00000001 0x00000000 1735855580
   0x04  9 0x00  0x6e795  0x000d  0x00000000192ea33d  0x010001ff 0x0000.000.00000000  0x00000001 0x00000000 1735855564
   0x05  9 0x00  0x6e79a  0x0016  0x00000000192ea4d1  0x010000a1 0x0000.000.00000000  0x00000001 0x00000000 1735855590
   0x06  9 0x00  0x6e792  0x000a  0x00000000192ea472  0x010000a1 0x0000.000.00000000  0x00000001 0x00000000 1735855584
   0x07  9 0x00  0x6e7a6  0x0009  0x00000000192ea3fb  0x010001ff 0x0000.000.00000000  0x00000001 0x00000000 1735855576
   0x08  9 0x00  0x6e7a7  0xffff  0x00000000192ea609  0x010000a1 0x0000.000.00000000  0x00000001 0x00000000 1735855611
   0x09  9 0x00  0x6e79c  0x0003  0x00000000192ea41a  0x010001ff 0x0000.000.00000000  0x00000001 0x00000000 1735855578
   0x0a  9 0x00  0x6e79e  0x0015  0x00000000192ea481  0x010000a1 0x0000.000.00000000  0x00000001 0x00000000 1735855585
   0x0b  9 0x00  0x6e7a3  0x0019  0x00000000192ea44c  0x010000a1 0x0000.000.00000000  0x00000002 0x00000000 1735855581
   0x0c  9 0x00  0x6e785  0x0018  0x00000000192ea391  0x010001ff 0x0000.000.00000000  0x00000001 0x00000000 1735855569
   0x0d  9 0x00  0x6e7ab  0x001a  0x00000000192ea355  0x010001ff 0x0000.000.00000000  0x00000001 0x00000000 1735855565
   0x0e  9 0x00  0x6e793  0x0000  0x00000000192ea51f  0x010000a1 0x0000.000.00000000  0x00000001 0x00000000 1735855596
   0x0f  9 0x00  0x6e7ad  0x000e  0x00000000192ea4fa  0x010000a1 0x0000.000.00000000  0x00000001 0x00000000 1735855593
   0x10  9 0x00  0x6e794  0x0013  0x00000000192ea550  0x010000a1 0x0000.000.00000000  0x00000001 0x00000000 1735855599
   0x11  9 0x00  0x6e792  0x0021  0x00000000192ea5df  0x010000a1 0x0000.000.00000000  0x00000001 0x00000000 1735855609
   0x12  9 0x00  0x6e766  0x001e  0x00000000192ea5c2  0x010000a1 0x0000.000.00000000  0x00000001 0x00000000 1735855607
   0x13  9 0x00  0x6e787  0x001c  0x00000000192ea561  0x010000a1 0x0000.000.00000000  0x00000001 0x00000000 1735855600
   0x14  9 0x00  0x6e7a3  0x0007  0x00000000192ea3d2  0x010001ff 0x0000.000.00000000  0x00000001 0x00000000 1735855574
   0x15  9 0x00  0x6e7a5  0x0020  0x00000000192ea498  0x010000a1 0x0000.000.00000000  0x00000001 0x00000000 1735855586
   0x16  9 0x00  0x6e791  0x0001  0x00000000192ea4dd  0x010000a1 0x0000.000.00000000  0x00000001 0x00000000 1735855591
   0x17  9 0x00  0x6e79f  0x0008  0x00000000192ea5f6  0x010000a1 0x0000.000.00000000  0x00000001 0x00000000 1735855610
   0x18  9 0x00  0x6e78a  0x001d  0x00000000192ea3a7  0x010001ff 0x0000.000.00000000  0x00000001 0x00000000 1735855571
   0x19  9 0x00  0x6e797  0x0006  0x00000000192ea45d  0x010000a1 0x0000.000.00000000  0x00000001 0x00000000 1735855583
   0x1a  9 0x00  0x6e796  0x001b  0x00000000192ea368  0x010001ff 0x0000.000.00000000  0x00000001 0x00000000 1735855567
   0x1b  9 0x00  0x6e78e  0x000c  0x00000000192ea37d  0x010001ff 0x0000.000.00000000  0x00000001 0x00000000 1735855568
   0x1c  9 0x00  0x6e798  0x0002  0x00000000192ea56f  0x010000a1 0x0000.000.00000000  0x00000001 0x00000000 1735855601
   0x1d  9 0x00  0x6e7a1  0x0014  0x00000000192ea3b5  0x010001ff 0x0000.000.00000000  0x00000001 0x00000000 1735855572
   0x1e  9 0x00  0x6e7a2  0x0011  0x00000000192ea5ce  0x010000a1 0x0000.000.00000000  0x00000001 0x00000000 1735855607
   0x1f  9 0x00  0x6e794  0x0012  0x00000000192ea5a0  0x010000a1 0x0000.000.00000000  0x00000001 0x00000000 1735855604
   0x20  9 0x00  0x6e7a2  0x0005  0x00000000192ea4a5  0x010000a1 0x0000.000.00000000  0x00000001 0x00000000 1735855587
   0x21  9 0x00  0x6e7a5  0x0017  0x00000000192ea5e9  0x010000a1 0x0000.000.00000000  0x00000001 0x00000000 1735855609
  EXT TRN CTL::
  usn: 3
  sp1:0x00000000 sp2:0x00000000 sp3:0x00000000 sp4:0x00000000
  sp5:0x00000000 sp6:0x00000000 sp7:0x00000000 sp8:0x00000000
  EXT TRN TBL::
  index      extflag    extHash    extPdbid/Spare1 extSpare2
  -----------------------------------------------------------
   0x00    0x00000000   0x00000000   0x00000001    0x00000000
   0x01    0x00000000   0x00000000   0x00000001    0x00000000
   0x02    0x00000000   0x00000000   0x00000001    0x00000000
   0x03    0x00000000   0x00000000   0x00000001    0x00000000
   0x04    0x00000000   0x00000000   0x00000001    0x00000000
   0x05    0x00000000   0x00000000   0x00000001    0x00000000
   0x06    0x00000000   0x00000000   0x00000001    0x00000000
   0x07    0x00000000   0x00000000   0x00000001    0x00000000
   0x08    0x00000000   0x00000000   0x00000001    0x00000000
   0x09    0x00000000   0x00000000   0x00000001    0x00000000
   0x0a    0x00000000   0x00000000   0x00000001    0x00000000
   0x0b    0x00000000   0x00000000   0x00000001    0x00000000
   0x0c    0x00000000   0x00000000   0x00000001    0x00000000
   0x0d    0x00000000   0x00000000   0x00000001    0x00000000
   0x0e    0x00000000   0x00000000   0x00000001    0x00000000
   0x0f    0x00000000   0x00000000   0x00000001    0x00000000
   0x10    0x00000000   0x00000000   0x00000001    0x00000000
   0x11    0x00000000   0x00000000   0x00000001    0x00000000
   0x12    0x00000000   0x00000000   0x00000001    0x00000000
   0x13    0x00000000   0x00000000   0x00000001    0x00000000
   0x14    0x00000000   0x00000000   0x00000001    0x00000000
   0x15    0x00000000   0x00000000   0x00000001    0x00000000
   0x16    0x00000000   0x00000000   0x00000001    0x00000000
   0x17    0x00000000   0x00000000   0x00000001    0x00000000
   0x18    0x00000000   0x00000000   0x00000001    0x00000000
   0x19    0x00000000   0x00000000   0x00000001    0x00000000
   0x1a    0x00000000   0x00000000   0x00000001    0x00000000
   0x1b    0x00000000   0x00000000   0x00000001    0x00000000
   0x1c    0x00000000   0x00000000   0x00000001    0x00000000
   0x1d    0x00000000   0x00000000   0x00000001    0x00000000
   0x1e    0x00000000   0x00000000   0x00000001    0x00000000
   0x1f    0x00000000   0x00000000   0x00000001    0x00000000
   0x20    0x00000000   0x00000000   0x00000001    0x00000000
   0x21    0x00000000   0x00000000   0x00000001    0x00000000
Dump of buffer cache for pdb 1 tsn 2 rdba 0x10000a0 at level 10 done.

基于上述信息基本上可以确认是由于我们修复的undo文件和seq$中的file#=1 block#=1161不匹配导致该问题,对其进行处理之后,数据库open成功

2025-03-11T13:12:52.837511+08:00
alter database open
2025-03-11T13:12:52.860336+08:00
Smart fusion block transfer is disabled:
  instance mounted in exclusive mode.
Buffer Cache Full DB Caching mode changing from FULL CACHING DISABLED to FULL CACHING ENABLED 
2025-03-11T13:12:52.864333+08:00
Crash Recovery excluding pdb 2 which was cleanly closed.
2025-03-11T13:12:52.864333+08:00
Crash Recovery excluding pdb 3 which was cleanly closed.
Endian type of dictionary set to little
2025-03-11T13:12:52.886321+08:00
Redo log for group 1, sequence 7 is not located on DAX storage
Thread 1 opened at log sequence 7
  Current log# 1 seq# 7 mem# 0: H:\TEMP\ORCLCDB\REDO01.LOG
Successful open of redo thread 1
2025-03-11T13:12:52.894310+08:00
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Stopping change tracking
Undo initialization recovery: Parallel FPTR complete: start:824531531 end:824531531 diff:0 ms (0.0 seconds)
Undo initialization recovery: err:0 start: 824531531 end: 824531531 diff: 0 ms (0.0 seconds)
[13696] Successfully onlined Undo Tablespace 5.
Undo initialization online undo segments: err:0 start: 824531531 end: 824531609 diff: 78 ms (0.1 seconds)
Undo initialization finished serial:0 start:824531531 end:824531609 diff:78 ms (0.1 seconds)
Database Characterset is AL32UTF8
No Resource Manager plan active
Starting background process RCBG
2025-03-11T13:12:53.502478+08:00
RCBG started with pid=48, OS id=17560_12292 
replication_dependency_tracking turned off (no async multimaster replication found)
Starting background process AQPC
2025-03-11T13:12:53.724661+08:00
AQPC started with pid=49, OS id=17560_12948 
PDB$SEED(2):Pluggable database PDB$SEED opening in read only
PDB$SEED(2):Autotune of undo retention is turned on. 
PDB$SEED(2):Endian type of dictionary set to little
PDB$SEED(2):Undo initialization finished serial:0 start:824532375 end:824532375 diff:0 ms (0.0 seconds)
PDB$SEED(2):Database Characterset for PDB$SEED is AL32UTF8
PDB$SEED(2):*********************************************************************
PDB$SEED(2):WARNING: The following temporary tablespaces in container(PDB$SEED)
PDB$SEED(2):         contain no files.
PDB$SEED(2):         This condition can occur when a backup controlfile has
PDB$SEED(2):         been restored.  It may be necessary to add files to these
PDB$SEED(2):         tablespaces.  That can be done using the SQL statement:
PDB$SEED(2): 
PDB$SEED(2):         ALTER TABLESPACE <tablespace_name> ADD TEMPFILE
PDB$SEED(2): 
PDB$SEED(2):         Alternatively, if these temporary tablespaces are no longer
PDB$SEED(2):         needed, then they can be dropped.
PDB$SEED(2):           Empty temporary tablespace: TEMP
PDB$SEED(2):*********************************************************************
2025-03-11T13:12:53.936415+08:00
PDB$SEED(2):SUPLOG: Set PDB SUPLOG SGA at PDB OPEN, old 0x0, new 0x0 (no suplog)
PDB$SEED(2):Opening pdb with no Resource Manager plan active
Starting background process CJQ0
2025-03-11T13:12:54.249096+08:00
CJQ0 started with pid=77, OS id=17560_17968 
Completed: alter database open

所幸客户需要的数据直接在cdb中,直接使用expdp导出客户需要的数据,完成本次恢复

删除redo导致ORA-00313 ORA-00312故障处理

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：删除redo导致ORA-00313 ORA-00312故障处理

有客户由于误操作直接rm 删除了redo文件,导致数据库启动报ORA-00313 ORA-00312错

2025-03-07T14:49:16.325723+08:00
ALTER DATABASE OPEN
2025-03-07T14:50:00.124620+08:00
Ping without log force is disabled:
  instance mounted in exclusive mode.
2025-03-07T14:50:00.198907+08:00
Crash Recovery excluding pdb 2 which was cleanly closed.
2025-03-07T14:50:00.238450+08:00
Beginning crash recovery of 1 threads
 parallel recovery started with 15 processes
 Thread 1: Recovery starting at checkpoint rba (logseq 2966 block 74686), scn 0
2025-03-07T14:50:00.325246+08:00
Started redo scan
2025-03-07T14:50:00.341193+08:00
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_2681.trc:
ORA-00313: open failed for members of log group 2 of thread 1
ORA-00312: online log 2 thread 1: '/u01/app/oracle/oradata/orcl/redo02.log'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 7
2025-03-07T14:50:00.372632+08:00
Slave encountered ORA-10388 exception during crash recovery
…………
2025-03-07T14:50:00.385698+08:00
Slave encountered ORA-10388 exception during crash recovery
2025-03-07T14:50:00.388594+08:00
Aborting crash recovery due to error 313
2025-03-07T14:50:00.388739+08:00
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_2681.trc:
ORA-00313: open failed for members of log group 2 of thread 1
ORA-00312: online log 2 thread 1: '/u01/app/oracle/oradata/orcl/redo02.log'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 7
2025-03-07T14:50:00.389243+08:00
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_2681.trc:
ORA-00313: open failed for members of log group 2 of thread 1
ORA-00312: online log 2 thread 1: '/u01/app/oracle/oradata/orcl/redo02.log'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 7
ORA-313 signalled during: ALTER DATABASE OPEN...

然后客户把历史的redo文件拷贝过来,尝试恢复数据库,报ORA-00314 ORA-00312错误

2025-03-07T15:07:30.784759+08:00
ALTER DATABASE OPEN
Ping without log force is disabled:
  instance mounted in exclusive mode.
2025-03-07T15:07:30.808497+08:00
Crash Recovery excluding pdb 2 which was cleanly closed.
2025-03-07T15:07:30.838664+08:00
Beginning crash recovery of 1 threads
 parallel recovery started with 15 processes
 Thread 1: Recovery starting at checkpoint rba (logseq 2966 block 74686), scn 0
2025-03-07T15:07:30.897547+08:00
Started redo scan
2025-03-07T15:07:30.898222+08:00
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_4106.trc:
ORA-00314: log 2 of thread 1, expected sequence# 2966 doesn't match 1646
ORA-00312: online log 2 thread 1: '/u01/app/oracle/oradata/orcl/redo02.log'
2025-03-07T15:07:30.930089+08:00
Slave encountered ORA-10388 exception during crash recovery
…………
2025-03-07T15:07:30.940051+08:00
Slave encountered ORA-10388 exception during crash recovery
2025-03-07T15:07:30.942274+08:00
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_mz00_4138.trc:
ORA-00312: online log 1 thread 1: '/u01/app/oracle/oradata/orcl/redo01.log'
2025-03-07T15:07:30.945509+08:00
Slave encountered ORA-10388 exception during crash recovery
2025-03-07T15:07:30.945512+08:00
Slave encountered ORA-10388 exception during crash recovery
2025-03-07T15:07:30.948369+08:00
Aborting crash recovery due to error 314
2025-03-07T15:07:30.948488+08:00
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_4106.trc:
ORA-00314: log 2 of thread 1, expected sequence# 2966 doesn't match 1646
ORA-00312: online log 2 thread 1: '/u01/app/oracle/oradata/orcl/redo02.log'
2025-03-07T15:07:30.949390+08:00
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_4106.trc:
ORA-00314: log 2 of thread 1, expected sequence# 2966 doesn't match 1646
ORA-00312: online log 2 thread 1: '/u01/app/oracle/oradata/orcl/redo02.log'
ORA-314 signalled during: ALTER DATABASE OPEN...

使用Oracle数据库异常恢复检查脚本(Oracle Database Recovery Check)脚本收集信息之后数据文件头状态和所需要redo信息
df_header

数据库需要sequence#为2966的redo日志,但是当前已经被删除,基于当前情况,只能进行强制非一致性恢复,尝试强制打开库

SQL> recover database;                 
ORA-00283: recovery session canceled due to errors
ORA-00314: log 2 of thread 1, expected sequence# 2966 doesn't match 1646
ORA-00312: online log 2 thread 1: '/u01/app/oracle/oradata/orcl/redo02.log'

QL> select group#,status,sequence# from v$log;

	  GROUP# STATUS 		 SEQUENCE#
---------------- ---------------- ----------------
	       1 UNUSED 			 0
	       3 CURRENT		      2967
	       2 ACTIVE 		      2966

SQL> 
SQL> 
SQL> recover database until cancel;
ORA-00279: change 163033183 generated at 03/07/2025 14:04:20 needed for thread 1
ORA-00289: suggestion : /u01/app/oracle/recovery_area/orcl/archivelog/2025_03_08/o1_mf_1_2966_%u_.arc
ORA-00280: change 163033183 for thread 1 is in sequence #2966


Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
cancel
ORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error below
ORA-01194: file 1 needs more recovery to be consistent
ORA-01110: data file 1: '/u01/app/oracle/oradata/orcl/system01.dbf'


ORA-01112: media recovery not started


SQL> alter database open resetlogs;

Database altered.

运气不错,直接打开数据库成功,然后逻辑导出数据,完成此处恢复.这个让我想起来了一些类似案例:
Oracle 23ai rm redo*.log恢复
 清空redo,导致ORA-27048: skgfifi: file header information is invalid
由于默认情况下oracle的redo文件扩展名是.log,然后被当做是不重要文件从而被清理导致数据库故障,在oracle服务器上清理数据之前建议查询v$datafile,v$logfile,v$tempfile,v$controlfile来确认是否是数据库文件

linux rm -rf 删除数据文件恢复

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：linux rm -rf 删除数据文件恢复

有客户由于误操作删除了oracle的部分数据文件(rm -rf 方式删除),然后自己尝试进行恢复操作,对部分文件执行了offline,导致比较麻烦的后果

offline-file

接手故障之后,第一时间对其进行了镜像(因为有部分文件句柄已经释放,为了方式覆盖进一步破坏),对于没有释放的句柄可以通过类似方法进行恢复,参考以前类似恢复:
Oracle误删除数据文件恢复
Solaris rm datafile recovery—利用句柄误删除数据文件恢复

!cp  269  /u01/app/oracle/oradata/orcl/XXXXXX_DATA01.dbf
alter database datafile 12 offline;
recover datafile 12;
alter database datafile 12 online;

对于删除文件,而且句柄已经释放的文件,通过文件系统层面反删除进行恢复,参考以前类似恢复:
rm -rf误删Oracle数据库恢复
 记录一次rm -rf 删除数据文件异常恢复
 rm -rf 删除数据文件恢复方法—文件系统反删除+oracle碎片重组

在这个恢复过程中,由于客户linux是物理机,而且本地空间不足,无法对其进行镜像,采用dd命令直接写镜像到其他的linux机器上（通过nfs方式),然后在win机器上直接挂载该nfs,记录下win上挂载nfs操作
nfs

CDM备份缺少归档打开数据库报ORA-600 kcbzib_kcrsds_1故障处理

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：CDM备份缺少归档打开数据库报ORA-600 kcbzib_kcrsds_1故障处理

有客户联系我们,说一个19c的库,由于产生归档较多在备份过程中把部分归档删除而导致没有备份成功,从而使得他们那边一个误操作需要恢复使用备份做不完全恢复,备份厂商进行恢复并尝试强制打开库报ORA-600 kcbzib_kcrsds_1错误
ORA-600-kcbzibz_kcrsds_1

由于客户那边使用的是CDM方式备份,可以较快的准备出来一个新环境,观察客户在应用日志过程中,文件头的scn一直不变,怀疑文件头由于begin backup冻结,对其进行dump,发现确实做了hot backup操作,而且没有end backup
scn

由于缺少归档,不完全恢复没有成功,begin backup 也无法正常结束,对于这种情况,先尝试调整到文件头最大的scn值,尝试打开库

SQL> alter database open resetlogs ;
alter database open resetlogs 
*
ERROR at line 1:
ORA-00603: ORACLE server session terminated by fatal error
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-00600: internal error code, arguments: [kcbzib_kcrsds_1], [], [], [], [],
[], [], [], [], [], [], []
Process ID: 3949615
Session ID: 5111 Serial number: 30040


--重启到mount状态
SQL> set numw 16
col CHECKPOINT_TIME for a40
set lines 150
set pages 1000
SELECT status,
to_char(checkpoint_time,'yyyy-mm-dd hh24:mi:ss') checkpoint_time,FUZZY,checkpoint_change#,
count(*) ROW_NUM
FROM v$datafile_header
GROUP BY status, checkpoint_change#, to_char(checkpoint_time,'yyyy-mm-dd hh24:mi:ss'),fuzzy
ORDER BY status, checkpoint_change#, checkpoint_time;SQL> SQL> SQL> SQL>   2    3    4    5    6  

STATUS  CHECKPOINT_TIME                          FUZ CHECKPOINT_CHANGE#          ROW_NUM
------- ---------------------------------------- --- ------------------ ----------------
ONLINE  2025-02-08 22:43:01                      YES     15626238353558               56

打开库失败,只能找出来数据库最大的block中最大scn,然后调整文件头scn的值,实现数据库open

SQL> oradebug setmypid
Statement processed.
SQL> oradebug DUMPvar SGA kcsgscn_
SQL> kscn8 kcsgscn_ [060017E98, 060017EA0) = 00000000 00000000
SQL> oradebug DUMPvar SGA kcsgscn_
kscn8 kcsgscn_ [060017E98, 060017EA0) = 8CD9C896 00000E4D
SQL> 
SQL> 
SQL> 
SQL> alter database open ;
alter database open 
*
ERROR at line 1:
ORA-01113: file 1 needs media recovery
ORA-01110: data file 1: '/cdmbak/db/xifenfei/ob_data_D-XIFENFEI-SYSTEM_FNO-1_t13930nd'

SQL> recover database;
Media recovery complete.
SQL> alter database open;

Database altered.

SQL>

使用exp导出客户需要的表,完成本次恢复任务
对于ORA-600 kcbzib_kcrsds_1恢复的情况,以前有过大量恢复案例,修改数据库scn即可
kcbzib_kcrsds_1报错汇总
 12C数据库报ORA-600 kcbzib_kcrsds_1故障处理
 存储故障,强制拉库报ORA-600 kcbzib_kcrsds_1处理
 Patch SCN工具一键恢复ORA-600 kcbzib_kcrsds_1
此类故障处理太多,不一一列举,解决这个错误之后,数据库open成功,然后安排逻辑迁移即可

ORA-07445: exception encountered: core dump [expgod()+43] [IN_PAGE_ERROR]

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：ORA-07445: exception encountered: core dump [expgod()+43] [IN_PAGE_ERROR]

数据库在运行过程中报O/S-Error: (OS 23) 数据错误(循环冗余检查)错误

Thu Jan 30 22:00:02 2025
Begin automatic SQL Tuning Advisor run for special tuning task  "SYS_AUTO_SQL_TUNING_TASK"
Thu Jan 30 22:00:04 2025
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_j000_12576.trc:
ORA-12012: error on auto execute of job 155962
ORA-01115: IO error reading block from file  (block # )
ORA-01110: data file 1: 'D:\APP\ADMINISTRATOR\ORADATA\ORCL\SYSTEM01.DBF'
ORA-27070: async read/write failed
OSD-04006: ReadFile() 失败, 无法读取文件
O/S-Error: (OS 23) 数据错误(循环冗余检查)。
ORA-06512: at "SYS.DBMS_STATS", line 25836
ORA-06512: at "SYS.DBMS_STATS", line 26171
End automatic SQL Tuning Advisor run for special tuning task  "SYS_AUTO_SQL_TUNING_TASK"
Fri Jan 31 02:00:00 2025
Clearing Resource Manager plan via parameter
Fri Jan 31 08:15:46 2025
Thread 1 advanced to log sequence 4420 (LGWR switch)
  Current log# 1 seq# 4420 mem# 0: D:\APP\ADMINISTRATOR\ORADATA\ORCL\REDO01.LOG
Fri Jan 31 10:53:57 2025
Process J000 died, see its trace file
kkjcre1p: unable to spawn jobq slave process 
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_cjq0_1140.trc:
Fri Jan 31 10:53:57 2025
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_j000_7916.trc:
ORA-27102: out of memory
OSD-00043: 附加错误信息
O/S-Error: (OS 1455) 页面文件太小，无法完成操作。
Process J000 died, see its trace file
kkjcre1p: unable to spawn jobq slave process 
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_cjq0_1140.trc:
Fri Jan 31 10:54:03 2025
Exception [type: ACCESS_VIOLATION, UNABLE_TO_READ] [ADDR:0x18] [PC:0xB778B02, clsdcxini()+90]
ERROR: Unable to normalize symbol name for the following short stack (at offset 199):
dbgexProcessError()+193<-dbgeExecuteForError()+65<-dbgePostErrorKGE()+1726<-dbkePostKGE_kgsf()+75
<-kgeade()+560<-kgerev()+125<-kgerec5()+60<-sss_xcpt_EvalFilterEx()+1869<-sss_xcpt_EvalFilter()+174
<-.1.4_5+59<-00007FFD0245F306<-00007FFD024735AF<-00007FFD023D4AAF<-00007FFD0247231E<-clsdcxini()+90
<-clsdinit()+124<-ksdnfy()+225<-kscnfy()+778<-opirip()+86<-opidrv()+909<-sou2o()+98<-opimai_real()+299
<-opimai()+191<-BackgroundThreadStart()+693<-00007FFD020E7E94<-00007FFD02437AD1
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_j000_9920.trc  (incident=39621):
ORA-07445: exception encountered: core dump [clsdcxini()+90][ACCESS_VIOLATION][ADDR:0x18][PC:0xB778B02][UNABLE_TO_READ]
Incident details in: d:\app\administrator\diag\rdbms\orcl\orcl\incident\incdir_39621\orcl_j000_9920_i39621.trc

然后无法正常启动,报Exception [type: IN_PAGE_ERROR, ] [] [PC:0x2C9C015, expgod()+43]错误

Wed Feb 05 09:43:51 2025
Sweep [inc][39621]: completed
Successful mount of redo thread 1, with mount id 1720066005
Database mounted in Exclusive Mode
Lost write protection disabled
Completed: alter database mount exclusive
alter database open
Beginning crash recovery of 1 threads
 parallel recovery started with 3 processes
Started redo scan
Completed redo scan
 read 140 KB redo, 62 data blocks need recovery
Started redo application at
 Thread 1: logseq 4420, block 42375
Wed Feb 05 09:44:00 2025
Recovery of Online Redo Log: Thread 1 Group 1 Seq 4420 Reading mem 0
  Mem# 0: D:\APP\ADMINISTRATOR\ORADATA\ORCL\REDO01.LOG
Completed redo application of 0.09MB
Completed crash recovery at
 Thread 1: logseq 4420, block 42656, scn 94456019
 62 data blocks read, 62 data blocks written, 140 redo k-bytes read
Wed Feb 05 09:44:01 2025
Exception [type: IN_PAGE_ERROR, ] [] [PC:0x2C9C015, expgod()+43]
ERROR: Unable to normalize symbol name for the following short stack (at offset 199):
dbgexProcessError()+193<-dbgeExecuteForError()+65<-dbgePostErrorKGE()+1726
<-dbkePostKGE_kgsf()+75<-kgeade()+560<-kgerev()+125<-kgerec5()+60<-sss_xcpt_EvalFilterEx()+1869
<-sss_xcpt_EvalFilter()+174<-.1.4_5+59<-00007FFD0245F306<-00007FFD024735AF<-00007FFD023D4AAF
<-00007FFD0247231E<-expgod()+43<-xtyopncb()+241<-qctcopn()+613<-qctcopn()+392<-qctcpqb()+290
<-qctcpqbl()+52<-xtydrv()+148<-opitca()+1091<-kksLoadChild()+9008<-kxsGetRuntimeLock()+2320
<-kksfbc()+15225<-kkspsc0()+2117<-kksParseCursor()+181<-opiosq0()+2538<-opiosq()+23<-opiodr()+1662
<-rpidrus()+862<-rpidru()+154<-rpiswu2()+2757<-rpidrv()+6105<-rpisplu()+1607<-kqldFixedTableLoadCols()+345
<-kqldcor()+2534<-kglslod()+352<-kqlslod()+52<-PGOSF455_kqlsublod()+125<-kqllod()+7284<-kglobld()+1354
<-kglobpn()+1900<-kglpim()+336<-qcdlgtd()+260<-qcsfplob()+166<-qcsprfro()+903<-qcsprfro_tree()
+292<-qcsprfro_tree()+373<-qcspafq()+96
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_mmon_15468.trc  (incident=39749):
ORA-07445: exception encountered: core dump [expgod()+43] [IN_PAGE_ERROR] [] [PC:0x2C9C015] [] []
Incident details in: d:\app\administrator\diag\rdbms\orcl\orcl\incident\incdir_39749\orcl_mmon_15468_i39749.trc
Wed Feb 05 09:44:02 2025
Thread 1 advanced to log sequence 4421 (thread open)
Thread 1 opened at log sequence 4421
  Current log# 2 seq# 4421 mem# 0: D:\APP\ADMINISTRATOR\ORADATA\ORCL\REDO02.LOG
Successful open of redo thread 1
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Wed Feb 05 09:44:02 2025
SMON: enabling cache recovery
Wed Feb 05 09:44:11 2025
Exception [type: IN_PAGE_ERROR, ] [] [PC:0x2C9C015, expgod()+43]
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_ora_9308.trc  (incident=39781):
ORA-07445: ??????: ???? [expgod()+43] [IN_PAGE_ERROR] [] [PC:0x2C9C015] [] []
Incident details in: d:\app\administrator\diag\rdbms\orcl\orcl\incident\incdir_39781\orcl_ora_9308_i39781.trc
Wed Feb 05 09:44:19 2025
PMON (ospid: 12376): terminating the instance due to error 397
Instance terminated by PMON, pid = 12376

基于上述的Exception [type: IN_PAGE_ERROR, ] [] [PC:0x2C9C015, expgod()+43]错误,第一反应就是可能由于底层损坏导致数据块损坏,dbv检查文件是否报错
dbv-system

检查系统日志确认异常
20250208203059

尝试拷贝文件也报错

已经比较明确由于底层问题,解决给问题之前,需要先对文件系统进行处理,然后再对恢复出来的数据文件恢复数据

分类目录归档：Oracle备份恢复