ORA-xxxxx – Database SOS

最近接到一个客户有一个数据文件offline的恢复咨询,通过分析日志,当时是由于在启动的时候数据文件被占用导致后续数据库open之后,该文件被强制offline掉

Fri May 16 20:01:05 2025
Database mounted in Exclusive Mode
Completed: ALTER DATABASE   MOUNT
Fri May 16 20:01:05 2025
ALTER DATABASE OPEN
Fri May 16 20:01:06 2025
LGWR: STARTING ARCH PROCESSES
ARC0 started with pid=70, OS id=4628
Fri May 16 20:01:06 2025
ARC0: Archival started
ARC1 started with pid=74, OS id=4840
Fri May 16 20:01:06 2025
ARC1: Archival started
LGWR: STARTING ARCH PROCESSES COMPLETE
Fri May 16 20:01:06 2025
Errors in file d:\oracle\product\10.2.0\admin\orcl\bdump\orcl_lgwr_4080.trc:
ORA-01110: data file 14: 'D:\ORADATA\XIFENFEI105_DAT_1.DBF'
ORA-01114: IO error writing block to file 14 (block # 1)
ORA-27041: unable to open file
OSD-04002: 无法打开文件
O/S-Error: (OS 32) 另一个程序正在使用此文件，进程无法访问。

Thread 1 opened at log sequence 172421
  Current log# 1 seq# 172421 mem# 0: D:\ORACLE\PRODUCT\10.2.0\ORADATA\ORCL\REDO01.LOG
Fri May 16 20:01:06 2025
ARC1: STARTING ARCH PROCESSES
Fri May 16 20:01:06 2025
Successful open of redo thread 1
Fri May 16 20:01:06 2025
ARC0: Becoming the 'no FAL' ARCH
ARC0: Becoming the 'no SRL' ARCH
Fri May 16 20:01:06 2025
ARC2: Archival started
ARC1: STARTING ARCH PROCESSES COMPLETE
ARC2 started with pid=78, OS id=4056
Fri May 16 20:01:06 2025
ARC1: Becoming the heartbeat ARCH
Fri May 16 20:01:06 2025
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Fri May 16 20:01:06 2025
SMON: enabling cache recovery
Fri May 16 20:01:07 2025
Successfully onlined Undo Tablespace 1.
Fri May 16 20:01:07 2025
SMON: enabling tx recovery
Fri May 16 20:01:08 2025
Database Characterset is ZHS16GBK
replication_dependency_tracking turned off (no async multimaster replication found)
Starting background process QMNC
QMNC started with pid=86, OS id=4492
Fri May 16 20:01:12 2025
db_recovery_file_dest_size of 51200 MB is 1.97% used. This is a
user-specified limit on the amount of space that will be used by this
database for recovery-related files, and does not reflect the amount of
space available in the underlying filesystem or ASM diskgroup.
Fri May 16 20:01:13 2025
Completed: ALTER DATABASE OPEN
Fri May 16 20:06:44 2025
Restarting dead background process MMON
MMON started with pid=98, OS id=4232
Fri May 16 20:07:06 2025
Shutting down archive processes
Fri May 16 20:07:11 2025
ARCH shutting down
ARC2: Archival stopped
Fri May 16 20:10:32 2025
Thread 1 advanced to log sequence 172422
  Current log# 2 seq# 172422 mem# 0: D:\ORACLE\PRODUCT\10.2.0\ORADATA\ORCL\REDO02.LOG
Fri May 16 20:15:33 2025
Errors in file d:\oracle\product\10.2.0\admin\orcl\bdump\orcl_ckpt_2496.trc:
ORA-01171: datafile 14 going offline due to error advancing checkpoint
ORA-01122: database file 14 failed verification check
ORA-01110: data file 14: 'D:\ORADATA\XIFENFEI105_DAT_1.DBF'
ORA-01208: data file is an old version - not accessing current version

Fri May 16 20:23:09 2025
Starting background process EMN0
EMN0 started with pid=82, OS id=2660

通过dbv检查报错文件,确认被offline文件本身正常

本身这个故障相对比较简单,只要归档存在直接recover datafile,然后online即可,但是由于备份软件定时工作,导致对应的归档被备份走

Fri May 16 21:55:10 2025
Control autobackup written to SBT_TAPE device
	comment 'API Version 2.0,MMS Version 10.0.0.116',
	media 'V_6746190_6959024'
	handle 'c-1300253653-20250516-00'
Fri May 16 21:56:03 2025
Thread 1 cannot allocate new log, sequence 172423
Private strand flush not complete
  Current log# 2 seq# 172422 mem# 0: D:\ORACLE\PRODUCT\10.2.0\ORADATA\ORCL\REDO02.LOG

而且被异常的数据文件不是核心业务文件,导致客户没有及时发现,等到发现之时尝试recover datafile,提示缺少归档

Wed May 28 17:26:01 2025
alter database recover datafile list clear
Wed May 28 17:26:01 2025
Completed: alter database recover datafile list clear
Wed May 28 17:26:01 2025
alter database recover if needed
 datafile 14

Media Recovery Start
 parallel recovery started with 16 processes
ORA-279 signalled during: alter database recover if needed
 datafile 14
...
Wed May 28 17:26:11 2025
alter database recover cancel
Wed May 28 17:26:13 2025
Media Recovery Canceled
Completed: alter database recover cancel
Wed May 28 17:38:58 2025
ALTER DATABASE RECOVER  datafile 'D:\ORADATA\XIFENFEI105_DAT_1.DBF'  
Wed May 28 17:38:58 2025
Media Recovery Start
 parallel recovery started with 16 processes
ORA-279 signalled during: ALTER DATABASE RECOVER  datafile 'D:\ORADATA\XIFENFEI105_DAT_1.DBF'  ...
Wed May 28 18:26:37 2025
ALTER DATABASE RECOVER    CONTINUE DEFAULT  
Wed May 28 18:26:38 2025
Media Recovery Log D:\ORACLE\PRODUCT\10.2.0\FLASH_RECOVERY_AREA\ORCL\ARCHIVELOG\2025_05_28\O1_MF_1_172421_%U_.ARC
Errors with log D:\ORACLE\PRODUCT\10.2.0\FLASH_RECOVERY_AREA\ORCL\ARCHIVELOG\2025_05_28\O1_MF_1_172421_%U_.ARC
ORA-308 signalled during: ALTER DATABASE RECOVER    CONTINUE DEFAULT  ...
Wed May 28 18:26:38 2025
ALTER DATABASE RECOVER    CONTINUE DEFAULT  
Wed May 28 18:26:38 2025
Media Recovery Log D:\ORACLE\PRODUCT\10.2.0\FLASH_RECOVERY_AREA\ORCL\ARCHIVELOG\2025_05_28\O1_MF_1_172421_%U_.ARC
Errors with log D:\ORACLE\PRODUCT\10.2.0\FLASH_RECOVERY_AREA\ORCL\ARCHIVELOG\2025_05_28\O1_MF_1_172421_%U_.ARC
ORA-308 signalled during: ALTER DATABASE RECOVER    CONTINUE DEFAULT  ...
Wed May 28 18:26:38 2025
ALTER DATABASE RECOVER CANCEL 
Wed May 28 18:26:40 2025
Media Recovery Canceled
Completed: ALTER DATABASE RECOVER CANCEL

这个客户运气还不错,带库中的需要恢复的归档日志都还在,通过指定带库通道,直接recover datafile成功

RUN {
  ALLOCATE CHANNEL ch1 DEVICE TYPE 'sbt_tape' 
  PARMS="BLKSIZE=262144,ENV=(CV_mmsApiVsn=2,CV_channelPar=ch1)";
  ALLOCATE CHANNEL ch2 DEVICE TYPE 'sbt_tape' 
  PARMS="BLKSIZE=262144,ENV=(CV_mmsApiVsn=2,CV_channelPar=ch2)";
 recover datafile 14;
}

至此完美解决该问题,通过这个case,的出来的经验有:
1. 数据库重启之后,要检查数据库日志和查询数据库数据文件状态(主要防止一些不太常用的文件异常,不能及时发现)
2. 需要需要数据库的基本情况,比如备份,容灾,asm磁盘组冗余,存储冗余,网络冗余等情况,这样出现问题好排查解决

ORA-600 ksvworkmsgalloc: bad reaper

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：ORA-600 ksvworkmsgalloc: bad reaper

有一个朋友说他们想把12c的库还原到19c版本中然后进行升级测试,结果在打开库的过程中发现几个错误,让我给帮忙分析下
resetlogs 报ORA-00392 ORA-00312

SQL> alter database open resetlogs upgrade;
alter database open resetlogs upgrade
*
ERROR at line 1:
ORA-00392: log 7 of thread 1 is being cleared, operation not allowed
ORA-00312: online log 7 thread 1: '/DBS1/data/NDBS/onlinelog/redo07_m1.log '
ORA-00312: online log 7 thread 1: '/DBS1/arch/NDBS/onlinelog/redo07_m2.log '

这个错误一般是由于redo状态不对,比如标记为了CLEARING_CURRENT,处理操作

SQL> select group#,status from v$log;

          GROUP# STATUS
---------------- ----------------
               1 CLEARING
               2 CLEARING
               3 CLEARING
               4 CLEARING
              10 CLEARING
               6 CLEARING
               7 CLEARING_CURRENT
               8 CLEARING
               9 CLEARING
               5 CLEARING

10 rows selected.


SQL> alter database clear logfile group 7;

Database altered.

SQL> select group#,status from v$log;

          GROUP# STATUS
---------------- ----------------
               1 CLEARING
               2 CLEARING
               3 CLEARING
               4 CLEARING
              10 CLEARING
               6 CLEARING
               7 CURRENT
               8 CLEARING
               9 CLEARING
               5 CLEARING

10 rows selected.

再次reseltogs报ORA-600 ksvworkmsgalloc: bad reaper错误

SQL> alter database open resetlogs upgrade;
alter database open resetlogs upgrade
*
ERROR at line 1:
ORA-00600: internal error code, arguments: [ksvworkmsgalloc: bad reaper], [0x080010003], [], [], []

这个错误通过查询MOS 发现Open Resetlogs Fail with ORA-00600[ksvworkmsgalloc: bad reaper] (Doc ID 2728106.1)文章中描述,由于non-ASM to ASM环境redo文件在clear的时候触发该问题

是由于db_create_online_log_dest_1参数没有设置导致,对于该库是由asm环境到文件系统,估计也是在resetlogs的时候clear redo报出来该错误,解决办法给该库设置上
db_create_online_log_dest_1=/DBS1/data,db_create_online_log_dest_2=/DBS1/arch,然后打开库成功

ORA-600 krccfl_chunk故障处理

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：ORA-600 krccfl_chunk故障处理

一个数据库启动包ORA-600 krccfl_chunk错误

2025-05-06T10:37:47.428203+08:00
Completed: ALTER DATABASE MOUNT /* db agent *//* {2:50212:2} */
ALTER DATABASE OPEN /* db agent *//* {2:50212:2} */
2025-05-06T10:37:47.433709+08:00
This instance was first to open
Block change tracking file is current.
Ping without log force is disabled:
  not an Exadata system.
start recovery: pdb 0, passed in flags x4 (domain enable 5) 
2025-05-06T10:37:48.203383+08:00
Beginning crash recovery of 2 threads
2025-05-06T10:37:48.568120+08:00
 parallel recovery started with 32 processes
2025-05-06T10:37:48.610951+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.611037+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.611243+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.611438+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.614947+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.616591+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.617188+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.617253+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.617428+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.617606+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.617676+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.617809+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.636568+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.636568+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.636620+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.637156+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.637300+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.637881+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.637999+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.638112+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.638241+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.638304+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.638338+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.638347+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.641621+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.642926+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.643092+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.643192+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.643204+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.643372+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.643516+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.643573+08:00
start recovery: pdb 0, passed in flags x5 (domain enable 5) 
2025-05-06T10:37:48.748956+08:00
Started redo scan
2025-05-06T10:37:49.849382+08:00
Completed redo scan
 read 469347 KB redo, 1213 data blocks need recovery
2025-05-06T10:37:50.007840+08:00
Started redo application at
 Thread 1: logseq 369323, block 651514, offset 0
 Thread 2: logseq 132962, block 1319944, offset 0
2025-05-06T10:37:50.016910+08:00
Recovery of Online Redo Log: Thread 1 Group 13 Seq 369323 Reading mem 0
  Mem# 0: +DATA/orcl/ONLINELOG/group_13.349.978709791
  Mem# 1: +FRA/orcl/ONLINELOG/group_13.12992.978709793
2025-05-06T10:37:50.025725+08:00
Recovery of Online Redo Log: Thread 2 Group 18 Seq 132962 Reading mem 0
  Mem# 0: +DATA/orcl/ONLINELOG/group_18.354.978710003
  Mem# 1: +FRA/orcl/ONLINELOG/group_18.12997.978710005
2025-05-06T10:37:51.063556+08:00
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_ora_68031.trc(incident=868005)(PDBNAME=CDB$ROOT):
ORA-00600: internal error code, arguments: [krccfl_chunk], [0x7F9BBB30BE58], [166528],[],[],[],[],[],[],[],[],[]
Incident details in: /u01/app/oracle/diag/rdbms/orcl/orcl2/incident/incdir_868005/orcl2_ora_68031_i868005.trc
2025-05-06T10:37:52.269823+08:00
Dumping diagnostic data in directory=[cdmp_20250506103752],requested by(instance=2,osid=68031),summary=[incident=868005].
2025-05-06T10:37:52.306517+08:00
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
2025-05-06T10:37:52.310723+08:00
Slave encountered ORA-10388 exception during crash recovery
2025-05-06T10:37:52.310813+08:00
Slave encountered ORA-10388 exception during crash recovery
2025-05-06T10:37:52.310820+08:00
Slave encountered ORA-10388 exception during crash recovery
2025-05-06T10:37:52.310853+08:00
Slave encountered ORA-10388 exception during crash recovery
2025-05-06T10:37:52.310902+08:00
Slave encountered ORA-10388 exception during crash recovery
2025-05-06T10:37:52.310907+08:00
Slave encountered ORA-10388 exception during crash recovery
2025-05-06T10:37:52.310945+08:00
Recovery slave process is holding some recovery locks. Killing the instance now.
2025-05-06T10:37:52.310950+08:00
Slave encountered ORA-10388 exception during crash recovery
2025-05-06T10:37:52.310987+08:00
Slave encountered ORA-10388 exception during crash recovery
2025-05-06T10:37:52.311002+08:00
Slave encountered ORA-10388 exception during crash recovery
2025-05-06T10:37:52.311009+08:00
Slave encountered ORA-10388 exception during crash recovery
2025-05-06T10:37:52.311017+08:00
Slave encountered ORA-10388 exception during crash recovery
2025-05-06T10:37:52.311055+08:00
Recovery slave process is holding some recovery locks. Killing the instance now.
2025-05-06T10:37:52.311055+08:00
Slave encountered ORA-10388 exception during crash recovery
2025-05-06T10:37:52.311064+08:00
Slave encountered ORA-10388 exception during crash recovery
2025-05-06T10:37:52.311071+08:00
Recovery slave process is holding some recovery locks. Killing the instance now.
2025-05-06T10:37:52.311080+08:00
Recovery slave process is holding some recovery locks. Killing the instance now.
2025-05-06T10:37:52.311107+08:00
Recovery slave process is holding some recovery locks. Killing the instance now.
2025-05-06T10:37:52.311119+08:00
Recovery slave process is holding some recovery locks. Killing the instance now.
2025-05-06T10:37:52.311126+08:00
Recovery slave process is holding some recovery locks. Killing the instance now.
2025-05-06T10:37:52.311135+08:00
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_p000_69617.trc:
ORA-10388: parallel query server interrupt (failure)
2025-05-06T10:37:52.311156+08:00
Recovery slave process is holding some recovery locks. Killing the instance now.
2025-05-06T10:37:52.311184+08:00
Recovery slave process is holding some recovery locks. Killing the instance now.
2025-05-06T10:37:52.311203+08:00
Recovery slave process is holding some recovery locks. Killing the instance now.
2025-05-06T10:37:52.311205+08:00
Recovery slave process is holding some recovery locks. Killing the instance now.
2025-05-06T10:37:52.311211+08:00
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_p001_69619.trc:
ORA-10388: parallel query server interrupt (failure)
2025-05-06T10:37:52.311276+08:00
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_p002_69621.trc:
ORA-10388: parallel query server interrupt (failure)
2025-05-06T10:37:52.311276+08:00
Recovery slave process is holding some recovery locks. Killing the instance now.
2025-05-06T10:37:52.311280+08:00
Recovery slave process is holding some recovery locks. Killing the instance now.
2025-05-06T10:37:52.311308+08:00
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_p003_69623.trc:
ORA-10388: parallel query server interrupt (failure)
2025-05-06T10:37:52.311329+08:00
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_p004_69625.trc:
ORA-10388: parallel query server interrupt (failure)
2025-05-06T10:37:52.311341+08:00
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_p005_69627.trc:
ORA-10388: parallel query server interrupt (failure)
2025-05-06T10:37:52.311345+08:00
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_p007_69631.trc:
ORA-10388: parallel query server interrupt (failure)
2025-05-06T10:37:52.311353+08:00
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_p008_69633.trc:
ORA-10388: parallel query server interrupt (failure)
2025-05-06T10:37:52.311374+08:00
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_p006_69629.trc:
ORA-10388: parallel query server interrupt (failure)
2025-05-06T10:37:52.311386+08:00
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_p009_69635.trc:
ORA-10388: parallel query server interrupt (failure)
2025-05-06T10:37:52.311402+08:00
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_p00a_69637.trc:
ORA-10388: parallel query server interrupt (failure)
2025-05-06T10:37:52.311513+08:00
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_p00c_69641.trc:
ORA-10388: parallel query server interrupt (failure)
2025-05-06T10:37:52.311515+08:00
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_p00b_69639.trc:
ORA-10388: parallel query server interrupt (failure)
2025-05-06T10:37:52.348331+08:00
USER (ospid: 69617): terminating the instance due to error 10388
2025-05-06T10:37:52.585589+08:00
System state dump requested by (instance=2, osid=69617 (P000)), summary=[abnormal instance termination].
System State dumped to trace file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_diag_67490_20250506103752.trc
2025-05-06T10:37:54.016704+08:00
License high water mark = 34
2025-05-06T10:37:55.387072+08:00
Instance terminated by USER, pid = 69617
2025-05-06T10:37:55.388683+08:00
Warning: 2 processes are still attach to shmid 2850830:
 (size: 45056 bytes, creator pid: 65902, last attach/detach pid: 67492)
2025-05-06T10:37:56.018027+08:00
USER (ospid: 69907): terminating the instance
2025-05-06T10:37:56.021711+08:00
Instance terminated by USER, pid = 69907

查询mos发现类似文章:
Database doesn’t open after crash ORA-00600 [krccfl_chunk] (Doc ID 2967548.1)
Bug 33251482 – ORA-487 / ORA-600 [krccfl_chunk] : CTWR process terminated during PDB creation (Doc ID 33251482.8)

分析这个客户情况，通过trace信息:Block change tracking file is current. 可以确认是启用了BCT,而且日志信息也反应出来是pdb环境。进一步分析客户的情况,发现他们在以前有一个数据文件创建到了本地（实际是rac环境）

2024-12-23T11:07:09.168322+08:00
PDBODS(5):Completed: alter tablespace PDBODS_DATA add datafile 'D:\APP\ADMINISTRATOR\ORADATA\ORCL\USERS02.DBF'
 size 5000M autoextend on next 1000M maxsize 32000M

数据库中现在实际存储路径/u01/app/oracle/product/12.2.0.1/dbhome_1/dbs/D:APPADMINISTRATORORADATAORCLUSERS 02.DBF
基于这种情况,解决问题比较简单:在本地数据文件所在节点禁用BCT,然后open库,把数据文件拷贝到asm中即可

ORA-600 kddummy_blkchk 数据库循环重启

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：ORA-600 kddummy_blkchk 数据库循环重启

一个运行在hp平台的10g rac突然异常,之后运行一段时间就自动重启,客户让对其进行分析和解决

Thu May  8 06:23:21 2025
ALTER DATABASE OPEN
Picked broadcast on commit scheme to generate SCNs
Thu May  8 06:23:21 2025
Thread 1 opened at log sequence 74302
  Current log# 1 seq# 74302 mem# 0: /dev/vgdata/rrac_redo01
Successful open of redo thread 1
Thu May  8 06:23:21 2025
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Thu May  8 06:23:21 2025
SMON: enabling cache recovery
Thu May  8 06:23:22 2025
Successfully onlined Undo Tablespace 1.
Thu May  8 06:23:22 2025
SMON: enabling tx recovery
Thu May  8 06:23:22 2025
Database Characterset is ZHS16CGB231280
Opening with internal Resource Manager plan
where NUMA PG = 1, CPUs = 4
replication_dependency_tracking turned off (no async multimaster replication found)
Thu May  8 06:23:23 2025
Errors in file /users/oracle/admin/orcl/bdump/orcl1_smon_15721.trc:
ORA-00600: internal error code, arguments: [kddummy_blkchk], [118], [333578], [18019], [], [], [], []
Starting background process QMNC
QMNC started with pid=22, OS id=15792
Thu May  8 06:23:25 2025
ORACLE Instance orcl1 (pid = 13) - Error 607 encountered while recovering transaction (9, 33) on object 775794.
Thu May  8 06:23:25 2025
Errors in file /users/oracle/admin/orcl/bdump/orcl1_smon_15721.trc:
ORA-00607: Internal error occurred while making a change to a data block
ORA-00600: internal error code, arguments: [kddummy_blkchk], [118], [333578], [18019], [], [], [], []
Thu May  8 06:23:26 2025
Completed: ALTER DATABASE OPEN
Thu May  8 06:23:26 2025
Doing block recovery for file 118 block 333578
Block recovery from logseq 74302, block 22 to scn 46740761996
Thu May  8 06:23:26 2025
Recovery of Online Redo Log: Thread 1 Group 1 Seq 74302 Reading mem 0
  Mem# 0: /dev/vgdata/rrac_redo01
Block recovery stopped at EOT rba 74302.33.16
Block recovery completed at rba 74302.33.16, scn 10.3791089036
Thu May  8 06:23:33 2025
Trace dumping is performing id=[cdmp_20250508062324]
Thu May  8 06:25:55 2025
Errors in file /users/oracle/admin/orcl/bdump/orcl1_smon_15721.trc:
ORA-00600: internal error code, arguments: [kddummy_blkchk], [118], [333578], [18019], [], [], [], []
Thu May  8 06:25:58 2025
ORACLE Instance orcl1 (pid = 13) - Error 607 encountered while recovering transaction (9, 33) on object 775794.
Thu May  8 06:27:32 2025
Errors in file /users/oracle/admin/orcl/bdump/orcl1_smon_15721.trc:
ORA-00600: internal error code, arguments: [kddummy_blkchk], [118], [333578], [18019], [], [], [], []
Errors in file /users/oracle/admin/orcl/bdump/orcl1_smon_15721.trc:
Doing block recovery for file 118 block 333578
Block recovery from logseq 74302, block 372 to scn 46740952565
Thu May  8 06:27:41 2025
Errors in file /users/oracle/admin/orcl/bdump/orcl1_smon_15721.trc:
ORA-00600: internal error code, arguments: [kddummy_blkchk], [118], [333578], [18019], [], [], [], []
ORACLE Instance orcl1 (pid = 13) - Error 607 encountered while recovering transaction (9, 33) on object 775794.
Thu May  8 06:27:43 2025
Errors in file /users/oracle/admin/orcl/bdump/orcl1_smon_15721.trc:
ORA-00607: Internal error occurred while making a change to a data block
ORA-00600: internal error code, arguments: [kddummy_blkchk], [118], [333578], [18019], [], [], [], []
Doing block recovery for file 118 block 333578
Block recovery from logseq 74302, block 372 to scn 46740952565
Thu May  8 06:27:45 2025
Recovery of Online Redo Log: Thread 1 Group 1 Seq 74302 Reading mem 0
  Mem# 0: /dev/vgdata/rrac_redo01
Block recovery completed at rba 74302.394.16, scn 10.3791279606
Thu May  8 06:27:47 2025
Errors in file /users/oracle/admin/orcl/bdump/orcl1_smon_15721.trc:
ORA-00600: internal error code, arguments: [kddummy_blkchk], [118], [333578], [18019], [], [], [], []
Thu May  8 06:28:07 2025
Errors in file /users/oracle/admin/orcl/bdump/orcl1_pmon_15690.trc:
ORA-00474: SMON process terminated with error
Thu May  8 06:28:07 2025
PMON: terminating instance due to error 474

这个数据库重启是由于smon进程异常导致数据库关闭,然后rac自动拉起数据库,从而出现了循环重启启动,smon异常的主要原因是由于“ORACLE Instance orcl1 (pid = 13) – Error 607 encountered while recovering transaction (9, 33) on object 775794.”这个表示有实物要回滚,但是遇到了ORA-600 kddummy_blkchk异常,无法完成回滚从而使得smon进程异常进而使得数据实例crash.对于这个问题处理相对比较简单
1. 通过ORA-600 kddummy_blkchk找出来报错对象

ORA-00600: internal error code, arguments: [kddummy_blkchk], [a], [b], 1

ARGUMENTS:
Arg [a] Absolute file number
Arg [b] Block number
Arg 1 Internal error code returned from kcbchk() which indicates the problem encountered.

2. 屏蔽实物回滚,打开数据库,然后对通过dba_extents 查询出来的对象/或者根据事务报错的object信息查询出来对象进行重建,完成本次恢复任务

关于ORA-00600: internal error code, arguments: [kddummy_blkchk], [a], [b], 相关目前已知bug

Bug	Fixed	Description
12349316	11.2.0.4, 12.1.0.2, 12.2.0.1	DBMS_SPACE_ADMIN.TABLESPACE_FIX_BITMAPS fails with ORA-600 [kddummy_blkchk] / ORA-600 [kdBlkCheckError] / ORA-607
17325413	11.2.0.3.BP23, 11.2.0.4.2, 11.2.0.4.BP04, 12.1.0.1.3, 12.1.0.2, 12.2.0.1	Drop column with DEFAULT value and NOT NULL definition ends up with Dropped Column Data still on Disk leading to Corruption
13715932	11.2.0.4, 12.1.0.1	ORA-600 [kddummy_blkchk] [18038] while adding extents to a large datafile
12417369	11.2.0.2.5, 11.2.0.2.BP13, 11.2.0.2.GIPSU05, 11.2.0.3, 12.1.0.1	Block corruption from rollback on compressed table
10324526	10.2.0.5.4, 11.1.0.7.8, 11.2.0.2.3, 11.2.0.2.BP06, 11.2.0.3, 12.1.0.1	ORA-600 [kddummy_blkchk] [6106] / corruption on COMPRESS table in TTS
10113224	11.2.0.3, 12.1.0.1	Index coalesce may generate invalid redo if blocks in the buffer cache are invalid/corrupted
9726702	11.2.0.3, 12.1.0.1	DBMS_SPACE_ADMIN.assm_segment_verify reports HWM related inconsistencies
9724970	11.2.0.1.BP08, 11.2.0.2.2, 11.2.0.2.BP02, 11.2.0.3, 12.1.0.1	Block Corruption with PDML UPDATE. ORA_600 [4511] OERI[kdblkcheckerror] by block check
9711859	10.2.0.5.1, 11.1.0.7.6, 11.2.0.2, 12.1.0.1	ORA-600 [ktsptrn_fix-extmap] / ORA-600 [kdblkcheckerror] during extent allocation caused by bug 8198906
9581240	11.1.0.7.9, 11.2.0.2, 12.1.0.1	Corruption / ORA-600 [kddummy_blkchk] [6101] / ORA-600 [7999] after RENAME operation during ROLLBACK
9350204	11.2.0.3, 12.1.0.1	Spurious ORA-600 [kddummy_blkchk] .. [6145] during CR operations on tables with ROWDEPENDENCIES
9231605	11.1.0.7.4, 11.2.0.1.3, 11.2.0.1.BP02, 11.2.0.2, 12.1.0.1	Block corruption with missing row on a compressed table after DELETE
9119771	11.2.0.2, 12.1.0.1	OERI [kddummy_blkchk]…[6108] from ‘SHRINK SPACE CASCADE’
9019113	11.2.0.1.BP02, 11.2.0.2, 12.1.0.1	ORA-600 [17182] ORA-7445 [memcpy] ORA-600 [kdBlkCheckError] for OLTP COMPRESS table in OLTP Compression REDO during RECOVERY
8951812	11.2.0.2, 12.1.0.1	Corrupt index by rebuild online. Possible OERI [kddummy_blkchk] by SMON
8720802	10.2.0.5, 11.2.0.1.BP07, 11.2.0.2, 12.1.0.1	Add check for row piece pointing to itself (db_block_checking,dbv,rman,analyze)
8331063	11.2.0.3, 12.1.0.1	Corrupt Undo. ORA-600 [2015] in Undo Block During Rollback
6523037	11.2.0.1.BP07, 11.2.0.2.2, 11.2.0.2.BP01, 11.2.0.3, 12.1.0.1	Corruption / ORA-600 [kddummy_blkchk] [6110] on update
8277580	11.1.0.7.2, 11.2.0.1, 11.2.0.2, 12.1.0.1	Corruption on compressed tables during Recovery and Quick Multi Delete (QMD).
9964102	11.2.0.1	OERI:2015 / OERI:kddummy_blkchk / undo corruption from supplementat logging with compressed tables
8613137	11.1.0.7.2, 11.2.0.1	ORA-600 updating table with DEFERRED constraints
8360192	11.1.0.7.6, 11.2.0.1	ORA-600 [kdBlkCheckError] [6110] / corruption from insert
8239658	10.2.0.5, 11.2.0.1	Dump / corruption writing row to compressed table
8198906	10.2.0.5, 11.2.0.1	OERI [kddummy_blkchk] / OERI [5467] for an aborted transaction of allocating extents
7715244	11.1.0.7.2, 11.2.0.1	Corruption on compressed tables. Error codes 6103 / 6110
7662491	10.2.0.4.2, 10.2.0.5, 11.1.0.7.4, 11.2.0.1	Array Update can corrupt a row. Errors OERI[kghstack_free1] or OERI[kddummy_blkchk][6110]
7411865	10.2.0.4.2, 10.2.0.5, 11.1.0.7.1, 11.2.0.1	OERI:13030 / ORA-1407 / block corruption from UPDATE .. RETURNING DML with trigger
7331181	11.2.0.1	ORA-1555 or OERI [kddummy_blkchk] [file#] [block#] [6126] during CR Rollback in query
7293156	11.1.0.7, 11.2.0.1	ORA-600 [2023] by Parallel Transaction Rollback when applying Multi-block undo Head-piece / Tail-piece
7041254	11.1.0.7.5, 11.2.0.1	ORA-19661 during RMAN restore check logical of compressed backup / IOT dummy key
6760697	10.2.0.4.3, 10.2.0.5, 11.1.0.7, 11.2.0.1	DBMS_SPACE_ADMIN.ASSM_SEGMENT_VERIFY does not detect certain segment header block corruption
6647480	10.2.0.4.4, 10.2.0.5, 11.1.0.7.3, 11.2.0.1	Corruption / OERI [kddummy_blkchk] .. [18021] with ASSM
6134368	10.2.0.5, 11.2.0.1	ORA-1407 / block corruption from UPDATE .. RETURNING DML with trigger – SUPERCEDED
6057203	10.2.0.4, 11.1.0.7, 11.2.0.1	Corruption with zero length column (ZLC) / OERI [kcbchg1_6] from Parallel update
6653934	10.2.0.4.2, 10.2.0.5, 11.1.0.7	Dump / block corruption from ONLINE segment shrink with ROWDEPENDENCIES
6674196	10.2.0.4, 10.2.0.5, 11.1.0.6	OERI / buffer cache corruption using ASM, OCFS or any ksfd client like ODM
5599596	10.2.0.4, 11.1.0.6	Block corruption / OERI [kddummy_blkchk] on clustered or compressed tables
5496041	10.2.0.4, 11.1.0.6	OERI[6006] / index corruption on compressed index
5386204	10.2.0.4.1, 10.2.0.5, 11.1.0.6	Block corruption / OERI[kddummy_blkchk] after direct load of ASSM segment
5363584	10.2.0.4, 11.1.0.6	Array insert into table can corrupt redo
4602031	10.2.0.2, 11.1.0.6	Block corruption from UPDATE or MERGE into compressed table
4493447	11.1.0.6	Spurious ORA-600 [kddummy_blkchk] [file#] [block#] [6145] on rollback of array update
4329302	11.1.0.6	OERI [kddummy_blkchk] [file#] [block#] [6145] on rollback of update with logminer
6075487	10.2.0.4	OERI[kddummy_blkchk]..[18020/18026] for DDL on plugged ASSM tablespace with FLASHBACK
4054640	10.1.0.5, 10.2.0.1	Block corruption / OERI [kddummy_blkchk] at physical standby
4000840	10.1.0.4, 10.2.0.1, 9.2.0.7	Update of a row with more than 255 columns can cause block corruption
3772033	9.2.0.7, 10.1.0.4, 10.2.0.1	OERI[ktspfmb_create1] creating a LOB in ASSM using 2k blocksize

ora-600 kdsgrp1 错误描述

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：ora-600 kdsgrp1 错误描述

当 fetch作找不到预期的行时，会引发 ora-600 [kdsgrp1] 错误。该错误在内存中命中，因此可能是仅内存错误或由磁盘损坏导致的错误。

此错误可能表示（但不限于）以下任何情况：

丢失写入
并行 DML 问题
索引损坏
数据块损坏
一致性读取 [CR] 问题
缓冲区缓存损坏

说明 285586.1 - ORA-600 [kdsgrp1] 中
提供了已知问题的完整列表：
每个错误都有一个简短描述，指示遇到它的情况。可以通过选择您的数据库版本来缩短 bug 列表，以仅显示可能影响您的问题。

此问题可能是间歇性的，也可能持续存在，直到修复底层磁盘级别损坏为止。间歇性问题可能是基于内存的（但是，对损坏的间歇性访问可能会与间歇性内存问题相混淆）。

常见的解决方法

如果问题仅在内存中，我们可以尝试通过刷新缓冲区缓存来立即解决问题，但请记住考虑对生产系统的性能影响：

更改系统刷新buffer_cache;

如果我们遇到间歇性一致性读取问题，我们可以尝试禁用 rowCR，这是一种优化，通过在初始化文件中设置 _row_cr=FALSE 来减少查询期间的一致性读取回滚。但是，这可能会导致查询的性能下降。请检查“RowCR hits”/“RowCR attempts”这两个统计信息的比率，以确定是否要使用解决方法。

如果这是索引损坏的结果，那么我们可以删除并重新构建索引。请注意，这将需要在生产系统上有一个 maintenance window。

根本原因确定
现在让我们看看我们如何发现问题的根本原因：查找此问题根本原因的第一步是检查生成的跟踪文件。ora-600 将在跟踪目录中生成跟踪文件，并在事件目录中的事件 ID 下生成事件文件。
跟踪文件的顶部告诉我们遇到错误时正在运行的 SQL：

—–此会话的当前 SQL 语句（sql_id=9mamr7xn4wg7x） —–

这立即向我们显示了访问的数据对象。在跟踪文件中搜索文本字符串 ‘Plan Table’ 将找到此跟踪文件中转储的 SQL 执行计划。对于持久性问题，这允许我们确定哪些索引已被访问，从而确定应验证以检查块损坏的索引：

SQL>分析索引 <OWNER>.<INDEX NAME>在线验证结构;

指数分析。

我们可以采取的另一种方法是使用 trace 文件中包含的 file 和 block 信息。在跟踪文件的顶部，我们将找到有关发现损坏的块的信息：

会话 ID：（3202.5644） 2011-03-19 04：12：16.910
行 07c7c8c7.a 在
文件# 31 块# 510151插槽 11 未找到的延续

此信息可用于识别 dba_extents 中的对象详细信息：

从 dba_extents 中选择 owner、segment_name、segment_type、partition_name，tablespace_name
其中 relative_fno = <文件 id>
并且 <block#> 在 block_id 和（block_id+blocks-1）之间;

然后我们可以验证这个对象，例如一个表和它的所有索引：

分析表 <OWNER>.<TABLE NAME>在线验证结构级联;

请记住，我们可能正在处理不在对象块本身中的永久损坏。这方面的示例包括：

可传输表空间作导致的字典损坏问题：检查 dba_tablespaces 以查看表空间是否已插入。
ASM 磁盘组镜像中的写入丢失 – 最有可能在存在大量 IO 和磁盘重新同步活动时看到。要检查此内容，请运行 dbms_diskgroup.checkfile 以检测镜像差异

如果 analyze 报告没有损坏，则检查表上是否有任何链接的行。如果存在这些，则可能存在未检测到的损坏，并且每当运行 SQL 时，问题都会再次出现。导出表也会检测到此问题。

如果 analyze 和 export 表（在存在链式行的情况下）都报告没有错误，则应将其视为一致性读取问题。

了解问题的性质后，您可以查看已知 bug 列表并确定哪个 bug 与您的条件匹配。如果您无法确定哪个问题影响了您，请向 Oracle 技术支持提交服务请求，并上传所有节点的 RDBMS 和 ASM（如果适用）实例警报日志、生成的任何跟踪和事件文件以及问题性质的完整描述。

Bug	Fixed	Description
32311758	23.1.0.0.0	ORA-600: internal error code, arguments: [kdsgrp1] on spatial physical standby database
32065006	23.1.0.0.0	Sdo_filter() fails with ORA-600: internal error code, arguments: [kdsgrp1]
32022223	19.12, 21.3.0.0.0	Sdo_filter fails with ORA-600: internal error code, arguments: [kdsgrp1]
28392179	19.11, 21.1.0.0.0	ORA-00600 [kdsgrp1] error on standby after intensive insert on the primary DB
29506942	18.11, 18.18, 19.8, 20.1	sdo_filter fails with ORA-600: internal error code, arguments: [kdsgrp1]
29311927	18.11, 18.18, 19.8, 20.1	sdo_filter fails with ORA-600: internal error code, arguments: [kdsgrp1]
28547478	12.2.0.1.DBRU:200714, 18.11, 18.18, 19.2, 20.1	ORA-600 [kdsgrp1] When Running Workload
27869764	19.1	Sdo_filter() call coredumps with [kdsgrp1] exception [optimized mbrs]
27397048	12.1.0.2.190115, 12.2.0.1.DBRU:190115, 18.18, 18.5, 19.1	Intermittent ORA-600[kdsgrp1] Raised By Query Using Index
26203182	11.2.0.4.200114, 12.1.0.2.190716, 12.2.0.1.DBRU:190115, 18.1	Lost Writes on ZFS if DNFS is enabled causing several Internal Errors. ORA-600 [kdsgrp1] ORA-8103 ORA-600 [3020] ORA-752 ORA-756
22581771	12.2.0.1.DBRU:180417, 18.1	ORA-600 [kdsgrp1] On Domain Index With Concurrent Insert And Select (With Clause)
21180699	18.1	ORA-7445/ ORA-00600 argument [kdibowrite()] / [kdibc3position()+78] / [20003] / [kcfrbd_3] / [25027] [kdsgrp1] with Execution plan ‘BITMAP’ access
22267274	12.2.0.1	CDB: Hit ORA-600 [kdsgrp1] and ORA-600 [4042]
17273253	12.1.0.1.1, 12.2.0.1	Various ORA-600 corruption errors with ASM
16195231	11.2.0.3.BP21, 11.2.0.4, 12.1.0.2, 12.2.0.1	ORA-7445 / ORA-600 from COMPRESSED table with LONG column
14576755	12.1.0.1.4, 12.1.0.2, 12.2.0.1	Corruption type ORA-600 errors from heavy concurrent DML on index cluster table
33005241	19.16, 21.7	ORA-00600 [kdsgrp1] error when using row CR
33599665	19.17	ORA-600 [kdsgrp_lost_piece] / ORA-600 [kdsgrp1-kdsgrp] While Running Flashback Query on FDA Enabled Table
31843845	19.13, 21.5	ORA-600 [kdsgrp1] Error or Wrong / Duplicate Results When Advanced Compressed Index Skip Scan Used to Access Rows
32417227	19.12	OLTP Compression Lock Bit Not Respected In Uncompressed Blocks
31228670	12.1.0.2.201020, 12.2.0.1.DBRU:201020, 18.12, 19.9	Corruption LOST Write : Rebalance disk resync causing lost write, mirror mismatches , several errors can be reported
31192039	12.2.0.1.DBRU:201020, 18.12, 18.18, 19.9	ORA-1554 and/or ORA-600 [kdsgrp1] While Deleting From A Compressed Index
31642462	19.14	ORA-600 [kdsgrp1-kdsgrp] when doing a version query using rowid having large row data with hybrid columnar compression enabled.
30651570	18.14, 18.18, 19.10	ORA-600: [kdsgrp1] After INSERT With APPEND Hint In Compressed Partitioned Table
32596207	21.0	ORA-600[kdsgrp1] failure using sdo_filter() function
29428230	18.11, 18.18, 19.8, 20.1	sdo_filter fails with ORA-600: internal error code, arguments: [kdsgrp1]
29362596	18.11.0.0.200714DBRU, 18.11, 18.18, 19.8.0.0.200714DBRU, 19.8, 20.1	sdo_filter fails with ORA-600: internal error code, arguments: [kdsgrp1]
29350868	18.11.0.0.200714DBRU, 18.11, 18.18, 19.8.0.0.200714DBRU, 19.8, 20.1	sdo_filter fails with ORA-600: internal error code, arguments: [kdsgrp1]
29139070	18.11.0.0.200714DBRU, 18.11, 18.18, 19.8.0.0.200714DBRU, 19.8, 20.1	very small adjacent insert causes index corruption ORA-600[kdsgrp1]
29048605	19.3.0.0.190416DBRU, 19.3, 20.1	index truncation causes index corruption ORA-600[kdsgrp1]
28881035	19.2, 19.2.0.0.181005R, 20.1	very small update causes index corruption ORA-600[kdsgrp1]
28802077	19.2, 19.2.0.0.181005R, 20.1	sdo_filter() fails with ORA-600[kdsgrp1]
27063461	19.11, 20.1	Physical Standby Hits ORA-600[kdbdmp_full:non-KDDBTDATA block. Use kcbtdu for it.]
28511632	23.4	Corruption LOST Write : Incomplete RMAN DUPLICATE can allow data file overwrites at Source database
27394954	19.1	sdo_filter fails with ORA-600 [kdsgrp1] after delete,insert,delete,insert,commit
27658186	12.2.0.1, 12.2.0.1.DBRU:190115, 18.5	ORA-600 [kdsgrp1] / Some rows not indexed in Text index in highly concurrent environment
24699619	12.2.0.1.171121DBRU, 12.2.0.1.171130WINDBBP, 12.2.0.1.DBRU:171121, 18.1	xdbstress hit ora 600 [kdsgrp1]
12690729	18.1	ORA-600 [kdsgrp1] errors when the active standby database recovery is enabled using CURRENT LOGFILE
22575209	12.2.0.1	ORA-600 [kdsgrp1] ORA-600 [25027] ORA-8103 ORA-1578 ORA-3254 in ADG Standby Database for Full Scan on ASSM segment – superseded
22519146	12.1.0.2.171017, 12.2.0.1	ORA-600 [kdsgrp1] or ORA-600 [kdsgrpcalcblockcount: hwmbno<=dbabno] or ORA-8103 in 12c on HCC Table in EXADATA
22241601	12.2.0.1	ORA-600 [kdsgrp1] ORA-1555 / ORA-600 [ktbdchk1: bad dscn] due to Invalid Commit SCN in INDEX block
21973601	12.2.0.0, 12.2.0.1	Querying a partitioned table may fail with ORA-00600 [kdsgrp1]
21634686	12.2.0.1	ORA-600 [kdsgrp1] / ORA-600 [ktfbhget:clsviol_kcbgcur_9] With Hybrid Columnar Compression (HCC)
21532755	11.2.0.4.171017, 12.1.0.2.171017, 12.2.0.1	ORA-600 [25027] By Concurrent queries while Create Index Online or ORA-8102 Table/Index mistmatch after Create Index Online or ONLINE_INDEX_CLEAN wait for DMLs
21096955	12.2.0.1	ORA-600 [kdsgrp1] / ORA-600 [ktfbhget:clsviol_kcbgcur_9] With Hybrid Columnar Compression (HCC)
19689979	11.2.0.4.170718, 12.1.0.2.160119, 12.1.0.2.DBBP07, 12.2.0.1	ORA-8103 or ORA-600 [ktecgsc:kcbz_objdchk] or Wrong Results on PARTITION table after TRUNCATE in 11.2.0.4 or above
19630914	12.2.0.1	ORA-600 [kdsgrp1] And Other Errors ORA-600 [6033] When BigSCN Testing Is Enabled
19614585	11.2.0.4.BP17, 12.1.0.2.DBBP03, 12.2.0.1	Wrong Results / ORA-600 [kksgaGetNoAlloc_Int0] / ORA-600 [12406] / ORA-7445 / ORA-8103 / ORA-1555 from query on RAC ADG Physical Standby Database
18607546	11.2.0.4.6, 11.2.0.4.BP16, 12.1.0.2.3, 12.1.0.2.DBBP06, 12.2.0.1	ORA-600 [kdblkcheckerror]..[6266] corruption with self-referenced chained row. ORA-600 [kdsgrp1] / Wrong Results / ORA-8102
18311351	12.2.0.1	ORA-1/ORA-10388 ORA-7445 [kdzsbuffercupiece_col] ORA-600 [kdsgrp1]/ORA-1499 Wrong Results, Index Inconsistency after Parallel Direct Path Insert of HCC table in EXADATA
17779978	12.2.0.1	ORA-00600 [kdsgrp1] & ORA-7445 [hshhsv] & ORA-7445 [pkrcd] errors on CDB
17761775	11.2.0.3.9, 11.2.0.3.BP22, 11.2.0.4.2, 11.2.0.4.BP03, 12.1.0.1.3, 12.1.0.2, 12.2.0.1	ORA-600 [kclchkblkdma_3] ORA-600 [3020] or ORA-600 [kcbchg1_16] Join of temp and permanent table in RAC might lead to corruption – superseded
17357359	12.1.0.2, 12.2.0.1	ORA-600 [kdsgrp1] during fetch by rowid
17160362	12.1.0.2, 12.2.0.1	ORA-600 [kdsgrp1] & [kclchkblk_3] & [kclchkblkdma_3] in rdbms
16849623	12.1.0.2, 12.2.0.1	ORA-600 [kdsgrp1] While Running Workload On Tables With Chained Rows
16698629	12.1.0.2, 12.2.0.1	ORA-600 [kdsgrp1] executing SELECT on table modified by a loosely coupled clusterwide global transaction
16555614	12.1.0.2, 12.2.0.1	mdidxridchk() causes buffer overrun problem when more than 4000 rows selected
16345143	12.2.0.1	Event 10231 does not skip row for IOT with non-existent nrid
14044260	12.1.0.2, 12.2.0.1	Update DML with long bind LOB that moves row to new partition fails with ORA-600 [kdsgrp1] – superseded
17449815	11.2.0.4.4, 11.2.0.4.BP11, 12.1.0.2, 12.2.0.1	ORA-8102 ORA-1499 after ORA-1/ORA-2291 by MERGE with DML ERROR LOGGING
17204397	12.1.0.2, 12.2.0.1	ORA-8005 ORA-8103 ORA-1410 ORA-600 [kdsgrp1] on Bitmap Index. Root Block may be repeatedly pinned/unpinned
16844448	11.2.0.3.9, 11.2.0.3.BP22, 11.2.0.4, 12.1.0.2, 12.2.0.1	ORA-600 [3020] after flashback database in a RAC
16563781	12.1.0.2, 12.1.0.2.180116, 12.2.0.1	version query may return wrong result on a table in TTS tablespace
21425496	11.2.0.4.190416, 12.1.0.1, 12.1.0.2.190716	ORA-752 or ORA-600 [3020] on recovery of Block Cleanout Operation OP:4.6
17518816	12.1.0.0, 12.1.0.1	ORA-600 [kdsgrp1] on select statements on a Active Dataguard Standby database
14790903	11.2.0.4, 12.1.0.1	ora 600 [kdsgrp1]
14527172	12.1.0.1	ORA-600 [4097] And [kdsgrp1] After unplugging and plugging the PDB In RAC Environment
13614906	12.1.0.1	ORA-600 [kdsgrp1] due to missing weak changes from an XA transaction in RAC – superceded
13399500	11.2.0.3.BP15, 11.2.0.4, 12.1.0.1	ORA-600 [kdsgrp1] when updating a chained rows on a ehcc table
13146182	11.2.0.2.11, 11.2.0.2.BP17, 11.2.0.3.10, 11.2.0.3.BP07, 11.2.0.4, 12.1.0.1	ORA-1499 ORA-8102 ORA-600 [kdsgrp1] Bitmap Index / Table mismatch
12821418	11.2.0.3.8, 11.2.0.3.BP18, 11.2.0.4, 12.1.0.1	Direct NFS appears to be sending zero length windows to storage device. It may also cause Lost Writes
12619529	11.2.0.3.BP18, 11.2.0.4, 12.1.0.1	ORA-600[kdsgrp1] from SELECT on plugged in tablespace with FLASHBACK
12330911	12.1.0.1	EXADATA LSI firmware for lost writes
10633840	11.2.0.2.7, 11.2.0.2.BP17, 11.2.0.3, 12.1.0.1	ORA-1502 on insert statement on INTERVAL partitioned table. ORA-8102 / ORA-1499 Index inconsistency
10245259	11.2.0.2.BP03, 11.2.0.3, 12.1.0.1	PARALLEL INSERT with +NOAPPEND hint or if PARALLEL INSERT plan is executed in SERIAL corrupts index and causes wrong results
10209232	11.1.0.7.7, 11.2.0.1.BP08, 11.2.0.2.1, 11.2.0.2.BP02, 11.2.0.2.GIBUNDLE01, 11.2.0.3, 12.1.0.1	ORA-1578 / ORA-600 [3020] Corruption. Misplaced Blocks and Lost Write in ASM
10205230	11.2.0.1.6, 11.2.0.1.BP09, 11.2.0.2.2, 11.2.0.2.BP04, 11.2.0.3, 12.1.0.1	ORA-600 / corruption possible during shutdown in RAC
9770451	10.2.0.5.3, 11.2.0.2.1, 11.2.0.2.BP02, 11.2.0.3, 12.1.0.1	ORA-600 [20022] with bitmap indexes
9734539	11.2.0.2, 12.1.0.1	ORA-8102 / ORA-1499 corrupt index after update/merge using QUERY REWRITE
9469117	10.2.0.5.4, 11.2.0.1.BP04, 11.2.0.2, 12.1.0.1	Corrupt index after PDML executed in serial. Wrong results. OERI[kdsgrp1]/ORA-1499 by analyze
9457185	11.2.0.1.BP12, 11.2.0.2, 12.1.0.1	Intermittent ORA-600 [kdsgrp1] during CR read
9231605	11.1.0.7.4, 11.2.0.1.3, 11.2.0.1.BP02, 11.2.0.2, 12.1.0.1	Block corruption with missing row on a compressed table after DELETE
9145541	11.1.0.7.4, 11.2.0.1.2, 11.2.0.2, 12.1.0.1	OERI[25027]/OERI[4097]/OERI[4000]/ORA-1555 in plugged datafile after CREATE CONTROLFILE in 11g
9061269	11.2.0.2, 12.1.0.1	ORA-600 [kdsgrp1] executing CTX_QUERY.COUNT_HITS during concurrent sync Text index
8951812	11.2.0.2, 12.1.0.1	Corrupt index by rebuild online. Possible OERI [kddummy_blkchk] by SMON
8837919	11.2.0.2, 12.1.0.1	DBV / RMAN enhanced to detect ASSM blocks with ktbfbseg but not ktbfexthd flag set as in Bug 8803762
8803762	11.1.0.7.6, 11.2.0.1.2, 11.2.0.1.BP06, 11.2.0.2, 12.1.0.1	ORA-600[kdsgrp1], ORA-600[25027] or wrong results on 11g database upgrade from 9i
8771916	10.2.0.5.3, 11.1.0.7.6, 11.2.0.1.BP12, 11.2.0.2, 12.1.0.1	OERI [kdsgrp1] during CR read
8635179	10.2.0.5, 11.2.0.2, 12.1.0.1	Solaris: directio may be disabled for RAC file access. Corruption / Lost Write
8597106	11.2.0.1.BP06, 11.2.0.2, 12.1.0.1	Lost Write in ASM when normal redundancy is used
8546356	10.2.0.5.1, 11.2.0.1.3, 11.2.0.1.BP07, 11.2.0.2, 12.1.0.1	ORA-8102/ORA-1499/OERI[kdsgrp1] Composite Partitioned Index corruption after rebuild ONLINE in RAC
7710827	11.2.0.2, 12.1.0.1	Index rebuild or Merge partition causes wrong results in concurrent reads instead of ORA-8103
7705591	10.2.0.5, 11.2.0.1.1, 11.2.0.1.BP04, 11.2.0.2, 12.1.0.1	Corruption with self-referenced row in MSSM tablespace. Wrong Results / OERI[6749] / ORA-8102
7251049	11.2.0.1.BP08, 11.2.0.2, 12.1.0.1	Corruption in bitmap index introduced when using transportable tablespaces
16579042	11.2.0.4	ORA-600 [kjbmpocr:alh] ORA-600 [kclchkblkdma_3] by LMS in RAC which may lead to corruption
9527635	11.2.0.1.BP04, 11.2.0.2, 12.1.0.1	ORA-00600 [kdsgrp1] On Exadata
8650661	11.1.0.7.2, 11.2.0.1	OERI / corruption type errors using global transactions in RAC
8588540	11.1.0.7.2, 11.2.0.1	Corruption / ORA-8102 in RAC with loopback DB links between instances
7682186	11.2.0.1	ORA-600[kdsgrp1] on consistent read in RAC with global transaction
7329252	10.2.0.4.4, 10.2.0.5, 11.1.0.7.5, 11.2.0.1	ORA-8102/ORA-1499/OERI[kdsgrp1] Index corruption after rebuild index ONLINE
7289224	11.2.0.1	ORA-600 [kdsgrp1] on CR read with parallel query
6791996	11.2.0.1	ORA-600 errors for a DELETE with self referencing FK constraint and BITMAP index
6772911	10.2.0.5, 11.1.0.7.3, 11.2.0.1	OERI[12700] OERI[qertbFetchByRowID] OERI[kdsgrp1] due to bad CR rollback of INDEX block
6445948	10.2.0.4.4, 10.2.0.5, 11.1.0.7.8, 11.2.0.1	Intermitent ORA-600 [kdsgrp1] accessing table with a LONG
6404058	10.2.0.5, 11.1.0.7, 11.2.0.1	OERI:12700 OERI:kdsgrp1 OERI:qertbFetchByRowID wrong results from CR rollback of split index leaf
6129296	11.2.0.1	ORA-600 [kdsgrp1] by PARALLEL select for update with LOB
5621677	10.2.0.4, 11.1.0.6	Logical corruption with PARALLEL update
5374225	10.2.0.4, 11.1.0.6	SDO_FILTER query fails with OERI[kdsgrp1]
5368945	10.2.0.5, 11.1.0.6	ORA-600 [kdsgrp1] on Index Organized Table with Overflow
4883635	10.2.0.4, 11.1.0.6	MERGE (with DELETE) can produce wrong results or Logical corruption in chained rows
3408192	9.2.0.6, 10.1.0.3, 10.2.0.1	Heavy concurrent DML scenarios can cause $R table to contain deleted rowids

ORA-600 krhpfh_03-1208

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：ORA-600 krhpfh_03-1208

最近一个客户咨询一个问题,他正常的drop tbs,结果触发ORA-600 krhpfh_03-1208 错误,导致数据库crash

Wed Mar 26 14:33:20 2025
Thread 1 cannot allocate new log, sequence 478485
Checkpoint not complete
  Current log# 2 seq# 478484 mem# 0: /apps/data/oracle/orcl/redo02.log
Thread 1 advanced to log sequence 478485 (LGWR switch)
  Current log# 3 seq# 478485 mem# 0: /apps/data/oracle/orcl/redo03.log
Wed Mar 26 14:35:06 2025
Wed Mar 26 14:35:06 2025
drop tablespace XFF_MON_2016 including contents and datafiles cascade constraint
Read of datafile '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf' (fno 17) header failed with ORA-01208
Rereading datafile 17 header failed with ORA-01208
Errors in file /apps/svr/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_188213.trc  (incident=7677):
ORA-00600: 内部错误代码, 参数: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935],[],[],[],[],[]
ORA-01110: 数据文件 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-01122: 数据库文件 17 验证失败
ORA-01110: 数据文件 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-01208: 数据文件是旧的版本 - 不能访问当前版本
Incident details in: /apps/svr/oracle/diag/rdbms/orcl/orcl/incident/incdir_7677/orcl_ora_188213_i7677.trc
Wed Mar 26 14:35:07 2025
Trace dumping is performing id=[cdmp_20250326143507]
ORA-600 signalled during: drop tablespace XFF_MON_2016 including contents and datafiles cascade constraint...
Errors in file /apps/svr/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_188213.trc  (incident=7678):
ORA-00600: 内部错误代码, 参数: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935],[],[],[],[],[]
ORA-01110: 数据文件 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-00600: 内部错误代码, 参数: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935],[],[],[],[],[]
ORA-01110: 数据文件 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-01122: 数据库文件 17 验证失败
ORA-01110: 数据文件 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-01208: 数据文件是旧的版本 - 不能访问当前版本
Incident details in: /apps/svr/oracle/diag/rdbms/orcl/orcl/incident/incdir_7678/orcl_ora_188213_i7678.trc
Wed Mar 26 14:35:08 2025
Sweep [inc][7678]: completed
Sweep [inc][7677]: completed
Sweep [inc2][7677]: completed
Wed Mar 26 14:35:09 2025
Thread 1 cannot allocate new log, sequence 478486
Checkpoint not complete
  Current log# 3 seq# 478485 mem# 0: /apps/data/oracle/orcl/redo03.log
Errors in file /apps/svr/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_188213.trc  (incident=7679):
ORA-00600: 内部错误代码, 参数: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935],[],[],[],[],[]
ORA-01110: 数据文件 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-00600: 内部错误代码, 参数: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935],[],[],[],[],[]
ORA-01110: 数据文件 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-00600: 内部错误代码, 参数: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935],[],[],[],[],[]
ORA-01110: 数据文件 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-01122: 数据库文件 17 验证失败
ORA-01110: 数据文件 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-01208: 数据文件是旧的版本 - 不能访问当前版本
Incident details in: /apps/svr/oracle/diag/rdbms/orcl/orcl/incident/incdir_7679/orcl_ora_188213_i7679.trc
Trace dumping is performing id=[cdmp_20250326143511]
Errors in file /apps/svr/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_188213.trc:
ORA-00600: 内部错误代码, 参数: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935],[],[],[],[],[]
ORA-01110: 数据文件 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-00600: 内部错误代码, 参数: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935],[],[],[],[],[]
ORA-01110: 数据文件 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-00600: 内部错误代码, 参数: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935],[],[],[],[],[]
ORA-01110: 数据文件 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-01122: 数据库文件 17 验证失败
ORA-01110: 数据文件 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-01208: 数据文件是旧的版本 - 不能访问当前版本
Thread 1 advanced to log sequence 478486 (LGWR switch)
  Current log# 1 seq# 478486 mem# 0: /apps/data/oracle/orcl/redo01.log
Wed Mar 26 14:35:13 2025
Errors in file /apps/svr/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_188213.trc  (incident=15551):
ORA-00603: ORACLE server session terminated by fatal error
ORA-00600: internal error code, arguments: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935]
ORA-01110: data file 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-00600: internal error code, arguments: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935]
ORA-01110: data file 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-00600: internal error code, arguments: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935]
ORA-01110: data file 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-01122: database file 17 failed verification check
ORA-01110: data file 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-01208: data file is an old version - not accessing current version
Incident details in: /apps/svr/oracle/diag/rdbms/orcl/orcl/incident/incdir_15551/orcl_ora_188213_i15551.trc
Errors in file /apps/svr/oracle/diag/rdbms/orcl/orcl/incident/incdir_15551/orcl_ora_188213_i15551.trc:
ORA-00603: ORACLE server session terminated by fatal error
ORA-00600: internal error code, arguments: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935]
ORA-01110: data file 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-00600: internal error code, arguments: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935]
ORA-01110: data file 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-00600: internal error code, arguments: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935]
ORA-01110: data file 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-01122: database file 17 failed verification check
ORA-01110: data file 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-01208: data file is an old version - not accessing current version
Trace dumping is performing id=[cdmp_20250326143514]
Wed Mar 26 14:35:15 2025
Errors in file /apps/svr/oracle/diag/rdbms/orcl/orcl/trace/orcl_pmon_139367.trc  (incident=7224):
ORA-00600: internal error code, arguments: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935]
ORA-01110: data file 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
Incident details in: /apps/svr/oracle/diag/rdbms/orcl/orcl/incident/incdir_7224/orcl_pmon_139367_i7224.trc
Errors in file /apps/svr/oracle/diag/rdbms/orcl/orcl/trace/orcl_pmon_139367.trc:
ORA-00600: internal error code, arguments: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935]
ORA-01110: data file 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
Wed Mar 26 14:35:19 2025
drop tablespace XFF_MON_2016 including contents and datafiles cascade constraint
Wed Mar 26 14:35:20 2025
DBW0 (ospid: 139390): terminating the instance due to error 472
Instance terminated by DBW0, pid = 139390

这个报错信息看,但是发起drop tbs之后,数据库应该是检查file 17号文件的状态,发现这个版本状态过旧(ORA-01208: 数据文件是旧的版本),由于某种原因报出来了krhpfh_03-1208,导致数据库crash了,然后他尝试启动数据库报ORA-01113: file 17 needs media recovery

Wed Mar 26 17:11:00 2025
Starting ORACLE instance (normal)
Wed Mar 26 17:11:17 2025
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Picked latch-free SCN scheme 3
Wed Mar 26 17:11:28 2025
Using LOG_ARCHIVE_DEST_1 parameter default value as USE_DB_RECOVERY_FILE_DEST
Autotune of undo retention is turned on. 
IMODE=BR
ILAT =168
LICENSE_MAX_USERS = 0
SYS auditing is disabled
Starting up:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Partitioning, Oracle Label Security, OLAP, Data Mining,
Oracle Database Vault and Real Application Testing options.
Using parameter settings in server-side spfile /apps/svr/oracle/product/11.2.0/dbhome_1/dbs/spfileorcl.ora
System parameters with non-default values:
  processes                = 1000
  sga_target               = 0
  memory_target            = 66048M
  memory_max_target        = 66048M
  control_files            = "/apps/data/oracle/orcl/control01.ctl"
  control_files            = "/apps/svr/oracle/flash_recovery_area/orcl/control02.ctl"
  db_block_size            = 8192
  compatible               = "11.2.0.0.0"
  db_recovery_file_dest    = "/apps/svr/oracle/flash_recovery_area"
  db_recovery_file_dest_size= 3882M
  undo_tablespace          = "UNDOTBS1"
  remote_login_passwordfile= "EXCLUSIVE"
  db_domain                = ""
  dispatchers              = "(PROTOCOL=TCP) (SERVICE=orclXDB)"
  audit_file_dest          = "/apps/svr/oracle/admin/orcl/adump"
  audit_trail              = "DB"
  db_name                  = "orcl"
  open_cursors             = 300
  pga_aggregate_target     = 0
  diagnostic_dest          = "/apps/svr/oracle"
Wed Mar 26 17:11:29 2025
PMON started with pid=2, OS id=28315 
Wed Mar 26 17:11:29 2025
VKTM started with pid=3, OS id=28317 at elevated priority
VKTM running at (10)millisec precision with DBRM quantum (100)ms
Wed Mar 26 17:11:29 2025
GEN0 started with pid=4, OS id=28324 
Wed Mar 26 17:11:29 2025
DIAG started with pid=5, OS id=28326 
Wed Mar 26 17:11:29 2025
DBRM started with pid=6, OS id=28328 
Wed Mar 26 17:11:29 2025
PSP0 started with pid=7, OS id=28330 
Wed Mar 26 17:11:29 2025
DIA0 started with pid=9, OS id=28334 
Wed Mar 26 17:11:29 2025
MMAN started with pid=8, OS id=28336 
Wed Mar 26 17:11:29 2025
DBW0 started with pid=10, OS id=28338 
Wed Mar 26 17:11:29 2025
DBW1 started with pid=11, OS id=28340 
Wed Mar 26 17:11:29 2025
DBW2 started with pid=12, OS id=28342 
Wed Mar 26 17:11:29 2025
DBW3 started with pid=13, OS id=28344 
Wed Mar 26 17:11:29 2025
LGWR started with pid=14, OS id=28346 
Wed Mar 26 17:11:29 2025
CKPT started with pid=15, OS id=28348 
Wed Mar 26 17:11:29 2025
SMON started with pid=16, OS id=28350 
Wed Mar 26 17:11:29 2025
RECO started with pid=17, OS id=28352 
Wed Mar 26 17:11:29 2025
MMON started with pid=18, OS id=28354 
starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
Wed Mar 26 17:11:29 2025
MMNL started with pid=19, OS id=28356 
starting up 1 shared server(s) ...
ORACLE_BASE from environment = /apps/svr/oracle
Wed Mar 26 17:11:29 2025
ALTER DATABASE   MOUNT
Wed Mar 26 17:11:32 2025
Sweep [inc][7679]: completed
Sweep [inc][7224]: completed
Sweep [inc][15551]: completed
Sweep [inc2][7679]: completed
Sweep [inc2][7678]: completed
Sweep [inc2][7224]: completed
Sweep [inc2][15551]: completed
Successful mount of redo thread 1, with mount id 1724539585
Database mounted in Exclusive Mode
Lost write protection disabled
Completed: ALTER DATABASE   MOUNT
Wed Mar 26 17:11:34 2025
ALTER DATABASE OPEN
Errors in file /apps/svr/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_28406.trc:
ORA-01113: file 17 needs media recovery
ORA-01110: data file 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-1113 signalled during: ALTER DATABASE OPEN...

由于现场已经破坏,无法分析当时库的情况和17号文件的具体情况做进一步判断,只能通过日志记录下这个类型的错误.
在oracle中关于ORA-600 krhpfh_03的bug也比较多

ORA-00069: cannot acquire lock — table locks disabled for xxxx

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：ORA-00069: cannot acquire lock — table locks disabled for xxxx

在oracle数据库中删除用户遭遇ORA-00069: cannot acquire lock — table locks disabled for HR_XXX_01错误

SQL>  drop user XFF cascade;
 drop user XFF cascade
*
ERROR at line 1:
ORA-00604: error occurred at recursive SQL level 1
ORA-00069: cannot acquire lock -- table locks disabled for HR_XXX_01

关于ORA-00069错误解释

[oracle@xifenfei.com ~]$ oerr ora 00069
00069, 00000, "cannot acquire lock -- table locks disabled for %s"
// *Cause: A command was issued that tried to lock the table indicated in
//         the message. Examples of commands that can lock tables are:
//         LOCK TABLE, ALTER TABLE ... ADD (...), and so on.
// *Action: Use the ALTER TABLE ... ENABLE TABLE LOCK command, and retry
//          the command.

尝试lock表,直接hang,强制终止

SQL> alter table XFF.HR_XXX_01 enable table lock; 



^Calter table XFF.HR_XXX_01 enable table lock
*
ERROR at line 1:
ORA-01013: user requested cancel of current operation

查询tab$.flags的值

SQL> col object_name for a30
SQL> set lines 150
SQL> select x. object_name,obj#, flags
  2  from sys.tab$,(
  3  select object_name, object_id
  4  from dba_objects
  5  where owner='XFF'
  6  and object_name in ('HR_XXX_01','HR_XXXCONTROL','XXXLZB_JD1')
  7  and object_type = 'TABLE') x
  8  where obj# = x.object_id;

OBJECT_NAME                          OBJ#      FLAGS
------------------------------ ---------- ----------
XXXLZB_JD1                         246416 1073742353
HR_XXXCONTROL                      246421 1073742353
HR_XXX_01                          246424 1073742359

发现报错表的flags和其他表不一样(其他表为1073742353,而报错表为1073742359),对于这种情况官方给出来的解决方法,关闭库,确保没有任何额外会话连接上来
ora-00069

因为本身要重启库维护,直接把库启动到upgrade模式进行操作

[oracle@xifenfei.com ~]$ sqlplus / as sysdba

SQL*Plus: Release 19.0.0.0.0 - Production on Fri Feb 14 20:29:28 2025
Version 19.24.0.0.0

Copyright (c) 1982, 2024, Oracle.  All rights reserved.


Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.24.0.0.0

SQL> alter system checkpoint;

System altered.

SQL> /

System altered.

SQL> shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup upgrade;
ORACLE instance started.

Total System Global Area 4.2950E+10 bytes
Fixed Size                 23149944 bytes
Variable Size            9529458688 bytes
Database Buffers         3.3286E+10 bytes
Redo Buffers              111067136 bytes
Database mounted.
Database opened.

SQL> startup upgrade;
ORACLE instance started.

Total System Global Area 4.2950E+10 bytes
Fixed Size                 23149944 bytes
Variable Size            9529458688 bytes
Database Buffers         3.3286E+10 bytes
Redo Buffers              111067136 bytes
Database mounted.
Database opened.
SQL>  drop user XFF cascade;
 drop user FZHR cascade
*
ERROR at line 1:
ORA-00604: error occurred at recursive SQL level 1
ORA-00069: cannot acquire lock -- table locks disabled for HR_XXX_01


SQL> alter table XFF.HR_XXX_01 enable table lock; 

Table altered.

SQL>  drop user XFF cascade;

User dropped.

SQL>

ORA-65088: database open should be retried

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：ORA-65088: database open should be retried

在12.2以及后续的cdb版本中,如果重建ctl并且resetlogs库,很可能会遇到ORA-65088: database open should be retried错误

SQL> startup nomount force pfile='/<path>/<filename>.ora';
ORACLE instance started.

Total System Global Area 1593835520 bytes
Fixed Size                  8793256 bytes
Variable Size             402654040 bytes
Database Buffers         1174405120 bytes
Redo Buffers                7983104 bytes
SQL> !vi ctl.sql

SQL> @ctl.sql

Control file created.

SQL> select count(*) ,fhsta from x$kcvfh group by fhsta;

  COUNT(*)      FHSTA
---------- ----------
        11      32768
         4      40960

SQL> select count(*) ,FHSCN from x$kcvfh group by FHSCN;

  COUNT(*) FHSCN
---------- --------------------
         3 1820866
         4 2281969
         4 2281978
         4 2281982

SQL> select file#,error from v$datafile_header where length(error)>=1;

no rows selected

SQL> select count(*) ,fhrba_seq from x$kcvfh group by fhrba_seq;

  COUNT(*)  FHRBA_SEQ
---------- ----------
         3         20
        12         32

SQL> recover database using backup controlfile until cancel;
ORA-00279: change 2281978 generated at 09/19/2018 00:52:00 needed for thread 1
ORA-00289: suggestion : /<path>/1_32_981800889.dbf
ORA-00280: change 2281978 for thread 1 is in sequence #32


Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
/<path>/1_32_981800889.dbf
ORA-00279: change 2282008 generated at 09/19/2018 00:52:13 needed for thread 1
ORA-00289: suggestion : /<path>/1_33_981800889.dbf
ORA-00280: change 2282008 for thread 1 is in sequence #33
ORA-00278: log file '/<path>/1_32_981800889.dbf' no longer needed for this recovery


Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
cancel
Media recovery cancelled.    << Expected message "Media recovery complete." !!
SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       MOUNTED
         3 _###_UNKNOWN_PDB_#_3           MOUNTED
         4 _###_UNKNOWN_PDB_#_4           MOUNTED
SQL> alter database open resetlogs;
alter database open resetlogs
*
ERROR at line 1:
ORA-00603: ORACLE server session terminated by fatal error
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-65088: database open should be retried
Process ID: 32688
Session ID: 10 Serial number: 38416

alert日志类似错误

 Dictionary check beginning
 Pluggable Database <pdb_name_1> (#3) found in data dictionary,
 but not in the control file. Adding it to control file.
 Pluggable Database <pdb_name_2> (#4) found in data dictionary,
 but not in the control file. Adding it to control file.
 Tablespace '<tablespace_name>' #3 found in data dictionary,
 but not in the controlfile. Adding to controlfile.
 --
 File 8 not verified due to error ORA-01122
 File 9 not verified due to error ORA-01122
 File 11 not verified due to error ORA-01122
 File 16 not verified due to error ORA-01122
 File 17 not verified due to error ORA-01122
 File 18 not verified due to error ORA-01122
 File 19 not verified due to error ORA-01122
 File 20 not verified due to error ORA-01122
  --
 ORA-65088: database open should be retried
 2018-09-19T01:00:54.083814+05:30
 Errors in file /<path>/trace/<oracle_sid>_ora_12412.trc:
 ORA-65088: database open should be retried
 Error 65088 happened during db open, shutting down database
 Errors in file /<path>/trace/<oracle_sid>_ora_12412.trc  (incident=12289) (PDBNAME=CDB$ROOT):
 ORA-00603: ORACLE server session terminated by fatal error
 ORA-01092: ORACLE instance terminated. Disconnection forced
 ORA-65088: database open should be retried

出现这类故障的原因是由于:
we see that the created controlfile is not aware of PDB and open resetlogs process trying to add information in newly created file . Hence, recovery process ,in newly created controlfile didn’t applied the archives to datafiles part of PDB which says later it will ask for recovery once controlfile is aware of PDB files During the resetlogs process, its pushing the required information to controlfile and shutting the database with suggestion to re-try opening the DB.

$ sqlplus "/as sysdba"

SQL*Plus: Release 12.2.0.1.0 Production on Wed Sep 19 01:34:01 2018

Copyright (c) 1982, 2016, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup nomount force pfile='/<path>/<filename>.ora';
ORACLE instance started.

Total System Global Area 1593835520 bytes
Fixed Size                  8793256 bytes
Variable Size             402654040 bytes
Database Buffers         1174405120 bytes
Redo Buffers                7983104 bytes
SQL> alter database mount;

Database altered.

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       MOUNTED
         3 PDB1                           MOUNTED
         4 APDB                           MOUNTED
        
SQL> select count(*) ,FHSCN from x$kcvfh group by FHSCN;

  COUNT(*) FHSCN
---------- --------------------
         3 1820866
         4 2281969
         4 2281982
         4 2282012        
        
//* Here , we see controlfile is aware of PDB



$ sqlplus "/as sysdba"

SQL*Plus: Release 12.2.0.1.0 Production on Wed Sep 19 01:02:13 2018

Copyright (c) 1982, 2016, Oracle.  All rights reserved.

Connected to an idle instance.

SQL>  startup nomount force pfile='/<path>/<filename>.ora';
ORACLE instance started.

Total System Global Area 1593835520 bytes
Fixed Size                  8793256 bytes
Variable Size             402654040 bytes
Database Buffers         1174405120 bytes
Redo Buffers                7983104 bytes
SQL> alter database mount;

Database altered.

SQL> recover database;
ORA-00279: change 2281969 generated at 09/19/2018 00:51:35 needed for thread 1
ORA-00289: suggestion : /<path>/1_32_981800889.dbf
ORA-00280: change 2281969 for thread 1 is in sequence #32


Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
/<path>/1_32_981800889.dbf
Log applied.
Media recovery complete.
SQL> alter database open;

Database altered.

SQL>

官方的进一步解释:
We clearly see that the recovery steps applies the same archivelog file twice. When a controlfile is recreated, the recovery initiated will apply archivelog files to only the CDB datafiles, not to the PDB. Once the database open returns the ORA-65088 error, the next database re-start will apply the archivelog files to the PDB for the sake of database consistency.This should explain why Oracle is looking to apply the same archivelog sequence a second time. The following bugs report similar issues. They have both been closed as ‘not a bug’ as this is expected behavior:
BUG 24951417 – ERROR OPENING DATABASE WITH RESETLOGS AFTER CREATE CONTROLFILE
BUG 25172530 – MULTITENANT RESTORE FAILED WITH ORA-65088: DATABASE OPEN
参考:ORA-65088 while opening DB with resetlogs for multi-tenant DB in 12.2 (Doc ID 2449591.1)

ORA-600 12807（CON$.CON#达到最大值）处理

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：ORA-600 12807（CON$.CON#达到最大值）处理

这次阳了有点严重，客户现场打patch无法去，在家里远程值守,在电脑前面闲着就查询和重现了最近朋友和我说的他们的客户遇到ORA-600 12807的故障.查询了下mos,基本上可以确认是由于CON$.CON#达到理论最大值无法继续增加从而报该错误,参考文档:
Mechanism to Recycle Database Constraint Identifiers (Doc ID 2925056.1)
Bug 13781691 – ORA-600 [12807] if CON$.CON# very high due to bug 13784384 (Doc ID 13781691.8)
Bug 25343563 – Mechanism to Implement Constraint Identifier (con#) Recycling (Doc ID 25343563.8)
在12及其之后的版本中oracle发布了patch 25343563 并设置event启用该patch进行解决.但是如果是12c之前版本,官方没有提供直接的解决方案.最基本的解决方法就是进行数据逻辑迁移,以及避免频繁创建约束导致con$.con#消耗太大

通过试验重现该错误

SQL> create table t_xff  (id number not null,name varchar2(100) not null);
create table t_xff  (id number not null,name varchar2(100) not null)
*
ERROR at line 1:
ORA-00600: internal error code, arguments: [12807], [], [], [], [], [], [], [],
[], [], [], []

SQL>  create table t_xff  (id number,name varchar2(100));

Table created.

SQL> alter table t_xff add primary key(id);
alter table t_xff add primary key(id)
*
ERROR at line 1:
ORA-00600: internal error code, arguments: [12807], [], [], [], [], [], [], [],
[], [], [], []


SQL> select con# from sys.con$ where name='_NEXT_CONSTRAINT';

      CON#
----------
4294967294

通过一些底层分析,并对部分底层基表进行patch实现数据库可以继续创建约束

SQL>  alter table t_xff add primary key(id);

Table altered.

使用临时的patch方法,可以快速的恢复业务,后续找适当时间点安排迁移.
在此提醒:对于一些创建中间对象或者临时对象频繁的系统(特别是大量主键,not null等)注意检查该值距离天花板距离,如果比较接近了最好安排一次逻辑迁移和找出来原因(是oracle bug还是应用触发)

ORA-600 ksuloget2 恢复

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：ORA-600 ksuloget2 恢复

客户在win 32位的操作系统上调至sga超过2G,数据库运行过程中报ORA-600 ksuloget2错误

Thread 1 cannot allocate new log, sequence 43586
Checkpoint not complete
  Current log# 1 seq# 43585 mem# 0: D:\ORACLE\ORADATA\ORCL\REDO01.LOG
Fri Aug 04 14:57:02 2023
Errors in file d:\oracle\diag\rdbms\orcl\orcl\trace\orcl_ora_42996.trc  (incident=67481):
ORA-00600: 内部错误代码, 参数: [ksuloget2], [0xFEBA6208], [0xFEBA3B08], [500], [0xFEBA622C], [], [], [], [], []
Thread 1 advanced to log sequence 43586 (LGWR switch)
  Current log# 2 seq# 43586 mem# 0: D:\ORACLE\ORADATA\ORCL\REDO02.LOG

重启数据库,进行尝试恢复继续报ORA-600 ksuloget2

Thu Aug 17 17:38:27 2023
ALTER DATABASE RECOVER  database using backup controlfile  
Media Recovery Start
 started logmerger process
Parallel Media Recovery started with 24 slaves
ORA-279 signalled during: ALTER DATABASE RECOVER  database using backup controlfile  ...
Thu Aug 17 17:39:01 2023
ALTER DATABASE RECOVER LOGFILE 'D:\oracle\flash_recovery_area\orcl\ARCHIVELOG\2023_08_04\REDO03.LOG'  
Media Recovery Log D:\oracle\flash_recovery_area\orcl\ARCHIVELOG\2023_08_04\REDO03.LOG
Thu Aug 17 17:39:01 2023
Errors in file d:\oracle\diag\rdbms\orcl\orcl\trace\orcl_pr00_5528.trc  (incident=110724):
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
ORA-00342: archived log does not have expected resetlogs SCN 685171428
ORA-00334: archived log: 'D:\ORACLE\FLASH_RECOVERY_AREA\ORCL\ARCHIVELOG\2023_08_04\REDO03.LOG'
Errors in file d:\oracle\diag\rdbms\orcl\orcl\trace\orcl_pr00_5528.trc:
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
ORA-00342: archived log does not have expected resetlogs SCN 685171428
ORA-00334: archived log: 'D:\ORACLE\FLASH_RECOVERY_AREA\ORCL\ARCHIVELOG\2023_08_04\REDO03.LOG'
Errors in file d:\oracle\diag\rdbms\orcl\orcl\trace\orcl_ora_5604.trc  (incident=110709):
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
ORA-00342: archived log does not have expected resetlogs SCN 685171428
ORA-00334: archived log: 'D:\ORACLE\FLASH_RECOVERY_AREA\ORCL\ARCHIVELOG\2023_08_04\REDO03.LOG'
Incident details in: d:\oracle\diag\rdbms\orcl\orcl\incident\incdir_110709\orcl_ora_5604_i110709.trc
ORA-600 signalled during:ALTER DATABASE RECOVER LOGFILE 'D:\oracle\flash_recovery_area\orcl\2023_08_04\REDO03.LOG'
ALTER DATABASE RECOVER CANCEL 
Errors in file d:\oracle\diag\rdbms\orcl\orcl\trace\orcl_pr00_5528.trc  (incident=110725):
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
Errors in file d:\oracle\diag\rdbms\orcl\orcl\trace\orcl_pr00_5528.trc  (incident=110726):
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
Errors in file d:\oracle\diag\rdbms\orcl\orcl\trace\orcl_pr00_5528.trc:
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
Errors in file d:\oracle\diag\rdbms\orcl\orcl\trace\orcl_pr00_5528.trc  (incident=110727):
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
Errors in file d:\oracle\diag\rdbms\orcl\orcl\trace\orcl_pr00_5528.trc  (incident=110728):
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
Errors in file d:\oracle\diag\rdbms\orcl\orcl\trace\orcl_ora_5604.trc  (incident=110710):
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [], []
ORA-00600: internal error code, arguments: [ksuloget2], [0xFEBA6E38], [0xFEBA3B08], [500], [0xFEBA6E5C], [], [], [
Incident details in: d:\oracle\diag\rdbms\orcl\orcl\incident\incdir_110710\orcl_ora_5604_i110710.trc

由于是应用日志失败,屏蔽日志一致性,强制打开数据库,检查数据ok,业务可以直接使用,对于这类问题,官方建议:ORA-600: [Ksuloget2] Hit on Windows When SGA Greater Than 1G (Doc ID 836109.1)
20230819105750

分类目录归档：ORA-xxxxx