有客户晚上对数据库进行正常的重启维护,结果去遭遇数据库无法正常启动故障,报ORA-600 kkdlcob-objn-exist 40错误.

alert日志信息

Sat Jul 06 12:05:50 2019
alter database open
Sat Jul 06 12:05:51 2019
LGWR: STARTING ARCH PROCESSES
Sat Jul 06 12:05:51 2019
ARC0 started with pid=19, OS id=7056
ARC0: Archival started
LGWR: STARTING ARCH PROCESSES COMPLETE
ARC0: STARTING ARCH PROCESSES
Thread 1 opened at log sequence 12
  Current log# 3 seq# 12 mem# 0: D:\APP\ADMINISTRATOR\ORADATA\ORCL\REDO03.LOG
Successful open of redo thread 1
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Sat Jul 06 12:05:52 2019
SMON: enabling cache recovery
Sat Jul 06 12:05:52 2019
ARC1 started with pid=21, OS id=7948
Sat Jul 06 12:05:52 2019
ARC2 started with pid=22, OS id=11552
Sat Jul 06 12:05:52 2019
ARC3 started with pid=23, OS id=5504
ARC1: Archival started
ARC2: Archival started
ARC1: Becoming the 'no FAL' ARCH
ARC1: Becoming the 'no SRL' ARCH
ARC2: Becoming the heartbeat ARCH
ARC3: Archival started
ARC0: STARTING ARCH PROCESSES COMPLETE
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\trace\orcl_ora_3092.trc  (incident=20562):
ORA-00600: internal error code, arguments: [kkdlcob-objn-exists], [40], [], [], [], [], [], [], [], [], [], []
Incident details in: D:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\incident\incdir_20562\orcl_ora_3092_i20562.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\trace\orcl_ora_3092.trc:
ORA-00704: bootstrap process failure
ORA-00600: internal error code, arguments: [kkdlcob-objn-exists], [40], [], [], [], [], [], [], [], [], [], []
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\trace\orcl_ora_3092.trc:
ORA-00704: bootstrap process failure
ORA-00600: internal error code, arguments: [kkdlcob-objn-exists], [40], [], [], [], [], [], [], [], [], [], []
Error 704 happened during db open, shutting down database
USER (ospid: 3092): terminating the instance due to error 704
Instance terminated by USER, pid = 3092
ORA-1092 signalled during: alter database open...
opiodr aborting process unknown ospid (3092) as a result of ORA-1092
Sat Jul 06 12:05:56 2019
ORA-1092 : opitsk aborting process

ORA-600 kkdlcob-objn-exists官方解释，一般是由于ddl操作时dataobj#不合适引起,最大的可能是_NEXT_OBJECT值不对

对数据库启动过程进行跟踪

CREATE UNIQUE INDEX I_OBJ5 ON OBJ$(SPARE3,NAME,NAMESPACE,TYPE#,OWNER#,REMOTEOWNER,LINKNAME,SUBNAME,OBJ#)
PCTFREE 10 INITRANS 2 MAXTRANS 255 STORAGE (  INITIAL 64K NEXT 1024K MINEXTENTS 1
MAXEXTENTS 2147483645 PCTINCREASE 0 OBJNO 40 EXTENTS (FILE 1 BLOCK 368))
END OF STMT
PARSE #299964504:c=0,e=368,p=0,cr=0,cu=0,mis=1,r=0,dep=1,og=4,plh=440321312,tim=8823997737
ORA-00600: internal error code, arguments: [kkdlcob-objn-exists], [40], [], [], [], [], [], [], [], [], [], []
EXEC #299964504:c=875000,e=1213391,p=0,cr=0,cu=0,mis=0,r=0,dep=1,og=4,plh=440321312,tim=8825211183
ERROR #299964504:err=600 tim=8825211203
ORA-00704: bootstrap process failure
ORA-00600: internal error code, arguments: [kkdlcob-objn-exists], [40], [], [], [], [], [], [], [], [], [], []
ORA-00704: bootstrap process failure
ORA-00600: internal error code, arguments: [kkdlcob-objn-exists], [40], [], [], [], [], [], [], [], [], [], []

发现是I_OBJ5无法正常创建,导致数据库无法正常启动,由于这个是obj$对象的一个index(而且是11.2之后版本才引进),不是数据库启动必须的对象,通过bbed修改相关基表,让数据库启动不在跳过该index,数据库启动成功.因为进行了非常规的方法恢复,建议客户导出数据,导入到新库,数据库恢复工作完成.