数据库版本信息
SQL> select * from v$version;
BANNER
--------------------------------------------------------------------------------
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - Production
PL/SQL Release 11.2.0.3.0 - Production
CORE 11.2.0.3.0 Production
TNS for Linux: Version 11.2.0.3.0 - Production
NLSRTL Version 11.2.0.3.0 - Production
SQL> select to_char(sysdate,'yyyy-mm-dd hh24:mi:ss') "www.xifenfei.com" from dual;
www.xifenfei.com
-------------------
2012-06-12 14:44:53
数据库启动报ORA-00600[ktubko_1]错误
Database Characterset is ZHS16GBK
Errors in file /u01/diag/rdbms/xff/XFF1/trace/XFF1_smon_17248.trc (incident=21754):
ORA-00600: internal error code, arguments: [ktubko_1], [], [], [], [], [], [], [], [], [], [], []
Incident details in: /u01/diag/rdbms/xff/XFF1/incident/incdir_21754/XFF1_smon_17248_i21754.trc
Tue Jun 12 10:37:10 2012
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
ORACLE Instance XFF1 (pid = 19) - Error 600 encountered while recovering transaction (4, 2) on object 5887.
Errors in file /u01/diag/rdbms/xff/XFF1/trace/XFF1_smon_17248.trc:
ORA-00600: internal error code, arguments: [ktubko_1], [], [], [], [], [], [], [], [], [], [], []
查看trace文件
Incorrect next uba in kturCurrBackoutOneChg while backing out xid: 0x0004.002.0000022b uba: 0x00c02068.00af.3b
Undo record:
ktubu redo: slt: 2 rci: 58 opc: 10.22 objn: 5887 objd: 5887 tsn: 1
Undo type: Regular undo Undo type: Last buffer split: No
Tablespace Undo: No
0x00000000
index undo for leaf key operations
KTB Redo
op: 0x04 ver: 0x01
compat bit: 4 (post-11) padding: 1
op: L itl: xid: 0x0003.003.0000030a uba: 0x00c0200c.010a.15
flg: C--- lkc: 0 scn: 0x0000.00118c55
Dump kdilk : itl=3, kdxlkflg=0x1 sdc=0 indexid=0x800df2 block=0x00800df3
(kdxlre): restore leaf row (clear leaf delete flags)
key :(5): 02 c1 0d 01 80
keydata/bitmap: (6): 00 81 0f 41 00 01
Undo block: tsn 0x2 rdba: 0xc02068
Dump of buffer cache at level 4 for tsn=2 rdba=12591208
BH (0x33ff7264) file#: 3 rdba: 0x00c02068 (3/8296) class: 24 ba: 0x33f24000
*** 2012-06-12 10:36:40.265
set: 5 pool: 3 bsz: 8192 bsi: 0 sflg: 2 pwc: 0,0
dbwrid: 0 obj: -1 objn: 0 tsn: 2 afn: 3 hint: f
hash: [0x3e78a49c,0x3e78a49c] lru: [0x33ff73ec,0x33ff723c]
obj-flags: object_ckpt_list
ckptq:[0x3e05cd68,0x33ff7f0c]fileq:[0x3e05cda4,0x3e05cda4]objq:[0x3b9094a4,0x3b9094a4]objaq:[0x33ff7acc,0x3b909494]
st: XCURRENT md: NULL fpin: 'ktuwh23: ktubko' tch: 1
flags: buffer_dirty redo_since_read
LRBA: [0x15.26.0] LSCN: [0x0.11acb6] HSCN: [0x0.11acb6] HSUB: [1]
Data block dump: tsn: 0x1 rdba: 0x800df3
Dump of buffer cache at level 3 for tsn=1 rdba=8392179
BH (0x33ff70b4) file#: 2 rdba: 0x00800df3 (2/3571) class: 1 ba: 0x33f20000
set: 5 pool: 3 bsz: 8192 bsi: 0 sflg: 2 pwc: 0,0
dbwrid: 0 obj: 5887 objn: 5887 tsn: 1 afn: 2 hint: f
hash: [0x3e7d85e4,0x3e7d85e4] lru: [0x33ff723c,0x3e05c760]
ckptq: [NULL] fileq: [NULL] objq: [0x3b9092a8,0x3b9092a8] objaq: [0x3b9092a0,0x3b9092a0]
st: XCURRENT md: NULL fpin: 'kdiwh27: kdiulk' tch: 1
flags:
LRBA: [0x0.0.0] LSCN: [0x0.0] HSCN: [0xffff.ffffffff] HSUB: [65535]
buffer tsn: 1 rdba: 0x00800df3 (2/3571)
scn: 0x0000.00118c67 seq: 0x01 flg: 0x06 tail: 0x8c670601
frmt: 0x02 chkval: 0x5d04 type: 0x06=trans data
Block header dump: 0x00800df3
Object id on Block? Y
seg/obj: 0x16ff csc: 0x00.118c60 itc: 3 flg: E typ: 2 - INDEX
brn: 0 bdba: 0x800df0 ver: 0x01 opc: 0
inc: 0 exflg: 0
Itl Xid Uba Flag Lck Scn/Fsc
0x01 0x0000.000.00000000 0x00000000.0000.00 ---- 0 fsc 0x0000.00000000
0x02 0x0006.01d.00000319 0x00c00332.009b.15 --U- 1 fsc 0x000f.00118c67
0x03 0x0003.003.0000030a 0x00c0200c.010a.15 C--- 0 scn 0x0000.00118c55
kcra_dump_redo_internal: skipped for critical process
Dumping redo for undo$
kcra_dump_redo_internal: skipped for critical process
*** 2012-06-12 10:36:43.906
Incident 21754 created, dump file: /u01/diag/rdbms/xff/XFF1/incident/incdir_21754/XFF1_smon_17248_i21754.trc
ORA-00600: internal error code, arguments: [ktubko_1], [], [], [], [], [], [], [], [], [], [], []
ORACLE Instance XFF1 (pid = 19) - Error 600 encountered while recovering transaction (4, 2) on object 5887.
*** 2012-06-12 10:37:10.646
dbkedDefDump(): Starting a non-incident diagnostic dump (flags=0x0, level=3, mask=0x0)
----- Error Stack Dump -----
ORA-00600: internal error code, arguments: [ktubko_1], [], [], [], [], [], [], [], [], [], [], []
----- SQL Statement (None) -----
Current SQL information unavailable - no cursor.
----- Call Stack Trace -----
calling call entry argument values in hex
location type point (? means dubious value)
-------------------- -------- -------------------- ----------------------------
skdstdst()+41 call kgdsdst() BFFD84FC ? 2 ?
ksedst1()+77 call skdstdst() BFFD84FC ? 0 ? 1 ? 8592A48 ?
85928C6 ? 8592A48 ?
ksedst()+33 call ksedst1() 0 ? 1 ?
dbkedDefDump()+2704 call ksedst() 0 ? 0 ? 3FFE17CC ? 3E7C5344 ?
1 ? 1 ?
ksedmp()+47 call dbkedDefDump() 3 ? 0 ?
kturRecoverTxn()+52 call ksedmp() 3 ? B6B3FEA4 ? 2C ?
471 B6B3FE38 ? 1 ? B6B3FE58 ?
kturRecoverUndoSegm call kturRecoverTxn() BFFD9078 ? 2 ? 1 ? 0 ? 11 ?
ent()+1091 4 ? 1 ?
kturRecoverActiveTx call kturRecoverUndoSegm 4 ? 0 ? 1 ? 0 ? FFFF ? 11 ?
ns()+931 ent() 4 ?
ktprbeg()+281 call kturRecoverActiveTx 10 ? 0 ?
ns()
ktmmon()+13050 call ktprbeg() 0 ? 1 ? 0 ? B6B5B72C ? 0 ?
0 ?
ktmSmonMain()+174 call ktmmon() 20018C2C ? B6B46D2C ?
114E7B00 ? 0 ? 0 ? B6B59F50 ?
ksbrdp()+826 call 00000000 20018C2C ? 432A884E ? 0 ? 0 ?
0 ? 0 ?
opirip()+559 call ksbrdp() 0 ? 0 ? 0 ? 0 ? 0 ? 0 ?
opidrv()+515 call opirip() 32 ? 4 ? BFFDB20C ?
sou2o()+80 call opidrv() 32 ? 4 ? BFFDB20C ?
opimai_real()+230 call sou2o() BFFDB1F0 ? 32 ? 4 ?
BFFDB20C ?
ssthrdmain()+212 call 00000000 3 ? BFFDB338 ? 0 ? 4318AF14 ?
BFFDB2F4 ? 4317E670 ?
main()+147 call ssthrdmain() 3 ? BFFDB338 ?
__libc_start_main() call 00000000 1 ? BFFDB434 ? BFFDB43C ?
+220 4317E828 ? 0 ? 1 ?
_start()+33 call __libc_start_main() 856F1C4 ? 1 ? BFFDB434 ?
BCF1440 ? BCF1430 ?
43170790 ?
--------------------- Binary Stack Dump ---------------------
通过上面的trace可以看出是2/3571中包含了事务,但是和3/8296[4号回滚段]回滚中的信息不相符,从而出现了在数据库启动回滚的时候出现该错误.查询mos[ID 1318986.1]发现这个是数据库的Bug 10205230比较相似,虽说在11.2.0.2中修复而且在asm中不受该影响,我这里库是11.2.0.3的asm rac照样出现该bug.
解决方法
通过alert日志提示object可以找到object_id=5887.当然也可以通过trace中的rdba来确定
SQL> col OBJECT_NAME for a30
SQL> select object_name,object_type,owner from dba_objects where object_id=5887;
OBJECT_NAME OBJECT_TYPE OWNER
------------------------------ ------------------- ------------------------------
WRI$_ADV_MESSAGE_GROUPS_PK INDEX SYS
SQL> alter index sys.WRI$_ADV_MESSAGE_GROUPS_PK rebuild online;
Index altered.
补充说明:如果损坏对象是表,需要使用DBMS_REPAIR跳过坏块,然后重建表
重启数据库观察
数据库已经正常,开始报undo回滚段错误的记录已经不再存在,数据库恢复正常
Tue Jun 12 13:50:43 2012
SMON: enabling tx recovery
Database Characterset is ZHS16GBK
Tue Jun 12 13:51:11 2012
No Resource Manager plan active
Tue Jun 12 13:52:01 2012
Starting background process GTX0
Tue Jun 12 13:52:01 2012
GTX0 started with pid=29, OS id=14234
Starting background process RCBG
Tue Jun 12 13:52:04 2012
RCBG started with pid=41, OS id=14238
replication_dependency_tracking turned off (no async multimaster replication found)
Tue Jun 12 13:54:01 2012
Starting background process QMNC
Tue Jun 12 13:54:01 2012
QMNC started with pid=42, OS id=14279
Tue Jun 12 13:57:26 2012
Completed: ALTER DATABASE OPEN