联系:手机/微信(+86 17813235971) QQ(107644445)
标题:ORA-00600: internal error code, arguments: [16703], [1403], [4] 故障处理
作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]
有一个客户数据库遭遇ORA-600 16703错误,故障原因见:警告:互联网中有oracle介质被注入恶意程序导致—ORA-600 16703,这种故障已经恢复比较多,在这次的恢复中遇到一个新错误,进行记录
接手给客户报错情况ORA-00600: internal error code, arguments: [16703], [1403], [20]
Thu May 13 22:36:11 2021 SMON: enabling cache recovery Thu May 13 22:36:11 2021 NSA2 started with pid=61, OS id=6261 Archived Log entry 90224 added for thread 1 sequence 50454 ID 0x19ae1c6c dest 1: Errors in file /oracle/diag/rdbms/xff/xff/trace/xff_ora_5931.trc (incident=741052): ORA-00600: internal error code, arguments: [16703], [1403], [20], [], [], [], [], [], [], [], [], [] Incident details in: /oracle/diag/rdbms/xff/xff/incident/incdir_741052/xff_ora_5931_i741052.trc Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. Errors in file /oracle/diag/rdbms/xff/xff/trace/xff_ora_5931.trc: ORA-00704: bootstrap process failure ORA-00704: bootstrap process failure ORA-00600: internal error code, arguments: [16703], [1403], [20], [], [], [], [], [], [], [], [], [] Errors in file /oracle/diag/rdbms/xff/xff/trace/xff_ora_5931.trc: ORA-00704: bootstrap process failure ORA-00704: bootstrap process failure ORA-00600: internal error code, arguments: [16703], [1403], [20], [], [], [], [], [], [], [], [], [] Error 704 happened during db open, shutting down database USER (ospid: 5931): terminating the instance due to error 704 Instance terminated by USER, pid = 5931 ORA-1092 signalled during: alter database open... opiodr aborting process unknown ospid (5931) as a result of ORA-1092 Thu May 13 22:36:13 2021 ORA-1092 : opitsk aborting process
这种故障,由于是恶意脚本在数据库启动的时候清空tab$所致,使用bbed对tab$进行反向删除操作,实现数据库打开.
在这次的恢复过程中遇到ORA-600 16703 1403 4的错误
SQL> startup mount pfile='/tmp/pfile'; ORACLE instance started. Total System Global Area 7.0818E+10 bytes Fixed Size 2260448 bytes Variable Size 1.3422E+10 bytes Database Buffers 5.7177E+10 bytes Redo Buffers 217030656 bytes Database mounted. SQL> alter database open ; alter database open * ERROR at line 1: ORA-01092: ORACLE instance terminated. Disconnection forced ORA-00704: bootstrap process failure ORA-00704: bootstrap process failure ORA-00600: internal error code, arguments: [16703], [1403], [4], [], [], [], [], [], [], [], [], [] Process ID: 51886 Session ID: 3497 Serial number: 3
根据ora-600 16703 1403 4基本上可以判断是由于tab$这个表中缺少obj#=4的对象导致,通过查询正常库,确认是该对象为tab$,也就是说由于tab$对象中少了tab$记录.通过bbed分析确认
SQL> SELECT a.OBJ# 2 ,TAB# 3 ,a.DATAOBJ# 4 ,BOBJ# 5 ,NAME 6 ,DBMS_ROWID.ROWID_RELATIVE_FNO (a.ROWID) FILE_ID 7 ,DBMS_ROWID.ROWID_BLOCK_NUMBER (a.ROWID) BLOCK_ID 8 FROM TAB$ a, obj$ b 9 WHERE a.obj# = b.obj# 10 AND A.OBJ# IN (4); OBJ# TAB# DATAOBJ# BOBJ# NAME ---------- ---------- ---------- ---------- ------------------------------ FILE_ID BLOCK_ID ---------- ---------- 4 1 2 2 TAB$ 1 147 BBED> set dba 1,147 DBA 0x00400093 (4194451 1,147) BBED> x /rnnnnnnnnnnnnncnnnnnnnntnnnnnnnnnncct *kdbr[14] rowdata[6848] @7349 ------------- flag@7349: 0x20 (KDRHFH) lock@7350: 0x02 cols@7351: 0 nrid@7352:0x00407b09.1 BBED> set dba 0x00407b09 DBA 0x00407b09 (4225801 1,31497) BBED> p kdbt[1] struct kdbt[1], 4 bytes @110 sb2 kdbtoffs @110 10 sb2 kdbtnrow @112 2 BBED> x /rnnnnnnnnnnnnncnnnnnnnntnnnnnnnnnncct *kdbr[11] rowdata[815] @4436 ------------ flag@4436: 0x5c (KDRHFL, KDRHFF, KDRHFD, KDRHFC) lock@4437: 0x02 cols@4438: 0 ckix@4439: 8 BBED> x /rn *kdbr[8] rowdata[950] @4571 ------------ flag@4571: 0xac (KDRHFL, KDRHFF, KDRHFH, KDRHFK) lock@4572: 0x00 cols@4573: 1 kref@4574: 1 hrid@4576:0x00400093.8 nrid@4582:0x00400094.0 col 0[2] @4590: 4
确认该记录发生了行迁移导致该问题,对其对应的block进行修复,数据库正常打开.