联系：手机/微信(+86 17813235971) QQ(107644445)
作者：惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]
由于raid掉盘过多,强制raid上线,然后启动数据库报以下错误
Mon Apr 19 23:19:28 2021 ALTER DATABASE OPEN Beginning crash recovery of 1 threads parallel recovery started with 15 processes Started redo scan Completed redo scan read 106750 KB redo, 9080 data blocks need recovery Mon Apr 19 23:19:45 2021 Slave exiting with ORA-1115 exception Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_p000_3277.trc: ORA-01115: IO error reading block from file 9 (block # 339) ORA-01110: data file 9: '/u01/app/oracle/oradata/orcl/dev02.dbf' ORA-27072: File I/O error Additional information: 4 Additional information: 326 Additional information: 24576 ORA-27072: File I/O error Additional information: 4 Additional information: 326 Additional information: 24576 ORA-27072: File I/O error Additional information: 4 Additional information: 326 Additional information: 24576 ORA-27072: File I/O error Additional information: 4 Additional information: 326 Additional information: 24576 ORA-27072: File I/O error Additional information: 4 Additional information: 326 Additional information: 24576 ORA-27072: File I/O error Additional information: 4 Additional information: 326 Additional information: 24576 ORA-27072: File I/O error Additional information: 4 Additional information: 326 Additional information: 24576 ORA-27072: File I/O error Additional information: 4 Additional information: 326 Additional information: 24576 ORA-27072: File I Mon Apr 19 23:19:45 2021 Aborting crash recovery due to slave death, attempting serial crash recovery Beginning crash recovery of 1 threads Started redo scan Completed redo scan read 106750 KB redo, 9080 data blocks need recovery Aborting crash recovery due to error 1115 Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_3275.trc: ORA-01115: IO error reading block from file 9 (block # 329) ORA-01110: data file 9: '/u01/app/oracle/oradata/orcl/dev02.dbf' ORA-1115 signalled during: ALTER DATABASE OPEN...
错误提示比较明显IO error,结合客户强行上线raid的操作,比较明显是由于底层io问题导致该错误,直接对此文件dbv检查
[oracle@database orcl]$ dbv file=dev02.dbf DBVERIFY: Release 11.2.0.4.0 - Production on Mon Apr 19 23:59:03 2021 Copyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved. DBVERIFY - Verification starting : FILE = /u01/app/oracle/oradata/orcl/dev02.dbf DBV-00600: Fatal Error - [28] [27061] [0] [0]
对于此类情况,通过工具进行处理
DUL> copy file from /u01/app/oracle/oradata/orcl/dev02.dbf to /oradata/dev02.dbf starting copy datafile '/u01/app/oracle/oradata/orcl/dev02.dbf' to '/oradata/dev02.dbf' read data error from file '/u01/app/oracle/oradata/orcl/dev02.dbf'.error message:Input/output error read block# error: 303 read data error from file '/u01/app/oracle/oradata/orcl/dev02.dbf'.error message:Input/output error read block# error: 304 read data error from file '/u01/app/oracle/oradata/orcl/dev02.dbf'.error message:Input/output error read block# error: 329 datafile copy completed with 2 block error.
dbv校验文件
[oracle@database oradata]$ dbv file=dev02.dbf DBVERIFY: Release 11.2.0.4.0 - Production on Tue Apr 20 00:28:31 2021 Copyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved. DBVERIFY - Verification starting : FILE = /oradata/dev02.dbf Page 303 is marked corrupt Corrupt block relative dba: 0x0240012f (file 9, block 303) Completely zero block found during dbv: Page 304 is marked corrupt Corrupt block relative dba: 0x02400130 (file 9, block 304) Completely zero block found during dbv: Page 329 is marked corrupt Corrupt block relative dba: 0x02400149 (file 9, block 329) Completely zero block found during dbv: DBVERIFY - Verification complete Total Pages Examined : 3932160 Total Pages Processed (Data) : 3213723 Total Pages Failing (Data) : 0 Total Pages Processed (Index): 714294 Total Pages Failing (Index): 0 Total Pages Processed (Other): 4139 Total Pages Processed (Seg) : 0 Total Pages Failing (Seg) : 0 Total Pages Empty : 1 Total Pages Marked Corrupt : 3 Total Pages Influx : 0 Total Pages Encrypted : 0 Highest block SCN : 85078875 (6.85078875)
通过对io error的文件进行处理,最终损坏三个block,最大限度抢救数据.使用被恢复出来的文件,尝试open库遭遇以下错误
SQL> alter database open resetlogs; alter database open resetlogs * ERROR at line 1: ORA-00603: ORACLE server session terminated by fatal error ORA-00600: internal error code, arguments: [2662], [6], [85035771], [6], [85084136], [12583040], [], [], [], [], [], [] ORA-00600: internal error code, arguments: [2662], [6], [85035770], [6], [85084136], [12583040], [], [], [], [], [], [] ORA-01092: ORACLE instance terminated. Disconnection forced ORA-00600: internal error code, arguments: [2662], [6], [85035764], [6], [85084136], [12583040], [], [], [], [], [], [] Process ID: 6733 Session ID: 570 Serial number: 3
ora-600 2662这个错误比较明显,处理文件头scn,继续open库
SQL> alter database open ; alter database open * ERROR at line 1: ORA-03113: end-of-file on communication channel Process ID: 6840 Session ID: 570 Serial number: 3
查看alert日志信息
Tue Apr 20 01:22:27 2021 alter database open upgrade Beginning crash recovery of 1 threads parallel recovery started with 15 processes Started redo scan Completed redo scan read 1 KB redo, 3 data blocks need recovery Started redo application at Thread 1: logseq 1, block 3 Recovery of Online Redo Log: Thread 1 Group 1 Seq 1 Reading mem 0 Mem# 0: /u01/app/oracle/oradata/orcl/redo01.log Completed redo application of 0.00MB Completed crash recovery at Thread 1: logseq 1, block 5, scn 25854859541 3 data blocks read, 3 data blocks written, 1 redo k-bytes read Tue Apr 20 01:22:28 2021 Thread 1 advanced to log sequence 2 (thread open) Thread 1 opened at log sequence 2 Current log# 2 seq# 2 mem# 0: /u01/app/oracle/oradata/orcl/redo02.log Successful open of redo thread 1 MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set Tue Apr 20 01:22:28 2021 SMON: enabling cache recovery [6840] Successfully onlined Undo Tablespace 2. Undo initialization finished serial:0 start:5902014 end:5905574 diff:3560 (35 seconds) Dictionary check beginning Dictionary check complete Verifying file header compatibility for 11g tablespace encryption.. Verifying 11g file header compatibility for tablespace encryption completed SMON: enabling tx recovery Database Characterset is ZHS16GBK Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_smon_6824.trc (incident=63970): ORA-00600: internal error code, arguments: [6006], [1], [], [], [], [], [], [], [], [], [], [] Incident details in: /u01/app/oracle/diag/rdbms/orcl/orcl/incident/incdir_63970/orcl_smon_6824_i63970.trc Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. ORACLE Instance orcl (pid = 14) - Error 600 encountered while recovering transaction (24, 2) on object 89023. Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_smon_6824.trc: ORA-00600: internal error code, arguments: [6006], [1], [], [], [], [], [], [], [], [], [], [] Tue Apr 20 01:22:38 2021 ORACLE Instance orcl (pid = 14) - Error 600 encountered while recovering transaction (63, 3) on object 89023. Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_smon_6824.trc: ORA-00600: internal error code, arguments: [6006], [1], [], [], [], [], [], [], [], [], [], [] Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_smon_6824.trc (incident=63974): ORA-00600: internal error code, arguments: [6006], [1], [], [], [], [], [], [], [], [], [], [] Incident details in: /u01/app/oracle/diag/rdbms/orcl/orcl/incident/incdir_63974/orcl_smon_6824_i63974.trc Tue Apr 20 01:22:55 2021 PMON (ospid: 6798): terminating the instance due to error 474
这个错误是比较常见的错误,参考:ORACLE Instance XFF (pid = 18) – Error 600 encountered while recovering transaction ，通过处理之后,数据库open成功
SQL> startup mount pfile='/tmp/pfile'; ORACLE instance started. Total System Global Area 1603411968 bytes Fixed Size 2253664 bytes Variable Size 1023413408 bytes Database Buffers 570425344 bytes Redo Buffers 7319552 bytes Database mounted. SQL> recover database; Media recovery complete. SQL> alter database open; Database altered.
后续安排逻辑导出,导入新库