分享一次ORA-01113 ORA-01110故障处理过程

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：分享一次ORA-01113 ORA-01110故障处理过程

作者：惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

数据库报ORA-01113 ORA-01110无法正常open

SQL> alter database open;
alter database open
*
ERROR at line 1:
ORA-01113: file 1 needs media recovery
ORA-01110: data file 1: 'D:\APP\ADMINISTRATOR\ORADATA\ORCL\SYSTEM01.DBF'

alert日志

Wed Jul 16 17:17:56 2014
ALTER DATABASE   MOUNT
Successful mount of redo thread 1, with mount id 1380870980
Database mounted in Exclusive Mode
Lost write protection disabled
Completed: ALTER DATABASE   MOUNT
Wed Jul 16 17:19:01 2014
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\trace\orcl_m000_3540.trc  (incident=367575):
ORA-00700: soft internal error, arguments: [dbgrmblcp_corrupt_page], [D:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\metadata\HM_FINDING.ams],
[1245], [], [], [], [], [], [], [], [], []
ORA-01113: file 1 needs media recovery
ORA-01110: data file 1: 'D:\APP\ADMINISTRATOR\ORADATA\ORCL\SYSTEM01.DBF'
Incident details in: D:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\incident\incdir_367575\orcl_m000_3540_i367575.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\trace\orcl_m000_3540.trc  (incident=367576):
ORA-00600: internal error code, arguments: [dbgrmblgp_get_page_1], [1245], [0], [80], [], [], [], [], [], [], [], []
ORA-01113: file 1 needs media recovery
ORA-01110: data file 1: 'D:\APP\ADMINISTRATOR\ORADATA\ORCL\SYSTEM01.DBF'
Incident details in: D:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\incident\incdir_367576\orcl_m000_3540_i367576.trc
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\trace\orcl_m000_3540.trc:
ORA-51106: check failed to complete due to an error.  See error below
ORA-00600: internal error code, arguments: [dbgrmblgp_get_page_1], [1245], [0], [80], [], [], [], [], [], [], [], []
ORA-01113: file 1 needs media recovery
ORA-01110: data file 1: 'D:\APP\ADMINISTRATOR\ORADATA\ORCL\SYSTEM01.DBF'

这里出现ORA-00700[dbgrmblcp_corrupt_page]与ORA-00600[dbgrmblgp_get_page_1]是Bug 10321285 : ORA-600 [DBGRMBLCP_CORRUPT_PAGE]

尝试recover database 遭遇ORA-00283 ORA-16433

SQL> recover database;
ORA-00283: recovery session canceled due to errors
ORA-16433: The database must be opened in read/write mode.

alert日志信息

Wed Jul 16 17:36:33 2014
ALTER DATABASE RECOVER  database
Media Recovery Start
 started logmerger process
Wed Jul 16 17:36:33 2014
Media Recovery failed with error 16433
Slave exiting with ORA-283 exception
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\trace\orcl_pr00_3868.trc:
ORA-00283: recovery session canceled due to errors
ORA-16433: The database must be opened in read/write mode.
Recovery Slave PR00 previously exited with exception 283
ORA-283 signalled during: ALTER DATABASE RECOVER  database  ...
Wed Jul 16 17:36:45 2014
alter database open
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\trace\orcl_ora_3236.trc:
ORA-01113: file 1 needs media recovery
ORA-01110: data file 1: 'D:\APP\ADMINISTRATOR\ORADATA\ORCL\SYSTEM01.DBF'
ORA-1113 signalled during: alter database open...

重建控制文件后恢复,遭遇ORA-600 2662错误

SQL> shutdown abort
ORACLE instance shut down.
SQL> startup nomount
ORACLE instance started.
Total System Global Area      10255212544 bytes
Fixed Size                        2264088 bytes
Variable Size                  4529849320 bytes
Database Buffers               5704253440 bytes
Redo Buffers                     18845696 bytes
SQL> @d:\ctl.sql
Control file created.
SQL> recover database using backup controlfile;
ORA-00279: change 13953689551797 generated at 07/16/2014 17:53:41 needed for
thread 1
ORA-00289: suggestion :
D:\APP\ADMINISTRATOR\FAST_RECOVERY_AREA\ORCL\ARCHIVELOG\2014_07_16\O1_MF_1_1_%U_
.ARC
ORA-00280: change 13953689551797 for thread 1 is in sequence #1
Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
d:\app\administrator\oradata\orcl\redo01.log
Log applied.
Media recovery complete.
SQL> alter database open resetlogs;
alter database open resetlogs
*
ERROR at line 1:
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-00704: bootstrap process failure
ORA-00704: bootstrap process failure
ORA-00600: internal error code, arguments: [2662], [3248], [3635774399],
[3249], [3580470066], [4194545], [], [], [], [], [], []
Process ID: 3932
Session ID: 200 Serial number: 1

因为该数据库是11.2.0.3未打上scn patch,直接推进scn(可以使用event,隐含参数,oradebug,bbed等),数据库就正常open

One thought on “分享一次ORA-01113 ORA-01110故障处理过程”

  1. Bug 10321285 : ORA-600 [DBGRMBLCP_CORRUPT_PAGE]

    Hdr: 10321285 11.1.0.7.0 RDBMS 11.1.0.7.0 DIAG ADR PRODID-5 PORTID-212 ORA-600
Abstract: ORA-600 [DBGRMBLCP_CORRUPT_PAGE]
*** 11/21/10 07:41 am *** (CHG: RDBMS Ver.-&gt; NULL -&gt; 11.1.0.7.0)
*** 11/21/10 07:41 am ***
----
PROBLEM:
--------
1. Clear description of the problem encountered
On Sep. 5th, the following internal errors were written to the alert log:
ORA-600 [dbgrmblcp_corrupt_page],
[/pextapps/admin/diag/rdbms/pextapps/pextapps/metadata/HM_FINDING.ams], [16],
[]
ORA-600 [dbgrmblgp_get_page_1], [16], [675499854], [42], []
can also see:
ORA-51108: unable to access diagnostic repository - retry command
ORA-48223: Interrupt Requested - Fetch Aborted - Return Code [51014]
ORA-6512: at &quot;SYS.DBMS_IR&quot;, line 522
This was a weekend - should have been little to no activity
These errors were raised several times on one day, did not reappear since
then.
2. Pertinent configuration information (MTS/OPS/distributed/etc)
3. Indication of the frequency and predictability of the problem
So far, this happened only on Sep. 5th at Customer-side.
Not reproducible
4. Sequence of events leading to the problem
undetermined
5. Technical impact on the customer. Include persistent after effects.
Customer is concerned about their ADR repository.
Is this the visible sign of something being corrupt in their ADR Repository?
DIAGNOSTIC ANALYSIS:
--------------------
Alert log shows:
Sun Sep 05 11:36:41 2010
Errors in file
/pextapps/admin/diag/rdbms/pextapps/pextapps/trace/pextapps_j001_3485708.trc
(incident=326889):
ORA-600: internal error code, arguments: [dbgrmblcp_corrupt_page],
[/pextapps/admin/diag/rdbms/pextapps/pextapps/metadata/HM_FINDING.ams], [16],
[], [], [], [], [], [], [], [], []
Incident details in:
/pextapps/admin/diag/rdbms/pextapps/pextapps/incident/incdir_326889/pextapps_j
001_3485708_i326889.trc
Errors in file
/pextapps/admin/diag/rdbms/pextapps/pextapps/trace/pextapps_j001_3485708.trc
(incident=326890):
ORA-600: internal error code, arguments: [dbgrmblgp_get_page_1], [16],
[675499854], [42], [], [], [], [], [], [], [], []
Incident details in:
/pextapps/admin/diag/rdbms/pextapps/pextapps/incident/incdir_326890/pextapps_j
001_3485708_i326890.trc
Sun Sep 05 11:36:45 2010
Trace dumping is performing id=[cdmp_20100905113645]
Sun Sep 05 11:36:45 2010
Sweep Incident[326889]: completed
Errors in file
/pextapps/admin/diag/rdbms/pextapps/pextapps/trace/pextapps_j001_3485708.trc:
ORA-12012: error on auto execute of job 11375
ORA-51108: unable to access diagnostic repository - retry command
ORA-48223: Interrupt Requested - Fetch Aborted - Return Code [51014]
ORA-6512: at &quot;SYS.DBMS_IR&quot;, line 522
Sun Sep 05 11:37:41 2010
Errors in file
/pextapps/admin/diag/rdbms/pextapps/pextapps/trace/pextapps_j000_1769628.trc
(incident=326831):
ORA-600: internal error code, arguments: [dbgrmblcp_corrupt_page],
[/pextapps/admin/diag/rdbms/pextapps/pextapps/metadata/HM_FINDING.ams], [16],
[], [], [], [], [], [], [], [], []
Incident details in:
/pextapps/admin/diag/rdbms/pextapps/pextapps/incident/incdir_326831/pextapps_j
000_1769628_i326831.trc
Sun Sep 05 11:37:45 2010
Sweep Incident[326890]: completed
Sun Sep 05 11:38:00 2010
Errors in file
/pextapps/admin/diag/rdbms/pextapps/pextapps/trace/pextapps_j000_1769628.trc
(incident=326832):
ORA-600: internal error code, arguments: [dbgrmblgp_get_page_1], [16],
[675499854], [42], [], [], [], [], [], [], [], []
Incident details in:
/pextapps/admin/diag/rdbms/pextapps/pextapps/incident/incdir_326832/pextapps_j
000_1769628_i326832.trc
Sun Sep 05 11:38:00 2010
Trace dumping is performing id=[cdmp_20100905113800]
Trace dumping is performing id=[cdmp_20100905113803]
Sun Sep 05 11:38:43 2010
Errors in file
/pextapps/admin/diag/rdbms/pextapps/pextapps/trace/pextapps_j000_1769628.trc
(incident=326833):
ORA-600: internal error code, arguments: [dbgrmblcp_corrupt_page],
[/pextapps/admin/diag/rdbms/pextapps/pextapps/metadata/HM_FINDING.ams], [16],
[], [], [], [], [], [], [], [], []
Incident details in:
/pextapps/admin/diag/rdbms/pextapps/pextapps/incident/incdir_326833/pextapps_j
000_1769628_i326833.trc
Sun Sep 05 11:38:45 2010
Sweep Incident[326832]: completed
Sweep Incident[326831]: completed
Errors in file
/pextapps/admin/diag/rdbms/pextapps/pextapps/trace/pextapps_j000_1769628.trc
(incident=326834):
ORA-600: internal error code, arguments: [dbgrmblgp_get_page_1], [16],
[675499854], [42], [], [], [], [], [], [], [], []
Incident details in:
/pextapps/admin/diag/rdbms/pextapps/pextapps/incident/incdir_326834/pextapps_j
000_1769628_i326834.trc
Sun Sep 05 11:38:47 2010
Trace dumping is performing id=[cdmp_20100905113847]
Trace dumping is performing id=[cdmp_20100905113850]
Sun Sep 05 11:39:45 2010
Sweep Incident[326834]: completed
Sweep Incident[326833]: completed
Asked CT to run the following commands:
 adrci&gt; set home rdbms
 adrci&gt; show catalog
 adrci&gt; describe hm_finding
 adrci&gt; query (RUN_ID, NAME) HM_FINDING
which all worked fine
Obtained the content of their /diag/rdbms/.../.../metadata/' folder
WORKAROUND:
-----------
RELATED BUGS:
-------------
Bug 10303857 - QCN: ORA-700[DBGRMBLCP_CORRUPT_PAGE] /
ORA-600[DBGRMBLGP_GET_PAGE_1] ON NT
Bug 6595098 - ORA-48132 AND ORA-600 [DBGRMBLCP_CORRUPT_PAGE] ETC DURING OPEN
DATABASE
REPRODUCIBILITY:
----------------
For the time being, these errors did not reappear.
TEST CASE:
----------
STACK TRACE:
------------
skdstdst ksedst1 ksedst dbkedDefDump ksedmp ksfdmp dbgexPhaseII
dbgexProcessError dbgeExecuteForError dbgePostErrorKGE dbkePostKGE_kgsf
kgeadse kgerinv_internal kgesoftnmierr
dbgrmblcp_corrupt_page dbgrmblgp_get_page dbgrmblrp_read_page
dbgrmqmgp_get_page dbgrmqmif_internal_fetch dbgrmqmf_fetch_real
dbgrmqmf_fetch
dbgrip_fetch_record dbgrip_relation_iterator dbghmo_close_findin
dbkh_close_finding  kcidr_file_header_check dbkird_df_check
dbkh_run_check_internal dbkh_run_revalidate check dbkifRevalidateFinding
dbkifReevaluateOpenFailures dbkiicd_reeval pevm_icd_call_commo pfrinstr_ICAL
pfrrun_no_tool pfrrun plsql_run peidxr_run peidxexe kkxdexe kkxmpexe kgmexwi
kgmexec evapls evaopn2 kkxmexcs opiexe kpoal8 opiodr kpoodr upirtrc kpurcsc
kpuexec OCIStmtExecute jslvec_execcb1 jslvswu jslve_execute0 jslve_execute
rpiswu2 kkjex1e kkjsexe kkjrdp opirip opidrv sou2o opimai_real main start
    回复

发表评论

邮箱地址不会被公开。 必填项已用*标注

2 × 2 =