DataGuard ora-16157故障解决

由于硬件故障，需要激活备库，由于登录错误服务器对本不该激活的服务器进行了如下操作
STANDBY DATABASE FINISH操作

[oracle@app73 ~]$ ss
SQL*Plus: Release 11.2.0.4.0 Production on Sun Jun 26 23:33:19 2016
Copyright (c) 1982, 2013, Oracle.  All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning and Data Mining options
SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL;
Database altered.
SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE FINISH;
Database altered.

对应的alert日志提示

Sun Jun 26 23:33:22 2016
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL
Sun Jun 26 23:33:22 2016
MRP0: Background Media Recovery cancelled with status 16037
Errors in file /Data/oracle/diag/rdbms/commentdbdg/commentdb/trace/commentdb_pr00_11871.trc:
ORA-16037: user requested cancel of managed recovery operation
Managed Standby Recovery not using Real Time Apply
Recovery interrupted!
Recovered data files to a consistent state at change 47440782676
Sun Jun 26 23:33:23 2016
MRP0: Background Media Recovery process shutdown (commentdb)
Managed Standby Recovery Canceled (commentdb)
Completed: ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL
Sun Jun 26 23:33:44 2016
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE FINISH
Attempt to do a Terminal Recovery (commentdb)
Media Recovery Start: Managed Standby Recovery (commentdb)
 started logmerger process
Sun Jun 26 23:33:44 2016
Managed Standby Recovery not using Real Time Apply
Parallel Media Recovery started with 4 slaves
Media Recovery Waiting for thread 1 sequence 21196 (in transit)
Killing 4 processes with pids 11881,11867,11869,14314 (all RFS, wait for I/O)
in order to disallow current and future RFS connections. Requested by OS process 7372
Begin: Standby Redo Logfile archival
End: Standby Redo Logfile archival
Terminal Recovery timestamp is '06/26/2016 23:33:48'
Terminal Recovery: applying standby redo logs.
Terminal Recovery: thread 1 seq# 21196 redo required
Terminal Recovery:
Recovery of Online Redo Log: Thread 1 Group 10 Seq 21196 Reading mem 0
  Mem# 0: /Data/oracle/oradata/commentdb/std_redo10.log
Identified End-Of-Redo (failover) for thread 1 sequence 21196 at SCN 0xffff.ffffffff
Incomplete Recovery applied until change 47440782709 time 06/26/2016 23:29:51
Media Recovery Complete (commentdb)
Terminal Recovery: Enabled archive destination LOG_ARCHIVE_DEST_2
Terminal Recovery: successful completion
Forcing ARSCN to IRSCN for TR 11:196142453
Attempt to set limbo arscn 11:196142453 irscn 11:196142453
Resetting standby activation ID 3880004483 (0xe7442b83)
Sun Jun 26 23:33:48 2016
ARCH: Archival stopped, error occurred. Will continue retrying
ORACLE Instance commentdb - Archival Error
ORA-16014: log 10 sequence# 21196 not archived, no available destinations
ORA-00312: online log 10 thread 1: '/Data/oracle/oradata/commentdb/std_redo10.log'
Completed: ALTER DATABASE RECOVER MANAGED STANDBY DATABASE FINISH
Sun Jun 26 23:39:24 2016
RFS[4]: Assigned to RFS process 7392
RFS[4]: No connections allowed during/after terminal recovery.

很明显数据库已经挺尸mrp并且成功执行了ALTER DATABASE RECOVER MANAGED STANDBY DATABASE FINISH

尝试重新启动mrp应用日志

SQL> shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup mount;
ORACLE instance started.
Total System Global Area 2.1379E+10 bytes
Fixed Size                  2262656 bytes
Variable Size            2684356992 bytes
Database Buffers         1.8656E+10 bytes
Redo Buffers               36073472 bytes
Database mounted.
SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT  LOGFILE DISCONNECT FROM SESSION;
Database altered.

alert日志提示

Sun Jun 26 23:40:39 2016
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT  LOGFILE DISCONNECT FROM SESSION
Attempt to start background Managed Standby Recovery process (commentdb)
Sun Jun 26 23:40:39 2016
MRP0 started with pid=24, OS id=7458
MRP0: Background Managed Standby Recovery process started (commentdb)
 started logmerger process
Sun Jun 26 23:40:44 2016
Managed Standby Recovery starting Real Time Apply
MRP0: Background Media Recovery terminated with error 16157
Errors in file /Data/oracle/diag/rdbms/commentdbdg/commentdb/trace/commentdb_pr00_7460.trc:
ORA-16157: media recovery not allowed following successful FINISH recovery
Managed Standby Recovery not using Real Time Apply
Completed: ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT  LOGFILE DISCONNECT FROM SESSION
Recovery Slave PR00 previously exited with exception 16157
MRP0: Background Media Recovery process shutdown (commentdb)
[oracle@app73 trace]$ oerr ora 16157
16157, 00000, "media recovery not allowed following successful FINISH recovery"
// *Cause:  A RECOVER MANAGED STANDBY DATABASE FINISH command has previously
//          completed successfully.  Another media recovery is not allowed.
// *Action: Issue one of these operations following a FINISH recocvery:
//          ALTER DATABASE OPEN READ ONLY or
//          ALTER DATABASE COMMIT TO SWITCHOVER TO PRIMARY.

前台虽然提示mrp启动成功，但是alert日志提示启动mrp进程失败，原因是由于media recovery not allowed following successful FINISH recovery，也就是说由于我开始已经执行了ALTER DATABASE RECOVER MANAGED STANDBY DATABASE FINISH导致无法继续启动mrp进程，也就无法继续应用日志。

查看mos看看是否有解决方案
ora-16157

根据官方的说法只能重建dg,实在不甘心,个人感觉ALTER DATABASE RECOVER MANAGED STANDBY DATABASE FINISH执行完成，但是我没有failover命令执行，应该数据文件没有改变，只是控制文件发生了改变。

解决ora-16157问题
从主库重新生成standby controlfile并且传输到备库，再次尝试启动mrp

--主库操作
SQL> ALTER DATABASE CREATE standby CONTROLFILE AS '/tmp/controlfs01.ctl';
Database altered.
--scp到备库
SQL> shutdown immediate;
ORA-01109: database not open
Database dismounted.
ORACLE instance shut down.
SQL> startup nomount;
ORACLE instance started.
Total System Global Area 2.1379E+10 bytes
Fixed Size                  2262656 bytes
Variable Size            2684356992 bytes
Database Buffers         1.8656E+10 bytes
Redo Buffers               36073472 bytes
[oracle@app73 ~]$ cp /tmp/controlfs01.ctl /Data/oracle/oradata/commentdb/control01.ctl
[oracle@app73 ~]$ cp /tmp/controlfs01.ctl /Data/oracle/fast_recovery_area/commentdb/control02.ctl
SQL> alter database mount;
Database altered.
SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT  LOGFILE DISCONNECT FROM SESSION;
Database altered.

观察alert日志

ARC2: Becoming the active heartbeat ARCH
Completed: alter database mount
ARC3: Archival started
ARC0: STARTING ARCH PROCESSES COMPLETE
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT  LOGFILE DISCONNECT FROM SESSION
Attempt to start background Managed Standby Recovery process (commentdb)
Sun Jun 26 23:46:50 2016
MRP0 started with pid=24, OS id=7547
MRP0: Background Managed Standby Recovery process started (commentdb)
Sun Jun 26 23:46:54 2016
RFS[1]: Assigned to RFS process 7553
RFS[1]: Selected log 10 for thread 1 sequence 21197 dbid -414945661 branch 893285763
Sun Jun 26 23:46:54 2016
Primary database is in MAXIMUM PERFORMANCE mode
RFS[2]: Assigned to RFS process 7555
RFS[2]: Selected log 11 for thread 1 sequence 21198 dbid -414945661 branch 893285763
Archived Log entry 1 added for thread 1 sequence 21197 ID 0xe7442b83 dest 1:
 started logmerger process
Sun Jun 26 23:46:55 2016
Managed Standby Recovery starting Real Time Apply
Parallel Media Recovery started with 4 slaves
Waiting for all non-current ORLs to be archived...
All non-current ORLs have been archived.
Sun Jun 26 23:46:56 2016
Completed: ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT  LOGFILE DISCONNECT FROM SESSION
Sun Jun 26 23:47:06 2016
Media Recovery Waiting for thread 1 sequence 21196
Fetching gap sequence in thread 1, gap sequence 21196-21196
Sun Jun 26 23:47:06 2016
RFS[3]: Assigned to RFS process 7567
RFS[3]: Opened log for thread 1 sequence 21196 dbid -414945661 branch 893285763
Archived Log entry 2 added for thread 1 sequence 21196 rlc 893285763 ID 0xe7442b83 dest 2:
Media Recovery Log /Data/oracle/fast_recovery_area/COMMENTDBDG/archivelog/2016_06_26/o1_mf_1_21196_cpzy7tjc_.arc
Media Recovery Log /Data/oracle/fast_recovery_area/COMMENTDBDG/archivelog/2016_06_26/o1_mf_1_21197_cpzy7gtl_.arc
Media Recovery Waiting for thread 1 sequence 21198 (in transit)
Recovery of Online Redo Log: Thread 1 Group 11 Seq 21198 Reading mem 0
  Mem# 0: /Data/oracle/oradata/commentdb/std_redo11.log

到这里已经证明，通过重建standby controlfile实现了即使执行了ALTER DATABASE RECOVER MANAGED STANDBY DATABASE FINISH(ORA-16157)也可以通过不重建dg,让其恢复正常(恢复dg状态).

pvid=yes导致asm无法mount

今天凌晨接到客户恢复请求，对于aix rac，两个ibm存储做mirror的环境中，客户做存储容灾演练，发现磁盘的名称发生改变，然后对其中一个磁盘设置pvid，结果悲剧了导致asm一个磁盘组无法正常起来。然后又aix端删除这些设备，然后重新扫描设备。结果不是一个磁盘组不能mount，而是整个gi就无法正常启动。希望我们给予技术支持。
查看asm 日志，确定asm disk信息
asm-disk1
asm-disk2

从这里可以确定，一共有两个asm diskgroup，每个group有两个磁盘，hdisk2和hdisk3 为hisdata，hdisk4,和hdisk5为emrdata.

使用kfed分析磁盘头

dd if=/dev/rhdisk2 of=/tmp/xifenfei/rhdisk2.dd bs=1024k count=10
dd if=/dev/rhdisk3 of=/tmp/xifenfei/rhdisk3.dd bs=1024k count=10
dd if=/dev/rhdisk4 of=/tmp/xifenfei/rhdisk4.dd bs=1024k count=10
dd if=/dev/rhdisk5 of=/tmp/xifenfei/rhdisk5.dd bs=1024k count=10
--传输到我电脑上分析
C:\Users\FAL>kfed read H:\temp\xifenfei\tmp\xifenfei\rhdisk2.dd|grep name
kfdhdb.dskname:            HISDATA_0000 ; 0x028: length=12
kfdhdb.grpname:                 HISDATA ; 0x048: length=7
kfdhdb.fgname:             HISDATA_0000 ; 0x068: length=12
kfdhdb.capname:                         ; 0x088: length=0
C:\Users\FAL>kfed read H:\temp\xifenfei\tmp\xifenfei\rhdisk3.dd|grep name
kfdhdb.dskname:            HISDATA_0001 ; 0x028: length=12
kfdhdb.grpname:                 HISDATA ; 0x048: length=7
kfdhdb.fgname:             HISDATA_0001 ; 0x068: length=12
kfdhdb.capname:                         ; 0x088: length=0
C:\Users\FAL>kfed read H:\temp\xifenfei\tmp\xifenfei\rhdisk4.dd|grep name
kfdhdb.dskname:            EMRDATA_0000 ; 0x028: length=12
kfdhdb.grpname:                 EMRDATA ; 0x048: length=7
kfdhdb.fgname:             EMRDATA_0000 ; 0x068: length=12
kfdhdb.capname:                         ; 0x088: length=0
C:\Users\FAL>kfed read H:\temp\xifenfei\tmp\xifenfei\rhdisk5.dd|grep name
C:\Users\FAL>kfed read H:\temp\xifenfei\tmp\xifenfei\rhdisk5.dd
kfbh.endian:                        201 ; 0x000: 0xc9
kfbh.hard:                          194 ; 0x001: 0xc2
kfbh.type:                          212 ; 0x002: *** Unknown Enum ***
kfbh.datfmt:                        193 ; 0x003: 0xc1
kfbh.block.blk:                       0 ; 0x004: blk=0
kfbh.block.obj:                       0 ; 0x008: file=0
kfbh.check:                           0 ; 0x00c: 0x00000000
kfbh.fcn.base:                        0 ; 0x010: 0x00000000
kfbh.fcn.wrap:                        0 ; 0x014: 0x00000000
kfbh.spare1:                          0 ; 0x018: 0x00000000
kfbh.spare2:                          0 ; 0x01c: 0x00000000
000000000 C1D4C2C9 00000000 00000000 00000000  [................]
000000010 00000000 00000000 00000000 00000000  [................]
  Repeat 254 times
KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type][][212]
C:\Users\FAL>kfed read H:\temp\xifenfei\tmp\xifenfei\rhdisk5.dd blkn=2|grep kfbh
kfbh.endian:                          0 ; 0x000: 0x00
kfbh.hard:                          130 ; 0x001: 0x82
kfbh.type:                            3 ; 0x002: KFBTYP_ALLOCTBL
kfbh.datfmt:                          2 ; 0x003: 0x02
kfbh.block.blk:                33554432 ; 0x004: blk=33554432
kfbh.block.obj:                16777344 ; 0x008: file=128
kfbh.check:                  2654889601 ; 0x00c: 0x9e3e6681
kfbh.fcn.base:               1696071680 ; 0x010: 0x65180000
kfbh.fcn.wrap:                        0 ; 0x014: 0x00000000
kfbh.spare1:                          0 ; 0x018: 0x00000000
kfbh.spare2:                          0 ; 0x01c: 0x00000000
C:\Users\FAL>kfed read H:\temp\xifenfei\tmp\xifenfei\rhdisk5.dd blkn=510|grep name
kfdhdb.dskname:            EMRDATA_0001 ; 0x028: length=12
kfdhdb.grpname:                 EMRDATA ; 0x048: length=7
kfdhdb.fgname:             EMRDATA_0001 ; 0x068: length=12
kfdhdb.capname:                         ; 0x088: length=0

通过上述分析，基本上确定由于对hdisk5设置了pvid导致该asm disk的磁盘头损坏.这个可以直接使用asm repair功能修复(注意要clear pvid)

C:\Users\FAL>kfed read H:\temp\xifenfei\tmp\xifenfei\rhdisk5.dd |grep name
kfdhdb.dskname:            EMRDATA_0001 ; 0x028: length=12
kfdhdb.grpname:                 EMRDATA ; 0x048: length=7
kfdhdb.fgname:             EMRDATA_0001 ; 0x068: length=12
kfdhdb.capname:                         ; 0x088: length=0

启动crs到cssd进程报错分析
1. 由于删除磁盘，扫描设备导致hdisk[2-5] 权限和用户组不对
2. 由于删除，扫描磁盘导致磁盘共享模式不对
修复磁盘头和解决这两个问题之后,gi启动正常,磁盘组也正常mount，数据库也正常启动,数据0丢失,至此完美恢复
oracle-open

类似客户恢复案例:asm disk误设置pvid导致asm diskgroup无法mount恢复
如果您遇到此类情况,无法解决请联系我们，提供专业ORACLE数据库恢复技术支持
Phone:17813235971    Q Q:107644445QQ咨询惜分飞    E-Mail:dba@xifenfei.com

使用losetup实现linux普通文件做asm disk

上一篇文章《使用_asm_allow_only_raw_disks实现普通文件做asm disk》中已经介绍使用_asm_allow_only_raw_disks参数使得oracle asm可以使用文件作为asm disk,这篇文章介绍在linux中还可以通过losetup来实现文件系统模拟磁盘实现使用文件系统做asm disk的效果
通过dd构造文件

[oracle@xifenfei ~]$ mkdir /u01/oracle/oradata/asmdisk
[oracle@xifenfei ~]$ dd if=/dev/zero of=/u01/oracle/oradata/asmdisk/xifenfei01.dd bs=10240k count=100
100+0 records in
100+0 records out
1048576000 bytes (1.0 GB) copied, 21.9158 seconds, 47.8 MB/s
[oracle@xifenfei ~]$ dd if=/dev/zero of=/u01/oracle/oradata/asmdisk/xifenfei02.dd bs=10240k count=100
100+0 records in
100+0 records out
1048576000 bytes (1.0 GB) copied, 22.392 seconds, 46.8 MB/s
[oracle@xifenfei ~]$ ls -lh /u01/oracle/oradata/asmdisk/
total 3.0G
-rw-r--r-- 1 oracle oinstall 1000M Feb 27 22:58 xifenfei01.dd
-rw-r--r-- 1 oracle oinstall 1000M Feb 27 23:00 xifenfei02.dd

使用losetup模拟磁盘

[root@xifenfei asmdisk]# ls -l /dev/lo
log    loop0  loop1  loop2  loop3  loop4  loop5  loop6  loop7
[root@xifenfei asmdisk]# losetup /dev/loop1 xifenfei01.dd
[root@xifenfei asmdisk]# losetup /dev/loop2 xifenfei02.dd

使用raw实现磁盘转换为裸设备

[root@xifenfei asmdisk]# raw  /dev/raw/raw10 /dev/loop1
/dev/raw/raw10: bound to major 7, minor 1
[root@xifenfei asmdisk]# raw  /dev/raw/raw11 /dev/loop2
/dev/raw/raw11: bound to major 7, minor 2
[root@xifenfei asmdisk]# ls -l /dev/raw/raw1[0-1]
crw------- 1 root root 162, 10 Feb 27 23:16 /dev/raw/raw10
crw------- 1 root root 162, 11 Feb 27 23:16 /dev/raw/raw11
[root@xifenfei asmdisk]# chown oracle.dba /dev/raw/raw1[0-1]
[root@xifenfei asmdisk]# ls -l /dev/raw/raw1[0-1]
crw------- 1 oracle dba 162, 10 Feb 27 23:16 /dev/raw/raw10
crw------- 1 oracle dba 162, 11 Feb 27 23:16 /dev/raw/raw11

创建磁盘组

[oracle@xifenfei ~]$ export ORACLE_SID=+ASM
[oracle@xifenfei ~]$ sqlplus / as sysdba
SQL*Plus: Release 10.2.0.4.0 - Production on Thu Feb 27 23:19:28 2014
Copyright (c) 1982, 2007, Oracle.  All Rights Reserved.
Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
SQL>  create diskgroup xff external redundancy disk '/dev/raw/raw10','/dev/raw/raw11';
Diskgroup created.
SQL> select group_number,name from v$asm_diskgroup;
GROUP_NUMBER NAME
------------ ------------------------------------------------------------
           1 DATA
           2 XFF
SQL> select path,TOTAL_MB from v$asm_disk where group_number=2;
PATH                   TOTAL_MB
-------------------- ----------
/dev/raw/raw11             1000
/dev/raw/raw10             1000

通过kfed验证asm disk是数据文件

[oracle@xifenfei tmp]$ kfed read /dev/raw/raw10|grep XFF
kfdhdb.dskname:                XFF_0000 ; 0x028: length=8
kfdhdb.grpname:                     XFF ; 0x048: length=3
kfdhdb.fgname:                 XFF_0000 ; 0x068: length=8
[oracle@xifenfei tmp]$ kfed read /dev/raw/raw11|grep XFF
kfdhdb.dskname:                XFF_0001 ; 0x028: length=8
kfdhdb.grpname:                     XFF ; 0x048: length=3
kfdhdb.fgname:                 XFF_0001 ; 0x068: length=8
[oracle@xifenfei tmp]$ kfed read /u01/oracle/oradata/asmdisk/xifenfei01.dd |grep XFF
kfdhdb.dskname:                XFF_0000 ; 0x028: length=8
kfdhdb.grpname:                     XFF ; 0x048: length=3
kfdhdb.fgname:                 XFF_0000 ; 0x068: length=8
[oracle@xifenfei tmp]$ kfed read /u01/oracle/oradata/asmdisk/xifenfei02.dd |grep XFF
kfdhdb.dskname:                XFF_0001 ; 0x028: length=8
kfdhdb.grpname:                     XFF ; 0x048: length=3
kfdhdb.fgname:                 XFF_0001 ; 0x068: length=8

通过kfed命令,确定asm本质是用了dd出来的数据文件做asm disk.

使用_asm_allow_only_raw_disks实现普通文件做asm disk

不少人可能都有一个烦恼，我们平时使用的是win系统，但是想测试下asm，很可能还要到linux的虚拟机里面，这样太麻烦了。如果能够使用win文件系统里面的文件直接做asm disk，直接给asm 使用那就省事了。这篇文章就描述10g中如何实现win 文件系统中的文件做asm disk.注意:如果11g需要安装grid
准备两个文件

h:\ASMDISK>dd if=/dev/zero of=asmdisk1.dd bs=10240k count=200
rawwrite dd for windows version 0.6beta3.
Written by John Newbigin <jn@it.swin.edu.au>
This program is covered by terms of the GPL Version 2.
200+0 records in
200+0 records out
h:\ASMDISK>dd if=/dev/zero of=asmdisk2.dd bs=10240k count=200
rawwrite dd for windows version 0.6beta3.
Written by John Newbigin <jn@it.swin.edu.au>
This program is covered by terms of the GPL Version 2.
200+0 records in
200+0 records out
h:\ASMDISK>dir
 驱动器 H 中的卷没有标签。
 卷的序列号是 360E-41A8
 h:\ASMDISK 的目录
2016-06-17  22:02    <DIR>          .
2016-06-17  22:02    <DIR>          ..
2016-06-17  22:30     2,097,152,000 asmdisk1.dd
2016-06-17  22:30     2,097,152,000 asmdisk2.dd
               2 个文件  4,194,304,000 字节
               2 个目录 574,891,098,112 可用字节

这里使用dd来构造文件，当然你也可以通过asmtools来实现

增加css服务

C:\Windows\system32>d:/app\product\10.2.0\db_1\bin\localconfig add
Step 1:  creating new OCR repository
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'fal', privgrp ''..
Operation successful.
Step 2:  creating new CSS service
successfully created local CSS service
successfully added CSS to home

使用文件创建asm 磁盘组
dbca选择自动存储管理，并且到选择磁盘步骤终止,因为没有磁盘分区给asm使用

h:\ASMDISK>set ORACLE_SID=+ASM
D:\app\product\10.2.0\db_1\bin>SQLPLUS / AS SYSDBA
SQL*Plus: Release 10.2.0.3.0 - Production on 星期五 6月 17 22:08:57 2016
Copyright (c) 1982, 2006, Oracle.  All Rights Reserved.
连接到:
Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - Production
With the Partitioning, OLAP and Data Mining options
SQL> alter system set "_asm_allow_only_raw_disks"=false
  2  ;
alter system set "_asm_allow_only_raw_disks"=false
                 *
第 1 行出现错误:
ORA-02095: ????????????
SQL> alter system set "_asm_allow_only_raw_disks"=false scope=spfile;
系统已更改。
SQL> shutdown immediate;
ORA-15100: ??????????
ASM 实例已关闭
SQL> startup
ASM 实例已启动
Total System Global Area   83886080 bytes
Fixed Size                  1289028 bytes
Variable Size              57431228 bytes
ASM Cache                  25165824 bytes
ORA-15110: ??????
SQL> create diskgroup data external redundancy disk 'H:\asmdisk\ASMDISK1.dd','H:\asmdisk\ASMDISK2.dd';
create diskgroup data external redundancy disk 'H:\asmdisk\ASMDISK1.dd','H:\asmdisk\ASMDISK2.dd'
*
第 1 行出现错误:
ORA-15018: ???????
ORA-15031: ???? 'H:\asmdisk\ASMDISK2.dd' ?????????
ORA-15014: ?? 'H:\ASMDISK\ASMDISK2.DD' ??????
ORA-15031: ???? 'H:\asmdisk\ASMDISK1.dd' ?????????
ORA-15014: ?? 'H:\ASMDISK\ASMDISK1.DD' ??????
SQL> show parameter asm;
NAME                                 TYPE
------------------------------------ ----------------------
VALUE
------------------------------
_asm_allow_only_raw_disks            boolean
FALSE
asm_diskgroups                       string
asm_diskstring                       string
asm_power_limit                      integer
1
SQL> alter system set asm_diskstring="H:\asmdisk\*.dd" ;
系统已更改。
SQL> create diskgroup data external redundancy disk 'H:\asmdisk\ASMDISK1.dd','H:\asmdisk\ASMDISK2.dd';
磁盘组已创建。
SQL> select path from v$asm_disk;
PATH
--------------------------------------------------------------------------------
H:\ASMDISK\ASMDISK1.DD
H:\ASMDISK\ASMDISK2.DD

这一步你也可以通过ORADIM创建asm服务,然后创建asm参数文件,然后启动asm,创建磁盘组

dbca创建数据库选择asm
asm-win-file

实现效果(普通file做asm disk)
win-file-asm-disk

同理在linux等平台上也可以使用_asm_allow_only_raw_disks参数实现文件系统做asm disk

oracle asm disk格式化恢复—格式化为ext4文件系统

昨天中午接到一位朋友紧急求救电话,大概场景如下,asm data磁盘组一共把个asm disk,但是使用4个lun实现的(也就是说每个lun使用fdisk进行分区),该主机上还有一个lun是用来存放备份的挂载在/xifenfei目录.客户记得他们的/xifenfei目录是/dev/sdh1这个分区,上次主机重启之后,rac工作正常,但是/xifenfei这个目录对应的/dev/sdh1无法挂载上去(提示无法找到超级块).然后我这位朋友上去也不管三七二一直接执行了mkfs.ext4 /dev/sdh1.结果整个asm 磁盘组异常了.最后他通过fdisk命令一看发现完蛋了,以前的/dev/sdh 已经变为了/dev/sdc,而现在的/dev/sdh是以前的asm disk.也就是说,他把asm 磁盘组中的一个disk进行了格式化为ext4文件系统操作.
ext4文件系统大概结构
通过这里大概可以发现在格式化为ext4文件系统并非把所有的磁盘数据全部重写主要就是覆盖一些ext4必要的一些元数据信息,理论上没有覆盖部分的数据依旧可以恢复
ext4

fdisk 分析现在磁盘情况

[root@db3 ~]#  fdisk -l
Disk /dev/sda: 171.8 GB, 171798691840 bytes
255 heads, 63 sectors/track, 20886 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00055b12
   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          64      512000   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2              64        8223    65536000   82  Linux swap / Solaris
/dev/sda3            8223       20887   101723136   8e  Linux LVM
Disk /dev/sdb: 751.6 GB, 751619276800 bytes
255 heads, 63 sectors/track, 91379 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00b6888f
   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1       91379   734001786    5  Extended
/dev/sdb5               1       15000   120487437   83  Linux
Disk /dev/sdd: 1073 MB, 1073741824 bytes
34 heads, 61 sectors/track, 1011 cylinders
Units = cylinders of 2074 * 512 = 1061888 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00083b68
   Device Boot      Start         End      Blocks   Id  System
Disk /dev/sdf: 526.1 GB, 526133493760 bytes
255 heads, 63 sectors/track, 63965 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000c10d8
   Device Boot      Start         End      Blocks   Id  System
/dev/sdf1               1       13054   104856223+  83  Linux
/dev/sdf2           13055       63965   408942607+  83  Linux
Disk /dev/sde: 1073 MB, 1073741824 bytes
34 heads, 61 sectors/track, 1011 cylinders
Units = cylinders of 2074 * 512 = 1061888 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000b120f
   Device Boot      Start         End      Blocks   Id  System
Disk /dev/sdc: 1795.3 GB, 1795296329728 bytes
255 heads, 63 sectors/track, 218265 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x78c377f7
   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1      218265  1753213581   83  Linux
Disk /dev/sdg: 526.1 GB, 526133493760 bytes
255 heads, 63 sectors/track, 63965 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000ee648
   Device Boot      Start         End      Blocks   Id  System
/dev/sdg1               1       13054   104856223+  83  Linux
/dev/sdg2           13055       63965   408942607+  83  Linux
Disk /dev/sdi: 526.1 GB, 526133493760 bytes
255 heads, 63 sectors/track, 63965 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00027e4b
   Device Boot      Start         End      Blocks   Id  System
/dev/sdi1               1       13054   104856223+  83  Linux
/dev/sdi2           13055       63965   408942607+  83  Linux
Disk /dev/sdh: 526.1 GB, 526133493760 bytes
255 heads, 63 sectors/track, 63965 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000938dc
   Device Boot      Start         End      Blocks   Id  System
/dev/sdh1               1       13054   104856223+  83  Linux
/dev/sdh2           13055       63965   408942607+  83  Linux
Disk /dev/mapper/rootvg-rootfs: 31.7 GB, 31708938240 bytes
255 heads, 63 sectors/track, 3855 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/mapper/rootvg-lv01: 32.2 GB, 32212254720 bytes
255 heads, 63 sectors/track, 3916 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

通过asm alert日志确定asm disk信息
这里可以确定asm disk是VOL01-VOL08一共8个盘

SQL> CREATE DISKGROUP DATA EXTERNAL REDUNDANCY  DISK '/dev/oracleasm/disks/VOL01',
'/dev/oracleasm/disks/VOL02',
'/dev/oracleasm/disks/VOL03',
'/dev/oracleasm/disks/VOL04' ATTRIBUTE 'compatible.asm'='11.2.0.0.0','au_size'='1M' /* ASMCA */
SQL> ALTER DISKGROUP DATA ADD  DISK '/dev/oracleasm/disks/VOL05' SIZE 399358M ,
'/dev/oracleasm/disks/VOL06' SIZE 399358M ,
'/dev/oracleasm/disks/VOL07' SIZE 399358M ,
'/dev/oracleasm/disks/VOL08' SIZE 399358M /* ASMCA */
SQL> ALTER DISKGROUP ALL MOUNT /* asm agent call crs *//* {0:0:2} */
NOTE: Diskgroup used for Voting files is:
 	 DATA
Diskgroup with spfile:DATA
Diskgroup used for OCR is:DATA
NOTE: cache registered group DATA number=1 incarn=0x6f3f6eb8
NOTE: cache began mount (not first) of group DATA number=1 incarn=0x6f3f6eb8
NOTE: Assigning number (1,7) to disk (/dev/oracleasm/disks/VOL08)
NOTE: Assigning number (1,6) to disk (/dev/oracleasm/disks/VOL07)
NOTE: Assigning number (1,5) to disk (/dev/oracleasm/disks/VOL06)
NOTE: Assigning number (1,4) to disk (/dev/oracleasm/disks/VOL05)
NOTE: Assigning number (1,3) to disk (/dev/oracleasm/disks/VOL04)
NOTE: Assigning number (1,1) to disk (/dev/oracleasm/disks/VOL02)
NOTE: Assigning number (1,2) to disk (/dev/oracleasm/disks/VOL03)
NOTE: Assigning number (1,0) to disk (/dev/oracleasm/disks/VOL01)
GMON querying group 1 at 2 for pid 23, osid 32026
NOTE: cache opening disk 0 of grp 1: DATA_0000 path:/dev/oracleasm/disks/VOL01
NOTE: cache opening disk 1 of grp 1: DATA_0001 path:/dev/oracleasm/disks/VOL02
NOTE: cache opening disk 2 of grp 1: DATA_0002 path:/dev/oracleasm/disks/VOL03
NOTE: cache opening disk 3 of grp 1: DATA_0003 path:/dev/oracleasm/disks/VOL04
NOTE: cache opening disk 4 of grp 1: DATA_0004 path:/dev/oracleasm/disks/VOL05
NOTE: cache opening disk 5 of grp 1: DATA_0005 path:/dev/oracleasm/disks/VOL06
NOTE: cache opening disk 6 of grp 1: DATA_0006 path:/dev/oracleasm/disks/VOL07
NOTE: cache opening disk 7 of grp 1: DATA_0007 path:/dev/oracleasm/disks/VOL08
NOTE: cache mounting (not first) external redundancy group 1/0x6F3F6EB8 (DATA)

查询asm对应的disk信息
这部分信息,可以确定VOL03被格式化,而且应该就是朋友反馈的被他格式化成ext4的/dev/sdh1
asm1

[root@db3 disks]# ls
VOL01  VOL02  VOL04  VOL05  VOL06  VOL07  VOL08
[root@db3 disks]# oracleasm querydisk -p 'VOL01'
Disk "VOL01" is a valid ASM disk
/dev/sdf1: LABEL="VOL01" TYPE="oracleasm"
[root@db3 disks]# oracleasm querydisk -p 'VOL02'
Disk "VOL02" is a valid ASM disk
/dev/sdg1: LABEL="VOL02" TYPE="oracleasm"
[root@db3 disks]# oracleasm querydisk -p 'VOL03'
Disk "VOL03" does not exist or is not instantiated
[root@db3 disks]# oracleasm querydisk -p 'VOL04'
Disk "VOL04" is a valid ASM disk
/dev/sdi1: LABEL="VOL04" TYPE="oracleasm"
[root@db3 disks]# oracleasm querydisk -p 'VOL05'
Disk "VOL05" is a valid ASM disk
/dev/sdf2: LABEL="VOL05" TYPE="oracleasm"
[root@db3 disks]# oracleasm querydisk -p 'VOL06'
Disk "VOL06" is a valid ASM disk
/dev/sdg2: LABEL="VOL06" TYPE="oracleasm"
[root@db3 disks]# oracleasm querydisk -p 'VOL07'
Disk "VOL07" is a valid ASM disk
/dev/sdh2: LABEL="VOL07" TYPE="oracleasm"
[root@db3 disks]# oracleasm querydisk -p 'VOL08'
Disk "VOL08" is a valid ASM disk
/dev/sdi2: LABEL="VOL08" TYPE="oracleasm"

通过kfed分析被格式化成ext4的磁盘信息

[grid@db3 ~]$ kfed read /dev/sdh1
kfbh.endian:                          0 ; 0x000: 0x00
kfbh.hard:                            0 ; 0x001: 0x00
kfbh.type:                            0 ; 0x002: KFBTYP_INVALID
kfbh.datfmt:                          0 ; 0x003: 0x00
kfbh.block.blk:                       0 ; 0x004: blk=0
kfbh.block.obj:                       0 ; 0x008: file=0
kfbh.check:                   810307429 ; 0x00c: 0x304c4f65
kfbh.fcn.base:                        0 ; 0x010: 0x00000000
kfbh.fcn.wrap:                        0 ; 0x014: 0x00000000
kfbh.spare1:                          0 ; 0x018: 0x00000000
kfbh.spare2:                          0 ; 0x01c: 0x00000000
000000000 00000000 00000000 00000000 304C4F65  [............eOL0]
000000010 00000000 00000000 00000000 00000000  [................]
000000020 4C43524F 44524C43 00000000 00000000  [ORCLCLRD........]
000000030 00000000 00000000 00000000 00000000  [................]
  Repeat 252 times
KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type][][0]
[grid@db3 ~]$ kfed read /dev/sdh1 aun=1
kfbh.endian:                          0 ; 0x000: 0x00
kfbh.hard:                          129 ; 0x001: 0x81
kfbh.type:                            0 ; 0x002: KFBTYP_INVALID
kfbh.datfmt:                          0 ; 0x003: 0x00
kfbh.block.blk:                   98560 ; 0x004: blk=98560
kfbh.block.obj:                  164096 ; 0x008: file=164096
kfbh.check:                      229632 ; 0x00c: 0x00038100
kfbh.fcn.base:                   295168 ; 0x010: 0x00048100
kfbh.fcn.wrap:                   819456 ; 0x014: 0x000c8100
kfbh.spare1:                     884992 ; 0x018: 0x000d8100
kfbh.spare2:                    1605888 ; 0x01c: 0x00188100
000100000 00008100 00018100 00028100 00038100  [................]
000100010 00048100 000C8100 000D8100 00188100  [................]
000100020 00288100 003E8100 00798100 00AB8100  [..(...>...y.....]
000100030 01388100 016C8100 00000000 00000000  [..8...l.........]
000100040 00000000 00000000 00000000 00000000  [................]
  Repeat 251 times
KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type][][0]
[grid@db3 ~]$ kfed read /dev/sdh1 aun=1 blkn=254
kfbh.endian:                        254 ; 0x000: 0xfe
kfbh.hard:                          129 ; 0x001: 0x81
kfbh.type:                            0 ; 0x002: KFBTYP_INVALID
kfbh.datfmt:                          0 ; 0x003: 0x00
kfbh.block.blk:                   98814 ; 0x004: blk=98814
kfbh.block.obj:                  164350 ; 0x008: file=164350
kfbh.check:                      229886 ; 0x00c: 0x000381fe
kfbh.fcn.base:                   295422 ; 0x010: 0x000481fe
kfbh.fcn.wrap:                   819710 ; 0x014: 0x000c81fe
kfbh.spare1:                     885246 ; 0x018: 0x000d81fe
kfbh.spare2:                    1606142 ; 0x01c: 0x001881fe
0001FE000 000081FE 000181FE 000281FE 000381FE  [................]
0001FE010 000481FE 000C81FE 000D81FE 001881FE  [................]
0001FE020 002881FE 003E81FE 007981FE 00AB81FE  [..(...>...y.....]
0001FE030 013881FE 016C81FE 00000000 00000000  [..8...l.........]
0001FE040 00000000 00000000 00000000 00000000  [................]
  Repeat 251 times
KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type][][0]
[grid@db3 ~]$ kfed read /dev/sdh1 aun=9
kfbh.endian:                          0 ; 0x000: 0x00
kfbh.hard:                            0 ; 0x001: 0x00
kfbh.type:                            0 ; 0x002: KFBTYP_INVALID
kfbh.datfmt:                          0 ; 0x003: 0x00
kfbh.block.blk:                       0 ; 0x004: blk=0
kfbh.block.obj:                       0 ; 0x008: file=0
kfbh.check:                           0 ; 0x00c: 0x00000000
kfbh.fcn.base:                        0 ; 0x010: 0x00000000
kfbh.fcn.wrap:                        0 ; 0x014: 0x00000000
kfbh.spare1:                          0 ; 0x018: 0x00000000
kfbh.spare2:                          0 ; 0x01c: 0x00000000
000900000 00000000 00000000 00000000 00000000  [................]
  Repeat 255 times
KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type][][0]

从这里可以看出来该asm的元数据信息大量被破坏(asm 备份的磁盘头也损坏),直接修复的可能性基本上为0,现在为了最大限度的恢复数据,通过底层扫描asm au结合数据库block信息,进行重组出来数据文件(这次的恢复有点难度,因为该asm磁盘组中有两个数据库,需要通过结合asm file+datafile来区分文件)
恢复出来两套库的数据文件信息

/orabak/xifenfei/oradb/1.dbf
/orabak/xifenfei/oradb/2.dbf
/orabak/xifenfei/oradb/3.dbf
/orabak/xifenfei/oradb/4.dbf
/orabak/xifenfei/oradb/5.dbf
/orabak/xifenfei/oradb/6.dbf
/orabak/xifenfei/oradb/7.dbf
/orabak/xifenfei/oradb/8.dbf
/orabak/xifenfei/oradb/9.dbf
/orabak/xifenfei/oradb/10.dbf
/orabak/xifenfei/oradb/11.dbf
/orabak/xifenfei/oradb/12.dbf
/orabak/xifenfei/oradb/13.dbf
/orabak/xifenfei/oradb/14.dbf
/orabak/xifenfei/oradb/15.dbf
/orabak/xifenfei/oradb/16.dbf
/orabak/xifenfei/oradb/17.dbf
/orabak/xifenfei/oradb/18.dbf
/orabak/xifenfei/oradb/19.dbf
/orabak/xifenfei/xgdb/1.dbf
/orabak/xifenfei/xgdb/2.dbf
/orabak/xifenfei/xgdb/3.dbf
/orabak/xifenfei/xgdb/4.dbf
/orabak/xifenfei/xgdb/5.dbf
/orabak/xifenfei/xgdb/6.dbf
/orabak/xifenfei/xgdb/7.dbf
/orabak/xifenfei/xgdb/8.dbf
/orabak/xifenfei/xgdb/9.dbf
/orabak/xifenfei/xgdb/10.dbf
/orabak/xifenfei/xgdb/11.dbf
/orabak/xifenfei/xgdb/12.dbf
/orabak/xifenfei/xgdb/13.dbf
/orabak/xifenfei/xgdb/14.dbf
/orabak/xifenfei/xgdb/15.dbf
/orabak/xifenfei/xgdb/16.dbf

然后使用工具拷贝出来redo信息,最后通过重建控制文件.其中一套库完美直接open,另外一套库system中的c_obj$被覆盖,不过使用一个多月以前的备份的system文件强制打开库成功,数据基本上完美导出,实现完美恢复.由于在格式化为ext4的时候,会在磁盘中部分位置写入一些
数据文件恢复参考:asm disk header 彻底损坏恢复
另外有一次win平台类似恢复经历:asm disk格式化为ntfs恢复
如果您遇到此类情况,无法解决请联系我们，提供专业ORACLE数据库恢复技术支持
Phone:17813235971    Q Q:107644445QQ咨询惜分飞    E-Mail:dba@xifenfei.com

oracle dul 11 正式发布

千呼万唤oracle dul 11终于发布了,迫不及待的下载来测试,现在版本号为dul 11.2.0.0.1,目前只发布了for linux,其他版本估计要等等.期待该版本有引进新功能
oracle dul 11发布
dul-11.2
oracle-dul-11

dul 11 测试恢复oracle 12c版本

[root@ora1221 dul11]# ./dul
Data UnLoader: 11.2.0.0.1 - Internal Only - on Mon Jun 13 00:04:50 2016
with 64-bit io functions
Copyright (c) 1994 2016 Bernard van Duijnen All rights reserved.
 Strictly Oracle Internal Use Only
DUL: Warning: Recreating file "dul.log"
Found db_id = 1414800805
Found db_name = ORCL
DUL> bootstrap;
Probing file = 1, block = 520
. unloading table                BOOTSTRAP$
DUL: Warning: block number is non zero but marked deferred trying to process it anyhow
      60 rows unloaded
DUL: Warning: Dictionary cache DC_BOOTSTRAP is empty
Reading BOOTSTRAP.dat 60 entries loaded
Parsing Bootstrap$ contents
DUL: Warning: Recreating file "dict.ddl"
Generating dict.ddl for version 12
 OBJ$: segobjno 18, file 1 block 240
 TAB$: segobjno 2, tabno 1, file 1  block 144
 COL$: segobjno 2, tabno 5, file 1  block 144
 USER$: segobjno 10, tabno 1, file 1  block 208
Running generated file "@dict.ddl" to unload the dictionary tables
. unloading table                      OBJ$   72547 rows unloaded
. unloading table                      TAB$    2137 rows unloaded
. unloading table                      COL$  115645 rows unloaded
. unloading table                     USER$     129 rows unloaded
Reading USER.dat 129 entries loaded
Reading OBJ.dat
DUL: FATAL Error: File OBJ.dat, line 22094: identifier too long

dul 11 恢复oracle 12c 报错分析
看报错OBJ.dat报错数据,从定义上判断是长度过长

[root@ora1221 dul11]# vi OBJ.dat
  22091 "22435" "" "0" "JVMFCB" "" "9" ""
  22092 "22436" "" "0" "JVMFCB" "" "11" ""
  22093 "22437" "56417" "0" "JAVA$CBOBJ$" "" "2" ""
----  22094 "22439" "" "0" "sun/util/xml/PlatformXmlPropertiesProvider$Resolver" "" "29" ""
  22095 "22440" "" "0" "sun/util/xml/PlatformXmlPropertiesProvider$EH" "" "29" ""
  22096 "22441" "" "0" "sun/util/xml/PlatformXmlPropertiesProvider$1" "" "29" ""
  22097 "22442" "" "0" "sun/util/resources/LocaleNamesBundle" "" "29" ""

对象名长度分析

[oracle@ora1221 ~]$ ss
SQL*Plus: Release 12.2.0.0.3 Production on Fri Aug 7 02:50:35 2015
Copyright (c) 1982, 2016, Oracle.  All rights reserved.
Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.0.3 - 64bit Production
SQL> select length(object_name) from dba_objects
  2  where object_name='sun/util/xml/PlatformXmlPropertiesProvider$Resolver';
LENGTH(OBJECT_NAME)
-------------------
                 51
SQL> select max(length(object_name)) from dba_objects;
MAX(LENGTH(OBJECT_NAME))
------------------------
                     123
SQL> desc dba_objects
 Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 OWNER                                              VARCHAR2(128)
 OBJECT_NAME                                        VARCHAR2(128)
 SUBOBJECT_NAME                                     VARCHAR2(128)
 OBJECT_ID                                          NUMBER
 DATA_OBJECT_ID                                     NUMBER
 OBJECT_TYPE                                        VARCHAR2(23)
 CREATED                                            DATE
 LAST_DDL_TIME                                      DATE
 TIMESTAMP                                          VARCHAR2(19)
 STATUS                                             VARCHAR2(7)
 TEMPORARY                                          VARCHAR2(1)
 GENERATED                                          VARCHAR2(1)
 SECONDARY                                          VARCHAR2(1)
 NAMESPACE                                          NUMBER
 EDITION_NAME                                       VARCHAR2(128)
 SHARING                                            VARCHAR2(18)
 EDITIONABLE                                        VARCHAR2(1)
 ORACLE_MAINTAINED                                  VARCHAR2(1)
 APPLICATION                                        VARCHAR2(1)
 DEFAULT_COLLATION                                  VARCHAR2(100)
 DUPLICATED                                         VARCHAR2(1)
 SHARDED                                            VARCHAR2(1)
 CREATED_APPID                                      NUMBER
 CREATED_VSNID                                      NUMBER
 MODIFIED_APPID                                     NUMBER
 MODIFIED_VSNID                                     NUMBER

在oracle 12c中对象名长度已经变为128,但是dul依旧不能支持列长度，既然作者已经在init.dul的Compatible参数曾增加了对12c的支持,那对这样明显的bug应该在不久的将来即可修复。当然这个问题,可以通过一些人工技巧绕过去

_use_single_log_writer和_max_outstanding_log_writes

SCALABLE LGWR是12cR1中引入的一个令人激动的特性， 这是由于在OLTP环境中LGWR写日志往往成为系统的主要性能瓶颈， 如果LGWR进程能像DBWR(DBW0~DBWn)那样多进程（LGNN）写出redo到LOGFILE那么就可能大幅释放OLTP的并发能力，增长Transcation系统的单位时间事务处理能力。这里在12.2版本中进行测试，确定_use_single_log_writer和_max_outstanding_log_writes参数对于SCALABLE LGWR特性的影响
数据库版本

SQL> select * from v$version;
BANNER                                                                                         CON_ID
------------------------------------------------------------------------------------------ ----------
Oracle Database 12c Enterprise Edition Release 12.2.0.0.3 - 64bit Production                        0
PL/SQL Release 12.2.0.0.3 - Production                                                              0
CORE    12.2.0.0.3      Production                                                                  0
TNS for Linux: Version 12.2.0.0.3 - Production                                                      0
NLSRTL Version 12.2.0.0.3 - Production                                                              0

_use_single_log_writer和_max_outstanding_log_writes默认值

SQL> /
Enter value for param: _use_single_log_writer
old   6:    and upper(a.ksppinm) LIKE upper('%&param%')
new   6:    and upper(a.ksppinm) LIKE upper('%_use_single_log_writer%')
NAME                                                 VALUE                    DESCRIPTION
---------------------------------------------------- ------------------------ -------------------------------------------
_use_single_log_writer                               ADAPTIVE                 Use a single process for redo log writing
SQL> /
Enter value for param: _max_outstanding_log_writes
old   6:    and upper(a.ksppinm) LIKE upper('%&param%')
new   6:    and upper(a.ksppinm) LIKE upper('%_max_outstanding_log_writes%')
NAME                                                 VALUE                    DESCRIPTION
---------------------------------------------------- ------------------------ ----------------------------------------------
_max_outstanding_log_writes                          2                        Maximum number of outstanding redo log writes

lg进程数量
这里可以看出来，有一个lgwr进程，两个lg进程和_max_outstanding_log_writes参数配置匹配

[oracle@ora1221 ~]$ ps -ef|grep ora_lg
oracle    49790      1  0 10:32 ?        00:00:00 ora_lgwr_orcl12c2
oracle    49794      1  0 10:32 ?        00:00:00 ora_lg00_orcl12c2
oracle    49798      1  0 10:32 ?        00:00:00 ora_lg01_orcl12c2

修改_max_outstanding_log_writes参数
通过修改_max_outstanding_log_writes参数为4，发现lg进程数量也变为了4，证明_max_outstanding_log_writes进程决定lg进程数量

SQL> alter system set "_max_outstanding_log_writes"=4 ;
alter system set "_max_outstanding_log_writes"=4
                 *
ERROR at line 1:
ORA-02095: specified initialization parameter cannot be modified
SQL> alter system set "_max_outstanding_log_writes"=4 scope=spfile;
System altered.
SQL> shutdown immediate
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup
ORACLE instance started.
Total System Global Area 2516582400 bytes
Fixed Size                  8260048 bytes
Variable Size             671090224 bytes
Database Buffers         1828716544 bytes
Redo Buffers                8515584 bytes
Database mounted.
Database opened.
SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.2.0.0.3 - 64bit Production
[oracle@ora1221 ~]$ ps -ef|grep lg
oracle    72339      1  0 13:45 ?        00:00:00 ora_lgwr_orcl12c2
oracle    72343      1  0 13:45 ?        00:00:00 ora_lg00_orcl12c2
oracle    72347      1  0 13:45 ?        00:00:00 ora_lg01_orcl12c2
oracle    72351      1  0 13:45 ?        00:00:00 ora_lg02_orcl12c2
oracle    72359      1  0 13:45 ?        00:00:00 ora_lg03_orcl12c2

修改_use_single_log_writer参数
通过测试_use_single_log_writer参数修改，我们可以确定_use_single_log_writer修改为true后，数据库恢复到12c之前的lgwr管理方式

[oracle@ora1221 ~]$ ss
SQL*Plus: Release 12.2.0.0.3 Production on Thu Aug 6 13:45:33 2015
Copyright (c) 1982, 2016, Oracle.  All rights reserved.
Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.0.3 - 64bit Production
SQL> alter system set "_use_single_log_writer"=1 ;
alter system set "_use_single_log_writer"=1
                 *
ERROR at line 1:
ORA-02095: specified initialization parameter cannot be modified
SQL>  alter system set "_use_single_log_writer"=1 scope=spfile;
 alter system set "_use_single_log_writer"=1 scope=spfile
*
ERROR at line 1:
ORA-00096: invalid value 1 for parameter _use_single_log_writer, must be from
among ADAPTIVE, FALSE, TRUE
SQL> alter system set "_use_single_log_writer"=TRUE scope=spfile;
System altered.
SQL> shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.2.0.0.3 - 64bit Production
[oracle@ora1221 ~]$ ps -ef|grep lg
oracle    72702  71510  0 13:46 pts/0    00:00:00 grep lg
[oracle@ora1221 ~]$ ss
SQL*Plus: Release 12.2.0.0.3 Production on Thu Aug 6 13:46:50 2015
Copyright (c) 1982, 2016, Oracle.  All rights reserved.
Connected to an idle instance.
SQL> startup
ORACLE instance started.
Total System Global Area 2516582400 bytes
Fixed Size                  8260048 bytes
Variable Size             671090224 bytes
Database Buffers         1828716544 bytes
Redo Buffers                8515584 bytes
Database mounted.
Database opened.
SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.2.0.0.3 - 64bit Production
[oracle@ora1221 ~]$ ps -ef|grep lg
oracle    72754      1  0 13:46 ?        00:00:00 ora_lgwr_orcl12c2
oracle    73008  71510  0 13:47 pts/0    00:00:00 grep lg

从这里可以确定_use_single_log_writer确定是否启用SCALABLE LGWR(多个lg子进程),_max_outstanding_log_writes确定lg进程个数

ORA-01555 ORA-600 kdiulk:kcbz_objdchk ORA-600 kdBlkCheckError等错误恢复

数据库启动ORA-00704,0RA-00604,ORA-01555导致数据库无法启动

Tue May 31 17:32:42 2016
SMON: enabling cache recovery
SUCCESS: diskgroup RECOVERY was mounted
ARC3: Archival started
ARC0: STARTING ARCH PROCESSES COMPLETE
ORA-01555 caused by SQL statement below (SQL ID: 4krwuz0ctqxdt, SCN: 0x0004.3af84bee):
select ctime, mtime, stime from obj$ where obj# = :1
Archived Log entry 5 added for thread 1 sequence 10 ID 0x86a261e7 dest 1:
Errors in file /opt/app/oracle/diag/rdbms/xifenfei/xifenfei/trace/xifenfei_ora_12779.trc:
ORA-00704: bootstrap process failure
ORA-00704: bootstrap process failure
ORA-00604: error occurred at recursive SQL level 1
ORA-01555: snapshot too old: rollback segment number 7 with name "_SYSSMU7_1592079335$" too small
Errors in file /opt/app/oracle/diag/rdbms/xifenfei/xifenfei/trace/xifenfei_ora_12779.trc:
ORA-00704: bootstrap process failure
ORA-00704: bootstrap process failure
ORA-00604: error occurred at recursive SQL level 1
ORA-01555: snapshot too old: rollback segment number 7 with name "_SYSSMU7_1592079335$" too small
Error 704 happened during db open, shutting down database
USER (ospid: 12779): terminating the instance due to error 704

通过bbed修改事务之后启动数据库

Tue May 31 17:35:49 2016
SMON: enabling tx recovery
*********************************************************************
WARNING: The following temporary tablespaces contain no files.
         This condition can occur when a backup controlfile has
         been restored.  It may be necessary to add files to these
         tablespaces.  That can be done using the SQL statement:
         ALTER TABLESPACE <tablespace_name> ADD TEMPFILE
         Alternatively, if these temporary tablespaces are no longer
         needed, then they can be dropped.
           Empty temporary tablespace: TEMP
*********************************************************************
Updating character set in controlfile to AL32UTF8
Tue May 31 17:35:50 2016
Errors in file /opt/app/oracle/diag/rdbms/xifenfei/xifenfei/trace/xifenfei_p021_13862.trc  (incident=166002):
ORA-00600: 内部错误代码, 参数: [kdiulk:kcbz_objdchk], [0], [0], [1], [], [], [], [], [], [], [], []
Tue May 31 17:35:50 2016
Errors in file /opt/app/oracle/diag/rdbms/xifenfei/xifenfei/trace/xifenfei_p010_13818.trc  (incident=165914):
ORA-00600: 内部错误代码, 参数: [kdiulk:kcbz_objdchk], [0], [0], [1], [], [], [], [], [], [], [], []
Tue May 31 17:35:50 2016
Errors in file /opt/app/oracle/diag/rdbms/xifenfei/xifenfei/trace/xifenfei_p004_13794.trc  (incident=165866):
ORA-00600: 内部错误代码, 参数: [kdiulk:kcbz_objdchk], [0], [0], [1], [], [], [], [], [], [], [], []
Tue May 31 17:35:50 2016
Errors in file /opt/app/oracle/diag/rdbms/xifenfei/xifenfei/trace/xifenfei_p011_13822.trc  (incident=165922):
ORA-00600: 内部错误代码, 参数: [kdiulk:kcbz_objdchk], [0], [0], [1], [], [], [], [], [], [], [], []
Tue May 31 17:35:50 2016
Errors in file /opt/app/oracle/diag/rdbms/xifenfei/xifenfei/trace/xifenfei_p016_13842.trc  (incident=165962):
ORA-00600: 内部错误代码, 参数: [kdiulk:kcbz_objdchk], [0], [0], [1], [], [], [], [], [], [], [], []

ORA-600 [kdiulk:kcbz_objdchk] trace文件

*** SESSION ID:(3.5) 2016-05-31 17:35:50.068
OBJD MISMATCH typ=6, seg.obj=-2, diskobj=222225, dsflg=0, dsobj=285890, tid=285890, cls=1
ORA-00600: 内部错误代码, 参数: [kdiulk:kcbz_objdchk], [0], [0], [1], [], [], [], [], [], [], [], []
Parallel Transaction recovery server caught exception 600
begin Parallel Recovery Context Dump
nsi: 48, nsactive: 48
, nirsi: 1, nidti: 1, ndt: 1, rescan: 0, ptrs: 48
[ktprsi] wdone: 50
[ktpritp 378651b8] ktprsi:
37903b60 37903b78 37903b90 37903ba8 37903bc0 37903bd8 37903bf0 37903c08 37903c20 37903c38 37903c50
37903c68 37903c80 37903c98 37903cb0 37903cc8 37903ce0 37903cf8 37903d10 37903d28 37903d40 37903d58
37903d70 37903d88 37903da0 37903db8 37903dd0 37903de8 37903e00 37903e18 37903e30 37903e48 37903e60
37903e78 37903e90 37903ea8 37903ec0 37903ed8 37903ef0 37903f08 37903f20 37903f38 37903f50 37903f68
37903f80 37903f98 37903fb0 37903fc8
[ktprht] nhb: 47, nfl: 20247, flg: 2
*** 2016-05-31 17:36:08.584
[ktprhb] nfl: 1, nelem: 97, flg: 0, sqn: 1
flist: 37698940
nhe: [ktprhe 32] sqn: -1297235803
[kturur] uoff: -1797708320, sqn: 4
uba: 0x098004cd.07e4.0b
*-----------------------------
* Rec #0xb  slt: 0x07  objn: 123986(0x0001e452)  objd: 285891  tblspc: 10(0x0000000a)
*       Layer:  10 (Index)   opc: 22   rci 0x0a
Undo type:  Regular undo   Last buffer split:  No
Temp Object:  No
Tablespace Undo:  No
rdba: 0x00000000

这里基本上可以确定是由于undo index中的dataobj#和block中的dataobj# 不匹配.在数据库undo回滚之时出现该错误.可以通过跳过undo回滚,然后重建对象

Tue May 31 17:36:06 2016
Simulated error on redo application.
Block recovery from logseq 12, block 959 to scn 20401094719
Recovery of Online Redo Log: Thread 1 Group 3 Seq 12 Reading mem 0
  Mem# 0: +DATA/xifenfei/onlinelog/group_3.263.802446627
Block recovery completed at rba 12.1012.16, scn 4.3221225536
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Simulated error for redo application done.
Errors in file /opt/app/oracle/diag/rdbms/xifenfei/xifenfei/trace/xifenfei_p009_13814.trc  (incident=165906):
ORA-00600: 内部错误代码, 参数: [kdBlkCheckError], [26], [950417], [18025], [], [], [], [], [], [], [], []

这些错误是由于数据库block逻辑异常导致,错过参数含义
在10g中ORA-600 kddummy_blkchk 在11g中ORA-600 kdBlkCheckError

ARGUMENTS:
Arg [a] Absolute file number
Arg [b] Bock number
Arg  Internal error code returned from kcbchk() which indicates the problem encountered.
See Note 46389.1 for details of block check codes.

根据QREF kddummy_blkchk / kdBlkCheckError Check Codes Listing (Full) (Doc ID 1264040.1)分析
这里的18025是代码的KCBTEMAP_EC_START + KTS4_EC_SBFREE部分异常，主要表现在Incorrect firstfree or nfree 可以通过设置一些参数进行屏蔽

在恢复过程中还有其他错误

ORA-600 encountered when generating server alert SMG-4128
ORA-00600: internal error code, arguments: [ktcpoptx:!cmt top lvl], [], [], [], [], [], [], [], [], [], [], []
ORA-00600: internal error code, arguments: [4406], [0x1026B65348], [0x000000000], [2], [6215], [], [], [], [], [], [], []
ORA-00600: internal error code, arguments: [ktcpoptx:!cmt top lvl], [], [], [], [], [], [], [], [], [], [], []
ORA-00600: internal error code, arguments: [4194], [], [], [], [], [], [], [], [], [], [], []
ORACLE Instance xifenfei (pid = 15) - Error 600 encountered while recovering transaction (10, 7) on object 123986.
ORA-00600: internal error code, arguments: [], [], [], [], [], [], [], [], [], [], [], []
ORA-00600: internal error code, arguments: [kdsgrp1], [], [], [], [], [], [], [], [], [], [], []
ORA-00600: internal error code, arguments: [kturbleurec1], [], [], [], [], [], [], [], [], [], [], []
ORA-00600: internal error code, arguments: [kewrose_1], [600],
  [ORA-00600: internal error code, arguments: [4194], [], [], [], [], [], [], [], [], [], [], []
Non-fatal internal error happenned while SMON was doing logging scn->time mapping.

通过整体分析错误主要是由于undo异常导致,通过设置_corrupted_rollback_segments设置db_block_checking等相关参数,清理SMON_SCN_TIME等操作数据库没有其他异常报错,让其通过逻辑方式重建库

12.1中出现大量Result Cache: RC Latch处理

昨天有个朋友找到我说他们的12.1的库在业务高峰期非常慢，希望我们给予优化支持,经过awr分析，定位到问题为latch free问题,具体定位为:Result Cache: RC Latch.
优化之前awr部分信息
awr整体负载情况，证明当前这个库已经比较忙，业务反馈很慢
awr1

addr信息和top wait信息，确定是latch free问题比较突出
awr2
awr3

latch信息统计和ash信息，找出来突出的latch,定位为Result Cache: RC Latch引起该问题
awr4
awr5

补充大量异常sql
awr6

类似sql语句

SELECT /* DS_SVC */ /*+ dynamic_sampling(0) no_sql_tune no_monitoring optimizer_features_enable(default) no_parallel */
 SUM(C1) FROM (SELECT /*+ qb_name("innerQuery") INDEX_FFS( "ACCOUNT" "ACC_USER_ID") */
1 AS C1 FROM "ACCOUNT" SAMPLE BLOCK(39.3701, 8) SEED(1) "ACCOUNT" WHERE ( "ACCOUNT".USER_ID IS NOT NULL)) innerQuery

查询mos发现
The cause of this issue is automatic dynamic statistics which is enabled by default in 12c automatically decides whether dynamic statistics are useful and which statistics level to use for all SQL statements. It collects dynamic statistics when the optimizer deems it necessary.
When Automatic Dynamic Sampling is used for the SQL statements, it can decide, based upon these statistics, that a better response time could be achieved by using the result cache for those queries. This can cause heavy usage of the result cache leading to the contention on latch free for “Result Cache: RC Latch”.
也就是说,12c在自动采样有改进，而且默认使用result cache特性，从而引起该问题，即使你设置了 RESULT_CACHE_MODE = MANUAL，依旧会有大量动态采样引起 Result Cache: RC Latch，彻底解决给问题就是通过隐含参数禁止Automatic Dynamic Statistics使用result cache

alter system set "_optimizer_ads_use_result_cache" = FALSE;

设置该参数之后效果
这里看，通过上述处理后，系统db time 大量减少，业务反馈已经运行正常
hawr1

latch free和Result Cache: RC Latch已经基本上消失
hawr2
hawr3
hawr4

当然这个异常是由于动态采样导致,可以通过收集数据库统计信息，设置动态采样级别，也可以从一定程度上缓解该情况.
参考mos
Very Long Parse Time for Queries in InMemory Database (Doc ID 2102106.1)
High “Latch Free” Waits with Contention on ‘Result Cache: RC Latch’ when RESULT_CACHE_MODE = MANUAL on Oracle 12c (Doc ID 2002089.1)

csc higher than block scn类型坏块修复

数据库虽然正常open了，但是由于system有坏块，导致数据库导出有部分表报错，客户希望通过修复坏块完美解决该问题
exp-ORA-1578

bbed检查system报坏块

C:\Users\FAL>dbv file=D:\BAIDUYUNDOWNLOAD\ORADATA\CHEASDB\SYSTEM01.DBF
DBVERIFY: Release 11.2.0.1.0 - Production on 星期六 5月 14 15:40:55 2016
Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.
DBVERIFY - 开始验证: FILE = D:\BAIDUYUNDOWNLOAD\ORADATA\CHEASDB\SYSTEM01.DBF
csc(0x0000.1f8adab3) higher than block scn(0x0000.00000000)
页 122146 失败, 校验代码为 6054
DBVERIFY - 验证完成
检查的页总数: 252160
处理的页总数 (数据): 178921
失败的页总数 (数据): 0
处理的页总数 (索引): 52576
失败的页总数 (索引): 1
处理的页总数 (其他): 3201
处理的总页数 (段)  : 1
失败的总页数 (段)  : 0
空的页总数: 17462
标记为损坏的总页数: 0
流入的页总数: 0
加密的总页数        : 0
最高块 SCN            : 529420540 (0.529420540)

bbed修复坏块

C:\Users\FAL>bbed password=blockedit filename=D:\BAIDUYUNDOWNLOAD\ORADATA\CHEASDB\SYSTEM01.DBF
BBED: Release 2.0.0.0.0 - Limited Production on Sat May 14 15:37:01 2016
Copyright (c) 1982, 2002, Oracle Corporation.  All rights reserved.
************* !!! For Oracle Internal Use only !!! ***************
BBED> set blocksize 8192
        BLOCKSIZE       8192
BBED> set block 2
        BLOCK#          2
BBED> map
 File: D:\BAIDUYUNDOWNLOAD\ORADATA\CHEASDB\SYSTEM01.DBF (0)
 Block: 2                                     Dba:0x00000000
------------------------------------------------------------
 Data File Header
 struct kcvfh, 360 bytes                    @0
 ub4 tailchk                                @8188
BBED> set block 122147
        BLOCK#          122147
BBED> map
 File: D:\BAIDUYUNDOWNLOAD\ORADATA\CHEASDB\SYSTEM01.DBF (0)
 Block: 122147                                Dba:0x00000000
------------------------------------------------------------
 KTB Data Block (Index Leaf)
 struct kcbh, 20 bytes                      @0
 struct ktbbh, 3720 bytes                   @20
 struct kdxle, 32 bytes                     @3740
 b2 kd_off[188]                             @3772
 ub1 freespace[1534]                        @4148
 ub1 rowdata[2442]                          @5682
 ub4 tailchk                                @8188
BBED> p kcbh
struct kcbh, 20 bytes                       @0
   ub1 type_kcbh                            @0        0x06
   ub1 frmt_kcbh                            @1        0xa2
   ub1 spare1_kcbh                          @2        0x00
   ub1 spare2_kcbh                          @3        0x00
   ub4 rdba_kcbh                            @4        0x0041dd22
   ub4 bas_kcbh                             @8        0x00000000
   ub2 wrp_kcbh                             @12       0x0000
   ub1 seq_kcbh                             @14       0xff
   ub1 flg_kcbh                             @15       0x04 (KCBHFCKV)
   ub2 chkval_kcbh                          @16       0x6cee
   ub2 spare3_kcbh                          @18       0x0000
BBED> verify
DBVERIFY - Verification starting
FILE = D:\BAIDUYUNDOWNLOAD\ORADATA\CHEASDB\SYSTEM01.DBF
BLOCK = 122146
Block Checking: DBA = 4316450, Block Type = KTB-managed data block
Found block already marked corrupted
DBVERIFY - Verification complete
Total Blocks Examined         : 1
Total Blocks Processed (Data) : 0
Total Blocks Failing   (Data) : 0
Total Blocks Processed (Index): 1
Total Blocks Failing   (Index): 0
Total Blocks Empty            : 0
Total Blocks Marked Corrupt   : 0
Total Blocks Influx           : 0
BBED> set offset 8188
        OFFSET          8188
BBED> map
 File: D:\BAIDUYUNDOWNLOAD\ORADATA\CHEASDB\SYSTEM01.DBF (0)
 Block: 122147                                Dba:0x00000000
------------------------------------------------------------
 KTB Data Block (Index Leaf)
 struct kcbh, 20 bytes                      @0
 struct ktbbh, 3720 bytes                   @20
 struct kdxle, 32 bytes                     @3740
 b2 kd_off[188]                             @3772
 ub1 freespace[1534]                        @4148
 ub1 rowdata[2442]                          @5682
 ub4 tailchk                                @8188
BBED> d
 File: D:\BAIDUYUNDOWNLOAD\ORADATA\CHEASDB\SYSTEM01.DBF (0)
 Block: 122147           Offsets: 8188 to 8191           Dba:0x00000000
------------------------------------------------------------------------
 ff060000
 <32 bytes per line>
BBED> set mode edit
        MODE            Edit
BBED> m /x 01 offset 14
 File: D:\BAIDUYUNDOWNLOAD\ORADATA\CHEASDB\SYSTEM01.DBF (0)
 Block: 122147           Offsets:   14 to  525           Dba:0x00000000
------------------------------------------------------------------------
 0104ee6c 00000200 00003500 0000b3da 8a1f0000 00009a00 030021dd 41000900
 0700443d 0600531c c000628b 010000c0 00002c3f 5a1f0300 0000ca5e 05002b0a
 c0008685 01000080 000073d7 8a1f0400 200057a2 0500d530 c000a88e 2c000080
 0000b3d7 8a1f0a00 01002848 08001101 c00084be 29000080 0000e3d7 8a1f0800
 0d00d612 06008f0a c000258d 3c000080 000026d8 8a1f0100 1f0063c1 04006007
 c0002887 4b000080 000046d8 8a1f0700 1e005403 0600a707 c000bd8e 40000080
 00006bd8 8a1f0a00 0b000c48 08001301 c00084be 41000080 0000a2d8 8a1f0600
 1b0055e6 0500c508 c0000f8b 23000080 0000fad8 8a1f0300 0600ca5e 0500220a
 c0008685 34000080 000040d9 8a1f0800 1300d412 0600900a c000258d 2e000080
 000064d9 8a1f0600 0b0092e6 0500c608 c0000f8b 1c000080 000089d9 8a1f0a00
 05001d48 08001401 c00084be 05000080 0000aed9 8a1f0400 100030a2 0500d730
 c000a88e 0e000080 000016da 8a1f0800 1700c712 0600910a c000258d 33000080
 000022da 8a1f0200 1100450d 0500930c c000c085 28000080 00002fda 8a1f0200
 0800610d 0500940c c000c085 20000080 00004eda 8a1f0700 12003303 0600aa07
 c000bd8e 0a000120 0f00b7da 8a1f0a00 13000048 08001501 c00084be 07000220
 1e00f5da 8a1f0900 07000b44 0600650f c000968d 05000120 0f0004db 8a1f0300
 <32 bytes per line>
BBED> sum apply
Check value for File 0, Block 122147:
current = 0x6cee, required = 0x6cee
BBED> verify
DBVERIFY - Verification starting
FILE = D:\BAIDUYUNDOWNLOAD\ORADATA\CHEASDB\SYSTEM01.DBF
BLOCK = 122146
DBVERIFY - Verification complete
Total Blocks Examined         : 1
Total Blocks Processed (Data) : 0
Total Blocks Failing   (Data) : 0
Total Blocks Processed (Index): 1
Total Blocks Failing   (Index): 0
Total Blocks Empty            : 0
Total Blocks Marked Corrupt   : 0
Total Blocks Influx           : 0
BBED> m /x c1da8a1f offset 8
 File: D:\BAIDUYUNDOWNLOAD\ORADATA\CHEASDB\SYSTEM01.DBF (0)
 Block: 122147           Offsets:    8 to  519           Dba:0x00000000
------------------------------------------------------------------------
 c1da8a1f 00000104 ee6c0000 02000000 35000000 b3da8a1f 00000000 9a000300
 21dd4100 09000700 443d0600 531cc000 628b0100 00c00000 2c3f5a1f 03000000
 ca5e0500 2b0ac000 86850100 00800000 73d78a1f 04002000 57a20500 d530c000
 a88e2c00 00800000 b3d78a1f 0a000100 28480800 1101c000 84be2900 00800000
 e3d78a1f 08000d00 d6120600 8f0ac000 258d3c00 00800000 26d88a1f 01001f00
 63c10400 6007c000 28874b00 00800000 46d88a1f 07001e00 54030600 a707c000
 bd8e4000 00800000 6bd88a1f 0a000b00 0c480800 1301c000 84be4100 00800000
 a2d88a1f 06001b00 55e60500 c508c000 0f8b2300 00800000 fad88a1f 03000600
 ca5e0500 220ac000 86853400 00800000 40d98a1f 08001300 d4120600 900ac000
 258d2e00 00800000 64d98a1f 06000b00 92e60500 c608c000 0f8b1c00 00800000
 89d98a1f 0a000500 1d480800 1401c000 84be0500 00800000 aed98a1f 04001000
 30a20500 d730c000 a88e0e00 00800000 16da8a1f 08001700 c7120600 910ac000
 258d3300 00800000 22da8a1f 02001100 450d0500 930cc000 c0852800 00800000
 2fda8a1f 02000800 610d0500 940cc000 c0852000 00800000 4eda8a1f 07001200
 33030600 aa07c000 bd8e0a00 01200f00 b7da8a1f 0a001300 00480800 1501c000
 84be0700 02201e00 f5da8a1f 09000700 0b440600 650fc000 968d0500 01200f00
 <32 bytes per line>
BBED> p kcbh
struct kcbh, 20 bytes                       @0
   ub1 type_kcbh                            @0        0x06
   ub1 frmt_kcbh                            @1        0xa2
   ub1 spare1_kcbh                          @2        0x00
   ub1 spare2_kcbh                          @3        0x00
   ub4 rdba_kcbh                            @4        0x0041dd22
   ub4 bas_kcbh                             @8        0x1f8adac1
   ub2 wrp_kcbh                             @12       0x0000
   ub1 seq_kcbh                             @14       0x01
   ub1 flg_kcbh                             @15       0x04 (KCBHFCKV)
   ub2 chkval_kcbh                          @16       0x6cee
   ub2 spare3_kcbh                          @18       0x0000
BBED> sum apply
Check value for File 0, Block 122147:
current = 0x7364, required = 0x7364
BBED> verify
DBVERIFY - Verification starting
FILE = D:\BAIDUYUNDOWNLOAD\ORADATA\CHEASDB\SYSTEM01.DBF
BLOCK = 122146
DBVERIFY - Verification complete
Total Blocks Examined         : 1
Total Blocks Processed (Data) : 0
Total Blocks Failing   (Data) : 0
Total Blocks Processed (Index): 1
Total Blocks Failing   (Index): 0
Total Blocks Empty            : 0
Total Blocks Marked Corrupt   : 0
Total Blocks Influx           : 0

dbv再次验证，坏块完美修复

C:\Users\FAL>dbv file=D:\BAIDUYUNDOWNLOAD\ORADATA\CHEASDB\SYSTEM01.DBF
DBVERIFY: Release 11.2.0.1.0 - Production on 星期六 5月 14 22:00:06 2016
Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.
DBVERIFY - 开始验证: FILE = D:\BAIDUYUNDOWNLOAD\ORADATA\CHEASDB\SYSTEM01.DBF
DBVERIFY - 验证完成
检查的页总数: 259840
处理的页总数 (数据): 181934
失败的页总数 (数据): 0
处理的页总数 (索引): 57381
失败的页总数 (索引): 0
处理的页总数 (其他): 3218
处理的总页数 (段)  : 1
失败的总页数 (段)  : 0
空的页总数: 17307
标记为损坏的总页数: 0
流入的页总数: 0
加密的总页数        : 0
最高块 SCN            : 530218225 (0.530218225)

通过修复之后重新导出数据
exp-ok

参考相似篇章:file 1 block 128 corrupted/坏块恢复—system rollback坏块修复