非常规恢复 – 第4页

oracle asm中drop pdb恢复方法

最近分析研究了一种asm disk 数据文件丢失的新恢复方法,通过cod和acd进行恢复,我们对于asm 磁盘组中的文件的改变(创建,删除,扩大,缩小等)操作会体现在cod和acd中有一些体现,类似oracle 数据库中数据的变化都会体现在redo和undo中类似.可以通过对他们的分析,确认文件在asm磁盘组中的分配关系,从而实现数据文件的恢复.我这里通过模拟创建表空间,插入数据,删除表空间(同时也删除文件),然后相关cod和acd分析,实现数据文件恢复.
创建表空间

SQL> create tablespace xifenfei datafile '+data' size 1G;

Tablespace created.

SQL> alter tablespace xifenfei add datafile '+data' size 128M autoextend on;

Tablespace altered.

创建模拟表并插入数据

SQL> create table t_xifenfei tablespace xifenfei as
  2  select * from dba_objects;

Table created.

SQL> insert into t_xifenfei select * from t_xifenfei;

73013 rows created.

…………

SQL> insert into t_xifenfei select * from t_xifenfei;

18691328 rows created.

SQL> COMMIT;

Commit complete.

SQL> SELECT COUNT(1) FROM T_XIFENFEI;

  COUNT(1)
----------
  37382656

SQL> alter system checkpoint;

System altered.

SQL> select bytes/1024/1024/1024,TABLESPACE_NAME FROM USER_SEGMENTS  where segment_name='T_XIFENFEI';

BYTES/1024/1024/1024 TABLESPACE_NAME
-------------------- ------------------------------
          5.56738281 XIFENFEI

删除表空间

SQL> drop tablespace xifenfei including contents and datafiles;

Tablespace dropped.

查看alert日志信息

2020-04-23T18:23:43.088997-04:00
drop tablespace xifenfei including contents and datafiles
2020-04-23T18:23:46.226654-04:00
Deleted Oracle managed file +DATA/ORA18C/DATAFILE/xifenfei.262.1035571131
Deleted Oracle managed file +DATA/ORA18C/DATAFILE/xifenfei.263.1038507123
Completed: drop tablespace xifenfei including contents and datafiles

这里我们可以看到被删除的两个数据文件的asm number为262和263,如果要恢复该表空间数据需要先恢复出来该数据文件.由于文件被删除,文件对应存储在asm里面的找出来相关的数据分配关系才可以对其恢复出来.尝试找该文件的分配extent映射关系
尝试直接读取extent map

[root@rac18c2 ~]#  kfed read /dev/xifenfei-sdb  aus=4194304 blkn=0|grep f1b1locn
kfdhdb.f1b1locn:                     10 ; 0x0d4: 0x0000000a
[root@rac18c2 ~]#  kfed read /dev/xifenfei-sdb  aus=4194304 aun=10 blkn=262|more
kfbh.endian:                          1 ; 0x000: 0x01
kfbh.hard:                          130 ; 0x001: 0x82
kfbh.type:                            4 ; 0x002: KFBTYP_FILEDIR
kfbh.datfmt:                          1 ; 0x003: 0x01
kfbh.block.blk:                     262 ; 0x004: blk=262
kfbh.block.obj:                       1 ; 0x008: file=1
kfbh.check:                  4132734069 ; 0x00c: 0xf6548475
kfbh.fcn.base:                     6741 ; 0x010: 0x00001a55
kfbh.fcn.wrap:                        0 ; 0x014: 0x00000000
kfbh.spare1:                          0 ; 0x018: 0x00000000
kfbh.spare2:                          0 ; 0x01c: 0x00000000
kfffdb.node.incarn:          1035571132 ; 0x000: A=0 NUMM=0x1edcc7de
kfffdb.node.frlist.number:          264 ; 0x004: 0x00000108
kfffdb.node.frlist.incarn:            0 ; 0x008: A=0 NUMM=0x0
kfffdb.hibytes:                       0 ; 0x00c: 0x00000000
kfffdb.lobytes:              1073750016 ; 0x010: 0x40002000
kfffdb.xtntcnt:                       0 ; 0x014: 0x00000000
kfffdb.xtnteof:                     257 ; 0x018: 0x00000101
kfffdb.blkSize:                    8192 ; 0x01c: 0x00002000
kfffdb.flags:                         1 ; 0x020: O=1 S=0 S=0 D=0 C=0 I=0 R=0 A=0
kfffdb.fileType:                      2 ; 0x021: 0x02
…………
kfffdb.mxshad:                        0 ; 0x498: 0x0000
kfffdb.mxprnt:                        0 ; 0x49a: 0x0000
kfffdb.fmtBlks:                  131073 ; 0x49c: 0x00020001
kfffde[0].xptr.au:           4294967295 ; 0x4a0: 0xffffffff
kfffde[0].xptr.disk:              65535 ; 0x4a4: 0xffff
kfffde[0].xptr.flags:                 0 ; 0x4a6: L=0 E=0 D=0 S=0 R=0 I=0
kfffde[0].xptr.chk:                  42 ; 0x4a7: 0x2a
kfffde[1].xptr.au:           4294967295 ; 0x4a8: 0xffffffff
kfffde[1].xptr.disk:              65535 ; 0x4ac: 0xffff
kfffde[1].xptr.flags:                 0 ; 0x4ae: L=0 E=0 D=0 S=0 R=0 I=0
kfffde[1].xptr.chk:                  42 ; 0x4af: 0x2a

这里的kfffdb.blkSize为8192证明以前很可能是数据文件,但是kfffde中的au和disk全部被置为f,说明extent的直接映射表已经被置空,更不用说间接extent分配映射表,也就是说这条路无法走通.变换一种思路,既然文件从asm中删除掉的extent映射关系被清空,那是否可以通过对应的acd记录来找到相关数据.通过对acd进行分析,发现在删除drop的时间点相关类似记录,通过分析对应的acd记录,发现直接extent和扩展extent分配全部被置空,无法通过该思路进行恢复
尝试acd恢复extent map

kfracdb2.lge[2].bcd[2].kfbl.blk:    262 ; 0x1cc: blk=262
kfracdb2.lge[2].bcd[2].kfbl.obj:      1 ; 0x1d0: file=1
kfracdb2.lge[2].bcd[2].kfcn.base:  6216 ; 0x1d4: 0x00001848
kfracdb2.lge[2].bcd[2].kfcn.wrap:     0 ; 0x1d8: 0x00000000
kfracdb2.lge[2].bcd[2].oplen:        20 ; 0x1dc: 0x0014
kfracdb2.lge[2].bcd[2].blkIndex:    262 ; 0x1de: 0x0106
kfracdb2.lge[2].bcd[2].flags:        28 ; 0x1e0: F=0 N=0 F=1 L=1 V=1 A=0 C=0 R=0
kfracdb2.lge[2].bcd[2].opcode:      162 ; 0x1e2: 0x00a2
kfracdb2.lge[2].bcd[2].kfbtyp:        4 ; 0x1e4: KFBTYP_FILEDIR
kfracdb2.lge[2].bcd[2].redund:       17 ; 0x1e5: SCHE=0x1 NUMB=0x1
kfracdb2.lge[2].bcd[2].pad:       63903 ; 0x1e6: 0xf99f
kfracdb2.lge[2].bcd[2].KFFFD_PEXT.xtntcnt:0 ; 0x1e8: 0x00000000
kfracdb2.lge[2].bcd[2].KFFFD_PEXT.xtntblk:0 ; 0x1ec: 0x0000
kfracdb2.lge[2].bcd[2].KFFFD_PEXT.xnum:0 ; 0x1ee: 0x0000
kfracdb2.lge[2].bcd[2].KFFFD_PEXT.xcnt:1 ; 0x1f0: 0x0001
kfracdb2.lge[2].bcd[2].KFFFD_PEXT.setflg:0 ; 0x1f2: 0x00
kfracdb2.lge[2].bcd[2].KFFFD_PEXT.flags:0 ; 0x1f3: O=0 S=0 S=0 D=0 C=0 I=0 R=0 A=0
kfracdb2.lge[2].bcd[2].KFFFD_PEXT.xptr[0].au:4294967292 ; 0x1f4: 0xfffffffc
kfracdb2.lge[2].bcd[2].KFFFD_PEXT.xptr[0].disk:0 ; 0x1f8: 0x0000
kfracdb2.lge[2].bcd[2].KFFFD_PEXT.xptr[0].flags:0 ; 0x1fa: L=0 E=0 D=0 S=0 R=0 I=0
kfracdb2.lge[2].bcd[2].KFFFD_PEXT.xptr[0].chk:41 ; 0x1fb: 0x29
kfracdb2.lge[2].bcd[2].au[0]:        10 ; 0x1fc: 0x0000000a
kfracdb2.lge[2].bcd[2].disks[0]:      0 ; 0x200: 0x0000


kfracdb2.lge[20].bcd[1].kfbl.blk:2147483648 ; 0xe54: blk=0 (indirect)
kfracdb2.lge[20].bcd[1].kfbl.obj:   262 ; 0xe58: file=262
kfracdb2.lge[20].bcd[1].kfcn.base: 3280 ; 0xe5c: 0x00000cd0
kfracdb2.lge[20].bcd[1].kfcn.wrap:    0 ; 0xe60: 0x00000000
kfracdb2.lge[20].bcd[1].oplen:       16 ; 0xe64: 0x0010
kfracdb2.lge[20].bcd[1].blkIndex:     0 ; 0xe66: 0x0000
kfracdb2.lge[20].bcd[1].flags:       28 ; 0xe68: F=0 N=0 F=1 L=1 V=1 A=0 C=0 R=0
kfracdb2.lge[20].bcd[1].opcode:     163 ; 0xe6a: 0x00a3
kfracdb2.lge[20].bcd[1].kfbtyp:      12 ; 0xe6c: KFBTYP_INDIRECT
kfracdb2.lge[20].bcd[1].redund:      17 ; 0xe6d: SCHE=0x1 NUMB=0x1
kfracdb2.lge[20].bcd[1].pad:      63903 ; 0xe6e: 0xf99f
kfracdb2.lge[20].bcd[1].KFFIX_PEXT.xtntblk:0 ; 0xe70: 0x0000
kfracdb2.lge[20].bcd[1].KFFIX_PEXT.xnum:0 ; 0xe72: 0x0000
kfracdb2.lge[20].bcd[1].KFFIX_PEXT.xcnt:1 ; 0xe74: 0x0001
kfracdb2.lge[20].bcd[1].KFFIX_PEXT.ub2spare:0 ; 0xe76: 0x0000
kfracdb2.lge[20].bcd[1].KFFIX_PEXT.xptr[0].au:4294967292 ; 0xe78: 0xfffffffc
kfracdb2.lge[20].bcd[1].KFFIX_PEXT.xptr[0].disk:0 ; 0xe7c: 0x0000
kfracdb2.lge[20].bcd[1].KFFIX_PEXT.xptr[0].flags:0 ; 0xe7e: L=0 E=0 D=0 S=0 R=0 I=0
kfracdb2.lge[20].bcd[1].KFFIX_PEXT.xptr[0].chk:41 ; 0xe7f: 0x29
kfracdb2.lge[20].bcd[1].au[0]:      296 ; 0xe80: 0x00000128
kfracdb2.lge[20].bcd[1].disks[0]:     0 ; 0xe84: 0x0000

直接通过删除的acd记录来找出来数据文件分配的extent也行不通,通过分析相关acd block,终于找到了对应的extent分配的相关记录

kfracdb2.lge[21].bcd[0].kfbl.blk:     2 ; 0x918: blk=2
kfracdb2.lge[21].bcd[0].kfbl.obj:2147483648 ; 0x91c: disk=0
kfracdb2.lge[21].bcd[0].kfcn.base: 2820 ; 0x920: 0x00000b04
kfracdb2.lge[21].bcd[0].kfcn.wrap:    0 ; 0x924: 0x00000000
kfracdb2.lge[21].bcd[0].oplen:       28 ; 0x928: 0x001c
kfracdb2.lge[21].bcd[0].blkIndex:     2 ; 0x92a: 0x0002
kfracdb2.lge[21].bcd[0].flags:       28 ; 0x92c: F=0 N=0 F=1 L=1 V=1 A=0 C=0 R=0
kfracdb2.lge[21].bcd[0].opcode:      73 ; 0x92e: 0x0049
kfracdb2.lge[21].bcd[0].kfbtyp:       3 ; 0x930: KFBTYP_ALLOCTBL
kfracdb2.lge[21].bcd[0].redund:      18 ; 0x931: SCHE=0x1 NUMB=0x2
kfracdb2.lge[21].bcd[0].pad:      63903 ; 0x932: 0xf99f
kfracdb2.lge[21].bcd[0].KFDAT_ALLOC2.vaa.curidx:2416 ; 0x934: 0x0970
kfracdb2.lge[21].bcd[0].KFDAT_ALLOC2.vaa.nxtidx:8 ; 0x936: 0x0008
kfracdb2.lge[21].bcd[0].KFDAT_ALLOC2.vaa.prvidx:8 ; 0x938: 0x0008
kfracdb2.lge[21].bcd[0].KFDAT_ALLOC2.vaa.asz:0 ; 0x93a: KFDASZ_1X
kfracdb2.lge[21].bcd[0].KFDAT_ALLOC2.vaa.frag:0 ; 0x93b: 0x00
kfracdb2.lge[21].bcd[0].KFDAT_ALLOC2.vaa.total:0 ; 0x93c: 0x0000
kfracdb2.lge[21].bcd[0].KFDAT_ALLOC2.vaa.free:0 ; 0x93e: 0x0000
kfracdb2.lge[21].bcd[0].KFDAT_ALLOC2.vaa.fnum:262 ; 0x940: 0x00000106
kfracdb2.lge[21].bcd[0].KFDAT_ALLOC2.vaa.xnum:0 ; 0x944: 0x00000000
kfracdb2.lge[21].bcd[0].KFDAT_ALLOC2.vaa.flags:8388608 ; 0x948: 0x00800000
kfracdb2.lge[21].bcd[0].KFDAT_ALLOC2.lxnum:3 ; 0x94c: 0x03
kfracdb2.lge[21].bcd[0].KFDAT_ALLOC2.spare1:0 ; 0x94d: 0x00
kfracdb2.lge[21].bcd[0].KFDAT_ALLOC2.spare2:0 ; 0x94e: 0x0000
kfracdb2.lge[21].bcd[0].au[0]:        0 ; 0x950: 0x00000000
kfracdb2.lge[21].bcd[0].au[1]:       11 ; 0x954: 0x0000000b
kfracdb2.lge[21].bcd[0].disks[0]:     0 ; 0x958: 0x0000
kfracdb2.lge[21].bcd[0].disks[1]:     0 ; 0x95a: 0x0000
kfracdb2.lge[21].bcd[1].kfbl.blk:   262 ; 0x95c: blk=262
kfracdb2.lge[21].bcd[1].kfbl.obj:     1 ; 0x960: file=1
kfracdb2.lge[21].bcd[1].kfcn.base: 3018 ; 0x964: 0x00000bca
kfracdb2.lge[21].bcd[1].kfcn.wrap:    0 ; 0x968: 0x00000000
kfracdb2.lge[21].bcd[1].oplen:       20 ; 0x96c: 0x0014
kfracdb2.lge[21].bcd[1].blkIndex:   262 ; 0x96e: 0x0106
kfracdb2.lge[21].bcd[1].flags:       28 ; 0x970: F=0 N=0 F=1 L=1 V=1 A=0 C=0 R=0
kfracdb2.lge[21].bcd[1].opcode:     162 ; 0x972: 0x00a2
kfracdb2.lge[21].bcd[1].kfbtyp:       4 ; 0x974: KFBTYP_FILEDIR
kfracdb2.lge[21].bcd[1].redund:      17 ; 0x975: SCHE=0x1 NUMB=0x1
kfracdb2.lge[21].bcd[1].pad:      63903 ; 0x976: 0xf99f
kfracdb2.lge[21].bcd[1].KFFFD_PEXT.xtntcnt:0 ; 0x978: 0x00000000
kfracdb2.lge[21].bcd[1].KFFFD_PEXT.xtntblk:0 ; 0x97c: 0x0000
kfracdb2.lge[21].bcd[1].KFFFD_PEXT.xnum:0 ; 0x97e: 0x0000
kfracdb2.lge[21].bcd[1].KFFFD_PEXT.xcnt:1 ; 0x980: 0x0001
kfracdb2.lge[21].bcd[1].KFFFD_PEXT.setflg:0 ; 0x982: 0x00
kfracdb2.lge[21].bcd[1].KFFFD_PEXT.flags:0 ; 0x983: O=0 S=0 S=0 D=0 C=0 I=0 R=0 A=0
kfracdb2.lge[21].bcd[1].KFFFD_PEXT.xptr[0].au:297 ; 0x984: 0x00000129
kfracdb2.lge[21].bcd[1].KFFFD_PEXT.xptr[0].disk:0 ; 0x988: 0x0000
kfracdb2.lge[21].bcd[1].KFFFD_PEXT.xptr[0].flags:0 ; 0x98a: L=0 E=0 D=0 S=0 R=0 I=0
kfracdb2.lge[21].bcd[1].KFFFD_PEXT.xptr[0].chk:2 ; 0x98b: 0x02
kfracdb2.lge[21].bcd[1].au[0]:       10 ; 0x98c: 0x0000000a
kfracdb2.lge[21].bcd[1].disks[0]:     0 ; 0x990: 0x0000
kfracdb2.lge[21].bcd[2].kfbl.blk:     2 ; 0x994: blk=2
kfracdb2.lge[21].bcd[2].kfbl.obj:     4 ; 0x998: file=4
kfracdb2.lge[21].bcd[2].kfcn.base: 3019 ; 0x99c: 0x00000bcb
kfracdb2.lge[21].bcd[2].kfcn.wrap:    0 ; 0x9a0: 0x00000000
kfracdb2.lge[21].bcd[2].oplen:        8 ; 0x9a4: 0x0008
kfracdb2.lge[21].bcd[2].blkIndex:     2 ; 0x9a6: 0x0002
kfracdb2.lge[21].bcd[2].flags:       28 ; 0x9a8: F=0 N=0 F=1 L=1 V=1 A=0 C=0 R=0
kfracdb2.lge[21].bcd[2].opcode:     211 ; 0x9aa: 0x00d3
kfracdb2.lge[21].bcd[2].kfbtyp:      16 ; 0x9ac: KFBTYP_COD_DATA
kfracdb2.lge[21].bcd[2].redund:      17 ; 0x9ad: SCHE=0x1 NUMB=0x1
kfracdb2.lge[21].bcd[2].pad:      63903 ; 0x9ae: 0xf99f
kfracdb2.lge[21].bcd[2].KFRCOD_DATA.offset:60 ; 0x9b0: 0x003c
kfracdb2.lge[21].bcd[2].KFRCOD_DATA.length:4 ; 0x9b2: 0x0004
kfracdb2.lge[21].bcd[2].KFRCOD_DATA.data[0]:1 ; 0x9b4: 0x01
kfracdb2.lge[21].bcd[2].KFRCOD_DATA.data[1]:0 ; 0x9b5: 0x00
kfracdb2.lge[21].bcd[2].KFRCOD_DATA.data[2]:0 ; 0x9b6: 0x00
kfracdb2.lge[21].bcd[2].KFRCOD_DATA.data[3]:0 ; 0x9b7: 0x00
kfracdb2.lge[21].bcd[2].au[0]:       16 ; 0x9b8: 0x00000010
kfracdb2.lge[21].bcd[2].disks[0]:     0 ; 0x9bc: 0x0000

对于类似这样的记录,通过汇总处理获得所有的file number对应的au extent分配记录,并且生成dd语句,然后生成文件

dbv检查恢复文件

[oracle@rac18c2 tmp]$ dbv file=262.dbf

DBVERIFY: Release 18.0.0.0.0 - Production on Tue Apr 28 09:45:37 2020

Copyright (c) 1982, 2018, Oracle and/or its affiliates.  All rights reserved.

DBVERIFY - Verification starting : FILE = /tmp/262.dbf


DBVERIFY - Verification complete

Total Pages Examined         : 131072
Total Pages Processed (Data) : 123400
Total Pages Failing   (Data) : 0
Total Pages Processed (Index): 0
Total Pages Failing   (Index): 0
Total Pages Processed (Other): 631
Total Pages Processed (Seg)  : 0
Total Pages Failing   (Seg)  : 0
Total Pages Empty            : 7041
Total Pages Marked Corrupt   : 0
Total Pages Influx           : 0
Total Pages Encrypted        : 0
Highest block SCN            : 10001146011 (2.1411211419)

[oracle@rac18c2 tmp]$ dbv file=263.dbf

DBVERIFY: Release 18.0.0.0.0 - Production on Tue Apr 28 09:51:05 2020

Copyright (c) 1982, 2018, Oracle and/or its affiliates.  All rights reserved.

DBVERIFY - Verification starting : FILE = /tmp/263.dbf



DBVERIFY - Verification complete

Total Pages Examined         : 643584
Total Pages Processed (Data) : 595146
Total Pages Failing   (Data) : 0
Total Pages Processed (Index): 0
Total Pages Failing   (Index): 0
Total Pages Processed (Other): 821
Total Pages Processed (Seg)  : 0
Total Pages Failing   (Seg)  : 0
Total Pages Empty            : 36865
Total Pages Marked Corrupt   : 0
Total Pages Influx           : 0
Total Pages Encrypted        : 0
Highest block SCN            : 10001153042 (2.1411218450)
[oracle@rac18c2 tmp]$

dul确认恢复文件中数据

DUL>  scan database
start scan database in parallel 1...
scan database completed.
DUL>  sample all segment 
start get segment info: data_obj#: 74635
finish get segment info: data_obj#: 74635
guess col def: 22
write segment info: 74635, 1, 8, 22
write sample rows: 10000
DUL>  unload 74635
 2020-04-24 22:32:11 unloading table segment 74635...
 2020-04-24 22:35:36 unloaded 37382656 rows.
DUL>

通过dbv和实际数据条数对比,此种恢复恢复的数据完全正常,不用使用底层碎片扫描,亦可恢复asm中被删除数据文件数据.在某些特殊情况下,此类方法配合底层碎片恢复,可以实现更加完美的恢复效果.对于比较典型的oracle pdb被删除(因为有多个数据文件的文件号是一样的,无法直接通过底层碎片扫描恢复),通过此类方法可以非常好的恢复出来.
类似文章参考:
asm disk header 彻底损坏恢复
 ASM未正常启动,使用dd找回数据文件
 asm磁盘组操作不当导致数据文件丢失恢复
如果你不幸遭遇asm 数据文件被删除/丢失,或者误删除pdb等相关事宜,如果需要恢复可以联系我们,提供专业数据库恢复服务
Phone:17813235971 Q Q:107644445 E-Mail:dba@xifenfei.com

asm disk被加入vg恢复

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：asm disk被加入vg恢复

接到客户恢复请求:把oracle asm datagroup中的一个磁盘增加到vg中,现在磁盘组无法mount,数据库无法正常启动.远程登录现场进行分析发现情况如下:
操作系统层面分析
history操作记录

这里比较明显把一个磁盘做成pv,并且加入到vg中,然后再分配199G给lv_home,系统层面分析lvm信息

--查看pv信息
[root@xff1 ~]# pvdisplay 
  --- Physical volume ---
  PV Name               /dev/sda2
  VG Name               VolGroup
  PV Size               277.98 GiB / not usable 3.00 MiB
  Allocatable           yes (but full)
  PE Size               4.00 MiB
  Total PE              71161
  Free PE               0
  Allocated PE          71161
  PV UUID               F6QO3f-065n-mwTW-Xbq2-Xx2y-c8HD-Tkr7V7
   
  --- Physical volume ---
  PV Name               /dev/sdg    <----新加入的磁盘
  VG Name               VolGroup
  PV Size               200.00 GiB / not usable 4.00 MiB
  Allocatable           yes 
  PE Size               4.00 MiB
  Total PE              51199
  Free PE               255
  Allocated PE          50944
  PV UUID               i69vUG-nCIK-dtxL-FvpD-2WZd-bvLv-n7lwrb

[root@xff1 ~]# lvdisplay 
  --- Logical volume ---
  LV Path                /dev/VolGroup/lv_root
  LV Name                lv_root
  VG Name                VolGroup
  LV UUID                JUNnkN-m4zq-D0gh-h42b-cUM1-Wh1q-ZMtQE4
  LV Write Access        read/write
  LV Creation host, time localhost.localdomain, 2017-07-19 20:08:47 +0800
  LV Status              available
  # open                 1
  LV Size                50.00 GiB
  Current LE             12800
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:0
   
  --- Logical volume ---
  LV Path                /dev/VolGroup/lv_home
  LV Name                lv_home
  VG Name                VolGroup
  LV UUID                eZTkLt-cNGX-371i-m8Bd-VdD9-q6Hz-wYDRIJ
  LV Write Access        read/write
  LV Creation host, time localhost.localdomain, 2017-07-19 20:08:54 +0800
  LV Status              available
  # open                 1
  LV Size                422.97 GiB      <-----lv大小编程422G,应该是被扩了199G后结果
  Current LE             108281
  Segments               2
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:2
   
  --- Logical volume ---
  LV Path                /dev/VolGroup/lv_swap
  LV Name                lv_swap
  VG Name                VolGroup
  LV UUID                54P9ok-VpwO-zM68-hvwY-9GBf-89yb-8xQAMn
  LV Write Access        read/write
  LV Creation host, time localhost.localdomain, 2017-07-19 20:09:23 +0800
  LV Status              available
  # open                 1
  LV Size                4.00 GiB
  Current LE             1024
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:1


[root@xff1 ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root
                       50G  3.9G   43G   9% /
tmpfs                  63G  509M   63G   1% /dev/shm
/dev/sda1             477M   44M  408M  10% /boot
/dev/mapper/VolGroup-lv_home
                      417G  226G  170G  58% /home  <----增加了199g空间,剩余只剩170G,证明增加空间之后最少使用了30G以上

基于这样的情况,基本上可以确定sdg盘加入VolGroup中并且被分配给 lv_home中,而且还写入了数据(/home空闲空间只剩余170G,lv_home当时扩了199G).
asm层面分析
asm磁盘组无法mount,提示缺少一块磁盘

SQL> ALTER DISKGROUP DATA MOUNT  /* asm agent *//* {1:12056:279} */ 
NOTE: cache registered group DATA number=1 incarn=0xa1dbff16
NOTE: cache began mount (first) of group DATA number=1 incarn=0xa1dbff16
NOTE: Assigning number (1,2) to disk (/dev/asmdisk3)
NOTE: Assigning number (1,1) to disk (/dev/asmdisk2)
Sat Apr 25 13:04:58 2020
ERROR: no read quorum in group: required 1, found 0 disks
NOTE: cache dismounting (clean) group 1/0xA1DBFF16 (DATA) 
NOTE: messaging CKPT to quiesce pins Unix process pid: 81552, image: oracle@rac2db1 (TNS V1-V3)
NOTE: dbwr not being msg'd to dismount
NOTE: lgwr not being msg'd to dismount
NOTE: cache dismounted group 1/0xA1DBFF16 (DATA) 
NOTE: cache ending mount (fail) of group DATA number=1 incarn=0xa1dbff16
NOTE: cache deleting context for group DATA 1/0xa1dbff16
GMON dismounting group 1 at 19 for pid 30, osid 81552
NOTE: Disk DATA_0001 in mode 0x9 marked for de-assignment
NOTE: Disk DATA_0002 in mode 0x9 marked for de-assignment
ERROR: diskgroup DATA was not mounted
ORA-15032: not all alterations performed
ORA-15017: diskgroup "DATA" cannot be mounted
ORA-15040: diskgroup is incomplete
ERROR: ALTER DISKGROUP DATA MOUNT  /* asm agent *//* {1:12056:279} */

kfed分析磁盘信息

报错比较明显asm disk磁盘头被lvm的信息取代(因为asm disk 被加入到vg中),根据前面的分析,该磁盘被写入数据很可能超过30G,使用kfed分析一个随意au,确认被破坏,证明开始判断基本正确

root@xff1:/home/oracle11g$kfed read /dev/asmdisk1 aun=10000
kfbh.endian:                         51 ; 0x000: 0x33
kfbh.hard:                           55 ; 0x001: 0x37
kfbh.type:                           32 ; 0x002: *** Unknown Enum ***
kfbh.datfmt:                         42 ; 0x003: 0x2a
kfbh.block.blk:              1329801248 ; 0x004: blk=1329801248
kfbh.block.obj:              1128615502 ; 0x008: file=347726
kfbh.check:                  1094999892 ; 0x00c: 0x41445f54
kfbh.fcn.base:                675103060 ; 0x010: 0x283d4154
kfbh.fcn.wrap:               1448232275 ; 0x014: 0x56524553
kfbh.spare1:                 1598374729 ; 0x018: 0x5f454349
kfbh.spare2:                 1162690894 ; 0x01c: 0x454d414e
7F7843EAD400 2A203733 4F432820 43454E4E 41445F54  [37 * (CONNECT_DA]
7F7843EAD410 283D4154 56524553 5F454349 454D414E  [TA=(SERVICE_NAME]
7F7843EAD420 6361723D 29626432 44494328 5250283D  [=rac2db)(CID=(PR]
7F7843EAD430 4152474F 3A443D4D 4341505C DFCF3153  [OGRAM=D:\PACS1..]
7F7843EAD440 B3BEB7BB 6369445C 65536D6F 72657672  [....\DicomServer]
7F7843EAD450 445C524D 6D6F6369 76726553 524D7265  [MR\DicomServerMR]
7F7843EAD460 6578652E 4F482829 573D5453 362D4E49  [.exe)(HOST=WIN-6]
7F7843EAD470 51414C38 54553645 28294A30 52455355  [8LAQE6UT0J)(USER]
7F7843EAD480 6D64413D 73696E69 74617274 2929726F  [=Administrator))]
7F7843EAD490 202A2029 44444128 53534552 5250283D  [) * (ADDRESS=(PR]
7F7843EAD4A0 434F544F 743D4C4F 28297063 54534F48  [OTOCOL=tcp)(HOST]
7F7843EAD4B0 2E30313D 2E303831 30332E31 4F502829  [=10.180.1.30)(PO]
7F7843EAD4C0 343D5452 37333539 2A202929 74736520  [RT=49537)) * est]
7F7843EAD4D0 696C6261 2A206873 63617220 20626432  [ablish * rac2db ]
7F7843EAD4E0 3231202A 0A343135 2D534E54 31353231  [* 12514.TNS-1251]
7F7843EAD4F0 54203A34 6C3A534E 65747369 2072656E  [4: TNS:listener ]
7F7843EAD500 73656F64 746F6E20 72756320 746E6572  [does not current]
7F7843EAD510 6B20796C 20776F6E 7320666F 69767265  [ly know of servi]
7F7843EAD520 72206563 65757165 64657473 206E6920  [ce requested in ]
7F7843EAD530 6E6E6F63 20746365 63736564 74706972  [connect descript]
………………
7F7843EAE300 6F636944 7265536D 4D726576 69445C52  [DicomServerMR\Di]
7F7843EAE310 536D6F63 65767265 2E524D72 29657865  [comServerMR.exe)]
7F7843EAE320 534F4828 49573D54 4F302D4E 314B304A  [(HOST=WIN-0OJ0K1]
7F7843EAE330 4955304E 55282954 3D524553 696D6441  [N0UIT)(USER=Admi]
7F7843EAE340 7473696E 6F746172 29292972 28202A20  [nistrator))) * (]
7F7843EAE350 52444441 3D535345 4F525028 4F434F54  [ADDRESS=(PROTOCO]
7F7843EAE360 63743D4C 48282970 3D54534F 312E3031  [L=tcp)(HOST=10.1]
7F7843EAE370 312E3038 2930332E 524F5028 35353D54  [80.1.30)(PORT=55]
7F7843EAE380 29383632 202A2029 61747365 73696C62  [268)) * establis]
7F7843EAE390 202A2068 32636172 2A206264 35323120  [h * rac2db * 125]
7F7843EAE3A0 540A3431 312D534E 34313532 4E54203A  [14.TNS-12514: TN]
7F7843EAE3B0 696C3A53 6E657473 64207265 2073656F  [S:listener does ]
7F7843EAE3C0 20746F6E 72727563 6C746E65 6E6B2079  [not currently kn]
7F7843EAE3D0 6F20776F 65732066 63697672 65722065  [ow of service re]
7F7843EAE3E0 73657571 20646574 63206E69 656E6E6F  [quested in conne]
7F7843EAE3F0 64207463 72637365 6F747069 34320A72  [ct descriptor.24]
KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type][][32]

通过上述kfed可以看到第10000 au的位置被写入的是数据库异常之后listener.log的信息(该数据库安装在/home目录中),进一步证明覆盖,通过以下信息证明sdg就是asmdisk1

[root@xff1 dev]# ls -l sdg
brw-rw---- 1 root disk 8,  96 Apr 25 00:05 sdg
[root@xff1 dev]# ls -l asmdisk1
brw-rw---- 1 grid asmadmin 8,  96 Apr 25 00:05 asmdisk1

基于现在的情况,data磁盘组是由三块 200G的磁盘组成,第一块磁盘被意外加入vg,并且写入数据大于30G,无法从asm层面直接通过kfed修复磁盘组,然后直接mount,只能通过oracle asm磁盘数据块重组技术(asm disk header 彻底损坏恢复)实现没有覆盖数据的恢复.

该客户运气还不错,通过仅剩的2019年12月份几天的不成功备份找出来所有的数据文件（无归档）,然后强制拉库成功.通过碎片恢复的最新的数据文件数据结合2019年12月份备份,实现绝大部分业务数据恢复,最大限度减少客户损失.对于oracle rac数据库服务器磁盘操作需要谨慎.
如果不幸有类似oracle asm disk被破坏(格式化,dd部分,做成lv等),需要进行恢复支持,可以联系我们,做专业的恢复评估,最大限度,最快速度抢救数据,减少损失
Phone:17813235971 Q Q:107644445 E-Mail:dba@xifenfei.com
恢复过部分asm被格式化案例:
又一例asm格式化文件系统恢复
一次完美的asm disk被格式化ntfs恢复
oracle asm disk格式化恢复—格式化为ext4文件系统
oracle asm disk格式化恢复—格式化为ntfs文件系统

aix平台tab$被删除可能出现ORA-600 [16703], [1403], [28]错误

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：aix平台tab$被删除可能出现ORA-600 [16703], [1403], [28]错误

以前在恢复过程中遇到过ORA-00600: internal error code, arguments: [16703], [1403], [28]错误(10g数据库遭遇ORA-600 16703)以为是因为10g版本的tab$记录被删除的原因导致报错和最常见的ORA-00600: internal error code, arguments: [16703], [1403], [20]不完全一致(警告：互联网中有oracle介质被注入恶意程序导致—ORA-600 16703),最近又遇到一个ORA-600 16703 1403 28错误的case,而且数据库版本是11.2.0.4 for aix平台,进一步说明该问题不是由于10g和11g的tab$被删除的区别导致,更多可能是由于操作系统不一样,数据库启动基表访问顺序不一致导致,特此进行说明.
数据库启动成功后报错

Wed Apr 01 22:36:19 2020
Completed: ALTER DATABASE OPEN /* db agent *//* {2:54387:2} */
Wed Apr 01 22:36:19 2020
Starting background process CJQ0
Wed Apr 01 22:36:19 2020
CJQ0 started with pid=53, OS id=7078224 
Wed Apr 01 22:36:21 2020
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_ora_7668220.trc  (incident=40337):
ORA-00600: internal error code, arguments: [kzrini:!uprofile], [], [], [], [], [], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/orcl/orcl2/incident/incdir_40337/orcl2_ora_7668220_i40337.trc
Wed Apr 01 22:36:21 2020
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_ora_6881718.trc  (incident=40369):
ORA-00600: internal error code, arguments: [kzrini:!uprofile], [], [], [], [], [], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/orcl/orcl2/incident/incdir_40369/orcl2_ora_6881718_i40369.trc
Setting Resource Manager plan SCHEDULER[0x32DB]:DEFAULT_MAINTENANCE_PLAN via scheduler window
Setting Resource Manager plan DEFAULT_MAINTENANCE_PLAN via parameter

Wed Apr 01 22:51:16 2020
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_smon_7078802.trc:
ORA-00604: error occurred at recursive SQL level 1
ORA-00957: duplicate column name
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_smon_7078802.trc:
ORA-00604: error occurred at recursive SQL level 1
ORA-00957: duplicate column name
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_smon_7078802.trc:
ORA-00604: error occurred at recursive SQL level 1
ORA-00957: duplicate column name

Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_m000_4850312.trc  (incident=48045):
ORA-00600: internal error code, arguments: [kdfReserveSingle_1], [0], [65280], [], [], [], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/orcl/orcl2/incident/incdir_48045/orcl2_m000_4850312_i48045.trc
Thu Apr 02 00:59:35 2020
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_smon_7078802.trc:
ORA-00604: error occurred at recursive SQL level 1
ORA-00957: duplicate column name
Thu Apr 02 00:59:36 2020
DDE: Problem Key 'ORA 600 [kzrini:!uprofile]' was completely flood controlled (0x6)
Further messages for this problem key will be suppressed for up to 10 minutes
Thu Apr 02 00:59:37 2020
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_m000_4850312.trc  (incident=48046):
ORA-00600: internal error code, arguments: [kewrose_1], [600], 
[ORA-00600: internal error code, arguments: [kdfReserveSingle_1], [0], [65280], [], [], [], []

数据库再次重启报错

Completed: ALTER DATABASE MOUNT /* db agent *//* {1:25340:2} */
ALTER DATABASE OPEN /* db agent *//* {1:25340:2} */
This instance was first to open
Picked broadcast on commit scheme to generate SCNs
Thu Apr 02 02:17:59 2020
Thread 2 opened at log sequence 13485
  Current log# 3 seq# 13485 mem# 0: +DATA/orcl/onlinelog/group_3.265.1003137665
  Current log# 3 seq# 13485 mem# 1: +FLASH/orcl/onlinelog/group_3.259.1003137677
Successful open of redo thread 2
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Thu Apr 02 02:17:59 2020
SMON: enabling cache recovery
Instance recovery: looking for dead threads
Instance recovery: lock domain invalid but no dead threads
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_ora_7799020.trc  (incident=48395):
ORA-00600: internal error code, arguments: [16703], [1403], [28], [], [], [], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/orcl/orcl2/incident/incdir_48395/orcl2_ora_7799020_i48395.trc
Thu Apr 02 02:18:01 2020
Thu Apr 02 02:18:01 2020
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_ora_7799020.trc:
ORA-00704: bootstrap process failure
ORA-00704: bootstrap process failure
ORA-00600: internal error code, arguments: [16703], [1403], [28], [], [], [], [], [], [], [], [], []
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_ora_7799020.trc:
ORA-00704: bootstrap process failure
ORA-00704: bootstrap process failure
ORA-00600: internal error code, arguments: [16703], [1403], [28], [], [], [], [], [], [], [], [], []
Error 704 happened during db open, shutting down database
USER (ospid: 7799020): terminating the instance due to error 704
Instance terminated by USER, pid = 7799020
ORA-1092 signalled during: ALTER DATABASE OPEN /* db agent *//* {1:25340:2} */...

对于此类问题,通过分析,确定也是由于DBMS_SUPPORT_DBMONITORP恶意脚本导致tab$记录被删除,导致数据库启动异常,处理方法基本上就是对tab$进行恢复,然后open数据库.

expdp dmp被加密破坏恢复

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：expdp dmp被加密破坏恢复

有朋友oracle数据库dmp备份被加密,后缀为:.DMP.voyager,通过分析发现文件加密2M左右

这里可以看出来dmp文件为expdp方式导出(expdp本质上xml方式存储,exp使用直接二进制方式存储),通过工具分析可恢复表情况.
通过工具对该dmp文件进行分析

CPFL> OPEN F:\BaiduNetdisk\KINGDEE85GH_2020-03-17.DMP.voyager
TABLE_NAME                                         START_POS       DATA_BYTE     
-------------------------------------------------- --------------- --------------- 
KINGDEE85GH.T_WFD_PROCESSDEF                       116300288       648396035       
KINGDEE85GH.T_DYN_DYNAMICCONFIGURE                 864710656       181453794       
KINGDEE85GH.T_RPTS_STORAGEFILEDATA                 1078767616      21548951        
KINGDEE85GH.T_BOT_RULESEGMENT                      1100324864      10372516        
KINGDEE85GH.T_LOG_APP                              1110712320      12603573        
KINGDEE85GH.T_PM_PERMITEM                          1123336192      7282412         
KINGDEE85GH.T_PM_USERORGPERM                       1130635264      6692320         
KINGDEE85GH.T_DYN_APPSOLUTION                      1137336320      801697          
KINGDEE85GH.T_PM_MAINMENUITEM                      1138155520      3573943         
KINGDEE85GH.T_PM_PERMUIGROUP                       1141751808      2159245         
KINGDEE85GH.T_SYS_ENTITYREF                        1143922688      4183869         
KINGDEE85GH.T_PM_ROLEPERM                          1148116992      2758960         
KINGDEE85GH.T_BAS_SYSMENUITEM                      1150885888      3304627         
KINGDEE85GH.T_JP_PAGE                              1154211840      3019174  
…………      
KINGDEE85GH.T_XT_CHECKTIME                         1212776448      41              
KINGDEE85GH.T_XT_SYNCHTIME                         1212784640      41              
SYSTEM.SYS_EXPORT_SCHEMA_02                        1212792832      215423380    
-------------------------------------------------- --------------- --------------- 
Scanned Find 895 segments.

通过这个基本上可以确定丢失了100多M数据，其他数据理论上可以恢复.
创建用户

SQL> create user KINGDEE85GHidentified by oracle;

User created.

SQL> grant dba to KINGDEE85GH;

Grant succeeded.

unexpdp数据(自动创建表和导入数据)

CPFL> unexpdp table KINGDEE85GH.T_WFD_PROCESSDEF

unexpdp table: KINGDEE85GH.T_WFD_PROCESSDEF storage(START_POSITION:116300288 DATA_BYTE:748396035)
824 rows unexpdp

查看恢复结果

如果你有oracle expdp dmp被加密或者破坏,无法正常导入数据库,可以联系我们对其进行恢复处理:提供(ORACLE数据库恢复技术支持)：
Phone:17813235971 Q Q:107644445 E-Mail:dba@xifenfei.com
如果你的oracle dmp是exp方式导出,也可以联系我们对其进行处理,参见:
exp dmp文件损坏恢复
oracle dmp被加密恢复

又一例system大量坏块恢复

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：又一例system大量坏块恢复

有朋友找到我们,说数据库服务可以启动,但是无法登陆,类似报错

C:\Users\XIFENFEI>D:\app\XIFENFEI\product\11.2.0.1\dbhome_2\bin\sqlplus / as sys
dba

SQL*Plus: Release 11.2.0.1.0 Production on 星期四 3月 12 15:04:32 2020

Copyright (c) 1982, 2010, Oracle.  All rights reserved.

ERROR:
ORA-01075: 您现在已登录


请输入用户名:
ERROR:
ORA-01017: 用户名/口令无效; 登录被拒绝


请输入用户名:
ERROR:
ORA-01017: 用户名/口令无效; 登录被拒绝


SP2-0157: 在 3 次尝试之后无法连接到 ORACLE, 退出 SQL*Plus

通过分析发现数据库启动报错（未正常open成功）

C:\Users\XIFENFEI>D:\app\XIFENFEI\product\11.2.0.1\dbhome_2\bin\sqlplus / as sysdba

SQL*Plus: Release 11.2.0.1.0 Production on 星期四 3月 12 14:58:28 2020

Copyright (c) 1982, 2010, Oracle.  All rights reserved.

已连接到空闲例程。

SQL> startup mount pfile='F:\pfile.txt';
ORACLE 例程已经启动。

Total System Global Area 3307048960 bytes
Fixed Size                  2180264 bytes
Variable Size            1811942232 bytes
Database Buffers         1476395008 bytes
Redo Buffers               16531456 bytes
数据库装载完毕。

SQL> alter database open;
alter database open
*
第 1 行出现错误:
ORA-00604: 递归 SQL 级别 1 出现错误
ORA-01578: ORACLE 数据块损坏 (文件号 1, 块号 48396)
ORA-01110: 数据文件 1: 'F:\ORADATA\SYSTEM01.DBF'

数据库没有正常启动成功的原因是由于system文件有坏块导致,通过dbv检查system文件发现有大量连续坏块

DBVERIFY: Release 11.2.0.1.0 - Production on 星期四 3月 12 19:12:34 2020

Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.


DBVERIFY - 开始验证: FILE = F:\ORADATA\SYSTEM01.DBF
页 48064 流入 - 很可能是介质损坏
Corrupt block relative dba: 0x0040bbc0 (file 1, block 48064)
Fractured block found during dbv: 
Data in bad block:
 type: 6 format: 2 rdba: 0x0040bbc0
 last change scn: 0x0000.0006f69c seq: 0x1 flg: 0x06
 spare1: 0x0 spare2: 0x0 spare3: 0x0
 consistency value in tail: 0x43455474
 check value in block header: 0x3893
 computed block checksum: 0xadfa

…………

页 48412 标记为损坏
Corrupt block relative dba: 0x0040bd1c (file 1, block 48412)
Bad header found during dbv: 
Data in bad block:
 type: 36 format: 2 rdba: 0x6dce856d
 last change scn: 0xfc44.d24c936f seq: 0xdc flg: 0x3b
 spare1: 0x9c spare2: 0x92 spare3: 0xcf67
 consistency value in tail: 0x43455474
 check value in block header: 0x2598
 block checksum disabled


DBVERIFY - 验证完成

检查的页总数: 249600
处理的页总数 (数据): 209467
失败的页总数 (数据): 0
处理的页总数 (索引): 13616
失败的页总数 (索引): 0
处理的页总数 (其他): 3369
处理的总页数 (段)  : 1
失败的总页数 (段)  : 0
空的页总数: 22799
标记为损坏的总页数: 349
流入的页总数: 1
加密的总页数        : 0
最高块 SCN            : 84123103 (0.84123103)

数据库alert日志信息

Thu Mar 12 14:52:20 2020
SMON: enabling cache recovery
Successfully onlined Undo Tablespace 2.
Verifying file header compatibility for 11g tablespace encryption..
Verifying 11g file header compatibility for tablespace encryption completed
SMON: enabling tx recovery
Database Characterset is ZHS16GBK
Hex dump of (file 1, block 48403) in trace file d:\app\xifenfei\diag\rdbms\qdbdc\o11201gbk\trace\o11201gbk_ora_9480.trc
Corrupt block relative dba: 0x0040bd13 (file 1, block 48403)
Bad header found during multiblock buffer read
Data in bad block:
 type: 36 format: 2 rdba: 0x6dce856d
 last change scn: 0xfc44.d24c936f seq: 0xdc flg: 0x3b
 spare1: 0x9c spare2: 0x92 spare3: 0xcf67
 consistency value in tail: 0x43455474
 check value in block header: 0x2598
 block checksum disabled
Reading datafile 'F:\ORADATA\SYSTEM01.DBF' for corruption at rdba: 0x0040bd13 (file 1, block 48403)
Reread (file 1, block 48403) found same corrupt data
Hex dump of (file 1, block 48404) in trace file d:\app\xifenfei\diag\rdbms\qdbdc\o11201gbk\trace\o11201gbk_ora_9480.trc
Corrupt block relative dba: 0x0040bd14 (file 1, block 48404)
Corrupt Block Found
Bad header found during multiblock buffer read
         TSN = 0, TSNAME = SYSTEM
Data in bad block:
         RFN = 1, BLK = 48403, RDBA = 4242707
 type: 36 format: 2 rdba: 0x6dce856d
 last change scn: 0xfc44.d24c936f seq: 0xdc flg: 0x3b
 spare1: 0x9c spare2: 0x92 spare3: 0xcf67
 consistency value in tail: 0x43455474
 check value in block header: 0x2598
 block checksum disabled
Reading datafile 'F:\ORADATA\SYSTEM01.DBF' for corruption at rdba: 0x0040bd14 (file 1, block 48404)
Reread (file 1, block 48404) found same corrupt data
Errors in file d:\app\xifenfei\diag\rdbms\qdbdc\o11201gbk\trace\o11201gbk_ora_9480.trc  (incident=3784):
ORA-01578: ORACLE 数据块损坏 (文件号 1, 块号 48404)
ORA-01110: 数据文件 1: 'F:\ORADATA\SYSTEM01.DBF'
Incident details in: d:\app\xifenfei\diag\rdbms\qdbdc\o11201gbk\incident\incdir_3784\o11201gbk_ora_9480_i3784.trc
         OBJN = 36, OBJD = 36, OBJECT = I_OBJ1, SUBOBJECT = 
         SEGMENT OWNER = SYS, SEGMENT TYPE = Index Segment
Corrupt Block Found
         TSN = 0, TSNAME = SYSTEM
         RFN = 1, BLK = 48404, RDBA = 4242708
         OBJN = 36, OBJD = 36, OBJECT = I_OBJ1, SUBOBJECT = 
         SEGMENT OWNER = SYS, SEGMENT TYPE = Index Segment
Errors in file d:\app\xifenfei\diag\rdbms\qdbdc\o11201gbk\trace\o11201gbk_ora_9480.trc:
ORA-00604: 递归 SQL 级别 1 出现错误
ORA-01578: ORACLE 数据块损坏 (文件号 1, 块号 48404)
ORA-01110: 数据文件 1: 'F:\ORADATA\SYSTEM01.DBF'
Errors in file d:\app\xifenfei\diag\rdbms\qdbdc\o11201gbk\trace\o11201gbk_ora_9480.trc:
ORA-00604: 递归 SQL 级别 1 出现错误
ORA-01578: ORACLE 数据块损坏 (文件号 1, 块号 48404)
ORA-01110: 数据文件 1: 'F:\ORADATA\SYSTEM01.DBF'
Error 604 happened during db open, shutting down database
USER (ospid: 9480): terminating the instance due to error 604

这里可以看出来数据库不能正常启动的原因,主要是由于I_OBJ1(obj$表的index)刚好被损坏,导致数据库无法,通过分析定位确定是如下sql导致启动失败

Dump continued from file: d:\app\xifenfei\diag\rdbms\qdbdc\o11201gbk\trace\o11201gbk_ora_9480.trc
ORA-01578: ORACLE 数据块损坏 (文件号 1, 块号 48404)
ORA-01110: 数据文件 1: 'F:\ORADATA\SYSTEM01.DBF'

========= Dump for incident 3784 (ORA 1578) ========

*** 2020-03-12 14:52:20.614
dbkedDefDump(): Starting incident default dumps (flags=0x2, level=3, mask=0x0)
----- Current SQL Statement for this session (sql_id=cq514nkrp38hv) -----
select distinct d.p_obj#,d.p_timestamp from sys.dependency$ d, obj$ o where d.p_obj#>=:1 and 
d.d_obj#=o.obj# and o.status not in (5,6)

通过对其i_obj1损坏block进行修复,数据库正启动成功

C:\Users\XIFENFEI>D:\app\XIFENFEI\product\11.2.0.1\dbhome_2\bin\sqlplus / as sysdba

SQL*Plus: Release 11.2.0.1.0 Production on 星期四 3月 12 15:05:48 2020

Copyright (c) 1982, 2010, Oracle.  All rights reserved.

已连接到空闲例程。

SQL> startup pfile='f:/pfile.txt' mount;
ORACLE 例程已经启动。

Total System Global Area 3307048960 bytes
Fixed Size                  2180264 bytes
Variable Size            1811942232 bytes
Database Buffers         1476395008 bytes
Redo Buffers               16531456 bytes
数据库装载完毕。
SQL> alter database open;

数据库已更改。

尝试导出数据

C:\Windows\system32>exp system/oracle owner=gis_sys file=f:/gis_sys.dmp FEEDBACK
=10000  COMPRESS=NO BUFFER=102400000 STATISTICS=none

Export: Release 11.2.0.1.0 - Production on 星期四 3月 12 15:46:19 2020

Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.


连接到: Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
已导出 ZHS16GBK 字符集和 AL16UTF16 NCHAR 字符集

即将导出指定的用户...
. 正在导出 pre-schema 过程对象和操作
. 正在导出用户 GIS_SYS 的外部函数库名
. 导出 PUBLIC 类型同义词
EXP-00008: 遇到 ORACLE 错误 604
ORA-00604: 递归 SQL 级别 1 出现错误
ORA-01410: 无效的 ROWID
EXP-00000: 导出终止失败

分析trace文件

*** SESSION ID:(192.3) 2020-03-12 15:05:36.132
OBJD MISMATCH typ=6, seg.obj=18, diskobj=224, dsflg=100100, dsobj=18, tid=18, cls=1
ORA-00604: 递归 SQL 级别 1 出现错误
ORA-01410: 无效的 ROWID
ORA-00604: 递归 SQL 级别 1 出现错误
ORA-01410: 无效的 ROWID

由于obj id为18(obj$)的对象和对应的数据库中实际block存储的表的block为224(aud$)不匹配,从而出现该错误,通过分析是由于i_obj1记录错误导致该问题,通过修复该记录之后,数据实现完美导出.

类似system文件坏块案例有:
通过拷贝block实现system文件大量坏块恢复
记录一次system表空间坏块(ORA-01578)数据库恢复
SYSTEM表空间坏块恢复—C_TS#对象坏块恢复(file 1 block 60)
file 1 block 128 corrupted/坏块恢复—system rollback坏块修复

oracle文件被删除且部分被覆盖恢复案例

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：oracle文件被删除且部分被覆盖恢复案例

有客户数据库异常,让我们对其进行分析,判断是否可以恢复,让客户通过Oracle数据库异常恢复检查脚本(Oracle Database Recovery Check)收集数据库信息,发现有三个数据文件异常

通过和客户确认,大概情况是这样的:由于/opt目录满了,客户把/opt/oracle整体迁移到/home目录中,然后通过link目录的方式实现迁移,但是在迁移之前把/home/oracle中的所有数据文件给rm掉了,然后把/opt中的数据拷贝到/home/中,在启动数据库的时候提示/home/oracle/JXWR.dbf文件丢失,然后又人工创建了一个该文件,从而出现了上述的三个文件异常(其实删除的数据库文件有十几个,涉及该库的有三个,客户只要恢复JXWR表空间数据即可).对于这种情况,有可能有覆盖的风险,让客户提供空间对现有/home 分区进行镜像,通过工具分析镜像文件

发现需要恢复文件大小/时间均不对,查看内容发现是oracle的审计trace,证明该文件对应的位置已经有覆盖,对于这样的情况,无法从os层面反删除进行恢复,考虑通过oracle碎片层面进行处理,对其分析发现大量block依旧存在(情况有点复杂,因为该目录涉及多个库,通过分析确认相关段为该数据库文件)

检查重组出来的数据文件效果

检查效果比较乐观,因为根据这样的情况,丢失15%左右的block算是非常理想的效果,然后通过oracle dul恢复客户需要的数据

完成数据库恢复任务.
这次的恢复效果不是太好主要就是由于客户删除文件之后,对被删除文件所在分区进行了大量写操作导致不少数据库block被覆盖,最终只能抱着试试看的态度最大限度恢复,属于比较侥幸的恢复成功,再次提醒各位:在数据被误删除之后,应该先保护现场(不要对其分区进行写操作),覆盖的越少,恢复的效果越好.

数字后缀名加密数据库恢复

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：数字后缀名加密数据库恢复

一个oracle dmp文件被加密破坏的case,加密提示如下

通过工具分析发现该文件前面1M被破坏

通过我们工具对头部损坏的1M数据进行特殊处理,数据直接使用imp命令导入

如果你有各种被类似病毒加密的数据库(oracle,sql server,mysql),我们可以提供专业的恢复支持,实现不给黑客交钱的情况下,数据几乎完美恢复
Tel/微信:17813235971 Q Q:107644445 E-Mail:dba@xifenfei.com提供专业的恢复服务.

xfs删除数据文件恢复

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：xfs删除数据文件恢复

在linux7的环境（xfs文件系统格式）中,由于误操作执行了rm -rf /操作,导致系统大部分文件被删除（oracle数据库相关文件）.我们对其现场进行分析,确认相关数据在磁盘底层依旧存在

运气不错通过底层恢复,实现数据库完美open

oracle dmp被加密恢复

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：oracle dmp被加密恢复

接到朋友恢复请求,oracle dmp文件被加密

通过分析文件发现该加密主要是对头尾部分block按照每16byte中8个byte置空和部分加密

通过进行恢复,对损坏部分进行跳过,剩余数据直接导入数据库,通过show=y测试数据可以正常入库,实现了dmp文件表数据的完美恢复

再一起asm disk被格式化成ext3文件系统故障恢复

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：再一起asm disk被格式化成ext3文件系统故障恢复

国庆节前夕接到朋友求救电话asm disk被格式化成ext3格式了,具体操作如下

并且把这个分区直接挂载到/目录

由于/被挂载新格式化的控盘,导致asm磁盘组访问其他盘报错

Sun Sep 29 18:15:02 2019
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_b000_8094.trc:
ORA-15025: could not open disk "/dev/asmdisk/sdh"
ORA-27041: unable to open file
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_b000_8094.trc:
ORA-15025: could not open disk "/dev/asmdisk/sdh"
ORA-27041: unable to open file
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
WARNING: cache failed reading from group=1(DATA) fn=9 blk=0 count=1 from
disk= 5 (DATA_0005) kfkist=0x20 status=0x02 osderr=0x0 file=kfc.c line=11596
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_b000_8094.trc:
ORA-15025: could not open disk "/dev/asmdisk/sdh"
ORA-27041: unable to open file
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-15080: synchronous I/O operation to a disk failed
WARNING: cache succeeded reading from group=1(DATA) fn=9 blk=0 count=1 from
disk= 7 (DATA_0007) kfkist=0x20 status=0x01 osderr=0x0 file=kfc.c line=11637
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_b000_8094.trc:
ORA-15025: could not open disk "/dev/asmdisk/sdh"
ORA-27041: unable to open file
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
WARNING: PST-initiated drop of 1 disk(s) in group 1(.2380027701))

重启系统之后,重试mount 磁盘组

GMON dismounting group 1 at 18 for pid 31, osid 44279
NOTE: Disk DATA_0000 in mode 0x7f marked for de-assignment
NOTE: Disk DATA_0001 in mode 0x7f marked for de-assignment
NOTE: Disk DATA_0002 in mode 0x7f marked for de-assignment
NOTE: Disk DATA_0003 in mode 0x7f marked for de-assignment
NOTE: Disk DATA_0004 in mode 0x1 marked for de-assignment
NOTE: Disk DATA_0005 in mode 0x7f marked for de-assignment
NOTE: Disk DATA_0006 in mode 0x7f marked for de-assignment
NOTE: Disk DATA_0007 in mode 0x7f marked for de-assignment
NOTE: Disk  in mode 0x7f marked for de-assignment
NOTE: Disk DATA_0009 in mode 0x7f marked for de-assignment
NOTE: Disk DATA_0010 in mode 0x7f marked for de-assignment
NOTE: Disk DATA_0011 in mode 0x7f marked for de-assignment
NOTE: Disk DATA_0012 in mode 0x7f marked for de-assignment
NOTE: Disk DATA_0013 in mode 0x7f marked for de-assignment
NOTE: Disk DATA_0014 in mode 0x7f marked for de-assignment
ERROR: diskgroup DATA was not mounted
ORA-15032: not all alterations performed
ORA-15040: diskgroup is incomplete
ORA-15042: ASM disk "8" is missing from group number "1"
ERROR: ALTER DISKGROUP DATA MOUNT  /* asm agent *//* {1:2587:2} */

由于sdb（asm disk 8）被格式化,导致data磁盘组无法正常mount.这个客户运气比较好data 磁盘组是normal模式,但是由于mount到/,导致disk 4被强制drop,因此无法mount成功,但是通过一系列处理数据实现完美恢复,0数据丢失

如果磁盘组是外部冗余,请参考:
又一例asm格式化文件系统恢复
 一次完美的asm disk被格式化ntfs恢复
 oracle asm disk格式化恢复—格式化为ext4文件系统
 oracle asm disk格式化恢复—格式化为ntfs文件系统

分类目录归档：非常规恢复