联系:手机/微信(+86 17813235971) QQ(107644445)
标题:ORA-15063: ASM discovered an insufficient number of disks for diskgroup 恢复
作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]
客户反馈三个磁盘组无法正常mount,报错类似ORA-15032 ORA-15017 ORA-15063
SQL> ALTER DISKGROUP ASM_DATA MOUNT /* asm agent *//* {0:0:2} */ NOTE: cache registered group ASM_DATA number=1 incarn=0xffa85ccd NOTE: cache began mount (first) of group ASM_DATA number=1 incarn=0xffa85ccd ERROR: no read quorum in group: required 2, found 0 disks NOTE: cache dismounting (clean) group 1/0xFFA85CCD (ASM_DATA) NOTE: messaging CKPT to quiesce pins Unix process pid: 5709, image: oracle@XFF (TNS V1-V3) NOTE: dbwr not being msg'd to dismount NOTE: lgwr not being msg'd to dismount NOTE: cache dismounted group 1/0xFFA85CCD (ASM_DATA) NOTE: cache ending mount (fail) of group ASM_DATA number=1 incarn=0xffa85ccd NOTE: cache deleting context for group ASM_DATA 1/0xffa85ccd Tue Jun 21 12:24:38 2022 NOTE: No asm libraries found in the system ASM Health Checker found 1 new failures GMON dismounting group 1 at 16 for pid 19, osid 5709 ERROR: diskgroup ASM_DATA was not mounted ORA-15032: not all alterations performed ORA-15017: diskgroup "ASM_DATA" cannot be mounted ORA-15063: ASM discovered an insufficient number of disks for diskgroup "ASM_DATA" ERROR: ALTER DISKGROUP ASM_DATA MOUNT /* asm agent *//* {0:0:2} */
初步判断是asm disk异常导致(比如asm disk不能被扫描到,或者丢失,或者磁盘头损坏等),分析客户的asm disk的udev文件配置
KERNEL=="sdd1", NAME="asm_grid", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sde1", NAME="asm_system", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sdf1", NAME="asm_data", OWNER="grid", GROUP="asmadmin", MODE="0660"
从udev的配置中可以看出来,客户以前是对3个磁盘进行分析,然后使用udev映射别名给asm使用的.通过对其中一个磁盘进行分析
通过上述winhex查看,可以确认该分区的磁盘头信息异常[该信息属于磁盘刚分区的时候信息,而不是asm disk的信息],和kfed看到的结果一致[磁盘头位置肯定损坏,其他位置目前未知]
H:\TEMP\dd>kfed read sdf_sdf1.dd kfbh.endian: 0 ; 0x000: 0x00 kfbh.hard: 0 ; 0x001: 0x00 kfbh.type: 0 ; 0x002: KFBTYP_INVALID kfbh.datfmt: 0 ; 0x003: 0x00 kfbh.block.blk: 0 ; 0x004: blk=0 kfbh.block.obj: 0 ; 0x008: file=0 kfbh.check: 0 ; 0x00c: 0x00000000 kfbh.fcn.base: 0 ; 0x010: 0x00000000 kfbh.fcn.wrap: 0 ; 0x014: 0x00000000 kfbh.spare1: 0 ; 0x018: 0x00000000 kfbh.spare2: 0 ; 0x01c: 0x00000000 0064D8400 00000000 00000000 00000000 00000000 [................] Repeat 26 times 0064D85B0 00000000 00000000 00000000 02000000 [................] 0064D85C0 FE8E0001 003FFFFF DFFC0000 0000257F [......?......%..] 0064D85D0 00000000 00000000 00000000 00000000 [................] Repeat 1 times 0064D85F0 00000000 00000000 00000000 AA550000 [..............U.] 0064D8600 00000000 00000000 00000000 00000000 [................] Repeat 223 times KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type][][0]
分析其他位置的block情况,初步看基本上ok[运气还不错]
H:\TEMP\dd>kfed read sdf_sdf1.dd blkn=2|grep kfbh.type kfbh.type: 3 ; 0x002: KFBTYP_ALLOCTBL H:\TEMP\dd>kfed read sdf_sdf1.dd blkn=3|grep kfbh.type kfbh.type: 3 ; 0x002: KFBTYP_ALLOCTBL H:\TEMP\dd>kfed read sdf_sdf1.dd blkn=1 aun=2|grep kfbh.type kfbh.type: 3 ; 0x002: KFBTYP_ALLOCTBL
通过检索备份出来的部分磁盘文件,找出来ORCLDISK信息部分(asm disk header)
然后利用这个部分对损坏的磁盘头进行修复,并且dd回生产环境中,并尝试mount磁盘组,数据库open成功
至此这个数据库运气不错,没有过多损坏,算完美恢复,可以进行了逻辑导出和rman备份,全部正常.为了后续安全,建议对其进行迁移