update undo$ – Database SOS

有客户数据库启动报ORA-01558: out of transaction ID’s in rollback segment SYSTorcl无法open成功,而且对应的备库也无法正常open(报同样错误)

对ORA-01558错误分析

[oracle@iZbp1hx0enix3hix1kvyrxZ ~]$  oerr ora 1558
01558, 00000, "out of transaction ID's in rollback segment %s"
// *Cause: All the available transaction id's have been used
// *Action: Shutdown the instance and restart using other rollback segment(s),
//          then drop the rollback segment that has no more transaction id's.

官方对于该错误的解释为由于回滚段的事务ID被使用完,无法再分配从而报ORA-01558,解决方法是重启库使用其他回滚段,并删除这个异常回滚段.对于此次故障的报错,这个方法原则上行不通,因为异常的是systorcl里面的rollback回滚段(也就是我们通常说的系统回滚段).通过alert日志分析客户这个故障的前因后果。
正常运行的库,突然报该错误

Fri Jun 03 19:37:36 2022
Thread 1 advanced to log sequence 2766146 (LGWR switch)
  Current log# 3 seq# 2766146 morcl# 0: /oradata/orclr/group_3.310.962631159
Fri Jun 03 19:37:36 2022
LNS: Standby redo logfile selected for thread 1 sequence 2766146 for destination LOG_ARCHIVE_DEST_2
Fri Jun 03 19:37:37 2022
Archived Log entry 3560157 added for thread 1 sequence 2766145 ID 0xefc56f40 dest 1:
Fri Jun 03 19:37:58 2022
Errors in file /u01/app/oracle/diag/rdbms/orclr/orclr/trace/orclr_smon_9974.trc:
ORA-00604: error occurred at recursive SQL level 1
ORA-01558: out of transaction ID's in rollback segment SYSTorcl
Errors in file /u01/app/oracle/diag/rdbms/orclr/orclr/trace/orclr_smon_9974.trc:
ORA-00604: error occurred at recursive SQL level 1
ORA-01558: out of transaction ID's in rollback segment SYSTorcl
Errors in file /u01/app/oracle/diag/rdbms/orclr/orclr/trace/orclr_smon_9974.trc:
ORA-00604: error occurred at recursive SQL level 1
ORA-01558: out of transaction ID's in rollback segment SYSTorcl
Errors in file /u01/app/oracle/diag/rdbms/orclr/orclr/trace/orclr_smon_9974.trc:
ORA-00604: error occurred at recursive SQL level 1
ORA-01558: out of transaction ID's in rollback segment SYSTorcl
Errors in file /u01/app/oracle/diag/rdbms/orclr/orclr/trace/orclr_smon_9974.trc:
ORA-00604: error occurred at recursive SQL level 1
ORA-01558: out of transaction ID's in rollback segment SYSTorcl
Fri Jun 03 19:38:13 2022
Errors in file /u01/app/oracle/diag/rdbms/orclr/orclr/trace/orclr_smon_9974.trc:
ORA-00604: error occurred at recursive SQL level 1
ORA-01558: out of transaction ID's in rollback segment SYSTorcl

这个错误一直报了很长时间,等到有时间窗口,客户安排重启操作

Mon Sep 26 14:10:12 2022
Shutting down instance (immediate)
Shutting down instance: further logons disabled
License high water mark = 205
All dispatchers and shared servers shutdown
Errors in file /u01/app/oracle/diag/rdbms/orclr/orclr/trace/orclr_smon_9974.trc:
ORA-00604: error occurred at recursive SQL level 1
ORA-01558: out of transaction ID's in rollback segment SYSTorcl
ALTER DATABASE CLOSE NORMAL
ORA-1558 signalled during: ALTER DATABASE CLOSE NORMAL...
Mon Sep 26 14:10:22 2022
Errors in file /u01/app/oracle/diag/rdbms/orclr/orclr/trace/orclr_smon_9974.trc:
ORA-00604: error occurred at recursive SQL level 1
ORA-01558: out of transaction ID's in rollback segment SYSTorcl
Mon Sep 26 14:10:33 2022
Errors in file /u01/app/oracle/diag/rdbms/orclr/orclr/trace/orclr_smon_9974.trc:
ORA-00604: error occurred at recursive SQL level 1
ORA-01558: out of transaction ID's in rollback segment SYSTorcl
Mon Sep 26 14:10:43 2022
Errors in file /u01/app/oracle/diag/rdbms/orclr/orclr/trace/orclr_smon_9974.trc:
ORA-00604: error occurred at recursive SQL level 1
ORA-01558: out of transaction ID's in rollback segment SYSTorcl
Mon Sep 26 14:10:53 2022
Errors in file /u01/app/oracle/diag/rdbms/orclr/orclr/trace/orclr_smon_9974.trc:
ORA-00604: error occurred at recursive SQL level 1
ORA-01558: out of transaction ID's in rollback segment SYSTorcl
Mon Sep 26 14:10:55 2022
Shutting down instance (abort)
License high water mark = 205
USER (ospid: 25049): terminating the instance
Instance terminated by USER, pid = 25049

正常shutdown immediate执行失败,直接abort方式关闭库,然后尝试重启库

Mon Sep 26 14:12:16 2022
ARC3 started with pid=44, OS id=25978 
ARC1: Archival started
ARC2: Archival started
ARC1: Becoming the 'no FAL' ARCH
ARC1: Becoming the 'no SRL' ARCH
ARC2: Becoming the heartbeat ARCH
ARC3: Archival started
ARC0: STARTING ARCH PROCESSES COMPLETE
Thread 1 opened at log sequence 2782827
  Current log# 1 seq# 2782827 morcl# 0: /oradata/orclr/group_1.296.962631151
Successful open of redo thread 1
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Mon Sep 26 14:12:16 2022
SMON: enabling cache recovery
Mon Sep 26 14:12:16 2022
NSA2 started with pid=46, OS id=25982 
Archived Log entry 3593518 added for thread 1 sequence 2782826 ID 0xefc56f40 dest 1:
ARC3: Standby redo logfile selected for thread 1 sequence 2782826 for destination LOG_ARCHIVE_DEST_2
Errors in file /u01/app/oracle/diag/rdbms/orclr/orclr/trace/orclr_ora_25879.trc:
ORA-00604: error occurred at recursive SQL level 1
ORA-01558: out of transaction ID's in rollback segment SYSTorcl
Errors in file /u01/app/oracle/diag/rdbms/orclr/orclr/trace/orclr_ora_25879.trc:
ORA-00604: error occurred at recursive SQL level 1
ORA-01558: out of transaction ID's in rollback segment SYSTorcl
Error 604 happened during db open, shutting down database
USER (ospid: 25879): terminating the instance due to error 604
Instance terminated by USER, pid = 25879
ORA-1092 signalled during: ALTER DATABASE OPEN...
opiodr aborting process unknown ospid (25879) as a result of ORA-1092
Mon Sep 26 14:12:17 2022
ORA-1092 : opitsk aborting process

数据库重启失败,报ORA-604和ORA-01558错误.
对于这种情况,对于数据库启动过程进行跟踪确认在UPDATE UNDO$的时候无法分配事务,导致无法继续

PARSING IN CURSOR #139696084476000 len=160 dep=1 uid=0 oct=6 lid=0 tim=1664353484223278 hv=1292341136 
ad='6ff1ffa8' sqlid='8vyjutx6hg3wh'
update /*+ rule */ undo$ set name=:2,file#=:3,block#=:4,status$=:5,user#=:6,undosqn=:7,xactsqn=:8,
scnbas=:9,scnwrp=:10,inst#=:11,ts#=:12,spare1=:13 where us#=:1
END OF STMT
PARSE #139696084476000:c=5998,e=10708,p=7,cr=53,cu=0,mis=1,r=0,dep=1,og=3,plh=0,tim=1664353484223277
BINDS #139696084476000:
 Bind#0
  oacdty=01 mxl=32(21) mxlc=00 mal=00 scl=00 pre=00
  oacflg=18 fl2=0001 frm=01 csi=852 siz=32 off=0
  kxsbbbfp=6ff376da  bln=32  avl=21  flg=09
  value="_SYSSMU14_3733658264$"
 Bind#1
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=7f0d877d7a68  bln=24  avl=02  flg=05
  value=15
 Bind#2
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=7f0d877d7a38  bln=24  avl=03  flg=05
  value=336
 Bind#3
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=7f0d877d7a00  bln=24  avl=02  flg=05
  value=5
 Bind#4
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=7f0d877d79d0  bln=24  avl=02  flg=05
  value=1
 Bind#5
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=7f0d877d79a0  bln=24  avl=03  flg=05
  value=5747
 Bind#6
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=7f0d877d7970  bln=24  avl=04  flg=05
  value=22103
 Bind#7
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=7f0d877d7938  bln=24  avl=06  flg=05
  value=1026171661
 Bind#8
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=7f0d877d7680  bln=24  avl=03  flg=05
  value=3399
 Bind#9
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=7f0d877d7650  bln=24  avl=02  flg=05
  value=2
 Bind#10
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=7f0d877d7620  bln=24  avl=02  flg=05
  value=5
 Bind#11
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=7f0d877d75f0  bln=24  avl=02  flg=05
  value=2
 Bind#12
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=7f0d877d7a98  bln=22  avl=02  flg=05
  value=14
EXEC #139696084476000:c=1000,e=1713,p=0,cr=1,cu=2,mis=1,r=0,dep=1,og=3,plh=3078630091,tim=1664353484225072
ERROR #139696084476000:err=1558 tim=1664353484225094
STAT #139696084476000 id=1 cnt=0 pid=0 pos=1 obj=0 op='UPDATE  UNDO$ (cr=0 pr=0 pw=0 time=5 us)'
STAT #139696084476000 id=2 cnt=1 pid=1 pos=1 obj=34 op='INDEX UNIQUE SCAN I_UNDO1 (cr=1 pr=0 pw=0 time=6 us)'
KQRCMT: Write failed with error=604 po=0x6ff375d0 cid=3
diagnostics : cid=3 hash=f2114ab9 flag=2a
ORA-00604: error occurred at recursive SQL level 1
ORA-01558: out of transaction ID's in rollback segment SYSTorcl
ORA-00604: error occurred at recursive SQL level 1
ORA-01558: out of transaction ID's in rollback segment SYSTorcl

既然报了事务ID不足,那对block进行分析,确实几乎都达到了数据库设计的理论最大值

  index  state cflags  wrap#    uel         scn            dba            parent-xid    nub     stmt_num
  ------------------------------------------------------------------------------------------------
   0x00    9    0x00  0xfffe  0x0026  0x111f.4d598aae  0x00400188  0x0000.000.00000000  0x00000001    0x00000000
   0x01    9    0x00  0xfffe  0x0054  0x111f.4d598a63  0x00400186  0x0000.000.00000000  0x00000001    0x00000000
   0x02    9    0x00  0xfffe  0x001a  0x111f.4d598a77  0x00400187  0x0000.000.00000000  0x00000001    0x00000000
   0x03    9    0x00  0xfffe  0x005c  0x111f.4d598a7d  0x00400187  0x0000.000.00000000  0x00000001    0x00000000
   0x04    9    0x00  0xfffe  0x003d  0x111f.4d598a4d  0x00400185  0x0000.000.00000000  0x00000001    0x00000000
   0x05    9    0x00  0xfffe  0x0061  0x111f.4d598a74  0x00400186  0x0000.000.00000000  0x00000001    0x00000000
   0x06    9    0x00  0xfffe  0x0002  0x111f.4d598a76  0x00400187  0x0000.000.00000000  0x00000001    0x00000000
   0x07    9    0x00  0xfffe  0x001b  0x111f.4d598a90  0x00400187  0x0000.000.00000000  0x00000001    0x00000000
   0x08    9    0x00  0xfffe  0x0011  0x111f.4d598a8c  0x00400187  0x0000.000.00000000  0x00000001    0x00000000
   0x09    9    0x00  0xfffe  0x0042  0x111f.4d598a6e  0x00400186  0x0000.000.00000000  0x00000001    0x00000000
   0x0a    9    0x00  0xfffe  0x0003  0x111f.4d598a7c  0x00400187  0x0000.000.00000000  0x00000001    0x00000000
   0x0b    9    0x00  0xfffe  0x000a  0x111f.4d598a7b  0x00400187  0x0000.000.00000000  0x00000001    0x00000000
   0x0c    9    0x00  0xfffe  0x0005  0x111f.4d598a73  0x00400186  0x0000.000.00000000  0x00000001    0x00000000
   0x0d    9    0x00  0xfffe  0x005d  0x111f.4d598a87  0x00400187  0x0000.000.00000000  0x00000001    0x00000000
   0x0e    9    0x00  0xfffe  0x0050  0x111f.4d598a68  0x00400186  0x0000.000.00000000  0x00000001    0x00000000
   0x0f    9    0x00  0xfffe  0x0047  0x111f.4d598a6a  0x00400186  0x0000.000.00000000  0x00000001    0x00000000
   0x10    9    0x00  0xfffe  0x0033  0x111f.4d598a95  0x00400187  0x0000.000.00000000  0x00000001    0x00000000
   0x11    9    0x00  0xfffe  0x0031  0x111f.4d598a8d  0x00400187  0x0000.000.00000000  0x00000001    0x00000000
   0x12    9    0x00  0xfffe  0x0020  0x111f.4d598a81  0x00400187  0x0000.000.00000000  0x00000001    0x00000000
   0x13    9    0x00  0xfffe  0x0014  0x111f.4d598a85  0x00400187  0x0000.000.00000000  0x00000001    0x00000000
   0x14    9    0x00  0xfffe  0x000d  0x111f.4d598a86  0x00400187  0x0000.000.00000000  0x00000001    0x00000000
   0x15    9    0x00  0xfffe  0x0013  0x111f.4d598a83  0x00400187  0x0000.000.00000000  0x00000001    0x00000000
   0x16    9    0x00  0xfffe  0x003b  0x111f.4d598aa4  0x00400188  0x0000.000.00000000  0x00000001    0x00000000
   0x17    9    0x00  0xfffe  0x005b  0x111f.4d598a70  0x00400186  0x0000.000.00000000  0x00000001    0x00000000
   0x18    9    0x00  0xfffe  0x001f  0x111f.4d598a8a  0x00400187  0x0000.000.00000000  0x00000001    0x00000000
   0x19    9    0x00  0xfffe  0x0010  0x111f.4d598a94  0x00400187  0x0000.000.00000000  0x00000001    0x00000000
   0x1a    9    0x00  0xfffe  0x001c  0x111f.4d598a79  0x00400187  0x0000.000.00000000  0x00000001    0x00000000
   0x1b    9    0x00  0xfffe  0x0027  0x111f.4d598a91  0x00400187  0x0000.000.00000000  0x00000001    0x00000000
   0x1c    9    0x00  0xfffe  0x000b  0x111f.4d598a7a  0x00400187  0x0000.000.00000000  0x00000001    0x00000000
   0x1d    9    0x00  0xfffe  0x0012  0x111f.4d598a80  0x00400187  0x0000.000.00000000  0x00000001    0x00000000
   0x1e    9    0x00  0xfffe  0x0045  0x111f.4d598aba  0x00400188  0x0000.000.00000000  0x00000001    0x00000000
   0x1f    9    0x00  0xfffe  0x0008  0x111f.4d598a8b  0x00400187  0x0000.000.00000000  0x00000001    0x00000000
   0x20    9    0x00  0xfffe  0x0015  0x111f.4d598a82  0x00400187  0x0000.000.00000000  0x00000001    0x00000000
   0x21    9    0x00  0xfffe  0x0038  0x111f.4d598ab2  0x00400188  0x0000.000.00000000  0x00000001    0x00000000
   0x22    9    0x00  0xfffd  0x003e  0x111f.4d598ab4  0x00400188  0x0000.000.00000000  0x00000001    0x00000000
   0x23    9    0x00  0xfffe  0x0028  0x111f.4d598a9e  0x00400188  0x0000.000.00000000  0x00000001    0x00000000
   0x24    9    0x00  0xfffd  0x0060  0x111f.4d598a5e  0x00400186  0x0000.000.00000000  0x00000001    0x00000000
   0x25    9    0x00  0xfffe  0x0021  0x111f.4d598ab0  0x00400188  0x0000.000.00000000  0x00000001    0x00000000
   0x26    9    0x00  0xfffe  0x0025  0x111f.4d598aaf  0x00400188  0x0000.000.00000000  0x00000001    0x00000000
   0x27    9    0x00  0xfffe  0x0019  0x111f.4d598a93  0x00400187  0x0000.000.00000000  0x00000001    0x00000000
   0x28    9    0x00  0xfffe  0x0046  0x111f.4d598a9f  0x00400188  0x0000.000.00000000  0x00000001    0x00000000
   0x29    9    0x00  0xfffe  0x0023  0x111f.4d598a9d  0x00400188  0x0000.000.00000000  0x00000001    0x00000000
   0x2a    9    0x00  0xfffe  0x002c  0x111f.4d598a98  0x00400187  0x0000.000.00000000  0x00000001    0x00000000
   0x2b    9    0x00  0xfffe  0x0040  0x111f.4d598aa7  0x00400188  0x0000.000.00000000  0x00000001    0x00000000
   0x2c    9    0x00  0xfffe  0x0030  0x111f.4d598a99  0x00400188  0x0000.000.00000000  0x00000001    0x00000000
   0x2d    9    0x00  0xfffe  0x003c  0x111f.4d598aab  0x00400188  0x0000.000.00000000  0x00000001    0x00000000
   0x2e    9    0x00  0xfffe  0x0056  0x111f.4d598abf  0x00400189  0x0000.000.00000000  0x00000001    0x00000000
   0x2f    9    0x00  0xfffe  0x0037  0x111f.4d598aa2  0x00400188  0x0000.000.00000000  0x00000001    0x00000000
   0x30    9    0x00  0xfffe  0x0034  0x111f.4d598a9a  0x00400188  0x0000.000.00000000  0x00000001    0x00000000
   0x31    9    0x00  0xfffe  0x0007  0x111f.4d598a8f  0x00400187  0x0000.000.00000000  0x00000001    0x00000000
   0x32    9    0x00  0xfffe  0x001e  0x111f.4d598ab9  0x00400188  0x0000.000.00000000  0x00000001    0x00000000
   0x33    9    0x00  0xfffe  0x002a  0x111f.4d598a97  0x00400187  0x0000.000.00000000  0x00000001    0x00000000
   0x34    9    0x00  0xfffe  0x0029  0x111f.4d598a9c  0x00400188  0x0000.000.00000000  0x00000001    0x00000000
   0x35    9    0x00  0xfffd  0x005f  0x111f.4d598a5b  0x00400186  0x0000.000.00000000  0x00000001    0x00000000
   0x36    9    0x00  0xfffe  0x0049  0x111f.4d598ac6  0x00400189  0x0000.000.00000000  0x00000001    0x00000000
   0x37    9    0x00  0xfffe  0x0016  0x111f.4d598aa3  0x00400188  0x0000.000.00000000  0x00000001    0x00000000
   0x38    9    0x00  0xfffe  0x0022  0x111f.4d598ab3  0x00400188  0x0000.000.00000000  0x00000001    0x00000000
   0x39    9    0x00  0xfffd  0x0058  0x111f.4d598a52  0x00400186  0x0000.000.00000000  0x00000001    0x00000000
   0x3a    9    0x00  0xfffd  0x0048  0x111f.4d598a57  0x00400186  0x0000.000.00000000  0x00000001    0x00000000
   0x3b    9    0x00  0xfffe  0x002b  0x111f.4d598aa5  0x00400188  0x0000.000.00000000  0x00000001    0x00000000
   0x3c    9    0x00  0xfffe  0x0000  0x111f.4d598aac  0x00400188  0x0000.000.00000000  0x00000001    0x00000000
   0x3d    9    0x00  0xfffd  0x0043  0x111f.4d598a4f  0x00400185  0x0000.000.00000000  0x00000001    0x00000000
   0x3e    9    0x00  0xfffe  0x003f  0x111f.4d598ab6  0x00400188  0x0000.000.00000000  0x00000001    0x00000000
   0x3f    9    0x00  0xfffe  0x0032  0x111f.4d598ab7  0x00400188  0x0000.000.00000000  0x00000001    0x00000000
   0x40    9    0x00  0xfffe  0x005a  0x111f.4d598aa8  0x00400188  0x0000.000.00000000  0x00000001    0x00000000
   0x41    9    0x00  0xfffe  0x004b  0x111f.4d598abc  0x00400188  0x0000.000.00000000  0x00000001    0x00000000
   0x42    9    0x00  0xfffd  0x0017  0x111f.4d598a6f  0x00400186  0x0000.000.00000000  0x00000001    0x00000000
   0x43    9    0x00  0xfffd  0x004d  0x111f.4d598a50  0x00400185  0x0000.000.00000000  0x00000001    0x00000000
   0x44    9    0x00  0xfffe  0x005e  0x111f.4d598ac2  0x00400189  0x0000.000.00000000  0x00000001    0x00000000
   0x45    9    0x00  0xfffe  0x0041  0x111f.4d598abb  0x00400188  0x0000.000.00000000  0x00000001    0x00000000
   0x46    9    0x00  0xfffe  0x002f  0x111f.4d598aa0  0x00400188  0x0000.000.00000000  0x00000001    0x00000000
   0x47    9    0x00  0xfffd  0x0059  0x111f.4d598a6b  0x00400186  0x0000.000.00000000  0x00000001    0x00000000
   0x48    9    0x00  0xfffd  0x0052  0x111f.4d598a58  0x00400186  0x0000.000.00000000  0x00000001    0x00000000
   0x49    9    0x00  0xfffe  0xffff  0x111f.4d598ac7  0x00400189  0x0000.000.00000000  0x00000001    0x00000000
   0x4a    9    0x00  0xfffd  0x003a  0x111f.4d598a56  0x00400186  0x0000.000.00000000  0x00000001    0x00000000
   0x4b    9    0x00  0xfffe  0x002e  0x111f.4d598abe  0x00400188  0x0000.000.00000000  0x00000001    0x00000000
   0x4c    9    0x00  0xfffd  0x0024  0x111f.4d598a5d  0x00400186  0x0000.000.00000000  0x00000001    0x00000000
   0x4d    9    0x00  0xfffd  0x0039  0x111f.4d598a51  0x00400185  0x0000.000.00000000  0x00000001    0x00000000
   0x4e    9    0x00  0xfffd  0x0001  0x111f.4d598a62  0x00400186  0x0000.000.00000000  0x00000001    0x00000000
   0x4f    9    0x00  0xfffd  0x000e  0x111f.4d598a66  0x00400186  0x0000.000.00000000  0x00000001    0x00000000
   0x50    9    0x00  0xfffd  0x000f  0x111f.4d598a69  0x00400186  0x0000.000.00000000  0x00000001    0x00000000
   0x51    9    0x00  0xfffe  0x0036  0x111f.4d598ac5  0x00400189  0x0000.000.00000000  0x00000001    0x00000000
   0x52    9    0x00  0xfffd  0x0035  0x111f.4d598a59  0x00400186  0x0000.000.00000000  0x00000001    0x00000000
   0x53    9    0x00  0xfffd  0x004a  0x111f.4d598a55  0x00400186  0x0000.000.00000000  0x00000001    0x00000000
   0x54    9    0x00  0xfffd  0x0055  0x111f.4d598a64  0x00400186  0x0000.000.00000000  0x00000001    0x00000000
   0x55    9    0x00  0xfffd  0x004f  0x111f.4d598a65  0x00400186  0x0000.000.00000000  0x00000001    0x00000000
   0x56    9    0x00  0xfffe  0x0044  0x111f.4d598ac1  0x00400189  0x0000.000.00000000  0x00000001    0x00000000
   0x57    9    0x00  0xfffd  0x004e  0x111f.4d598a60  0x00400186  0x0000.000.00000000  0x00000001    0x00000000
   0x58    9    0x00  0xfffd  0x0053  0x111f.4d598a53  0x00400186  0x0000.000.00000000  0x00000001    0x00000000
   0x59    9    0x00  0xfffd  0x0009  0x111f.4d598a6c  0x00400186  0x0000.000.00000000  0x00000001    0x00000000
   0x5a    9    0x00  0xfffd  0x002d  0x111f.4d598aa9  0x00400188  0x0000.000.00000000  0x00000001    0x00000000
   0x5b    9    0x00  0xfffd  0x000c  0x111f.4d598a71  0x00400186  0x0000.000.00000000  0x00000001    0x00000000
   0x5c    9    0x00  0xfffd  0x001d  0x111f.4d598a7f  0x00400187  0x0000.000.00000000  0x00000001    0x00000000
   0x5d    9    0x00  0xfffd  0x0018  0x111f.4d598a88  0x00400187  0x0000.000.00000000  0x00000001    0x00000000
   0x5e    9    0x00  0xfffe  0x0051  0x111f.4d598ac3  0x00400189  0x0000.000.00000000  0x00000001    0x00000000
   0x5f    9    0x00  0xfffd  0x004c  0x111f.4d598a5c  0x00400186  0x0000.000.00000000  0x00000001    0x00000000
   0x60    9    0x00  0xfffd  0x0057  0x111f.4d598a5f  0x00400186  0x0000.000.00000000  0x00000001    0x00000000
   0x61    9    0x00  0xfffd  0x0006  0x111f.4d598a75  0x00400187  0x0000.000.00000000  0x00000001    0x00000000

通过bbed修改相关值,规避掉此类问题

m /x 6c000000 offset 6513
m /x 7c000000 offset 6752
m /x 8c000000 offset 7048

启动数据库成功

Exception [type: SIGSEGV, Address not mapped to object] [] [ kgegpa()+36]

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：Exception [type: SIGSEGV, Address not mapped to object] [] [ kgegpa()+36]

有朋友找我，由于客户硬件故障,导致控制文件损坏，重建控制文件的时候,由于失误依赖了100多个数据文件,并且进行了resetlogs操作,然后通过bbed修改文件头,再进行恢复,数据库在open过程中报ORA-03113错误

SQL> recover database;
Media recovery complete.
SQL> alter database Open ;
ERROR:
ORA-03113: end-of-file on communication channel
Process ID: 11207284
Session ID: 4537 Serial number: 1

对应的alert日志信息

Successful open of redo thread 1
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Sat Dec 04 23:17:02 2021
SMON: enabling cache recovery
ARC3: Archival started
ARC0: STARTING ARCH PROCESSES COMPLETE
Sat Dec 04 23:17:03 2021
Exception [type: SIGSEGV, Address not mapped to object][ADDR:0x1604971A0][PC:0x1000E1764,kgegpa()+36][flags:0x0,count:1]
Exception [type: SIGSEGV, Address not mapped to object][ADDR:0x1604971A0][PC:0x1000E1764,kgegpa()+36][flags:0x2,count:2]
Exception [type: SIGSEGV, Address not mapped to object][ADDR:0x1604971A0][PC:0x1000E1764,kgegpa()+36][flags:0x2,count:2]
Sat Dec 04 23:17:07 2021
PMON (ospid: 10879866): terminating the instance due to error 397
Sat Dec 04 23:17:08 2021
System state dump requested by (instance=1, osid=10879866 (PMON)), summary=[abnormal instance termination].
System State dumped to trace file /oracle/app/oracle/diag/rdbms/xff/xff1/trace/xff1_diag_11076320.trc
Dumping diagnostic data in directory=[cdmp_20211204231708], requested by (instance=1, osid=10879866 (PMON)), 
summary=[abnormal instance termination].
Sat Dec 04 23:17:09 2021
ORA-1092 : opitsk aborting process

通过对其启动过程进行跟踪确认在update undo$ 语句中异常

PARSING IN CURSOR #11529215044940435280 len=148 dep=1 uid=0 oct=6 lid=0 tim=223080942765 
hv=3540833987 ad='c000000d67a42778' sqlid='5ansr7r9htpq3'
update undo$ set name=:2,file#=:3,block#=:4,status$=:5,user#=:6,undosqn=:7,xactsqn=:8,
scnbas=:9,scnwrp=:10,inst#=:11,ts#=:12,spare1=:13 where us#=:1
END OF STMT
PARSE #11529215044940435280:c=10000,e=8182,p=6,cr=55,cu=0,mis=1,r=0,dep=1,og=4,plh=0,tim=223080942764
BINDS #11529215044940435280:
 Bind#0
  oacdty=01 mxl=32(20) mxlc=00 mal=00 scl=00 pre=00
  oacflg=18 fl2=0001 frm=01 csi=873 siz=32 off=0
  kxsbbbfp=c000000d5fd299aa  bln=32  avl=20  flg=09
  value="_SYSSMU29_3935275865$"
 Bind#1
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=9fffffffbcc6e078  bln=24  avl=02  flg=05
  value=3
 Bind#2
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=9fffffffbcc6e048  bln=24  avl=03  flg=05
  value=128
 Bind#3
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=9fffffffbcc6e010  bln=24  avl=02  flg=05
  value=5
 Bind#4
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=9fffffffbcc6dfe0  bln=24  avl=02  flg=05
  value=1
 Bind#5
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=9fffffffbcc6dfb0  bln=24  avl=04  flg=05
  value=28921
 Bind#6
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=9fffffffbcc6df80  bln=24  avl=05  flg=05
  value=1245262
 Bind#7
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=9fffffffbcc6df48  bln=24  avl=06  flg=05
  value=1217986655
 Bind#8
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=9fffffffbcc6dc90  bln=24  avl=03  flg=05
  value=3621
 Bind#9
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=9fffffffbcc6dc60  bln=24  avl=01  flg=05
  value=0
 Bind#10
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=9fffffffbcc6dc30  bln=24  avl=02  flg=05
  value=2
 Bind#11
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=9fffffffbcc6dc00  bln=24  avl=02  flg=05
  value=2
 Bind#12
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=9fffffffbcc6e0a8  bln=22  avl=02  flg=05
  value=1
WAIT #4574659112: nam='db file sequential read' ela= 6835 file#=1 block#=842169 blocks=1 obj#=0 tim=90821816220340

通过分析,确认是由于undo事务异常,通过一系列处理之后,数据库启动报

SQL> alter database open;
alter database open
*
ERROR at line 1:
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-00600: internal error code, arguments: [4193], [], [], [], [], [], [], [], [], [], [], []
Process ID: 11206912
Session ID: 4537 Serial number: 1

对应alert日志报错

Successful open of redo thread 1
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Sun Dec 05 02:42:02 2021
SMON: enabling cache recovery
SUCCESS: diskgroup FRA was mounted
Archived Log entry 1 added for thread 1 sequence 1 ID 0x14ba19ad dest 1:
Sun Dec 05 02:42:03 2021
NOTE: dependency between database xff and diskgroup resource ora.FRA.dg is established
Errors in file /oracle/app/oracle/diag/rdbms/xff/xff1/trace/xff1_ora_11206912.trc  (incident=5681243):
ORA-00600: internal error code, arguments: [4193], [], [], [], [], [], [], [], [], [], [], []
Incident details in: /oracle/app/oracle/diag/rdbms/xff/xff1/incident/incdir_5681243/xff1_ora_11206912_i5681243.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
ARC3: Archival started
ARC0: STARTING ARCH PROCESSES COMPLETE
Sun Dec 05 02:42:05 2021
Dumping diagnostic data in directory=[cdmp_20211205024205], requested by (instance=1, osid=11206912), 
summary=[incident=5681243].
Sun Dec 05 02:42:05 2021
Block recovery from logseq 2, block 3 to scn 13867049493730
Recovery of Online Redo Log: Thread 1 Group 2 Seq 2 Reading mem 0
  Mem# 0: +DATA/xff/onlinelog/group_2.log
Block recovery stopped at EOT rba 2.5.16
Block recovery completed at rba 2.5.16, scn 3228.2895062240
Block recovery from logseq 2, block 3 to scn 13867049493727
Recovery of Online Redo Log: Thread 1 Group 2 Seq 2 Reading mem 0
  Mem# 0: +DATA/xff/onlinelog/group_2.log
Block recovery completed at rba 2.5.16, scn 3228.2895062240
Errors in file /oracle/app/oracle/diag/rdbms/xff/xff1/trace/xff1_ora_11206912.trc:
ORA-00600: internal error code, arguments: [4193], [], [], [], [], [], [], [], [], [], [], []
Errors in file /oracle/app/oracle/diag/rdbms/xff/xff1/trace/xff1_ora_11206912.trc:
ORA-00600: internal error code, arguments: [4193], [], [], [], [], [], [], [], [], [], [], []
Error 600 happened during db open, shutting down database
USER (ospid: 11206912): terminating the instance due to error 600
Instance terminated by USER, pid = 11206912
ORA-1092 signalled during: alter database open...
opiodr aborting process unknown ospid (11206912) as a result of ORA-1092
Sun Dec 05 02:42:07 2021
ORA-1092 : opitsk aborting process

trace中关键信息和bbed对看到结果对应,通过bbed清空,数据库启动正常

Version: 0x01
  FREE BLOCK POOL::
    uba: 0x004cd9b9.02fb.0c ext: 0x19 spc: 0x1462
    uba: 0x00000000.02fb.02 ext: 0x19 spc: 0x1f06
    uba: 0x00000000.02fb.0d ext: 0x19 spc: 0x1970
    uba: 0x00000000.02fa.03 ext: 0x18 spc: 0x1e34
    uba: 0x00000000.02fa.03 ext: 0x18 spc: 0x1e34

   struct ktuxcfbp[0], 12 bytes             @4192    
      struct ktufbuba, 8 bytes              @4192    
         ub4 kubadba                        @4192     0x004cd9b9
         ub2 kubaseq                        @4196     0x02fb
         ub1 kubarec                        @4198     0x0c
      sb2 ktufbext                          @4200     25
      sb2 ktufbspc                          @4202     5218

以前有过类似恢复硬件恢复之后,数据库无法open故障恢复

硬件恢复之后,数据库无法open故障恢复

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：硬件恢复之后,数据库无法open故障恢复

由于硬件故障,客户恢复硬件之后,数据库无法正常启动,报ORA-00354 ORA-00353错误

/tmp/> sqlplus / as sysdba

SQL*Plus: Release 11.2.0.3.0 Production on Mon Mar 1 17:10:30 2021

Copyright (c) 1982, 2011, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> recover database;

ORA-00283: recovery session canceled due to errors
ORA-00354: corrupt redo log block header
ORA-00353: log corruption near block 86088 change 16135545783340 time
02/23/2021 13:53:24
ORA-00312: online log 2 thread 1: '/oradata02/redo02b.log'
ORA-00312: online log 2 thread 1: '/oradata01/redo02a.log'

由于redo损坏,数据库无法继续正常恢复,通过屏蔽一致性,force open库

SQL> alter database open resetlogs;
alter database open resetlogs 
*
ERROR at line 1:
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-00704: bootstrap process failure
ORA-00704: bootstrap process failure
ORA-00600: internal error code, arguments: [2662], [3756], [3571444619], [3756], [3648471803], [4194545]
Process ID: 5104
Session ID: 576 Serial number: 3

这个错误比较简单,是由于scn问题导致,修改数据库scn启动库

SQL> alter database open;
alter database open
*
ERROR at line 1:
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-00600: internal error code, arguments: [4193], [], [], [], [], [], []
Process ID: 5536
Session ID: 576 Serial number: 1

这个错误比较明显，修改回滚段，尝试启动库

SQL> alter database open;
alter database open
*
ERROR at line 1:
ORA-03113: end-of-file on communication channel
Process ID: 6033
Session ID: 576 Serial number: 3

数据库依旧无法正常open,alert日志报错如下

ARC3 started with pid=30, OS id=6078 
ARC1: Archival started
ARC2: Archival started
ARC1: Becoming the 'no FAL' ARCH
ARC1: Becoming the 'no SRL' ARCH
ARC2: Becoming the heartbeat ARCH
Exception[type:SIGSEGV Address not mapped to object][ADDR:0x60173487F5][PC:0xC003B1C20,_memcpy()+64][flags:0x0,count:1]
Exception[type:SIGSEGV,Address not mapped to object][ADDR:0x60173487F5][PC:0xC003B1C20,_memcpy()+64][flags:0x2,count:2]
Exception[type:SIGSEGV,Address not mapped to object][ADDR:0x60173487F5][PC:0xC003B1C20,_memcpy()+64][flags:0x2,count:2]
Archived Log entry 2 added for thread 1 sequence 2 ID 0x506cafbb dest 1:
ARC3: Archival started
ARC0: STARTING ARCH PROCESSES COMPLETE
Mon Mar 01 17:44:44 2021
PMON (ospid: 5993): terminating the instance due to error 397
Mon Mar 01 17:44:45 2021
System state dump requested by (instance=1, osid=5993 (PMON)), summary=[abnormal instance termination].
System State dumped to trace file /oracle/diag/rdbms/xff/xff/trace/xff_diag_6001.trc
Instance terminated by PMON, pid = 5993

通过其启动过程分析,发现数据库卡在如下对象：

PARSING IN CURSOR #11529215044940435280 len=148 dep=1 uid=0 oct=6 lid=0 tim=223080942765 
hv=3540833987 ad='c000000d67a42778' sqlid='5ansr7r9htpq3'
update undo$ set name=:2,file#=:3,block#=:4,status$=:5,user#=:6,undosqn=:7,xactsqn=:8,
scnbas=:9,scnwrp=:10,inst#=:11,ts#=:12,spare1=:13 where us#=:1
END OF STMT
PARSE #11529215044940435280:c=10000,e=8182,p=6,cr=55,cu=0,mis=1,r=0,dep=1,og=4,plh=0,tim=223080942764
BINDS #11529215044940435280:
 Bind#0
  oacdty=01 mxl=32(20) mxlc=00 mal=00 scl=00 pre=00
  oacflg=18 fl2=0001 frm=01 csi=873 siz=32 off=0
  kxsbbbfp=c000000d5fd299aa  bln=32  avl=20  flg=09
  value="_SYSSMU1_3935275865$"
 Bind#1
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=9fffffffbcc6e078  bln=24  avl=02  flg=05
  value=3
 Bind#2
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=9fffffffbcc6e048  bln=24  avl=03  flg=05
  value=128
 Bind#3
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=9fffffffbcc6e010  bln=24  avl=02  flg=05
  value=5
 Bind#4
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=9fffffffbcc6dfe0  bln=24  avl=02  flg=05
  value=1
 Bind#5
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=9fffffffbcc6dfb0  bln=24  avl=04  flg=05
  value=28921
 Bind#6
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=9fffffffbcc6df80  bln=24  avl=05  flg=05
  value=1245262
 Bind#7
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=9fffffffbcc6df48  bln=24  avl=06  flg=05
  value=1217986655
 Bind#8
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=9fffffffbcc6dc90  bln=24  avl=03  flg=05
  value=3621
 Bind#9
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=9fffffffbcc6dc60  bln=24  avl=01  flg=05
  value=0
 Bind#10
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=9fffffffbcc6dc30  bln=24  avl=02  flg=05
  value=2
 Bind#11
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=9fffffffbcc6dc00  bln=24  avl=02  flg=05
  value=2
 Bind#12
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=9fffffffbcc6e0a8  bln=22  avl=02  flg=05
  value=1
WAIT #11529215044940435280: nam='db file sequential read' ela= 21 file#=1 block#=530 blocks=1 obj#=0 tim=223080944352
Incident 528204 created, dump file: /oracle/diag/rdbms/xff/xff/incident/incdir_528204/xff_ora_6593_i528204.trc
ORA-00600: internal error code, arguments: [4193], [], [], [], [], [], [], [], [], [], [], []

至此基本上可以确认是由于出现回滚段异常,继续查看日志发现

Error 600 in redo application callback
Dump of change vector:
TYP:0 CLS:16 AFN:1 DBA:0x00400212 OBJ:4294967295 SCN:0x0ea6.f4f2da14 SEQ:1 OP:5.1 ENC:0 RBL:0
ktudb redo: siz: 320 spc: 5892 flg: 0x0012 seq: 0x0072 rec: 0x08
            xid:  0x0000.004.000000bc  
ktubl redo: slt: 4 rci: 0 opc: 11.1 [objn: 15 objd: 15 tsn: 0]
Undo type:  Regular undo        Begin trans    Last buffer split:  No 
Temp Object:  No 
Tablespace Undo:  No 
             0x00000000  prev ctl uba: 0x00400212.0072.07 
prev ctl max cmt scn:  0x0eac.d42963be  prev tx cmt scn:  0x0eac.d4296f48 
txn start scn:  0xffff.ffffffff  logon user: 0  prev brb: 4194446  prev bcl: 0 BuExt idx: 0 flg2: 0
KDO undo record:
KTB Redo 
op: 0x04  ver: 0x01  
compat bit: 4 (post-11) padding: 1
op: L  itl: xid:  0x0000.060.000000bb uba: 0x00400212.0072.04
                      flg: C---    lkc:  0     scn: 0x0eac.d9736b46
KDO Op code: URP row dependencies Disabled
  xtype: XA flags: 0x00000000  bdba: 0x004000e1  hdba: 0x004000e0
itli: 4  ispac: 0  maxfr: 4863
tabn: 0 slot: 1(0x1) flag: 0x2c lock: 0 ckix: 0
ncol: 17 nnew: 12 size: 0
col  1: [20]  5f 53 59 53 53 4d 55 31 5f 33 39 33 35 32 37 35 38 36 35 24
col  2: [ 2]  c1 02
col  3: [ 2]  c1 04
col  4: [ 3]  c2 02 1d
col  5: [ 6]  c5 0d 12 63 43 38
col  6: [ 3]  c2 25 16
col  7: [ 5]  c4 02 19 35 3f
col  8: [ 4]  c3 03 5a 16
col  9: [ 1]  80
col 10: [ 2]  c1 04
col 11: [ 2]  c1 03
col 16: [ 2]  c1 03
Block after image is corrupt: 
buffer tsn: 0 rdba: 0x00400212 (1/530)
scn: 0x0ea6.f4f2da14 seq: 0x01 flg: 0x04 tail: 0xda140201
frmt: 0x02 chkval: 0x9dd8 type: 0x02=KTU UNDO BLOCK

使用bbed对file 1 block 530进行处理

   struct ktuxcscn, 8 bytes                 @4148    
      ub4 kscnbas                           @4148     0xd42963be
      ub2 kscnwrp                           @4152     0x0eac
   struct ktuxcuba, 8 bytes                 @4156    
      ub4 kubadba                           @4156     0x00400212
      ub2 kubaseq                           @4160     0x0072
      ub1 kubarec                           @4162     0x07
   sb2 ktuxcflg                             @4164     1 (KTUXCFSK)
   ub2 ktuxcseq                             @4166     0x0072
   sb2 ktuxcnfb                             @4168     1
   ub4 ktuxcinc                             @4172     0x00000000
   sb2 ktuxcchd                             @4176     4
   sb2 ktuxcctl                             @4178     3
   ub2 ktuxcmgc                             @4180     0x8002
   ub4 ktuxcopt                             @4188     0x7ffffffe

数据库顺利open成功

后续建议客户逻辑导出数据,导入到新库

分类目录归档：update undo$