Database SOS – Professional Database Recovery Support@+8617813235971

记录一次云存储资源到期之后引起的数据库恢复

客户环境是运行在xx厂商上的云环境的Oracle RAC,由于其中一块asm disk资源到期没有进行续费,云厂商对该盘进行了限速(1MB/s,并且异步写入),导致运行在该rac上的数据库直接crash,客户续费之后磁盘速度恢复正常但是数据库启动报ORA-00353 ORA-00312错误(redo异常),客户找了以前的服务商进行恢复尝试没有成功.对接手后的现场进行检测,发现有20个数据文件被初始化(dbv检测全部是空块,没有写入任何表和索引数据)[直接变成了空文件,一般sqlplus中执行alter database create datafile或者在没有对应的数据文件备份的情况下执行restore datafile会出现这个情况],数据库被强制拉库,但是启动没有成功报ORA-00704 ORA-01555错误.由于客户明天领导会过来检查,所以着急启动系统,我们只能基于现在的现况给客户强制拉起来库,赌运气看看核心业务是否可以跑起来

回溯处理过程,续费欠费资源之后,数据库启动报错(估计是当时限速之后,云存储层面异步写入导致部分写丢失)

Mon Oct 26 11:38:19 2026
alter database open
This instance was first to open
Beginning crash recovery of 1 threads
 parallel recovery started with 15 processes
Started redo scan
Incomplete read from log member '+DATA/xff/onlinelog/group_3.312.1210628353'. Trying next member.
Incomplete read from log member '+DATA/xff/onlinelog/group_3.312.1210628353'. Trying next member.
Errors in file /oracle/diag/rdbms/xff/xff1/trace/xff1_ora_19144.trc  (incident=682004):
ORA-00353: log corruption near block 47146 change 16921129940671 time 10/26/2026 08:57:37
ORA-00312: online log 3 thread 1: '+DATA/xff/onlinelog/group_3.312.1210628353'
Incident details in: /oracle/diag/rdbms/xff/xff1/incident/incdir_682004/xff1_ora_19144_i682004.trc
Abort recovery for domain 0
Aborting crash recovery due to error 368
Errors in file /oracle/diag/rdbms/xff/xff1/trace/xff1_ora_19144.trc:
ORA-00368: checksum error in redo log block
ORA-00353: log corruption near block 47146 change 16921129940671 time 10/26/2026 08:57:37
ORA-00312: online log 3 thread 1: '+DATA/xff/onlinelog/group_3.312.1210628353'
Abort recovery for domain 0
Mon Oct 26 11:38:22 2026
Sweep [inc][682004]: completed
Errors in file /oracle/diag/rdbms/xff/xff1/trace/xff1_ora_19144.trc:
ORA-00368: checksum error in redo log block
ORA-00353: log corruption near block 47146 change 16921129940671 time 10/26/2026 08:57:37
ORA-00312: online log 3 thread 1: '+DATA/xff/onlinelog/group_3.312.1210628353'
ORA-368 signalled during: alter database open...

屏蔽一致性,强制resetlogs的时候报ORA-00704 ORA-01555错误

Mon Oct 26 15:47:43 2026
Setting recovery target incarnation to 2
Initializing SCN for created control file
Database SCN compatibility initialized to 3
Warning - High Database SCN: Current SCN value is 16917769616373, threshold SCN value is 0
Mon Oct 26 15:47:43 2026
Assigning activation ID 2556751686 (0x9864ef46)
LGWR: STARTING ARCH PROCESSES
Mon Oct 26 15:47:43 2026
ARC0 started with pid=32, OS id=14392 
ARC0: Archival started
LGWR: STARTING ARCH PROCESSES COMPLETE
ARC0: STARTING ARCH PROCESSES
Mon Oct 26 15:47:44 2026
ARC1 started with pid=33, OS id=14446 
Mon Oct 26 15:47:44 2026
ARC2 started with pid=34, OS id=14448 
Mon Oct 26 15:47:44 2026
ARC3 started with pid=35, OS id=14450 
ARC1: Archival started
ARC2: Archival started
ARC1: Becoming the 'no FAL' ARCH
ARC1: Becoming the 'no SRL' ARCH
ARC2: Becoming the heartbeat ARCH
Thread 1 opened at log sequence 1
  Current log# 1 seq# 1 mem# 0: +DATA/xff/onlinelog/group_1.314.1210628351
Successful open of redo thread 1
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Mon Oct 26 15:47:44 2026
SMON: enabling cache recovery
ORA-01555 caused by SQL statement below (SQL ID: 7bd391hat42zk, SCN: 0x0f62.f9a5fbf8):
select /*+ rule */ name,file#,block#,status$,user#,undosqn,xactsqn,scnbas,scnwrp,
DECODE(inst#,0,NULL,inst#),ts#,spare1 from undo$ where us#=:1
Errors in file /oracle/diag/rdbms/xff/xff1/trace/xff1_ora_13439.trc:
ORA-00704: bootstrap process failure
ORA-00704: bootstrap process failure
ORA-00604: error occurred at recursive SQL level 2
ORA-01555: snapshot too old: rollback segment number 0 with name "SYSTEM" too small
Errors in file /oracle/diag/rdbms/xff/xff1/trace/xff1_ora_13439.trc:
ORA-00704: bootstrap process failure
ORA-00704: bootstrap process failure
ORA-00604: error occurred at recursive SQL level 2
ORA-01555: snapshot too old: rollback segment number 0 with name "SYSTEM" too small
Error 704 happened during db open, shutting down database
USER (ospid: 13439): terminating the instance due to error 704
Instance terminated by USER, pid = 13439
ORA-1092 signalled during: ALTER DATABASE OPEN RESETLOGS...
opiodr aborting process unknown ospid (13439) as a result of ORA-1092

从这个时候开始我接手这个故障,数据完全恢复已经不太可能,只能基于现在的情况,给客户尝试拉起来库,看看核心业务是否可以跑起来.
数据库open过程报ORA-01555错误,这个是一个比较常见的问题,以前专门写过类似文章:
在数据库open过程中常遇到ORA-01555汇总
 数据库open过程遭遇ORA-1555对应sql语句补充
对于这个客户问题,我也一样尝试修改scn的方式来解决

SQL> set numw 16
col CHECKPOINT_TIME for a40
set lines 150
set pages 1000
SELECT status,
to_char(checkpoint_time,'yyyy-mm-dd hh24:mi:ss') checkpoint_time,FUZZY,checkpoint_change#,
count(*) ROW_NUM
FROM v$datafile_header
GROUP BY status, checkpoint_change#, to_char(checkpoint_time,'yyyy-mm-dd hh24:mi:ss'),fuzzy
ORDER BY status, checkpoint_change#, checkpoint_time;SQL> SQL> SQL> SQL>   2    3    4    5    6  

STATUS  CHECKPOINT_TIME                          FUZ CHECKPOINT_CHANGE#          ROW_NUM
------- ---------------------------------------- --- ------------------ ----------------
ONLINE  2026-10-26 15:47:44                      YES     16917769616374              191

使用obet的patch_scn功能来修改数据库scn（obet的patch_scn功能）

OBET> patch_scn 89103 0x06001AE70 0xF62FBA5FBF6
 
=== Oracle SCN Patch ===
PID:     89103
Address: 0x06001AE70
New Value: 0xF62FBA5FBF6 (16917803170806)
Original SCN: 0x0
 
Are you sure you want to modify Oracle SCN? (yes/no): y
 
SCN modified: 0x0 -> 0xF62FBA5FBF6
Oracle SCN successfully modified.


SQL> oradebug DUMPvar SGA kcsgscn_ 
kcslf kcsgscn_ [06001AE70, 06001AEA0) = FBA5FBF6 00000F62 00000000 00000000 00000000

然后尝试打开数据库依旧报ORA-01555错误,证明我们处理的scn不正确

SQL> recover database using backup controlfile;
ORA-00279: change 16917769616374 generated at 10/26/2026 15:47:44 needed for
thread 1
ORA-00289: suggestion : +ARCH
ORA-00280: change 16917769616374 for thread 1 is in sequence #1


Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
+DATA/xff/onlinelog/group_1.314.1210628351
Log applied.
Media recovery complete.
SQL> alter database open resetlogs;
alter database open resetlogs
*
ERROR at line 1:
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-00704: bootstrap process failure
ORA-00704: bootstrap process failure
ORA-00604: error occurred at recursive SQL level 2
ORA-01555: snapshot too old: rollback segment number 0 with name "SYSTEM" too small
Process ID: 18169
Session ID: 3681 Serial number: 2825

分析异常block的scn

*** 2026-10-27 01:05:42.156
Start dump data block from file +DATA/xff/system01.dbf minblk 558 maxblk 558
 V10 STYLE FILE HEADER:
        Compatibility Vsn = 186647552=0xb200400
        Db ID=2195614447=0x82de6aef, Db Name='xff'
        Activation ID=0=0x0
        Control Seq=542=0x21e, File size=446720=0x6d100
        File Number=1, Blksiz=8192, File Type=3 DATA
Dump all the blocks in range:
buffer tsn: 0 rdba: 0x0040022e (1/558)
scn: 0x0f63.c1efd2ab seq: 0x01 flg: 0x04 tail: 0xd2ab0201
frmt: 0x02 chkval: 0x75a3 type: 0x02=KTU UNDO BLOCK
Hex dump of block: st=0, typ_found=1

发现该block的scn非常大0x0f63.c1efd2ab,我之前处理的scn明显不合适,继续换上新的scn,再次打开数据库成功

OBET> patch_scn 89103 0x06001AE70 0xF64C1EFD2AB
 
=== Oracle SCN Patch ===
PID:     89103
Address: 0x06001AE70
New Value: 0xF64C1EFD2AB(16925424865963)
Original SCN: 0x0
 
Are you sure you want to modify Oracle SCN? (yes/no): y
 
SCN modified: 0x0 -> 0xF64C1EFD2AB
Oracle SCN successfully modified.


SQL> oradebug DUMPvar SGA kcsgscn_ 
kcslf kcsgscn_ [06001AE70, 06001AEA0) = C1EFD2AB 00000F64 00000000 00000000 00000000
SQL> 

SQL> 
SQL> 
SQL> alter database open;

Database altered.
[/sql]
后续大量异常undo问题
1
ORA-00376: file 182 cannot be read at this time
ORA-01111: name for data file 182 is unknown - rename to correct file
ORA-01110: data file 182: '/oracle/product/11.2.0/db_1/dbs/MISSING00182'
ORACLE Instance xff1 (pid = 20) - Error 1578 encountered while recovering transaction (21, 29) on object 99404.
Errors in file /oracle/diag/rdbms/xff/xff1/trace/xff1_smon_29435.trc:
ORA-01578: ORACLE data block corrupted (file # 8, block # 1471227)
ORA-01110: data file 8: '+DATA/xff/users04.dbf'
ORACLE Instance xff1 (pid = 20) - Error 376 encountered while recovering transaction (2, 27) on object 599926.
Errors in file /oracle/diag/rdbms/xff/xff1/trace/xff1_smon_29435.trc:
ORA-00376: file 211 cannot be read at this time
ORA-01111: name for data file 211 is unknown - rename to correct file
ORA-01110: data file 211: '/oracle/product/11.2.0/db_1/dbs/MISSING00211'
ORACLE Instance xff1 (pid = 20) - Error 376 encountered while recovering transaction (10, 6) on object 99591.
Errors in file /oracle/diag/rdbms/xff/xff1/trace/xff1_smon_29435.trc:
ORA-00376: file 182 cannot be read at this time
ORA-01111: name for data file 182 is unknown - rename to correct file
ORA-01110: data file 182: '/oracle/product/11.2.0/db_1/dbs/MISSING0018

由于这些文件已经被初始化掉了,恢复已经没有意义,直接没有加入到ctl中,现在主要是由于有些事务涉及这些文件,导致smon回滚的时候报这些错误,对于这样的异常,直接对于undo进行重建更换新undo即可.处理好这些问题之后,数据库open运行问题,客户自行安排业务测试和论证

Firebird 损坏数据库数据抢救工具—firebirdext

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：Firebird 损坏数据库数据抢救工具—firebirdext

它是做什么的
firebirdext 是一个针对 Firebird 数据库 .fdb 文件的数据抢救工具。
当数据库文件因为断电、磁盘坏道、误操作、备份中断、病毒查杀等原因损坏，导致数据库无法打开、官方工具 gbak/gfix 也救不回来时，fbrescue 可以绕过数据库引擎，直接以二进制方式解析磁盘文件，把还能找回的数据一页一页”捞”出来——导出成 CSV、SQL 脚本，甚至直接重建一个可用的新数据库。
一句话：引擎打不开的库，它照样能把数据掏出来。

解决什么问题
你可能会遇到这些情况：

工具的信条是宁缺毋滥：损坏到无法确认的行，如实计入”失败”，绝不编造半真半假的数据出来。

核心功能
1. 全景扫描（scan） — 动手之前先看病情
输出每页的类型与归属（哪些表还剩多少数据页、哪些页被毁），立刻知道”还有多少可救”。

firebirdext scan 损坏库.FDB

2. 零配置数据提取 — 不用提供表结构
自动建库、建表、写入数据，产出一个可以正常连接使用的 Firebird 数据库。

firebirdext extract 损坏库.FDB --format db --target 新库.FDB

3. 直接重建新库 — 数据一步到位
自动建库、建表、写入数据，产出一个可以正常连接使用的 Firebird 数据库。

firebirdext extract 损坏库.FDB --format db --target 新库.FDB

4. 对象结构恢复 — 不只救数据，结构也不丢
恢复索引、主键、视图、存储过程、触发器、序列，在数据恢复好的新库上执行一遍，对象全部重建。

5. 单表定向抢救

firebirdext extract 损坏库.FDB --table ORDERS --format db --target 订单.FDB

6. 三种输出格式：csv（Excel 核对）/ sql（INSERT 脚本）/ db（新数据库）。

支持的数据类型
整数、BIGINT、SMALLINT、NUMERIC/DECIMAL(p,s)、FLOAT、DOUBLE、DATE、TIMESTAMP、VARCHAR/CHAR（多字节字符集支持）、BLOB（自动重组内联大对象，实测 4000 字符文本完整还原）

上手：三分钟走完全流程

# 环境：Windows/Linux 均可，Python 3.8+，无需安装任何第三方库
# （只有"直接建新库"功能会调用 Firebird 自带的 isql.exe）

# 第 1 步：复制副本（永远不要对原文件操作）
copy D:\prod\MYDATA.FDB E:\work\抢救副本.FDB

# 第 2 步：诊断，看看能救多少
firebirdext scan E:\work\抢救副本.FDB

# 第 3 步：抢救 → 新库 + 对象脚本，一条命令
firebirdext extract E:\work\抢救副本.FDB --format db --target E:\work\恢复库.FDB

# 第 4 步：把索引/视图/过程/触发器也建起来
isql -q -b -i E:\work\恢复库_ddl.sql

# 第 5 步：连上新库验证数据
isql -q
SQL> connect 'E:/work/恢复库.FDB' user 'SYSDBA' password '***';
SQL> select count(*) from 任意表;

适用范围与限制
1).适用 Firebird 3 / 4 / 5（ODS12/13）；更老的 2.5 及以前版本需要小幅适配
2).已被物理覆写清零的记录、已被垃圾回收的数据，任何工具都救不回
3).表结构（列类型）依赖文件内的系统表页；如果系统表页也烂光了，可以用 –ddl 手动提供建表语句兜底
4).view、index、trigger等如果字典完善可以恢复
5).外键、CHECK 约束暂无法恢复，后续进行完善

Oracle 19c 202610补丁（RUs+OJVM）-19.33

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：Oracle 19c 202610补丁（RUs+OJVM）-19.33

Release	Database Update	GI Update	Windows Bundle Patch
OCT2026 (19.33.0.0.0)	39915792	39922757	39898799
JUL2026 (19.32.0.0.0)	39472050	39467003	39418910
APR2026 (19.31.0.0.0)	39034528	39036936	38818049
JAN2026 (19.30.0.0.0)	38632161	38629535	38597735
OCT2025 (19.29.0.0.0)	38291812	38298204	38111211
JUL2025 (19.28.0.0.0)	37960098	37957391	37962957
APR2025 (19.27.0.0.0)	37642901	37641958	37532350
JAN2025 (19.26.0.0.0)	37260974	37257886	37486199
OCT2024 (19.25.0.0.0)	36912597	36916690	36878821
JUL2024 (19.24.0.0.0)	36582781	36582629	36521936
APR2024 (19.23.0.0.0)	36233263	36233126	36219938
JAN2024 (19.22.0.0.0)	35943157	35940989	35962832
OCT2023 (19.21.0.0.0)	35643107	35642822	35681552
JUL2023 (19.20.0.0.0)	35320081	35319490	35348034
APR2023 (19.19.0.0.0)	35042068	35037840	35046439
JAN2023 (19.18.0.0.0)	34765931	34762026	34750795
Oct2022 (19.17.0.0.0)	34419443	34416665	34468114
JUL2022 (19.16.0.0.0)	34133642	34130714	34110685
APR2022 (19.15.0.0.0)	33806152	33803476	33829175
JAN2022 (19.14.0.0.0)	33515361	33509923	33575656
OCT2021(19.13.0.0.0)	33192793	33182768	33155330
JUL2021 (19.12.0.0.0)	32904851	32895426	32832237
APR2021 (19.11.0.0.0)	32545013	32545008	32409154
JAN2021 (19.10.0.0.0)	32218454	32226239	32062765
OCT2020 (19.9.0.0.0)	31771877	31750108	31719903
JUL2020 (19.8.0.0.0)	31281355	31305339	31247621
APR2020 (19.7.0.0.0)	30869156	30899722	30901317
JAN2020 (19.6.0.0.0)	30557433	30501910	30445947
OCT2019 (19.5.0.0.0)	30125133	30116789	30151705
JUL2019 (19.4.0.0.0)	29834717	29708769	-
APR2019 (19.3.0.0.0)	29517242	29517302	-

Release	OJVM Update	OJVM + DB Update	OJVM + GI Update
OCT2026 (19.33.0.0.261020)	39803650	40121635	40121665
JUL2026 (19.32.0.0.260721)	39222882	39618649	39618711
APR2026 (19.31.0.0.260421)	38906621	39062931	39062956
JAN2026 (19.30.0.0.260120)	38523609	38658587	38658588
OCT2025 (19.29.0.0.251021)	38194382	38273545	38273558
JUL2025 (19.28.0.0.250715)	37847857	37952354	37952382
APR2025 (19.27.0.0.250415)	37499406	37591483	37591516
JAN2025 (19.26.0.0.250121)	37102264	37262172	37262208
OCT2024 (19.25.0.0.241015)	36878697	36866623	36866740
JUL2024 (19.24.0.0.240716)	36414915	36522340	36522439
APR2024 (19.23.0.0.240416)	36199232	36209492	36209493
JAN2024 (19.22.0.0.240116)	35926646	36031426	36031453
OCT2023 (19.21.0.0.231017)	35648110	35742413	35742441
JUL2023 (19.20.0.0.230718)	35354406	35370174	35370167
APR2023 (19.19.0.0.230418)	35050341	35058163	35058172
JAN2023 (19.18.0.0.230117)	34786990	34773489	34773504
OCT2022 (19.17.0.0.221018)	34411846	34449114	34449117
JUL2022 (19.16.0.0.220719)	34086870	34160831	34160854
APR2022 (19.15.0.0.220419)	33808367	33859194	33859214
JAN2022 (19.14.0.0.220118)	33561310	33567270	33567274
OCT2021 (19.13.0.0.211019)	33192694	33248420	33248471
JUL2021 (19.12.0.0.210720)	32876380	32900021	32900083
APR2021 (19.11.0.0.210420)	32399816	32578972	32578973
JAN2021 (19.10.0.0.210119)	32067171	32126828	32126842
OCT2020 (19.9.0.0.201020)	31668882	31720396	31720429
JUL2020 (19.8.0.0.200714)	31219897	31326362	31326369
APR2020 (19.7.0.0.200414)	30805684	30783543	30783556
JAN2020 (19.6.0.0.200114)	30484981	30463595	30463609
OCT2019 (19.5.0.0.191015)	30128191	30133124	30133178
JUL2019 (19.4.0.0.190716)	29774421	29699079	29699097
APR2019 (19.3.0.0.190416)	29548437	29621253	29621299

参考：Assistant: Download Reference for Oracle Database/GI Update, Revision, PSU, SPU(CPU), Bundle Patches, Patchsets and Base Releases KA958

由于con$.con#值达到极限，触发数据库增加约束报ORA-600 12807错误

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：由于con$.con#值达到极限，触发数据库增加约束报ORA-600 12807错误

有客户数据库报ORA-600 12807错误，导致数据库无法创建任何约束(带约束条件的表也无法创建)，alert 日志报错如下

2026-10-19T09:33:41.779082+08:00
Errors in file /oracle/database/diag/rdbms/oradb/oradb1/trace/oradb1_ora_184458.trc
ORA-00600: 内部错误代码, 参数: [12807], [], [], [], [], [], [], [], [], [], [], []
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
2026-10-19T09:33:42.080313+08:00
Errors in file /oracle/database/diag/rdbms/oradb/oradb1/trace/oradb1_ora_221081.trc
ORA-00600: 内部错误代码, 参数: [12807], [], [], [], [], [], [], [], [], [], [], []
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
2026-10-19T09:33:42.401635+08:00
DDE: Problem Key 'ORA 600 [12807]' was completely flood controlled (0x6)
Further messages for this problem key will be suppressed for up to 10 minutes

对应的trace文件报错

*** 2026-10-20T23:01:14.835258+08:00
dbkedDefDump(): Starting incident default dumps (flags=0x2, level=3, mask=0x0)
[TOC00004]
----- Current SQL Statement for this session (sql_id=czyqxdfu45w1c) -----
CREATE TABLE VT10RMC7BHYU76DDQ3DHV75RMV (KSQL_SEQ INT, FID VARCHAR2 (44) NOT NULL
…………………… 
FInPriceBizDate TIMESTAMP, CONSTRAINT ZC8SOAHBADTGB7OIBG PRIMARY KEY (FID)) NOLOGGING
[TOC00004-END]

[TOC00005]
----- Call Stack Trace -----
calling              call     entry                argument values in hex      
location             type     point                (? means dubious value)     
-------------------- -------- -------------------- ----------------------------
ksedst1()+95         call     kgdsdst()            7FFF09990520 000000002
                                                   7FFF0998A860 ? 7FFF0998A978 ?
                                                   000000000 000000082 ?
ksedst()+58          call     ksedst1()            000000000 000000001
                                                   7FFF0998A860 ? 7FFF0998A978 ?
                                                   000000000 ? 000000082 ?
dbkedDefDump()+2344  call     ksedst()             000000000 000000001 ?
8                                                  7FFF0998A860 ? 7FFF0998A978 ?
                                                   000000000 ? 000000082 ?
ksedmp()+577         call     dbkedDefDump()       000000003 000000002
                                                   7FFF0998A860 ? 7FFF0998A978 ?
                                                   000000000 ? 000000082 ?
dbgexPhaseII()+2092  call     ksedmp()             0000003EB 000000002 ?
                                                   7FFF0998A860 ? 7FFF0998A978 ?
                                                   000000000 ? 000000082 ?
dbgexExplicitEndInc  call     dbgexPhaseII()       7FBAB95976D8 7FBAB47D3148
()+285                                             7FFF09991700 7FFF0998A978 ?
                                                   000000000 ? 000000082 ?
dbgeEndDDEInvocatio  call     dbgexExplicitEndInc  7FBAB95976D8 7FBAB47D3148
nImpl()+314                   ()                   7FBAB47D39F0 7FFF0998A978 ?
                                                   000000000 ? 000000082 ?
kqrHashTableInsert(  call     dbgeEndDDEInvocatio  7FBAB95976D8 7FBAB47D3148
)+2446                        nImpl()              7FBAB95D79C0 7FFF0998A978 ?
                                                   7FBAB95D79C0 000000082 ?
kqrRelinkToHashTabl  call     kqrHashTableInsert(  000000000 ? 16F5F05F70 ?
e()+226                       )                    15B7ED3EC0 ? FFFFFFFFF6B64B6B
                                                   7FBAB95D79C0 ? 2400000000
kqrUpdateHashTable(  call     kqrRelinkToHashTabl  000000000 ? 16F5F05F70 ?
)+417                         e()                  15B7ED3EC0 ?
                                                   FFFFFFFFF6B64B6B ?
                                                   7FBAB95D79C0 ? 2400000000 ?
kqrfpo()+729         call     kqrUpdateHashTable(  7FBAB95D79C0 16F5F05F70 ?
                              )                    FFFFFFFFF5F05F70 00000000B
                                                   000000003 ? 2400000000 ?
kkdcnco()+1749       call     kqrfpo()             16F5F05F70 16F5F05F70 ?
                                                   FFFFFFFFF5F05F70 ?
                                                   00000000B ? 24886FB6D8
                                                   2400000000 ?
kkdcico()+339        call     kkdcnco()            000000057 7FBAB39D5420
                                                   7FFF09992B80 16167E3500
                                                   000000000 2400000000 ?
ctcdrv()+40399       call     kkdcico()            000000057 7FBAB39D53F8
                                                   7FBAB9478950 16167E3500
                                                   14BF81D188 2400000000 ?
opiexe()+25553       call     ctcdrv()             7FBAB9479388 7FFF099B5D60
                                                   7FFF099B5E50 16167E3500 ?
                                                   14BF81D188 ? 2400000000 ?
opiosq0()+4494       call     opiexe()             000000004 7FFF099B5D60 ?
                                                   7FFF099B64A0 16167E3500 ?
                                                   14BF81D188 ? 2400000000 ?
kpooprx()+287        call     opiosq0()            000000003 7FFF099B5D60 ?
                                                   7FBAB95D79C0 ? 0000000A4
                                                   000000000 000000001
kpoal8()+821         call     kpooprx()            7FFF099BA5C4 7FBAB3967B58
                                                   000000736 000000001 000000000
                                                   0000000A4
opiodr()+1202        call     kpoal8()             00000005E 000000026
                                                   7FFF099BA5C0 000000001 ?
                                                   000000000 ? 0000000A4 ?
ttcpip()+1218        call     opiodr()             00000005E 000000026
                                                   7FFF099BA5C0 ? 000000000
                                                   000000000 ? 0000000A4 ?
opitsk()+1900        call     ttcpip()             7FBAB9601B30 ? 000000026 ?
                                                   7FFF099BA5C0 000000000 ?
                                                   7FFF099BA020 7FFF099BA810 ?
opiino()+936         call     opitsk()             000000000 000000000
                                                   7FFF099BA5C0 ? 000000000 ?
                                                   7FFF099BA020 ? 7FFF099BA810 ?
opiodr()+1202        call     opiino()             00000003C 000000004
                                                   7FFF099BC198 000000000 ?
                                                   7FFF099BA020 ? 7FFF099BA810 ?
opidrv()+1094        call     opiodr()             00000003C 000000004
                                                   7FFF099BC198 ? 000000000
                                                   7FFF099BA020 ? 7FFF099BA810 ?
sou2o()+165          call     opidrv()             00000003C 000000004
                                                   7FFF099BC198 000000000 ?
                                                   7FFF099BA020 ? 7FFF099BA810 ?
opimai_real()+422    call     sou2o()              7FFF099BC131 00000003C
                                                   000000004 7FFF099BC198
                                                   7FFF099BA020 ? 7FFF099BA810 ?
ssthrdmain()+417     call     opimai_real()        000000000 7FFF099BC988
                                                   000000004 ? 7FFF099BC198 ?
                                                   7FFF099BA020 ? 7FFF099BA810 ?
main()+256           call     ssthrdmain()         000000000 000000002
                                                   7FFF099BC988 000000001
                                                   000000000 7FFF099BA810 ?
__libc_start_main()  call     main()               000000002 7FFF099BCBC8
+245                                               7FFF099BC988 ? 000000001 ?
                                                   000000000 ? 7FFF099BA810 ?
_start()+41          call     __libc_start_main()  000DCD110 000000002
                                                   7FFF099BCBC8 000000000 ?
                                                   000000000 ? 7FFF099BA810 ?
[TOC00005-END]

问题重现

[oracle@oradb1 ~]$ sqlplus / as sysdba

SQL*Plus: Relorae 19.0.0.0.0 - Production on Tue Oct 20 21:45:57 2026
Version 19.14.0.0.0

Copyright (c) 1982, 2021, Oracle.  All rights reserved.


Connected to:
Oracle Database 19c Enterprise Edition Relorae 19.0.0.0.0 - Production
Version 19.14.0.0.0

SQL> create table t_test1020(id number not null);
create table t_test1020(id number not null)
*
ERROR at line 1:
ORA-00600: internal error code, arguments: [12807], [], [], [], [], [], [], [],
[], [], [], []

分析故障原因(con$.con#=4294967294(_NEXT_CONSTRAINT)达到了理论最大值,无法继续增加所以报ORA-600 12807错误)

SQL> select 4294967294 - con# from sys.con$ where name='_NEXT_CONSTRAINT';

4294967294-CON#
---------------
              0

SQL> select max(con#) from sys.con$;

 MAX(CON#)
----------
4294967294

根据oracle官方描述打上25343563 patch可以实现con$.con#循环利用

[oracle@oradb1 25343563]$ opatch lspatches
25343563;MECHANISM TO REUSE CON#
33561310;OJVM RELoraE UPDATE: 19.14.0.0.220118 (33561310)
33529556;OCW RELoraE UPDATE 19.14.0.0.0 (33529556)
33515361;Database Relorae Update : 19.14.0.0.220118 (33515361)

OPatch succeeded.

测试问题解决

SQL> create table t_test1020(id number not null);

Table created.

SQL> 
SQL> select constraint_name
  2  from user_cons_columns
  3  where table_name = 'T_TEST1020'
  4    and column_name = 'ID';

CONSTRAINT_NAME
--------------------------------------------------------------------------------
SYS_C0012094

SQL> SELECT CON# FROM CON$ WHERE NAME='SYS_C0012094';

      CON#
----------
     12094

SQL> SELECT COUNT(1) FROM con$recycle;

  COUNT(1)
----------
 100032699

SQL> select  con# from sys.con$ where name='_NEXT_CONSTRAINT';

      CON#
----------
4294967294

分析实现原因，其实oracle是通过创建sys.con$recycle表，把使用过的记录存储到这个表里面(新删除的约束的con#也会自动增加到这个表里面)，后续使用直接从这个表中拿数据出来使用，从而实现con$中的con#循环使用，从而解决了ORA-600 12807问题,如果是12c之前版本参考:ORA-600 12807（CON$.CON#达到最大值）处理

MongoDB 离线提取工具—mongoext

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：MongoDB 离线提取工具—mongoext

在某些情况下,由于断电、硬件故障、软件bug等导致mongodb数据库无法正常启动,甚至使用repair依然无法启动,对于这样的情况.我专门开发了mongodb离线提取工具mongoext,在离线情况下直接提取mongodb的数据
整体命令支持

G:\mongodb\mongoext>mongoext.exe -h
usage: mongoext [-h] {scan,recover,info} ...

脱机 MongoDB 数据文件提取工具

positional arguments:
  {scan,recover,info}
    scan               扫描 dbPath 并导出
    recover            恢复孤立的单个 collection 文件（无 catalog）
    info               打印 dbPath 元信息

options:
  -h, --help           show this help message and exit

scan 是基于mongodb字典直接提取数据
recover是直接恢复单个集合文件中数据
info是获取字典信息以及集合对应关系

mongoext scan 命令介绍

G:\mongodb\mongoext>mongoext.exe scan -h
usage: mongoext scan [-h] [--db DB] [--collection COLLECTION] [--include-system] [--out OUT]
                     [--format {jsonl,sql,bson,mongo}] [--uri URI] [--batch BATCH] [--no-indexes] [--ttl-indexes]
                     [--keep-existing] [--sql-dialect {postgres,mysql}] [--dedup] [--no-progress] [--dry-run]
                     [--allocation-size ALLOCATION_SIZE]
                     dbpath

positional arguments:
  dbpath                dbPath 路径

options:
  -h, --help            show this help message and exit
  --db DB               只导出指定数据库
  --collection COLLECTION
                        只导出指定集合
  --include-system      连 admin/config/local 也一起导入（默认跳过，会污染目标实例）
  --out OUT             输出目录（--format mongo 时不需要）
  --format {jsonl,sql,bson,mongo}
                        输出格式；mongo = 直接写入运行中的 mongod（BSON 类型保真）
  --uri URI             --format mongo 的目标连接串
  --batch BATCH         --format mongo 每批 insert_many 的条数
  --no-indexes          --format mongo 导入后不重建索引
  --ttl-indexes         --format mongo 连 TTL 索引也重建（默认不建：建了会立刻按过期时间删数据）
  --keep-existing       --format mongo 不 drop 目标集合，追加写入
  --sql-dialect {postgres,mysql}
                        SQL 方言
  --dedup               按 _id 去重（多版本 leaf page 残留时开启）
  --no-progress         关闭 tqdm 进度条
  --dry-run             只统计不写盘，用于核对提取完整性与文档计数
  --allocation-size ALLOCATION_SIZE
                        手工指定块对齐粒度（默认从 turtle / 元数据读，读不到才穷举探测）

mongoext recover命令介绍

G:\mongodb\mongoext>mongoext.exe recover -h
usage: mongoext recover [-h] [--orphans DBPATH] [--db DB] [--collection COLLECTION] [--meta-dir META_DIR] [--out OUT]
                        [--format {jsonl,sql,bson,mongo}] [--uri URI] [--batch BATCH] [--no-indexes] [--ttl-indexes]
                        [--keep-existing] [--sql-dialect {postgres,mysql}] [--dedup] [--no-progress] [--dry-run]
                        [--allocation-size ALLOCATION_SIZE]
                        [wt_file]

positional arguments:
  wt_file               孤立的 collection-*.wt 文件

options:
  -h, --help            show this help message and exit
  --orphans DBPATH      批量恢复该 dbPath 里 catalog 未记录的 collection 文件
  --db DB               手工指定库名（与 --collection 一起用）
  --collection COLLECTION
                        手工指定集合名（与 --db 一起用）
  --meta-dir META_DIR   伴生文件（turtle / _mdb_catalog.wt / WiredTiger.wt）所在目录，用来反查 namespace；
                        文件已拷走但原 dbPath 还在时指过去即可
  --out OUT             输出目录（--format mongo 时不需要）
  --format {jsonl,sql,bson,mongo}
                        输出格式；mongo = 直接写入运行中的 mongod（BSON 类型保真）
  --uri URI             --format mongo 的目标连接串
  --batch BATCH         --format mongo 每批 insert_many 的条数
  --no-indexes          --format mongo 导入后不重建索引
  --ttl-indexes         --format mongo 连 TTL 索引也重建（默认不建：建了会立刻按过期时间删数据）
  --keep-existing       --format mongo 不 drop 目标集合，追加写入
  --sql-dialect {postgres,mysql}
                        SQL 方言
  --dedup               按 _id 去重（多版本 leaf page 残留时开启）
  --no-progress         关闭 tqdm 进度条
  --dry-run             只统计不写盘，用于核对提取完整性与文档计数
  --allocation-size ALLOCATION_SIZE
                        手工指定块对齐粒度（默认从 turtle / 元数据读，读不到才穷举探测）

常使用命令说明

# 看库结构
mongoext.exe info DBPATH

# 只统计不写盘
mongoext.exe scan DBPATH --out ./out --dry-run --dedup

# 全库导 JSON Lines（人看 / 给别的程序吃）
mongoext.exe scan DBPATH --out ./out --format jsonl --dedup

# 全库导 BSON（给 mongorestore 用）
mongoext.exe scan DBPATH --out ./out --format bson --dedup

# 全库导 SQL
mongoext.exe scan DBPATH --out ./out --format sql --sql-dialect mysql --dedup

# 保真导入运行中的 MongoDB（推荐路径）
mongoext.exe scan DBPATH --format mongo --dedup

# 只导一个集合
mongoext.exe scan DBPATH --format mongo --db mydb --collection users --dedup

# 导到远程实例
mongoext.exe scan DBPATH --format mongo \
    --uri "mongodb://user:pass@host:27017/?authSource=admin" --dedup

# 恢复孤立的单个 collection 文件（无 dbPath / 无 catalog）
mongoext.exe recover FILE.wt --dry-run --dedup          # 先看探测结论
mongoext.exe recover FILE.wt --db mydb --collection users --format mongo --dedup
mongoext.exe recover FILE.wt --meta-dir /原/dbPath --format mongo --dedup

# 批量恢复一个 dbPath 里的孤儿文件
mongoext.exe recover --orphans DBPATH --out ./out --dedup

.bres勒索病毒加密数据库dmp文件可恢复

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：.bres勒索病毒加密数据库dmp文件可恢复

近期.bres勒索病毒较多,加密文件名称一般为:原始文件名.id.bres

勒索信息为:id.readme文件

一般是勒索5000或者7500美元.通过我们对加密的oracle数据库文件和dmp的逻辑备份分析,确认该病毒对于两种类型的文件破坏比例不一样
一、对于bres勒索病毒数据库层面恢复探讨
1. 对于dbf/ora类型文件破坏比例非常大,达到1/3左右,而且还是间隔加密,对于这种情况,基于数据库文件基本上没有太大恢复价值,但是我们也遇到一些客户,由于某些原因导致这类病毒加密文件没有全部完成(数据文件被加密中途终止),这样的情况下，可以恢复没有被破坏的block中数据.

2.对于exp/expdp备份出来的dmp文件,该病毒一般只是加密前面30M的block,后续其他block全部是完整的,可以恢复出来后续数据,整体数据丢失较少,如果有历史完整的备份,可以实现较为完整的恢复效果

二、病毒入侵渠道与内网扩散逻辑
.bres病毒主要有四类入侵入口，其中内网共享权限漏洞是最高发渠道。
第一，共享存储与NAS弱口令入侵。工贸企业、工程公司大量使用NAS存放图纸、合同，为方便异地办公，直接开放外网访问，账号密码长期不修改。黑客会批量扫描公网设备，尝试默认密码登录，拿到权限后植入后门程序，长期潜伏在内网。这也是.bres最标志性的入侵方式，区别于其他病毒以钓鱼邮件为主要入口。
第二，钓鱼邮件投递。伪装成供应商对账函、项目验收单、物流通知单，压缩包内放置恶意程序。员工解压运行之后，木马悄悄植入电脑，建立持久后门，为后续内网横向扩散铺路。
第三，RDP远程桌面爆破。部分小微企业为远程管理服务器，3389端口对外暴露，使用简单密码。黑客利用工具批量爆破账号密码，登陆服务器投放病毒载荷。
第四，根据我们近期恢复的一些case，大量是同一个软件厂商的软件在不同客户现场几乎同一时间被勒索加密,怀疑可能是由于应用某组件有漏洞被黑客利用进行攻击,建议加强应用版本管理,及时更新应用修复漏洞。

ORA-600 krsm_mrp_rebuild_lf_list.lno_mismatch

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：ORA-600 krsm_mrp_rebuild_lf_list.lno_mismatch

客户由于断电,导致一个19c数据库无法正常启动,接手时候报错为：ORA-600 krsm_mrp_rebuild_lf_list.lno_mismatch

2026-09-15T06:36:24.908317-07:00
ALTER DATABASE OPEN RESETLOGS
2026-09-15T06:36:24.912251-07:00
RESETLOGS is being done without consistancy checks. This may result
in a corrupted database. The database should be recreated.
RESETLOGS after incomplete recovery UNTIL CHANGE 22297188690 time 
NET  (PID:36760): Unable to create archive log file '/u19/dbs/arch1_1_1243990641.dbf'
2026-09-15T06:36:24.919696-07:00
Errors in file /u19/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_36760.trc:
ORA-19504: failed to create file "/u19/dbs/arch1_1_1243990641.dbf"
ORA-27038: created file already exists
Additional information: 1
NET  (PID:36760): Error 19504 Creating archive log file to '/u19/app/dbs/arch1_1_1243990641.dbf'
NET  (PID:36760): Archiving not possible: error count exceeded
Errors in file /u19/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_36760.trc  (incident=119876):
ORA-00600: internal error code, arguments: [krsm_mrp_rebuild_lf_list.lno_mismatch], [3], [1], [1], [0], []
Incident details in: /u19/app/oracle/diag/rdbms/orcl/orcl/incident/incdir_119876/orcl_ora_36760_i119876.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
NET  (PID:36760): Archive all ORLs failed, error=600
ORA-600 signalled during: ALTER DATABASE OPEN RESETLOGS...

通过alert日志大概回溯现场人员的恢复过程,最初数据库断电重启之后报ORA-00742错误

2026-09-14T20:52:37.294285-07:00
ALTER DATABASE OPEN
Ping without log force is disabled:
  instance mounted in exclusive mode.
2026-09-14T20:52:37.313266-07:00
Beginning crash recovery of 1 threads
 Thread 1: Recovery starting at checkpoint rba (logseq 63111 block 286483), scn 0
2026-09-14T20:52:37.370330-07:00
Started redo scan
2026-09-14T20:52:37.419944-07:00
Aborting crash recovery due to error 742
2026-09-14T20:52:37.420039-07:00
Errors in file /u19/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_68055.trc:
ORA-00742: Log read detects lost write in thread 1 sequence 63111 block 326157
ORA-00312: online log 3 thread 1: '/u19/app/oracle/oradata/ORCL/redo03.log'
2026-09-14T20:52:37.420977-07:00
Errors in file /u19/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_68055.trc:
ORA-00742: Log read detects lost write in thread 1 sequence 63111 block 326157
ORA-00312: online log 3 thread 1: '/u19/app/oracle/oradata/ORCL/redo03.log'
ORA-742 signalled during: ALTER DATABASE OPEN...

现场人员加上隐含参数强制拉库,然后数据库报ORA-600 kcbzib_kcrsds_1错误

2026-09-14T21:35:21.425310-07:00
ALTER DATABASE OPEN RESETLOGS
2026-09-14T21:35:21.429136-07:00
RESETLOGS is being done without consistancy checks. This may result
in a corrupted database. The database should be recreated.
RESETLOGS after incomplete recovery UNTIL CHANGE 22297188666 time 
Resetting resetlogs activation ID 1680209845 (0x6425f7b5)
2026-09-14T21:35:34.017515-07:00
Setting recovery target incarnation to 3
2026-09-14T21:35:34.028360-07:00
Ping without log force is disabled:
  instance mounted in exclusive mode.
Endian type of dictionary set to little
2026-09-14T21:35:34.043771-07:00
Assigning activation ID 1771732361 (0x699a7d89)
2026-09-14T21:35:34.125473-07:00
TT00 (PID:47466): Gap Manager starting
2026-09-14T21:35:34.127554-07:00
Redo log for group 1, sequence 1 is not located on DAX storage
Thread 1 opened at log sequence 1
  Current log# 1 seq# 1 mem# 0: /u19/app/oracle/oradata/ORCL/redo01.log
Successful open of redo thread 1
2026-09-14T21:35:34.165332-07:00
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
stopping change tracking
2026-09-14T21:35:35.507453-07:00
ORA-00600: internal error code, arguments: [kcbzib_kcrsds_1],  , [], [], [], [], []
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
SYSTEM DUMP REDO DBA MIN 4 128 DBA MAX 4 128 SCN MIN 1;
2026-09-14T21:35:37.154962-07:00
*****************************************************************
An internal routine has requested a dump of selected redo.
This usually happens following a specific internal error, when
analysis of the redo logs will help Oracle Support with the
diagnosis.
It is recommended that you retain all the redo logs generated (by
all the instances) during the past 12 hours, in case additional
redo dumps are required to help with the diagnosis.
*****************************************************************
Undo initialization recovery: err:600 start: 3433752 end: 3435865 diff: 2113 ms (2.1 seconds)
2026-09-14T21:35:37.475752-07:00
Errors in file /u19/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_46217.trc:
ORA-00600: internal error code, arguments: [kcbzib_kcrsds_1],  , [], [], [], [], []
2026-09-14T21:35:37.475831-07:00
Errors in file /u19/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_46217.trc:
ORA-00600: internal error code, arguments: [kcbzib_kcrsds_1],  , [], [], [], [], []
Error 600 happened during db open, shutting down database
Errors in file /u19/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_46217.trc  (incident=48128):
ORA-00603: ORACLE server session terminated by fatal error
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-00600: internal error code, arguments: [kcbzib_kcrsds_1],  , [], [], [], [], []

现场人员重建控制文件,然后启动库报ORA-600 krsm_mrp_rebuild_lf_list.lno_mismatch

2026-09-15T04:26:41.297637-07:00
Successful mount of redo thread 1, with mount id 1771762176
Completed: CREATE CONTROLFILE REUSE SET DATABASE "ORCL" RESETLOGS ARCHIVELOG
    MAXLOGFILES 16
    MAXLOGMEMBERS 3
    MAXDATAFILES 100
    MAXINSTANCES 1
    MAXLOGHISTORY 16
LOGFILE
  GROUP 1 '/u19/app/oracle/oradata/ORCL_NEW/redo01.log' SIZE 200M,
  GROUP 2 '/u19/app/oracle/oradata/ORCL_NEW/redo02.log' SIZE 200M
DATAFILE
  '/u19/app/oracle/oradata/ORCL_NEW/system01.dbf',
  '/u19/app/oracle/oradata/ORCL_NEW/sysaux01.dbf',
  '/u19/app/oracle/oradata/ORCL_NEW/undotbs01.dbf',
…………
2026-09-15T04:26:52.121793-07:00
ALTER DATABASE OPEN RESETLOGS
2026-09-15T04:26:52.126848-07:00
RESETLOGS is being done without consistancy checks. This may result
in a corrupted database. The database should be recreated.
RESETLOGS after incomplete recovery UNTIL CHANGE 22297188686 time 
NET  (PID:27130): Unable to create archive log file '/u19/dbs/arch1_1_1243990641.dbf'
2026-09-15T04:26:52.233538-07:00
Errors in file /u19/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_27130.trc:
ORA-19504: failed to create file "/u19/dbs/arch1_1_1243990641.dbf"
ORA-27038: created file already exists
Additional information: 1
NET  (PID:27130): Error 19504 Creating archive log file to '/u19/dbs/arch1_1_1243990641.dbf'
NET  (PID:27130): Archiving not possible: error count exceeded
Errors in file /u19/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_27130.trc  (incident=102102):
ORA-00600: internal error code, arguments: [krsm_mrp_rebuild_lf_list.lno_mismatch], [2], [1], [1], [0], []
Incident details in: /u19/app/oracle/diag/rdbms/orcl/orcl/incident/incdir_102102/orcl_ora_27130_i102102.trc

接手故障开始处理,尝试open库,重现ORA-600 krsm_mrp_rebuild_lf_list.lno_mismatch错误

SQL> recover database using backup controlfile;
ORA-00279: change 22297188689 generated at 09/15/2026 00:57:22 needed for thread 1
ORA-00289: suggestion : /u19/dbs/arch/1_1_1243990641.dbf
ORA-00280: change 22297188689 for thread 1 is in sequence #1


Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
/u19/app/oracle/oradata/ORCL/redo01.log
Log applied.
Media recovery complete.
SQL>        
SQL> 
SQL> alter database open resetlogs;
alter database open resetlogs
*
ERROR at line 1:
ORA-00600: internal error code, arguments: [krsm_mrp_rebuild_lf_list.lno_mismatch], [3], [1], [1], [0]

分析alert日志,是由于归档无法被正常创建导致该问题,对/u19/dbs/arch/进行创建并授权之后,再次尝试打开数据库,数据库报ORA-600 kcbzib_kcrsds_1错误

SQL> startup nomount pfile='/tmp/pfile';
ORACLE instance started.

Total System Global Area       6442448984 bytes
Fixed Size			  8910936 bytes
Variable Size		       3858759680 bytes
Database Buffers	       2566914048 bytes
Redo Buffers			  7864320 bytes
SQL> alter database mount;

Database altered.

SQL> alter database open resetlogs;
alter database open resetlogs
*
ERROR at line 1:
ORA-00603: ORACLE server session terminated by fatal error
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-00600: internal error code, arguments: [kcbzib_kcrsds_1], [], [], [], []
Process ID: 42259
Session ID: 1 Serial number: 19675

使用obet的patch scn功能修改scn之后（obet patch_scn使用说明）,顺利打开库

记录一次0丢失的ORA-00354: 损坏重做日志块标头故障恢复

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：记录一次0丢失的ORA-00354: 损坏重做日志块标头故障恢复

由于服务器断电导致数据库启动报错

Mon Sep 07 12:47:52 2026
ALTER DATABASE OPEN
Beginning crash recovery of 1 threads
 parallel recovery started with 32 processes
Started redo scan
ORA-00354: 损坏重做日志块标头
ORA-00353: 日志损坏接近块 41735 更改 2385313823 时间 09/07/2026 10:35:04
ORA-00312: 联机日志 5 线程 1: 'D:\APP\LIMSADMIN\ORADATA\ORCL\REDO05.LOG'
Mon Sep 07 12:47:58 2026
Trace dumping is performing id=[cdmp_20260907124758]
Errors in file d:\app\limsadmin\diag\rdbms\primary\orcl\trace\orcl_ora_3040.trc:
ORA-00354: 损坏重做日志块标头
ORA-00353: 日志损坏接近块 41735 更改 2385313823 时间 09/07/2026 10:35:04
ORA-00312: 联机日志 5 线程 1: 'D:\APP\LIMSADMIN\ORADATA\ORCL\REDO05.LOG'
ORA-354 signalled during: ALTER DATABASE OPEN...

从这里看,数据库在启动的时候实例恢复需要读取REDO05.LOG日志,但是发现该日志中有block损坏,而且损坏日志的时间点为09/07/2026 10:35:04,scn为2385313823（ORA-00353: 日志损坏接近块 41735 更改 2385313823 时间 09/07/2026 10:35:04）,导致实例恢复无法完成,从而数据库无法正常启动.通过Oracle Recovery Check脚本检查数据库当前状态,数据库的checkpoint time为:09/07/2026 10:37:37,checkpoint scn为:2385314357

这个就比较明显,数据库文件正在需要的日志是sequence 19001 REDO01.LOG(而不是报错的sequence 19000 REDO05.LOG),基于这样的情况,出现这种问题,是由于control中记录的Thread Checkpoint RBA不正确导致.解决这个问题相对比较简单,重建ctl即可

由于redo05.log有损坏,该库为归档模式,需要先clear redo05.log（不然会导致redo无法归档,数据库hang住）,然后对全库发起备份

不当数据库恢复操作导致一个月数据丢失

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：不当数据库恢复操作导致一个月数据丢失

今天分享一个本不该发生的故障,结果由于现场人员没有合适的判断导致客户丢失近一个月数据,事故经过大概是这样的
1. 客户数据库由于断电,启动之后报ORA-600 kcratr_scan_lastbwr错误

2026-09-03T11:22:58.998379+08:00
Beginning crash recovery of 2 threads
 parallel recovery started with 31 processes
 Thread 1: Recovery starting at checkpoint rba (logseq 37738 block 7724), scn 0
 Thread 2: Recovery starting at checkpoint rba (logseq 43821 block 37937), scn 0
2026-09-03T11:22:59.158232+08:00
Started redo scan
Hex dump of (file 20, block 2720) in trace file /u01/app/oracle/diag/rdbms/orcl/orcl1/trace/orcl1_ora_34049.trc

Reading datafile '+DATA/ORCL/DATAFILE/undotbs01.dbf' for corrupt data at rdba: 0x05000aa0 (file 20, block 2720)
Reread Not Encrypted (file 20, block 2720) found same data 
physically corrupt:No, logically corrupt:No, and is not expected version by upper layer. 
Write verification failed for File 20 Block 2720 (rdba 0x5000aa0)
*****************************************************************
An internal routine has requested a dump of selected redo.
This usually happens following a specific internal error, when
analysis of the redo logs will help Oracle Support with the
diagnosis.
It is recommended that you retain all the redo logs generated (by
all the instances) during the past 12 hours, in case additional
redo dumps are required to help with the diagnosis.
*****************************************************************
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl1/trace/orcl1_ora_34049.trc  (incident=430645):
ORA-00600: internal error code, arguments: [kcratr_scan_lastbwr], [], [], [], [], [], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/orcl/orcl1/incident/incdir_430645/orcl1_ora_34049_i430645.trc
2026-09-03T11:23:00.484486+08:00
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
2026-09-03T11:23:00.510512+08:00
Slave encountered ORA-10388 exception during crash recovery

这个是一个比较常见的错误一般是由于redo写丢失导致(数据库open报ORA-600 kcratr_scan_lastbwr故障处理)

2. 现场操作的技术人员看了下这个错误,觉得数据库有备份，直接决定使用备份进行恢复

结果发现读取的是20260719的备份,而且原始的数据文件在asm里面，已经被覆盖。这个时候如果归档或者增量备份完整,还能够最大限度抢救数据,检查备份发现只有8月7日的增量备份,没有任何归档的备份(使用crontab 直接清理3天之前的归档)

基于这样的情况,应用到最后一个增量备份(也就是8月7日),后面的数据由于归档已经被清理(而且没有备份)

出现这个问题的原因是由于备份脚本本身备份归档,导致备份无法按照策略清理历史备份,导致磁盘空间满,从而后续备份无法继续(直接清理归档了)

恢复只能到此为止,查询最终的文件头恢复时间点

3. 基于这种情况只能尝试强制打开数据库,结果报ORA-600 kcbzibmlt_kcrsds_1错误

SQL> alter database open resetlogs;
alter database open resetlogs
*
ERROR at line 1:
ORA-00603: ORACLE server session terminated by fatal error
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-00600: internal error code, arguments: [kcbzibmlt_kcrsds_1], [], [], []
Process ID: 30365
Session ID: 2180 Serial number: 43414

4. 这种错误类似ORA-600 kcbzib_kcrsds_1的处理思路,使用obet的patch_scn功能直接进行处理

OBET> patch_scn 89103 0x60017e98 3182707622

=== Oracle SCN Patch ===
PID:     89103
Address: 0x60017e98
New Value: 0xbdb443a6 (3182707622)
Original SCN: 0x0

Are you sure you want to modify Oracle SCN? (yes/no): y

SCN modified: 0x0 -> 0xbdb443a6
Oracle SCN successfully modified.

然后打开数据库

SQL> alter database open;

Database altered.

导出数据,导入新库完成本次恢复任务,可惜没有办法帮客户找回来丢失的那一个月数据.

案例总结:
1. rman备份脚本尽量不要有直接删除归档的操作,尽可能是先备份然后再删除(确认备份成功之后删除)
2. 备份还原操作之前,需要看备份文件和日志,大概评估下备份的可用性
3. 尽可能不要在损坏的现场环境上直接做rman还原操作(万一备份无法正常还原,比如中途有备份文件异常呢？)
4. 这个故障,如果不做还原操作,直接异常恢复,基本上也就丢失了当前redo中的数据,损失是最小的

断电引起ORA-600 3005故障处理