segment header异常对象删除处理

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:segment header异常对象删除处理

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

对于某些极端情况下,segment header出现损坏的对象该如何处理,这里通过一个实验来说明这类情况该如何处理,创建表并查询相关segment信息

SQL> create tablespace t_xff datafile '/u01/app/oracle/oradata/orcl/t_xifenfei01.dbf' size 128M autoextend on;

Tablespace created.

SQL> create table t_xifenfei tablespace t_xff as select * from dba_objects;

Table created.

SQL> select header_file,header_block from dba_SEGMENTS where segment_name='T_XIFENFEI'  and owner='SYS';

HEADER_FILE HEADER_BLOCK
----------- ------------
          5          130
----segment header 为 file 5,block 130

SQL> select   EXTENT_ID , FILE_ID ,BLOCK_ID  from dba_extents where segment_name='T_XIFENFEI'  and owner='SYS';

 EXTENT_ID    FILE_ID   BLOCK_ID
---------- ---------- ----------
         0          5        128
         1          5        136
         2          5        144
         3          5        152
         4          5        160
         5          5        168
         6          5        176
         7          5        184
         8          5        192
         9          5        200
        10          5        208
        11          5        216
        12          5        224
        13          5        232
        14          5        240
        15          5        248
        16          5        256
        17          5        384
        18          5        512
        19          5        640
        20          5        768
        21          5        896
        22          5       1024
        23          5       1152
        24          5       1280

25 rows selected.

---有25个extent信息

SQL> select * from dba_free_space where file_id=5;

TABLESPACE_NAME                   FILE_ID   BLOCK_ID      BYTES     BLOCKS
------------------------------ ---------- ---------- ---------- ----------
RELATIVE_FNO
------------
T_XFF                                   5       1408  122683392      14976
           5
--空闲block是从1408开始

模拟segment header损坏(通过dd破坏block)

SQL> shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> 
SQL> 
SQL> 
SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
[oracle@xifenfei ~]$ 
[oracle@xifenfei ~]$ 
[oracle@xifenfei ~]$echo xifenfei.com|dd of=/u01/app/oracle/oradata/orcl/t_xifenfei01.dbf bs=8192 conv=notrunc seek=130
0+1 records in
0+1 records out
17 bytes (17 B) copied, 5.4389e-05 s, 313 kB/s
[oracle@xifenfei ~]$ dbv file=/u01/app/oracle/oradata/orcl/t_xifenfei01.dbf

DBVERIFY: Release 11.2.0.4.0 - Production on Wed Apr 13 20:29:41 2022

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.

DBVERIFY - Verification starting : FILE = /u01/app/oracle/oradata/orcl/t_xifenfei01.dbf
Page 130 is marked corrupt
Corrupt block relative dba: 0x01400082 (file 5, block 130)
Bad header found during dbv: 
Data in bad block:
 type: 119 format: 7 rdba: 0x65666978
 last change scn: 0x632e.6965666e seq: 0x6f flg: 0x6d
 spare1: 0x77 spare2: 0x2e spare3: 0x0
 consistency value in tail: 0xada72301
 check value in block header: 0xf30a
 computed block checksum: 0x5eb9



DBVERIFY - Verification complete

Total Pages Examined         : 16384
Total Pages Processed (Data) : 1234
Total Pages Failing   (Data) : 0
Total Pages Processed (Index): 0
Total Pages Failing   (Index): 0
Total Pages Processed (Other): 154
Total Pages Processed (Seg)  : 0
Total Pages Failing   (Seg)  : 0
Total Pages Empty            : 14995
Total Pages Marked Corrupt   : 1
Total Pages Influx           : 0
Total Pages Encrypted        : 0
Highest block SCN            : 6466979 (0.6466979)

查询表数据报错

[oracle@xifenfei ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.4.0 Production on Wed Apr 13 20:29:48 2022

Copyright (c) 1982, 2013, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup
ORACLE instance started.

Total System Global Area  734892032 bytes
Fixed Size                  2256872 bytes
Variable Size             452984856 bytes
Database Buffers          276824064 bytes
Redo Buffers                2826240 bytes
Database mounted.
Database opened.
SQL> select count(1) from t_xifenfei;
select count(1) from t_xifenfei
                     *
ERROR at line 1:
ORA-01578: ORACLE data block corrupted (file # 5, block # 130)
ORA-01110: data file 5: '/u01/app/oracle/oradata/orcl/t_xifenfei01.dbf'

查询extent信息为空

SQL> select   EXTENT_ID   , FILE_ID   ,BLOCK_ID  from dba_extents where segment_name='T_XIFENFEI'  and owner='SYS';

no rows selected

SQL> select header_file,header_block from dba_SEGMENTS where segment_name='T_XIFENFEI'  and owner='SYS';

HEADER_FILE HEADER_BLOCK
----------- ------------
          5          130

SQL> select * from dba_free_space where file_id=5;

TABLESPACE_NAME                   FILE_ID   BLOCK_ID      BYTES     BLOCKS
------------------------------ ---------- ---------- ---------- ----------
RELATIVE_FNO
------------
T_XFF                                   5       1408  122683392      14976
           5

尝试删除表报错

SQL> drop table t_xifenfei;
drop table t_xifenfei
           *
ERROR at line 1:
ORA-01578: ORACLE data block corrupted (file # 5, block # 130)
ORA-01110: data file 5: '/u01/app/oracle/oradata/orcl/t_xifenfei01.dbf'

通过加purge删除成功

SQL> drop table t_xifenfei purge;

Table dropped.

删除表成功,但是磁盘空间未释放,通过查询确认变为一个临时段

SQL> select * from dba_free_space where file_id=5;

TABLESPACE_NAME                   FILE_ID   BLOCK_ID      BYTES     BLOCKS
------------------------------ ---------- ---------- ---------- ----------
RELATIVE_FNO
------------
T_XFF                                   5       1408  122683392      14976
           5


SQL> select segment_name,segment_type,owner from dba_segments where header_file=5;

SEGMENT_NAME
--------------------------------------------------------------------------------
SEGMENT_TYPE       OWNER
------------------ ------------------------------
5.130
TEMPORARY          SYS

清理临时段,彻底删除segment header异常对象删除后遗症

SQL> exec dbms_space_admin.segment_corrupt('T_XFF',5,130); 

PL/SQL procedure successfully completed.

SQL> exec dbms_space_admin.segment_drop_corrupt('T_XFF',5,130);

PL/SQL procedure successfully completed.

SQL> exec DBMS_SPACE_ADMIN.TABLESPACE_REBUILD_BITMAPS ('T_XFF');

PL/SQL procedure successfully completed.

SQL> select segment_name,segment_type,owner from dba_segments where header_file=5;

no rows selected

SQL> select * from dba_free_space where file_id=5;

TABLESPACE_NAME                   FILE_ID   BLOCK_ID      BYTES     BLOCKS
------------------------------ ---------- ---------- ---------- ----------
RELATIVE_FNO
------------
T_XFF                                   5        128  133169152      16256
           5

ORA-600 ktbsdp2 处理

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:ORA-600 ktbsdp2 处理

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

客户反馈数据库异常:两个节点rac,两个节点都启动,其中一个节点无法正常open,另外一个节点一段时间后也会挂。以下是无法正常open节点报错信息(正常open节点最终挂掉报错信息也是类似)

2022-03-30T19:19:12.870813+08:00
[84321] Successfully onlined Undo Tablespace 4.
Undo initialization finished serial:0 start:1728252021 end:1728256302 diff:4281 ms (4.3 seconds)
Verifying minimum file header compatibility for tablespace encryption..
Verifying file header compatibility for tablespace encryption completed for pdb 0
2022-03-30T19:19:13.953252+08:00
Database Characterset is ZHS16GBK
2022-03-30T19:19:14.538155+08:00
Errors in file /oracle/app/oracle/diag/rdbms/xff/xff2/trace/xff2_p02o_85718.trc  (incident=1093927):
ORA-00600: internal error code, arguments: [ktbsdp2], [18446744073709551615], [], [], [], [], [], [], [], [], [], []
Incident details in: /oracle/app/oracle/diag/rdbms/xff/xff2/incident/incdir_1093927/xff2_p02o_85718_i1093927.trc
2022-03-30T19:19:15.536582+08:00
ORACLE Instance xff2 (pid = 57) - Error 607 encountered while recovering transaction (73, 12) on object 112841.
2022-03-30T19:19:33.699944+08:00
Errors in file /oracle/app/oracle/diag/rdbms/xff/xff2/trace/xff2_smon_84007.trc:
ORA-00607: Internal error occurred while making a change to a data block
ORA-00600: internal error code, arguments: [ktbsdp2], [18446744073709551615], [], [], [], [], [], [], [], [], [], []
2022-03-30T19:19:34.673840+08:00
Errors in file /oracle/app/oracle/diag/rdbms/xff/xff2/trace/xff2_smon_84007.trc:
ORA-00607: Internal error occurred while making a change to a data block
ORA-00600: internal error code, arguments: [ktbsdp2], [18446744073709551615], [], [], [], [], [], [], [], [], [], []
2022-03-30T19:19:34.673954+08:00
Errors in file /oracle/app/oracle/diag/rdbms/xff/xff2/trace/xff2_smon_84007.trc:
ORA-00607: Internal error occurred while making a change to a data block
ORA-00600: internal error code, arguments: [ktbsdp2], [18446744073709551615], [], [], [], [], [], [], [], [], [], []
Errors in file /oracle/app/oracle/diag/rdbms/xff/xff2/trace/xff2_smon_84007.trc  (incident=1092704):
ORA-607 [] [] [] [] [] [] [] [] [] [] [] []
Incident details in: /oracle/app/oracle/diag/rdbms/xff/xff2/incident/incdir_1092704/xff2_smon_84007_i1092704.trc
2022-03-30T19:19:35.422779+08:00
*****************************************************************
An internal routine has requested a dump of selected redo.
This usually happens following a specific internal error, when
analysis of the redo logs will help Oracle Support with the
diagnosis.
It is recommended that you retain all the redo logs generated (by
all the instances) during the past 12 hours, in case additional
redo dumps are required to help with the diagnosis.
*****************************************************************
2022-03-30T19:19:36.154689+08:00
Starting background process GTX0
2022-03-30T19:19:36.169007+08:00
GTX0 started with pid=370, OS id=87409 
2022-03-30T19:19:36.645876+08:00
USER (ospid: 84007): terminating the instance due to error 607
2022-03-30T19:19:36.680109+08:00
opiodr aborting process unknown ospid (87439) as a result of ORA-1092
2022-03-30T19:19:36.681091+08:00
ORA-1092 : opitsk aborting process
2022-03-30T19:19:36.740357+08:00
System state dump requested by (instance=2, osid=84007 (SMON)), summary=[abnormal instance termination].
System State dumped to trace file /oracle/app/oracle/diag/rdbms/xff/xff2/trace/xff2_diag_83895_20220330191936.trc
2022-03-30T19:19:40.135579+08:00
Instance terminated by USER, pid = 84007

对于上述报错信息分析,初步判断是由于事务异常导致,查询mos发现类似报错Bug 32208691 – After upgrade from 12.1 to 19.3 drop columns fails ORA-600[ktbsdp2] ORA-600[4512] (Doc ID 32208691.8),通过咨询客户,确认他们这边是通过plsql dev工具对id为112841表进行增加列的时候网络中断导致增加失败,后续我尝试对该表进行查询发现也报该错误,基本上可以确认由于该表事务异常导致,通过dul把该表数据恢复,然后drop 该表,数据库启动正常,未见其他报错,通过hcheck检查,数据库字典基本一致(除一些统计信息异常,原则上不影响数据库运行)

[oracle@xifenfei2 ~]$ sqlplus / as sysdba @hcheck.sql

SQL*Plus: Release 12.2.0.1.0 Production on Thu Mar 31 00:38:32 2022

Copyright (c) 1982, 2016, Oracle.  All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

HCheck Version 07MAY18 on 31-MAR-2022 00:38:34
----------------------------------------------
Catalog Version 12.2.0.1.0 (1202000100)
db_name: xff
Is CDB?: NO

                                   Catalog       Fixed
Procedure Name                     Version    Vs Release    Timestamp
Result
------------------------------ ... ---------- -- ---------- --------------
------
.- LobNotInObj                 ... 1202000100 <=  *All Rel* 03/31 00:38:34 PASS
.- MissingOIDOnObjCol          ... 1202000100 <=  *All Rel* 03/31 00:38:34 PASS
.- SourceNotInObj              ... 1202000100 <=  *All Rel* 03/31 00:38:34 PASS
.- OversizedFiles              ... 1202000100 <=  *All Rel* 03/31 00:38:38 PASS
.- PoorDefaultStorage          ... 1202000100 <=  *All Rel* 03/31 00:38:38 PASS
.- PoorStorage                 ... 1202000100 <=  *All Rel* 03/31 00:38:38 PASS
.- TabPartCountMismatch        ... 1202000100 <=  *All Rel* 03/31 00:38:38 PASS
.- OrphanedTabComPart          ... 1202000100 <=  *All Rel* 03/31 00:38:38 PASS
.- MissingSum$                 ... 1202000100 <=  *All Rel* 03/31 00:38:38 PASS
.- MissingDir$                 ... 1202000100 <=  *All Rel* 03/31 00:38:38 PASS
.- DuplicateDataobj            ... 1202000100 <=  *All Rel* 03/31 00:38:40 PASS
.- ObjSynMissing               ... 1202000100 <=  *All Rel* 03/31 00:38:42 PASS
.- ObjSeqMissing               ... 1202000100 <=  *All Rel* 03/31 00:38:42 PASS
.- OrphanedUndo                ... 1202000100 <=  *All Rel* 03/31 00:38:44 PASS
.- OrphanedIndex               ... 1202000100 <=  *All Rel* 03/31 00:38:44 PASS
.- OrphanedIndexPartition      ... 1202000100 <=  *All Rel* 03/31 00:38:45 PASS
.- OrphanedIndexSubPartition   ... 1202000100 <=  *All Rel* 03/31 00:38:45 PASS
.- OrphanedTable               ... 1202000100 <=  *All Rel* 03/31 00:38:45 PASS
.- OrphanedTablePartition      ... 1202000100 <=  *All Rel* 03/31 00:38:45 PASS
.- OrphanedTableSubPartition   ... 1202000100 <=  *All Rel* 03/31 00:38:45 PASS
.- MissingPartCol              ... 1202000100 <=  *All Rel* 03/31 00:38:45 PASS
.- OrphanedSeg$                ... 1202000100 <=  *All Rel* 03/31 00:38:45 PASS
.- OrphanedIndPartObj#         ... 1202000100 <=  *All Rel* 03/31 00:38:45 PASS
.- DuplicateBlockUse           ... 1202000100 <=  *All Rel* 03/31 00:38:45 PASS
.- FetUet                      ... 1202000100 <=  *All Rel* 03/31 00:38:45 PASS
.- Uet0Check                   ... 1202000100 <=  *All Rel* 03/31 00:38:45 PASS
.- SeglessUET                  ... 1202000100 <=  *All Rel* 03/31 00:38:45 PASS
.- BadInd$                     ... 1202000100 <=  *All Rel* 03/31 00:38:45 PASS
.- BadTab$                     ... 1202000100 <=  *All Rel* 03/31 00:38:45 PASS
.- BadIcolDepCnt               ... 1202000100 <=  *All Rel* 03/31 00:38:45 PASS
.- ObjIndDobj                  ... 1202000100 <=  *All Rel* 03/31 00:38:45 PASS
.- TrgAfterUpgrade             ... 1202000100 <=  *All Rel* 03/31 00:38:45 PASS
.- ObjType0                    ... 1202000100 <=  *All Rel* 03/31 00:38:45 PASS
.- BadOwner                    ... 1202000100 <=  *All Rel* 03/31 00:38:45 PASS
.- StmtAuditOnCommit           ... 1202000100 <=  *All Rel* 03/31 00:38:45 PASS
.- BadPublicObjects            ... 1202000100 <=  *All Rel* 03/31 00:38:46 PASS
.- BadSegFreelist              ... 1202000100 <=  *All Rel* 03/31 00:38:50 PASS
.- BadDepends                  ... 1202000100 <=  *All Rel* 03/31 00:38:50 PASS
.- CheckDual                   ... 1202000100 <=  *All Rel* 03/31 00:38:57 PASS
.- ObjectNames                 ... 1202000100 <=  *All Rel* 03/31 00:38:57 WARN

HCKW-0018: OBJECT name clashes with SCHEMA name (Doc ID 2363142.1)
Schema=MHWZ PACKAGE=MHWZ.MHWZ
Schema=MHWZ PACKAGE BODY=MHWZ.MHWZ

.- BadCboHiLo                  ... 1202000100 <=  *All Rel* 03/31 00:39:01 WARN

HCKW-0019: HIST_HEAD$.LOWVAL > HIVAL (Doc ID 1361047.1)
OBJ# 324163 INTCOL#=22
OBJ# 482668 INTCOL#=4
OBJ# 442865 INTCOL#=31
OBJ# 436924 INTCOL#=31
OBJ# 580529 INTCOL#=8
OBJ# 459432 INTCOL#=31
OBJ# 451260 INTCOL#=31
OBJ# 530980 INTCOL#=21
OBJ# 498442 INTCOL#=5
OBJ# 652114 INTCOL#=8
OBJ# 701695 INTCOL#=21
OBJ# 831961 INTCOL#=31
OBJ# 831962 INTCOL#=31
OBJ# 831963 INTCOL#=31

.- ChkIotTs                    ... 1202000100 <=  *All Rel* 03/31 00:39:09 PASS
.- NoSegmentIndex              ... 1202000100 <=  *All Rel* 03/31 00:39:09 PASS
.- BadNextObject               ... 1202000100 <=  *All Rel* 03/31 00:39:09 PASS
.- DroppedROTS                 ... 1202000100 <=  *All Rel* 03/31 00:39:09 PASS
.- FilBlkZero                  ... 1202000100 <=  *All Rel* 03/31 00:39:09 PASS
.- DbmsSchemaCopy              ... 1202000100 <=  *All Rel* 03/31 00:39:09 PASS
.- OrphanedIdnseqObj           ... 1202000100 >  1201000000 03/31 00:39:09 PASS
.- OrphanedIdnseqSeq           ... 1202000100 >  1201000000 03/31 00:39:09 PASS
.- OrphanedObjError            ... 1202000100 >  1102000000 03/31 00:39:09 PASS
.- ObjNotLob                   ... 1202000100 <=  *All Rel* 03/31 00:39:09 PASS
.- MaxControlfSeq              ... 1202000100 <=  *All Rel* 03/31 00:39:09 PASS
.- SegNotInDeferredStg         ... 1202000100 >  1102000000 03/31 00:39:13 PASS
.- SystemNotRfile1             ... 1202000100 >   902000000 03/31 00:39:13 PASS
.- DictOwnNonDefaultSYSTEM     ... 1202000100 <=  *All Rel* 03/31 00:39:13 PASS
.- OrphanTrigger               ... 1202000100 <=  *All Rel* 03/31 00:39:13 PASS
.- ObjNotTrigger               ... 1202000100 <=  *All Rel* 03/31 00:39:13 PASS
---------------------------------------
31-MAR-2022 00:39:13  Elapsed: 39 secs
---------------------------------------
Found 0 potential problem(s) and 16 warning(s)
Contact Oracle Support with the output and trace file
to check if the above needs attention or not

PL/SQL procedure successfully completed.

Statement processed.

Complete output is in trace file:
/oracle/app/oracle/diag/rdbms/xff/xff2/trace/xff2_ora_26887_HCHECK.trc

12c启动报kcratr_nab_less_than_odr

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:12c启动报kcratr_nab_less_than_odr

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

ORA-600 kcratr_nab_less_than_odr这个错误,以前的认知里面,主要是11.2.0.1中比较常见,这次在12c中见到了,记录下
数据库版本

Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options.
Windows NT Version V6.1  

数据库启动报错

Fri Apr 30 23:11:16 2021
Completed redo scan
 read 20 KB redo, 26 data blocks need recovery
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\orcl12c\orcl12c\trace\orcl12c_ora_5744.trc  (incident=28863):
ORA-00600: 内部错误代码, 参数: [kcratr_nab_less_than_odr], [1], [6872], [23015], [23017], [], [], [], [], [], [], []
Incident details in: D:\APP\ADMINISTRATOR\diag\rdbms\orcl12c\orcl12c\incident\incdir_28863\orcl12c_ora_5744_i28863.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Fri Apr 30 23:11:17 2021
Slave encountered ORA-10388 exception during crash recovery
Fri Apr 30 23:11:17 2021
Slave encountered ORA-10388 exception during crash recovery
Fri Apr 30 23:11:17 2021
Aborting crash recovery due to error 600
Fri Apr 30 23:11:17 2021
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\orcl12c\orcl12c\trace\orcl12c_ora_5744.trc:
ORA-00600: 内部错误代码, 参数: [kcratr_nab_less_than_odr], [1], [6872], [23015], [23017], [], [], [], [], [], [], []
Fri Apr 30 23:11:17 2021
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\orcl12c\orcl12c\trace\orcl12c_ora_5744.trc:
ORA-00600: 内部错误代码, 参数: [kcratr_nab_less_than_odr], [1], [6872], [23015], [23017], [], [], [], [], [], [], []
ORA-600 signalled during: alter database open...

这个故障处理比较简单参考:ORA-600 kcratr_nab_less_than_odr故障解决

KGL-heap-size-exceeded报错

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:KGL-heap-size-exceeded报错

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

alert日志报错

2021-04-19T15:06:08.208159+08:00
Memory Notification: Library Cache Object loaded into SGA
Heap size 512000K exceeds notification threshold (51200K)
Details in trace file /mnt/app/diag/rdbms/xff/xff/trace/xff_ora_12300.trc
2021-04-19T15:06:08.208212+08:00
KGL object name :select DISTINCT(BEGIN_TIME) beginTime,
    MILE mile,
    END_TIME endTime,
    LAST_TIME lastTime,
    BEGIN_TIME beginTime2,
    END_TIME endTime2,
    TEMPERATURE1 temperature1,
    TEMPERATURE2 temperature2,
    TEMPERATURE3 temperature3,
    TEMPERATURE4 temperature4,
    LATITUDE latitude,
    LONGITUDE longitude
    from T_XIFENFEI
    where VEHICLE_ID = :1 
    and END_TIME is not null
    and BEGIN_TIME BETWEEN TO_DATE(:2 , 'yyyy-MM-dd hh24:mi:ss') and TO_DATE(:3 , 'yyyy-MM-dd hh24
Errors in file /mnt/app/diag/rdbms/xff/xff/trace/xff_ora_12300.trc  (incident=162089):
ORA-00600: internal error code, arguments: [KGL-heap-size-exceeded], 
[0x1C4782708], [0], [524288008], [], [], [], [], [], [], [], []
Incident details in: /mnt/app/diag/rdbms/xff/xff/incident/incdir_162089/xff_ora_12300_i162089.trc
Use ADRCI or Support Workbench to package the incident.

数据库版本信息

Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production
Build label:    RDBMS_12.2.0.1.0_LINUX.X64_170125
ORACLE_HOME:    /mnt/app/product/12.2.0/dbhome_1
System name:    Linux
Node name:      izbp1399kdj96ra6bjtwyxz
Release:        3.10.0-693.2.2.el7.x86_64
Version:        #1 SMP Tue Sep 12 22:26:13 UTC 2017
Machine:        x86_64

Call Stack信息

----- Call Stack Trace -----
calling              call     entry                argument values in hex
location             type     point                (? means dubious value)
-------------------- -------- -------------------- ----------------------------
ksedst()+119         call     kgdsdst()            7FFD6CB9A258 000000002
                                                   7FFD6CB7BCC0 ? 7FFD6CB7BDD8 ?
                                                   000000000 000000082 ?
dbkedDefDump()+1200  call     ksedst()             000000000 000000002 ?
                                                   7FFD6CB7BCC0 ? 7FFD6CB7BDD8 ?
                                                   000000000 ? 000000082 ?
ksedmp()+259         call     dbkedDefDump()       000000003 000000002
                                                   7FFD6CB7BCC0 ? 7FFD6CB7BDD8 ?
                                                   000000000 ? 000000082 ?
dbgexPhaseII()+2130  call     ksedmp()             0000003EB 000000002 ?
                                                   7FFD6CB7BCC0 ? 7FFD6CB7BDD8 ?
                                                   000000000 ? 000000082 ?
dbgexExplicitEndInc  call     dbgexPhaseII()       7F4853DB46C0 7F4853DB77A0
()+602                                             7FFD6CB9E388 7FFD6CB7BDD8 ?
                                                   000000000 ? 000000082 ?
dbgeEndDDEInvocatio  call     dbgexExplicitEndInc  7F4853DB46C0 7F4853DB77A0
nImpl()+658                   ()                   7FFD6CB9E388 ? 7FFD6CB7BDD8 ?
                                                   000000000 ? 000000082 ?
kglLargeHeapWarning  call     dbgeEndDDEInvocatio  7F4853DB46C0 7F4853DB77A0
()+1544                       nImpl()              7F4853DF49A0 7FFD6CB7BDD8 ?
                                                   7F4853DF49A0 000000082 ?
kglHeapAllocCbk()+2  call     kglLargeHeapWarning  7F4853DF49A0 ? 1C4782708 ?
27                            ()                   000000000 01F400060 ?
                                                   0B6DADD88 000000082 ?
kghalp()+1086        call     kglHeapAllocCbk()    7F4853DF49A0 ? 1C4782708 ?
                                                   2EE171B70 01F400060 ?
                                                   000000058 ? 000000082 ?
kkocsRegBindEqvCtxC  call     kghalp()             7F4853DF49A0 ? 0B6DADD88 ?
ommon()+145                                        2EE171B70 ? 000000001 ?
                                                   000000058 ? 0119E7860 ?
kkocsAddBeElem()+81  call     kkocsRegBindEqvCtxC  7F4853DF49A0 ? 7FFD6CB9F460 ?
1                             ommon()              2EE171B70 ? 000000001 ?
                                                   000000002 ? 0119E7860 ?
kkocsProcBSensPredC  call     kkocsAddBeElem()     7F484E2EF998 ? 7F484E2EFAC8 ?
B()+431                                            0BBED4798 ? 209DFEA60 ?
                                                   000000002 ? 0119E7860 ?
qksopVisitPreds()+5  call     kkocsProcBSensPredC  7F484E2EF998 ? 7FFD6CB9F640 ?
39                            B()                  0BBED4798 ? 209DFEA60 ?
                                                   000000002 ? 0119E7860 ?
kkocsBuildBESelCtx(  call     qksopVisitPreds()    7F484E2EF998 ? 7FFD6CB9F640 ?
)+153                                              0BBED4798 ? 209DFEA60 ?
                                                   000000002 ? 0119E7860 ?
kkocsBindSensCheck(  call     kkocsBuildBESelCtx(  7F484E2EF998 ? 7FFD6CB9F640 ?
)+585                         )                    0BBED4798 ? 209DFEA60 ?
                                                   000000002 ? 0119E7860 ?
kkoipt()+901         call     kkocsBindSensCheck(  7F484E2EFA20 ? 7FFD6CB9F640 ?
                              )                    0BBED4798 ? 209DFEA60 ?
                                                   000000002 ? 0119E7860 ?
kkoqbc()+4677        call     kkoipt()             7F484E31F1E0 ? 7FFD6CB9F640 ?
                                                   0BBED4798 ? 209DFEA60 ?
                                                   000000002 ? 000000000
apakkoqb()+183       call     kkoqbc()             7F484E31F1E0 ? 7F4853CC2958 ?
                                                   0BBED4798 ? 209DFEA60 ?
                                                   000000001 000000000 ?
apaqbdDescendents()  call     apakkoqb()           7FFD6CBA1870 ? 000000EB5 ?
+503                                               0BBED4C18 ? 209DFEA60 ?
                                                   000000001 ? 000000000 ?
apaqbd()+136         call     apaqbdDescendents()  7FFD6CBA1870 ? 7F4853CC2958 ?
                                                   0BBED4C18 ? 209DFEA60 ?
                                                   000000001 ? 000000000 ?
apadrv()+1228        call     apaqbd()             7FFD6CBA1870 ? 7F4853CC2958 ?
                                                   0BBED4C18 ? 209DFEA60 ?
                                                   000000001 ? 000000000 ?
opitca()+2139        call     apadrv()             0BBED4C18 ? 7F4853CC2958 ?
                                                   0BBED4C18 ? 209DFEA60 ?
                                                   000000001 ? 000000000 ?
kksFullTypeCheck()+  call     opitca()             7F484E2F2A08 0BBED4C18
86                                                 7FFD6CBA42F0 209DFEA60 ?
                                                   000000001 ? 000000000 ?
rpiswu2()+627        call     kksFullTypeCheck()   7FFD6CBA2FF8 ? 0BBED4C18 ?
                                                   7FFD6CBA42F0 ? 209DFEA60 ?
                                                   000000001 ? 000000000 ?
kksLoadChild()+8003  call     rpiswu2()            7FFD6CBA2FF8 ? 0BBED4C18 ?
                                                   7FFD6CBA42F0 ? 209DFEA60 ?
                                                   7FFD6CBA24F8 ? 00000006A ?
kxsGetRuntimeLock()  call     kksLoadChild()       7F4853DF49A0 0BBED4C18 ?
+2029                                              7FFD6CBA42F0 ? 209DFEA60 ?
                                                   7FFD6CBA24F8 ? 00000006A ?
kksfbc()+15091       call     kxsGetRuntimeLock()  7F4853DF49A0 7F484E2F2A08
                                                   7FFD6CBA4270 209DFEA60 ?
                                                   7FFD6CBA24F8 ? 1C4782708
opiexe()+2932        call     kksfbc()             7F484E2F2A08 7F484E2F2A08 ?
                                                   7FFD6CBA4270 ? 209DFEA60 ?
                                                   7FFD6CBA24F8 ? 1C4782708 ?
kpoal8()+2679        call     opiexe()             000000049 7F484E2F2A08 ?
                                                   7FFD6CBA58F0 209DFEA60 ?
                                                   7FFD6CBA24F8 ? 1C4782708 ?
opiodr()+1229        call     kpoal8()             00000005E 000000026
                                                   7FFD6CBA9300 209DFEA60 ?
                                                   7FFD6CBA24F8 ? 1C4782708 ?
ttcpip()+1257        call     opiodr()             00000005E 000000026
                                                   7FFD6CBA9300 000000000
                                                   7FFD6CBA24F8 ? 1C4782708 ?
opitsk()+1940        call     ttcpip()             7F4853E11750 ? 000000026 ?
                                                   7FFD6CBA9300 00000017D ?
                                                   7FFD6CBA8D50 7FFD6CBA955C
opiino()+941         call     opitsk()             000000000 000000000
                                                   7FFD6CBA9300 ? 00000017D ?
                                                   7FFD6CBA8D50 ? 7FFD6CBA955C ?
opiodr()+1229        call     opiino()             00000003C 000000004
                                                   7FFD6CBAAA18 00000017D ?
                                                   7FFD6CBA8D50 ? 7FFD6CBA955C ?
opidrv()+1021        call     opiodr()             00000003C 000000004
                                                   7FFD6CBAAA18 000000000
                                                   7FFD6CBA8D50 ? 7FFD6CBA955C ?
sou2o()+145          call     opidrv()             00000003C 000000004
                                                   7FFD6CBAAA18 000000000 ?
                                                   7FFD6CBA8D50 ? 7FFD6CBA955C ?
opimai_real()+455    call     sou2o()              7FFD6CBAA9F0 00000003C
                                                   000000004 7FFD6CBAAA18
                                                   7FFD6CBA8D50 ? 7FFD6CBA955C ?
ssthrdmain()+417     call     opimai_real()        000000000 7FFD6CBAAD00
                                                   000000004 ? 7FFD6CBAAA18 ?
                                                   7FFD6CBA8D50 ? 7FFD6CBA955C ?
main()+262           call     ssthrdmain()         000000000 000000002
                                                   7FFD6CBAAD00 000000001
                                                   000000000 7FFD6CBA955C ?
__libc_start_main()  call     main()               000000000 7FFD6CBAAF38
+245                                               7FFD6CBAAD00 ? 000000001 ?
                                                   000000000 ? 7FFD6CBA955C ?
_start()+41          call     __libc_start_main()  000D04D40 000000002
                                                   7FFD6CBAAF38 7F484F241555 ?
                                                   000000000 ? 7FFD6CBA955C ?
--------------------- Binary Stack Dump ---------------------

根据官方描述,该问题是由于Bug 20917487 – CORRUPT KKOCS STRUCTURES AFTER PARTIAL PURGE OF CURSOR HEAP引起
解决方案:
1. 升级数据库到19.1+版本
2. 应用 Patch 20917487
3. Disable adaptive cursor sharing

alter system set "_optimizer_adaptive_cursor_sharing" = false;
alter system set "_optimizer_extended_cursor_sharing_rel" = "none";

4. 通过隐含参数设置一个足够大的值

alter system set "_kgl_large_heap_warning_threshold"=83886080 scope=spfile; --一个足够大的值
--or
alter system set "_kgl_large_heap_warning_threshold"=0 scope=spfile;
alter system set "_kgl_large_heap_assert_threshold"=0 scope=spfile;

shutdown immediate
startup

参考文档:
ORA-600 [KGL-heap-size-exceeded] Errors in the Alert Log (Doc ID 2628072.1)
Memory Notification: Library Cache Object loaded into SGA / ORA-600 [KGL-heap-size-exceeded] (Doc ID 330239.1)

ORA-21779错误处理

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:ORA-21779错误处理

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

有客户win 10.2.0.4的rac,查看alert日志发现如下错误

Mon May 04 17:25:18 2020
Errors in file d:\oracle\product\10.2.0\admin\orcl\bdump\orcl2_smon_2548.trc:
ORA-21779: 持续时间不活动
ORA-06512: 在 line 1

Mon May 04 17:25:28 2020
Errors in file d:\oracle\product\10.2.0\admin\orcl\bdump\orcl2_smon_2548.trc:
ORA-21779: 持续时间不活动
ORA-06512: 在 line 1

Mon May 04 17:25:38 2020
Errors in file d:\oracle\product\10.2.0\admin\orcl\bdump\orcl2_smon_2548.trc:
ORA-21779: 持续时间不活动
ORA-06512: 在 line 1

Mon May 04 17:25:39 2020
Errors in file d:\oracle\product\10.2.0\admin\orcl\bdump\orcl2_smon_2548.trc:
ORA-21779: 持续时间不活动
ORA-06512: 在 line 1

查看对应trace

Dump file d:\oracle\product\10.2.0\admin\orcl\bdump\orcl2_smon_2548.trc
Sat Aug 31 15:02:39 2019
ORACLE V10.2.0.4.0 - 64bit Production vsnsta=0
vsnsql=14 vsnxtr=3
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Data Mining
and Real Application Testing options
Windows Server 2003 Version V5.2 Service Pack 2
CPU                 : 32 - type 8664, 2 Physical Cores
Process Affinity    : 0x0000000000000000
Memory (Avail/Total): Ph:17543M/32742M, Ph+PgF:19550M/33833M

…………

*** 2020-05-04 18:40:35.687
SMON: following errors trapped and ignored:
ORA-21779: 持续时间不活动
ORA-06512: 在 line 1
*** 2020-05-04 18:40:45.734
	 Drop transient type:   SYSTPzpEA6olJSImGURLObkiE6w==
*** 2020-05-04 18:40:45.734
SMON: following errors trapped and ignored:
ORA-21779: 持续时间不活动
ORA-06512: 在 line 1
*** 2020-05-04 18:40:55.812
	 Drop transient type:   SYSTPzpEA6olJSImGURLObkiE6w==
*** 2020-05-04 18:40:55.812
SMON: following errors trapped and ignored:
ORA-21779: 持续时间不活动
ORA-06512: 在 line 1
*** 2020-05-04 18:41:05.875
	 Drop transient type:   SYSTPzpEA6olJSImGURLObkiE6w==
*** 2020-05-04 18:41:05.875
SMON: following errors trapped and ignored:
ORA-21779: 持续时间不活动
ORA-06512: 在 line 1

出现该问题是由于oracle的smon进程无法清理掉 transient types,从而出现该问题,根据官方的说法,这个错误是不会影响数据库正常使用,但是可以通过以下方法暂时规避这种错误:
1)通过设置alter system set events ’22834 trace name context forever, level 1′禁止smon清理transient types,从而来规避该错误,但是可能会引起transient types对象越来越多,当然你可以通过以下sql查询出来

select o.* from obj$ o, type$ t
where o.oid$ = t.tvoid and
bitand(t.properties,8388608) = 8388608 and (sysdate-o.ctime) > 0.0007;

然后删除掉相关记录DROP TYPE “SYSTPf/r2wN4keX7gQKjA3AFMSw==” FORCE;【这个删除不是必须的】
2) flush shared_pool也可以临时规避这个问题
3) 重启数据库,可以暂时规避

具体参考:
SMON: Following Errors Trapped And Ignored ORA-21779 (Doc ID 988663.1)
Receiving ORA-21780 Continuously in the Alert Log and SMON Trace Reports “Drop transient type”. (Doc ID 1081950.1)

Oracle数据库O/S-Error: (OS 1453)错误

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:Oracle数据库O/S-Error: (OS 1453)错误

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

客户数据库在运行过程中突然报错

Fri Apr 03 15:00:17 2020
Thread 1 cannot allocate new log, sequence 8201
Private strand flush not complete
  Current log# 1 seq# 8200 mem# 0: D:\APP\ADMINISTRATOR\ORADATA\ORCL\REDO01.LOG
Thread 1 advanced to log sequence 8201 (LGWR switch)
  Current log# 2 seq# 8201 mem# 0: D:\APP\ADMINISTRATOR\ORADATA\ORCL\REDO02.LOG
Fri Apr 03 20:25:06 2020
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_ckpt_2292.trc:
ORA-00202: control file: 'D:\APP\ADMINISTRATOR\ORADATA\ORCL\CONTROL01.CTL'
ORA-27070: async read/write failed
OSD-04006: ReadFile() 失败, 无法读取文件
O/S-Error: (OS 1453) 配额不足,无法完成请求的服务。
Fri Apr 03 20:27:32 2020
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_m000_3754676.trc:
ORA-00202: control file: 'D:\APP\ADMINISTRATOR\ORADATA\ORCL\CONTROL01.CTL'
ORA-27070: async read/write failed
OSD-04006: ReadFile() 失败, 无法读取文件
O/S-Error: (OS 1453) 配额不足,无法完成请求的服务。
Fri Apr 03 20:28:42 2020
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_ckpt_2292.trc:
ORA-00206: error in writing (block 3, # blocks 1) of control file
ORA-00202: control file: 'D:\APP\ADMINISTRATOR\FLASH_RECOVERY_AREA\ORCL\CONTROL02.CTL'
ORA-27072: File I/O error
OSD-04008: WriteFile() 失败, 无法写入文件
O/S-Error: (OS 1453) 配额不足,无法完成请求的服务。
ORA-00206: error in writing (block 3, # blocks 1) of control file
ORA-00202: control file: 'D:\APP\ADMINISTRATOR\ORADATA\ORCL\CONTROL01.CTL'
ORA-27072: File I/O error
OSD-04008: WriteFile() 失败, 无法写入文件
O/S-Error: (OS 1453) 配额不足,无法完成请求的服务。
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_ckpt_2292.trc:
ORA-00221: error on write to control file
ORA-00206: error in writing (block 3, # blocks 1) of control file
ORA-00202: control file: 'D:\APP\ADMINISTRATOR\FLASH_RECOVERY_AREA\ORCL\CONTROL02.CTL'
ORA-27072: File I/O error
OSD-04008: WriteFile() 失败, 无法写入文件
O/S-Error: (OS 1453) 配额不足,无法完成请求的服务。
ORA-00206: error in writing (block 3, # blocks 1) of control file
ORA-00202: control file: 'D:\APP\ADMINISTRATOR\ORADATA\ORCL\CONTROL01.CTL'
ORA-27072: File I/O error
OSD-04008: WriteFile() 失败, 无法写入文件
O/S-Error: (OS 1453) 配额不足,无法完成请求的服务。
CKPT (ospid: 2292): terminating the instance due to error 221
Fri Apr 03 20:28:42 2020
opiodr aborting process unknown ospid (3753944) as a result of ORA-1092

这个错误提示io error,但是有O/S-Error: (OS 1453)的错误,根据经验判断很可能不是真的io错误,查询mos发现相关记录,发现在How To Resolve (OS 1453) Insufficient Quota To Complete The Requested Service Errors (Doc ID 758595.1)文章中有类似描述.
20200410124105


是由于oracle请求的过程中发现该应用可以使用的物理内存不足导致.该数据库恰好是32位的,最简单的做法就是换64位数据库,可以大大的避免这种错误发生

SQL> select * from v$version;

BANNER
--------------------------------------------------------------------------------

Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - Production
PL/SQL Release 11.2.0.1.0 - Production
CORE    11.2.0.1.0      Production
TNS for 32-bit Windows: Version 11.2.0.1.0 - Production
NLSRTL Version 11.2.0.1.0 - Production

ORA-04020导致adg异常

联系:手机/微信(+86 17813235971) QQ(107644445)

标题:ORA-04020导致adg异常

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

今日早上有客户反馈adg停止同步了,通过检查alert日志发现

Tue Dec 24 18:17:41 2019
Media Recovery Waiting for thread 1 sequence 56655 (in transit)
Recovery of Online Redo Log: Thread 1 Group 11 Seq 56655 Reading mem 0
  Mem# 0: Y:\ORACLE\ORADATA\ORACLE11\STD_REDO11.LOG
Archived Log entry 56248 added for thread 1 sequence 56654 ID 0x5b6bcf9b dest 1:
Tue Dec 24 18:18:11 2019
Errors in file C:\APP\ADMINISTRATOR\diag\rdbms\oracle11dg\oracle11\trace\oracle11_lgwr_3252.trc:
ORA-04020: deadlock detected while trying to lock object SYS.orcl
LGWR (ospid: 3252): terminating the instance due to error 4020
Tue Dec 24 18:18:11 2019
System state dump requested by (instance=1, osid=3252 (LGWR)), summary=[abnormal instance termination].
System State dumped to trace file C:\APP\ADMINISTRATOR\diag\rdbms\oracle11dg\trace\oracle11_diag_3236_20191224181811.trc
Dumping diagnostic data in directory=[cdmp_20191224181811],
      requested by (instance=1, osid=3252 (LGWR)), summary=[abnormal instance termination].
Instance terminated by LGWR, pid = 3252

由于lgwr进程遭遇ORA-04020,从而使得lgwr进程异常,进而整个数据库crash.

分析trace文件

Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
Windows NT Version V6.1 Service Pack 1
CPU                 : 8 - type 8664, 8 Physical Cores
Process Affinity    : 0x0x0000000000000000
Memory (Avail/Total): Ph:8395M/32733M, Ph+PgF:41002M/65464M
Instance name: oracle11
Redo thread mounted by this instance: 1
Oracle process number: 10
Windows thread id: 3252, image: ORACLE.EXE (LGWR)
*** 2019-12-24 18:18:11.072
*** SESSION ID:(384.1) 2019-12-24 18:18:11.072
*** CLIENT ID:() 2019-12-24 18:18:11.072
*** SERVICE NAME:(SYS$BACKGROUND) 2019-12-24 18:18:11.072
*** MODULE NAME:() 2019-12-24 18:18:11.072
*** ACTION NAME:() 2019-12-24 18:18:11.072
A deadlock among DDL and parse locks is detected.
This deadlock is usually due to user errors in
the design of an application or from issuing a set
of concurrent statements which can cause a deadlock.
This should not be reported to Oracle Support.
The following information may aid in finding
the errors which cause the deadlock:
ORA-04020: deadlock detected while trying to lock object SYS.orcl
--------------------------------------------------------
  object   waiting  waiting       blocking blocking
  handle   session     lock mode   session     lock mode
--------  -------- -------- ----  -------- -------- ----
0000000676C20F08  000000066D22BE10 00000006738AB970    X  000000066D22BE10 00000006738A04B0    S
0000000677DF2E80  00000006792E2880 0000000673B13AE8    X  000000066D22BE10 00000006738A19B8    S
--------------------------------------------------------
---------- DUMP OF WAITING AND BLOCKING LOCKS ----------
--------------------------------------------------------
------------- WAITING LOCK -------------
----------------------------------------
SO: 0x00000006738AB970, type: 78, owner: 0x000000065D440498, flag: INIT/-/-/0x00 if: 0x3 c: 0x3
 proc=0x000000066CDE7AD0, name=LIBRARY OBJECT LOCK, file=kgl.h LINE:8751, pg=0
LibraryObjectLock:Address=00000006738AB970 Handle=0000000676C20F08
  RequestMode=X CanBeBrokenCount=2 Incarnation=3 ExecutionCount=0
………………
SO: 0x00000006738A19B8, type: 78, owner: 0x000000065A38D6C0, flag: INIT/-/-/0x00 if: 0x3 c: 0x3
 proc=0x000000066CDE7AD0, name=LIBRARY OBJECT LOCK, file=kgl.h LINE:8751, pg=0
LibraryObjectLock:Address=00000006738A19B8 Handle=0000000677DF2E80
   Mode=S CanBeBrokenCount=1 Incarnation=1 ExecutionCount=0
  User=000000066D22BE10 Session=000000066D22BE10 ReferenceCount=1 Flags=CNB/[0001] SavepointNum=1b749
  LibraryHandle:  Address=0000000677DF2E80 Hash=675351da LockMode=S PinMode=0 LoadLockMode=0 Status=0
  ObjectName:  Name=SYS.orcl
    FullHashValue=285b654fe3f440652c403c98675351da Namespace=DBINSTANCE(74) Type=CURSOR(00) Identifier=1 OwnerIdn=0
  Statistics:  InvalidationCount=0 ExecutionCount=0 LoadCount=0 ActiveLocks=1 TotalLockCount=74719 TotalPinCount=0
  Counters:  BrokenCount=1 RevocablePointer=1 KeepDependency=0 Version=0
        BucketInUse=2 HandleInUse=2 HandleReferenceCount=0
  Concurrency:  DependencyMutex=0000000677DF2F30(0, 0, 0, 0) Mutex=0000000677DF2FC0(0, 149471, 1, 0)
  Flags=RON/PIN/KEP/BSO/[00810003]
  WaitersLists:
    Lock=0000000677DF2F10[0000000673B13B58,000000067382E2F0]
    Pin=0000000677DF2EF0[0000000677DF2EF0,0000000677DF2EF0]
    LoadLock=0000000677DF2F68[0000000677DF2F68,0000000677DF2F68]
  Timestamp:
  HandleReference:  Address=0000000677DF3030 Handle=0000000000000000 Flags=[00] ---------------------------------
This lock request was aborted.
error 4020 detected in background process
ORA-04020: deadlock detected while trying to lock object SYS.orcl
kjzduptcctx: Notifying DIAG for crash event
----- Abridged Call Stack Trace -----
ksedsts()+585<-kjzdssdmp()+329<-kjzduptcctx()+288<-kjzdicrshnfy()+99<-ksuitm()+1525<-ksbrdp()+4578<-opirip()
+853<-opidrv()+906<-sou2o()+98<-opimai_real()+280<-opimai()+191
    <-BackgroundThreadStart()+646<-0000000076CF59CD<-0000000076E2A561
----- End of Abridged Call Stack Trace -----
*** 2019-12-24 18:18:11.165
LGWR (ospid: 3252): terminating the instance due to error 4020
*** 2019-12-24 18:18:17.483
ksuitm: waiting up to [5] seconds before killing DIAG(3236)

日志显示由于lgwr进程等待LIBRARY OBJECT LOCK超时,从而引起异常,根据经验此类问题一般是由于bug导致,查询mos发现匹配bug信息Bug 18515268 ORA-4020 in ADG Standby Database causing instance crash by LGWR
20191225112622


可以根据需要打上相关Patch 18515268: ACTIVE DATAGUARD STANDBY CRASHES DUE TO AN ORA-4020 ENCOUNTERED BY LGWR

ORA-00600: internal error code, arguments: [2252], [3987]

联系:手机/微信(+86 17813235971) QQ(107644445)

标题:ORA-00600: internal error code, arguments: [2252], [3987]

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

客户数据库版本10.2.0.4,启动成功之后立马crash,让我们协助解决
version


Thu Jul  4 13:03:10 2019
Completed: ALTER DATABASE OPEN
Thu Jul  4 13:03:10 2019
db_recovery_file_dest_size of 2048 MB is 0.00% used. This is a
user-specified limit on the amount of space that will be used by this
database for recovery-related files, and does not reflect the amount of
space available in the underlying filesystem or ASM diskgroup.
Thu Jul  4 13:04:01 2019
Errors in file /oracle/app/oracle/admin/orcl/bdump/orcl_reco_22268.trc:
ORA-00600: internal error code, arguments: [2252], [3987], [3375047096], [], [], [], [], []
Thu Jul  4 13:04:01 2019
Errors in file /oracle/app/oracle/admin/orcl/bdump/orcl_reco_22268.trc:
ORA-00600: internal error code, arguments: [2252], [3987], [3375047096], [], [], [], [], []
Thu Jul  4 13:04:02 2019
Errors in file /oracle/app/oracle/admin/orcl/bdump/orcl_reco_22268.trc:
ORA-00600: internal error code, arguments: [2252], [3987], [3375047096], [], [], [], [], []
Thu Jul  4 13:04:02 2019
RECO: terminating instance due to error 476
Instance terminated by RECO, pid = 22268

根据以往经验记录一次ORA-00600[2252]故障解决,很可能是scn过大引起.通过Oracle数据库异常恢复检查脚本(Oracle Database Recovery Check)检查scn相关信息
scn


从ORA-600 2252错误信息看,由于scn可能超过该数据库的天花板理论上而导致该问题,而reco进程主要是由于分布式事务引起,通过和客户确认,该库有通过dblink去访问11204版本oracle,而从2019年6月23日之后scn的算法发生了一些改变(SCN Compatibility问题汇总-2019年6月23日),导致数据库可以支持更大的scn,从而当低版本需要进行分布式事务操作之时,可能导致数据库异常.

处理方案:通过临时屏蔽分布式事务,让数据库临时正常工作;长期解决方案需要把数据库版本升级,避免scn引起相关问题

ORA-07217: sltln: environment variable cannot be evaluated.

联系:手机/微信(+86 17813235971) QQ(107644445)

标题:ORA-07217: sltln: environment variable cannot be evaluated.

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

dg备库mount报ORA-07217
在一次搭建dg过程中,备库无法mount,报ORA-07217: sltln: environment variable cannot be evaluated.错误

[oracle@localhost orcl]$ sqlplus / as sysdba
SQL*Plus: Release 11.2.0.4.0 Production on Wed Feb 13 22:27:31 2019
Copyright (c) 1982, 2013, Oracle.  All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
SQL> alter database mount;
alter database mount
*
ERROR at line 1:
ORA-07217: sltln: environment variable cannot be evaluated.

alert日志报错

Set as converted control file due to db_unique_name mismatch
Changing di2dbun from ORCL to ORCLDG
ORA-7217 signalled during: alter database mount...
Wed Feb 13 20:33:41 2019
alter database mount
ORA-7217 signalled during: alter database mount...
Wed Feb 13 20:36:11 2019
alter database mount
ORA-7217 signalled during: alter database mount...

根据mos排查
根据mos ORA-07217: sltln: environment variable cannot be evaluated (Doc ID 2487898.1)描述排查,确认不是该问题

SQL> show parameter diag;
NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
diagnostic_dest                      string      /u01/app/oracle
[oracle@dg:/opt/app/oracle/arch]$env|grep ORA
ORACLE_SID=orcl
ORACLE_BASE=/opt/app/oracle
ORACLE_TERM=xterm
ORACLE_HOME=/opt/app/oracle/product/11.2.0/db_1

检查发现数据文件异常

SQL> select name from v$datafile where name like '%$%';
NAME
------------------------------------------------------
+DATA/orcl/datafile/ts_his$emr01
+DATA/orcl/datafile/ts_his$emr02

通过分析,注释掉db_file_name_convert=’+DATA/orcl/datafile/’,’/u01/app/oradata/orcl/’即可正常mount成功,数据文件命令一定要规范(不要有特殊符号),不然给后人和自己留下巨坑

下调虚拟化资源导致ORA-00494

联系:手机/微信(+86 17813235971) QQ(107644445)

标题:下调虚拟化资源导致ORA-00494

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

在虚拟化环境中的Oracle数据库近期报大量ORA-00494: 入队 [CF] 被持有的时间过长 (more than 900 seconds) 错误,每次客户重启一段时间之后又不行

Mon Nov 26 14:04:39 中国标准时间 2018
Thread 1 advanced to log sequence 97327 (LGWR switch)
  Current log# 1 seq# 97327 mem# 0: D:\ORADATA\xifenfei\REDO01.LOG
Mon Nov 26 14:20:02 中国标准时间 2018
Errors in file c:\oracle\product\10.2.0\admin\xifenfei\bdump\xifenfei_arc1_1860.trc:
ORA-00494: 入队 [CF] 被持有的时间过长 (more than 900 seconds) (由 'inst 1, osid 3264')
Mon Nov 26 14:20:03 中国标准时间 2018
Errors in file c:\oracle\product\10.2.0\admin\xifenfei\bdump\xifenfei_arc4_1872.trc:
ORA-00494: 入队 [CF] 被持有的时间过长 (more than 900 seconds) (由 'inst 1, osid 3264')
Mon Nov 26 14:20:03 中国标准时间 2018
Errors in file c:\oracle\product\10.2.0\admin\xifenfei\bdump\xifenfei_arc3_1868.trc:
ORA-00494: 入队 [CF] 被持有的时间过长 (more than 900 seconds) (由 'inst 1, osid 3264')
Mon Nov 26 14:20:03 中国标准时间 2018
Errors in file c:\oracle\product\10.2.0\admin\xifenfei\bdump\xifenfei_arc0_1856.trc:
ORA-00494: 入队 [CF] 被持有的时间过长 (more than 900 seconds) (由 'inst 1, osid 3264')

因为这个系统有一段历史,以前是客户的核心生产库,虚拟化资源给的比较多128G,后来核心生产迁移到其他系统,该系统跑一些边缘小业务,所以怀疑客户有可能下调了虚拟化的资源(cpu和内存),检查机器硬件资源

C:\Users\Administrator>systeminfo
主机名:           HIS-VM
OS 名称:          Microsoft Windows Server 2008 R2 Enterprise
OS 版本:          6.1.7601 Service Pack 1 Build 7601
OS 制造商:        Microsoft Corporation
OS 配置:          独立服务器
OS 构件类型:      Multiprocessor Free
注册的所有人:     Windows 用户
注册的组织:
产品 ID:          00486-OEM-8400691-20006
初始安装日期:     2013/4/19, 12:22:44
系统启动时间:     2018/11/27, 8:06:00
系统制造商:       VMware, Inc.
系统型号:         VMware Virtual Platform
系统类型:         x64-based PC
处理器:           安装了 2 个处理器。
                  [01]: Intel64 Family 6 Model 45 Stepping 2 GenuineIntel
Mhz
                  [02]: Intel64 Family 6 Model 45 Stepping 2 GenuineIntel
Mhz
BIOS 版本:        Phoenix Technologies LTD 6.00, 2014/4/14
Windows 目录:     C:\Windows
系统目录:         C:\Windows\system32
启动设备:         \Device\HarddiskVolume1
系统区域设置:     zh-cn;中文(中国)
输入法区域设置:   zh-cn;中文(中国)
时区:             (UTC+08:00) 北京,重庆,香港特别行政区,乌鲁木齐
物理内存总量:     32,767 MB
可用的物理内存:   22,740 MB
虚拟内存: 最大值: 95,047 MB
虚拟内存: 可用:   10,426 MB
虚拟内存: 使用中: 84,621 MB
页面文件位置:     C:\pagefile.sys
域:               WORKGROUP

果然系统内存从128G下调到了32G,但是数据库sga估计没有对应降低

SQL> show parameter sga;
NAME                                 TYPE        VALUE
------------------------------------ ----------- ------
lock_sga                             boolean     FALSE
pre_page_sga                         boolean     FALSE
sga_max_size                         big integer 80G
sga_target                           big integer 80G

基本上就是由于虚拟机内存下调,但是数据库sga配置过大导致该问题.