admin – 第18页 – Database SOS

osw配置私网监控(oswprvtnet)

OS Watcher Black Box (OSWbb) 是一种 UNIX shell 脚本的集合，主要用于收集和归档操作系统和网络的度量，以便为诊断性能问题提供支持。OSWbb 作为服务器上的一组后台进程来运行，定期收集OS 数据，其调用 Unix 实用程序，如 vmstat、netstat 和 iostat 等。在最近的gi psu中已经把osw包含在tfa中了，但是默认不启用私网的监控，如果需要可以进行人工配置
OSWbb 使用 traceroute 命令获取这些私有网络的状态。每个操作系统对 traceroute 命令使用的参数都略有不同。基本 oswbb 目录中的示例Exampleprivate.net 文件中包含每个操作系统使用的语法。因此，这将导致在 UNIX 平台中的输出显示有所不同。OSWbb 按指定间隔运行 private.net 文件，并在归档目录下的 oswprvtnet 子目录中存储数据。数据存储在每小时归档的文件中。文件中的每个条目都包含一个时间戳，前缀为 ***，嵌入在顶部输出中。
tfa中含osw

[root@xifenfei01 ~]# ps -ef|grep osw
grid      17673      1  0 Jan30 ?        00:00:07 /bin/sh ./OSWatcher.sh 30 48 NONE /u01/app/grid/tfa/repository/suptools/xifenfei01/oswbb/grid/archive
grid      17841  17673  0 Jan30 ?        00:00:01 /bin/sh ./OSWatcherFM.sh 48 /u01/app/grid/tfa/repository/suptools/xifenfei01/oswbb/grid/archive
root      64159  64120  0 00:30 pts/0    00:00:00 grep osw

测试prvtnet.sh

[root@xifenfei01 oswbb]# cd /u01/app/grid/tfa/repository/suptools/xifenfei01/oswbb/grid/oswbb
[root@xifenfei01 oswbb]# more prvtnet.sh
echo "zzz ***"`date`
traceroute -r -F xifenfei01-priv
traceroute -r -F xifenfei02-priv
######################################################################
# DO NOT DELETE THE FOLLOWING LINE!!!!!!!!!!!!!!!!!!!!!
######################################################################
rm locks/lock.file
[root@xifenfei01 oswbb]# ./prvtnet.sh
zzz ***Tue Jan 30 18:02:49 CST 2018
traceroute to xifenfei01-priv (192.168.102.11), 30 hops max, 60 byte packets
 1  xifenfei01-priv (192.168.102.11)  0.010 ms  0.003 ms  0.002 ms
traceroute to xifenfei02-priv (192.168.102.14), 30 hops max, 60 byte packets
 1  xifenfei02-priv (192.168.102.14)  0.081 ms  0.076 ms  0.087 ms
rm: cannot remove `locks/lock.file': No such file or directory

配置private.net正式收集私网信息

[root@xifenfei01 oswbb]# find . -name OSWatcher.sh | xargs grep -i "private"
# Run traceroute for private networks if file private.net exists
if [ -x private.net ]; then
   ./private.net >> $OSWBB_ARCHIVE_DEST/oswprvtnet/${hostn}_prvtnet_$hour 2>&1 &

这里重要的是：
./OSWatcher.sh:# Run traceroute for private networks if file private.net exists
表示，在OSWatcher.sh中调用的文件名称是private.net
将prvtnet.sh复制为供OSWatcher.sh脚本调用的文件名称

[root@xifenfei01 oswbb]# cp prvtnet.sh private.net
[root@xifenfei01 oswbb]# chown grid:oinstall prvtnet.net
[root@xifenfei01 oswbb]# chmod +x prvtnet.net

关注内容

示例 1： 接口处于启动和响应状态：
traceroute to X.X.X.X, (X.X.X.X) 30 hops max, 1492 byte packets
1 X.X.X.X 1.015 ms 0.766 ms 0.755 ms
示例 2： 目标接口不是在直接连接的网络上，所以需要验证地址
         是否正确或其插入的交换机是否位于相同的 VLAN 上（或其他问题）：
traceroute to X.X.X.X, (X.X.X.X) 30 hops max, 40 byte packets
traceroute: host X.X.X.X is not on a directly-attached network
示例 3： 网络无法访问：
traceroute to X.X.X.X, (X.X.X.X) 30 hops max, 40 byte packets
Network is unreachable
示例 4： 网络无法访问：
xifenfei01_prvtnet_18.01.30.2000.dat-zzz ***Tue Jan 30 20:27:43 CST 2018
xifenfei01_prvtnet_18.01.30.2000.dat-traceroute to xifenfei01-priv (192.168.102.11), 30 hops max, 60 byte packets
xifenfei01_prvtnet_18.01.30.2000.dat- 1  xifenfei01-priv (192.168.102.11)  0.014 ms  0.004 ms  0.005 ms
xifenfei01_prvtnet_18.01.30.2000.dat-traceroute to xifenfei02-priv (192.168.102.14), 30 hops max, 60 byte packets
xifenfei01_prvtnet_18.01.30.2000.dat- 1  * * *
xifenfei01_prvtnet_18.01.30.2000.dat- 2  * * *
xifenfei01_prvtnet_18.01.30.2000.dat- 3  * * *
xifenfei01_prvtnet_18.01.30.2000.dat- 4  * * *
xifenfei01_prvtnet_18.01.30.2000.dat- 5  * * *
xifenfei01_prvtnet_18.01.30.2000.dat- 6  * * *
xifenfei01_prvtnet_18.01.30.2000.dat- 7  * * *
xifenfei01_prvtnet_18.01.30.2000.dat- 8  * * *
xifenfei01_prvtnet_18.01.30.2000.dat- 9  * * *
xifenfei01_prvtnet_18.01.30.2000.dat-10  * * *
xifenfei01_prvtnet_18.01.30.2000.dat-11  * * *
xifenfei01_prvtnet_18.01.30.2000.dat-12  * * *
xifenfei01_prvtnet_18.01.30.2000.dat-13  * * *
xifenfei01_prvtnet_18.01.30.2000.dat-14  * * *
xifenfei01_prvtnet_18.01.30.2000.dat-15  * * *
xifenfei01_prvtnet_18.01.30.2000.dat-16  * * *
xifenfei01_prvtnet_18.01.30.2000.dat-17  * * *
xifenfei01_prvtnet_18.01.30.2000.dat-18  * * *
xifenfei01_prvtnet_18.01.30.2000.dat-19  * * *
xifenfei01_prvtnet_18.01.30.2000.dat-20  * * *
xifenfei01_prvtnet_18.01.30.2000.dat-21  * * *
xifenfei01_prvtnet_18.01.30.2000.dat-22  * * *
xifenfei01_prvtnet_18.01.30.2000.dat-23  * * *
xifenfei01_prvtnet_18.01.30.2000.dat-24  * * *
xifenfei01_prvtnet_18.01.30.2000.dat-25  * * *
xifenfei01_prvtnet_18.01.30.2000.dat-26  * * *
xifenfei01_prvtnet_18.01.30.2000.dat-27  * * *
xifenfei01_prvtnet_18.01.30.2000.dat-28  * * *
xifenfei01_prvtnet_18.01.30.2000.dat-29  * * *
xifenfei01_prvtnet_18.01.30.2000.dat-30  * * *
xifenfei01_prvtnet_18.01.30.2000.dat:zzz ***Warning. Traceroute response is spanning snapshot intervals.

reliable message等待

在一次给客户的rac poc测试中，发现大量reliable message等待，严重影响性能
数据库版本
11.2.0.4+最新psu(180116)

SYS@xffdb1>select * from v$version;
BANNER
--------------------------------------------------------------------------------
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
PL/SQL Release 11.2.0.4.0 - Production
CORE    11.2.0.4.0      Production
TNS for Linux: Version 11.2.0.4.0 - Production
NLSRTL Version 11.2.0.4.0 - Production
Elapsed: 00:00:00.00
SYS@xffdb1>exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
[oracle@xifenfei01 ~]$ opatch lspatches
26609929;OCW Patch Set Update : 11.2.0.4.170814 (26609929)
26925576;Database Patch Set Update : 11.2.0.4.180116 (26925576)
OPatch succeeded.

监控em发现大量other等待

查询发现大量reliable message等待

SYS@xffdb1>@event
         INST_ID USERNAME                       EVENT                                                      COUNT(*)
---------------- ------------------------------ -------------------------------------------------- ----------------
               1 SOE                            reliable message                                                 16
               1 SOE                            gc current request                                                4
               1 SOE                            gc cr request                                                     5
               1 SOE                            db file sequential read                                          17
               1 SYS                            db file scattered read                                            1
               2 SOE                            library cache: mutex X                                            2
               2 SOE                            latch: ges resource hash list                                     2
               2 SOE                            gc current request                                                1
               2 SOE                            gc cr request                                                     4
               2 SOE                            db file sequential read                                          12
               2                                db file parallel write                                            1
               2 SOE                            SQL*Net message to client                                         1
12 rows selected.

进一步查询发现大量Result Cache: Channel

SYS@xffdb1>SELECT CHANNEL,
  2    SUM(wait_count) sum_wait_count
  3  FROM GV$CHANNEL_WAITS
  4  GROUP BY CHANNEL
  5  ORDER BY SUM(wait_count) DESC;
CHANNEL                                                            SUM_WAIT_COUNT
---------------------------------------------------------------- ----------------
Result Cache: Channel                                                   379367698
RBR channel                                                                128968
kxfp control signal channel                                                115091
MMON remote action broadcast channel                                         4646
obj broadcast channel                                                         395
LCK0 ksbxic channel                                                            17
parameters to cluster db instances - broadcast channel                          4
service operations - broadcast channel                                          3
kill job broadcast - broadcast channel                                          2

查看result_cache_max_size参数

SYS@xffdb1>show parameter result_cache_max_size;
NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
result_cache_max_size                big integer 335552K

调整result_cache_max_size参数

SYS@xffdb1>alter system set result_cache_max_size=0;
System altered.
SYS@xffdb1>show parameter result_cache_max_size;
NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
result_cache_max_size                big integer 0

重启数据库，再次测试，发现相关等待消失，系统也更加稳定

SYS@xffdb1>@event
         INST_ID USERNAME                       EVENT                                                      COUNT(*)
---------------- ------------------------------ -------------------------------------------------- ----------------
               1 SOE                            library cache: mutex X                                            1
               1 SOE                            gc current request                                                3
               1 DBSNMP                         gc cr request                                                     1
               1 SOE                            gc cr request                                                     7
               1 SOE                            db file sequential read                                          18
               1                                db file parallel write                                            1
               2 SOE                            library cache: mutex X                                            2
               2 SOE                            latch free                                                        1
               2 SOE                            gc current request                                                4
               2 SOE                            gc cr request                                                     4
               2 SOE                            db file sequential read                                          12
               2                                db file parallel write                                            3
12 rows selected.

参考：Very High Waits for ‘reliable message’ After Upgrade to 11.2.0.4 When Using Result Cache (Doc ID 1951729.1)

重建oraInventory解决ORA-20001

数据库启动报ORA-20001: Latest xml inventory is not loaded into table错误

Completed: ALTER DATABASE OPEN
2018-01-23T23:46:27.924841+08:00
CJQ0 started with pid=54, OS id=6653
2018-01-23T23:46:31.705550+08:00
Unable to obtain current patch information due to error: 20001,
  ORA-20001: Latest xml inventory is not loaded into table
ORA-06512: at "SYS.DBMS_QOPATCH", line 777
ORA-06512: at "SYS.DBMS_QOPATCH", line 864
ORA-06512: at "SYS.DBMS_QOPATCH", line 2222
ORA-06512: at "SYS.DBMS_QOPATCH", line 740
ORA-06512: at "SYS.DBMS_QOPATCH", line 2247
===========================================================
Dumping current patch information
===========================================================
Unable to obtain current patch information due to error: 20001
===========================================================

查询相关sql报错

SYS@xffdb>select * from OPATCH_XML_INV ;
ERROR:
ORA-29913: error in executing ODCIEXTTABLEFETCH callout
ORA-29400: data cartridge error
KUP-04004: error while reading file /u03/app/oracle/product/12.2.0.1/dbhome/QOpatch/qopiprep.bat
no rows selected
Elapsed: 00:00:00.58
SYS@xffdb>select xmltransform(dbms_qopatch.get_opatch_lsinventory(), dbms_qopatch.GET_OPATCH_XSLT()) from dual ;
ERROR:
ORA-20001: Latest xml inventory is not loaded into table
ORA-06512: at "SYS.DBMS_QOPATCH", line 777
ORA-06512: at "SYS.DBMS_QOPATCH", line 864
ORA-06512: at "SYS.DBMS_QOPATCH", line 2222
ORA-06512: at "SYS.DBMS_QOPATCH", line 740
ORA-06512: at "SYS.DBMS_QOPATCH", line 2247
no rows selected
Elapsed: 00:00:00.63

datapatch -prereq报错

[oracle@xifenfei ~]$ $ORACLE_HOME/OPatch/datapatch -prereq
SQL Patching tool version 12.2.0.1.0 Production on Tue Jan 23 18:11:32 2018
Copyright (c) 2012, 2017, Oracle.  All rights reserved.
Connecting to database...OK
Note:  Datapatch will only apply or rollback SQL fixes for PDBs
       that are in an open state, no patches will be applied to closed PDBs.
       Please refer to Note: Datapatch: Database 12c Post Patch SQL Automation
       (Doc ID 1585822.1)
Queryable inventory could not determine the current opatch status.
Execute 'select dbms_sqlpatch.verify_queryable_inventory from dual'
and/or check the invocation log
/u03/app/oracle/cfgtoollogs/sqlpatch/sqlpatch_4909_2018_01_23_18_11_32/sqlpatch_invocation.log
for the complete error.
Prereq check failed, exiting without installing any patches.
Please refer to MOS Note 1609718.1 and/or the invocation log
/u03/app/oracle/cfgtoollogs/sqlpatch/sqlpatch_4909_2018_01_23_18_11_32/sqlpatch_invocation.log
for information on how to resolve the above errors.
SQL Patching tool complete on Tue Jan 23 18:11:45 2018

分析qopiprep.bat文件

cd $ORACLE_HOME
PATH=/bin:/usr/bin
export PATH
# sed tried to convert from one encoding to other in presence of LC_ALL
# or LANG settings. Since opatch returning UTF-8 based encoding we do not
# need such a conversion. So safely skip it
LANG=en_US.UTF-8
export LANG
LC_ALL=''
export LC_ALL
# Option: "-retry 0" avoids retries in case of locked inventory.
# Option: "-invPtrLoc" is required for non-central-inventory
# locations. $OPATCH_PREP_LSINV_OPTS which may set by users
# in the environment to configure special OPatch options
# ("-jdk" is another good candidate that may require configuration!).
# Option: "-all" gives information on all Oracle Homes
# installed in the central inventory.  With that information, the
# patches of non-RDBMS homes could be fetched.
DBSID=$ORACLE_SID
ORABASE=`$ORACLE_HOME/bin/orabasehome`
rm -rf $ORABASE/rdbms/log/xml_file_$DBSID.xml
$ORACLE_HOME/OPatch/opatch lsinventory -xml  $ORABASE/rdbms/log/xml_file_$DBSID.xml -retry 0 -invPtrLoc $ORACLE_HOME/oraInst.loc >> $ORABASE/rdbms/log/stout_$DBSID.txt
cat $ORABASE/rdbms/log/xml_file_$DBSID.xml | sed 's/^ *//' | tr '\n' ' '
echo "UIJSVTBOEIZBEFFQBL"
rm $ORABASE/rdbms/log/xml_file_$DBSID.xml
rm $ORABASE/rdbms/log/stout_$DBSID.txt

这里主要是$ORACLE_HOME/OPatch/opatch lsinventory可能异常，测试该功能
qopatch_log日志

[oracle@xifenfei ~]$ tail -f /u03/app/oracle/product/12.2.0.1/dbhome/QOpatch/qopatch_log.log
 LOG file opened at 01/23/18 18:48:55
KUP-05007:   Warning: Intra source concurrency disabled because the preprocessor option is being used.
Field Definitions for table OPATCH_XML_INV
  Record format DELIMITED BY NEWLINE
  Data in file has same endianness as the platform
  Reject rows with all null fields
  Fields in Data Source:
    XML_INVENTORY                   CHAR (100000000)
      Terminated by "UIJSVTBOEIZBEFFQBL"
      Trim whitespace same as SQL Loader
KUP-04004: error while reading file /u03/app/oracle/product/12.2.0.1/dbhome/QOpatch/qopiprep.bat
KUP-04017: OS message: Error 0
KUP-04017: OS message: LsInventorySession failed: RawInventory gets null OracleHomeInfo
cat: /u03/app/oracle/product/12.2.0.1/dbhome/rdbms/log/xml_file_xffdb.xml: No such file or direc
KUP-04118: operation "pipe read", location "skudmir:2"

opatch lsinventory验证

[oracle@xifenfei ~]$ /u03/app/oracle/product/12.2.0.1/dbhome/OPatch/opatch lsinventory
Oracle Interim Patch Installer version 12.2.0.1.6
Copyright (c) 2018, Oracle Corporation.  All rights reserved.
Oracle Home       : /u03/app/oracle/product/12.2.0.1/dbhome
Central Inventory : /u01/app/oraInventory
   from           : /u03/app/oracle/product/12.2.0.1/dbhome/oraInst.loc
OPatch version    : 12.2.0.1.6
OUI version       : 12.2.0.1.4
Log file location : /u03/app/oracle/product/12.2.0.1/dbhome/cfgtoollogs/opatch/opatch2018-01-23_23-50-29PM_1.log
List of Homes on this system:
  Home name= OraDB12Home1, Location= "/u01/app/oracle/product/12.2.0/dbhome_1"
LsInventorySession failed: RawInventory gets null OracleHomeInfo
OPatch failed with error code 73

现在到这一步，可以确定判断opatch lsinventory运行异常，导致DBMS_QOPATCH无法正常工作，而引起opatch异常的原因是由于RawInventory gets null OracleHomeInfo
分析inventory.xml 文件

[oracle@xifenfei ContentsXML]$ cat inventory.xml
<?xml version="1.0" standalone="yes" ?>
<!-- Copyright (c) 1999, 2016, Oracle and/or its affiliates.
All rights reserved. -->
<!-- Do not modify the contents of this file by hand. -->
<INVENTORY>
<VERSION_INFO>
   <SAVED_WITH>12.2.0.1.0</SAVED_WITH>
   <MINIMUM_VER>2.1.0.6.0</MINIMUM_VER>
</VERSION_INFO>
<HOME_LIST>
<HOME NAME="OraDB12Home1" LOC="/u01/app/oracle/product/12.2.0/dbhome_1" TYPE="O" IDX="1"/>
</HOME_LIST>
<COMPOSITEHOME_LIST>
</COMPOSITEHOME_LIST>
</INVENTORY>

因为该机器上安装过三个版本的oracle，12.2 beta，11.2.0.4,12.2.0.1，现在oracle home只有第一个beta的，因此这个部分肯定异常，导致后面的12.2正式版无法获取到oraclehome
重建oraInventory

[oracle@xifenfei app]$ cd $ORACLE_HOME/oui/bin
[oracle@xifenfei bin]$ ./runInstaller -silent -ignoreSysPrereqs -attachHome ORACLE_HOME="/u01/app/oracle/product/12.2.0/dbhome_1" ORACLE_HOME_NAME="OraDB12betaHome1"
Starting Oracle Universal Installer...
Checking swap space: must be greater than 500 MB.   Actual 3935 MB    Passed
The inventory pointer is located at /etc/oraInst.loc
'AttachHome' was successful.
[oracle@xifenfei bin]$ ./runInstaller -silent -ignoreSysPrereqs -attachHome ORACLE_HOME="/u02/app/oracle/product/11.2.0.4/dbhome" ORACLE_HOME_NAME="OraDb11g_home1"
Starting Oracle Universal Installer...
Checking swap space: must be greater than 500 MB.   Actual 3935 MB    Passed
The inventory pointer is located at /etc/oraInst.loc
'AttachHome' was successful.
[oracle@xifenfei bin]$ ./runInstaller -silent -ignoreSysPrereqs -attachHome ORACLE_HOME="/u03/app/oracle/product/12.2.0.1/dbhome" ORACLE_HOME_NAME="OraDb122g_home1"
Starting Oracle Universal Installer...
Checking swap space: must be greater than 500 MB.   Actual 3935 MB    Passed
The inventory pointer is located at /etc/oraInst.loc
'AttachHome' was successful.
--验证inventory.xml 文件
[oracle@xifenfei ContentsXML]$ cat inventory.xml
<?xml version="1.0" standalone="yes" ?>
<!-- Copyright (c) 1999, 2018, Oracle and/or its affiliates.
All rights reserved. -->
<!-- Do not modify the contents of this file by hand. -->
<INVENTORY>
<VERSION_INFO>
   <SAVED_WITH>12.2.0.1.4</SAVED_WITH>
   <MINIMUM_VER>2.1.0.6.0</MINIMUM_VER>
</VERSION_INFO>
<HOME_LIST>
<HOME NAME="OraDB12betaHome1" LOC="/u01/app/oracle/product/12.2.0/dbhome_1" TYPE="O" IDX="1"/>
<HOME NAME="OraDb11g_home1" LOC="/u02/app/oracle/product/11.2.0.4/dbhome" TYPE="O" IDX="2"/>
<HOME NAME="OraDb122g_home1" LOC="/u03/app/oracle/product/12.2.0.1/dbhome" TYPE="O" IDX="3"/>
</HOME_LIST>
<COMPOSITEHOME_LIST>
</COMPOSITEHOME_LIST>
</INVENTORY>

验证opatch lsinventory

[oracle@xifenfei bin]$ opatch lsinv
Oracle Interim Patch Installer version 12.2.0.1.6
Copyright (c) 2018, Oracle Corporation.  All rights reserved.
Oracle Home       : /u03/app/oracle/product/12.2.0.1/dbhome
Central Inventory : /u01/app/oraInventory
   from           : /u03/app/oracle/product/12.2.0.1/dbhome/oraInst.loc
OPatch version    : 12.2.0.1.6
OUI version       : 12.2.0.1.4
Log file location : /u03/app/oracle/product/12.2.0.1/dbhome/cfgtoollogs/opatch/opatch2018-01-24_00-19-55AM_1.log
Lsinventory Output file location : /u03/app/oracle/product/12.2.0.1/dbhome/cfgtoollogs/opatch/lsinv/lsinventory2018-01-24_00-19-55AM.txt
--------------------------------------------------------------------------------
Local Machine Information::
Hostname: xifenfei
ARU platform id: 226
ARU platform description:: Linux x86-64
Installed Top-level Products (1):
Oracle Database 12c                                                  12.2.0.1.0
There are 1 products installed in this Oracle Home.
There are no Interim patches installed in this Oracle Home.
--------------------------------------------------------------------------------
OPatch succeeded.

验证dbms_qopatch工作正常

[oracle@xifenfei ContentsXML]$ $ORACLE_HOME/OPatch/datapatch -prereq
SQL Patching tool version 12.2.0.1.0 Production on Wed Jan 24 00:21:48 2018
Copyright (c) 2012, 2017, Oracle.  All rights reserved.
Connecting to database...OK
Note:  Datapatch will only apply or rollback SQL fixes for PDBs
       that are in an open state, no patches will be applied to closed PDBs.
       Please refer to Note: Datapatch: Database 12c Post Patch SQL Automation
       (Doc ID 1585822.1)
Determining current state...done
Adding patches to installation queue and performing prereq checks...done
Installation queue:
  For the following PDBs: CDB$ROOT PDB$SEED
    Nothing to roll back
    Nothing to apply
SQL Patching tool complete on Wed Jan 24 00:21:55 2018
SYS@xffdb>select dbms_sqlpatch.verify_queryable_inventory from dual;
VERIFY_QUERYABLE_INVENTORY
--------------------------------------------------------------------------------------------------
OK
Elapsed: 00:00:01.03
SYS@xffdb>select xmltransform(dbms_qopatch.get_opatch_lsinventory(), dbms_qopatch.GET_OPATCH_XSLT()) from dual ;
XMLTRANSFORM(DBMS_QOPATCH.GET_OPATCH_LSINVENTORY(),DBMS_QOPATCH.GET_OPATCH_XSLT())
----------------------------------------------------------------------------------------------------------
Oracle Querayable Patch Interface 1.0
-----------------------------------------
Elapsed: 00:00:01.09
SYS@xffdb>

通过修复错误的oraInventory解决ORA-20001问题

18c新特性：Scalable Sequences(自适应序列)

作为18c的新特性,其实在oracle 12.2 引入了Scalable Sequences作为一个隐藏特性，为了改善在高并发系统中，特别是使用seq作为index，大量插入记录导致index的争用，oracle自己实现了以前我们需要人工去自定义seq的方式(instance+sid+seq的类似算法方式）,该功能将在oracle 18c中正式推出,我这里带领大家先体会下
SCALE/NOSCALE
When SCALE is specified, a numeric offset is affixed to the beginning of the sequence. This offset if of the form iii||sss||, where,
iii denotes a three digit instance offset given by (instance_id % 100) + 100,
sss denotes a three digits session offset given by (session_id % 1000), and
|| is the concatenation operator

EXTEND/NOEXTEND
When EXTEND is specified with the SCALE keyword, the generated sequence values are all of length (x+y), where x is the length of the scalable offset (default 6), and y is the maximum number of digits in the sequence maxvalue/minvalue. Thus, for an ascending sequence with maxvalue 100 and SCALABLE EXTEND specified, the generated sequence values are of the form iii||sss||001, iii||sss||002, …,iii||sss||100
The default setting for the SCALE clause is NOEXTEND. With the NOEXTEND setting, the generated sequence values are at most as wide as the maximum number of digits in the sequence maxvalue/minvalue. This setting is useful for integration with existing applications where sequences are used to populate fixed width columns. On invocation of NEXTVAL on a sequence with SCALABLE NOEXTEND specified, a user error is thrown if the generated value requires more digits of representation than the sequence’s maxvalue/minvalue.

SCALE的算法就是(instance_id % 100)(_kqdsn_instance_digits) + 100||(session_id % 1000)(_kqdsn_cpu_digits)+seq(EXTEND/NOEXTEND确定是否固定宽度）

Scalable Sequences语法

CREATE | ALTER SEQUENCE [ schema. ]sequence
[ { INCREMENT BY | START WITH } integer
| { MAXVALUE integer | NOMAXVALUE }
| { MINVALUE integer | NOMINVALUE }
| { CYCLE | NOCYCLE }
| { CACHE integer | NOCACHE }
| { ORDER | NOORDER }
| { SCALE {EXTEND | NOEXTEND} | NOSCALE}
]

Scalable Sequences测试

[oracle@xifenfei ~]$ sqlplus / as sysdba
SQL*Plus: Release 12.2.0.1.0 Production on Tue Jan 23 02:33:49 2018
Copyright (c) 1982, 2016, Oracle.  All rights reserved.
Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production
BANNER                                                                               CON_ID
-------------------------------------------------------------------------------- ----------
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production              0
PL/SQL Release 12.2.0.1.0 - Production                                                    0
CORE    12.2.0.1.0      Production                                                        0
TNS for Linux: Version 12.2.0.1.0 - Production                                            0
NLSRTL Version 12.2.0.1.0 - Production                                                    0
SQL> set line 150
SQL> col "Paramete"  for a30
SQL> col "Session Value"  for a20
SQL> col "Instance Value" for a20
SQL> col "Is Default?" for a20
SQL> SELECT a.ksppinm "Parameter", b.ksppstvl "Session Value", c.ksppstvl "Instance Value",c.ksppstdf "Default?"
  2  FROM x$ksppi a, x$ksppcv b, x$ksppsv c WHERE a.indx = b.indx  AND a.indx = c.indx  AND lower(a.ksppinm)
  3  in ('_kqdsn_instance_digits', '_kqdsn_cpu_digits');
Parameter                                                Session Value        Instance Value       Default?
-------------------------------------------------------- -------------------- -------------------- ---------
_kqdsn_instance_digits                                   2                    2                    TRUE
_kqdsn_cpu_digits                                        3                    3                    TRUE
SQL> select instance_number from v$instance;
INSTANCE_NUMBER
---------------
              1
SQL> select sys_context('userenv','sid') from dual;
SYS_CONTEXT('USERENV','SID')
----------------------------------------------------------------------------
275
SQL> create sequence seq_xff start with 1 increment by 1 minvalue 1 maxvalue 100 scale extend;
Sequence created.
SQL> select seq_xff.nextval from dual;
   NEXTVAL
----------
 101275001

验证效果

--另外一个会话
[oracle@xifenfei ~]$ sqlplus / as sysdba
SQL*Plus: Release 12.2.0.1.0 Production on Tue Jan 23 02:45:14 2018
Copyright (c) 1982, 2016, Oracle.  All rights reserved.
Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production
SQL> select sys_context('userenv','sid') from dual;
SYS_CONTEXT('USERENV','SID')
---------------------------------------------------------
4
SQL> select instance_number from v$instance;
INSTANCE_NUMBER
---------------
              1
SQL> select seq_xff.nextval from dual;
   NEXTVAL
----------
 101004002

这里可以看出来seq的值是固定长度的.而且随着sid或者inst_id 不同而不同,从而实现减少大量数据集中在一个block而引起的各种争用
测试scale noextend

SQL> create sequence seq_xifenfei start with 1 increment by 1 minvalue 1 maxvalue 100 scale noextend;
Sequence created.
SQL> select seq_xifenfei.nextval from dual;
select seq_xifenfei.nextval from dual
       *
ERROR at line 1:
ORA-64603: NEXTVAL cannot be instantiated for seq_xifenfei. Widen the sequence
by 4 digits or alter sequence with SCALE EXTEND.
SQL> drop sequence seq_xifenfei;
Sequence dropped.
SQL> create sequence seq_xifenfei start with 1 increment by 1 minvalue 1 maxvalue 1000000 scale noextend;
Sequence created.
SQL>  select seq_xifenfei.nextval from dual;
   NEXTVAL
----------
   1010041

这里可以看出来scale noextend的长度是随着seq值的改变而改变,而且max值不能小于seq本身长度.

ORA-20001: Latest xml inventory is not loaded into table

12.2数据库启动报ORA-20001错

2018-01-22T04:51:19.574350-05:00
Unable to obtain current patch information due to error: 20001,
  ORA-20001: Latest xml inventory is not loaded into table
ORA-06512: at "SYS.DBMS_QOPATCH", line 777
ORA-06512: at "SYS.DBMS_QOPATCH", line 864
ORA-06512: at "SYS.DBMS_QOPATCH", line 2222
ORA-06512: at "SYS.DBMS_QOPATCH", line 740
ORA-06512: at "SYS.DBMS_QOPATCH", line 2247
===========================================================
Dumping current patch information
===========================================================
Unable to obtain current patch information due to error: 20001
===========================================================

patch日志

KUP-05007:   Warning: Intra source concurrency disabled because the preprocessor option is being used.
Field Definitions for table OPATCH_XML_INV
  Record format DELIMITED BY NEWLINE
  Data in file has same endianness as the platform
  Reject rows with all null fields
  Fields in Data Source:
    XML_INVENTORY                   CHAR (100000000)
      Terminated by "UIJSVTBOEIZBEFFQBL"
      Trim whitespace same as SQL Loader

DBMS_QOPATCH包是提供PLSQL/SQL接口方式访问数据库补丁安装情况，oracle在启动的时候会去调用这个包查询patch情况，由于ORA-20001 Latest XML inventory is not loaded into table.导致失败，从而出现上述启动错误。
主要相关的sql

CREATE OR REPLACE DIRECTORY "OPATCH_LOG_DIR" AS '/u01/app/oracle/product/11.2.0/dbhome_1/QOpatch';
CREATE OR REPLACE DIRECTORY "OPATCH_SCRIPT_DIR" AS '/u01/app/oracle/product/11.2.0/dbhome_1/QOpatch';
CREATE TABLE "SYS"."OPATCH_XML_INV"
   (    "XML_INVENTORY" CLOB
   )
   ORGANIZATION EXTERNAL
    ( TYPE ORACLE_LOADER
      DEFAULT DIRECTORY "OPATCH_SCRIPT_DIR"
      ACCESS PARAMETERS
      ( RECORDS DELIMITED BY NEWLINE CHARACTERSET UTF8
      DISABLE_DIRECTORY_LINK_CHECK
      READSIZE 8388608
      preprocessor opatch_script_dir:'qopiprep.bat'
      BADFILE opatch_script_dir:'qopatch_bad.bad'
      LOGFILE opatch_log_dir:'qopatch_log.log'
      FIELDS TERMINATED BY 'UIJSVTBOEIZBEFFQBL'
      MISSING FIELD VALUES ARE NULL
      REJECT ROWS WITH ALL NULL FIELDS
      (
        xml_inventory    CHAR(100000000)
      )
        )
      LOCATION
       ( "OPATCH_SCRIPT_DIR":'qopiprep.bat'
       )
    )
   REJECT LIMIT UNLIMITED;

解决方案

---方案1
SQL> alter system set "_disable_directory_link_check"=true scope=spfile;
System altered.
SQL> shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup
ORACLE instance started.
Total System Global Area 2432696320 bytes
Fixed Size                  8795664 bytes
Variable Size             654313968 bytes
Database Buffers         1761607680 bytes
Redo Buffers                7979008 bytes
Database mounted.
Database opened.
SQL> SELECT a.ksppinm "Parameter",
  2  b.ksppstvl "Session Value",
  3  c.ksppstvl "Instance Value"
  4  FROM x$ksppi a,
  5  x$ksppcv b,
  6  x$ksppsv c
  7  WHERE a.indx = b.indx
  8  AND a.indx = c.indx
  9  AND a.ksppinm LIKE '/_disable_direc%' escape '/' ;
Parameter
--------------------------------------------------------------------------------
Session Value
--------------------------------------------------------------------------------
Instance Value
--------------------------------------------------------------------------------
_disable_directory_link_check
TRUE
TRUE
SQL> exit
$ORACLE_HOME/sqlpatch/datapatch
---方案2
SQL> create or replace directory OPATCH_LOG_DIR as '<complete path to QOpatch>';
Directory created.
SQL> create or replace directory OPATCH_SCRIPT_DIR as '<complete path to QOpatch>';
Directory created.
---方案3
cd $ORACLE_HOME/sqlpatch
./datapatch -verbose
---方案4
chmod 775 -r $ORACLE_HOME/rdbms/log
$ORACLE_HOME/OPatch/datapatch -prereq

参考MOS
Queryable Patch Inventory – Issues/Solutions for ORA-20001: Latest xml inventory is not loaded into table (Doc ID 1602089.1)
12.2:DB Alert.log shows ORA-20001: Latest xml inventory,ORA-06512: at “SYS.DBMS_QOPATCH”,”KUP-04004: error while reading file” (Doc ID 2323937.1)

Assistant: Download Reference for Oracle Database/GI PSU, SPU(CPU), Bundle Patches, Patchsets and Base Releases—201801

Patchsets
12.1.0.2 (12.1.0.2.0 PATCH SET FOR ORACLE DATABASE SERVER)	21419221
11.2.0.4 (11.2.0.4.0 PATCH SET FOR ORACLE DATABASE SERVER)	13390677
11.2.0.3 (11.2.0.3.0 PATCH SET FOR ORACLE DATABASE SERVER)	10404530
11.2.0.2 (11.2.0.2.0 PATCH SET FOR ORACLE DATABASE SERVER)	10098816
11.1.0.7 (11.1.0.7.0 PATCH SET FOR ORACLE DATABASE SERVER)	6890831
10.2.0.5 (10.2.0.5 PATCH SET FOR ORACLE DATABASE SERVER)	8202632
^A10.2.0.4 (10.2.0.4.0 PATCH SET FOR ORACLE DATABASE SERVER)	6810189
^B10.2.0.3 (10.2.0.3 PATCH SET FOR ORACLE DATABASE SERVER)	5337014
10.2.0.2 (10.2.0.2 PATCH SET FOR ORACLE DATABASE SERVER)	4547817
10.1.0.5 (10.1.0.5 PATCH SET FOR ORACLE DATABASE SERVER)	4505133
10.1.0.4 (10.1.0.4 PATCH SET FOR ORACLE DATABASE SERVER)	4163362
10.1.0.3 (10.1.0.3 PATCH SET FOR ORACLE DATABASE SERVER)	3761843
9.2.0.8 (9.2.0.8 PATCH SET FOR ORACLE DATABASE SERVER)	4547809
9.2.0.7 (9.2.0.7 PATCH SET FOR ORACLE DATABASE SERVER)	4163445
9.2.0.6 (9.2.0.6 PATCH SET FOR ORACLE DATABASE SERVER)	3948480
9.2.0.5 (ORACLE 9I DATABASE SERVER RELEASE 2 – PATCH SET 4 VERSION 9.2.0.5.0)	3501955
9.2.0.4 (9.2.0.4 PATCH SET FOR ORACLE DATABASE SERVER)	3095277
9.2.0.3 (9.2.0.3 PATCH SET FOR ORACLE DATABASE SERVER)	2761332
9.2.0.2 (9.2.0.2 PATCH SET FOR ORACLE DATABASE SERVER)	2632931
9.0.1.5 (9.0.1.5 PATCHSET)	3301544
9.0.1.4 (9.0.1.4 PATCH SET FOR ORACLE DATABASE SERVER)	2517300
9.0.1.3 (9.0.1.3. PATCH SET FOR ORACLE DATA SERVER)	2271678
8.1.7.4 (8.1.7.4 PATCH SET FOR ORACLE DATA SERVER)	2376472
8.1.7.3 (8.1.7.3 PATCH SET FOR ORACLE DATA SERVER)	2189751
8.1.7.2 (8.1.7.2.1 PATCH SET FOR ORACLE DATA SERVER)	1909158

12.2.0.1
Description	Database RU	GI RU	Windows Bundle Patch
JAN2018 (12.2.0.1.180116)	27105253	27100009	27162931
NOV2017 (12.2.0.1.171121)		27010638
OCT2017 (12.2.0.1.171017)	26710464	26737266	26758841
AUG2017 (12.2.0.1.170814)	26609817	26610291	26204214
JUL2017 (12.2.0.1.170718)	26123830	26133434	26204212

12.1.0.2
Description	PSU	GI PSU	Proactive Bundle Patch	Bundle Patch(Windows 32bit & 64bit)
JAN2017 (12.1.0.2.180116)	26925311	27010872	27010930	27162953
OCT2017 (12.1.0.2.171017)	26713565	26635815	26635880	26720785
AUG2017(12.1.0.2.170814)	26609783	26610308	26610322	26161726
JUL2017 (12.1.0.2.170718)	25755742	25901062	26022196	26161724
APR2017 (12.1.0.2.170418)	25171037	25434003	25433352	25632533
JAN2017 (12.1.0.2.170117)	24732082	24917825	24968615	25115951
OCT2016 (12.1.0.2.161018)	24006101	24412235	24448103	24591642
JUL2016 (12.1.0.2.160719)	23054246	23273629	23273686	23530387
APR2016 (12.1.0.2.160419)	22291127	22646084	22899531	22809813
JAN2016 (12.1.0.2.160119)	21948354	22191349	22243551	22310559
OCT2015	21359755(12.1.0.2.5)	21523234(12.1.0.2.5)	21744410(12.1.0.2.13)	21821214(12.1.0.2.10)
JUL2015	20831110(12.1.0.2.4)	20996835(12.1.0.2.4)	21188742(12.1.0.2.10)	21126814(12.1.0.2.7)
APR2015	20299023(12.1.0.2.3)	20485724(12.1.0.2.3)	20698050(12.1.0.2.7)	20684004(12.1.0.2.4)
JAN2015	19769480(12.1.0.2.2)	19954978(12.1.0.2.2)	20141343(12.1.0.2.4)	19720843(12.1.0.2.1)
OCT2014	19303936(12.1.0.2.1)	19392646(12.1.0.2.1)	19404326(12.1.0.2.1)	N/A

12.1.0.1
Description	PSU	GI PSU		Bundle Patch
Description	PSU	A	B	Windows 64 bit	Windows 32 bit
JUL2016 (12.1.0.1.160719)	23054354	23273935	23273958	23530410
APR2016 (12.1.0.1.160419)	22291141	22654153	22654166	22839614
JAN2016 (12.1.0.1.160119)	21951844	22191492	22191511	22494866
OCT2015	21352619(12.1.0.1.9)	21551666(12.1.0.1.9)	21551685(12.1.0.1.9)	21744907 (12.1.0.1.21)
JUL2015	20831107(12.1.0.1.8)	20996901(12.1.0.1.8)	20996911(12.1.0.1.8)	21076681 (12.1.0.1.20)
APR2015	20299016(12.1.0.1.7)	20485762(12.1.0.1.7)	19971331(12.1.0.1.7)	20558101 (12.1.0.1.18)
JAN2015	19769486(12.1.0.1.6)	19971324(12.1.0.1.6)	19971331(12.1.0.1.6)	20160748 (12.1.0.1.16)
OCT2014	19121550(12.1.0.1.5)	19392372(12.1.0.1.5)	19392451(12.1.0.1.5)	19542943 (12.1.0.1.14)
JUL2014	18522516(12.1.0.1.4)	18705901(12.1.0.1.4)	18705972(12.1.0.1.4)	19062327 (12.1.0.1.11)
APR2014	18031528(12.1.0.1.3)	18139660(12.1.0.1.3)	18413105(12.1.0.1.3)	18448604 (12.1.0.1.7)
JAN2014	17552800(12.1.0.1.2)	17735306 (12.1.0.1.2)		17977915 (12.1.0.1.3)
OCT2013	17027533(12.1.0.1.1)	17272829 (12.1.0.1.1)		17363796(12.1.0.1.1)	17363795(12.1.0.1.1)

11.2.0.4
Description	PSU	SPU(CPU)	GI PSU	Bundle Patch (Windows 32bit & 64bit)
JAN2018 (11.2.0.4.180116)	26925576	N/A	27107360	27162965
OCT2017 (11.2.0.4.171017)	26392168	26474853	26635745	26581376
AUG2017 (11.2.0.4.170814)	26609445	N/A	26610246	26194138
JUL2017 (11.2.0.4.170718)	25869727	25879656	26030799	26194136
APR2017 (11.2.0.4.170418)	24732075	25369547	25476126	25632525
JAN2017	N/A	N/A	N/A	N/A
OCT2016 (11.2.0.4.161018)	24006111	24433711	24436338	24591646
JUL2016 (11.2.0.4.160719)	23054359	23177648	23274134	23530402
APR2016 (11.2.0.4.160419)	22502456	22502493	22646198	22839608
JAN2016 (11.2.0.4.160119)	21948347	21972320	22191577	22310544
OCT2015	21352635 (11.2.0.4.8)	21352646	21523375 (11.2.0.4.8)	21821802 (11.2.0.4.20)
JUL2015	20760982 (11.2.0.4.7)	20803583	20996923 (11.2.0.4.7)	21469106 (11.2.0.4.18)
APR2015	20299013 (11.2.0.4.6)	20299015	20485808 (11.2.0.4.6)	20544696 (11.2.0.4.15)
JAN2015	19769489 (11.2.0.4.5)	19854503	19955028 (11.2.0.4.5)	20127071 (11.2.0.4.12)
OCT2014	19121551 (11.2.0.4.4)	19271443	19380115 (11.2.0.4.4)	19651773 (11.2.0.4.10)
JUL2014	18522509 (11.2.0.4.3)	18681862	18706472 (11.2.0.4.3)	18842982 (11.2.0.4.7)
APR2014	18031668 (11.2.0.4.2)	18139690	18139609 (11.2.0.4.2)	18296644 (11.2.0.4.4)
JAN2014	17478514 (11.2.0.4.1)	17551709	N/A	17987366 (11.2.0.4.1)

11.2.0.3
Description	PSU	SPU(CPU)	GI PSU	Bundle Patch (Windows 64bit)	Bundle Patch (Windows 32bit)
JUL2015	20760997 (11.2.0.3.15)	20803576	20996944 (11.2.0.3.15)	21104036	21104035
APR2015	20299017 (11.2.0.3.14)	20299010	20485830 (11.2.0.3.14)	20420395	20420394
JAN2015	19769496 (11.2.0.3.13)	19854461	19971343 (11.2.0.3.13)	20233168	20233167
OCT2014	19121548 (11.2.0.3.12)	19271438	19440385 (11.2.0.3.12)	19618575	19618574
JUL2014	18522512 (11.2.0.3.11)	18681866	18706488 (11.2.0.3.11)	18940194	18940193
APR2014	18031683 (11.2.0.3.10)	18139695	18139678 (11.2.0.3.10)	18372244	18372243
JAN2014	17540582 (11.2.0.3.9)	17478415	17735354 (11.2.0.3.9)	18075406	17906981
OCT2013	16902043 (11.2.0.3.8)	17082364	17272731 (11.2.0.3.8)	17363850	17363844
JUL2013	16619892 (11.2.0.3.7)	16742095	16742216 (11.2.0.3.7)	16803775	16803774
APR2013	16056266 (11.2.0.3.6)	16294378	16083653 (11.2.0.3.6)	16345834	16345833
JAN2013	14727310 (11.2.0.3.5)	14841409	14727347 (11.2.0.3.5)	16042648	16042647
OCT2012	14275605 (11.2.0.3.4)	14390252	14275572 (11.2.0.3.4)	14613223	14613222
JUL2012	13923374 (11.2.0.3.3)	14038787	13919095 (11.2.0.3.3)	14223718	14223717
APR2012	13696216 (11.2.0.3.2)	13632717	13696251 (11.2.0.3.2)	13885389	13885388
JAN2012	13343438 (11.2.0.3.1)	13466801	13348650 (11.2.0.3.1)	13413168	13413167

11.2.0.2
Description	PSU	SPU(CPU)	GI PSU	Bundle Patch (Windows 64bit)	Bundle Patch (Windows 32bit)
OCT2013	17082367 (11.2.0.2.12)	17082375	17272753 (11.2.0.2.12)	17363838	17363837
JUL2013	16619893 (11.2.0.2.11)	16742100	16742320 (11.2.0.2.11)	16345852	16345851
APR2013	16056267 (11.2.0.2.10)	16294412	16166868 (11.2.0.2.10)	16345846	16345845
JAN2013	14727315 (11.2.0.2.9)	14841437	14841385 (11.2.0.2.9)	16100399	16100398
OCT2012	14275621 (11.2.0.2.8)	14390377	14390437 (11.2.0.2.8)	14672268	14672267
JUL2012	13923804 (11.2.0.2.7)	14038791	14192201 (11.2.0.2.7)	14134043	14134042
APR2012	13696224 (11.2.0.2.6)	13632725	13696242 (11.2.0.2.6)	13697074	13697073
JAN2012	13343424 (11.2.0.2.5)	13343244	13653086 (11.2.0.2.5)	13413155	13413154
OCT2011	12827726 (11.2.0.2.4)	12828071	12827731 (11.2.0.2.4)	13038788	13038787
JUL2011	12419331 (11.2.0.2.3)	12419321	12419353 (11.2.0.2.3)	12714463	12714462
APR2011	11724916 (11.2.0.2.2)	11724984	12311357 (11.2.0.2.2)	11896292	11896290
JAN2011	10248523 (11.2.0.2.1)	N/A	N/A	10432053	10432052

11.2.0.1
Description	PSU	CPU	Bundle Patch (Windows 64bit)	Bundle Patch (Windows 32bit)
JUL2011	12419378 (11.2.0.1.6)	12419278	12429529	12429528
APR2011	11724930 (11.2.0.1.5)	11724991	11731176	11883240
JAN2011	10248516 (11.2.0.1.4)	10249532	10432045	10432044
OCT2010	9952216 (11.2.0.1.3)	9952260	10100101	10100100
JUL2010	9654983 (11.2.0.1.2)	9655013	9736865	9736864
APR2010	9352237 (11.2.0.1.1)	9369797	N/A	N/A

10.2.0.5
Description	PSU	SPU(CPU)	Bundle Patch (Windows 64bit)	Bundle Patch (Windows 32bit)	Bundle Patch (Windows Itanium)
JUL2015	20299014(10.2.0.5.19)	20299021	20420387	20420386	N/A
APR2015	N/A	N/A	N/A	N/A	N/A
JAN2015	19769505(10.2.0.5.18)	19854436	20126868	20126867	N/A
OCT2014	19274523(10.2.0.5.17)	19274521	19618565	19618563	N/A
JUL2014	18522511(10.2.0.5.16)	18681879	18940198	18940196	N/A
APR2014	18031728(10.2.0.5.15)	18139709	18372261	18372259	N/A
JAN2014	17465584(10.2.0.5.14)	17551414	17906974	17906972	N/A
OCT2013	17082365(10.2.0.5.13)	17082371	N/A	17363822	N/A
JUL2013	16619894(10.2.0.5.12)	16742123	16803782	16803780	16803781
APR2013	16056270(10.2.0.5.11)	16270946	16345857	16345855	16345856
JAN2013	14727319(10.2.0.5.10)	14841459	15848062	15848060	15848061
OCT2012	14275629(10.2.0.5.9)	14390396	14553358	14553356	14553357
JUL2012	13923855(10.2.0.5.8)	14038805	14134053	14134051	14134052
APR2012	13632743(10.2.0.5.7)	13632738	13654815	13654814	13870404
JAN2012	13343471(10.2.0.5.6)	13343467	13460968	13460967	N/A
OCT2011	12827745(10.2.0.5.5)	12828105	N/A	12914911	N/A
JUL2011	12419392(10.2.0.5.4)	12419258	12429524	12429523	N/A
APR2011	11724962(10.2.0.5.3)	11725006	12328269	12328268	N/A
JAN2011	10248542(10.2.0.5.2)	10249537	10352673	10352672	N/A
OCT2010	9952230(10.2.0.5.1)	9952270	10099855	10058290	N/A

10.2.0.4
Description	PSU	SPU(CPU)	Bundle Patch (Windows 32bit)	Bundle Patch (Windows 64bit)	Bundle Patch (Windows Itanium)
JUL2013	16619897 (10.2.0.4.17)	16742253	N/A	N/A	N/A
APR2013	16056269 (10.2.0.4.16)	16270931	N/A	N/A	N/A
JAN2013	14736542 (10.2.0.4.15)	14841471	N/A	N/A	N/A
OCT2012	14275630 (10.2.0.4.14)	14390410	N/A	N/A	N/A
JUL2012	13923851 (10.2.0.4.13)	14038814	N/A	N/A	N/A
APR2012	12879933 (10.2.0.4.12)	12879926	13928775	13928776	N/A
JAN2012	12879929 (10.2.0.4.11)	12879912	13654060	N/A	N/A
OCT2011	12827778 (10.2.0.4.10)	12828112	12914908	12914910	12914909
JUL2011	12419397 (10.2.0.4.9)	12419249	12429519	12429521	12429520
APR2011	11724977 (10.2.0.4.8)	11725015	12328501	12328503	12328502
JAN2011	10248636 (10.2.0.4.7)	10249540	10349197	10349200	10349198
OCT2010	9952234 (10.2.0.4.6)	9952272	10084980	10084982	10084981
JUL2010	9654991 (10.2.0.4.5)	9655017	9777076	9777078	9777077
APR2010	9352164 (10.2.0.4.4)	9352191	9393548	9393550	9393549
JAN2010	9119284 (10.2.0.4.3)	9119226	9169457	9169460	9169458
OCT2009	8833280 (10.2.0.4.2)	8836308	8880857	8880861	8880858
JUL2009	8576156 (10.2.0.4.1)	8534387	8559466	8559467	8541782
APR2009	N/A	8290506	8307237	8307238	8333678
JAN2009	N/A	7592346	7584866	7584867	N/A
OCT2008	N/A	7375644	7386320	7386321	N/A
JUL2008	N/A	7150470	7218676	7218677	N/A

10.2.0.3
Description	CPU (Unix/Linux)	Bundle Patch (Windows 32bit)	Bundle Patch (Windows Itanium)	Bundle Patch (Windows 64bit)
JAN2009	7592354	7631956	7631958	7631957
OCT2008	7369190	7353782	7353784	7353785
JUL2008	7150622	7252496	7252497	7252498
APR2008	6864068	6867054	6867055	6867056
JAN2008	6646853	6637237	6637238	6637239
OCT2007	6394981	6430171	6430173	6430174
JUL2007	6079591	6116131	6038242	6116139
APR2007	5901891	5948242	5916262	5948243
JAN2007	5881721	5846376	5846377	5846378

10.2.0.2
Description	CPU (Unix/Linux)	Bundle Patch (Windows 32bit)	Bundle Patch (Windows 64bit)	Bundle Patch (Windows Itanium)
JAN2009	7592355	N/A	N/A	N/A
OCT2008	7375660	N/A	N/A	N/A
JUL2008	7154083	N/A	N/A	N/A
APR2008	6864071	N/A	N/A	N/A
JAN2008	6646850	N/A	N/A	N/A
OCT2007	6394997	6397028	6397030	6397029
JUL2007	6079588	6013105	6013121	6013118
APR2007	5901881	5912173	5912179	5912176
JAN2007	5689957	5716143	5699839	5699824
OCT2006	5490848	5502226	5500921	5500894
JUL2006	5225799	5251025	5251028	5251026
APR2006	5079037	5140461	5140567	5140508

10.2.0.1
Description	CPU (Unix/Linux)	Bundle Patch (Windows 32bit)	Bundle Patch (Windows 64bit)	Bundle Patch (Windows Itanium)
APR2007	5901880	N/A	N/A	N/A
JAN2007	5689937	5695784	5695786	5695785
OCT2006	5490846	5500927	5500954	5500951
JUL2006	5225798	5239698	5239701	5239699
APR2006	5049080	5059238	5059261	5059251
JAN2006	4751931	4751539	4770480	4751549

10.1.0.5
Description	CPU (Unix/Linux)	Bundle Patch (Windows 32bit)	Bundle Patch (Windows Itanium)
JAN2012	13343482	13413002	13413003
OCT2011	12828135	12914905	12914906
JUL2011	12419228	12429517	12429518
APR2011	11725035	11731119	11731120
JAN2011	N/A	N/A	N/A
OCT2010	9952279	10089559	10089560
JUL2010	9655023	9683651	9683652
APR2010	9352208	9390288	9390289
JAN2010	9119261	9187104	9187105
OCT2009	8836540	8785211	8785212
JUL2009	8534394	8656224	8656226
APR2009	8290534	8300356	8300360
JAN2009	7592360	7486619	7586049
OCT2008	7375686	7367493	7367494
JUL2008	7154097	7047034	7047037
APR2008	6864078	6867107	6867108
JAN2008	6647005	6637274	6637275
OCT2007	6395024	6408393	6408394
JUL2007	6079585	6115804	6115818
APR2007	5901877	5907304	5907305
JAN2007	5689908	5716295	5634747
OCT2006	5490845	5500883	5500885
JUL2006	5225797	5251148	5251140
APR2006	5049074	5057606	5057609
JAN2006	4751932	4882231	4882236

10.1.0.4
Description	CPU (Unix/Linux)	Bundle Patch (Windows 32bit)	Bundle Patch (Windows Itanium)
APR2007	5901876	5909871	5909879
JAN2007	5689894	5695771	5695772
OCT2006	5490844	5500878	5500880
JUL2006	5225796	5239736	5239737
APR2006	5049067	5059200	5059227
JAN2006	4751928	4751259	4745040
OCT2005	4567866	4579182	4579188
JUL2005	4392423	4440706	4404600
APR2005	4210374	4287619	4287611

10.1.0.3
Description	CPU (Unix/Linux)	Bundle Patch (Windows 32bit)	Bundle Patch (Windows Itanium)
JAN2007	5923277	N/A	N/A
OCT2006	5566825	N/A	N/A
JUL2006	5435164	N/A	N/A
APR2006	5158022	N/A	N/A
JAN2006	4751926	4741077	4741084
OCT2005	4567863	4567518	4567523
JUL2005	4392409	4389012	4389014
APR2005	4193286	4269715	4158888
JAN2005	4003062	4074232	3990812

10.1.0.2
Description	CPU (Unix/Linux)	Bundle Patch (Windows 32bit)	Bundle Patch (Windows Itanium)
APR2005	4193293	4181849	4213305
JUL2005	4400766	4388944	4388948
JAN2005	4003051	4104364	4083038

9.2.0.8
Description	CPU (Unix/Linux)	Bundle Patch (Windows 32bit)	Bundle Patch (Windows Itanium)
JUL2010	9655027	9683644	9683645
APR2010	9352224	9390286	N/A
JAN2010	9119275	9187106	N/A
OCT2009	8836758	8785185	8785186
JUL2009	8534403	8427417	8427418
APR2009	8290549	8300340	8300346
JAN2009	7592365	7703210	7703212
OCT2008	7375695	7394394	7394402
JUL2008	7154111	7047026	7047029
APR2008	6864082	6867138	6867139
JAN2008	6646842	6637265	6637266
OCT2007	6395038	6417013	6417014
JUL2007	6079582	6130293	6130295
APR2007	5901875	5916268	5916275
JAN2007	N/A	N/A	N/A
OCT2006	5490859	5652380	5639519

9.2.0.7
Description	CPU (Unix/Linux)	Bundle Patch (Windows 32bit)	Bundle Patch (Windows Itanium)
JUL2007	6079579	6146759	6146748
APR2007	5901872	5907274	5907275
JAN2007	5689875	5654905	5654909
OCT2006	5490841	5500873	5500874
JUL2006	5225794	5250980	5250981
APR2006	5049060	5064365	5064364
JAN2006	4751923	4751528	4741074
OCT2005	4567854	4579590	4579599
JUL2005	4547566	N/A	N/A

9.2.0.6
Description	CPU (Unix/Linux)	Bundle Patch (Windows 32bit)	Bundle Patch (Windows Itanium)
OCT2006	5490840	5500865	5500871
JUL2006	5225793	5239794	5239793
APR2006	5049051	5059614	5059615
JAN2006	4751921	4751261	4751262
OCT2005	4567846	4579093	4579097
JUL2005	4392392	4445852	4401917
APR2005	4193295	4269928	4213298

9.2.0.5
Description	CPU (Unix/Linux)	Bundle Patch (Windows 32bit)	Bundle Patch (Windows Itanium)
OCT2006	5689708	N/A	N/A
JUL2006	5435138	N/A	N/A
APR2006	5219762	N/A	N/A
OCT2005	4560421	N/A	N/A
JUL2005	4392256	4387563	4391819
APR2005	4193299	4195791	4214192
JAN2005	4003006	4104374	3990809

9.2.0.4
Description	CPU (Unix/Linux)	Bundle Patch (Windows 32bit)	Bundle Patch (Windows Itanium)
JAN2005	4002994	4104369	4083202

8.1.7.4
Description	CPU (Unix/Linux)	Bundle Patch (Windows 32bit)
JAN2007	5689799	5686514
OCT2006	5490835	5496067
JUL2006	5225788	5236412
APR2006	5045247	5057601
JAN2006	4751906	4751570
OCT2005	4560405	4554818
JUL2005	4392446	4437058
APR2005	4193312	4180163
JAN2005	4002909	3921893

参考:Assistant: Download Reference for Oracle Database/GI PSU, SPU(CPU), Bundle Patches, Patchsets and Base Releases (文档 ID 2118136.2)

文件read only导致数据库无法open

在一次客户的恢复请求中,系统被黑客入侵,数据库无法启动.虽然system文件可以拷贝,但是无论拷贝到哪里,都无法dbv和open数据库,报O/S-Error: (OS 5)和DBV-00600错误.最后检查发现是文件设置了只读
数据库启动报错

ORACLE 例程已经启动。
Total System Global Area 6881345536 bytes
Fixed Size                  2130160 bytes
Variable Size            4026535696 bytes
Database Buffers         2818572288 bytes
Redo Buffers               34107392 bytes
数据库装载完毕。
ORA-01113: 文件 1 需要介质恢复
ORA-01110: 数据文件 1: 'D:\APP\ADMINISTRATOR\ORADATA\xifenfei\SYSTEM01.DBF'
SQL> recover datafile 1;
ORA-00283: 恢复会话因错误而取消
ORA-01114: 将块写入文件 1 时出现 IO 错误 (块 # 1)
ORA-01110: 数据文件 1: 'D:\APP\ADMINISTRATOR\ORADATA\xifenfei\SYSTEM01.DBF'
ORA-27091: 无法将 I/O 排队
ORA-27041: 无法打开文件
OSD-04002: 无法打开文件
O/S-Error: (OS 5) 拒绝访问。
SQL> recover database until cancel using backup controlfile;
ORA-00283: 恢复会话因错误而取消
ORA-01201: 文件 1 标头无法正确写入

alert日志报错

Completed: ALTER DATABASE   MOUNT
Sun Dec 10 12:03:20 2017
alter database open
Errors in file d:\app\administrator\diag\rdbms\xifenfei\xifenfei\trace\xifenfei_ora_6860.trc:
ORA-01110: 数据文件 1: 'D:\APP\ADMINISTRATOR\ORADATA\xifenfei\SYSTEM01.DBF'
ORA-01114: 将块写入文件 1 时出现 IO 错误 (块 # 1)
ORA-27041: 无法打开文件
OSD-04002: 无法打开文件
O/S-Error: (OS 5) 拒绝访问。
ORA-01201: 文件 1 标头无法正确写入
ORA-1113 signalled during: alter database open...
Sun Dec 10 12:03:33 2017
ALTER DATABASE RECOVER  datafile 1
Media Recovery Start
Fast Parallel Media Recovery NOT enabled
Media Recovery failed with error 1114
ORA-283 signalled during: ALTER DATABASE RECOVER  datafile 1  ...

dbv检查文件

D:\APP\ADMINISTRATOR\ORADATA>dbv file=system01.dbf
DBVERIFY: Release 11.1.0.7.0 - Production on 星期日 12月 10 19:06:55 2017
Copyright (c) 1982, 2007, Oracle.  All rights reserved.
DBV-00600: 致命错误 - [22] [27041] [0] [0]

这里很明显,文件不能写操作的原因,就是由于该属性导致

redhat和oracle linux kernel对应关系

参考：Comparison of Red Hat and Oracle Linux kernel versions and release strings (文档 ID 560992.1)

asm磁盘组操作不当导致数据文件丢失恢复

最近遇到数据库恢复case,客户是要更换存储,在数据库mount状态把使用omf方式存储数据的asm 磁盘组通过rman copy到新的通过别名方式存储的新的asm 磁盘组的存储中,但是由于操作人员粗心,copy语句中部分目标磁盘组的数据文件别名重复了,最后执行rename file之后,导致部分数据文件彻底丢失.我们通过底层碎片扫描(参考:asm disk header 彻底损坏恢复)对于该用户的数据实现完全恢复.
因为整个过程重现比较麻烦,这里测试从一个data磁盘组中有一个omf方式存储的含有两个数据文件的表空间,通过rman copy 把这个表空间的两个文件拷贝到datanew磁盘组中,但是由于粗心把两个数据文件的别名写成一样,结果导致该表空间的一个数据文件彻底丢失的测试.

创建测试表空间
在datanew磁盘组中创建omf方式管理的xifenfei表空间,含有两个数据文件,file#分别为14和15

SQL> create tablespace xifenfei datafile '+DATA' SIZE 128m;
Tablespace created.
SQL> ALTER TABLESPACE XIFENFEI ADD DATAFILE '+DATA' SIZE 128m AUTOEXTEND ON;
Tablespace altered.
SQL> SELECT FILE_NAME,FILE_ID FROM  DBA_DATA_FILES WHERE TABLESPACE_NAME='XIFENFEI';
FILE_NAME
--------------------------------------------------------------------------------
   FILE_ID
----------
+DATA/XFF/DATAFILE/xifenfei.276.961143809
        14
+DATA/XFF/DATAFILE/xifenfei.277.961143825
        15

rman copy datafile 14
通过rman copy把datafile 14拷贝到data磁盘组中,目标端为别名方式存储

RMAN> copy datafile 14 to '+datanew/xifenfei.dbf';
Starting backup at 27-NOV-17
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=24 device type=DISK
channel ORA_DISK_1: starting datafile copy
input datafile file number=00014 name=+DATA/XFF/DATAFILE/xifenfei.276.961143809
output file name=+DATANEW/xifenfei.dbf tag=TAG20171127T082643 RECID=4 STAMP=961144006
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:07
Finished backup at 27-NOV-17
[grid@localhost ~]$ asmcmd
ASMCMD> cd datanew
ASMCMD> ls
XFF/
xifenfei.dbf
ASMCMD> ls -l
Type      Redund  Striped  Time             Sys  Name
                                            Y    XFF/
DATAFILE  UNPROT  COARSE   NOV 27 08:00:00  N    xifenfei.dbf => +DATANEW/XFF/DATAFILE/XIFENFEI.256.961144003
ASMCMD>

这里通过asmcmd的ls命令,可以看到虽然我们存储的为datanew磁盘组的别名文件,实际上是link到asm的omf方式的文件(本质上asm中的文件都是omf方式存储,只是在使用的时候体现asm的客户端程序方式不一样,是直接asm中的omf方式,还是asm中的别名).

rman copy datafile 15
通过rman copy把datafile 15 拷贝到和datafile 14别名一样的文件了

RMAN> copy datafile 15 to '+datanew/xifenfei.dbf';
Starting backup at 27-NOV-17
using channel ORA_DISK_1
channel ORA_DISK_1: starting datafile copy
input datafile file number=00015 name=+DATA/XFF/DATAFILE/xifenfei.277.961143825
output file name=+DATANEW/xifenfei.dbf tag=TAG20171127T082731 RECID=5 STAMP=961144053
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:03
Finished backup at 27-NOV-17
ASMCMD> ls -l
Type      Redund  Striped  Time             Sys  Name
                                            Y    XFF/
DATAFILE  UNPROT  COARSE   NOV 27 08:00:00  N    xifenfei.dbf => +DATANEW/XFF/DATAFILE/XIFENFEI.256.961144003
ASMCMD> cd xff
ASMCMD> ls
DATAFILE/
ASMCMD> cd datafile
ASMCMD> ls
XIFENFEI.256.961144003
ASMCMD>

这里可以看出来,在data磁盘组中,file 14被file 15覆盖掉了

rename file
把data磁盘组中的数据文件rename 到datanew磁盘组中

SQL> alter database rename file '+DATA/XFF/DATAFILE/xifenfei.276.961143809' to '+datanew/xifenfei.dbf';
Database altered.
SQL> alter database rename file '+DATA/XFF/DATAFILE/xifenfei.277.961143825' to '+datanew/xifenfei.dbf';
alter database rename file '+DATA/XFF/DATAFILE/xifenfei.277.961143825' to '+datanew/xifenfei.dbf'
*
ERROR at line 1:
ORA-01511: error in renaming log/data files
ORA-01523: cannot rename data file to '+data/xifenfei.dbf' - file already part of database

这里我们可以看到,file 14 rename 成功,但是file 15 rename失败,因为在数据库中，已经有了别名的文件(数据文件的路径)

omf自动删除文件
查看原磁盘组datanew中,发现datafile 14被自动删除

ASMCMD> pwd
+DATA/XFF/DATAFILE
ASMCMD> ls -l
Type      Redund  Striped  Time             Sys  Name
DATAFILE  UNPROT  COARSE   NOV 27 08:00:00  Y    SYSAUX.257.942061433
DATAFILE  UNPROT  COARSE   NOV 27 08:00:00  Y    SYSTEM.256.942061393
DATAFILE  UNPROT  COARSE   NOV 27 08:00:00  Y    UNDOTBS1.258.942061449
DATAFILE  UNPROT  COARSE   NOV 27 08:00:00  Y    USERS.259.942061449
DATAFILE  UNPROT  COARSE   NOV 27 08:00:00  Y    XIFENFEI.277.961143825
ASMCMD>

alert日志证实数据文件被删除

2017-11-27T09:05:03.054741-05:00
alter database rename file '+DATA/XFF/DATAFILE/xifenfei.276.961143809' to '+datanew/xifenfei.dbf'
2017-11-27T09:05:03.114947-05:00
NOTE: Under CF enqueue, no dependency request for disk group DATANEW
Deleted Oracle managed file +DATA/XFF/DATAFILE/xifenfei.276.961143809
Completed: alter database rename file '+DATA/XFF/DATAFILE/xifenfei.276.961143809' to '+datanew/xifenfei.dbf'
2017-11-27T09:05:21.471474-05:00
alter database rename file '+DATA/XFF/DATAFILE/xifenfei.277.961143825' to '+data/xifenfei.dbf'
ORA-1511 signalled during:alter database rename file
      '+DATA/XFF/DATAFILE/xifenfei.277.961143825' to'+datanew/xifenfei.dbf'

这里可以证实，数据文件的omf方式管理,在数据文件执行rename file的时候,会自动删除掉老的数据文件.这里悲剧已经发生,由于rman copy 覆盖了datanew磁盘组中的datafile 14,rename file又导致data磁盘组中的datafile 14被自动删除,从而使得datafile 14这个数据文件在两个磁盘组中都丢失.从常规角度来说,如果没有合适的备份该文件无法恢复.如果遭遇到oracle asm中数据文件丢失或者部分覆盖,请保护现场,联系我们(ORACLE数据库恢复技术支持),将为您提供专业数据库技术支持：Phone:17813235971 Q Q:107644445 E-Mail:dba@xifenfei.com最大限度抢救您的数据

dbconsole和sysman用户

dbconsole一直是我不怎么推荐的oracle工具,但是总有一些客户使用,这里对于dbconsole中sysman用户状态,密码修改做了一些基本的测试
sysman用户被锁

SQL>  alter user sysman account  lock;
User altered.

重启dbconsole

[oracle@localhost ~]$ emctl stop dbconsole
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = (unset),
        LANG = "c"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
Oracle Enterprise Manager 11g Database Control Release 11.2.0.4.0
Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.
https://localhost:1158/em/console/aboutApplication
Stopping Oracle Enterprise Manager 11g Database Control ...
 ...  Stopped.
[oracle@localhost ~]$ emctl start dbconsole
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = (unset),
        LANG = "c"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
Oracle Enterprise Manager 11g Database Control Release 11.2.0.4.0
Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.
https://localhost:1158/em/console/aboutApplication
Starting Oracle Enterprise Manager 11g Database Control ..... started.
------------------------------------------------------------------
Logs are generated in directory /u01/app/oracle/product/11.2.0/db_1/localhost_test/sysman/log

部分错误日志提示（这里可以明显dbconsole无法正常使用是由于sysman被锁）

解锁sysman用户

SQL> alter user sysman account unlock;
User altered.

重启dbconsole

[oracle@localhost ~]$ emctl stop dbconsole
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = (unset),
        LANG = "c"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
Oracle Enterprise Manager 11g Database Control Release 11.2.0.4.0
Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.
https://localhost:1158/em/console/aboutApplication
Stopping Oracle Enterprise Manager 11g Database Control ...
 ...  Stopped.
[oracle@localhost ~]$ emctl start dbconsole
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = (unset),
        LANG = "c"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
Oracle Enterprise Manager 11g Database Control Release 11.2.0.4.0
Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.
https://localhost:1158/em/console/aboutApplication
Starting Oracle Enterprise Manager 11g Database Control ..... started.
------------------------------------------------------------------
Logs are generated in directory /u01/app/oracle/product/11.2.0/db_1/localhost_test/sysman/log

sysman解锁之后重启dbconsole恢复正常
修改sysman密码

SQL> alter user sysman identified by xifenfei;
User altered.
SQL> select account_status from dba_users where username='SYSMAN';
ACCOUNT_STATUS
--------------------------------
OPEN

重启dbconsole

[oracle@localhost ~]$  emctl stop dbconsole
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = (unset),
        LANG = "c"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
Oracle Enterprise Manager 11g Database Control Release 11.2.0.4.0
Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.
https://localhost:1158/em/console/aboutApplication
Stopping Oracle Enterprise Manager 11g Database Control ...
 ...  Stopped.
[oracle@localhost ~]$  emctl start dbconsole
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = (unset),
        LANG = "c"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
Oracle Enterprise Manager 11g Database Control Release 11.2.0.4.0
Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.
https://localhost:1158/em/console/aboutApplication
Starting Oracle Enterprise Manager 11g Database Control ..... started.
------------------------------------------------------------------
Logs are generated in directory /u01/app/oracle/product/11.2.0/db_1/localhost_test/sysman/log

SQL>  select account_status from dba_users where username='SYSMAN';
ACCOUNT_STATUS
--------------------------------
LOCKED(TIMED)

由于dbconsole的密码存储在配置文件中，修改sysman密码，而配置文件的没有修改，因此使用错误是syman密码登录，超过了profile里面设置的10次，导致用户被锁.

解决sysman密码不对导致异常问题

[oracle@localhost ~]$ emctl stop dbconsole
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = (unset),
        LANG = "c"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
Oracle Enterprise Manager 11g Database Control Release 11.2.0.4.0
Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.
https://localhost:1158/em/console/aboutApplication
Stopping Oracle Enterprise Manager 11g Database Control ...
 ...  Stopped.
[oracle@localhost config]$ pwd
/u01/app/oracle/product/11.2.0/db_1/localhost_test/sysman/config
[oracle@localhost config]$ ls -l emoms.properties
-rw-r--r--. 1 oracle oinstall 1669 Nov 25 15:20 emoms.properties
[oracle@localhost config]$ grep emdRepPwd emoms.properties|grep -v emdRepPwdSeed
oracle.sysman.eml.mntr.emdRepPwd=e3299b5db6162f5d
oracle.sysman.eml.mntr.emdRepPwdEncrypted=TRUE
[oracle@localhost config]$ grep emdRepPwd emoms.properties|grep -v emdRepPwdSeed
oracle.sysman.eml.mntr.emdRepPwd=xifenfei
oracle.sysman.eml.mntr.emdRepPwdEncrypted=FALSE

SQL>  alter user sysman account unlock;
User altered.

[oracle@localhost ~]$ emctl start dbconsole
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = (unset),
        LANG = "c"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
Oracle Enterprise Manager 11g Database Control Release 11.2.0.4.0
Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.
https://localhost:1158/em/console/aboutApplication
Starting Oracle Enterprise Manager 11g Database Control ..... started.
------------------------------------------------------------------
Logs are generated in directory /u01/app/oracle/product/11.2.0/db_1/localhost_test/sysman/log

通过人工修改$ORACLE_HOME/localhost_sid/sysman/config/emoms.properties文件中的oracle.sysman.eml.mntr.emdRepPwd=密码，oracle.sysman.eml.mntr.emdRepPwdEncrypted=false实现修改密码

[oracle@localhost config]$ grep emdRepPwd emoms.properties|grep -v emdRepPwdSeed
oracle.sysman.eml.mntr.emdRepPwd=aa537afcbab41e05
oracle.sysman.eml.mntr.emdRepPwdEncrypted=TRUE

dbconsole重启之后，密码又重新被修改为加密方式,在11.2之前修改sysman密码，注意要同步修改emoms.properties中相关密码信息

sysman修改密码( 11.2 and higher)

[oracle@localhost config]$ emctl stop dbconsole
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = (unset),
        LANG = "c"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
Oracle Enterprise Manager 11g Database Control Release 11.2.0.4.0
Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.
https://localhost:1158/em/console/aboutApplication
Stopping Oracle Enterprise Manager 11g Database Control ...
 ...  Stopped.

SQL> alter user sysman identified by xifenfei123;
User altered.

[oracle@localhost ~]$ emctl setpasswd dbconsole
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = (unset),
        LANG = "c"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
Oracle Enterprise Manager 11g Database Control Release 11.2.0.4.0
Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.
https://localhost:1158/em/console/aboutApplication
Please enter new repository password:
Repository password successfully updated.
[oracle@localhost ~]$ emctl start dbconsole
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = (unset),
        LANG = "c"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
Oracle Enterprise Manager 11g Database Control Release 11.2.0.4.0
Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.
https://localhost:1158/em/console/aboutApplication
Starting Oracle Enterprise Manager 11g Database Control ..... started.
------------------------------------------------------------------
Logs are generated in directory /u01/app/oracle/product/11.2.0/db_1/localhost_test/sysman/log

Red Hat Enterprise Linux Version / Update	Red Hat Enterprise Linux – Kernel version / redhat-release string	Oracle Linux – Kernel version / release strings
*Red Hat Enterprise Linux 7*
Red Hat Enterprise Linux 7 Update 6	3.10.0-957.el7.x86_64 redhat-release: Red Hat Enterprise Linux Server release 7.6 (Maipo)	4.14.35-1818.3.3.el7uek.x86_64 ^ * (x86_64 only) 3.10.0-957.el7.x86_64 ^ (x86_64 only) redhat-release: Red Hat Enterprise Linux Server release 7.6 (Maipo) system-release: Oracle Linux Server release 7.6 oracle-release: Oracle Linux Server release 7.6
Red Hat Enterprise Linux 7 Update 5	3.10.0-862.el7.x86_64 redhat-release: Red Hat Enterprise Linux Server release 7.5 (Maipo)	4.1.12-112.16.4.el7uek ^ * (x86_64 only) 3.10.0-862.el7.x86_64 ^ (x86_64 only) redhat-release: Red Hat Enterprise Linux Server release 7.5 (Maipo) system-release: Oracle Linux Server release 7.5 oracle-release: Oracle Linux Server release 7.5
Red Hat Enterprise Linux 7 Update 4	3.10.0-693.el7.x86_64 redhat-release: Red Hat Enterprise Linux Server release 7.4 (Maipo)	4.1.12-94.3.9.el7uek ^ * (x86_64 only) 3.10.0-693.el7.x86_64 ^ (x86_64 only) redhat-release: Red Hat Enterprise Linux Server release 7.4 (Maipo) system-release: Oracle Linux Server release 7.4 oracle-release: Oracle Linux Server release 7.4
Red Hat Enterprise Linux 7 Update 3	3.10.0-514.el7.x86_64 redhat-release: Red Hat Enterprise Linux Server release 7.3 (Maipo)	4.1.12-61.1.18.el7uek ^ * (x86_64 only) 3.10.0-514.el7.x86_64 ^ (x86_64 only) redhat-release: Red Hat Enterprise Linux Server release 7.3 (Maipo) system-release: Oracle Linux Server release 7.3 oracle-release: Oracle Linux Server release 7.3
Red Hat Enterprise Linux 7 Update 2	3.10.0-327.el7.x86_64 redhat-release: Red Hat Enterprise Linux Server release 7.2 (Maipo)	3.8.13-98.6.1.el7uek ^ * (x86_64 only) 3.10.0-327.el7.x86_64 ^ (x86_64 only) redhat-release: Red Hat Enterprise Linux Server release 7.2 (Maipo) system-release: Oracle Linux Server release 7.2 oracle-release: Oracle Linux Server release 7.2
Red Hat Enterprise Linux 7 Update 1	3.10.0-229.el7.x86_64 redhat-release: Red Hat Enterprise Linux Server release 7.1 (Maipo)	3.8.13-55.1.6.el7uek ^ * (x86_64 only) 3.10.0-229.el7.x86_64 ^ (x86_64 only) redhat-release:Red Hat Enterprise Linux Server release 7.1 (Maipo) system-release:Oracle Linux Server release 7.1 oracle-release: Oracle Linux Server release 7.1
Red Hat Enterprise Linux 7 GA	3.10.0-123.el7.x86_64 redhat-release: Red Hat Enterprise Linux Server release 7.0 (Maipo)	3.8.13-35.3.1.el7uek ^ * (x86_64 only) 3.10.0-123.el7.x86_64 ^ (x86_64 only) redhat-release:Red Hat Enterprise Linux Server release 7.0 (Maipo) system-release:Oracle Linux Server release 7.0 oracle-release: Oracle Linux Server release 7.0
*Red Hat Enterprise Linux 6*
Red Hat Enterprise Linux Server 6 Update 10	2.6.32-754.el6 redhat-release: Red Hat Enterprise Linux Server release 6.10 (Santiago)	4.1.12-124.16.4.el6uek ^ * (x86_64 only) 2.6.39-400.299.3.el6uek ^ * (x86 only) 2.6.32-754.el6 ^ (x86, x86_64) redhat-release: Red Hat Enterprise Linux Server release 6.10 (Santiago) system-release: Oracle Linux Server release 6.10 oracle-release: Oracle Linux Server release 6.10
Red Hat Enterprise Linux Server 6 Update 9	2.6.32-696.el6 redhat-release: Red Hat Enterprise Linux Server release 6.9 (Santiago)	4.1.12-61.1.28.el6uek ^ * (x86_64 only) 2.6.39-400.294.3.el6uek ^ * (x86 only) 2.6.32-696.el6 ^ (x86, x86_64) redhat-release: Red Hat Enterprise Linux Server release 6.9 (Santiago) system-release: Oracle Linux Server release 6.9 oracle-release: Oracle Linux Server release 6.9
Red Hat Enterprise Linux Server 6 Update 8	2.6.32-642.el6 redhat-release: Red Hat Enterprise Linux Server release 6.8 (Santiago)	4.1.12-37.3.1.el6uek ^ * (x86_64 only) 2.6.39-400.278.2.el6uek ^ * (x86 only) 2.6.32-642.el6 ^ (x86, x86_64) redhat-release: Red Hat Enterprise Linux Server release 6.8 (Santiago) system-release: Oracle Linux Server release 6.8 oracle-release: Oracle Linux Server release 6.8
Red Hat Enterprise Linux Server 6 Update 7	2.6.32-573.el6 redhat-release: Red Hat Enterprise Linux Server release 6.7 (Santiago)	3.8.13-68.3.4.el6uek ^ * (x86_64 only) 2.6.39-400.250.7.el6uek ^ * (x86 only) 2.6.32-573.el6 ^ (x86, x86_64) redhat-release: Red Hat Enterprise Linux Server release 6.7 (Santiago) system-release: Oracle Linux Server release 6.7 oracle-release: Oracle Linux Server release 6.7
Red Hat Enterprise Linux Server 6 Update 6	2.6.32-504.el6 redhat-release: Red Hat Enterprise Linux Server release 6.6 (Santiago)	3.8.13-44.1.1.el6uek ^ * (x86_64 only) 2.6.39-400.215.10.el6uek ^ * (x86 only) 2.6.32-504.el6 ^ (x86, x86_64) redhat-release: Red Hat Enterprise Linux Server release 6.6 (Santiago) system-release: Oracle Linux Server release 6.6 oracle-release: Oracle Linux Server release 6.6
Red Hat Enterprise Linux Server 6 Update 5	2.6.32-431.el6 redhat-release: Red Hat Enterprise Linux Server release 6.5 (Santiago)	3.8.13-16.121.el6uek ^ * (x86_64 only) 2.6.39-400.211.1.el6uek ^ * (x86 only) 2.6.32-431.el6 ^ (x86, x86_64) redhat-release: Red Hat Enterprise Linux Server release 6.5 (Santiago) system-release: Oracle Linux Server release 6.5 oracle-release: Oracle Linux Server release 6.5
Red Hat Enterprise Linux Server 6 Update 4	2.6.32-358.el6 redhat-release: Red Hat Enterprise Linux Server release 6.4 (Santiago)	2.6.39-400.17.1.el6uek ^ * 2.6.32-358.el6 ^ redhat-release: Red Hat Enterprise Linux Server release 6.4 (Santiago) system-release: Oracle Linux Server release 6.4 oracle-release: Oracle Linux Server release 6.4
Red Hat Enterprise Linux Server 6 Update 3	2.6.32-279.el6 redhat-release: Red Hat Enterprise Linux Server release 6.3 (Santiago)	2.6.39-200.24.1.el6uek ^ * 2.6.32-279.el6 ^ redhat-release: Red Hat Enterprise Linux Server release 6.3 (Santiago) system-release: Oracle Linux Server release 6.3 oracle-release: Oracle Linux Server release 6.3
Red Hat Enterprise Linux Server 6 Update 2	2.6.32-220.el6 redhat-release: Red Hat Enterprise Linux Server release 6.2 (Santiago)	2.6.32-100.34.1.el6uek ^ * 2.6.32-220.el6 ^ redhat-release: Red Hat Enterprise Linux Server release 6.2 (Santiago) system-release: Oracle Linux Server release 6.2 oracle-release: Oracle Linux Server release 6.2
Red Hat Enterprise Linux Server 6 Update 1	2.6.32-131.el6 redhat-release: Red Hat Enterprise Linux Server release 6.1 (Santiago)	2.6.32-100.34.1.el6uek ^ * 2.6.32-131.0.15.el6 ^ redhat-release: Red Hat Enterprise Linux Server release 6.1 (Santiago) system-release: Oracle Linux Server release 6.1 oracle-release: Oracle Linux Server release 6.1
Red Hat Enterprise Linux Server 6 GA	2.6.32-71.el6 redhat-release: Red Hat Enterprise Linux Server release 6.0 (Santiago)	2.6.32-100.28.5.el6uek ^ * 2.6.32-71.el6 ^ redhat-release: Red Hat Enterprise Linux Server release 6.0 (Santiago) system-release: Oracle Linux Server release 6.0 oracle-release: Oracle Linux Server release 6.0
*Red Hat Enterprise Linux 5*
Red Hat Enterprise Linux Server 5 Update 11	2.6.18-398.el5 redhat-release: Red Hat Enterprise Linux Server release 5.11 (Tikanga)	*2.6.39-400.215.10.el5uek ^ (x86, x86_64) 2.6.18-398.el5 ^ (x86, x86_64) 2.6.18-398.0.0.0.1.el5 (x86, x86_64) redhat-release: Red Hat Enterprise Linux Server release 5.11 (Tikanga) enterprise-release: Enterprise Linux Enterprise Linux Server release 5.11 (Carthage) oracle-release: Oracle Linux Server release 5.11**
Red Hat Enterprise Linux Server 5 Update 10	2.6.18-371.el5 redhat-release: Red Hat Enterprise Linux Server release 5.10 (Tikanga)	2.6.39-400.209.1.el5uek ^ * (x86, x86_64) 2.6.18-371.el5 ^ (x86, x86_64) 2.6.18-371.0.0.0.1.el5 (x86, x86_64) redhat-release: Red Hat Enterprise Linux Server release 5.10 (Tikanga) enterprise-release: Enterprise Linux Enterprise Linux Server release 5.10 (Carthage) oracle-release: Oracle Linux Server release 5.10
Red Hat Enterprise Linux Server 5 Update 9	2.6.18-348.el5 redhat-release: Red Hat Enterprise Linux Server release 5.9 (Tikanga)	2.6.39-300.26.1.el5uek ^ * (x86, x86_64) 2.6.18-348.el5 ^ (x86, x86_64) 2.6.18-348.0.0.0.1.el5 (x86, x86_64) redhat-release: Red Hat Enterprise Linux Server release 5.9 (Tikanga) enterprise-release: Enterprise Linux Enterprise Linux Server release 5.9 (Carthage) oracle-release: Oracle Linux Server release 5.9
Red Hat Enterprise Linux Server 5 Update 8	2.6.18-308.el5 redhat-release: Red Hat Enterprise Linux Server release 5.8 (Tikanga)	2.6.32-300.10.1.el5uek ^ * (x86, x86_64) 2.6.18-308.el5 ^ (x86, x86_64) 2.6.18-308.0.0.0.1.el5 (x86, x86_64) redhat-release: Red Hat Enterprise Linux Server release 5.8 (Tikanga) enterprise-release: Enterprise Linux Enterprise Linux Server release 5.8 (Carthage) oracle-release: Oracle Linux Server release 5.8
Red Hat Enterprise Linux Server 5 Update 7	2.6.18-274.el5 redhat-release: Red Hat Enterprise Linux Server release 5.7 (Tikanga)	2.6.32-200.13.1.el5uek ^ * (x86, x86_64) 2.6.18-274.el5 ^ (x86, x86_64) 2.6.18-274.0.0.0.1.el5 (x86, x86_64) redhat-release: Red Hat Enterprise Linux Server release 5.7 (Tikanga) enterprise-release: Enterprise Linux Enterprise Linux Server release 5.7 (Carthage) oracle-release: Oracle Linux Server release 5.7
Red Hat Enterprise Linux Server 5 Update 6	2.6.18-238.el5 redhat-release: Red Hat Enterprise Linux Server release 5.6 (Tikanga)	2.6.32-100.26.2.el5uek ^ * (x86_64 only) 2.6.18-238.el5 ^ (x86, x86_64) 2.6.18-238.0.0.0.1.el5 (x86, x86_64) redhat-release: Red Hat Enterprise Linux Server release 5.6 (Tikanga) enterprise-release: Enterprise Linux Enterprise Linux Server release 5.6 (Carthage) oracle-release: Oracle Linux Server release 5.6
Red Hat Enterprise Linux Server 5 Update 5	2.6.18-194.el5 redhat-release: Red Hat Enterprise Linux Server release 5.5 (Tikanga)	2.6.18-194.el5 ^ * (x86, x86_64, ia64) 2.6.32-100.24.1.el5 # (x86_64 only) 2.6.18-194.0.0.0.3.el5 (x86, x86_64, ia64) redhat-release: Red Hat Enterprise Linux Server release 5.5 (Tikanga) enterprise-release: Enterprise Linux Enterprise Linux Server release 5.5 (Carthage) oracle-release: Oracle Linux Server release 5.5
Red Hat Enterprise Linux Server 5 Update 4	2.6.18-164.el5 redhat-release: Red Hat Enterprise Linux Server release 5.4 (Tikanga)	2.6.18-164.el5 ^ * (x86, x86_64, ia64) 2.6.18-164.0.0.0.1.el5 (x86, x86_64, ia64) UEK unavailable redhat-release: Red Hat Enterprise Linux Server release 5.4 (Tikanga) enterprise-release: Enterprise Linux Enterprise Linux Server release 5.4 (Carthage)
Red Hat Enterprise Linux Server 5 Update 3	2.6.18-128.el5 redhat-release: Red Hat Enterprise Linux Server release 5.3 (Tikanga)	2.6.18-128.el5 ^ * (x86, x86_64, ia64) 2.6.18-128.0.0.0.2.el5 (x86, x86_64, ia64) UEK unavailable redhat-release: Red Hat Enterprise Linux Server release 5.3 (Tikanga) enterprise-release: Enterprise Linux Enterprise Linux Server release 5.3 (Carthage)
Red Hat Enterprise Linux Server 5 Update 2	2.6.18-92.el5 redhat-release: Red Hat Enterprise Linux Server release 5.2 (Tikanga)	2.6.18-92.el5 ^ * (x86, x86_64, ia64) 2.6.18-92.0.0.0.1.el5 (x86, x86_64, ia64) UEK unavailable redhat-release: Enterprise Linux Enterprise Linux Server release 5.2 (Carthage) enterprise-release: Enterprise Linux Enterprise Linux Server release 5.2 (Carthage)
Red Hat Enterprise Linux Server 5 Update 1	2.6.18-53.el5 redhat-release: Red Hat Enterprise Linux Server release 5.1 (Tikanga)	2.6.18-53.el5 ^ * (x86, x86_64, ia64) 2.6.18-53.0.0.0.1.el5 (x86, x86_64, ia64) UEK unavailable redhat-release: Enterprise Linux Enterprise Linux Server release 5.1 (Carthage) enterprise-release: Enterprise Linux Enterprise Linux Server release 5.1 (Carthage)
Red Hat Enterprise Linux Server 5 GA	2.6.18-8.el5 redhat-release: Red Hat Enterprise Linux Server release 5 (Tikanga)	2.6.18-8.el5 ^ * (x86, x86_64, ia64) 2.6.18-8.0.0.4.1.el5 (x86, x86_64, ia64) UEK unavailable redhat-release: Enterprise Linux Enterprise Linux Server release 5 (Carthage) enterprise-release: Enterprise Linux Enterprise Linux Server release 5 (Carthage)
*Red Hat Enterprise Linux 4*
Red Hat Enterprise Linux 4 Update 9	2.6.9-100.EL redhat-release: Red Hat Enterprise Linux AS release 4 (Nahant Update 9)	*2.6.9-100.0.0.0.1.EL ^ (x86, x86_64, ia64) 2.6.9-100.EL (x86, x86_64, ia64) UEK unavailable redhat-release: Red Hat Enterprise Linux AS release 4 (Nahant Update 9) enterprise-release: Enterprise Linux Enterprise Linux AS release 4 (October Update 9)**
Red Hat Enterprise Linux 4 Update 8	2.6.9-89.EL redhat-release: Red Hat Enterprise Linux AS release 4 (Nahant Update 8)	2.6.9-89.0.0.0.1.EL ^ * (x86, x86_64, ia64) 2.6.9-89.EL (x86, x86_64, ia64) UEK unavailable redhat-release: Red Hat Enterprise Linux AS release 4 (Nahant Update 8) enterprise-release: Enterprise Linux Enterprise Linux AS release 4 (October Update 8)
Red Hat Enterprise Linux 4 Update 7	2.6.9-78.EL redhat-release: Red Hat Enterprise Linux AS release 4 (Nahant Update 7)	2.6.9-78.0.0.0.1.EL ^ * (x86, x86_64, ia64) 2.6.9-78.EL (x86, x86_64, ia64) UEK unavailable redhat-release: Enterprise Linux Enterprise Linux AS release 4 (October Update 7) enterprise-release: Enterprise Linux Enterprise Linux AS release 4 (October Update 7)
Red Hat Enterprise Linux 4 Update 6	2.6.9-67.EL redhat-release: Red Hat Enterprise Linux AS release 4 (Nahant Update 6)	2.6.9-67.0.0.0.1.EL ^ * (x86, x86_64, ia64) 2.6.9-67.EL ^ * (x86, x86_64, ia64) UEK unavailable redhat-release: Enterprise Linux Enterprise Linux AS release 4 (October Update 6) enterprise-release: Enterprise Linux Enterprise Linux AS release 4 (October Update 6)
Red Hat Enterprise Linux 4 Update 5	2.6.9-55.EL redhat-release: Red Hat Enterprise Linux AS release 4 (Nahant Update 5)	2.6.9-55.0.0.0.2.EL ^ * (x86, x86_64) 2.6.9-55.EL (x86, x86_64) UEK unavailable redhat-release: Enterprise Linux Enterprise Linux AS release 4 (October Update 5) enterprise-release: Enterprise Linux Enterprise Linux AS release 4 (October Update 5)
Red Hat Enterprise Linux 4 Update 4	2.6.9-42.EL redhat-release: Red Hat Enterprise Linux AS release 4 (Nahant Update 4	2.6.9-42.0.0.0.1.EL ^ * (x86, x86_64) 2.6.9-42.EL (x86, x86_64) UEK unavailable redhat-release: Enterprise Linux Enterprise Linux AS release 4 (October Update 4) enterprise-release: Enterprise Linux Enterprise Linux AS release 4 (October Update 4)
Red Hat Enterprise Linux 4 Update 3	2.6.9-34.EL redhat-release: Red Hat Enterprise Linux AS release 4 (Nahant Update 3)	No corresponding version
Red Hat Enterprise Linux 4 Update 2	2.6.9-22.EL redhat-release: Red Hat Enterprise Linux AS release 4 (Nahant Update 2)	No corresponding version
Red Hat Enterprise Linux 4 Update 1	2.6.9-11.EL redhat-release: Red Hat Enterprise Linux AS release 4 (Nahant Update 1)	No corresponding version
Red Hat Enterprise Linux 4 GA	2.6.9-5.EL redhat-release: Red Hat Enterprise Linux AS release 4 (Nahant)	No corresponding version

作者：admin