sqlplus 报 Segmentation Fault故障分析

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:sqlplus 报 Segmentation Fault故障分析

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

有客户反馈sqlplus 登录报错 Segmentation Fault
20210510190752


通过strace跟踪报错信息
20210510190943

上述显示和正常环境sqlplus一样,对应的lib信息一致
20210510191202

通过查询信息发现客户自行对glibc进行了升级
20210510190449
20210510191512

在mos上找到匹配文章说明,升级了glibc之后,sqlplus 报Segmentation Fault说明:Segmentation Fault when Running Sqlplus (Doc ID 2702934.1),在此提醒客户,运行oracle的数据库,升级glibc请谨慎

12c启动报kcratr_nab_less_than_odr

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:12c启动报kcratr_nab_less_than_odr

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

ORA-600 kcratr_nab_less_than_odr这个错误,以前的认知里面,主要是11.2.0.1中比较常见,这次在12c中见到了,记录下
数据库版本

Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options.
Windows NT Version V6.1  

数据库启动报错

Fri Apr 30 23:11:16 2021
Completed redo scan
 read 20 KB redo, 26 data blocks need recovery
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\orcl12c\orcl12c\trace\orcl12c_ora_5744.trc  (incident=28863):
ORA-00600: 内部错误代码, 参数: [kcratr_nab_less_than_odr], [1], [6872], [23015], [23017], [], [], [], [], [], [], []
Incident details in: D:\APP\ADMINISTRATOR\diag\rdbms\orcl12c\orcl12c\incident\incdir_28863\orcl12c_ora_5744_i28863.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Fri Apr 30 23:11:17 2021
Slave encountered ORA-10388 exception during crash recovery
Fri Apr 30 23:11:17 2021
Slave encountered ORA-10388 exception during crash recovery
Fri Apr 30 23:11:17 2021
Aborting crash recovery due to error 600
Fri Apr 30 23:11:17 2021
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\orcl12c\orcl12c\trace\orcl12c_ora_5744.trc:
ORA-00600: 内部错误代码, 参数: [kcratr_nab_less_than_odr], [1], [6872], [23015], [23017], [], [], [], [], [], [], []
Fri Apr 30 23:11:17 2021
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\orcl12c\orcl12c\trace\orcl12c_ora_5744.trc:
ORA-00600: 内部错误代码, 参数: [kcratr_nab_less_than_odr], [1], [6872], [23015], [23017], [], [], [], [], [], [], []
ORA-600 signalled during: alter database open...

这个故障处理比较简单参考:ORA-600 kcratr_nab_less_than_odr故障解决

oracle dmp被加密为.eking扩展名恢复

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:oracle dmp被加密为.eking扩展名恢复

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

又一客户数据库被勒索病毒加密,扩展名为:.id[32D2A259-3147].[mikolio@cock.li].eking

E:\BaiduNetdiskDownload>dir *.eking
 驱动器 E 中的卷是 SSD
 卷的序列号是 98A5-7F8E

 E:\BaiduNetdiskDownload 的目录

2021-05-04  01:55   162,604,986,658 ORACLEBAK20210503.DMP.id[32D2A259-3147].[mikolio@cock.li].eking
               1 个文件 162,604,986,658 字节
               0 个目录 262,026,616,832 可用字节

通过分析,确认只是少了的dmp数据被破坏
20210509174037


通过expdp dmp被加密破坏恢复工具进行恢复,实现绝大多数数据的完美恢复
20210509210046

如果此类的数据库(oracle,mysql,sql server)等被加密,需要专业恢复技术支持,请联系我们:
电话/微信:17813235971    Q Q:107644445QQ咨询惜分飞    E-Mail:dba@xifenfei.com

ORA-00704 ORA-00943故障恢复

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:ORA-00704 ORA-00943故障恢复

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

有朋友找到我们,他们数据库启动报ORA-01092 ORA-00704 ORA-00702错
ORA-01092-ORA-00704-ORA-00702


MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
SMON: enabling cache recovery
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\trace\orcl_ora_4868.trc:
ORA-00704: 引导程序进程失败
ORA-00702: 引导程序版本 '' 与版本 '8.0.0.0.0' 不一致
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\trace\orcl_ora_4868.trc:
ORA-00704: 引导程序进程失败
ORA-00702: 引导程序版本 '' 与版本 '8.0.0.0.0' 不一致
Error 704 happened during db open, shutting down database
USER (ospid: 4868): terminating the instance due to error 704
Thu May 06 09:20:28 2021
Instance terminated by USER, pid = 4868
ORA-1092 signalled during: ALTER DATABASE OPEN...

以前有过类似恢复经历:
2019恢复第一单—ORA-704-ORA-702恢复
恶意删除bootstrap$导致数据库无法正常启动
ORA-00702: bootstrap verison ” inconsistent with version ’8.0.0.0.0′
这个问题本身相对比较简单,客户根据网上介绍,自行尝试恢复,经过客户一系列的处理之后,数据库启动报ORA-00704 ORA-00943错

MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
SMON: enabling cache recovery
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\trace\orcl_ora_6600.trc:
ORA-00704: 引导程序进程失败
ORA-00943: 簇不存在
Errors in file D:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\trace\orcl_ora_6600.trc:
ORA-00704: 引导程序进程失败
ORA-00943: 簇不存在
Error 704 happened during db open, shutting down database
USER (ospid: 6600): terminating the instance due to error 704
Thu May 06 22:17:18 2021
Instance terminated by USER, pid = 6600
ORA-1092 signalled during: ALTER DATABASE OPEN...
opiodr aborting process unknown ospid (6600) as a result of ORA-1092

因为根据最初的错误知道是bootstrap$异常,现在报该错误,应该是bootstrap$没有处理正确导致,我们通过对bootstrap$重新处理,数据库直接open成功
open


win 11.2.0.4打patch后服务无法正常启动处理

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:win 11.2.0.4打patch后服务无法正常启动处理

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

最近一直困惑在win 11.2.0.4中打扩展patch的之后,oracle相关无法无法正常启动的问题,今天在朋友的指导下,终于搞定,这里具体描述下这次打patch的相关事宜.
0. win版本信息
20210508121352
1. 关闭oracle相关服务之后,打patch 提示CheckActiveFilesAndExecutables
20210508115852


对于这个问题,可以通过tasklist /M ora*命令找出相关进程,然后杀掉即可
20210508144815

2. 打上psu patch之后,相关服务无法启动
20210508143417
20210508143434

对于这个问题,是由于打上patch之后由于oracle程序调用的需要的MFC动态库不对,需要升级到Microsoft Visual C++ 2005 Service Pack 1 Redistributable Package MFC(https://www.microsoft.com/zh-cn/download/confirmation.aspx?id=26347)
20210508121006

启动相关服务正常
20210508144417

安装相关patch成功
20210508121147

Avaddon勒索病毒数据库恢复

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:Avaddon勒索病毒数据库恢复

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

接到朋友一个oracle数据库被加密的恢复请求,被加密文件为:
20210505193114


read.txt文件中信息

-------===    Your network has been infected!    ===-------





*****************DO NOT DELETE THIS FILE UNTIL ALL YOUR DATA HAVE BEEN RECOVERED*****************





All your documents, photos, databases and other important 

files have been encrypted and have the extension: .BCdadccBEA



You are not able to decrypt it by yourself. But don't worry, we can help you to restore all your files!



The only way to restore your files is to buy our special software. 
Only we can give you this software and only we can restore your files!



We have also downloaded a lot of private data from your network.

If you do not contact as in a 3 days we will post information about your breach 
on our public news website (avaddongun7rngel.onion) and after 7 days the whole downloaded info.



You can get more information on our page, which is located in a Tor hidden network.





How to get to our page

--------------------------------------------------------------------------------

|

|  1. Download Tor browser - https://www.torproject.org/

|

|  2. Install Tor browser

|

|  3. Open link in Tor browser - avaddonbotrxmuyl.onion

|

|  4. Follow the instructions on this page

|

--------------------------------------------------------------------------------



Your ID:

--------------------------------------------------------------------------------



MjQ4Ni1VeE5hL2hSVzJVeXU0Wm1CeHhhdDFLUDVGWTlqMnJFekZlczd3NlVFdnBROHYz…………



--------------------------------------------------------------------------------



* DO NOT TRY TO RECOVER FILES YOURSELF!



* DO NOT MODIFY ENCRYPTED FILES!



* * * OTHERWISE, YOU MAY LOSE ALL YOUR FILES FOREVER! * * *

YHSKC2aqLa0A1xzn

通过底层分析坏块情况,确认只是对文件头的127个block进行了破坏
20210505192823
由于客户是10g的版本,无法实现直接open库,然后expdp/exp导出数据.通过底层技术,直接恢复数据到新库,然后处理非表数据(index,view,proc,sequence等),实现最大限度恢复客户数据,最大程度减少客户整合数据的工作量
20210505194153


如果此类的数据库文件(oracle,mysql,sql server)等被加密,需要专业恢复技术支持,请联系我们:
电话/微信:17813235971    Q Q:107644445QQ咨询惜分飞    E-Mail:dba@xifenfei.com

expdp报ORA-39064 ORA-29285错误

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:expdp报ORA-39064 ORA-29285错误

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

expdp导出数据,报错ORA-39064 ORA-29285错误,导致datapump的logfile记录的日志不全

. . 导出了 "HCP"."HR_ADJ_SAL_ADV_SETUP_M"                  0 KB       0 行
ORA-39064: 无法写入日志文件
ORA-29285: 文件写入错误
. . 导出了 "HCP"."HR_ADJ_SAL_CONFIRM"                      0 KB       0 行

查询数据库NLS_CHARACTERSET

SQL> select NAME, value$ from props$ where name like 'NLS_CHARACTERSET';

NAME
------------------------------
VALUE$
--------------------------------------------------------------------------------
NLS_CHARACTERSET
UTF8

查看客户端字符集


SQL> select userenv('language') from dual;

USERENV('LANGUAGE')
--------------------------------------------------------------------------------
SIMPLIFIED CHINESE_CHINA.ZHS16GBK

出现这个问题,是由于expdp本身调用UTL_FILE,在Oracle Database PL/SQL Packages and Types Reference中有When data encoded in one character set is read and Globalization Support is told (such as by means of NLS_LANG) that it is encoded in another character set, the result is indeterminate. If NLS_LANG is set, it should be the same as the database character set.
基于这样的情况,通过设置NLS_LANG在客户端字符集和服务端一致,就不会出现该问题

ora-600 2662和ora-600 kclchkblk_4恢复

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:ora-600 2662和ora-600 kclchkblk_4恢复

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

这两天连续处理两个case,一个是12.1.0.2版本数据库屏蔽一致性,强制open之后,报ORA-600 2662故障
20210429220218


这个错误本身是一个非常常见的错误,直接推scn即可解决,但是问题是12.1.0.2版本,oracle不允许以前常规的操作方法,就连oradebug都报错oradebug poke ORA-32521/ORA-32519故障解决,而且可以是rac环境,bbed修改文件头也相当麻烦,最后我们使用patch方法轻松解决

另外一例是11.2.0.4版本,强制open库报ora-600 kclchkblk_4

Wed Apr 28 21:25:38 2021
SMON: enabling cache recovery
Instance recovery: looking for dead threads
Instance recovery: lock domain invalid but no dead threads
Errors in file /u01/app/oracle/diag/rdbms/dc/dc1/trace/dc1_ora_27832.trc  (incident=564430):
ORA-00600: internal error code, arguments: [kclchkblk_4], [2959], [904341694], [2959], [904131717], []
Incident details in: /u01/app/oracle/diag/rdbms/dc/dc1/incident/incdir_564430/dc1_ora_27832_i564430.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Errors in file /u01/app/oracle/diag/rdbms/dc/dc1/trace/dc1_ora_27832.trc:
ORA-00704: bootstrap process failure
ORA-00704: bootstrap process failure
ORA-00600: internal error code, arguments: [kclchkblk_4], [2959], [904341694], [2959], [904131717], []
Errors in file /u01/app/oracle/diag/rdbms/dc/dc1/trace/dc1_ora_27832.trc:
ORA-00704: bootstrap process failure
ORA-00704: bootstrap process failure
ORA-00600: internal error code, arguments: [kclchkblk_4], [2959], [904341694], [2959], [904131717], []
Error 704 happened during db open, shutting down database
USER (ospid: 27832): terminating the instance due to error 704
Instance terminated by USER, pid = 27832
ORA-1092 signalled during: alter database open resetlogs...

这个比较简单,参考redo异常 ORA-600 kclchkblk_4 故障恢复.

19c 打RU patch遇到oui-patch.xml (Permission denied)问题

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:19c 打RU patch遇到oui-patch.xml (Permission denied)问题

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

有一段时间没有做实施的活,今天安装一套19c rac,并且计划安装Patch 32545008 – GI Release Update 19.11.0.0.210420,在安装过程中遇到oui-patch.xml问题,记录下来供参考:
1. OPATCHAUTO-72088

[root@jbsbdb1 ~]# opatchauto apply /u01/soft/32545008

OPatchauto session is initiated at Wed Apr 28 14:20:24 2021

System initialization log file is /u01/app/19.0/grid/cfgtoollogs/opatchautodb/systemconfig2021-04-28_02-20-29PM.log.

Session log file is /u01/app/19.0/grid/cfgtoollogs/opatchauto/opatchauto2021-04-28_02-20-57PM.log
The id for this session is N2PG

Wrong OPatch software installed in following homes:
Host:jbsbdb2, Home:/u01/app/oracle/product/19.0/db_1

Host:jbsbdb2, Home:/u01/app/19.0/grid

OPATCHAUTO-72088: OPatch version check failed.
OPATCHAUTO-72088: OPatch software version in homes selected for patching are different.
OPATCHAUTO-72088: Please install same OPatch software in all homes.
OPatchAuto failed.

OPatchauto session completed at Wed Apr 28 14:21:15 2021
Time taken to complete the session 0 minute, 51 seconds

 opatchauto failed with error code 42

故障原因,只是升级了节点1的grid和oracle的opatch,把节点2的opatch也升级之后该问题解决

2. oui-patch.xml (Permission denied)文件处理

[root@jbsbdb2 soft]# opatchauto apply /u01/soft/32545008

OPatchauto session is initiated at Wed Apr 28 14:52:29 2021

System initialization log file is /u01/app/19.0/grid/cfgtoollogs/opatchautodb/systemconfig2021-04-28_02-52-32PM.log.

Session log file is /u01/app/19.0/grid/cfgtoollogs/opatchauto/opatchauto2021-04-28_02-52-58PM.log
The id for this session is T6ST

Executing OPatch prereq operations to verify patch applicability on home /u01/app/19.0/grid

Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/19.0/db_1
Patch applicability verified successfully on home /u01/app/19.0/grid

Patch applicability verified successfully on home /u01/app/oracle/product/19.0/db_1


Executing patch validation checks on home /u01/app/19.0/grid
Patch validation checks successfully completed on home /u01/app/19.0/grid


Executing patch validation checks on home /u01/app/oracle/product/19.0/db_1
Patch validation checks successfully completed on home /u01/app/oracle/product/19.0/db_1


Verifying SQL patch applicability on home /u01/app/oracle/product/19.0/db_1
SQL patch applicability verified successfully on home /u01/app/oracle/product/19.0/db_1


Preparing to bring down database service on home /u01/app/oracle/product/19.0/db_1
Successfully prepared home /u01/app/oracle/product/19.0/db_1 to bring down database service


Performing prepatch operations on CRS - bringing down CRS service on home /u01/app/19.0/grid
Prepatch operation log file location: 
/u01/app/grid/crsdata/jbsbdb2/crsconfig/crs_prepatch_apply_inplace_jbsbdb2_2021-04-28_02-54-34PM.log
CRS service brought down successfully on home /u01/app/19.0/grid


Performing prepatch operation on home /u01/app/oracle/product/19.0/db_1
Perpatch operation completed successfully on home /u01/app/oracle/product/19.0/db_1


Start applying binary patch on home /u01/app/oracle/product/19.0/db_1
Failed while applying binary patches on home /u01/app/oracle/product/19.0/db_1

Execution of [OPatchAutoBinaryAction] patch action failed, check log for more details. Failures:
Patch Target : jbsbdb2->/u01/app/oracle/product/19.0/db_1 Type[rac]
Details: [
---------------------------Patching Failed---------------------------------
Command execution failed during patching in home: /u01/app/oracle/product/19.0/db_1, host: jbsbdb2.
Command failed:  /u01/app/oracle/product/19.0/db_1/OPatch/opatchauto 
 apply /u01/soft/32545008 -oh /u01/app/oracle/product/19.0/db_1 -target_type rac_database
 -binary -invPtrLoc /u01/app/19.0/grid/oraInst.loc -jre /u01/app/19.0/grid/OPatch/jre -persistresult
/u01/app/oracle/product/19.0/db_1/opatchautocfg/db/sessioninfo/sessionresult_jbsbdb2_rac_2.ser -analyzedresult
 /u01/app/oracle/product/19.0/db_1/opatchautocfg/db/sessioninfo/sessionresult_analyze_jbsbdb2_rac_2.ser
Command failure output: 
==Following patches FAILED in apply:

Patch: /u01/soft/32545008/32545013
Log: /u01/app/oracle/product/19.0/db_1/cfgtoollogs/opatchauto/core/opatch/opatch2021-04-28_15-00-49PM_1.log
Reason: Failed during Patching: oracle.opatch.opatchsdk.OPatchException: 
ApplySession failed in system modification phase... 
'ApplySession::apply failed: java.io.IOException: oracle.sysman.oui.patch.PatchException: 
java.io.FileNotFoundException: /u01/app/oraInventory/ContentsXML/oui-patch.xml (Permission denied)' 

After fixing the cause of failure Run opatchauto resume

]
OPATCHAUTO-68061: The orchestration engine failed.
OPATCHAUTO-68061: The orchestration engine failed with return code 1
OPATCHAUTO-68061: Check the log for more details.
OPatchAuto failed.

OPatchauto session completed at Wed Apr 28 15:05:29 2021
Time taken to complete the session 13 minutes, 0 second

 opatchauto failed with error code 42

在节点1打patch成功之后,对节点2进行打patch,遇到类似:ApplySession::apply failed: java.io.IOException: oracle.sysman.oui.patch.PatchException:
java.io.FileNotFoundException: /u01/app/oraInventory/ContentsXML/oui-patch.xml (Permission denied)问题,通过查询mos发现类似文章opatchauto apply Results java.io.FileNotFoundException: /ContentsXML/oui-patch.xml (Permission denied) Error in Non-OUI Nodes (Doc ID 2582139.1),确认是bug 29859410 在20.1版本中修复.检查系统中对应oui-patch.xml信息.

[root@jbsbdb2 soft]# ls -l /u01/app/oraInventory/ContentsXML/oui-patch.xml
-rw-r----- 1 grid oinstall 174 Apr 28 14:04 /u01/app/oraInventory/ContentsXML/oui-patch.xml

确实文件权限不对,对其授权处理

[root@jbsbdb2 soft]# chmod 660 /u01/app/oraInventory/ContentsXML/oui-patch.xml
[root@jbsbdb2 soft]# ls -l /u01/app/oraInventory/ContentsXML/oui-patch.xml
-rw-rw---- 1 grid oinstall 174 Apr 28 14:04 /u01/app/oraInventory/ContentsXML/oui-patch.xml

处理oui-patch.xml文件异常之后,尝试回滚操作

[root@jbsbdb2 soft]# opatchauto rollback /u01/soft/32545008

OPatchauto session is initiated at Wed Apr 28 15:13:58 2021

System initialization log file is /u01/app/19.0/grid/cfgtoollogs/opatchautodb/systemconfig2021-04-28_03-14-00PM.log.

Session log file is /u01/app/19.0/grid/cfgtoollogs/opatchauto/opatchauto2021-04-28_03-14-28PM.log
The id for this session is KFYI

Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/19.0/db_1

Executing OPatch prereq operations to verify patch applicability on home /u01/app/19.0/grid
Patch applicability verified successfully on home /u01/app/19.0/grid

Patch applicability verification failed on home /u01/app/oracle/product/19.0/db_1

Execution of [OPatchAutoBinaryAction] patch action failed, check log for more details. Failures:
Patch Target : jbsbdb2->/u01/app/oracle/product/19.0/db_1 Type[rac]
Details: [
---------------------------Patching Failed---------------------------------
Command execution failed during patching in home: /u01/app/oracle/product/19.0/db_1, host: jbsbdb2.
Command failed:  /u01/app/oracle/product/19.0/db_1/OPatch/opatchauto  rollback
 /u01/soft/32545008 -oh /u01/app/oracle/product/19.0/db_1 
-target_type rac_database -binary -invPtrLoc /u01/app/19.0/grid/oraInst.loc 
-jre /u01/app/19.0/grid/OPatch/jre -persistresult 
/u01/app/oracle/product/19.0/db_1/opatchautocfg/db/sessioninfo/sessionresult_analyze_jbsbdb2_rac_2.ser
 -analyze -online -prepare_home
Command failure output: 
==Following patches FAILED in analysis for rollback:

Patch: /u01/soft/32545008/32579761
Log: 
Reason: Failed during listing in Analysis: java.lang.Exception: oracle.opatch.opatchsdk.OPatchException: 
Unable to create patchObject
Possible causes are:
   ORACLE_HOME/inventory/oneoffs/32545013 is corrupted. PatchObject constructor: 
Input file "/u01/app/oracle/product/19.0/db_1/inventory/oneoffs/32545013/etc/config/actions" 
or "/u01/app/oracle/product/19.0/db_1/inventory/oneoffs/32545013/etc/config/inventory" does not exist.


Patch: /u01/soft/32545008/32545013
Log: 
Reason: Failed during listing in Analysis: java.lang.Exception: oracle.opatch.opatchsdk.OPatchException: 
Unable to create patchObject
Possible causes are:
   ORACLE_HOME/inventory/oneoffs/32545013 is corrupted. PatchObject constructor:
 Input file "/u01/app/oracle/product/19.0/db_1/inventory/oneoffs/32545013/etc/config/actions" 
or "/u01/app/oracle/product/19.0/db_1/inventory/oneoffs/32545013/etc/config/inventory" does not exist. 

After fixing the cause of failure Run opatchauto resume

]
OPATCHAUTO-68061: The orchestration engine failed.
OPATCHAUTO-68061: The orchestration engine failed with return code 1
OPATCHAUTO-68061: Check the log for more details.
OPatchAuto failed.

OPatchauto session completed at Wed Apr 28 15:14:50 2021
Time taken to complete the session 0 minute, 53 seconds

 opatchauto failed with error code 42

检查发现/u01/app/oracle/product/19.0/db_1/inventory/oneoffs/32545013确实不存在,从节点1把该文件夹tar过来,然后再次回滚

[root@jbsbdb2 ~]# /u01/app/oracle/product/19.0/db_1/OPatch/opatchauto rollback /u01/soft/32545008 -oh 
>/u01/app/oracle/product/19.0/db_1

OPatchauto session is initiated at Wed Apr 28 16:12:33 2021

System initialization log file is 
/u01/app/oracle/product/19.0/db_1/cfgtoollogs/opatchautodb/systemconfig2021-04-28_04-12-36PM.log.

Session log file is /u01/app/oracle/product/19.0/db_1/cfgtoollogs/opatchauto/opatchauto2021-04-28_04-12-53PM.log
The id for this session is JRS3

Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/19.0/db_1
Patch applicability verified successfully on home /u01/app/oracle/product/19.0/db_1


Executing patch validation checks on home /u01/app/oracle/product/19.0/db_1
Patch validation checks successfully completed on home /u01/app/oracle/product/19.0/db_1


Verifying SQL patch applicability on home /u01/app/oracle/product/19.0/db_1
SQL patch applicability verified successfully on home /u01/app/oracle/product/19.0/db_1


Preparing to bring down database service on home /u01/app/oracle/product/19.0/db_1
Successfully prepared home /u01/app/oracle/product/19.0/db_1 to bring down database service


Bringing down database service on home /u01/app/oracle/product/19.0/db_1
Following database(s) and/or service(s) are stopped and will be restarted later during the session: racdb
Database service successfully brought down on home /u01/app/oracle/product/19.0/db_1


Performing prepatch operation on home /u01/app/oracle/product/19.0/db_1
Perpatch operation completed successfully on home /u01/app/oracle/product/19.0/db_1


Start rolling back binary patch on home /u01/app/oracle/product/19.0/db_1
Binary patch rolled back successfully on home /u01/app/oracle/product/19.0/db_1


Performing postpatch operation on home /u01/app/oracle/product/19.0/db_1
Postpatch operation completed successfully on home /u01/app/oracle/product/19.0/db_1


Starting database service on home /u01/app/oracle/product/19.0/db_1
Database service successfully started on home /u01/app/oracle/product/19.0/db_1


Preparing home /u01/app/oracle/product/19.0/db_1 after database service restarted
No step execution required.........
 

Trying to roll back SQL patch on home /u01/app/oracle/product/19.0/db_1
SQL patch rolled back successfully on home /u01/app/oracle/product/19.0/db_1

OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:jbsbdb2
RAC Home:/u01/app/oracle/product/19.0/db_1
Version:19.0.0.0.0
Summary:

==Following patches were SKIPPED:

Patch: /u01/soft/32545008/32576499
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /u01/soft/32545008/32585572
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /u01/soft/32545008/32584670
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /u01/soft/32545008/32579761
Reason: This Patch does not exist in the home, it cannot be rolled back.


==Following patches were SUCCESSFULLY rolled back:

Patch: /u01/soft/32545008/32545013
Log: /u01/app/oracle/product/19.0/db_1/cfgtoollogs/opatchauto/core/opatch/opatch2021-04-28_16-15-05PM_1.log

回滚成功,直接使用opatchauto apply /u01/soft/32545008打节点2成功
3. RU安装成功结果

[grid@jbsbdb2 ~]$ opatch lspatches
32585572;DBWLM RELEASE UPDATE 19.0.0.0.0 (32585572)
32584670;TOMCAT RELEASE UPDATE 19.0.0.0.0 (32584670)
32579761;OCW RELEASE UPDATE 19.11.0.0.0 (32579761)
32576499;ACFS RELEASE UPDATE 19.11.0.0.0 (32576499)
32545013;Database Release Update : 19.11.0.0.210420 (32545013)

OPatch succeeded.

[oracle@jbsbdb2 ~]$ opatch lspatches
32579761;OCW RELEASE UPDATE 19.11.0.0.0 (32579761)
32545013;Database Release Update : 19.11.0.0.210420 (32545013)

OPatch succeeded.
[grid@jbsbdb2 ~]$ crsctl query crs softwarepatch
Oracle Clusterware patch level on node jbsbdb2 is [3331580692].
[grid@jbsbdb2 ~]$ crsctl query crs activeversion -f
Oracle Clusterware active version on the cluster is [19.0.0.0.0]. 
The cluster upgrade state is [NORMAL]. The cluster active patch level is [3331580692].

SQL> select PATCH_ID,DESCRIPTION from dba_registry_sqlpatch;

  PATCH_ID
----------
DESCRIPTION
--------------------------------------------------------------------------------
  29517242
Database Release Update : 19.3.0.0.190416 (29517242)

  32545013
Database Release Update : 19.11.0.0.210420 (32545013)

在打patch之前已经知晓该问题,比较粗心的把oui-patch.xml文件从节点1scp拷贝到节点2,没有确认oui-patch.xml权限(拷贝过来是640)从而引起后续很多麻烦

permission.pl处理ORACLE目录权限被误操作故障

联系:手机/微信(+86 17813235971) QQ(107644445)QQ咨询惜分飞

标题:permission.pl处理ORACLE目录权限被误操作故障

作者:惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

在以前的案例中,多次出现由于误操作修改oracle rac相关目录权限,导致集群无法启动,以前官方给出来的解决方案,大部分情况是通过删除节点,增加解决的方式解决.在翻看最近的mos文档时发现Script to capture and restore file permission in a directory (for eg. ORACLE_HOME) (Doc ID 1515018.1),通过permission.pl来记录正常节点的权限,然后在异常节点执行(注意需要替换主机名).通过对该脚本简单测试,确认大概效果:
1. 上传脚本并且执行

[root@localhost tmp]# chmod +x permission.pl 
[root@localhost tmp]# ./permission.pl /u01

]
2. 生成对应文件

[root@localhost tmp]# ls -l *perm*
-rwxr-xr-x. 1 root root     2451 4ÔÂ  26 14:23 permission.pl
-rw-r--r--. 1 root root  6918174 4ÔÂ  27 10:56 permission-¶þ-4ÔÂ-27-10-55-19-2021
-rw-r--r--. 1 root root 13442294 4ÔÂ  27 10:56 restore-perm-¶þ-4ÔÂ-27-10-55-19-2021.cmd

3. 查看相关文件内容
permission-


restore-perm-