Quick Reference to Patch Numbers for Database/GI PSU, SPU(CPU), Bundle Patches and Patchsets—201708

Patchsets

12.1.0.2 (12.1.0.2.0 PATCH SET FOR ORACLE DATABASE SERVER)

21419221

11.2.0.4 (11.2.0.4.0 PATCH SET FOR ORACLE DATABASE SERVER)

13390677

11.2.0.3 (11.2.0.3.0 PATCH SET FOR ORACLE DATABASE SERVER)

10404530

11.2.0.2 (11.2.0.2.0 PATCH SET FOR ORACLE DATABASE SERVER)

10098816

11.1.0.7 (11.1.0.7.0 PATCH SET FOR ORACLE DATABASE SERVER)

6890831

10.2.0.5 (10.2.0.5 PATCH SET FOR ORACLE DATABASE SERVER)

8202632

A10.2.0.4 (10.2.0.4.0 PATCH SET FOR ORACLE DATABASE SERVER)

6810189

B10.2.0.3 (10.2.0.3 PATCH SET FOR ORACLE DATABASE SERVER)

5337014

10.2.0.2 (10.2.0.2 PATCH SET FOR ORACLE DATABASE SERVER)

4547817

10.1.0.5 (10.1.0.5 PATCH SET FOR ORACLE DATABASE SERVER)

4505133

10.1.0.4 (10.1.0.4 PATCH SET FOR ORACLE DATABASE SERVER)

4163362

10.1.0.3 (10.1.0.3 PATCH SET FOR ORACLE DATABASE SERVER)

3761843

9.2.0.8 (9.2.0.8 PATCH SET FOR ORACLE DATABASE SERVER)

4547809

9.2.0.7 (9.2.0.7 PATCH SET FOR ORACLE DATABASE SERVER)

4163445

9.2.0.6 (9.2.0.6 PATCH SET FOR ORACLE DATABASE SERVER)

3948480

9.2.0.5 (ORACLE 9I DATABASE SERVER RELEASE 2 – PATCH SET 4 VERSION 9.2.0.5.0)

3501955

9.2.0.4 (9.2.0.4 PATCH SET FOR ORACLE DATABASE SERVER)

3095277

9.2.0.3 (9.2.0.3 PATCH SET FOR ORACLE DATABASE SERVER)

2761332

9.2.0.2 (9.2.0.2 PATCH SET FOR ORACLE DATABASE SERVER)

2632931

9.0.1.5 (9.0.1.5 PATCHSET)

3301544

9.0.1.4 (9.0.1.4 PATCH SET FOR ORACLE DATABASE SERVER)

2517300

9.0.1.3 (9.0.1.3. PATCH SET FOR ORACLE DATA SERVER)

2271678

8.1.7.4 (8.1.7.4 PATCH SET FOR ORACLE DATA SERVER)

2376472

8.1.7.3 (8.1.7.3 PATCH SET FOR ORACLE DATA SERVER)

2189751

8.1.7.2 (8.1.7.2.1 PATCH SET FOR ORACLE DATA SERVER)

1909158

 
 

12.2.0.1

Description

Database RU

GI RU

Windows Bundle Patch

AUG2017 (12.2.0.1.170814)

26609817

26610291

26204214

JUL2017 (12.2.0.1.170718)

26123830

26133434

26204212

 

12.1.0.2

Description

PSU

GI PSU

Proactive Bundle Patch

Bundle Patch(Windows 32bit & 64bit)

AUG2017(12.1.0.2.170814)

26609783

26610308

26610322

26161726

JUL2017 (12.1.0.2.170718)

25755742

25901062

26022196

26161724

APR2017 (12.1.0.2.170418)

25171037

25434003

25433352

25632533

JAN2017 (12.1.0.2.170117)

24732082

24917825

24968615

25115951

OCT2016 (12.1.0.2.161018)

24006101

24412235

24448103

24591642

JUL2016 (12.1.0.2.160719)

23054246

23273629

23273686

23530387

APR2016 (12.1.0.2.160419)

22291127

22646084

22899531

22809813

JAN2016 (12.1.0.2.160119)

21948354

22191349

22243551

22310559

OCT2015

21359755(12.1.0.2.5)

21523234(12.1.0.2.5)

21744410(12.1.0.2.13)

21821214(12.1.0.2.10)

JUL2015

20831110(12.1.0.2.4)

20996835(12.1.0.2.4)

21188742(12.1.0.2.10)

21126814(12.1.0.2.7)

APR2015

20299023(12.1.0.2.3)

20485724(12.1.0.2.3)

20698050(12.1.0.2.7)

20684004(12.1.0.2.4)

JAN2015

19769480(12.1.0.2.2)

19954978(12.1.0.2.2)

20141343(12.1.0.2.4)

19720843(12.1.0.2.1)

OCT2014

19303936(12.1.0.2.1)

19392646(12.1.0.2.1)

19404326(12.1.0.2.1)

N/A

 

12.1.0.1

Description

PSU

GI PSU

Bundle Patch

A

B

Windows 64 bit

Windows 32 bit

JUL2016 (12.1.0.1.160719)

23054354

23273935

23273958

23530410

APR2016 (12.1.0.1.160419)

22291141

22654153

22654166

22839614

JAN2016 (12.1.0.1.160119)

21951844

22191492

22191511

22494866

OCT2015

21352619(12.1.0.1.9)

21551666(12.1.0.1.9)

21551685(12.1.0.1.9)

21744907 (12.1.0.1.21)

JUL2015

20831107(12.1.0.1.8)

20996901(12.1.0.1.8)

20996911(12.1.0.1.8)

21076681 (12.1.0.1.20)

APR2015

20299016(12.1.0.1.7)

20485762(12.1.0.1.7)

19971331(12.1.0.1.7)

20558101 (12.1.0.1.18)

JAN2015

19769486(12.1.0.1.6)

19971324(12.1.0.1.6)

19971331(12.1.0.1.6)

20160748 (12.1.0.1.16)

OCT2014

19121550(12.1.0.1.5)

19392372(12.1.0.1.5)

19392451(12.1.0.1.5)

19542943 (12.1.0.1.14)

JUL2014

18522516(12.1.0.1.4)

18705901(12.1.0.1.4)

18705972(12.1.0.1.4)

19062327 (12.1.0.1.11)

APR2014

18031528(12.1.0.1.3)

18139660(12.1.0.1.3)

18413105(12.1.0.1.3)

18448604 (12.1.0.1.7)

JAN2014

17552800(12.1.0.1.2)

17735306 (12.1.0.1.2)

17977915 (12.1.0.1.3)

OCT2013

17027533(12.1.0.1.1)

17272829 (12.1.0.1.1)

17363796(12.1.0.1.1)

17363795(12.1.0.1.1)

 

11.2.0.4

Description

PSU

SPU(CPU)

GI PSU

Bundle Patch (Windows 32bit & 64bit)

AUG2017 (11.2.0.4.170814)

26609445

N/A

26610246

26194138

JUL2017 (11.2.0.4.170718)

25869727

25879656

26030799

26194136

APR2017 (11.2.0.4.170418)

24732075

25369547

25476126

25632525

JAN2017

N/A

N/A

N/A

N/A

OCT2016 (11.2.0.4.161018)

24006111

24433711

24436338

24591646

JUL2016 (11.2.0.4.160719)

23054359

23177648

23274134

23530402

APR2016 (11.2.0.4.160419)

22502456

22502493

22646198

22839608

JAN2016 (11.2.0.4.160119)

21948347

21972320

22191577

22310544

OCT2015

21352635 (11.2.0.4.8)

21352646

21523375 (11.2.0.4.8)

21821802 (11.2.0.4.20)

JUL2015

20760982 (11.2.0.4.7)

20803583

20996923 (11.2.0.4.7)

21469106 (11.2.0.4.18)

APR2015

20299013 (11.2.0.4.6)

20299015

20485808 (11.2.0.4.6)

20544696 (11.2.0.4.15)

JAN2015

19769489 (11.2.0.4.5)

19854503

19955028 (11.2.0.4.5)

20127071 (11.2.0.4.12)

OCT2014

19121551 (11.2.0.4.4)

19271443

19380115 (11.2.0.4.4)

19651773 (11.2.0.4.10)

JUL2014

18522509 (11.2.0.4.3)

18681862

18706472 (11.2.0.4.3)

18842982 (11.2.0.4.7)

APR2014

18031668 (11.2.0.4.2)

18139690

18139609 (11.2.0.4.2)

18296644 (11.2.0.4.4)

JAN2014

17478514 (11.2.0.4.1)

17551709

N/A

17987366 (11.2.0.4.1)

11.2.0.3

Description

PSU

SPU(CPU)

GI PSU

Bundle Patch (Windows 64bit)

Bundle Patch (Windows 32bit)

JUL2015

20760997 (11.2.0.3.15)

20803576

20996944 (11.2.0.3.15)

21104036

21104035

APR2015

20299017 (11.2.0.3.14)

20299010

20485830 (11.2.0.3.14)

20420395

20420394

JAN2015

19769496 (11.2.0.3.13)

19854461

19971343 (11.2.0.3.13)

20233168

20233167

OCT2014

19121548 (11.2.0.3.12)

19271438

19440385 (11.2.0.3.12)

19618575

19618574

JUL2014

18522512 (11.2.0.3.11)

18681866

18706488 (11.2.0.3.11)

18940194

18940193

APR2014

18031683 (11.2.0.3.10)

18139695

18139678 (11.2.0.3.10)

18372244

18372243

JAN2014

17540582 (11.2.0.3.9)

17478415

17735354 (11.2.0.3.9)

18075406

17906981

OCT2013

16902043 (11.2.0.3.8)

17082364

17272731 (11.2.0.3.8)

17363850

17363844

JUL2013

16619892 (11.2.0.3.7)

16742095

16742216 (11.2.0.3.7)

16803775

16803774

APR2013

16056266 (11.2.0.3.6)

16294378

16083653 (11.2.0.3.6)

16345834

16345833

JAN2013

14727310 (11.2.0.3.5)

14841409

14727347 (11.2.0.3.5)

16042648

16042647

OCT2012

14275605 (11.2.0.3.4)

14390252

14275572 (11.2.0.3.4)

14613223

14613222

JUL2012

13923374 (11.2.0.3.3)

14038787

13919095 (11.2.0.3.3)

14223718

14223717

APR2012

13696216 (11.2.0.3.2)

13632717

13696251 (11.2.0.3.2)

13885389

13885388

JAN2012

13343438 (11.2.0.3.1)

13466801

13348650 (11.2.0.3.1)

13413168

13413167

 

11.2.0.2

Description

PSU

SPU(CPU)

GI PSU

Bundle Patch (Windows 64bit)

Bundle Patch (Windows 32bit)

OCT2013

17082367 (11.2.0.2.12)

17082375

17272753 (11.2.0.2.12)

17363838

17363837

JUL2013

16619893 (11.2.0.2.11)

16742100

16742320 (11.2.0.2.11)

16345852

16345851

APR2013

16056267 (11.2.0.2.10)

16294412

16166868 (11.2.0.2.10)

16345846

16345845

JAN2013

14727315 (11.2.0.2.9)

14841437

14841385 (11.2.0.2.9)

16100399

16100398

OCT2012

14275621 (11.2.0.2.8)

14390377

14390437 (11.2.0.2.8)

14672268

14672267

JUL2012

13923804 (11.2.0.2.7)

14038791

14192201 (11.2.0.2.7)

14134043

14134042

APR2012

13696224 (11.2.0.2.6)

13632725

13696242 (11.2.0.2.6)

13697074

13697073

JAN2012

13343424 (11.2.0.2.5)

13343244

13653086 (11.2.0.2.5)

13413155

13413154

OCT2011

12827726 (11.2.0.2.4)

12828071

12827731 (11.2.0.2.4)

13038788

13038787

JUL2011

12419331 (11.2.0.2.3)

12419321

12419353 (11.2.0.2.3)

12714463

12714462

APR2011

11724916 (11.2.0.2.2)

11724984

12311357 (11.2.0.2.2)

11896292

11896290

JAN2011

10248523 (11.2.0.2.1)

N/A

N/A

10432053

10432052

 

11.2.0.1

Description

PSU

CPU

Bundle Patch (Windows 64bit)

Bundle Patch (Windows 32bit)

JUL2011

12419378 (11.2.0.1.6)

12419278

12429529

12429528

APR2011

11724930 (11.2.0.1.5)

11724991

11731176

11883240

JAN2011

10248516 (11.2.0.1.4)

10249532

10432045

10432044

OCT2010

9952216 (11.2.0.1.3)

9952260

10100101

10100100

JUL2010

9654983 (11.2.0.1.2)

9655013

9736865

9736864

APR2010

9352237 (11.2.0.1.1)

9369797

N/A

N/A

10.2.0.5

Description

PSU

SPU(CPU)

Bundle Patch (Windows 64bit)

Bundle Patch (Windows 32bit)

Bundle Patch (Windows Itanium)

JUL2015

20299014(10.2.0.5.19)

20299021

20420387

20420386

N/A

APR2015

N/A

N/A

N/A

N/A

N/A

JAN2015

19769505(10.2.0.5.18)

19854436

20126868

20126867

N/A

OCT2014

19274523(10.2.0.5.17)

19274521

19618565

19618563

N/A

JUL2014

18522511(10.2.0.5.16)

18681879

18940198

18940196

N/A

APR2014

18031728(10.2.0.5.15)

18139709

18372261

18372259

N/A

JAN2014

17465584(10.2.0.5.14)

17551414

17906974

17906972

N/A

OCT2013

17082365(10.2.0.5.13)

17082371

N/A

17363822

N/A

JUL2013

16619894(10.2.0.5.12)

16742123

16803782

16803780

16803781

APR2013

16056270(10.2.0.5.11)

16270946

16345857

16345855

16345856

JAN2013

14727319(10.2.0.5.10)

14841459

15848062

15848060

15848061

OCT2012

14275629(10.2.0.5.9)

14390396

14553358

14553356

14553357

JUL2012

13923855(10.2.0.5.8)

14038805

14134053

14134051

14134052

APR2012

13632743(10.2.0.5.7)

13632738

13654815

13654814

13870404

JAN2012

13343471(10.2.0.5.6)

13343467

13460968

13460967

N/A

OCT2011

12827745(10.2.0.5.5)

12828105

N/A

12914911

N/A

JUL2011

12419392(10.2.0.5.4)

12419258

12429524

12429523

N/A

APR2011

11724962(10.2.0.5.3)

11725006

12328269

12328268

N/A

JAN2011

10248542(10.2.0.5.2)

10249537

10352673

10352672

N/A

OCT2010

9952230(10.2.0.5.1)

9952270

10099855

10058290

N/A

 

10.2.0.4

Description

PSU

SPU(CPU)

Bundle Patch (Windows 32bit)

Bundle Patch (Windows 64bit)

Bundle Patch (Windows Itanium)

JUL2013

16619897 (10.2.0.4.17)

16742253

N/A

N/A

N/A

APR2013

16056269 (10.2.0.4.16)

16270931

N/A

N/A

N/A

JAN2013

14736542 (10.2.0.4.15)

14841471

N/A

N/A

N/A

OCT2012

14275630 (10.2.0.4.14)

14390410

N/A

N/A

N/A

JUL2012

13923851 (10.2.0.4.13)

14038814

N/A

N/A

N/A

APR2012

12879933 (10.2.0.4.12)

12879926

13928775

13928776

N/A

JAN2012

12879929 (10.2.0.4.11)

12879912

13654060

N/A

N/A

OCT2011

12827778 (10.2.0.4.10)

12828112

12914908

12914910

12914909

JUL2011

12419397 (10.2.0.4.9)

12419249

12429519

12429521

12429520

APR2011

11724977 (10.2.0.4.8)

11725015

12328501

12328503

12328502

JAN2011

10248636 (10.2.0.4.7)

10249540

10349197

10349200

10349198

OCT2010

9952234 (10.2.0.4.6)

9952272

10084980

10084982

10084981

JUL2010

9654991 (10.2.0.4.5)

9655017

9777076

9777078

9777077

APR2010

9352164 (10.2.0.4.4)

9352191

9393548

9393550

9393549

JAN2010

9119284 (10.2.0.4.3)

9119226

9169457

9169460

9169458

OCT2009

8833280 (10.2.0.4.2)

8836308

8880857

8880861

8880858

JUL2009

8576156 (10.2.0.4.1)

8534387

8559466

8559467

8541782

APR2009

N/A

8290506

8307237

8307238

8333678

JAN2009

N/A

7592346

7584866

7584867

N/A

OCT2008

N/A

7375644

7386320

7386321

N/A

JUL2008

N/A

7150470

7218676

7218677

N/A

10.2.0.3

Description

CPU (Unix/Linux)

Bundle Patch (Windows 32bit)

Bundle Patch (Windows Itanium)

Bundle Patch (Windows 64bit)

JAN2009

7592354

7631956

7631958

7631957

OCT2008

7369190

7353782

7353784

7353785

JUL2008

7150622

7252496

7252497

7252498

APR2008

6864068

6867054

6867055

6867056

JAN2008

6646853

6637237

6637238

6637239

OCT2007

6394981

6430171

6430173

6430174

JUL2007

6079591

6116131

6038242

6116139

APR2007

5901891

5948242

5916262

5948243

JAN2007

5881721

5846376

5846377

5846378

 

10.2.0.2

Description

CPU (Unix/Linux)

Bundle Patch (Windows 32bit)

Bundle Patch (Windows 64bit)

Bundle Patch (Windows Itanium)

JAN2009

7592355

N/A

N/A

N/A

OCT2008

7375660

N/A

N/A

N/A

JUL2008

7154083

N/A

N/A

N/A

APR2008

6864071

N/A

N/A

N/A

JAN2008

6646850

N/A

N/A

N/A

OCT2007

6394997

6397028

6397030

6397029

JUL2007

6079588

6013105

6013121

6013118

APR2007

5901881

5912173

5912179

5912176

JAN2007

5689957

5716143

5699839

5699824

OCT2006

5490848

5502226

5500921

5500894

JUL2006

5225799

5251025

5251028

5251026

APR2006

5079037

5140461

5140567

5140508

 

10.2.0.1

Description

CPU (Unix/Linux)

Bundle Patch (Windows 32bit)

Bundle Patch (Windows 64bit)

Bundle Patch (Windows Itanium)

APR2007

5901880

N/A

N/A

N/A

JAN2007

5689937

5695784

5695786

5695785

OCT2006

5490846

5500927

5500954

5500951

JUL2006

5225798

5239698

5239701

5239699

APR2006

5049080

5059238

5059261

5059251

JAN2006

4751931

4751539

4770480

4751549

 

10.1.0.5

Description

CPU (Unix/Linux)

Bundle Patch (Windows 32bit)

Bundle Patch (Windows Itanium)

JAN2012

13343482

13413002

13413003

OCT2011

12828135

12914905

12914906

JUL2011

12419228

12429517

12429518

APR2011

11725035

11731119

11731120

JAN2011

N/A

N/A

N/A

OCT2010

9952279

10089559

10089560

JUL2010

9655023

9683651

9683652

APR2010

9352208

9390288

9390289

JAN2010

9119261

9187104

9187105

OCT2009

8836540

8785211

8785212

JUL2009

8534394

8656224

8656226

APR2009

8290534

8300356

8300360

JAN2009

7592360

7486619

7586049

OCT2008

7375686

7367493

7367494

JUL2008

7154097

7047034

7047037

APR2008

6864078

6867107

6867108

JAN2008

6647005

6637274

6637275

OCT2007

6395024

6408393

6408394

JUL2007

6079585

6115804

6115818

APR2007

5901877

5907304

5907305

JAN2007

5689908

5716295

5634747

OCT2006

5490845

5500883

5500885

JUL2006

5225797

5251148

5251140

APR2006

5049074

5057606

5057609

JAN2006

4751932

4882231

4882236

 

10.1.0.4

Description

CPU (Unix/Linux)

Bundle Patch (Windows 32bit)

Bundle Patch (Windows Itanium)

APR2007

5901876

5909871

5909879

JAN2007

5689894

5695771

5695772

OCT2006

5490844

5500878

5500880

JUL2006

5225796

5239736

5239737

APR2006

5049067

5059200

5059227

JAN2006

4751928

4751259

4745040

OCT2005

4567866

4579182

4579188

JUL2005

4392423

4440706

4404600

APR2005

4210374

4287619

4287611

 

10.1.0.3

Description

CPU (Unix/Linux)

Bundle Patch (Windows 32bit)

Bundle Patch (Windows Itanium)

JAN2007

5923277

N/A

N/A

OCT2006

5566825

N/A

N/A

JUL2006

5435164

N/A

N/A

APR2006

5158022

N/A

N/A

JAN2006

4751926

4741077

4741084

OCT2005

4567863

4567518

4567523

JUL2005

4392409

4389012

4389014

APR2005

4193286

4269715

4158888

JAN2005

4003062

4074232

3990812

 

10.1.0.2

Description

CPU (Unix/Linux)

Bundle Patch (Windows 32bit)

Bundle Patch (Windows Itanium)

APR2005

4193293

4181849

4213305

JUL2005

4400766

4388944

4388948

JAN2005

4003051

4104364

4083038

 

9.2.0.8

Description

CPU (Unix/Linux)

Bundle Patch (Windows 32bit)

Bundle Patch (Windows Itanium)

JUL2010

9655027

9683644

9683645

APR2010

9352224

9390286

N/A

JAN2010

9119275

9187106

N/A

OCT2009

8836758

8785185

8785186

JUL2009

8534403

8427417

8427418

APR2009

8290549

8300340

8300346

JAN2009

7592365

7703210

7703212

OCT2008

7375695

7394394

7394402

JUL2008

7154111

7047026

7047029

APR2008

6864082

6867138

6867139

JAN2008

6646842

6637265

6637266

OCT2007

6395038

6417013

6417014

JUL2007

6079582

6130293

6130295

APR2007

5901875

5916268

5916275

JAN2007

N/A

N/A

N/A

OCT2006

5490859

5652380

5639519

 

9.2.0.7

Description

CPU (Unix/Linux)

Bundle Patch (Windows 32bit)

Bundle Patch (Windows Itanium)

JUL2007

6079579

6146759

6146748

APR2007

5901872

5907274

5907275

JAN2007

5689875

5654905

5654909

OCT2006

5490841

5500873

5500874

JUL2006

5225794

5250980

5250981

APR2006

5049060

5064365

5064364

JAN2006

4751923

4751528

4741074

OCT2005

4567854

4579590

4579599

JUL2005

4547566

N/A

N/A

 

9.2.0.6

Description

CPU (Unix/Linux)

Bundle Patch (Windows 32bit)

Bundle Patch (Windows Itanium)

OCT2006

5490840

5500865

5500871

JUL2006

5225793

5239794

5239793

APR2006

5049051

5059614

5059615

JAN2006

4751921

4751261

4751262

OCT2005

4567846

4579093

4579097

JUL2005

4392392

4445852

4401917

APR2005

4193295

4269928

4213298

 

9.2.0.5

Description

CPU (Unix/Linux)

Bundle Patch (Windows 32bit)

Bundle Patch (Windows Itanium)

OCT2006

5689708

N/A

N/A

JUL2006

5435138

N/A

N/A

APR2006

5219762

N/A

N/A

OCT2005

4560421

N/A

N/A

JUL2005

4392256

4387563

4391819

APR2005

4193299

4195791

4214192

JAN2005

4003006

4104374

3990809

 

9.2.0.4

Description

CPU (Unix/Linux)

Bundle Patch (Windows 32bit)

Bundle Patch (Windows Itanium)

JAN2005

4002994

4104369

4083202

 

8.1.7.4

Description

CPU (Unix/Linux)

Bundle Patch (Windows 32bit)

JAN2007

5689799

5686514

OCT2006

5490835

5496067

JUL2006

5225788

5236412

APR2006

5045247

5057601

JAN2006

4751906

4751570

OCT2005

4560405

4554818

JUL2005

4392446

4437058

APR2005

4193312

4180163

JAN2005

4002909

3921893

参考:Assistant: Download Reference for Oracle Database/GI PSU, SPU(CPU), Bundle Patches, Patchsets and Base Releases (文档 ID 2118136.2)

12c ORA-01113 ORA-01110 恢复

数据库版本
VERSION


数据库启动报错

Completed: ALTER DATABASE   MOUNT
Thu Aug 17 12:34:52 2017
alter database open
Thu Aug 17 12:34:52 2017
Ping without log force is disabled
.
Thu Aug 17 12:34:52 2017
Errors in file D:\APP\ORACLE\diag\rdbms\XIFENFEI\XIFENFEI\trace\XIFENFEI_ora_5960.trc:
ORA-01113: 文件 5 需要介质恢复
ORA-01110: 数据文件 5: 'D:\APP\ORACLE\ORADATA\XIFENFEI\DATAFILE\O1_MF_UNDOTBS1_DN9MQJFK_.DBF'
ORA-1113 signalled during: alter database open

客户尝试恢复

SQL> startup mount;
ORACLE 例程已经启动。
Total System Global Area 1.0301E+10 bytes
Fixed Size                  3842760 bytes
Variable Size            1778388280 bytes
Database Buffers         8489271296 bytes
Redo Buffers               29708288 bytes
数据库装载完毕。
SQL> alter  database open;
alter  database open
*
第 1 行出现错误:
ORA-01113: 文件 5 需要介质恢复
ORA-01110: 数据文件 5: 'D:\APP\ORACLE\ORADATA\XIFENFEI\DATAFILE\O1_MF_UNDOTBS1_DN9MQJFK_.DBF'
SQL> alter database datafile 5 offline drop;
数据库已更改。
SQL> alter database open;
alter database open
*
第 1 行出现错误:
ORA-01113: 文件 6 需要介质恢复
ORA-01110: 数据文件 6: 'D:\APP\ORACLE\ORADATA\XIFENFEI\DATAFILE\O1_MF_USERS_DN9MQH75_.DBF'
SQL> recover datafile 6;
ORA-00283: 恢复会话因错误而取消
ORA-00322: 日志 3 (用于线程 1) 不是最新副本
ORA-00312: 联机日志 3 线程 1: 'D:\APP\ORACLE\FAST_RECOVERY_AREA\XIFENFEI\ONLINELOG\O1_MF_3_DN9MV6DW_.LOG'
ORA-00322: 日志 3 (用于线程 1) 不是最新副本
ORA-00312: 联机日志 3 线程 1: 'D:\APP\ORACLE\ORADATA\XIFENFEI\ONLINELOG\O1_MF_3_DN9MV6BX_.LOG'

使用Oracle Database Recovery Check检测结果
check_ctl
check_resulte


尝试恢复

SQL> recover database;
ORA-00283: recovery session canceled due to errors
ORA-00322: log 3 of thread 1 is not current copy
ORA-00312: online log 3 thread 1:
'D:\APP\ORACLE\FAST_RECOVERY_AREA\XIFENFEI\ONLINELOG\O1_MF_3_DN9MV6DW_.LOG'
ORA-00322: log 3 of thread 1 is not current copy
ORA-00312: online log 3 thread 1:
'D:\APP\ORACLE\ORADATA\XIFENFEI\ONLINELOG\O1_MF_3_DN9MV6BX_.LOG'
SQL> recover database until cancel
ORA-00279: 更改 9843709 (在 08/17/2017 07:04:02 生成) 对于线程 1 是必需的
ORA-00289: 建议:
D:\APP\ORACLE\FAST_RECOVERY_AREA\XIFENFEI\ARCHIVELOG\2017_08_17\O1_MF_1_717_%U_.ARC
ORA-00280: 更改 9843709 (用于线程 1) 在序列 #717 中
指定日志: {<RET>=suggested | filename | AUTO | CANCEL}
D:\APP\ORACLE\FAST_RECOVERY_AREA\XIFENFEI\ONLINELOG\O1_MF_2_DN9MV69G_.LOG
ORA-00310: archived log contains sequence 716; sequence 717 required
ORA-00334: archived log:
'D:\APP\ORACLE\FAST_RECOVERY_AREA\XIFENFEI\ONLINELOG\O1_MF_2_DN9MV69G_.LOG'
ORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error below
ORA-01194: file 1 needs more recovery to be consistent
ORA-01110: data file 1:
'D:\APP\ORACLE\ORADATA\XIFENFEI\DATAFILE\O1_MF_SYSTEM_DN9MN5OT_.DBF'
SQL> recover database until cancel
ORA-00279: 更改 9843709 (在 08/17/2017 07:04:02 生成) 对于线程 1 是必需的
ORA-00289: 建议:
D:\APP\ORACLE\FAST_RECOVERY_AREA\XIFENFEI\ARCHIVELOG\2017_08_17\O1_MF_1_717_%U_.AR
C
ORA-00280: 更改 9843709 (用于线程 1) 在序列 #717 中
指定日志: {<RET>=suggested | filename | AUTO | CANCEL}
D:\APP\ORACLE\FAST_RECOVERY_AREA\XIFENFEI\ONLINELOG\O1_MF_1_DN9MV661_.LOG
ORA-00310: archived log contains sequence 715; sequence 717 required
ORA-00334: archived log:
'D:\APP\ORACLE\FAST_RECOVERY_AREA\XIFENFEI\ONLINELOG\O1_MF_1_DN9MV661_.LOG'
ORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error below
ORA-01194: file 1 needs more recovery to be consistent
ORA-01110: data file 1:
'D:\APP\ORACLE\ORADATA\XIFENFEI\DATAFILE\O1_MF_SYSTEM_DN9MN5OT_.DBF'
SQL> recover database until cancel
ORA-00279: 更改 9843709 (在 08/17/2017 07:04:02 生成) 对于线程 1 是必需的
ORA-00289: 建议:
D:\APP\ORACLE\FAST_RECOVERY_AREA\XIFENFEI\ARCHIVELOG\2017_08_17\O1_MF_1_717_%U_.ARC
ORA-00280: 更改 9843709 (用于线程 1) 在序列 #717 中
指定日志: {<RET>=suggested | filename | AUTO | CANCEL}
D:\APP\ORACLE\FAST_RECOVERY_AREA\XIFENFEI\ONLINELOG\O1_MF_3_DN9MV6DW_.LOG
已应用的日志。
完成介质恢复。
SQL> alter database datafile 5 online;
数据库已更改。
SQL> recover database until cancel;
ORA-00279: 更改 9843709 (在 08/17/2017 07:04:02 生成) 对于线程 1 是必需的
ORA-00289: 建议:
D:\APP\ORACLE\FAST_RECOVERY_AREA\XIFENFEI\ARCHIVELOG\2017_08_17\O1_MF_1_717_%U_.AR
C
ORA-00280: 更改 9843709 (用于线程 1) 在序列 #717 中
指定日志: {<RET>=suggested | filename | AUTO | CANCEL}
D:\APP\ORACLE\FAST_RECOVERY_AREA\XIFENFEI\ONLINELOG\O1_MF_3_DN9MV6DW_.LOG
ORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error below
ORA-01152: file 1 was not restored from a sufficiently old backup
ORA-01110: data file 1:
'D:\APP\ORACLE\ORADATA\XIFENFEI\DATAFILE\O1_MF_SYSTEM_DN9MN5OT_.DBF'
ORA-01112: 未启动介质恢复
SQL> recover datafile 5;
ORA-00283: 恢复会话因错误而取消
ORA-00264: 不要求恢复
SQL> SELECT status,
  2  checkpoint_change#,
  3  checkpoint_time,FUZZY,
  4  count(*) ROW_NUM
  5  FROM v$datafile_header
  6  GROUP BY status, checkpoint_change#, checkpoint_time,fuzzy
  7  ORDER BY status, checkpoint_change#, checkpoint_time;
STATUS  CHECKPOINT_CHANGE# CHECKPOINT_TIME     FUZ          ROW_NUM
------- ------------------ ------------------- --- ----------------
ONLINE             9850826 2017-08-17 08:15:45 NO                 1
ONLINE             9857411 2017-08-17 08:15:45 NO                 7
ONLINE             9857411 2017-08-17 09:00:48 NO                 2
SQL> set numw 16
SQL> SELECT status,
  2  checkpoint_change#,
  3  checkpoint_time,last_change#,
  4  count(*) ROW_NUM
  5  FROM v$datafile
  6  GROUP BY status, checkpoint_change#, checkpoint_time,last_change#
  7  ORDER BY status, checkpoint_change#, checkpoint_time;
STATUS  CHECKPOINT_CHANGE# CHECKPOINT_TIME         LAST_CHANGE#          ROW_NUM
------- ------------------ ------------------- ---------------- ----------------
ONLINE             9857411 2017-08-17 09:00:48          9850826                1
ONLINE             9857411 2017-08-17 09:00:48          9857411                8
SYSTEM             9857411 2017-08-17 09:00:48          9857411                1
SQL> alter database open resetlogs;
alter database open resetlogs
*
第 1 行出现错误:
ORA-01152: 文件 1 没有从过旧的备份中还原
ORA-01110: 数据文件 1:
'D:\APP\ORACLE\ORADATA\XIFENFEI\DATAFILE\O1_MF_SYSTEM_DN9MN5OT_.DBF'
[/shell]
这里比较明显,由于controlfile的scn 大于db的scn,从而出现了ORA-01152的错误,重试重建控制文件
<br>
<strong>重建控制文件</strong>

SQL> alter database backup controlfile to trace as 'd:\app\ctl.txt';
数据库已更改。
SQL> startup nomount;
ORACLE 例程已经启动。
Total System Global Area      10301210624 bytes
Fixed Size                        3842760 bytes
Variable Size                  1778388280 bytes
Database Buffers               8489271296 bytes
Redo Buffers                     29708288 bytes
SQL> CREATE CONTROLFILE REUSE DATABASE "XIFENFEI" NORESETLOGS  NOARCHIVELOG
  2      MAXLOGFILES 16
  3      MAXLOGMEMBERS 3
  4      MAXDATAFILES 100
  5      MAXINSTANCES 8
  6      MAXLOGHISTORY 292
  7  LOGFILE
  8    GROUP 1 (
  9      'D:\APP\ORACLE\ORADATA\XIFENFEI\ONLINELOG\O1_MF_1_DN9MV652_.LOG',
 10      'D:\APP\ORACLE\FAST_RECOVERY_AREA\XIFENFEI\ONLINELOG\O1_MF_1_DN9MV661_.LO
G'
 11    ) SIZE 50M BLOCKSIZE 512,
 12    GROUP 2 (
 13      'D:\APP\ORACLE\ORADATA\XIFENFEI\ONLINELOG\O1_MF_2_DN9MV68H_.LOG',
 14      'D:\APP\ORACLE\FAST_RECOVERY_AREA\XIFENFEI\ONLINELOG\O1_MF_2_DN9MV69G_.LO
G'
 15    ) SIZE 50M BLOCKSIZE 512,
 16    GROUP 3 (
 17      'D:\APP\ORACLE\ORADATA\XIFENFEI\ONLINELOG\O1_MF_3_DN9MV6BX_.LOG',
 18      'D:\APP\ORACLE\FAST_RECOVERY_AREA\XIFENFEI\ONLINELOG\O1_MF_3_DN9MV6DW_.LO
G'
 19    ) SIZE 50M BLOCKSIZE 512
 20  DATAFILE
 21    'D:\APP\ORACLE\ORADATA\XIFENFEI\DATAFILE\O1_MF_SYSTEM_DN9MN5OT_.DBF',
 22    'D:\APP\ORACLE\ORADATA\XIFENFEI\DATAFILE\O1_MF_SYSAUX_DN9MK6B3_.DBF',
 23    'D:\APP\ORACLE\ORADATA\XIFENFEI\DATAFILE\O1_MF_UNDOTBS1_DN9MQJFK_.DBF',
 24    'D:\APP\ORACLE\ORADATA\XIFENFEI\DATAFILE\O1_MF_USERS_DN9MQH75_.DBF',
 25    'D:\APP\ORACLE\PRODUCT\12.1.0\DBHOME_1\DATABASE\XIFENFEIFILE06',
 26    'D:\APP\ORACLE\PRODUCT\12.1.0\DBHOME_1\DATABASE\XIFENFEIFILE05',
 27    'D:\APP\ORACLE\PRODUCT\12.1.0\DBHOME_1\DATABASE\XIFENFEIFILE04',
 28    'D:\APP\ORACLE\PRODUCT\12.1.0\DBHOME_1\DATABASE\XIFENFEIFILE03',
 29    'D:\APP\ORACLE\PRODUCT\12.1.0\DBHOME_1\DATABASE\XIFENFEIFILE02',
 30    'D:\APP\ORACLE\PRODUCT\12.1.0\DBHOME_1\DATABASE\XIFENFEIFILE01'
 31  CHARACTER SET ZHS16GBK
 32  ;
控制文件已创建。

尝试恢复

SQL> recover database;
完成介质恢复。
SQL> alter database open;
alter database open
*
第 1 行出现错误:
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-00600: internal error code, arguments: [4194], [41], [36], [], [], [], [],
[], [], [], [], []
进程 ID: 2864
会话 ID: 62 序列号: 54236

本来到这一步,错误比较明显,undo异常,这类直接对undo进行处理即可,可是运气不太好

异常crash之后redo损坏

SQL> startup pfile='d:/app/pfile.txt'
ORACLE 例程已经启动。
Total System Global Area 1.0301E+10 bytes
Fixed Size                  3842760 bytes
Variable Size            1778388280 bytes
Database Buffers         8489271296 bytes
Redo Buffers               29708288 bytes
数据库装载完毕。
ORA-00354: 损坏重做日志块标头
ORA-00353: 日志损坏接近块 60 更改 23924938639111 时间 08/17/2017 21:36:16
ORA-00312: 联机日志 1 线程 1:
'D:\APP\ORACLE\FAST_RECOVERY_AREA\XIFENFEI\ONLINELOG\O1_MF_1_DN9MV661_.LOG'
ORA-00312: 联机日志 1 线程 1:
'D:\APP\ORACLE\ORADATA\XIFENFEI\ONLINELOG\O1_MF_1_DN9MV652_.LOG'
SQL> recover database;
ORA-00283: recovery session canceled due to errors
ORA-00354: corrupt redo log block header
ORA-00353: log corruption near block 60 change 23924938639111 time 08/17/201721:36:16
ORA-00312: online log 1 thread 1:
'D:\APP\ORACLE\FAST_RECOVERY_AREA\XIFENFEI\ONLINELOG\O1_MF_1_DN9MV661_.LOG'
ORA-00312: online log 1 thread 1:
'D:\APP\ORACLE\ORADATA\XIFENFEI\ONLINELOG\O1_MF_1_DN9MV652_.LOG'
SQL> recover database until cancel;
ORA-00279: 更改 10050832 (在 08/17/2017 21:36:13 生成) 对于线程 1 是必需的
ORA-00289: 建议:
D:\APP\ORACLE\FAST_RECOVERY_AREA\XIFENFEI\ARCHIVELOG\2017_08_17\O1_MF_1_718_%U_.ARC
ORA-00280: 更改 10050832 (用于线程 1) 在序列 #718 中
指定日志: {<RET>=suggested | filename | AUTO | CANCEL}
D:\APP\ORACLE\FAST_RECOVERY_AREA\XIFENFEI\ONLINELOG\O1_MF_1_DN9MV661_.LOG
ORA-00283: recovery session canceled due to errors
ORA-00354: corrupt redo log block header
ORA-00353: log corruption near block 60 change 23924938639111 time 08/17/2017 21:36:16
ORA-00334: archived log:
'D:\APP\ORACLE\FAST_RECOVERY_AREA\XIFENFEI\ONLINELOG\O1_MF_1_DN9MV661_.LOG'

到这一步,只能通过屏蔽oracle 事务前滚,强制拉库恢复。

SQL> startup pfile='d:/app/pfile.txt' mount;
ORACLE 例程已经启动。
Total System Global Area 1.0301E+10 bytes
Fixed Size                  3842760 bytes
Variable Size            1778388280 bytes
Database Buffers         8489271296 bytes
Redo Buffers               29708288 bytes
数据库装载完毕。
SQL> recover database until cancel;
ORA-00279: 更改 10050832 (在 08/17/2017 21:36:13 生成) 对于线程 1 是必需的
ORA-00289: 建议: D:\APP\ORACLE\FAST_RECOVERY_AREA\XIFENFEI\ARCHIVELOG\2017_08_17\O
1_MF_1_718_%U_.ARC
ORA-00280: 更改 10050832 (用于线程 1) 在序列 #718 中
指定日志: {<RET>=suggested | filename | AUTO | CANCEL}
cancel
ORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error below
ORA-01194: file 1 needs more recovery to be consistent
ORA-01110: data file 1: 'D:\APP\ORACLE\ORADATA\XIFENFEI\DATAFILE\O1_MF_SYSTEM_DN9M
N5OT_.DBF'
ORA-01112: 未启动介质恢复
SQL> alter database open resetlogs;
数据库已更改。

mount: wrong fs type, bad option, bad superblock恢复Oracle

有朋友找到我们,说对lv进行收缩操作之后,导致文件系统无法mount,提示超级块损坏.
尝试mount失败

[root@GZGSDB data]# mount /dev/vg_gzgsdb/lv_home /home
mount: wrong fs type, bad option, bad superblock on /dev/mapper/vg_gzgsdb-lv_home,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so

系统日志报错

Aug 16 21:23:24 GZGSDB kernel: EXT4-fs (dm-5): ext4_check_descriptors: Block bitmap for group 1 not in group (block 0)!
Aug 16 21:23:24 GZGSDB kernel: EXT4-fs (dm-5): group descriptors corrupted!

主要操作
cz


这里可以看出来,明显操作错误,在没有收缩文件系统之前,直接收缩的lv.
list_history


这里可以看出来,在发生故障之后,又做了很多操作,包括fsck和testdisk等,最终无法恢复,请求我们协助处理。

找出来备份超级块

[root@GZGSDB dul]# dumpe2fs /dev/vg_gzgsdb/lv_home |grep superblock
dumpe2fs 1.41.12 (17-May-2010)
ext2fs_read_bb_inode: A block group is missing an inode table
  Primary superblock at 0, Group descriptors at 1-52
  Backup superblock at 32768, Group descriptors at 32769-32820
  Backup superblock at 98304, Group descriptors at 98305-98356
  Backup superblock at 163840, Group descriptors at 163841-163892
  Backup superblock at 229376, Group descriptors at 229377-229428
  Backup superblock at 294912, Group descriptors at 294913-294964
  Backup superblock at 819200, Group descriptors at 819201-819252
  Backup superblock at 884736, Group descriptors at 884737-884788
  Backup superblock at 1605632, Group descriptors at 1605633-1605684
  Backup superblock at 2654208, Group descriptors at 2654209-2654260
  Backup superblock at 4096000, Group descriptors at 4096001-4096052
  Backup superblock at 7962624, Group descriptors at 7962625-7962676
  Backup superblock at 11239424, Group descriptors at 11239425-11239476
  Backup superblock at 20480000, Group descriptors at 20480001-20480052
  Backup superblock at 23887872, Group descriptors at 23887873-23887924
  Backup superblock at 71663616, Group descriptors at 71663617-71663668
  Backup superblock at 78675968, Group descriptors at 78675969-78676020
  Backup superblock at 102400000, Group descriptors at 102400001-102400052
  Backup superblock at 214990848, Group descriptors at 214990849-214990900

使用fsck修复

[root@GZGSDB data]# fsck -y /dev/vg_gzgsdb/lv_home
fsck from util-linux-ng 2.17.2
e2fsck 1.41.12 (17-May-2010)
fsck.ext4: Group descriptors look bad... trying backup blocks...
fsck.ext4: The ext2 superblock is corrupt when using the backup blocks
fsck.ext4: going back to original superblock
fsck.ext4: Device or resource busy while trying to open /dev/mapper/vg_gzgsdb-lv_home
Filesystem mounted or opened exclusively by another program?
--指定超级块恢复
[root@GZGSDB data]# fsck -y -b 102400000 /dev/vg_gzgsdb/lv_home
…………
Illegal block #0 (1296647770) in inode 354315.  CLEARED.
Illegal block #1 (1398362886) in inode 354315.  CLEARED.
Illegal block #3 (453538936) in inode 354315.  CLEARED.
Illegal block #5 (808333361) in inode 354315.  CLEARED.
Illegal block #6 (775434798) in inode 354315.  CLEARED.
Illegal block #8 (1180306180) in inode 354315.  CLEARED.
Illegal block #9 (1413893971) in inode 354315.  CLEARED.
Illegal block #10 (1229347423) in inode 354315.  CLEARED.
Illegal block #11 (1498613325) in inode 354315.  CLEARED.
Illegal indirect block (1296389203) in inode 354315.  CLEARED.
Inode 354315 is too big.  Truncate? yes
Block #1074301965 (69632) causes directory to be too big.  CLEARED.
Warning... fsck.ext4 for device /dev/mapper/vg_gzgsdb-lv_home exited with signal 11.
[root@GZGSDB data]# mount /dev/vg_gzgsdb/lv_home /home
mount: wrong fs type, bad option, bad superblock on /dev/mapper/vg_gzgsdb-lv_home,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so

至此基本上可以判断,直接修复该文件系统,让其正常mount的概率很小.直接通过lun层面来恢复.

通过工具解析lun
disk_size


比较明显,客户经过一系列操作现在的现象是lv 1.03T,文件系统只有821G,明显和客户给我反馈的操作不符(应该是lv 821G),证明客户做了大量操作,已经导致文件系统损坏
inode_recovery


由于文件系统严重异常,工具获取到的文件都是没有名称,直接从inode里面读取的数据.获取到这些数据之后,然后结合oracle的特性,判断出来对应的文件号关系(这里有大量文件重复);另外有个别文件通过inode恢复丢失,通过底层碎片重组进行恢复出来文件,然后恢复出来其中数据(asm disk header 彻底损坏恢复).这个客户比较幸运,system文件在另外一个分区中,不然工作量会大很多.
再次提醒:对lv操作一定要谨慎,特别是lvreduce操作,另外出现发生误操作之后,应该第一时间保护现场,而不是百度着去乱操作,可能导致故障更加悲剧

Quick Reference to Patch Numbers for Database/GI PSU, SPU(CPU), Bundle Patches and Patchsets—201707

 

12.1.0.2 (12.1.0.2.0 PATCH SET FOR ORACLE DATABASE SERVER)

21419221

11.2.0.4 (11.2.0.4.0 PATCH SET FOR ORACLE DATABASE SERVER)

13390677

11.2.0.3 (11.2.0.3.0 PATCH SET FOR ORACLE DATABASE SERVER)

10404530

11.2.0.2 (11.2.0.2.0 PATCH SET FOR ORACLE DATABASE SERVER)

10098816

11.1.0.7 (11.1.0.7.0 PATCH SET FOR ORACLE DATABASE SERVER)

6890831

A10.2.0.4 (10.2.0.4.0 PATCH SET FOR ORACLE DATABASE SERVER)

6810189

B10.2.0.3 (10.2.0.3 PATCH SET FOR ORACLE DATABASE SERVER)

5337014

10.2.0.2 (10.2.0.2 PATCH SET FOR ORACLE DATABASE SERVER)

4547817

10.1.0.5 (10.1.0.5 PATCH SET FOR ORACLE DATABASE SERVER)

4505133

10.1.0.4 (10.1.0.4 PATCH SET FOR ORACLE DATABASE SERVER)

4163362

10.1.0.3 (10.1.0.3 PATCH SET FOR ORACLE DATABASE SERVER)

3761843

9.2.0.8 (9.2.0.8 PATCH SET FOR ORACLE DATABASE SERVER)

4547809

9.2.0.7 (9.2.0.7 PATCH SET FOR ORACLE DATABASE SERVER)

4163445

9.2.0.6 (9.2.0.6 PATCH SET FOR ORACLE DATABASE SERVER)

3948480

9.2.0.5 (ORACLE 9I DATABASE SERVER RELEASE 2 – PATCH SET 4 VERSION 9.2.0.5.0)

3501955

9.2.0.4 (9.2.0.4 PATCH SET FOR ORACLE DATABASE SERVER)

3095277

9.2.0.3 (9.2.0.3 PATCH SET FOR ORACLE DATABASE SERVER)

2761332

9.2.0.2 (9.2.0.2 PATCH SET FOR ORACLE DATABASE SERVER)

2632931

9.0.1.5 (9.0.1.5 PATCHSET)

3301544

9.0.1.4 (9.0.1.4 PATCH SET FOR ORACLE DATABASE SERVER)

2517300

9.0.1.3 (9.0.1.3. PATCH SET FOR ORACLE DATA SERVER)

2271678

8.1.7.4 (8.1.7.4 PATCH SET FOR ORACLE DATA SERVER)

2376472

8.1.7.3 (8.1.7.3 PATCH SET FOR ORACLE DATA SERVER)

2189751

8.1.7.2 (8.1.7.2.1 PATCH SET FOR ORACLE DATA SERVER)

1909158

 

12.2.0.1

Description

Database RU

GI RU

Windows Bundle Patch

JUL2017 (12.2.0.1.170718)

26123830

26133434

26204212

 
 

12.1.0.2

Description

PSU

GI PSU

Proactive Bundle Patch

Bundle Patch (Windows 32bit & 64bit)

JUL2017 (12.1.0.2.170718)

25755742

25901062

26022196

26161724

APR2017 (12.1.0.2.170418)

25171037

25434003

25433352

25632533

JAN2017 (12.1.0.2.170117)

24732082

24917825

24968615

25115951

OCT2016 (12.1.0.2.161018)

24006101

24412235

24448103

24591642

JUL2016 (12.1.0.2.160719)

23054246

23273629

23273686

23530387

APR2016 (12.1.0.2.160419)

22291127

22646084

22899531

22809813

JAN2016 (12.1.0.2.160119)

21948354

22191349

22243551

22310559

OCT2015

21359755 (12.1.0.2.5)

21523234 (12.1.0.2.5)

21744410 (12.1.0.2.13)

21821214 (12.1.0.2.10)

JUL2015

20831110 (12.1.0.2.4)

20996835 (12.1.0.2.4)

21188742 (12.1.0.2.10)

21126814 (12.1.0.2.7)

APR2015

20299023 (12.1.0.2.3)

20485724 (12.1.0.2.3)

20698050 (12.1.0.2.7)

20684004 (12.1.0.2.4)

JAN2015

19769480 (12.1.0.2.2)

19954978 (12.1.0.2.2)

20141343 (12.1.0.2.4)

19720843 (12.1.0.2.1)

OCT2014

19303936 (12.1.0.2.1)

19392646 (12.1.0.2.1)

19404326 (12.1.0.2.1)

N/A

 

12.1.0.1

Description

PSU

GI PSU

Bundle Patch

A

B

Windows 64 bit

Windows 32 bit

JUL2016 (12.1.0.1.160719)

23054354

23273935

23273958

23530410

APR2016 (12.1.0.1.160419)

22291141

22654153

22654166

22839614

JAN2016 (12.1.0.1.160119)

21951844

22191492

22191511

22494866

OCT2015

21352619(12.1.0.1.9)

21551666(12.1.0.1.9)

21551685(12.1.0.1.9)

21744907 (12.1.0.1.21)

JUL2015

20831107(12.1.0.1.8)

20996901(12.1.0.1.8)

20996911(12.1.0.1.8)

21076681 (12.1.0.1.20)

APR2015

20299016(12.1.0.1.7)

20485762(12.1.0.1.7)

19971331(12.1.0.1.7)

20558101 (12.1.0.1.18)

JAN2015

19769486(12.1.0.1.6)

19971324(12.1.0.1.6)

19971331(12.1.0.1.6)

20160748 (12.1.0.1.16)

OCT2014

19121550(12.1.0.1.5)

19392372(12.1.0.1.5)

19392451(12.1.0.1.5)

19542943 (12.1.0.1.14)

JUL2014

18522516(12.1.0.1.4)

18705901(12.1.0.1.4)

18705972(12.1.0.1.4)

19062327 (12.1.0.1.11)

APR2014

18031528(12.1.0.1.3)

18139660(12.1.0.1.3)

18413105(12.1.0.1.3)

18448604 (12.1.0.1.7)

JAN2014

17552800(12.1.0.1.2)

17735306 (12.1.0.1.2)

17977915 (12.1.0.1.3)

OCT2013

17027533(12.1.0.1.1)

17272829 (12.1.0.1.1)

17363796(12.1.0.1.1)

17363795(12.1.0.1.1)

 

11.2.0.4

Description

PSU

SPU(CPU)

GI PSU

Bundle Patch (Windows 32bit & 64bit)

JUL2017 (11.2.0.4.170718)

25869727

25879656

26030799

26194136

APR2017 (11.2.0.4.170418)

24732075

25369547

25476126

25632525

JAN2017

N/A

N/A

N/A

N/A

OCT2016 (11.2.0.4.161018)

24006111

24433711

24436338

24591646

JUL2016 (11.2.0.4.160719)

23054359

23177648

23274134

23530402

APR2016 (11.2.0.4.160419)

22502456

22502493

22646198

22839608

JAN2016 (11.2.0.4.160119)

21948347

21972320

22191577

22310544

OCT2015

21352635 (11.2.0.4.8)

21352646

21523375 (11.2.0.4.8)

21821802 (11.2.0.4.20)

JUL2015

20760982 (11.2.0.4.7)

20803583

20996923 (11.2.0.4.7)

21469106 (11.2.0.4.18)

APR2015

20299013 (11.2.0.4.6)

20299015

20485808 (11.2.0.4.6)

20544696 (11.2.0.4.15)

JAN2015

19769489 (11.2.0.4.5)

19854503

19955028 (11.2.0.4.5)

20127071 (11.2.0.4.12)

OCT2014

19121551 (11.2.0.4.4)

19271443

19380115 (11.2.0.4.4)

19651773 (11.2.0.4.10)

JUL2014

18522509 (11.2.0.4.3)

18681862

18706472 (11.2.0.4.3)

18842982 (11.2.0.4.7)

APR2014

18031668 (11.2.0.4.2)

18139690

18139609 (11.2.0.4.2)

18296644 (11.2.0.4.4)

JAN2014

17478514 (11.2.0.4.1)

17551709

N/A

17987366 (11.2.0.4.1)

Assistant: Download Reference for Oracle Database/GI PSU, SPU(CPU), Bundle Patches, Patchsets and Base Releases (文档 ID 2118136.2)

ORA-12012: error on auto execute of job "SYS"."ORA$AT_OS_OPT_SY_383"

12.2.0.1版本数据库报类似ORA-12012: error on auto execute of job “SYS”.”ORA$AT_OS_OPT_SY_383″错误

Errors in file /u01/app/oracle/diag/rdbms/xifenfei/xifenfei/trace/xifenfei_j000_10213.trc:
ORA-12012: error on auto execute of job "SYS"."ORA$AT_OS_OPT_SY_383"
ORA-20001: Statistics Advisor: Invalid task name for the current user
ORA-06512: at "SYS.DBMS_STATS", line 47207
ORA-06512: at "SYS.DBMS_STATS_ADVISOR", line 882
ORA-06512: at "SYS.DBMS_STATS_INTERNAL", line 20059
ORA-06512: at "SYS.DBMS_STATS_INTERNAL", line 22201
ORA-06512: at "SYS.DBMS_STATS", line 47197

trace文件信息

Trace file /u01/app/oracle/diag/rdbms/xifenfei/xifenfei/trace/xifenfei_j000_10379.trc
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production
Build label:    RDBMS_12.2.0.1.0_LINUX.X64_170125
ORACLE_HOME:    /u01/app/oracle/product/12.2.0/db_1
System name:    Linux
Node name:      yoridb2
Release:        3.10.0-514.26.2.el7.x86_64
Version:        #1 SMP Tue Jul 4 15:04:05 UTC 2017
Machine:        x86_64
Instance name: xifenfei
Redo thread mounted by this instance: 1
Oracle process number: 148
Unix process pid: 10379, image: oracle@yoridb2 (J000)
*** 2017-08-04T14:41:26.486692+08:00
*** SESSION ID:(508.52539) 2017-08-04T14:41:26.486716+08:00
*** CLIENT ID:() 2017-08-04T14:41:26.486722+08:00
*** SERVICE NAME:(SYS$USERS) 2017-08-04T14:41:26.486727+08:00
*** MODULE NAME:(DBMS_SCHEDULER) 2017-08-04T14:41:26.486732+08:00
*** ACTION NAME:(ORA$AT_OS_OPT_SY_384) 2017-08-04T14:41:26.486738+08:00
*** CLIENT DRIVER:() 2017-08-04T14:41:26.486743+08:00
ORA-12012: error on auto execute of job "SYS"."ORA$AT_OS_OPT_SY_384"
ORA-20001: Statistics Advisor: Invalid task name for the current user
ORA-06512: at "SYS.DBMS_STATS", line 47207
ORA-06512: at "SYS.DBMS_STATS_ADVISOR", line 882
ORA-06512: at "SYS.DBMS_STATS_INTERNAL", line 20059
ORA-06512: at "SYS.DBMS_STATS_INTERNAL", line 22201
ORA-06512: at "SYS.DBMS_STATS", line 47197

错误比较明显,是在执行统计信息分析的时候报错了,报错原因是由于SCHEDULER的task name无效.

解决方法

[oracle@yoridb2 trace]$ ss
SQL*Plus: Release 12.2.0.1.0 Production on Fri Aug 4 23:40:33 2017
Copyright (c) 1982, 2016, Oracle.  All rights reserved.
Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production
SQL> select name, ctime, how_created
  2    from sys.wri$_adv_tasks
  3   where owner_name = 'SYS'
   and name in ('AUTO_STATS_ADVISOR_TASK','INDIVIDUAL_STATS_ADVISOR_TASK');  4
no rows selected
SQL> EXEC dbms_stats.init_package();
PL/SQL procedure successfully completed.
SQL> select name, ctime, how_created
  2    from sys.wri$_adv_tasks
  3   where owner_name = 'SYS'
   and name in ('AUTO_STATS_ADVISOR_TASK','INDIVIDUAL_STATS_ADVISOR_TASK');  4
NAME
--------------------------------------------------------------------------------
CTIME     HOW_CREATED
--------- ------------------------------
AUTO_STATS_ADVISOR_TASK
04-AUG-17 CMD
INDIVIDUAL_STATS_ADVISOR_TASK
04-AUG-17 CMD

Disable Transparent HugePages

从redhat 6开始引入了Transparent HugePages,但是oracle一直建议disable 它,而使用标准HugePages方式.对于6和7的禁用方式有一些区别.
linux 6
修改/etc/grub.conf之后重启系统生效

vi /etc/grub.conf
title Oracle Linux Server (2.6.32-300.25.1.el6uek.x86_64)
        root (hd0,0)
        kernel /vmlinuz-2.6.32-300.25.1.el6uek.x86_64 ro root=LABEL=/  transparent_hugepage=never
        initrd /initramfs-2.6.32-300.25.1.el6uek.x86_64.img

linux 7
修改/etc/default/grub然后执行grub2-mkconfig并重启系统生效

[root@xifenfei u01]# vi /etc/default/grub
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="nomodeset vconsole.font=latarcyrheb-sun16 vconsole.keymap=us crashkernel=auto
                    biosdevname=0 transparent_hugepage=never"
GRUB_DISABLE_RECOVERY="true"
~
[root@xifenfei u01]# grub2-mkconfig -o /boot/grub2/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-3.10.0-514.26.2.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-514.26.2.el7.x86_64.img
Found linux image: /boot/vmlinuz-3.10.0-123.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-123.el7.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-cb0b6b4de89a4fe4acfc8774c2f01486
Found initrd image: /boot/initramfs-0-rescue-cb0b6b4de89a4fe4acfc8774c2f01486.img
done

临时禁用
该方法对于linux 6和7均有效,不用重启系统

if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
   echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
   echo never > /sys/kernel/mm/transparent_hugepage/defrag
fi

警告:互联网中有oracle介质被注入恶意程序导致—ORA-600 16703

继续上篇的tab$被清空(ORA-600 16703故障解析—tab$表被清空),导致数据库启动异常的case
ORA-600 16703报错
ora-600-16703


数据库日志分析
数据库open成功同时报ORA-7445 kokeicbegin和ORA-600 kzrini:!uprofile错误
ora-600-kzrini-uprofile


再次启动数据库直接报ORA-600 16703错误
ora-600-16703


这里有一个其他现象,就是数据库在open成功的同时(同一秒内),开始报异常.重启之后直接报
ORA-00704: bootstrap process failure
ORA-00704: bootstrap process failure
ORA-00600: internal error code, arguments: [16703], [1403], [20], [], [], [], [], [], [], [], [], []
根据10046分析结果

=====================
select rowcnt,blkcnt,empcnt,avgspc,chncnt,avgrln,nvl(degree,1), nvl(instances,1) from tab$ where obj# = :1
END OF STMT
PARSE #140048443935120:c=0,e=390,p=0,cr=0,cu=0,mis=1,r=0,dep=1,og=4,plh=0,tim=1499185905161433
=====================
select blevel, leafcnt, distkey, lblkkey, dblkkey, clufac,        nvl(degree,1), nvl(instances,1) from ind$ where bo# = :1 and obj# = :2
END OF STMT
PARSE #140048443934176:c=1000,e=601,p=0,cr=0,cu=0,mis=1,r=0,dep=1,og=4,plh=0,tim=1499185905162088
=====================
PARSING IN CURSOR #140048443933232 len=70 dep=1 uid=0 oct=3 lid=0 tim=1499185905162444 hv=3377894161 ad='84f13d70' sqlid='32d4jrb4pd4sj'
select charsetid, charsetform from col$  where obj# = :1 and col# = :2
END OF STMT
PARSE #140048443933232:c=0,e=294,p=0,cr=0,cu=0,mis=1,r=0,dep=1,og=4,plh=0,tim=1499185905162443
=====================
PARSING IN CURSOR #140048443932288 len=52 dep=1 uid=0 oct=3 lid=0 tim=1499185905247020 hv=429618617 ad='84f0f2d0' sqlid='4krwuz0ctqxdt'
select ctime, mtime, stime from obj$ where obj# = :1
END OF STMT
PARSE #140048443932288:c=0,e=549,p=0,cr=0,cu=0,mis=1,r=0,dep=1,og=4,plh=0,tim=1499185905247019
BINDS #140048443932288:
select blevel, leafcnt, distkey, lblkkey, dblkkey, clufac,        nvl(degree,1), nvl(instances,1) from ind$ where bo# = :1 and obj# = :2
END OF STMT
PARSE #140048443934176:c=1000,e=601,p=0,cr=0,cu=0,mis=1,r=0,dep=1,og=4,plh=0,tim=1499185905162088
=====================
PARSING IN CURSOR #140048443933232 len=70 dep=1 uid=0 oct=3 lid=0 tim=1499185905162444 hv=3377894161 ad='84f13d70' sqlid='32d4jrb4pd4sj'
select charsetid, charsetform from col$  where obj# = :1 and col# = :2
END OF STMT
PARSE #140048443933232:c=0,e=294,p=0,cr=0,cu=0,mis=1,r=0,dep=1,og=4,plh=0,tim=1499185905162443
=====================
PARSING IN CURSOR #140048443932288 len=52 dep=1 uid=0 oct=3 lid=0 tim=1499185905247020 hv=429618617 ad='84f0f2d0' sqlid='4krwuz0ctqxdt'
select ctime, mtime, stime from obj$ where obj# = :1
END OF STMT
PARSE #140048443932288:c=0,e=549,p=0,cr=0,cu=0,mis=1,r=0,dep=1,og=4,plh=0,tim=1499185905247019
BINDS #140048443932288:
 Bind#0
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=00 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=7f5f91b87bd0  bln=22  avl=02  flg=05
  value=20
EXEC #140048443932288:c=2000,e=2686,p=0,cr=0,cu=0,mis=1,r=0,dep=1,og=4,plh=1218588913,tim=1499185905249810
WAIT #140048443932288: nam='db file sequential read' ela= 6205 file#=1 block#=337 blocks=1 obj#=36 tim=1499185905256132
WAIT #140048443932288: nam='db file sequential read' ela= 3739 file#=1 block#=338 blocks=1 obj#=36 tim=1499185905266704
WAIT #140048443932288: nam='db file sequential read' ela= 4966 file#=1 block#=241 blocks=1 obj#=18 tim=1499185905271761
FETCH #140048443932288:c=0,e=21976,p=3,cr=3,cu=0,mis=0,r=1,dep=1,og=4,plh=1218588913,tim=1499185905271820
STAT #140048443932288 id=1 cnt=1 pid=0 pos=1 obj=18 op='TABLE ACCESS BY INDEX ROWID OBJ$ (cr=3 pr=3 pw=0 time=21993 us)'
STAT #140048443932288 id=2 cnt=1 pid=1 pos=1 obj=36 op='INDEX RANGE SCAN I_OBJ1 (cr=2 pr=2 pw=0 time=16923 us)'
CLOSE #140048443932288:c=0,e=63,dep=1,type=0,tim=1499185905271941
BINDS #140048443935120:
 Bind#0
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=7f5f91c07de8  bln=22  avl=02  flg=05
  value=20
EXEC #140048443935120:c=1000,e=795,p=0,cr=0,cu=0,mis=1,r=0,dep=1,og=4,plh=2970138452,tim=1499185905272802
WAIT #140048443935120: nam='db file sequential read' ela= 3197 file#=1 block#=169 blocks=1 obj#=3 tim=1499185905276069
WAIT #140048443935120: nam='db file sequential read' ela= 3459 file#=1 block#=170 blocks=1 obj#=3 tim=1499185905404084
WAIT #140048443935120: nam='db file sequential read' ela= 6358 file#=1 block#=145 blocks=1 obj#=4 tim=1499185905410548
FETCH #140048443935120:c=999,e=137805,p=3,cr=3,cu=0,mis=0,r=0,dep=1,og=4,plh=2970138452,tim=1499185905410635
STAT #140048443935120 id=1 cnt=0 pid=0 pos=1 obj=4 op='TABLE ACCESS CLUSTER TAB$ (cr=3 pr=3 pw=0 time=137810 us)'
STAT #140048443935120 id=2 cnt=1 pid=1 pos=1 obj=3 op='INDEX UNIQUE SCAN I_OBJ# (cr=2 pr=2 pw=0 time=131330 us)'
*** 2017-07-05 00:31:46.094
Incident 176395 created, dump file: /oracle/diag/rdbms/orcl/orcl2/incident/incdir_176395/orcl_ora_51261_i176395.trc
ORA-00600: internal error code, arguments: [16703], [1403], [20], [], [], [], [], [], [], [], [], []

以及以往恢复经验和mos,基本上可以确定是由于tab$和obj$记录不匹配导致该问题.而且是obj#=20的记录在tab$和obj$中不匹配.

分析tab$和obj$记录

Data UnLoader: 11.2.0.1.5 - Internal Only - on Wed Jul 05 01:28:53 2017
with 64-bit io functions and the decompression option
 
Copyright (c) 1994 2017 Bernard van Duijnen All rights reserved.
 
 Strictly Oracle Internal Use Only
 
 
Found db_id = 1334610369
Found db_name = orcl
DUL> unload table TAB$( OBJ# number, DATAOBJ# number,
  2      TS# number, FILE# number, BLOCK# number,
  3      BOBJ# number, TAB# number, COLS number, CLUCOLS number,
  4      PCTFREE$ ignore, PCTUSED$ ignore, INITRANS ignore, MAXTRANS ignore,
  5      FLAGS ignore, AUDIT$ ignore, ROWCNT ignore, BLKCNT ignore,
  6      EMPCNT ignore, AVGSPC ignore, CHNCNT ignore, AVGRLN ignore,
  7      AVGSPC_FLB ignore, FLBCNT ignore,
  8      ANALYZETIME ignore, SAMPLESIZE ignore,
  9      DEGREE ignore, INSTANCES ignore,
 10      INTCOLS ignore, KERNELCOLS number, PROPERTY number)
 11      cluster  C_OBJ#(OBJ#)
 12      storage ( tablespace 0 segobjno 2 tabno 1 file 1 block 144);
. unloading table                      TAB$       0 rows unloaded
DUL> unload table OBJ$( OBJ# number, DATAOBJ# number, OWNER# number,
  2      NAME clean varchar2(30), NAMESPACE ignore, SUBNAME clean varchar2(30),
  3      TYPE# number, CTIME ignore, MTIME ignore, STIME ignore,
  4      STATUS ignore, REMOTEOWNER ignore, LINKNAME ignore,
  5      FLAGS ignore, OID$ hexraw)
  6      storage ( tablespace 0 segobjno 18 file 1 block 240);
. unloading table                      OBJ$   89804 rows unloaded
DUL>

这里可以明确表示tab$无记录,obj$有记录,从而启动的过程出现ORA-600 16703错误可以很好解释.由于数据库启动成功和报错几乎同时进行,以及上面看到的tab$记录不存在的情况,初步怀疑是有startup触发器清空tab$表记录
工具分析触发器表trigger$
startup-trigger


这里果然看到一个after startup on database的触发器,名字为DBMS_SUPPORT_DBMONITOR,而它调用的是DBMS_SUPPORT_DBMONITORP存储.

工具分析pl/sql表source$
DBMS_SUPPORT_DBMONITOR


对wraped加密的程序进行解密
DBMS_SUPPORT_DBMONITOR-unwraped


这里可以明确的看到DBMS_SUPPORT_DBMONITORP存储过程备份tab$表到orachk中然后delete tab$表,现在已经明确是由于DBMS_SUPPORT_DBMONITOR触发器在数据库重启之后开始执行调用DBMS_SUPPORT_DBMONITORP程序,如果判断数据库创建时间大于等于300天,便干掉tab$表,实现数据库破坏.

查找DBMS_SUPPORT_DBMONITOR等程序源头
分析相关程序创建时间,通过obj$表的ctime和name来判断
DBMS_SUPPORT_DBMONITOR-ctime
bootstrap-ctime


这里可以看出来DBMS_SUPPORT_DBMONITOR和DBMS_SUPPORT_DBMONITORP的创建时间基本上和数据库核心对象的创建时间相差无几,我们可以大概排除掉,是由于pl sql等工具连接数据库导致该发问题(类似:plsql dev引起的数据库被黑勒索比特币实现原理分析和解决方案),很可能是在dbca创建库的过程中就已经带有了DBMS_SUPPORT_DBMONITOR等程序,如果这样那很可能是由于数据库的安装介质被破坏导致该问题.

分析DBMS_SUPPORT_DBMONITOR来源
prvtsupp
20170711001626


这里已经很清晰,由于prvtsupp.plb被人注入了恶意脚本从而使得数据库被创建了DBMS_SUPPORT_DBMONITOR的触发器和DBMS_SUPPORT_DBMONITORP的存储过程,从而实现了对数据库的而易破坏.

确定破坏文件prvtsupp.plb来源于介质
jar


这里比较明显,文件就是来源database\stage\Components\oracle.rdbms.dbscripts\11.2.0.4.0\1\DataFiles\filegroup2.jar\rdbms\admin\prvtsupp.plb文件被修改导致
md5


通过md5码对比,可以确定是有人对软件的安装介质进行了破坏,从而实现了恶意代码的注入,从而实现了数据库300天之后重启之后无法正常启动而是出现类似ORA-00600: internal error code, arguments: [16703], [1403], [20], [], [], [], [], [], [], [], [], []的错误.

温馨提示
各位一定要从官方途径下载oracle安装介质,如果是从其他互联网途径下载一定要验证md5,确保文件没有被人恶意篡改,造成无可挽回的损坏.如果真的不幸遇到这类问题,请保护现场联系我们
Tel:17813235971    Q Q:107644445QQ咨询惜分飞    E-Mail:dba@xifenfei.com
我们可以通过使用bbed对tab$表数据数据进行恢复实现数据库正常启动,实现数据0丢失,最大限度抢救您的数据和减少业务恢复时间

ORA-600 16703故障解析—tab$表被清空

最近连续遇到两次数据库启动报ORA-600 16703错误,而导致数据库无法正常启动的问题
ORA-600 16703报错
ora-600-16703


Completed: ALTER DATABASE RECOVER  database
alter database open
Beginning crash recovery of 1 threads
 parallel recovery started with 32 processes
Started redo scan
Completed redo scan
 read 0 KB redo, 0 data blocks need recovery
Started redo application at
 Thread 1: logseq 993752, block 2, scn 14872376881763
Recovery of Online Redo Log: Thread 1 Group 6 Seq 993752 Reading mem 0
  Mem# 0: /oracle/oradata/xifenfei/redo06.log
Completed redo application of 0.00MB
Completed crash recovery at
 Thread 1: logseq 993752, block 3, scn 14872376901765
 0 data blocks read, 0 data blocks written, 0 redo k-bytes read
Tue Jul 04 23:13:29 2017
Thread 1 advanced to log sequence 993753 (thread open)
Thread 1 opened at log sequence 993753
  Current log# 7 seq# 993753 mem# 0: /oracle/oradata/xifenfei/redo07.log
Successful open of redo thread 1
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Tue Jul 04 23:13:29 2017
SMON: enabling cache recovery
Errors in file /oracle/diag/rdbms/xifenfei/xifenfei/trace/xifenfei_ora_15886.trc  (incident=163595):
ORA-00600: internal error code, arguments: [16703], [1403], [20], [], [], [], [], [], [], [], [], []
Incident details in: /oracle/diag/rdbms/xifenfei/xifenfei/incident/incdir_163595/xifenfei_ora_15886_i163595.trc
Tue Jul 04 23:13:30 2017
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Errors in file /oracle/diag/rdbms/xifenfei/xifenfei/trace/xifenfei_ora_15886.trc:
ORA-00704: bootstrap process failure
ORA-00704: bootstrap process failure
ORA-00600: internal error code, arguments: [16703], [1403], [20], [], [], [], [], [], [], [], [], []
Errors in file /oracle/diag/rdbms/xifenfei/xifenfei/trace/xifenfei_ora_15886.trc:
ORA-00704: bootstrap process failure
ORA-00704: bootstrap process failure
ORA-00600: internal error code, arguments: [16703], [1403], [20], [], [], [], [], [], [], [], [], []
Error 704 happened during db open, shutting down database
USER (ospid: 15886): terminating the instance due to error 704
Instance terminated by USER, pid = 15886
ORA-1092 signalled during: alter database open...
opiodr aborting process unknown ospid (15886) as a result of ORA-1092

这里报错比较明显ORA-600 16703,而且是在启动时bootstrap$中的对象出现该问题.
10046分析启动过程

=====================
select rowcnt,blkcnt,empcnt,avgspc,chncnt,avgrln,nvl(degree,1), nvl(instances,1) from tab$ where obj# = :1
END OF STMT
PARSE #140048443935120:c=0,e=390,p=0,cr=0,cu=0,mis=1,r=0,dep=1,og=4,plh=0,tim=1499185905161433
=====================
select blevel, leafcnt, distkey, lblkkey, dblkkey, clufac,        nvl(degree,1), nvl(instances,1) from ind$ where bo# = :1 and obj# = :2
END OF STMT
PARSE #140048443934176:c=1000,e=601,p=0,cr=0,cu=0,mis=1,r=0,dep=1,og=4,plh=0,tim=1499185905162088
=====================
PARSING IN CURSOR #140048443933232 len=70 dep=1 uid=0 oct=3 lid=0 tim=1499185905162444 hv=3377894161 ad='84f13d70' sqlid='32d4jrb4pd4sj'
select charsetid, charsetform from col$  where obj# = :1 and col# = :2
END OF STMT
PARSE #140048443933232:c=0,e=294,p=0,cr=0,cu=0,mis=1,r=0,dep=1,og=4,plh=0,tim=1499185905162443
=====================
PARSING IN CURSOR #140048443932288 len=52 dep=1 uid=0 oct=3 lid=0 tim=1499185905247020 hv=429618617 ad='84f0f2d0' sqlid='4krwuz0ctqxdt'
select ctime, mtime, stime from obj$ where obj# = :1
END OF STMT
PARSE #140048443932288:c=0,e=549,p=0,cr=0,cu=0,mis=1,r=0,dep=1,og=4,plh=0,tim=1499185905247019
BINDS #140048443932288:
select blevel, leafcnt, distkey, lblkkey, dblkkey, clufac,        nvl(degree,1), nvl(instances,1) from ind$ where bo# = :1 and obj# = :2
END OF STMT
PARSE #140048443934176:c=1000,e=601,p=0,cr=0,cu=0,mis=1,r=0,dep=1,og=4,plh=0,tim=1499185905162088
=====================
PARSING IN CURSOR #140048443933232 len=70 dep=1 uid=0 oct=3 lid=0 tim=1499185905162444 hv=3377894161 ad='84f13d70' sqlid='32d4jrb4pd4sj'
select charsetid, charsetform from col$  where obj# = :1 and col# = :2
END OF STMT
PARSE #140048443933232:c=0,e=294,p=0,cr=0,cu=0,mis=1,r=0,dep=1,og=4,plh=0,tim=1499185905162443
=====================
PARSING IN CURSOR #140048443932288 len=52 dep=1 uid=0 oct=3 lid=0 tim=1499185905247020 hv=429618617 ad='84f0f2d0' sqlid='4krwuz0ctqxdt'
select ctime, mtime, stime from obj$ where obj# = :1
END OF STMT
PARSE #140048443932288:c=0,e=549,p=0,cr=0,cu=0,mis=1,r=0,dep=1,og=4,plh=0,tim=1499185905247019
BINDS #140048443932288:
 Bind#0
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=00 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=7f5f91b87bd0  bln=22  avl=02  flg=05
  value=20
EXEC #140048443932288:c=2000,e=2686,p=0,cr=0,cu=0,mis=1,r=0,dep=1,og=4,plh=1218588913,tim=1499185905249810
WAIT #140048443932288: nam='db file sequential read' ela= 6205 file#=1 block#=337 blocks=1 obj#=36 tim=1499185905256132
WAIT #140048443932288: nam='db file sequential read' ela= 3739 file#=1 block#=338 blocks=1 obj#=36 tim=1499185905266704
WAIT #140048443932288: nam='db file sequential read' ela= 4966 file#=1 block#=241 blocks=1 obj#=18 tim=1499185905271761
FETCH #140048443932288:c=0,e=21976,p=3,cr=3,cu=0,mis=0,r=1,dep=1,og=4,plh=1218588913,tim=1499185905271820
STAT #140048443932288 id=1 cnt=1 pid=0 pos=1 obj=18 op='TABLE ACCESS BY INDEX ROWID OBJ$ (cr=3 pr=3 pw=0 time=21993 us)'
STAT #140048443932288 id=2 cnt=1 pid=1 pos=1 obj=36 op='INDEX RANGE SCAN I_OBJ1 (cr=2 pr=2 pw=0 time=16923 us)'
CLOSE #140048443932288:c=0,e=63,dep=1,type=0,tim=1499185905271941
BINDS #140048443935120:
 Bind#0
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=7f5f91c07de8  bln=22  avl=02  flg=05
  value=20
EXEC #140048443935120:c=1000,e=795,p=0,cr=0,cu=0,mis=1,r=0,dep=1,og=4,plh=2970138452,tim=1499185905272802
WAIT #140048443935120: nam='db file sequential read' ela= 3197 file#=1 block#=169 blocks=1 obj#=3 tim=1499185905276069
WAIT #140048443935120: nam='db file sequential read' ela= 3459 file#=1 block#=170 blocks=1 obj#=3 tim=1499185905404084
WAIT #140048443935120: nam='db file sequential read' ela= 6358 file#=1 block#=145 blocks=1 obj#=4 tim=1499185905410548
FETCH #140048443935120:c=999,e=137805,p=3,cr=3,cu=0,mis=0,r=0,dep=1,og=4,plh=2970138452,tim=1499185905410635
STAT #140048443935120 id=1 cnt=0 pid=0 pos=1 obj=4 op='TABLE ACCESS CLUSTER TAB$ (cr=3 pr=3 pw=0 time=137810 us)'
STAT #140048443935120 id=2 cnt=1 pid=1 pos=1 obj=3 op='INDEX UNIQUE SCAN I_OBJ# (cr=2 pr=2 pw=0 time=131330 us)'
*** 2017-07-05 00:31:46.094
Incident 176395 created, dump file: /oracle/diag/rdbms/xifenfei/xifenfei/incident/incdir_176395/xifenfei_ora_51261_i176395.trc
ORA-00600: internal error code, arguments: [16703], [1403], [20], [], [], [], [], [], [], [], [], []

报错信息明显,由于select rowcnt,blkcnt,empcnt,avgspc,chncnt,avgrln,nvl(degree,1), nvl(instances,1) from tab$ where obj# = :1无法正常执行.从而出现了ORA-600 16703的错误,更加直接一点的解释就是obj#=20的对象在tab$中找不到记录,从而出现此类报错.和官方解释ORA-600 TAB$和obj$不匹配一致.
分析system文件
通过dul等工具分析system文件发现tab$表记录为空

Data UnLoader: 11.2.0.1.5 - Internal Only - on Wed Jul 05 01:28:53 2017
with 64-bit io functions and the decompression option
Copyright (c) 1994 2017 Bernard van Duijnen All rights reserved.
 Strictly Oracle Internal Use Only
Found db_id = 1334610369
Found db_name = xifenfei
DUL> unload table TAB$( OBJ# number, DATAOBJ# number,
  2      TS# number, FILE# number, BLOCK# number,
  3      BOBJ# number, TAB# number, COLS number, CLUCOLS number,
  4      PCTFREE$ ignore, PCTUSED$ ignore, INITRANS ignore, MAXTRANS ignore,
  5      FLAGS ignore, AUDIT$ ignore, ROWCNT ignore, BLKCNT ignore,
  6      EMPCNT ignore, AVGSPC ignore, CHNCNT ignore, AVGRLN ignore,
  7      AVGSPC_FLB ignore, FLBCNT ignore,
  8      ANALYZETIME ignore, SAMPLESIZE ignore,
  9      DEGREE ignore, INSTANCES ignore,
 10      INTCOLS ignore, KERNELCOLS number, PROPERTY number)
 11      cluster  C_OBJ#(OBJ#)
 12      storage ( tablespace 0 segobjno 2 tabno 1 file 1 block 144);
. unloading table                      TAB$       0 rows unloaded
DUL> unload table OBJ$( OBJ# number, DATAOBJ# number, OWNER# number,
  2      NAME clean varchar2(30), NAMESPACE ignore, SUBNAME clean varchar2(30),
  3      TYPE# number, CTIME ignore, MTIME ignore, STIME ignore,
  4      STATUS ignore, REMOTEOWNER ignore, LINKNAME ignore,
  5      FLAGS ignore, OID$ hexraw)
  6      storage ( tablespace 0 segobjno 18 file 1 block 240);
. unloading table                      OBJ$   89804 rows unloaded
DUL>

发现在obj$中有创建表ORACHKBEC66CBE055000000000001(ORACHK+16进制24位)的一个表名字
orachk


该表用途通过分析数据库日志发现
create table ORACHKBEC66CBE055000000000001 tablespace system as select * from sys.tab$;
也就是说,这个orachk的表是用来备份tab$的,然后进一步发现有delete from tab$.至此基本上分析清楚,tab$表备份到ORACHK表中,然后delete tab$表数据.实现数据库破坏以及难以恢复的效果.有点类似plsql dev引起的数据库被黑勒索比特币实现原理分析和解决方案的破坏案例
我们可以通过使用bbed对tab$表数据数据进行恢复实现数据库正常启动,实现数据0丢失,最大限度抢救您的数据和减少业务恢复时间
Phone:17813235971    Q Q:107644445QQ咨询惜分飞    E-Mail:dba@xifenfei.com
源头已经分析清楚,是有人篡改了oracle安装介质,导致数据库运行300天之后出现该问题,具体见:警告:互联网中有oracle介质被注入恶意程序导致—ORA-600 16703

Oracle 12c active dataguard switchover

从12.1开始adg的切换发生了一些改变,直接使用alter database switchover to [target standby db_unique_name] verify; alter database switchover to [target standby db_unique_name]; 即可完成切换,以下是一次生产环境的具体操作步骤
主库操作

SQL> select database_role from v$database;
DATABASE_ROLE
----------------
PRIMARY
SQL> alter database switchover to xifenfei verify;
数据库已更改。
--alert日志
Sun Jun 25 09:07:08 2017
diag_adl:SWITCHOVER VERIFY: Send VERIFY request to switchover target xifenfei
diag_adl:SWITCHOVER VERIFY COMPLETE
SQL> alter database switchover to xifenfei;
数据库已更改。
--alert日志
Sun Jun 25 09:07:46 2017
diag_adl:Starting switchover [Process ID: 37024]
Sun Jun 25 09:07:46 2017
diag_adl:ALTER DATABASE COMMIT TO SWITCHOVER TO PHYSICAL STANDBY [Process Id: 37024] (xifenfei)
diag_adl:Waiting for target standby to receive all redo
Sun Jun 25 09:07:46 2017
diag_adl:Waiting for all non-current ORLs to be archived...
Sun Jun 25 09:07:46 2017
diag_adl:All non-current ORLs have been archived.
Sun Jun 25 09:07:46 2017
diag_adl:Waiting for all FAL entries to be archived...
Sun Jun 25 09:07:46 2017
diag_adl:All FAL entries have been archived.
Sun Jun 25 09:07:46 2017
diag_adl:Waiting for dest_id 2 to become synchronized...
Sun Jun 25 09:07:47 2017
diag_adl:Active, synchronized Physical Standby switchover target has been identified
diag_adl:Preventing updates and queries at the Primary
diag_adl:Generating and shipping final logs to target standby
diag_adl:Switchover End-Of-Redo Log thread 1 sequence 96534 has been fixed
diag_adl:Switchover: Primary highest seen SCN set to 0x3.0x109d7502
diag_adl:ARCH: Noswitch archival of thread 1, sequence 96534
diag_adl:ARCH: End-Of-Redo Branch archival of thread 1 sequence 96534
diag_adl:ARCH: LGWR is scheduled to archive destination LOG_ARCHIVE_DEST_2 after log switch
diag_adl:ARCH: Standby redo logfile selected for thread 1 sequence 96534 for destination LOG_ARCHIVE_DEST_2
diag_adl:ARCH: Archiving is disabled due to current logfile archival
diag_adl:Primary will check for some target standby to have received all redo
diag_adl:Waiting for target standby to apply all redo
diag_adl:Backup controlfile written to trace file
            /u01/app/oracle/diag/rdbms/xifenfeildg/xifenfei/trace/xifenfei_ora_37024.trc
diag_adl:Converting the primary database to a new standby database
diag_adl:Clearing standby activation ID 612004791 (0x247a73b7)
diag_adl:The primary database controlfile was created using the
diag_adl:'MAXLOGFILES 16' clause.
diag_adl:There is space for up to 11 standby redo logfiles
diag_adl:Use the following SQL commands on the standby database to create
diag_adl:standby redo logfiles that match the primary database:
diag_adl:ALTER DATABASE ADD STANDBY LOGFILE 'srl1.f' SIZE 1073741824;
diag_adl:ALTER DATABASE ADD STANDBY LOGFILE 'srl2.f' SIZE 1073741824;
diag_adl:ALTER DATABASE ADD STANDBY LOGFILE 'srl3.f' SIZE 1073741824;
diag_adl:ALTER DATABASE ADD STANDBY LOGFILE 'srl4.f' SIZE 1073741824;
diag_adl:ALTER DATABASE ADD STANDBY LOGFILE 'srl5.f' SIZE 1073741824;
diag_adl:ALTER DATABASE ADD STANDBY LOGFILE 'srl6.f' SIZE 1073741824;
diag_adl:Archivelog for thread 1 sequence 96534 required for standby recovery
diag_adl:Switchover: Primary controlfile converted to standby controlfile succesfully.
diag_adl:Switchover: Complete - Database shutdown required
diag_adl:Sending request(convert to primary database) to switchover target xifenfei
Sun Jun 25 09:07:58 2017
diag_adl:Switchover complete. Database shutdown required
diag_adl:USER (ospid: 37024): terminating the instance
Sun Jun 25 09:07:59 2017
diag_adl:Instance terminated by USER, pid = 37024
diag_adl:Shutting down instance (abort)
diag_adl:License high water mark = 527
Sun Jun 25 09:07:59 2017
Instance shutdown complete

备库alert日志

Sun Jun 25 09:05:54 2017
SWITCHOVER VERIFY BEGIN
SWITCHOVER VERIFY COMPLETE
Sun Jun 25 09:06:35 2017
RFS[107]: Assigned to RFS process (PID:7330)
RFS[107]: Selected log 12 for thread 1 sequence 96534 dbid 588725663 branch 916962073
Sat Jun 24 20:06:35 2017
Archived Log entry 100576 added for thread 1 sequence 96534 ID 0x247a73b7 dest 1:
Sat Jun 24 20:06:35 2017
Resetting standby activation ID 612004791 (0x247a73b7)
Sat Jun 24 20:06:35 2017
Media Recovery End-Of-Redo indicator encountered
Sat Jun 24 20:06:35 2017
Media Recovery Continuing
Media Recovery Waiting for thread 1 sequence 96535
Sun Jun 25 09:06:36 2017
SWITCHOVER: received request 'ALTER DTABASE COMMIT TO SWITCHOVER  TO PRIMARY' from primary database.
Sun Jun 25 09:06:36 2017
ALTER DATABASE SWITCHOVER TO PRIMARY (xifenfei)
Maximum wait for role transition is 15 minutes.
Switchover: Media recovery is still active
Role Change: Canceling MRP - no more redo to apply
Sat Jun 24 20:06:36 2017
MRP0: Background Media Recovery cancelled with status 16037
Sat Jun 24 20:06:36 2017
Errors in file /u01/app/oracle/diag/rdbms/xifenfei/xifenfei/trace/xifenfei_pr00_4590.trc:
ORA-16037: user requested cancel of managed recovery operation
Managed Standby Recovery not using Real Time Apply
Recovery interrupted!
Sat Jun 24 20:06:36 2017
Errors in file /u01/app/oracle/diag/rdbms/xifenfei/xifenfei/trace/xifenfei_pr00_4590.trc:
ORA-16037: user requested cancel of managed recovery operation
Sat Jun 24 20:06:37 2017
MRP0: Background Media Recovery process shutdown (xifenfei)
Sun Jun 25 09:06:38 2017
Role Change: Canceled MRP
Killing 2 processes(PIDS:7328,4704)(all RFS)
      in order to disallow current and future RFS connections.Requested by OS process 7334
Stopping Emon pool
All dispatchers and shared servers shutdown
CLOSE: killing server sessions.
Active process 5428 user 'oracle' program 'oracle@kage7.hk0620.com (TNS V1-V3)'
Active process 5161 user 'oracle' program 'oracle@kage7.hk0620.com'
…………
Active process 5428 user 'oracle' program 'oracle@kage7.hk0620.com (TNS V1-V3)'
Active process 5161 user 'oracle' program 'oracle@kage7.hk0620.com'
Active process 5178 user 'oracle' program 'oracle@kage7.hk0620.com'
CLOSE: all sessions shutdown successfully.
Stopping Emon pool
Sat Jun 24 20:06:43 2017
SMON: disabling cache recovery
Sat Jun 24 20:06:44 2017
Buffer Cache Full DB Caching mode changing from FULL CACHING DISABLED to FULL CACHING ENABLED
Sun Jun 25 09:06:44 2017
Backup controlfile written to trace file /u01/app/oracle/diag/rdbms/xifenfei/xifenfei/trace/xifenfei_rmi_7334.trc
SwitchOver after complete recovery through change 13163656450
Online logfile pre-clearing operation disabled by switchover
Online log /u01/app/oracle/oradata/xifenfei/redo01n.log: Thread 1 Group 1 was previously cleared
Online log /u01/app/oracle/fast_recovery_area/xifenfei/redo01n.log: Thread 1 Group 1 was previously cleared
Online log /u01/app/oracle/oradata/xifenfei/redo02n.log: Thread 1 Group 2 was previously cleared
Online log /u01/app/oracle/fast_recovery_area/xifenfei/redo02n.log: Thread 1 Group 2 was previously cleared
Online log /u01/app/oracle/oradata/xifenfei/redo03n.log: Thread 1 Group 3 was previously cleared
Online log /u01/app/oracle/fast_recovery_area/xifenfei/redo03n.log: Thread 1 Group 3 was previously cleared
Online log /u01/app/oracle/oradata/xifenfei/redo04n.log: Thread 1 Group 4 was previously cleared
Online log /u01/app/oracle/fast_recovery_area/xifenfei/redo04n.log: Thread 1 Group 4 was previously cleared
Online log /u01/app/oracle/oradata/xifenfei/redo05n.log: Thread 1 Group 5 was previously cleared
Online log /u01/app/oracle/fast_recovery_area/xifenfei/redo05n.log: Thread 1 Group 5 was previously cleared
Standby became primary SCN: 13163656448
Switchover: Complete - Database mounted as primary
SWITCHOVER: completed request from primary database.
Sat Jun 24 20:07:12 2017
ARC0: Becoming the 'no SRL' ARCH

原备库(现主库)操作

SQL> conn / as sysdba
Connected.
SQL>  select database_role,open_mode from v$database;
DATABASE_ROLE    OPEN_MODE
---------------- --------------------
PRIMARY          MOUNTED
SQL> alter database open;
Database altered.

原主库(现备库)操作

[oracle@localhost scripts]$ ss
SQL*Plus: Release 12.1.0.2.0 Production on 星期日 6月 25 09:09:40 2017
Copyright (c) 1982, 2014, Oracle.  All rights reserved.
已连接到空闲例程。
SQL> startup
ORACLE 例程已经启动。
Total System Global Area 8.5899E+10 bytes
Fixed Size                  7654304 bytes
Variable Size            1.2616E+10 bytes
Database Buffers         7.3014E+10 bytes
Redo Buffers              260780032 bytes
数据库装载完毕。
数据库已经打开。
SQL> alter database recover managed standby database disconnect;
数据库已更改。