Oracle Apps DBA

My Photo
More than 5+ Years of IT Experience in administering Enterprise wide Multi Vendor UNIX Servers, Oracle Databases (8i to 11gR2), Middle-tiers, Applications and Clusters. I am a Sun Certified System Administrator (SCSA) for Solaris 10, oracle 10g database certified associate(OCA), Oracle 10g Database certified professional (OCP) and Oracle E-Business suite R12 certified professionl(OCP)

Oracle Database and Applications

Thursday, 17 April 2014

Upgrade RAC database from 11.2.0.1 to 11.2.0.4

This article illustrates step-by-step upgrading of a RAC database from 11.2.0. to 11.2.0.4.

High Level steps:

1) Install 11.2.0.4 RDBMS software with RAC option
2) Perform full database backup of 11.2.0.1 database in mount stage
3) Run DBUA from 11.2.0.4 Oracle_home
4) Verify number of INVALID objects and version of database


1) Install 11.2.0.4 RDBMS software with RAC option

- Make sure that proper DISPLAY variable is configured and execute ./runInstaller

- Follow screen shots
























- current version of database.


3) Run DBUA from 11.2.0.4 Oracle_home































Database has been successfully upgraded from 11.2.0.1 to 11.2.0.4.

thanks for reading.

regards,
X A H E E R

Wednesday, 26 March 2014

Excellent offer from PACKT publishers.

Here is an excellent opportunity to buy books from PACKT Publishers. They are celebrating 2000 titles for their published books. Now if you buy 1 book than you can get another book free of cost. Its 1 + 1 offer.




I am reading books from packt publishers since 3 years and  they are really very helpful for understanding latest technologies.

You can buy using URL's:
bit.ly/1j26nPN 
http://www.packtpub.com/?utm_source=referral&utm_medium=marketingPR&utm_campaign=2000thTitle

thanks,
X A H E E R

Monday, 24 March 2014

Upgrade Oracle Grid Infrastructure from 11.2.0.1 to 11.2.0.4

In this article I will demonstrate all steps required for upgrading Oracle Grid Infrastructure 11.2.0.1 to 11.2.0.4.  Current GI is running on 2 nodes cluster on Oracle enterprise Linux 5.7 with OCR and Voting Disk on ASM Disk group.

Node1 => linuxrac1
Node2 => linuxrac2


High Level steps for upgrading Grid Infrastructure:

1) Perform backup of OCR, voting disk and Database

2) Create new directory structure on both RAC nodes where 11.2.0.4 SW will be Installed and grant appropriate directory privileges

3) Run "runcluvfy.sh" to verify errors 

4) Apply required patches

5) Install and upgrade GI from 11.2.0.1 to 11.2.0.4

6) Verify upgrade


1) Perform local backup of OCR:

[root@linuxrac1 bin]# ./ocrconfig -manualbackup  
linuxrac2     2014/03/18 10:58:55     /u01/11g_grid/grid/cdata/scan-rac/backup_20140318_105855.ocr
[root@linuxrac1 bin]# ./ocrconfig -showbackup
linuxrac1     2014/02/19 15:28:32     /u01/11g_grid/grid/cdata/scan-rac/backup00.ocr
linuxrac1     2014/02/19 15:28:32     /u01/11g_grid/grid/cdata/scan-rac/day.ocr
linuxrac1     2014/02/19 15:28:32     /u01/11g_grid/grid/cdata/scan-rac/week.ocr
linuxrac2     2014/03/18 10:58:55     /u01/11g_grid/grid/cdata/scan-rac/backup_20140318_105855.ocr
[root@linuxrac1 bin]# 


2) Create new directories for 11.2.0.4 GI Home:

[root@linuxrac1 sw_home]# mkdir -p  /u01/11g_grid/11204_grid
[root@linuxrac1 sw_home]# chown -R oracle:dbarac /u01/11g_grid/11204_grid/
[root@linuxrac1 sw_home]# 


3) Run "runcluvfy.sh" to verify errors 

[oracle@linuxrac1 ~]$ /media/grid/runcluvfy.sh stage -pre crsinst -upgrade -n linuxrac1,linuxrac2 -rolling -src_crshome /u01/11g_grid/grid -dest_crshome /u01/11g_grid/11204_grid -dest_version 11.2.0.4.0 -fixup -fixupdir /home/oracle/fixup

Performing pre-checks for cluster services setup 

Checking node reachability...
Node reachability check passed from node "linuxrac1"


Checking user equivalence...
User equivalence check passed for user "oracle"

Checking CRS user consistency
CRS user consistency check successful

Checking node connectivity...

Checking hosts config file...

Verification of the hosts config file successful

Check: Node connectivity for interface "eth0"
Node connectivity passed for interface "eth0"
TCP connectivity check passed for subnet "192.168.1.0"


Check: Node connectivity for interface "eth1"
Node connectivity passed for interface "eth1"
TCP connectivity check passed for subnet "10.0.0.0"

Checking subnet mask consistency...
Subnet mask consistency check passed for subnet "192.168.1.0".
Subnet mask consistency check passed for subnet "10.0.0.0".
Subnet mask consistency check passed.

Node connectivity check passed

Checking multicast communication...

Checking subnet "192.168.1.0" for multicast communication with multicast group "230.0.1.0"...
Check of subnet "192.168.1.0" for multicast communication with multicast group "230.0.1.0" passed.

Checking subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0"...
Check of subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0" passed.

Check of multicast communication passed.

Checking OCR integrity...

OCR integrity check passed

Checking ASMLib configuration.
Check for ASMLib configuration passed.
Total memory check passed
Available memory check passed
Swap space check passed
Free disk space check passed for "linuxrac2:/u01/11g_grid/11204_grid"
Free disk space check passed for "linuxrac1:/u01/11g_grid/11204_grid"
Free disk space check passed for "linuxrac2:/tmp"
Free disk space check passed for "linuxrac1:/tmp"
Check for multiple users with UID value 500 passed 
User existence check passed for "oracle"
Group existence check passed for "dbarac"
Membership check for user "oracle" in group "dbarac" [as Primary] passed
Run level check passed
Hard limits check passed for "maximum open file descriptors"
Soft limits check passed for "maximum open file descriptors"
Hard limits check passed for "maximum user processes"
Soft limits check passed for "maximum user processes"
Check for Oracle patch "9413827 or 9706490" in home "/u01/11g_grid/grid" failed
Check failed on nodes: 
        linuxrac2,linuxrac1
There are no oracle patches required for home "/u01/11g_grid/11204_grid".
System architecture check passed
Kernel version check passed
Kernel parameter check passed for "semmsl"
Kernel parameter check passed for "semmns"
Kernel parameter check passed for "semopm"
Kernel parameter check passed for "semmni"
Kernel parameter check passed for "shmmax"
Kernel parameter check passed for "shmmni"
Kernel parameter check passed for "shmall"
Kernel parameter check passed for "file-max"
Kernel parameter check passed for "ip_local_port_range"
Kernel parameter check passed for "rmem_default"
Kernel parameter check passed for "rmem_max"
Kernel parameter check passed for "wmem_default"
Kernel parameter check passed for "wmem_max"
Kernel parameter check passed for "aio-max-nr"
Package existence check passed for "make"
Package existence check passed for "binutils"
Package existence check passed for "gcc"
Package existence check passed for "gcc-c++"
Package existence check passed for "libgomp"
Package existence check passed for "libaio"
Package existence check passed for "glibc"
Package existence check passed for "compat-libstdc++-33"
Package existence check passed for "elfutils-libelf"
Package existence check passed for "elfutils-libelf-devel"
Package existence check passed for "glibc-common"
Package existence check passed for "glibc-devel"
Package existence check passed for "glibc-headers"
Package existence check passed for "libaio-devel"
Package existence check passed for "libgcc"
Package existence check passed for "libstdc++"
Package existence check passed for "libstdc++-devel"
Package existence check passed for "sysstat"
Package existence check passed for "ksh"
Check for multiple users with UID value 0 passed 
Current group ID check passed

Starting check for consistency of primary group of root user

Check for consistency of root user's primary group passed
Package existence check passed for "cvuqdisk"

Starting Clock synchronization checks using Network Time Protocol(NTP)...

NTP Configuration file check started...
No NTP Daemons or Services were found to be running

Clock synchronization check using Network Time Protocol(NTP) passed

Core file name pattern consistency check passed.

User "oracle" is not part of "root" group. Check passed
Default user file creation mask check passed
Checking consistency of file "/etc/resolv.conf" across nodes

File "/etc/resolv.conf" does not have both domain and search entries defined
domain entry in file "/etc/resolv.conf" is consistent across nodes
search entry in file "/etc/resolv.conf" is consistent across nodes
PRVF-5636 : The DNS response time for an unreachable node exceeded "15000" ms on following nodes: linuxrac1

File "/etc/resolv.conf" is not consistent across nodes


UDev attributes check for OCR locations started...
UDev attributes check passed for OCR locations 


UDev attributes check for Voting Disk locations started...
UDev attributes check passed for Voting Disk locations 

Time zone consistency check passed
Checking VIP configuration.
Checking VIP Subnet configuration.
Check for VIP Subnet configuration passed.
Checking VIP reachability
Check for VIP reachability passed.

Checking Oracle Cluster Voting Disk configuration...

ASM Running check passed. ASM is running on all specified nodes

Oracle Cluster Voting Disk configuration check passed

Clusterware version consistency passed

Pre-check for cluster services setup was unsuccessful on all the nodes. 
[oracle@linuxrac1 ~]$ 


- cluvfy failed on Oracle Patch and DNS resolv.conf file

4) Apply required patches

I have applied these patches in rolling method:

- shutdown database running on node2
- Apply patch 9706450 on GI home then on RDBMS home
- startup database on node2
- Verify DB and GI status
- Apply patch on node1 with same sequence
- Verify all services are running without any errors

- All services are up and running on both nodes

[root@linuxrac2 bin]# ./crs_stat -t -v
Name           Type           R/RA   F/FT   Target    State     Host        
----------------------------------------------------------------------
ora.DBDATA.dg  ora....up.type 0/5    0/     ONLINE    ONLINE    linuxrac1   
ora.GDATA.dg   ora....up.type 0/5    0/     ONLINE    ONLINE    linuxrac1   
ora....ER.lsnr ora....er.type 0/5    0/     ONLINE    ONLINE    linuxrac1   
ora....N1.lsnr ora....er.type 0/5    0/0    ONLINE    ONLINE    linuxrac1   
ora.asm        ora.asm.type   0/5    0/     ONLINE    ONLINE    linuxrac1   
ora.eons       ora.eons.type  0/3    0/     ONLINE    ONLINE    linuxrac1   
ora.gsd        ora.gsd.type   0/5    0/     OFFLINE   OFFLINE               
ora.irprod.db  ora....se.type 0/2    0/1    ONLINE    ONLINE    linuxrac1   
ora....SM1.asm application    0/5    0/0    ONLINE    ONLINE    linuxrac1   
ora....C1.lsnr application    0/5    0/0    ONLINE    ONLINE    linuxrac1   
ora....ac1.gsd application    0/5    0/0    OFFLINE   OFFLINE               
ora....ac1.ons application    0/3    0/0    ONLINE    ONLINE    linuxrac1   
ora....ac1.vip ora....t1.type 0/0    0/0    ONLINE    ONLINE    linuxrac1   
ora....SM2.asm application    0/5    0/0    ONLINE    ONLINE    linuxrac2   
ora....C2.lsnr application    0/5    0/0    ONLINE    ONLINE    linuxrac2   
ora....ac2.gsd application    0/5    0/0    OFFLINE   OFFLINE               
ora....ac2.ons application    0/3    0/0    ONLINE    ONLINE    linuxrac2   
ora....ac2.vip ora....t1.type 0/0    0/0    ONLINE    ONLINE    linuxrac2   
ora....network ora....rk.type 0/5    0/     ONLINE    ONLINE    linuxrac1   
ora.oc4j       ora.oc4j.type  0/5    0/0    OFFLINE   OFFLINE               
ora.ons        ora.ons.type   0/3    0/     ONLINE    ONLINE    linuxrac1   
ora.scan1.vip  ora....ip.type 0/0    0/0    ONLINE    ONLINE    linuxrac1   
[root@linuxrac2 bin]#

- Shutdown database on node2

[oracle@linuxrac2 ~]$ srvctl status database -d IRPROD
Instance IRPROD1 is running on node linuxrac1
Instance IRPROD2 is not running on node linuxrac2
[oracle@linuxrac2 ~]$ 

- Run "rootcrs.pl -unlock" script before applying patch

[root@linuxrac2 bin]# /u01/11g_grid/grid/crs/install/rootcrs.pl -unlock
2014-03-19 09:45:10: Parsing the host name
2014-03-19 09:45:10: Checking for super user privileges
2014-03-19 09:45:10: User has super user privileges
Using configuration parameter file: /u01/11g_grid/grid/crs/install/crsconfig_params
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'linuxrac2'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'linuxrac2'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'linuxrac2'
CRS-2677: Stop of 'ora.gpnpd' on 'linuxrac2' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'linuxrac2'
CRS-2677: Stop of 'ora.mdnsd' on 'linuxrac2' succeeded
CRS-2677: Stop of 'ora.gipcd' on 'linuxrac2' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'linuxrac2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
Successfully unlock /u01/11g_grid/grid
[root@linuxrac2 bin]# 

- Make sure "opatch" executable in path

[oracle@linuxrac2 grid]$ export PATH=/u01/11g_grid/grid/OPatch:$PATH
[oracle@linuxrac2 grid]$ which opatch
/u01/11g_grid/grid/OPatch/opatch
[oracle@linuxrac2 grid]$ echo $ORACLE_HOME
/u01/11g_grid/grid
[oracle@linuxrac2 grid]$ 

- Apply patch on GI Home:

[oracle@linuxrac2 9706490]$ opatch napply -local -oh /u01/11g_grid/grid -id 9706490

- Run prepatch script for RDBMS home:

[oracle@linuxrac2 scripts]$ pwd
/u01/sw_home/Db_Patches/9706490/custom/scripts
[oracle@linuxrac2 scripts]$ sh prepatch.sh -dbhome /u01/11g_db/product/11.2.0/dbhome_1
prepatch.sh completed successfully.
[oracle@linuxrac2 scripts]$ 


- Apply patch on RDBMS Home

[oracle@linuxrac2 9706490]$ opatch napply custom/server/ -local -oh /u01/11g_db/product/11.2.0/dbhome_1 -id 9706490
Invoking OPatch 11.1.0.6.6

Oracle Interim Patch Installer version 11.1.0.6.6
Copyright (c) 2009, Oracle Corporation.  All rights reserved.

UTIL session

Oracle Home       : /u01/11g_db/product/11.2.0/dbhome_1
Central Inventory : /u01/ora/oraInventory
   from           : /etc/oraInst.loc
OPatch version    : 11.1.0.6.6
OUI version       : 11.2.0.1.0
OUI location      : /u01/11g_db/product/11.2.0/dbhome_1/oui
Log file location : /u01/11g_db/product/11.2.0/dbhome_1/cfgtoollogs/opatch/opatch2014-03-19_10-06-40AM.log

Patch history file: /u01/11g_db/product/11.2.0/dbhome_1/cfgtoollogs/opatch/opatch_history.txt

Invoking utility "napply"
Checking conflict among patches...
Checking if Oracle Home has components required by patches...
Checking conflicts against Oracle Home...
OPatch continues with these patches:   9706490  

Do you want to proceed? [y|n]

Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/srvm/mesg/procus.msg"
Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/srvm/mesg/proczhs.msb"
Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/srvm/mesg/proczht.msb"
Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/srvm/mesg/prvfus.msg"
Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/srvm/mesg/prvpus.msg"
Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/srvm/mesg/sclcus.msg"
Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/srvm/mesg/sclsus.msg"
Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/oui/jlib/srvm.jar"
Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/oui/jlib/srvmasm.jar"
Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/oui/lib/linux/libsrvm11.so"
ApplySession adding interim patch '9706490' to inventory

Verifying the update...
Inventory check OK: Patch ID 9706490 is registered in Oracle Home inventory with proper meta-data.
Files check OK: Files from Patch ID 9706490 are present in Oracle Home.
Running make for target install_srvm
Running make for target install

The local system has been patched and can be restarted.

UtilSession: N-Apply done.

OPatch succeeded.
[oracle@linuxrac2 9706490]$ 

-  Apply postpatch script on RDBMS home

[oracle@linuxrac2 scripts]$ sh postpatch.sh -dbhome /u01/11g_db/product/11.2.0/dbhome_1
Reading /u01/11g_db/product/11.2.0/dbhome_1/install/params.ora..
Reading /u01/11g_db/product/11.2.0/dbhome_1/install/params.ora..
Parsing file /u01/11g_db/product/11.2.0/dbhome_1/bin/racgwrap
Parsing file /u01/11g_db/product/11.2.0/dbhome_1/bin/srvctl
Parsing file /u01/11g_db/product/11.2.0/dbhome_1/bin/srvconfig
Parsing file /u01/11g_db/product/11.2.0/dbhome_1/bin/cluvfy
Verifying file /u01/11g_db/product/11.2.0/dbhome_1/bin/racgwrap
Verifying file /u01/11g_db/product/11.2.0/dbhome_1/bin/srvctl
Verifying file /u01/11g_db/product/11.2.0/dbhome_1/bin/srvconfig
Verifying file /u01/11g_db/product/11.2.0/dbhome_1/bin/cluvfy
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/racgwrap
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/srvctl
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/srvconfig
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/cluvfy
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/racgmain
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/racgeut
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/diskmon.bin
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/lsnodes
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/osdbagrp
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/rawutl
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/srvm/admin/ractrans
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/srvm/admin/getcrshome
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/gnsd
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/crsdiag.pl
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libhasgen11.so
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libclsra11.so
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libdbcfg11.so
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libocr11.so
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libocrb11.so
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libocrutl11.so
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libuini11.so
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/librdjni11.so
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libgns11.so
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libgnsjni11.so
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libagfw11.so
[oracle@linuxrac2 scripts]$ 

- run "rootcrl.pl" script:

[root@linuxrac2 bin]# /u01/11g_grid/grid/crs/install/rootcrs.pl -patch
2014-03-19 10:24:52: Parsing the host name
2014-03-19 10:24:52: Checking for super user privileges
2014-03-19 10:24:52: User has super user privileges
Using configuration parameter file: /u01/11g_grid/grid/crs/install/crsconfig_params
ADVM/ACFS is not supported on oraclelinux-release-5-7.0.2
CRS-4123: Oracle High Availability Services has been started.
[root@linuxrac2 bin]# 

- Verify GI services

[root@linuxrac2 bin]# ./crs_stat -t -v
Name           Type           R/RA   F/FT   Target    State     Host        
----------------------------------------------------------------------
ora.DBDATA.dg  ora....up.type 0/5    0/     ONLINE    ONLINE    linuxrac1   
ora.GDATA.dg   ora....up.type 0/5    0/     ONLINE    ONLINE    linuxrac1   
ora....ER.lsnr ora....er.type 0/5    0/     ONLINE    ONLINE    linuxrac1   
ora....N1.lsnr ora....er.type 0/5    0/0    ONLINE    ONLINE    linuxrac1   
ora.asm        ora.asm.type   0/5    0/     ONLINE    ONLINE    linuxrac1   
ora.eons       ora.eons.type  0/3    0/     ONLINE    ONLINE    linuxrac1   
ora.gsd        ora.gsd.type   0/5    0/     OFFLINE   OFFLINE               
ora.irprod.db  ora....se.type 0/2    0/1    ONLINE    ONLINE    linuxrac1   
ora....SM1.asm application    0/5    0/0    ONLINE    ONLINE    linuxrac1   
ora....C1.lsnr application    0/5    0/0    ONLINE    ONLINE    linuxrac1   
ora....ac1.gsd application    0/5    0/0    OFFLINE   OFFLINE               
ora....ac1.ons application    0/3    0/0    ONLINE    ONLINE    linuxrac1   
ora....ac1.vip ora....t1.type 0/0    0/0    ONLINE    ONLINE    linuxrac1   
ora....SM2.asm application    0/5    0/0    ONLINE    ONLINE    linuxrac2   
ora....C2.lsnr application    0/5    0/0    ONLINE    ONLINE    linuxrac2   
ora....ac2.gsd application    0/5    0/0    OFFLINE   OFFLINE               
ora....ac2.ons application    0/3    0/0    ONLINE    ONLINE    linuxrac2   
ora....ac2.vip ora....t1.type 0/0    0/0    ONLINE    ONLINE    linuxrac2   
ora....network ora....rk.type 0/5    0/     ONLINE    ONLINE    linuxrac1   
ora.oc4j       ora.oc4j.type  0/5    0/0    OFFLINE   OFFLINE               
ora.ons        ora.ons.type   0/3    0/     ONLINE    ONLINE    linuxrac1   
ora.scan1.vip  ora....ip.type 0/0    0/0    ONLINE    ONLINE    linuxrac1   
[root@linuxrac2 bin]# 

- Verify patch

[oracle@linuxrac2 9706490]$ opatch lsinventory -detail -oh  /u01/11g_grid/grid/

[oracle@linuxrac2 scripts]$ opatch lsinventory -detail -oh /u01/11g_db/product/11.2.0/dbhome_1

- Add following entry in /etc/resolv.conf 

options timeout:1 attempts:1 rotate

- Similarly perform all actions for applying patch on linuxrac1 (node1) 

- After successfull completion of patch on both nodes, please runcluvfy for installation verification:

[oracle@linuxrac1 ~]$ /media/grid/runcluvfy.sh stage -pre crsinst -upgrade -n linuxrac1,linuxrac2 -rolling -src_crshome /u01/11g_grid/grid -dest_crshome /u01/11g_grid/11204_grid -dest_version 11.2.0.4.0 -fixup -fixupdir /home/oracle/fixup

Performing pre-checks for cluster services setup 

Checking node reachability...
Node reachability check passed from node "linuxrac1"


Checking user equivalence...
User equivalence check passed for user "oracle"
l
Checking CRS user consistency
CRS user consistency check successful

Checking node connectivity...

Checking hosts config file...

Verification of the hosts config file successful

Check: Node connectivity for interface "eth0"
Node connectivity passed for interface "eth0"
TCP connectivity check passed for subnet "192.168.1.0"


Check: Node connectivity for interface "eth1"
Node connectivity passed for interface "eth1"
TCP connectivity check passed for subnet "10.0.0.0"

Checking subnet mask consistency...
Subnet mask consistency check passed for subnet "192.168.1.0".
Subnet mask consistency check passed for subnet "10.0.0.0".
Subnet mask consistency check passed.

Node connectivity check passed

Checking multicast communication...

Checking subnet "192.168.1.0" for multicast communication with multicast group "230.0.1.0"...
Check of subnet "192.168.1.0" for multicast communication with multicast group "230.0.1.0" passed.

Checking subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0"...
Check of subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0" passed.

Check of multicast communication passed.

Checking OCR integrity...

OCR integrity check passed

Checking ASMLib configuration.
Check for ASMLib configuration passed.
Total memory check passed
Available memory check passed
Swap space check passed
Free disk space check passed for "linuxrac2:/u01/11g_grid/11204_grid"
Free disk space check passed for "linuxrac1:/u01/11g_grid/11204_grid"
Free disk space check passed for "linuxrac2:/tmp"
Free disk space check passed for "linuxrac1:/tmp"
Check for multiple users with UID value 500 passed 
User existence check passed for "oracle"
Group existence check passed for "dbarac"
Membership check for user "oracle" in group "dbarac" [as Primary] passed
Run level check passed
Hard limits check passed for "maximum open file descriptors"
Soft limits check passed for "maximum open file descriptors"
Hard limits check passed for "maximum user processes"
Soft limits check passed for "maximum user processes"
Check for Oracle patch "9413827 or 9706490" in home "/u01/11g_grid/grid" passed
There are no oracle patches required for home "/u01/11g_grid/11204_grid".
System architecture check passed
Kernel version check passed
Kernel parameter check passed for "semmsl"
Kernel parameter check passed for "semmns"
Kernel parameter check passed for "semopm"
Kernel parameter check passed for "semmni"
Kernel parameter check passed for "shmmax"
Kernel parameter check passed for "shmmni"
Kernel parameter check passed for "shmall"
Kernel parameter check passed for "file-max"
Kernel parameter check passed for "ip_local_port_range"
Kernel parameter check passed for "rmem_default"
Kernel parameter check passed for "rmem_max"
Kernel parameter check passed for "wmem_default"
Kernel parameter check passed for "wmem_max"
Kernel parameter check passed for "aio-max-nr"
Package existence check passed for "make"
Package existence check passed for "binutils"
Package existence check passed for "gcc"
Package existence check passed for "gcc-c++"
Package existence check passed for "libgomp"
Package existence check passed for "libaio"
Package existence check passed for "glibc"
Package existence check passed for "compat-libstdc++-33"
Package existence check passed for "elfutils-libelf"
Package existence check passed for "elfutils-libelf-devel"
Package existence check passed for "glibc-common"
Package existence check passed for "glibc-devel"
Package existence check passed for "glibc-headers"
Package existence check passed for "libaio-devel"
Package existence check passed for "libgcc"
Package existence check passed for "libstdc++"
Package existence check passed for "libstdc++-devel"
Package existence check passed for "sysstat"
Package existence check passed for "ksh"
Check for multiple users with UID value 0 passed 
Current group ID check passed

Starting check for consistency of primary group of root user

Check for consistency of root user's primary group passed
Package existence check passed for "cvuqdisk"

Starting Clock synchronization checks using Network Time Protocol(NTP)...

NTP Configuration file check started...
No NTP Daemons or Services were found to be running

Clock synchronization check using Network Time Protocol(NTP) passed

Core file name pattern consistency check passed.

User "oracle" is not part of "root" group. Check passed
Default user file creation mask check passed
Checking consistency of file "/etc/resolv.conf" across nodes

File "/etc/resolv.conf" does not have both domain and search entries defined
domain entry in file "/etc/resolv.conf" is consistent across nodes
search entry in file "/etc/resolv.conf" is consistent across nodes
PRVF-5636 : The DNS response time for an unreachable node exceeded "15000" ms on following nodes: linuxrac2,linuxrac1

File "/etc/resolv.conf" is not consistent across nodes


UDev attributes check for OCR locations started...
UDev attributes check passed for OCR locations 


UDev attributes check for Voting Disk locations started...
UDev attributes check passed for Voting Disk locations 

Time zone consistency check passed
Checking VIP configuration.
Checking VIP Subnet configuration.
Check for VIP Subnet configuration passed.
Checking VIP reachability
Check for VIP reachability passed.

Checking Oracle Cluster Voting Disk configuration...

ASM Running check passed. ASM is running on all specified nodes

Oracle Cluster Voting Disk configuration check passed

Clusterware version consistency passed

Pre-check for cluster services setup was unsuccessful on all the nodes. 
[oracle@linuxrac1 ~]$ /media/grid/runcluvfy.sh stage -pre crsinst -upgrade -n linuxrac1,linuxrac2 -rolling -src_crshome /u01/11g_grid/grid -dest_crshome /u01/11g_grid/11204_grid -dest_version 11.2.0.4.0 -fixup -fixupdir /home/oracle/fixup

Performing pre-checks for cluster services setup 

Checking node reachability...
Node reachability check passed from node "linuxrac1"


Checking user equivalence...
User equivalence check passed for user "oracle"

Checking CRS user consistency
CRS user consistency check successful

Checking node connectivity...

Checking hosts config file...

Verification of the hosts config file successful

Check: Node connectivity for interface "eth0"
Node connectivity passed for interface "eth0"
TCP connectivity check passed for subnet "192.168.1.0"


Check: Node connectivity for interface "eth1"
Node connectivity passed for interface "eth1"
TCP connectivity check passed for subnet "10.0.0.0"

Checking subnet mask consistency...
Subnet mask consistency check passed for subnet "192.168.1.0".
Subnet mask consistency check passed for subnet "10.0.0.0".
Subnet mask consistency check passed.

Node connectivity check passed

Checking multicast communication...

Checking subnet "192.168.1.0" for multicast communication with multicast group "230.0.1.0"...
Check of subnet "192.168.1.0" for multicast communication with multicast group "230.0.1.0" passed.

Checking subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0"...
Check of subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0" passed.

Check of multicast communication passed.

Checking OCR integrity...

OCR integrity check passed

Checking ASMLib configuration.
Check for ASMLib configuration passed.
Total memory check passed
Available memory check passed
Swap space check passed
Free disk space check passed for "linuxrac2:/u01/11g_grid/11204_grid"
Free disk space check passed for "linuxrac1:/u01/11g_grid/11204_grid"
Free disk space check passed for "linuxrac2:/tmp"
Free disk space check passed for "linuxrac1:/tmp"
Check for multiple users with UID value 500 passed 
User existence check passed for "oracle"
Group existence check passed for "dbarac"
Membership check for user "oracle" in group "dbarac" [as Primary] passed
Run level check passed
Hard limits check passed for "maximum open file descriptors"
Soft limits check passed for "maximum open file descriptors"
Hard limits check passed for "maximum user processes"
Soft limits check passed for "maximum user processes"
Check for Oracle patch "9413827 or 9706490" in home "/u01/11g_grid/grid" passed
There are no oracle patches required for home "/u01/11g_grid/11204_grid".
System architecture check passed
Kernel version check passed
Kernel parameter check passed for "semmsl"
Kernel parameter check passed for "semmns"
Kernel parameter check passed for "semopm"
Kernel parameter check passed for "semmni"
Kernel parameter check passed for "shmmax"
Kernel parameter check passed for "shmmni"
Kernel parameter check passed for "shmall"
Kernel parameter check passed for "file-max"
Kernel parameter check passed for "ip_local_port_range"
Kernel parameter check passed for "rmem_default"
Kernel parameter check passed for "rmem_max"
Kernel parameter check passed for "wmem_default"
Kernel parameter check passed for "wmem_max"
Kernel parameter check passed for "aio-max-nr"
Package existence check passed for "make"
Package existence check passed for "binutils"
Package existence check passed for "gcc"
Package existence check passed for "gcc-c++"
Package existence check passed for "libgomp"
Package existence check passed for "libaio"
Package existence check passed for "glibc"
Package existence check passed for "compat-libstdc++-33"
Package existence check passed for "elfutils-libelf"
Package existence check passed for "elfutils-libelf-devel"
Package existence check passed for "glibc-common"
Package existence check passed for "glibc-devel"
Package existence check passed for "glibc-headers"
Package existence check passed for "libaio-devel"
Package existence check passed for "libgcc"
Package existence check passed for "libstdc++"
Package existence check passed for "libstdc++-devel"
Package existence check passed for "sysstat"
Package existence check passed for "ksh"
Check for multiple users with UID value 0 passed 
Current group ID check passed

Starting check for consistency of primary group of root user

Check for consistency of root user's primary group passed
Package existence check passed for "cvuqdisk"

Starting Clock synchronization checks using Network Time Protocol(NTP)...

NTP Configuration file check started...
No NTP Daemons or Services were found to be running

Clock synchronization check using Network Time Protocol(NTP) passed

Core file name pattern consistency check passed.

User "oracle" is not part of "root" group. Check passed
Default user file creation mask check passed
Checking consistency of file "/etc/resolv.conf" across nodes

File "/etc/resolv.conf" does not have both domain and search entries defined
domain entry in file "/etc/resolv.conf" is consistent across nodes
search entry in file "/etc/resolv.conf" is consistent across nodes
The DNS response time for an unreachable node is within acceptable limit on all nodes

File "/etc/resolv.conf" is consistent across nodes


UDev attributes check for OCR locations started...
UDev attributes check passed for OCR locations 


UDev attributes check for Voting Disk locations started...
UDev attributes check passed for Voting Disk locations 

Time zone consistency check passed
Checking VIP configuration.
Checking VIP Subnet configuration.
Check for VIP Subnet configuration passed.
Checking VIP reachability
Check for VIP reachability passed.

Checking Oracle Cluster Voting Disk configuration...

ASM Running check passed. ASM is running on all specified nodes

Oracle Cluster Voting Disk configuration check passed

Clusterware version consistency passed

Pre-check for cluster services setup was successful. 
[oracle@linuxrac1 ~]$ 


- Make sure full database backup has been performed and database in not running from both the nodes.

5) Install and upgrade Grid Infrastructure:

- set the proper display variable and execute "runInstaller" from grid software owner

- Follow screen shots




- Select "Upgrade"




- Verify SSH connectivity



- Provide location for 11.2.0.4 GI home location








- run "upgraderoot.sh" on both nodes as listed

root@linuxrac1 ~]# /u01/11g_grid/11204_grid/rootupgrade.sh
Performing root user operation for Oracle 11g 

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/11g_grid/11204_grid

Enter the full pathname of the local bin directory: [/usr/local/bin]: 
The contents of "dbhome" have not changed. No need to overwrite.
The file "oraenv" already exists in /usr/local/bin.  Overwrite it? (y/n) 
[n]: y
   Copying oraenv to /usr/local/bin ...
The file "coraenv" already exists in /usr/local/bin.  Overwrite it? (y/n) 
[n]: y
   Copying coraenv to /usr/local/bin ...

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/11g_grid/11204_grid/crs/install/crsconfig_params
Creating trace directory
Installing Trace File Analyzer

ASM upgrade has started on first node.


CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'linuxrac1'
CRS-2673: Attempting to stop 'ora.crsd' on 'linuxrac1'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'linuxrac1'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'linuxrac1'
CRS-2673: Attempting to stop 'ora.GDATA.dg' on 'linuxrac1'
CRS-2673: Attempting to stop 'ora.DBDATA.dg' on 'linuxrac1'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'linuxrac1' succeeded
CRS-2673: Attempting to stop 'ora.linuxrac1.vip' on 'linuxrac1'
CRS-2677: Stop of 'ora.linuxrac1.vip' on 'linuxrac1' succeeded
CRS-2672: Attempting to start 'ora.linuxrac1.vip' on 'linuxrac2'
CRS-2676: Start of 'ora.linuxrac1.vip' on 'linuxrac2' succeeded
CRS-2677: Stop of 'ora.GDATA.dg' on 'linuxrac1' succeeded
CRS-2677: Stop of 'ora.DBDATA.dg' on 'linuxrac1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'linuxrac1'
CRS-2677: Stop of 'ora.asm' on 'linuxrac1' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'linuxrac1'
CRS-2673: Attempting to stop 'ora.eons' on 'linuxrac1'
CRS-2677: Stop of 'ora.ons' on 'linuxrac1' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'linuxrac1'
CRS-2677: Stop of 'ora.net1.network' on 'linuxrac1' succeeded
CRS-2677: Stop of 'ora.eons' on 'linuxrac1' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'linuxrac1' has completed
CRS-2677: Stop of 'ora.crsd' on 'linuxrac1' succeeded
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'linuxrac1'
CRS-2673: Attempting to stop 'ora.ctssd' on 'linuxrac1'
CRS-2673: Attempting to stop 'ora.evmd' on 'linuxrac1'
CRS-2673: Attempting to stop 'ora.asm' on 'linuxrac1'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'linuxrac1'
CRS-2677: Stop of 'ora.cssdmonitor' on 'linuxrac1' succeeded
CRS-2677: Stop of 'ora.evmd' on 'linuxrac1' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'linuxrac1' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'linuxrac1' succeeded
CRS-2677: Stop of 'ora.asm' on 'linuxrac1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'linuxrac1'
CRS-2677: Stop of 'ora.cssd' on 'linuxrac1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'linuxrac1'
CRS-2673: Attempting to stop 'ora.diskmon' on 'linuxrac1'
CRS-2677: Stop of 'ora.gpnpd' on 'linuxrac1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'linuxrac1'
CRS-2677: Stop of 'ora.diskmon' on 'linuxrac1' succeeded
CRS-2677: Stop of 'ora.gipcd' on 'linuxrac1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'linuxrac1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
OLR initialization - successful
Replacing Clusterware entries in inittab
clscfg: EXISTING configuration version 5 detected.
clscfg: version 5 is 11g Release 2.
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
[root@linuxrac1 ~]# 


root@linuxrac2 ~]# /u01/11g_grid/11204_grid/rootupgrade.sh
Performing root user operation for Oracle 11g 

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/11g_grid/11204_grid

Enter the full pathname of the local bin directory: [/usr/local/bin]: 
The contents of "dbhome" have not changed. No need to overwrite.
The file "oraenv" already exists in /usr/local/bin.  Overwrite it? (y/n) 
[n]: y
   Copying oraenv to /usr/local/bin ...
The file "coraenv" already exists in /usr/local/bin.  Overwrite it? (y/n) 
[n]: y
   Copying coraenv to /usr/local/bin ...

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/11g_grid/11204_grid/crs/install/crsconfig_params
Creating trace directory
Installing Trace File Analyzer
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'linuxrac2'
CRS-2673: Attempting to stop 'ora.crsd' on 'linuxrac2'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'linuxrac2'
CRS-2673: Attempting to stop 'ora.GDATA.dg' on 'linuxrac2'
CRS-2673: Attempting to stop 'ora.DBDATA.dg' on 'linuxrac2'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'linuxrac2'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'linuxrac2'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'linuxrac2' succeeded
CRS-2673: Attempting to stop 'ora.linuxrac2.vip' on 'linuxrac2'
CRS-2677: Stop of 'ora.linuxrac2.vip' on 'linuxrac2' succeeded
CRS-2672: Attempting to start 'ora.linuxrac2.vip' on 'linuxrac1'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'linuxrac2' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'linuxrac2'
CRS-2677: Stop of 'ora.scan1.vip' on 'linuxrac2' succeeded
CRS-2672: Attempting to start 'ora.scan1.vip' on 'linuxrac1'
CRS-2676: Start of 'ora.linuxrac2.vip' on 'linuxrac1' succeeded
CRS-2676: Start of 'ora.scan1.vip' on 'linuxrac1' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'linuxrac1'
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'linuxrac1' succeeded
CRS-2677: Stop of 'ora.GDATA.dg' on 'linuxrac2' succeeded
CRS-2677: Stop of 'ora.DBDATA.dg' on 'linuxrac2' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'linuxrac2'
CRS-2677: Stop of 'ora.asm' on 'linuxrac2' succeeded
CRS-2673: Attempting to stop 'ora.eons' on 'linuxrac2'
CRS-2673: Attempting to stop 'ora.ons' on 'linuxrac2'
CRS-2677: Stop of 'ora.ons' on 'linuxrac2' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'linuxrac2'
CRS-2677: Stop of 'ora.net1.network' on 'linuxrac2' succeeded
CRS-2677: Stop of 'ora.eons' on 'linuxrac2' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'linuxrac2' has completed
CRS-2677: Stop of 'ora.crsd' on 'linuxrac2' succeeded
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'linuxrac2'
CRS-2673: Attempting to stop 'ora.ctssd' on 'linuxrac2'
CRS-2673: Attempting to stop 'ora.evmd' on 'linuxrac2'
CRS-2673: Attempting to stop 'ora.asm' on 'linuxrac2'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'linuxrac2'
CRS-2677: Stop of 'ora.cssdmonitor' on 'linuxrac2' succeeded
CRS-2677: Stop of 'ora.evmd' on 'linuxrac2' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'linuxrac2' succeeded
CRS-2677: Stop of 'ora.asm' on 'linuxrac2' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'linuxrac2' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'linuxrac2'
CRS-2677: Stop of 'ora.cssd' on 'linuxrac2' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'linuxrac2'
CRS-2673: Attempting to stop 'ora.diskmon' on 'linuxrac2'
CRS-2677: Stop of 'ora.gpnpd' on 'linuxrac2' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'linuxrac2'
CRS-2677: Stop of 'ora.gipcd' on 'linuxrac2' succeeded
CRS-2677: Stop of 'ora.diskmon' on 'linuxrac2' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'linuxrac2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
OLR initialization - successful
Replacing Clusterware entries in inittab
clscfg: EXISTING configuration version 5 detected.
clscfg: version 5 is 11g Release 2.
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Started to upgrade the Oracle Clusterware. This operation may take a few minutes.
Started to upgrade the CSS.
Started to upgrade the CRS.
The CRS was successfully upgraded.
Successfully upgraded the Oracle Clusterware.
Oracle Clusterware operating version was successfully set to 11.2.0.4.0

ASM upgrade has finished on last node.


Configure Oracle Grid Infrastructure for a Cluster ... succeeded
[root@linuxrac2 ~]#




6) Verify Upgrade:



- Verify services:


- startup database and make sure both Instances are working fine without any Issues.



11.2.0.1 Grid Infrastructure has been successfully upgraded to 11.2.0.4.

In next article i will demonstrate how to upgrade an Oracle RAC database from 11.2.0.1 to 11.2.0.4

Thanks for reading.

regards,
X A H E E R