Skip to main content

Upgrade Oracle Grid Infrastructure from 11.2.0.1 to 11.2.0.4

In this article I will demonstrate all steps required for upgrading Oracle Grid Infrastructure 11.2.0.1 to 11.2.0.4.  Current GI is running on 2 nodes cluster on Oracle enterprise Linux 5.7 with OCR and Voting Disk on ASM Disk group.

Node1 => linuxrac1
Node2 => linuxrac2


High Level steps for upgrading Grid Infrastructure:

1) Perform backup of OCR, voting disk and Database

2) Create new directory structure on both RAC nodes where 11.2.0.4 SW will be Installed and grant appropriate directory privileges

3) Run "runcluvfy.sh" to verify errors 

4) Apply required patches

5) Install and upgrade GI from 11.2.0.1 to 11.2.0.4

6) Verify upgrade


1) Perform local backup of OCR:

[root@linuxrac1 bin]# ./ocrconfig -manualbackup  
linuxrac2     2014/03/18 10:58:55     /u01/11g_grid/grid/cdata/scan-rac/backup_20140318_105855.ocr
[root@linuxrac1 bin]# ./ocrconfig -showbackup
linuxrac1     2014/02/19 15:28:32     /u01/11g_grid/grid/cdata/scan-rac/backup00.ocr
linuxrac1     2014/02/19 15:28:32     /u01/11g_grid/grid/cdata/scan-rac/day.ocr
linuxrac1     2014/02/19 15:28:32     /u01/11g_grid/grid/cdata/scan-rac/week.ocr
linuxrac2     2014/03/18 10:58:55     /u01/11g_grid/grid/cdata/scan-rac/backup_20140318_105855.ocr
[root@linuxrac1 bin]# 


2) Create new directories for 11.2.0.4 GI Home:

[root@linuxrac1 sw_home]# mkdir -p  /u01/11g_grid/11204_grid
[root@linuxrac1 sw_home]# chown -R oracle:dbarac /u01/11g_grid/11204_grid/
[root@linuxrac1 sw_home]# 


3) Run "runcluvfy.sh" to verify errors 

[oracle@linuxrac1 ~]$ /media/grid/runcluvfy.sh stage -pre crsinst -upgrade -n linuxrac1,linuxrac2 -rolling -src_crshome /u01/11g_grid/grid -dest_crshome /u01/11g_grid/11204_grid -dest_version 11.2.0.4.0 -fixup -fixupdir /home/oracle/fixup

Performing pre-checks for cluster services setup 

Checking node reachability...
Node reachability check passed from node "linuxrac1"


Checking user equivalence...
User equivalence check passed for user "oracle"

Checking CRS user consistency
CRS user consistency check successful

Checking node connectivity...

Checking hosts config file...

Verification of the hosts config file successful

Check: Node connectivity for interface "eth0"
Node connectivity passed for interface "eth0"
TCP connectivity check passed for subnet "192.168.1.0"


Check: Node connectivity for interface "eth1"
Node connectivity passed for interface "eth1"
TCP connectivity check passed for subnet "10.0.0.0"

Checking subnet mask consistency...
Subnet mask consistency check passed for subnet "192.168.1.0".
Subnet mask consistency check passed for subnet "10.0.0.0".
Subnet mask consistency check passed.

Node connectivity check passed

Checking multicast communication...

Checking subnet "192.168.1.0" for multicast communication with multicast group "230.0.1.0"...
Check of subnet "192.168.1.0" for multicast communication with multicast group "230.0.1.0" passed.

Checking subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0"...
Check of subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0" passed.

Check of multicast communication passed.

Checking OCR integrity...

OCR integrity check passed

Checking ASMLib configuration.
Check for ASMLib configuration passed.
Total memory check passed
Available memory check passed
Swap space check passed
Free disk space check passed for "linuxrac2:/u01/11g_grid/11204_grid"
Free disk space check passed for "linuxrac1:/u01/11g_grid/11204_grid"
Free disk space check passed for "linuxrac2:/tmp"
Free disk space check passed for "linuxrac1:/tmp"
Check for multiple users with UID value 500 passed 
User existence check passed for "oracle"
Group existence check passed for "dbarac"
Membership check for user "oracle" in group "dbarac" [as Primary] passed
Run level check passed
Hard limits check passed for "maximum open file descriptors"
Soft limits check passed for "maximum open file descriptors"
Hard limits check passed for "maximum user processes"
Soft limits check passed for "maximum user processes"
Check for Oracle patch "9413827 or 9706490" in home "/u01/11g_grid/grid" failed
Check failed on nodes: 
        linuxrac2,linuxrac1
There are no oracle patches required for home "/u01/11g_grid/11204_grid".
System architecture check passed
Kernel version check passed
Kernel parameter check passed for "semmsl"
Kernel parameter check passed for "semmns"
Kernel parameter check passed for "semopm"
Kernel parameter check passed for "semmni"
Kernel parameter check passed for "shmmax"
Kernel parameter check passed for "shmmni"
Kernel parameter check passed for "shmall"
Kernel parameter check passed for "file-max"
Kernel parameter check passed for "ip_local_port_range"
Kernel parameter check passed for "rmem_default"
Kernel parameter check passed for "rmem_max"
Kernel parameter check passed for "wmem_default"
Kernel parameter check passed for "wmem_max"
Kernel parameter check passed for "aio-max-nr"
Package existence check passed for "make"
Package existence check passed for "binutils"
Package existence check passed for "gcc"
Package existence check passed for "gcc-c++"
Package existence check passed for "libgomp"
Package existence check passed for "libaio"
Package existence check passed for "glibc"
Package existence check passed for "compat-libstdc++-33"
Package existence check passed for "elfutils-libelf"
Package existence check passed for "elfutils-libelf-devel"
Package existence check passed for "glibc-common"
Package existence check passed for "glibc-devel"
Package existence check passed for "glibc-headers"
Package existence check passed for "libaio-devel"
Package existence check passed for "libgcc"
Package existence check passed for "libstdc++"
Package existence check passed for "libstdc++-devel"
Package existence check passed for "sysstat"
Package existence check passed for "ksh"
Check for multiple users with UID value 0 passed 
Current group ID check passed

Starting check for consistency of primary group of root user

Check for consistency of root user's primary group passed
Package existence check passed for "cvuqdisk"

Starting Clock synchronization checks using Network Time Protocol(NTP)...

NTP Configuration file check started...
No NTP Daemons or Services were found to be running

Clock synchronization check using Network Time Protocol(NTP) passed

Core file name pattern consistency check passed.

User "oracle" is not part of "root" group. Check passed
Default user file creation mask check passed
Checking consistency of file "/etc/resolv.conf" across nodes

File "/etc/resolv.conf" does not have both domain and search entries defined
domain entry in file "/etc/resolv.conf" is consistent across nodes
search entry in file "/etc/resolv.conf" is consistent across nodes
PRVF-5636 : The DNS response time for an unreachable node exceeded "15000" ms on following nodes: linuxrac1

File "/etc/resolv.conf" is not consistent across nodes


UDev attributes check for OCR locations started...
UDev attributes check passed for OCR locations 


UDev attributes check for Voting Disk locations started...
UDev attributes check passed for Voting Disk locations 

Time zone consistency check passed
Checking VIP configuration.
Checking VIP Subnet configuration.
Check for VIP Subnet configuration passed.
Checking VIP reachability
Check for VIP reachability passed.

Checking Oracle Cluster Voting Disk configuration...

ASM Running check passed. ASM is running on all specified nodes

Oracle Cluster Voting Disk configuration check passed

Clusterware version consistency passed

Pre-check for cluster services setup was unsuccessful on all the nodes. 
[oracle@linuxrac1 ~]$ 


- cluvfy failed on Oracle Patch and DNS resolv.conf file

4) Apply required patches

I have applied these patches in rolling method:

- shutdown database running on node2
- Apply patch 9706450 on GI home then on RDBMS home
- startup database on node2
- Verify DB and GI status
- Apply patch on node1 with same sequence
- Verify all services are running without any errors

- All services are up and running on both nodes

[root@linuxrac2 bin]# ./crs_stat -t -v
Name           Type           R/RA   F/FT   Target    State     Host        
----------------------------------------------------------------------
ora.DBDATA.dg  ora....up.type 0/5    0/     ONLINE    ONLINE    linuxrac1   
ora.GDATA.dg   ora....up.type 0/5    0/     ONLINE    ONLINE    linuxrac1   
ora....ER.lsnr ora....er.type 0/5    0/     ONLINE    ONLINE    linuxrac1   
ora....N1.lsnr ora....er.type 0/5    0/0    ONLINE    ONLINE    linuxrac1   
ora.asm        ora.asm.type   0/5    0/     ONLINE    ONLINE    linuxrac1   
ora.eons       ora.eons.type  0/3    0/     ONLINE    ONLINE    linuxrac1   
ora.gsd        ora.gsd.type   0/5    0/     OFFLINE   OFFLINE               
ora.irprod.db  ora....se.type 0/2    0/1    ONLINE    ONLINE    linuxrac1   
ora....SM1.asm application    0/5    0/0    ONLINE    ONLINE    linuxrac1   
ora....C1.lsnr application    0/5    0/0    ONLINE    ONLINE    linuxrac1   
ora....ac1.gsd application    0/5    0/0    OFFLINE   OFFLINE               
ora....ac1.ons application    0/3    0/0    ONLINE    ONLINE    linuxrac1   
ora....ac1.vip ora....t1.type 0/0    0/0    ONLINE    ONLINE    linuxrac1   
ora....SM2.asm application    0/5    0/0    ONLINE    ONLINE    linuxrac2   
ora....C2.lsnr application    0/5    0/0    ONLINE    ONLINE    linuxrac2   
ora....ac2.gsd application    0/5    0/0    OFFLINE   OFFLINE               
ora....ac2.ons application    0/3    0/0    ONLINE    ONLINE    linuxrac2   
ora....ac2.vip ora....t1.type 0/0    0/0    ONLINE    ONLINE    linuxrac2   
ora....network ora....rk.type 0/5    0/     ONLINE    ONLINE    linuxrac1   
ora.oc4j       ora.oc4j.type  0/5    0/0    OFFLINE   OFFLINE               
ora.ons        ora.ons.type   0/3    0/     ONLINE    ONLINE    linuxrac1   
ora.scan1.vip  ora....ip.type 0/0    0/0    ONLINE    ONLINE    linuxrac1   
[root@linuxrac2 bin]#

- Shutdown database on node2

[oracle@linuxrac2 ~]$ srvctl status database -d IRPROD
Instance IRPROD1 is running on node linuxrac1
Instance IRPROD2 is not running on node linuxrac2
[oracle@linuxrac2 ~]$ 

- Run "rootcrs.pl -unlock" script before applying patch

[root@linuxrac2 bin]# /u01/11g_grid/grid/crs/install/rootcrs.pl -unlock
2014-03-19 09:45:10: Parsing the host name
2014-03-19 09:45:10: Checking for super user privileges
2014-03-19 09:45:10: User has super user privileges
Using configuration parameter file: /u01/11g_grid/grid/crs/install/crsconfig_params
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'linuxrac2'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'linuxrac2'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'linuxrac2'
CRS-2677: Stop of 'ora.gpnpd' on 'linuxrac2' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'linuxrac2'
CRS-2677: Stop of 'ora.mdnsd' on 'linuxrac2' succeeded
CRS-2677: Stop of 'ora.gipcd' on 'linuxrac2' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'linuxrac2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
Successfully unlock /u01/11g_grid/grid
[root@linuxrac2 bin]# 

- Make sure "opatch" executable in path

[oracle@linuxrac2 grid]$ export PATH=/u01/11g_grid/grid/OPatch:$PATH
[oracle@linuxrac2 grid]$ which opatch
/u01/11g_grid/grid/OPatch/opatch
[oracle@linuxrac2 grid]$ echo $ORACLE_HOME
/u01/11g_grid/grid
[oracle@linuxrac2 grid]$ 

- Apply patch on GI Home:

[oracle@linuxrac2 9706490]$ opatch napply -local -oh /u01/11g_grid/grid -id 9706490

- Run prepatch script for RDBMS home:

[oracle@linuxrac2 scripts]$ pwd
/u01/sw_home/Db_Patches/9706490/custom/scripts
[oracle@linuxrac2 scripts]$ sh prepatch.sh -dbhome /u01/11g_db/product/11.2.0/dbhome_1
prepatch.sh completed successfully.
[oracle@linuxrac2 scripts]$ 


- Apply patch on RDBMS Home

[oracle@linuxrac2 9706490]$ opatch napply custom/server/ -local -oh /u01/11g_db/product/11.2.0/dbhome_1 -id 9706490
Invoking OPatch 11.1.0.6.6

Oracle Interim Patch Installer version 11.1.0.6.6
Copyright (c) 2009, Oracle Corporation.  All rights reserved.

UTIL session

Oracle Home       : /u01/11g_db/product/11.2.0/dbhome_1
Central Inventory : /u01/ora/oraInventory
   from           : /etc/oraInst.loc
OPatch version    : 11.1.0.6.6
OUI version       : 11.2.0.1.0
OUI location      : /u01/11g_db/product/11.2.0/dbhome_1/oui
Log file location : /u01/11g_db/product/11.2.0/dbhome_1/cfgtoollogs/opatch/opatch2014-03-19_10-06-40AM.log

Patch history file: /u01/11g_db/product/11.2.0/dbhome_1/cfgtoollogs/opatch/opatch_history.txt

Invoking utility "napply"
Checking conflict among patches...
Checking if Oracle Home has components required by patches...
Checking conflicts against Oracle Home...
OPatch continues with these patches:   9706490  

Do you want to proceed? [y|n]

Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/srvm/mesg/procus.msg"
Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/srvm/mesg/proczhs.msb"
Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/srvm/mesg/proczht.msb"
Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/srvm/mesg/prvfus.msg"
Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/srvm/mesg/prvpus.msg"
Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/srvm/mesg/sclcus.msg"
Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/srvm/mesg/sclsus.msg"
Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/oui/jlib/srvm.jar"
Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/oui/jlib/srvmasm.jar"
Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/oui/lib/linux/libsrvm11.so"
ApplySession adding interim patch '9706490' to inventory

Verifying the update...
Inventory check OK: Patch ID 9706490 is registered in Oracle Home inventory with proper meta-data.
Files check OK: Files from Patch ID 9706490 are present in Oracle Home.
Running make for target install_srvm
Running make for target install

The local system has been patched and can be restarted.

UtilSession: N-Apply done.

OPatch succeeded.
[oracle@linuxrac2 9706490]$ 

-  Apply postpatch script on RDBMS home

[oracle@linuxrac2 scripts]$ sh postpatch.sh -dbhome /u01/11g_db/product/11.2.0/dbhome_1
Reading /u01/11g_db/product/11.2.0/dbhome_1/install/params.ora..
Reading /u01/11g_db/product/11.2.0/dbhome_1/install/params.ora..
Parsing file /u01/11g_db/product/11.2.0/dbhome_1/bin/racgwrap
Parsing file /u01/11g_db/product/11.2.0/dbhome_1/bin/srvctl
Parsing file /u01/11g_db/product/11.2.0/dbhome_1/bin/srvconfig
Parsing file /u01/11g_db/product/11.2.0/dbhome_1/bin/cluvfy
Verifying file /u01/11g_db/product/11.2.0/dbhome_1/bin/racgwrap
Verifying file /u01/11g_db/product/11.2.0/dbhome_1/bin/srvctl
Verifying file /u01/11g_db/product/11.2.0/dbhome_1/bin/srvconfig
Verifying file /u01/11g_db/product/11.2.0/dbhome_1/bin/cluvfy
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/racgwrap
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/srvctl
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/srvconfig
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/cluvfy
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/racgmain
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/racgeut
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/diskmon.bin
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/lsnodes
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/osdbagrp
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/rawutl
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/srvm/admin/ractrans
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/srvm/admin/getcrshome
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/gnsd
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/crsdiag.pl
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libhasgen11.so
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libclsra11.so
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libdbcfg11.so
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libocr11.so
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libocrb11.so
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libocrutl11.so
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libuini11.so
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/librdjni11.so
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libgns11.so
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libgnsjni11.so
Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libagfw11.so
[oracle@linuxrac2 scripts]$ 

- run "rootcrl.pl" script:

[root@linuxrac2 bin]# /u01/11g_grid/grid/crs/install/rootcrs.pl -patch
2014-03-19 10:24:52: Parsing the host name
2014-03-19 10:24:52: Checking for super user privileges
2014-03-19 10:24:52: User has super user privileges
Using configuration parameter file: /u01/11g_grid/grid/crs/install/crsconfig_params
ADVM/ACFS is not supported on oraclelinux-release-5-7.0.2
CRS-4123: Oracle High Availability Services has been started.
[root@linuxrac2 bin]# 

- Verify GI services

[root@linuxrac2 bin]# ./crs_stat -t -v
Name           Type           R/RA   F/FT   Target    State     Host        
----------------------------------------------------------------------
ora.DBDATA.dg  ora....up.type 0/5    0/     ONLINE    ONLINE    linuxrac1   
ora.GDATA.dg   ora....up.type 0/5    0/     ONLINE    ONLINE    linuxrac1   
ora....ER.lsnr ora....er.type 0/5    0/     ONLINE    ONLINE    linuxrac1   
ora....N1.lsnr ora....er.type 0/5    0/0    ONLINE    ONLINE    linuxrac1   
ora.asm        ora.asm.type   0/5    0/     ONLINE    ONLINE    linuxrac1   
ora.eons       ora.eons.type  0/3    0/     ONLINE    ONLINE    linuxrac1   
ora.gsd        ora.gsd.type   0/5    0/     OFFLINE   OFFLINE               
ora.irprod.db  ora....se.type 0/2    0/1    ONLINE    ONLINE    linuxrac1   
ora....SM1.asm application    0/5    0/0    ONLINE    ONLINE    linuxrac1   
ora....C1.lsnr application    0/5    0/0    ONLINE    ONLINE    linuxrac1   
ora....ac1.gsd application    0/5    0/0    OFFLINE   OFFLINE               
ora....ac1.ons application    0/3    0/0    ONLINE    ONLINE    linuxrac1   
ora....ac1.vip ora....t1.type 0/0    0/0    ONLINE    ONLINE    linuxrac1   
ora....SM2.asm application    0/5    0/0    ONLINE    ONLINE    linuxrac2   
ora....C2.lsnr application    0/5    0/0    ONLINE    ONLINE    linuxrac2   
ora....ac2.gsd application    0/5    0/0    OFFLINE   OFFLINE               
ora....ac2.ons application    0/3    0/0    ONLINE    ONLINE    linuxrac2   
ora....ac2.vip ora....t1.type 0/0    0/0    ONLINE    ONLINE    linuxrac2   
ora....network ora....rk.type 0/5    0/     ONLINE    ONLINE    linuxrac1   
ora.oc4j       ora.oc4j.type  0/5    0/0    OFFLINE   OFFLINE               
ora.ons        ora.ons.type   0/3    0/     ONLINE    ONLINE    linuxrac1   
ora.scan1.vip  ora....ip.type 0/0    0/0    ONLINE    ONLINE    linuxrac1   
[root@linuxrac2 bin]# 

- Verify patch

[oracle@linuxrac2 9706490]$ opatch lsinventory -detail -oh  /u01/11g_grid/grid/

[oracle@linuxrac2 scripts]$ opatch lsinventory -detail -oh /u01/11g_db/product/11.2.0/dbhome_1

- Add following entry in /etc/resolv.conf 

options timeout:1 attempts:1 rotate

- Similarly perform all actions for applying patch on linuxrac1 (node1) 

- After successfull completion of patch on both nodes, please runcluvfy for installation verification:

[oracle@linuxrac1 ~]$ /media/grid/runcluvfy.sh stage -pre crsinst -upgrade -n linuxrac1,linuxrac2 -rolling -src_crshome /u01/11g_grid/grid -dest_crshome /u01/11g_grid/11204_grid -dest_version 11.2.0.4.0 -fixup -fixupdir /home/oracle/fixup

Performing pre-checks for cluster services setup 

Checking node reachability...
Node reachability check passed from node "linuxrac1"


Checking user equivalence...
User equivalence check passed for user "oracle"
l
Checking CRS user consistency
CRS user consistency check successful

Checking node connectivity...

Checking hosts config file...

Verification of the hosts config file successful

Check: Node connectivity for interface "eth0"
Node connectivity passed for interface "eth0"
TCP connectivity check passed for subnet "192.168.1.0"


Check: Node connectivity for interface "eth1"
Node connectivity passed for interface "eth1"
TCP connectivity check passed for subnet "10.0.0.0"

Checking subnet mask consistency...
Subnet mask consistency check passed for subnet "192.168.1.0".
Subnet mask consistency check passed for subnet "10.0.0.0".
Subnet mask consistency check passed.

Node connectivity check passed

Checking multicast communication...

Checking subnet "192.168.1.0" for multicast communication with multicast group "230.0.1.0"...
Check of subnet "192.168.1.0" for multicast communication with multicast group "230.0.1.0" passed.

Checking subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0"...
Check of subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0" passed.

Check of multicast communication passed.

Checking OCR integrity...

OCR integrity check passed

Checking ASMLib configuration.
Check for ASMLib configuration passed.
Total memory check passed
Available memory check passed
Swap space check passed
Free disk space check passed for "linuxrac2:/u01/11g_grid/11204_grid"
Free disk space check passed for "linuxrac1:/u01/11g_grid/11204_grid"
Free disk space check passed for "linuxrac2:/tmp"
Free disk space check passed for "linuxrac1:/tmp"
Check for multiple users with UID value 500 passed 
User existence check passed for "oracle"
Group existence check passed for "dbarac"
Membership check for user "oracle" in group "dbarac" [as Primary] passed
Run level check passed
Hard limits check passed for "maximum open file descriptors"
Soft limits check passed for "maximum open file descriptors"
Hard limits check passed for "maximum user processes"
Soft limits check passed for "maximum user processes"
Check for Oracle patch "9413827 or 9706490" in home "/u01/11g_grid/grid" passed
There are no oracle patches required for home "/u01/11g_grid/11204_grid".
System architecture check passed
Kernel version check passed
Kernel parameter check passed for "semmsl"
Kernel parameter check passed for "semmns"
Kernel parameter check passed for "semopm"
Kernel parameter check passed for "semmni"
Kernel parameter check passed for "shmmax"
Kernel parameter check passed for "shmmni"
Kernel parameter check passed for "shmall"
Kernel parameter check passed for "file-max"
Kernel parameter check passed for "ip_local_port_range"
Kernel parameter check passed for "rmem_default"
Kernel parameter check passed for "rmem_max"
Kernel parameter check passed for "wmem_default"
Kernel parameter check passed for "wmem_max"
Kernel parameter check passed for "aio-max-nr"
Package existence check passed for "make"
Package existence check passed for "binutils"
Package existence check passed for "gcc"
Package existence check passed for "gcc-c++"
Package existence check passed for "libgomp"
Package existence check passed for "libaio"
Package existence check passed for "glibc"
Package existence check passed for "compat-libstdc++-33"
Package existence check passed for "elfutils-libelf"
Package existence check passed for "elfutils-libelf-devel"
Package existence check passed for "glibc-common"
Package existence check passed for "glibc-devel"
Package existence check passed for "glibc-headers"
Package existence check passed for "libaio-devel"
Package existence check passed for "libgcc"
Package existence check passed for "libstdc++"
Package existence check passed for "libstdc++-devel"
Package existence check passed for "sysstat"
Package existence check passed for "ksh"
Check for multiple users with UID value 0 passed 
Current group ID check passed

Starting check for consistency of primary group of root user

Check for consistency of root user's primary group passed
Package existence check passed for "cvuqdisk"

Starting Clock synchronization checks using Network Time Protocol(NTP)...

NTP Configuration file check started...
No NTP Daemons or Services were found to be running

Clock synchronization check using Network Time Protocol(NTP) passed

Core file name pattern consistency check passed.

User "oracle" is not part of "root" group. Check passed
Default user file creation mask check passed
Checking consistency of file "/etc/resolv.conf" across nodes

File "/etc/resolv.conf" does not have both domain and search entries defined
domain entry in file "/etc/resolv.conf" is consistent across nodes
search entry in file "/etc/resolv.conf" is consistent across nodes
PRVF-5636 : The DNS response time for an unreachable node exceeded "15000" ms on following nodes: linuxrac2,linuxrac1

File "/etc/resolv.conf" is not consistent across nodes


UDev attributes check for OCR locations started...
UDev attributes check passed for OCR locations 


UDev attributes check for Voting Disk locations started...
UDev attributes check passed for Voting Disk locations 

Time zone consistency check passed
Checking VIP configuration.
Checking VIP Subnet configuration.
Check for VIP Subnet configuration passed.
Checking VIP reachability
Check for VIP reachability passed.

Checking Oracle Cluster Voting Disk configuration...

ASM Running check passed. ASM is running on all specified nodes

Oracle Cluster Voting Disk configuration check passed

Clusterware version consistency passed

Pre-check for cluster services setup was unsuccessful on all the nodes. 
[oracle@linuxrac1 ~]$ /media/grid/runcluvfy.sh stage -pre crsinst -upgrade -n linuxrac1,linuxrac2 -rolling -src_crshome /u01/11g_grid/grid -dest_crshome /u01/11g_grid/11204_grid -dest_version 11.2.0.4.0 -fixup -fixupdir /home/oracle/fixup

Performing pre-checks for cluster services setup 

Checking node reachability...
Node reachability check passed from node "linuxrac1"


Checking user equivalence...
User equivalence check passed for user "oracle"

Checking CRS user consistency
CRS user consistency check successful

Checking node connectivity...

Checking hosts config file...

Verification of the hosts config file successful

Check: Node connectivity for interface "eth0"
Node connectivity passed for interface "eth0"
TCP connectivity check passed for subnet "192.168.1.0"


Check: Node connectivity for interface "eth1"
Node connectivity passed for interface "eth1"
TCP connectivity check passed for subnet "10.0.0.0"

Checking subnet mask consistency...
Subnet mask consistency check passed for subnet "192.168.1.0".
Subnet mask consistency check passed for subnet "10.0.0.0".
Subnet mask consistency check passed.

Node connectivity check passed

Checking multicast communication...

Checking subnet "192.168.1.0" for multicast communication with multicast group "230.0.1.0"...
Check of subnet "192.168.1.0" for multicast communication with multicast group "230.0.1.0" passed.

Checking subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0"...
Check of subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0" passed.

Check of multicast communication passed.

Checking OCR integrity...

OCR integrity check passed

Checking ASMLib configuration.
Check for ASMLib configuration passed.
Total memory check passed
Available memory check passed
Swap space check passed
Free disk space check passed for "linuxrac2:/u01/11g_grid/11204_grid"
Free disk space check passed for "linuxrac1:/u01/11g_grid/11204_grid"
Free disk space check passed for "linuxrac2:/tmp"
Free disk space check passed for "linuxrac1:/tmp"
Check for multiple users with UID value 500 passed 
User existence check passed for "oracle"
Group existence check passed for "dbarac"
Membership check for user "oracle" in group "dbarac" [as Primary] passed
Run level check passed
Hard limits check passed for "maximum open file descriptors"
Soft limits check passed for "maximum open file descriptors"
Hard limits check passed for "maximum user processes"
Soft limits check passed for "maximum user processes"
Check for Oracle patch "9413827 or 9706490" in home "/u01/11g_grid/grid" passed
There are no oracle patches required for home "/u01/11g_grid/11204_grid".
System architecture check passed
Kernel version check passed
Kernel parameter check passed for "semmsl"
Kernel parameter check passed for "semmns"
Kernel parameter check passed for "semopm"
Kernel parameter check passed for "semmni"
Kernel parameter check passed for "shmmax"
Kernel parameter check passed for "shmmni"
Kernel parameter check passed for "shmall"
Kernel parameter check passed for "file-max"
Kernel parameter check passed for "ip_local_port_range"
Kernel parameter check passed for "rmem_default"
Kernel parameter check passed for "rmem_max"
Kernel parameter check passed for "wmem_default"
Kernel parameter check passed for "wmem_max"
Kernel parameter check passed for "aio-max-nr"
Package existence check passed for "make"
Package existence check passed for "binutils"
Package existence check passed for "gcc"
Package existence check passed for "gcc-c++"
Package existence check passed for "libgomp"
Package existence check passed for "libaio"
Package existence check passed for "glibc"
Package existence check passed for "compat-libstdc++-33"
Package existence check passed for "elfutils-libelf"
Package existence check passed for "elfutils-libelf-devel"
Package existence check passed for "glibc-common"
Package existence check passed for "glibc-devel"
Package existence check passed for "glibc-headers"
Package existence check passed for "libaio-devel"
Package existence check passed for "libgcc"
Package existence check passed for "libstdc++"
Package existence check passed for "libstdc++-devel"
Package existence check passed for "sysstat"
Package existence check passed for "ksh"
Check for multiple users with UID value 0 passed 
Current group ID check passed

Starting check for consistency of primary group of root user

Check for consistency of root user's primary group passed
Package existence check passed for "cvuqdisk"

Starting Clock synchronization checks using Network Time Protocol(NTP)...

NTP Configuration file check started...
No NTP Daemons or Services were found to be running

Clock synchronization check using Network Time Protocol(NTP) passed

Core file name pattern consistency check passed.

User "oracle" is not part of "root" group. Check passed
Default user file creation mask check passed
Checking consistency of file "/etc/resolv.conf" across nodes

File "/etc/resolv.conf" does not have both domain and search entries defined
domain entry in file "/etc/resolv.conf" is consistent across nodes
search entry in file "/etc/resolv.conf" is consistent across nodes
The DNS response time for an unreachable node is within acceptable limit on all nodes

File "/etc/resolv.conf" is consistent across nodes


UDev attributes check for OCR locations started...
UDev attributes check passed for OCR locations 


UDev attributes check for Voting Disk locations started...
UDev attributes check passed for Voting Disk locations 

Time zone consistency check passed
Checking VIP configuration.
Checking VIP Subnet configuration.
Check for VIP Subnet configuration passed.
Checking VIP reachability
Check for VIP reachability passed.

Checking Oracle Cluster Voting Disk configuration...

ASM Running check passed. ASM is running on all specified nodes

Oracle Cluster Voting Disk configuration check passed

Clusterware version consistency passed

Pre-check for cluster services setup was successful. 
[oracle@linuxrac1 ~]$ 


- Make sure full database backup has been performed and database in not running from both the nodes.

5) Install and upgrade Grid Infrastructure:

- set the proper display variable and execute "runInstaller" from grid software owner

- Follow screen shots




- Select "Upgrade"




- Verify SSH connectivity



- Provide location for 11.2.0.4 GI home location








- run "upgraderoot.sh" on both nodes as listed

root@linuxrac1 ~]# /u01/11g_grid/11204_grid/rootupgrade.sh
Performing root user operation for Oracle 11g 

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/11g_grid/11204_grid

Enter the full pathname of the local bin directory: [/usr/local/bin]: 
The contents of "dbhome" have not changed. No need to overwrite.
The file "oraenv" already exists in /usr/local/bin.  Overwrite it? (y/n) 
[n]: y
   Copying oraenv to /usr/local/bin ...
The file "coraenv" already exists in /usr/local/bin.  Overwrite it? (y/n) 
[n]: y
   Copying coraenv to /usr/local/bin ...

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/11g_grid/11204_grid/crs/install/crsconfig_params
Creating trace directory
Installing Trace File Analyzer

ASM upgrade has started on first node.


CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'linuxrac1'
CRS-2673: Attempting to stop 'ora.crsd' on 'linuxrac1'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'linuxrac1'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'linuxrac1'
CRS-2673: Attempting to stop 'ora.GDATA.dg' on 'linuxrac1'
CRS-2673: Attempting to stop 'ora.DBDATA.dg' on 'linuxrac1'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'linuxrac1' succeeded
CRS-2673: Attempting to stop 'ora.linuxrac1.vip' on 'linuxrac1'
CRS-2677: Stop of 'ora.linuxrac1.vip' on 'linuxrac1' succeeded
CRS-2672: Attempting to start 'ora.linuxrac1.vip' on 'linuxrac2'
CRS-2676: Start of 'ora.linuxrac1.vip' on 'linuxrac2' succeeded
CRS-2677: Stop of 'ora.GDATA.dg' on 'linuxrac1' succeeded
CRS-2677: Stop of 'ora.DBDATA.dg' on 'linuxrac1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'linuxrac1'
CRS-2677: Stop of 'ora.asm' on 'linuxrac1' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'linuxrac1'
CRS-2673: Attempting to stop 'ora.eons' on 'linuxrac1'
CRS-2677: Stop of 'ora.ons' on 'linuxrac1' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'linuxrac1'
CRS-2677: Stop of 'ora.net1.network' on 'linuxrac1' succeeded
CRS-2677: Stop of 'ora.eons' on 'linuxrac1' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'linuxrac1' has completed
CRS-2677: Stop of 'ora.crsd' on 'linuxrac1' succeeded
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'linuxrac1'
CRS-2673: Attempting to stop 'ora.ctssd' on 'linuxrac1'
CRS-2673: Attempting to stop 'ora.evmd' on 'linuxrac1'
CRS-2673: Attempting to stop 'ora.asm' on 'linuxrac1'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'linuxrac1'
CRS-2677: Stop of 'ora.cssdmonitor' on 'linuxrac1' succeeded
CRS-2677: Stop of 'ora.evmd' on 'linuxrac1' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'linuxrac1' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'linuxrac1' succeeded
CRS-2677: Stop of 'ora.asm' on 'linuxrac1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'linuxrac1'
CRS-2677: Stop of 'ora.cssd' on 'linuxrac1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'linuxrac1'
CRS-2673: Attempting to stop 'ora.diskmon' on 'linuxrac1'
CRS-2677: Stop of 'ora.gpnpd' on 'linuxrac1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'linuxrac1'
CRS-2677: Stop of 'ora.diskmon' on 'linuxrac1' succeeded
CRS-2677: Stop of 'ora.gipcd' on 'linuxrac1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'linuxrac1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
OLR initialization - successful
Replacing Clusterware entries in inittab
clscfg: EXISTING configuration version 5 detected.
clscfg: version 5 is 11g Release 2.
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
[root@linuxrac1 ~]# 


root@linuxrac2 ~]# /u01/11g_grid/11204_grid/rootupgrade.sh
Performing root user operation for Oracle 11g 

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/11g_grid/11204_grid

Enter the full pathname of the local bin directory: [/usr/local/bin]: 
The contents of "dbhome" have not changed. No need to overwrite.
The file "oraenv" already exists in /usr/local/bin.  Overwrite it? (y/n) 
[n]: y
   Copying oraenv to /usr/local/bin ...
The file "coraenv" already exists in /usr/local/bin.  Overwrite it? (y/n) 
[n]: y
   Copying coraenv to /usr/local/bin ...

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/11g_grid/11204_grid/crs/install/crsconfig_params
Creating trace directory
Installing Trace File Analyzer
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'linuxrac2'
CRS-2673: Attempting to stop 'ora.crsd' on 'linuxrac2'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'linuxrac2'
CRS-2673: Attempting to stop 'ora.GDATA.dg' on 'linuxrac2'
CRS-2673: Attempting to stop 'ora.DBDATA.dg' on 'linuxrac2'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'linuxrac2'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'linuxrac2'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'linuxrac2' succeeded
CRS-2673: Attempting to stop 'ora.linuxrac2.vip' on 'linuxrac2'
CRS-2677: Stop of 'ora.linuxrac2.vip' on 'linuxrac2' succeeded
CRS-2672: Attempting to start 'ora.linuxrac2.vip' on 'linuxrac1'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'linuxrac2' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'linuxrac2'
CRS-2677: Stop of 'ora.scan1.vip' on 'linuxrac2' succeeded
CRS-2672: Attempting to start 'ora.scan1.vip' on 'linuxrac1'
CRS-2676: Start of 'ora.linuxrac2.vip' on 'linuxrac1' succeeded
CRS-2676: Start of 'ora.scan1.vip' on 'linuxrac1' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'linuxrac1'
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'linuxrac1' succeeded
CRS-2677: Stop of 'ora.GDATA.dg' on 'linuxrac2' succeeded
CRS-2677: Stop of 'ora.DBDATA.dg' on 'linuxrac2' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'linuxrac2'
CRS-2677: Stop of 'ora.asm' on 'linuxrac2' succeeded
CRS-2673: Attempting to stop 'ora.eons' on 'linuxrac2'
CRS-2673: Attempting to stop 'ora.ons' on 'linuxrac2'
CRS-2677: Stop of 'ora.ons' on 'linuxrac2' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'linuxrac2'
CRS-2677: Stop of 'ora.net1.network' on 'linuxrac2' succeeded
CRS-2677: Stop of 'ora.eons' on 'linuxrac2' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'linuxrac2' has completed
CRS-2677: Stop of 'ora.crsd' on 'linuxrac2' succeeded
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'linuxrac2'
CRS-2673: Attempting to stop 'ora.ctssd' on 'linuxrac2'
CRS-2673: Attempting to stop 'ora.evmd' on 'linuxrac2'
CRS-2673: Attempting to stop 'ora.asm' on 'linuxrac2'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'linuxrac2'
CRS-2677: Stop of 'ora.cssdmonitor' on 'linuxrac2' succeeded
CRS-2677: Stop of 'ora.evmd' on 'linuxrac2' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'linuxrac2' succeeded
CRS-2677: Stop of 'ora.asm' on 'linuxrac2' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'linuxrac2' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'linuxrac2'
CRS-2677: Stop of 'ora.cssd' on 'linuxrac2' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'linuxrac2'
CRS-2673: Attempting to stop 'ora.diskmon' on 'linuxrac2'
CRS-2677: Stop of 'ora.gpnpd' on 'linuxrac2' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'linuxrac2'
CRS-2677: Stop of 'ora.gipcd' on 'linuxrac2' succeeded
CRS-2677: Stop of 'ora.diskmon' on 'linuxrac2' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'linuxrac2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
OLR initialization - successful
Replacing Clusterware entries in inittab
clscfg: EXISTING configuration version 5 detected.
clscfg: version 5 is 11g Release 2.
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Started to upgrade the Oracle Clusterware. This operation may take a few minutes.
Started to upgrade the CSS.
Started to upgrade the CRS.
The CRS was successfully upgraded.
Successfully upgraded the Oracle Clusterware.
Oracle Clusterware operating version was successfully set to 11.2.0.4.0

ASM upgrade has finished on last node.


Configure Oracle Grid Infrastructure for a Cluster ... succeeded
[root@linuxrac2 ~]#




6) Verify Upgrade:



- Verify services:


- startup database and make sure both Instances are working fine without any Issues.



11.2.0.1 Grid Infrastructure has been successfully upgraded to 11.2.0.4.

In next article i will demonstrate how to upgrade an Oracle RAC database from 11.2.0.1 to 11.2.0.4

Thanks for reading.

regards,
X A H E E R

Comments

Abdul Mannan said…
Very well executed!!


Abdul Mannan said…
This comment has been removed by the author.
when we do a rolling upgrade of grid infrastructure,can we have to select only one node without impacting the operations on other node.

because when you execute a shell script at the end. your rac services get restarted.

Please suggest the best option.

Popular posts from this blog

Disable Firewall on Oracle Linux 8

In this blogpost we will see how we can stop/disable the firewall on Oracle Linux 8, the firewall command is same in both linux 7 an linux 8. The below listed is the procedure for stopping and disabling the  firewall on Oracle Linux 8. - Here we can see the firewall deamon in active state - Here when we stop the firewall in previous command, now the firewall daemon is dead - For permanent disabling the firewall on server, we can use "disable" option The following commands will be helpful: #systemctl status firewalld #systemctl stop firewalld #systemctl disable firewalld #systemctl enable firewalld #systemctl start firewalld Hope it helps !! Thanks for reading :) regards, X A H E E R

Enable Desktop on Oracle Solaris 11.4

Oracle Solaris 11 installation has multiple options to choose for installation of an Operating Environment, but mostly Oracle Solaris text install media is used and this installation media doesn't offer the GUI Desktop Environment by default after the installation. This blog post will explain how we can enable the desktop for Oracle Solaris 11.4 operating system, after the completion of installation. We have to install "solaris-desktop" package and reboot the machine and GUI desktop will be enabled for the Operating System. In this blog post my virtual machine is connected to the internet and hence I am able to use available pupblic repository for package installation, if in case internet is not available for the server/machine then we have configure the local/Server  repository for the installation. Follow the below steps for desktop package installation: After installation of dekstop package we are now able to login with GUI desktop environment

Oracle Applications R12.2 Installation on Linux using virtual box

Oracle E-Business suite R12.2 is released a month ago. I received too many requests from my followers to demonstrate this Installation. This is a step by step detailed article for Installing Oracle E-business suite R12,2 on Oracle Enterprise Linux 5.7 (64-bit). I've used oracle virtual box for Installing Linux. Note:  This Installation is demonstrated on virtual machine with 8GB RAM and Installation completed in 7 hours. I am not sure whether this Installation will work with less than this .  My Virtual Machine configuration: CPU    = 1 core RAM   = 8 GB HDD   =  380 GB Software's Required: 1) Oracle Virtual Box 2) Oracle Enterprise Linux 5.7 (64-bit) 3) Oracle E-Business suite R12.2 for LinuxX86-64-bit Installation Steps: Hardware requirements Create virtual machine and Install operating system Operating System pre-requisites Prepare Stage Area Install Oracle EBS R12.2 software Verify Installation 1. Hardware Requirement