In this article I will demonstrate all steps required for upgrading Oracle Grid Infrastructure 11.2.0.1 to 11.2.0.4. Current GI is running on 2 nodes cluster on Oracle enterprise Linux 5.7 with OCR and Voting Disk on ASM Disk group.
Node1 => linuxrac1
Node2 => linuxrac2
High Level steps for upgrading Grid Infrastructure:
1) Perform backup of OCR, voting disk and Database
2) Create new directory structure on both RAC nodes where 11.2.0.4 SW will be Installed and grant appropriate directory privileges
3) Run "runcluvfy.sh" to verify errors
4) Apply required patches
5) Install and upgrade GI from 11.2.0.1 to 11.2.0.4
6) Verify upgrade
1) Perform local backup of OCR:
2) Create new directories for 11.2.0.4 GI Home:
3) Run "runcluvfy.sh" to verify errors
- cluvfy failed on Oracle Patch and DNS resolv.conf file
4) Apply required patches
I have applied these patches in rolling method:
- shutdown database running on node2
- Apply patch 9706450 on GI home then on RDBMS home
- startup database on node2
- Verify DB and GI status
- Apply patch on node1 with same sequence
- Verify all services are running without any errors
- All services are up and running on both nodes
- Similarly perform all actions for applying patch on linuxrac1 (node1)
- After successfull completion of patch on both nodes, please runcluvfy for installation verification:
- Make sure full database backup has been performed and database in not running from both the nodes.
5) Install and upgrade Grid Infrastructure:
- set the proper display variable and execute "runInstaller" from grid software owner
- Follow screen shots
- Select "Upgrade"
- Verify SSH connectivity
- Provide location for 11.2.0.4 GI home location
- run "upgraderoot.sh" on both nodes as listed
6) Verify Upgrade:
- Verify services:
- startup database and make sure both Instances are working fine without any Issues.
11.2.0.1 Grid Infrastructure has been successfully upgraded to 11.2.0.4.
In next article i will demonstrate how to upgrade an Oracle RAC database from 11.2.0.1 to 11.2.0.4
Thanks for reading.
regards,
X A H E E R
Node1 => linuxrac1
Node2 => linuxrac2
High Level steps for upgrading Grid Infrastructure:
1) Perform backup of OCR, voting disk and Database
2) Create new directory structure on both RAC nodes where 11.2.0.4 SW will be Installed and grant appropriate directory privileges
3) Run "runcluvfy.sh" to verify errors
4) Apply required patches
5) Install and upgrade GI from 11.2.0.1 to 11.2.0.4
6) Verify upgrade
1) Perform local backup of OCR:
[root@linuxrac1 bin]# ./ocrconfig -manualbackup
linuxrac2 2014/03/18 10:58:55 /u01/11g_grid/grid/cdata/scan-rac/backup_20140318_105855.ocr
[root@linuxrac1 bin]# ./ocrconfig -showbackup
linuxrac1 2014/02/19 15:28:32 /u01/11g_grid/grid/cdata/scan-rac/backup00.ocr
linuxrac1 2014/02/19 15:28:32 /u01/11g_grid/grid/cdata/scan-rac/day.ocr
linuxrac1 2014/02/19 15:28:32 /u01/11g_grid/grid/cdata/scan-rac/week.ocr
linuxrac2 2014/03/18 10:58:55 /u01/11g_grid/grid/cdata/scan-rac/backup_20140318_105855.ocr
[root@linuxrac1 bin]#
2) Create new directories for 11.2.0.4 GI Home:
[root@linuxrac1 sw_home]# mkdir -p /u01/11g_grid/11204_grid [root@linuxrac1 sw_home]# chown -R oracle:dbarac /u01/11g_grid/11204_grid/ [root@linuxrac1 sw_home]#
3) Run "runcluvfy.sh" to verify errors
[oracle@linuxrac1 ~]$ /media/grid/runcluvfy.sh stage -pre crsinst -upgrade -n linuxrac1,linuxrac2 -rolling -src_crshome /u01/11g_grid/grid -dest_crshome /u01/11g_grid/11204_grid -dest_version 11.2.0.4.0 -fixup -fixupdir /home/oracle/fixup Performing pre-checks for cluster services setup Checking node reachability... Node reachability check passed from node "linuxrac1" Checking user equivalence... User equivalence check passed for user "oracle" Checking CRS user consistency CRS user consistency check successful Checking node connectivity... Checking hosts config file... Verification of the hosts config file successful Check: Node connectivity for interface "eth0" Node connectivity passed for interface "eth0" TCP connectivity check passed for subnet "192.168.1.0" Check: Node connectivity for interface "eth1" Node connectivity passed for interface "eth1" TCP connectivity check passed for subnet "10.0.0.0" Checking subnet mask consistency... Subnet mask consistency check passed for subnet "192.168.1.0". Subnet mask consistency check passed for subnet "10.0.0.0". Subnet mask consistency check passed. Node connectivity check passed Checking multicast communication... Checking subnet "192.168.1.0" for multicast communication with multicast group "230.0.1.0"... Check of subnet "192.168.1.0" for multicast communication with multicast group "230.0.1.0" passed. Checking subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0"... Check of subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0" passed. Check of multicast communication passed. Checking OCR integrity... OCR integrity check passed Checking ASMLib configuration. Check for ASMLib configuration passed. Total memory check passed Available memory check passed Swap space check passed Free disk space check passed for "linuxrac2:/u01/11g_grid/11204_grid" Free disk space check passed for "linuxrac1:/u01/11g_grid/11204_grid" Free disk space check passed for "linuxrac2:/tmp" Free disk space check passed for "linuxrac1:/tmp" Check for multiple users with UID value 500 passed User existence check passed for "oracle" Group existence check passed for "dbarac" Membership check for user "oracle" in group "dbarac" [as Primary] passed Run level check passed Hard limits check passed for "maximum open file descriptors" Soft limits check passed for "maximum open file descriptors" Hard limits check passed for "maximum user processes" Soft limits check passed for "maximum user processes" Check for Oracle patch "9413827 or 9706490" in home "/u01/11g_grid/grid" failed Check failed on nodes: linuxrac2,linuxrac1 There are no oracle patches required for home "/u01/11g_grid/11204_grid". System architecture check passed Kernel version check passed Kernel parameter check passed for "semmsl" Kernel parameter check passed for "semmns" Kernel parameter check passed for "semopm" Kernel parameter check passed for "semmni" Kernel parameter check passed for "shmmax" Kernel parameter check passed for "shmmni" Kernel parameter check passed for "shmall" Kernel parameter check passed for "file-max" Kernel parameter check passed for "ip_local_port_range" Kernel parameter check passed for "rmem_default" Kernel parameter check passed for "rmem_max" Kernel parameter check passed for "wmem_default" Kernel parameter check passed for "wmem_max" Kernel parameter check passed for "aio-max-nr" Package existence check passed for "make" Package existence check passed for "binutils" Package existence check passed for "gcc" Package existence check passed for "gcc-c++" Package existence check passed for "libgomp" Package existence check passed for "libaio" Package existence check passed for "glibc" Package existence check passed for "compat-libstdc++-33" Package existence check passed for "elfutils-libelf" Package existence check passed for "elfutils-libelf-devel" Package existence check passed for "glibc-common" Package existence check passed for "glibc-devel" Package existence check passed for "glibc-headers" Package existence check passed for "libaio-devel" Package existence check passed for "libgcc" Package existence check passed for "libstdc++" Package existence check passed for "libstdc++-devel" Package existence check passed for "sysstat" Package existence check passed for "ksh" Check for multiple users with UID value 0 passed Current group ID check passed Starting check for consistency of primary group of root user Check for consistency of root user's primary group passed Package existence check passed for "cvuqdisk" Starting Clock synchronization checks using Network Time Protocol(NTP)... NTP Configuration file check started... No NTP Daemons or Services were found to be running Clock synchronization check using Network Time Protocol(NTP) passed Core file name pattern consistency check passed. User "oracle" is not part of "root" group. Check passed Default user file creation mask check passed Checking consistency of file "/etc/resolv.conf" across nodes File "/etc/resolv.conf" does not have both domain and search entries defined domain entry in file "/etc/resolv.conf" is consistent across nodes search entry in file "/etc/resolv.conf" is consistent across nodes PRVF-5636 : The DNS response time for an unreachable node exceeded "15000" ms on following nodes: linuxrac1 File "/etc/resolv.conf" is not consistent across nodes UDev attributes check for OCR locations started... UDev attributes check passed for OCR locations UDev attributes check for Voting Disk locations started... UDev attributes check passed for Voting Disk locations Time zone consistency check passed Checking VIP configuration. Checking VIP Subnet configuration. Check for VIP Subnet configuration passed. Checking VIP reachability Check for VIP reachability passed. Checking Oracle Cluster Voting Disk configuration... ASM Running check passed. ASM is running on all specified nodes Oracle Cluster Voting Disk configuration check passed Clusterware version consistency passed Pre-check for cluster services setup was unsuccessful on all the nodes. [oracle@linuxrac1 ~]$
- cluvfy failed on Oracle Patch and DNS resolv.conf file
4) Apply required patches
I have applied these patches in rolling method:
- shutdown database running on node2
- Apply patch 9706450 on GI home then on RDBMS home
- startup database on node2
- Verify DB and GI status
- Apply patch on node1 with same sequence
- Verify all services are running without any errors
- All services are up and running on both nodes
[root@linuxrac2 bin]# ./crs_stat -t -v Name Type R/RA F/FT Target State Host ---------------------------------------------------------------------- ora.DBDATA.dg ora....up.type 0/5 0/ ONLINE ONLINE linuxrac1 ora.GDATA.dg ora....up.type 0/5 0/ ONLINE ONLINE linuxrac1 ora....ER.lsnr ora....er.type 0/5 0/ ONLINE ONLINE linuxrac1 ora....N1.lsnr ora....er.type 0/5 0/0 ONLINE ONLINE linuxrac1 ora.asm ora.asm.type 0/5 0/ ONLINE ONLINE linuxrac1 ora.eons ora.eons.type 0/3 0/ ONLINE ONLINE linuxrac1 ora.gsd ora.gsd.type 0/5 0/ OFFLINE OFFLINE ora.irprod.db ora....se.type 0/2 0/1 ONLINE ONLINE linuxrac1 ora....SM1.asm application 0/5 0/0 ONLINE ONLINE linuxrac1 ora....C1.lsnr application 0/5 0/0 ONLINE ONLINE linuxrac1 ora....ac1.gsd application 0/5 0/0 OFFLINE OFFLINE ora....ac1.ons application 0/3 0/0 ONLINE ONLINE linuxrac1 ora....ac1.vip ora....t1.type 0/0 0/0 ONLINE ONLINE linuxrac1 ora....SM2.asm application 0/5 0/0 ONLINE ONLINE linuxrac2 ora....C2.lsnr application 0/5 0/0 ONLINE ONLINE linuxrac2 ora....ac2.gsd application 0/5 0/0 OFFLINE OFFLINE ora....ac2.ons application 0/3 0/0 ONLINE ONLINE linuxrac2 ora....ac2.vip ora....t1.type 0/0 0/0 ONLINE ONLINE linuxrac2 ora....network ora....rk.type 0/5 0/ ONLINE ONLINE linuxrac1 ora.oc4j ora.oc4j.type 0/5 0/0 OFFLINE OFFLINE ora.ons ora.ons.type 0/3 0/ ONLINE ONLINE linuxrac1 ora.scan1.vip ora....ip.type 0/0 0/0 ONLINE ONLINE linuxrac1 [root@linuxrac2 bin]#
- Shutdown database on node2
[oracle@linuxrac2 ~]$ srvctl status database -d IRPROD Instance IRPROD1 is running on node linuxrac1 Instance IRPROD2 is not running on node linuxrac2 [oracle@linuxrac2 ~]$
- Run "rootcrs.pl -unlock" script before applying patch
[root@linuxrac2 bin]# /u01/11g_grid/grid/crs/install/rootcrs.pl -unlock 2014-03-19 09:45:10: Parsing the host name 2014-03-19 09:45:10: Checking for super user privileges 2014-03-19 09:45:10: User has super user privileges Using configuration parameter file: /u01/11g_grid/grid/crs/install/crsconfig_params CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'linuxrac2' CRS-2673: Attempting to stop 'ora.gpnpd' on 'linuxrac2' CRS-2673: Attempting to stop 'ora.mdnsd' on 'linuxrac2' CRS-2677: Stop of 'ora.gpnpd' on 'linuxrac2' succeeded CRS-2673: Attempting to stop 'ora.gipcd' on 'linuxrac2' CRS-2677: Stop of 'ora.mdnsd' on 'linuxrac2' succeeded CRS-2677: Stop of 'ora.gipcd' on 'linuxrac2' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'linuxrac2' has completed CRS-4133: Oracle High Availability Services has been stopped. Successfully unlock /u01/11g_grid/grid [root@linuxrac2 bin]#
- Make sure "opatch" executable in path
[oracle@linuxrac2 grid]$ export PATH=/u01/11g_grid/grid/OPatch:$PATH [oracle@linuxrac2 grid]$ which opatch /u01/11g_grid/grid/OPatch/opatch [oracle@linuxrac2 grid]$ echo $ORACLE_HOME /u01/11g_grid/grid [oracle@linuxrac2 grid]$
- Apply patch on GI Home:
[oracle@linuxrac2 9706490]$ opatch napply -local -oh /u01/11g_grid/grid -id 9706490
- Run prepatch script for RDBMS home:
[oracle@linuxrac2 scripts]$ pwd /u01/sw_home/Db_Patches/9706490/custom/scripts [oracle@linuxrac2 scripts]$ sh prepatch.sh -dbhome /u01/11g_db/product/11.2.0/dbhome_1 prepatch.sh completed successfully. [oracle@linuxrac2 scripts]$
- Apply patch on RDBMS Home
[oracle@linuxrac2 9706490]$ opatch napply custom/server/ -local -oh /u01/11g_db/product/11.2.0/dbhome_1 -id 9706490 Invoking OPatch 11.1.0.6.6 Oracle Interim Patch Installer version 11.1.0.6.6 Copyright (c) 2009, Oracle Corporation. All rights reserved. UTIL session Oracle Home : /u01/11g_db/product/11.2.0/dbhome_1 Central Inventory : /u01/ora/oraInventory from : /etc/oraInst.loc OPatch version : 11.1.0.6.6 OUI version : 11.2.0.1.0 OUI location : /u01/11g_db/product/11.2.0/dbhome_1/oui Log file location : /u01/11g_db/product/11.2.0/dbhome_1/cfgtoollogs/opatch/opatch2014-03-19_10-06-40AM.log Patch history file: /u01/11g_db/product/11.2.0/dbhome_1/cfgtoollogs/opatch/opatch_history.txt Invoking utility "napply" Checking conflict among patches... Checking if Oracle Home has components required by patches... Checking conflicts against Oracle Home... OPatch continues with these patches: 9706490 Do you want to proceed? [y|n] Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/srvm/mesg/procus.msg" Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/srvm/mesg/proczhs.msb" Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/srvm/mesg/proczht.msb" Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/srvm/mesg/prvfus.msg" Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/srvm/mesg/prvpus.msg" Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/srvm/mesg/sclcus.msg" Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/srvm/mesg/sclsus.msg" Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/oui/jlib/srvm.jar" Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/oui/jlib/srvmasm.jar" Copying file to "/u01/11g_db/product/11.2.0/dbhome_1/oui/lib/linux/libsrvm11.so" ApplySession adding interim patch '9706490' to inventory Verifying the update... Inventory check OK: Patch ID 9706490 is registered in Oracle Home inventory with proper meta-data. Files check OK: Files from Patch ID 9706490 are present in Oracle Home. Running make for target install_srvm Running make for target install The local system has been patched and can be restarted. UtilSession: N-Apply done. OPatch succeeded. [oracle@linuxrac2 9706490]$
- Apply postpatch script on RDBMS home
[oracle@linuxrac2 scripts]$ sh postpatch.sh -dbhome /u01/11g_db/product/11.2.0/dbhome_1 Reading /u01/11g_db/product/11.2.0/dbhome_1/install/params.ora.. Reading /u01/11g_db/product/11.2.0/dbhome_1/install/params.ora.. Parsing file /u01/11g_db/product/11.2.0/dbhome_1/bin/racgwrap Parsing file /u01/11g_db/product/11.2.0/dbhome_1/bin/srvctl Parsing file /u01/11g_db/product/11.2.0/dbhome_1/bin/srvconfig Parsing file /u01/11g_db/product/11.2.0/dbhome_1/bin/cluvfy Verifying file /u01/11g_db/product/11.2.0/dbhome_1/bin/racgwrap Verifying file /u01/11g_db/product/11.2.0/dbhome_1/bin/srvctl Verifying file /u01/11g_db/product/11.2.0/dbhome_1/bin/srvconfig Verifying file /u01/11g_db/product/11.2.0/dbhome_1/bin/cluvfy Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/racgwrap Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/srvctl Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/srvconfig Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/cluvfy Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/racgmain Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/racgeut Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/diskmon.bin Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/lsnodes Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/osdbagrp Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/rawutl Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/srvm/admin/ractrans Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/srvm/admin/getcrshome Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/gnsd Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/bin/crsdiag.pl Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libhasgen11.so Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libclsra11.so Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libdbcfg11.so Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libocr11.so Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libocrb11.so Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libocrutl11.so Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libuini11.so Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/librdjni11.so Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libgns11.so Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libgnsjni11.so Reapplying file permissions on /u01/11g_db/product/11.2.0/dbhome_1/lib/libagfw11.so [oracle@linuxrac2 scripts]$
- run "rootcrl.pl" script:
[root@linuxrac2 bin]# /u01/11g_grid/grid/crs/install/rootcrs.pl -patch 2014-03-19 10:24:52: Parsing the host name 2014-03-19 10:24:52: Checking for super user privileges 2014-03-19 10:24:52: User has super user privileges Using configuration parameter file: /u01/11g_grid/grid/crs/install/crsconfig_params ADVM/ACFS is not supported on oraclelinux-release-5-7.0.2 CRS-4123: Oracle High Availability Services has been started. [root@linuxrac2 bin]#
- Verify GI services
[root@linuxrac2 bin]# ./crs_stat -t -v Name Type R/RA F/FT Target State Host ---------------------------------------------------------------------- ora.DBDATA.dg ora....up.type 0/5 0/ ONLINE ONLINE linuxrac1 ora.GDATA.dg ora....up.type 0/5 0/ ONLINE ONLINE linuxrac1 ora....ER.lsnr ora....er.type 0/5 0/ ONLINE ONLINE linuxrac1 ora....N1.lsnr ora....er.type 0/5 0/0 ONLINE ONLINE linuxrac1 ora.asm ora.asm.type 0/5 0/ ONLINE ONLINE linuxrac1 ora.eons ora.eons.type 0/3 0/ ONLINE ONLINE linuxrac1 ora.gsd ora.gsd.type 0/5 0/ OFFLINE OFFLINE ora.irprod.db ora....se.type 0/2 0/1 ONLINE ONLINE linuxrac1 ora....SM1.asm application 0/5 0/0 ONLINE ONLINE linuxrac1 ora....C1.lsnr application 0/5 0/0 ONLINE ONLINE linuxrac1 ora....ac1.gsd application 0/5 0/0 OFFLINE OFFLINE ora....ac1.ons application 0/3 0/0 ONLINE ONLINE linuxrac1 ora....ac1.vip ora....t1.type 0/0 0/0 ONLINE ONLINE linuxrac1 ora....SM2.asm application 0/5 0/0 ONLINE ONLINE linuxrac2 ora....C2.lsnr application 0/5 0/0 ONLINE ONLINE linuxrac2 ora....ac2.gsd application 0/5 0/0 OFFLINE OFFLINE ora....ac2.ons application 0/3 0/0 ONLINE ONLINE linuxrac2 ora....ac2.vip ora....t1.type 0/0 0/0 ONLINE ONLINE linuxrac2 ora....network ora....rk.type 0/5 0/ ONLINE ONLINE linuxrac1 ora.oc4j ora.oc4j.type 0/5 0/0 OFFLINE OFFLINE ora.ons ora.ons.type 0/3 0/ ONLINE ONLINE linuxrac1 ora.scan1.vip ora....ip.type 0/0 0/0 ONLINE ONLINE linuxrac1 [root@linuxrac2 bin]#
- Verify patch
[oracle@linuxrac2 9706490]$ opatch lsinventory -detail -oh /u01/11g_grid/grid/ [oracle@linuxrac2 scripts]$ opatch lsinventory -detail -oh /u01/11g_db/product/11.2.0/dbhome_1
- Add following entry in /etc/resolv.conf
options timeout:1 attempts:1 rotate
- After successfull completion of patch on both nodes, please runcluvfy for installation verification:
[oracle@linuxrac1 ~]$ /media/grid/runcluvfy.sh stage -pre crsinst -upgrade -n linuxrac1,linuxrac2 -rolling -src_crshome /u01/11g_grid/grid -dest_crshome /u01/11g_grid/11204_grid -dest_version 11.2.0.4.0 -fixup -fixupdir /home/oracle/fixup Performing pre-checks for cluster services setup Checking node reachability... Node reachability check passed from node "linuxrac1" Checking user equivalence... User equivalence check passed for user "oracle" l Checking CRS user consistency CRS user consistency check successful Checking node connectivity... Checking hosts config file... Verification of the hosts config file successful Check: Node connectivity for interface "eth0" Node connectivity passed for interface "eth0" TCP connectivity check passed for subnet "192.168.1.0" Check: Node connectivity for interface "eth1" Node connectivity passed for interface "eth1" TCP connectivity check passed for subnet "10.0.0.0" Checking subnet mask consistency... Subnet mask consistency check passed for subnet "192.168.1.0". Subnet mask consistency check passed for subnet "10.0.0.0". Subnet mask consistency check passed. Node connectivity check passed Checking multicast communication... Checking subnet "192.168.1.0" for multicast communication with multicast group "230.0.1.0"... Check of subnet "192.168.1.0" for multicast communication with multicast group "230.0.1.0" passed. Checking subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0"... Check of subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0" passed. Check of multicast communication passed. Checking OCR integrity... OCR integrity check passed Checking ASMLib configuration. Check for ASMLib configuration passed. Total memory check passed Available memory check passed Swap space check passed Free disk space check passed for "linuxrac2:/u01/11g_grid/11204_grid" Free disk space check passed for "linuxrac1:/u01/11g_grid/11204_grid" Free disk space check passed for "linuxrac2:/tmp" Free disk space check passed for "linuxrac1:/tmp" Check for multiple users with UID value 500 passed User existence check passed for "oracle" Group existence check passed for "dbarac" Membership check for user "oracle" in group "dbarac" [as Primary] passed Run level check passed Hard limits check passed for "maximum open file descriptors" Soft limits check passed for "maximum open file descriptors" Hard limits check passed for "maximum user processes" Soft limits check passed for "maximum user processes" Check for Oracle patch "9413827 or 9706490" in home "/u01/11g_grid/grid" passed There are no oracle patches required for home "/u01/11g_grid/11204_grid". System architecture check passed Kernel version check passed Kernel parameter check passed for "semmsl" Kernel parameter check passed for "semmns" Kernel parameter check passed for "semopm" Kernel parameter check passed for "semmni" Kernel parameter check passed for "shmmax" Kernel parameter check passed for "shmmni" Kernel parameter check passed for "shmall" Kernel parameter check passed for "file-max" Kernel parameter check passed for "ip_local_port_range" Kernel parameter check passed for "rmem_default" Kernel parameter check passed for "rmem_max" Kernel parameter check passed for "wmem_default" Kernel parameter check passed for "wmem_max" Kernel parameter check passed for "aio-max-nr" Package existence check passed for "make" Package existence check passed for "binutils" Package existence check passed for "gcc" Package existence check passed for "gcc-c++" Package existence check passed for "libgomp" Package existence check passed for "libaio" Package existence check passed for "glibc" Package existence check passed for "compat-libstdc++-33" Package existence check passed for "elfutils-libelf" Package existence check passed for "elfutils-libelf-devel" Package existence check passed for "glibc-common" Package existence check passed for "glibc-devel" Package existence check passed for "glibc-headers" Package existence check passed for "libaio-devel" Package existence check passed for "libgcc" Package existence check passed for "libstdc++" Package existence check passed for "libstdc++-devel" Package existence check passed for "sysstat" Package existence check passed for "ksh" Check for multiple users with UID value 0 passed Current group ID check passed Starting check for consistency of primary group of root user Check for consistency of root user's primary group passed Package existence check passed for "cvuqdisk" Starting Clock synchronization checks using Network Time Protocol(NTP)... NTP Configuration file check started... No NTP Daemons or Services were found to be running Clock synchronization check using Network Time Protocol(NTP) passed Core file name pattern consistency check passed. User "oracle" is not part of "root" group. Check passed Default user file creation mask check passed Checking consistency of file "/etc/resolv.conf" across nodes File "/etc/resolv.conf" does not have both domain and search entries defined domain entry in file "/etc/resolv.conf" is consistent across nodes search entry in file "/etc/resolv.conf" is consistent across nodes PRVF-5636 : The DNS response time for an unreachable node exceeded "15000" ms on following nodes: linuxrac2,linuxrac1 File "/etc/resolv.conf" is not consistent across nodes UDev attributes check for OCR locations started... UDev attributes check passed for OCR locations UDev attributes check for Voting Disk locations started... UDev attributes check passed for Voting Disk locations Time zone consistency check passed Checking VIP configuration. Checking VIP Subnet configuration. Check for VIP Subnet configuration passed. Checking VIP reachability Check for VIP reachability passed. Checking Oracle Cluster Voting Disk configuration... ASM Running check passed. ASM is running on all specified nodes Oracle Cluster Voting Disk configuration check passed Clusterware version consistency passed Pre-check for cluster services setup was unsuccessful on all the nodes. [oracle@linuxrac1 ~]$ /media/grid/runcluvfy.sh stage -pre crsinst -upgrade -n linuxrac1,linuxrac2 -rolling -src_crshome /u01/11g_grid/grid -dest_crshome /u01/11g_grid/11204_grid -dest_version 11.2.0.4.0 -fixup -fixupdir /home/oracle/fixup Performing pre-checks for cluster services setup Checking node reachability... Node reachability check passed from node "linuxrac1" Checking user equivalence... User equivalence check passed for user "oracle" Checking CRS user consistency CRS user consistency check successful Checking node connectivity... Checking hosts config file... Verification of the hosts config file successful Check: Node connectivity for interface "eth0" Node connectivity passed for interface "eth0" TCP connectivity check passed for subnet "192.168.1.0" Check: Node connectivity for interface "eth1" Node connectivity passed for interface "eth1" TCP connectivity check passed for subnet "10.0.0.0" Checking subnet mask consistency... Subnet mask consistency check passed for subnet "192.168.1.0". Subnet mask consistency check passed for subnet "10.0.0.0". Subnet mask consistency check passed. Node connectivity check passed Checking multicast communication... Checking subnet "192.168.1.0" for multicast communication with multicast group "230.0.1.0"... Check of subnet "192.168.1.0" for multicast communication with multicast group "230.0.1.0" passed. Checking subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0"... Check of subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0" passed. Check of multicast communication passed. Checking OCR integrity... OCR integrity check passed Checking ASMLib configuration. Check for ASMLib configuration passed. Total memory check passed Available memory check passed Swap space check passed Free disk space check passed for "linuxrac2:/u01/11g_grid/11204_grid" Free disk space check passed for "linuxrac1:/u01/11g_grid/11204_grid" Free disk space check passed for "linuxrac2:/tmp" Free disk space check passed for "linuxrac1:/tmp" Check for multiple users with UID value 500 passed User existence check passed for "oracle" Group existence check passed for "dbarac" Membership check for user "oracle" in group "dbarac" [as Primary] passed Run level check passed Hard limits check passed for "maximum open file descriptors" Soft limits check passed for "maximum open file descriptors" Hard limits check passed for "maximum user processes" Soft limits check passed for "maximum user processes" Check for Oracle patch "9413827 or 9706490" in home "/u01/11g_grid/grid" passed There are no oracle patches required for home "/u01/11g_grid/11204_grid". System architecture check passed Kernel version check passed Kernel parameter check passed for "semmsl" Kernel parameter check passed for "semmns" Kernel parameter check passed for "semopm" Kernel parameter check passed for "semmni" Kernel parameter check passed for "shmmax" Kernel parameter check passed for "shmmni" Kernel parameter check passed for "shmall" Kernel parameter check passed for "file-max" Kernel parameter check passed for "ip_local_port_range" Kernel parameter check passed for "rmem_default" Kernel parameter check passed for "rmem_max" Kernel parameter check passed for "wmem_default" Kernel parameter check passed for "wmem_max" Kernel parameter check passed for "aio-max-nr" Package existence check passed for "make" Package existence check passed for "binutils" Package existence check passed for "gcc" Package existence check passed for "gcc-c++" Package existence check passed for "libgomp" Package existence check passed for "libaio" Package existence check passed for "glibc" Package existence check passed for "compat-libstdc++-33" Package existence check passed for "elfutils-libelf" Package existence check passed for "elfutils-libelf-devel" Package existence check passed for "glibc-common" Package existence check passed for "glibc-devel" Package existence check passed for "glibc-headers" Package existence check passed for "libaio-devel" Package existence check passed for "libgcc" Package existence check passed for "libstdc++" Package existence check passed for "libstdc++-devel" Package existence check passed for "sysstat" Package existence check passed for "ksh" Check for multiple users with UID value 0 passed Current group ID check passed Starting check for consistency of primary group of root user Check for consistency of root user's primary group passed Package existence check passed for "cvuqdisk" Starting Clock synchronization checks using Network Time Protocol(NTP)... NTP Configuration file check started... No NTP Daemons or Services were found to be running Clock synchronization check using Network Time Protocol(NTP) passed Core file name pattern consistency check passed. User "oracle" is not part of "root" group. Check passed Default user file creation mask check passed Checking consistency of file "/etc/resolv.conf" across nodes File "/etc/resolv.conf" does not have both domain and search entries defined domain entry in file "/etc/resolv.conf" is consistent across nodes search entry in file "/etc/resolv.conf" is consistent across nodes The DNS response time for an unreachable node is within acceptable limit on all nodes File "/etc/resolv.conf" is consistent across nodes UDev attributes check for OCR locations started... UDev attributes check passed for OCR locations UDev attributes check for Voting Disk locations started... UDev attributes check passed for Voting Disk locations Time zone consistency check passed Checking VIP configuration. Checking VIP Subnet configuration. Check for VIP Subnet configuration passed. Checking VIP reachability Check for VIP reachability passed. Checking Oracle Cluster Voting Disk configuration... ASM Running check passed. ASM is running on all specified nodes Oracle Cluster Voting Disk configuration check passed Clusterware version consistency passed Pre-check for cluster services setup was successful. [oracle@linuxrac1 ~]$
5) Install and upgrade Grid Infrastructure:
- set the proper display variable and execute "runInstaller" from grid software owner
- Follow screen shots
- Select "Upgrade"
- Verify SSH connectivity
- Provide location for 11.2.0.4 GI home location
- run "upgraderoot.sh" on both nodes as listed
root@linuxrac1 ~]# /u01/11g_grid/11204_grid/rootupgrade.sh Performing root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u01/11g_grid/11204_grid Enter the full pathname of the local bin directory: [/usr/local/bin]: The contents of "dbhome" have not changed. No need to overwrite. The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y Copying oraenv to /usr/local/bin ... The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y Copying coraenv to /usr/local/bin ... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Using configuration parameter file: /u01/11g_grid/11204_grid/crs/install/crsconfig_params Creating trace directory Installing Trace File Analyzer ASM upgrade has started on first node. CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'linuxrac1' CRS-2673: Attempting to stop 'ora.crsd' on 'linuxrac1' CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'linuxrac1' CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'linuxrac1' CRS-2673: Attempting to stop 'ora.GDATA.dg' on 'linuxrac1' CRS-2673: Attempting to stop 'ora.DBDATA.dg' on 'linuxrac1' CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'linuxrac1' succeeded CRS-2673: Attempting to stop 'ora.linuxrac1.vip' on 'linuxrac1' CRS-2677: Stop of 'ora.linuxrac1.vip' on 'linuxrac1' succeeded CRS-2672: Attempting to start 'ora.linuxrac1.vip' on 'linuxrac2' CRS-2676: Start of 'ora.linuxrac1.vip' on 'linuxrac2' succeeded CRS-2677: Stop of 'ora.GDATA.dg' on 'linuxrac1' succeeded CRS-2677: Stop of 'ora.DBDATA.dg' on 'linuxrac1' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'linuxrac1' CRS-2677: Stop of 'ora.asm' on 'linuxrac1' succeeded CRS-2673: Attempting to stop 'ora.ons' on 'linuxrac1' CRS-2673: Attempting to stop 'ora.eons' on 'linuxrac1' CRS-2677: Stop of 'ora.ons' on 'linuxrac1' succeeded CRS-2673: Attempting to stop 'ora.net1.network' on 'linuxrac1' CRS-2677: Stop of 'ora.net1.network' on 'linuxrac1' succeeded CRS-2677: Stop of 'ora.eons' on 'linuxrac1' succeeded CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'linuxrac1' has completed CRS-2677: Stop of 'ora.crsd' on 'linuxrac1' succeeded CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'linuxrac1' CRS-2673: Attempting to stop 'ora.ctssd' on 'linuxrac1' CRS-2673: Attempting to stop 'ora.evmd' on 'linuxrac1' CRS-2673: Attempting to stop 'ora.asm' on 'linuxrac1' CRS-2673: Attempting to stop 'ora.mdnsd' on 'linuxrac1' CRS-2677: Stop of 'ora.cssdmonitor' on 'linuxrac1' succeeded CRS-2677: Stop of 'ora.evmd' on 'linuxrac1' succeeded CRS-2677: Stop of 'ora.mdnsd' on 'linuxrac1' succeeded CRS-2677: Stop of 'ora.ctssd' on 'linuxrac1' succeeded CRS-2677: Stop of 'ora.asm' on 'linuxrac1' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'linuxrac1' CRS-2677: Stop of 'ora.cssd' on 'linuxrac1' succeeded CRS-2673: Attempting to stop 'ora.gpnpd' on 'linuxrac1' CRS-2673: Attempting to stop 'ora.diskmon' on 'linuxrac1' CRS-2677: Stop of 'ora.gpnpd' on 'linuxrac1' succeeded CRS-2673: Attempting to stop 'ora.gipcd' on 'linuxrac1' CRS-2677: Stop of 'ora.diskmon' on 'linuxrac1' succeeded CRS-2677: Stop of 'ora.gipcd' on 'linuxrac1' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'linuxrac1' has completed CRS-4133: Oracle High Availability Services has been stopped. OLR initialization - successful Replacing Clusterware entries in inittab clscfg: EXISTING configuration version 5 detected. clscfg: version 5 is 11g Release 2. Successfully accumulated necessary OCR keys. Creating OCR keys for user 'root', privgrp 'root'.. Operation successful. Configure Oracle Grid Infrastructure for a Cluster ... succeeded [root@linuxrac1 ~]#
root@linuxrac2 ~]# /u01/11g_grid/11204_grid/rootupgrade.sh Performing root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u01/11g_grid/11204_grid Enter the full pathname of the local bin directory: [/usr/local/bin]: The contents of "dbhome" have not changed. No need to overwrite. The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y Copying oraenv to /usr/local/bin ... The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y Copying coraenv to /usr/local/bin ... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Using configuration parameter file: /u01/11g_grid/11204_grid/crs/install/crsconfig_params Creating trace directory Installing Trace File Analyzer CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'linuxrac2' CRS-2673: Attempting to stop 'ora.crsd' on 'linuxrac2' CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'linuxrac2' CRS-2673: Attempting to stop 'ora.GDATA.dg' on 'linuxrac2' CRS-2673: Attempting to stop 'ora.DBDATA.dg' on 'linuxrac2' CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'linuxrac2' CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'linuxrac2' CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'linuxrac2' succeeded CRS-2673: Attempting to stop 'ora.linuxrac2.vip' on 'linuxrac2' CRS-2677: Stop of 'ora.linuxrac2.vip' on 'linuxrac2' succeeded CRS-2672: Attempting to start 'ora.linuxrac2.vip' on 'linuxrac1' CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'linuxrac2' succeeded CRS-2673: Attempting to stop 'ora.scan1.vip' on 'linuxrac2' CRS-2677: Stop of 'ora.scan1.vip' on 'linuxrac2' succeeded CRS-2672: Attempting to start 'ora.scan1.vip' on 'linuxrac1' CRS-2676: Start of 'ora.linuxrac2.vip' on 'linuxrac1' succeeded CRS-2676: Start of 'ora.scan1.vip' on 'linuxrac1' succeeded CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'linuxrac1' CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'linuxrac1' succeeded CRS-2677: Stop of 'ora.GDATA.dg' on 'linuxrac2' succeeded CRS-2677: Stop of 'ora.DBDATA.dg' on 'linuxrac2' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'linuxrac2' CRS-2677: Stop of 'ora.asm' on 'linuxrac2' succeeded CRS-2673: Attempting to stop 'ora.eons' on 'linuxrac2' CRS-2673: Attempting to stop 'ora.ons' on 'linuxrac2' CRS-2677: Stop of 'ora.ons' on 'linuxrac2' succeeded CRS-2673: Attempting to stop 'ora.net1.network' on 'linuxrac2' CRS-2677: Stop of 'ora.net1.network' on 'linuxrac2' succeeded CRS-2677: Stop of 'ora.eons' on 'linuxrac2' succeeded CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'linuxrac2' has completed CRS-2677: Stop of 'ora.crsd' on 'linuxrac2' succeeded CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'linuxrac2' CRS-2673: Attempting to stop 'ora.ctssd' on 'linuxrac2' CRS-2673: Attempting to stop 'ora.evmd' on 'linuxrac2' CRS-2673: Attempting to stop 'ora.asm' on 'linuxrac2' CRS-2673: Attempting to stop 'ora.mdnsd' on 'linuxrac2' CRS-2677: Stop of 'ora.cssdmonitor' on 'linuxrac2' succeeded CRS-2677: Stop of 'ora.evmd' on 'linuxrac2' succeeded CRS-2677: Stop of 'ora.mdnsd' on 'linuxrac2' succeeded CRS-2677: Stop of 'ora.asm' on 'linuxrac2' succeeded CRS-2677: Stop of 'ora.ctssd' on 'linuxrac2' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'linuxrac2' CRS-2677: Stop of 'ora.cssd' on 'linuxrac2' succeeded CRS-2673: Attempting to stop 'ora.gpnpd' on 'linuxrac2' CRS-2673: Attempting to stop 'ora.diskmon' on 'linuxrac2' CRS-2677: Stop of 'ora.gpnpd' on 'linuxrac2' succeeded CRS-2673: Attempting to stop 'ora.gipcd' on 'linuxrac2' CRS-2677: Stop of 'ora.gipcd' on 'linuxrac2' succeeded CRS-2677: Stop of 'ora.diskmon' on 'linuxrac2' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'linuxrac2' has completed CRS-4133: Oracle High Availability Services has been stopped. OLR initialization - successful Replacing Clusterware entries in inittab clscfg: EXISTING configuration version 5 detected. clscfg: version 5 is 11g Release 2. Successfully accumulated necessary OCR keys. Creating OCR keys for user 'root', privgrp 'root'.. Operation successful. Started to upgrade the Oracle Clusterware. This operation may take a few minutes. Started to upgrade the CSS. Started to upgrade the CRS. The CRS was successfully upgraded. Successfully upgraded the Oracle Clusterware. Oracle Clusterware operating version was successfully set to 11.2.0.4.0 ASM upgrade has finished on last node. Configure Oracle Grid Infrastructure for a Cluster ... succeeded [root@linuxrac2 ~]#
6) Verify Upgrade:
- Verify services:
- startup database and make sure both Instances are working fine without any Issues.
11.2.0.1 Grid Infrastructure has been successfully upgraded to 11.2.0.4.
In next article i will demonstrate how to upgrade an Oracle RAC database from 11.2.0.1 to 11.2.0.4
Thanks for reading.
regards,
X A H E E R
Comments
because when you execute a shell script at the end. your rac services get restarted.
Please suggest the best option.