Oracle Apps DBA

My photo
More than 5+ Years of IT Experience in administering Enterprise wide Multi Vendor UNIX Servers, Oracle Databases (8i to 11gR2), Middle-tiers, Applications and Clusters. I am a Sun Certified System Administrator (SCSA) for Solaris 10, oracle 10g database certified associate(OCA), Oracle 10g Database certified professional (OCP) and Oracle E-Business suite R12 certified professionl(OCP)

Oracle Database and Applications

Sunday, 11 May 2014

Oracle 12c RAC Installation on linux using virtualbox

This a step by step article for configuring oracle 12c RAC on Oracle Enterprise Linux 6.1.


Following software's used for this deployment:

1) Oracle Virtual Box
2) Oracle Enterprise Linux 6.1 (64-Bit)
3) Oracle 12c Grid Infrastructure (64-Bit)
4) Oracle 12c Database

Virtual machines configuration:




I used windows DNS Server 2008 for resolving scan IP to simulate it with real world deployments. You can use a single entry of scan in /etc/hosts file for TEST deployments.

The high level steps for deployment: (assuming virtual box - already Installed)

1) Create Virtual machines
2) Install Oracle Enterprise Linux 6.1
3) Configure additional Hardware required for RAC (SHARED STORAGE)
4) Clone virtual machine
5) Perform all OS pre-requisites
6) Install Oracle 12c Grid Infrastructure
7) Install Oracle 12c RDBMS  home
8) Create RAC database

1) Create Virtual Mahcine:

Follow step by step screens for creation of virtual machines:










>> use oracle enterprise linux 6.1 iso image for OS Installaiton



>> Configure Additional network card for private interconnect




2- INSTALLATION OF ORACLE ENTERPRISE LINUX:























































>> Disable SELINUX and FIREWALL





>> Install virtual box tools



Installation of Linux completed need to restart racnode1

3 - CONFIGURE ADDITIONAL HARDWARE REQUIRED FOR RAC:


RACDISK1               => 1 GB  (CRS)
RACDISK2               => 1 GB  (CRS)
DATADISK1             => 5 GB (DATA)
DATADISK2             => 5 GB (DATA)


>> Add scsci controller








>> I created racshare directory in which all share-able disks are created.



>> Similarly create for racdisk2, datadisk1 and datadisk2



>> Go to virtual device manager and change its settings for all these disks as share-able

.


>> Make sure that all share-able disks are configured as listed.



4-PERFORM ALL OS PRE-REQUISITES:

>> Add entries for SCAN in DNS Server










>> If you're using DNS windows server as SCAN then please make sure entries are configured as listed.

>> If you're using DNS in hosts file then this step is not required, just add 1 scan IP address with fully qualified domain name in /etc/hosts file


- Configure Kernel Parameters:

>> add the following entires in /etc/sysctl.conf file

fs.file-max = 6815744
kernel.sem = 250 32000 100 128
kernel.shmmni = 4096
kernel.shmall = 1073741824
kernel.shmmax = 4398046511104
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
fs.aio-max-nr = 1048576
net.ipv4.ip_local_port_range = 9000 65500

- Verify using command:


[root@racnode1 ~]# sysctl -p
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0
kernel.msgmnb = 65536
kernel.msgmax = 65536
fs.file-max = 6815744
kernel.sem = 250 32000 100 128
kernel.shmmni = 4096
kernel.shmall = 1073741824
kernel.shmmax = 4398046511104
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
fs.aio-max-nr = 1048576
net.ipv4.ip_local_port_range = 9000 65500
[root@racnode1 ~]# 

>> configure open files:



[root@racnode1 ~]# tail -7 /etc/security/limits.conf 
# End of file
oracle   soft   nofile    1024
oracle   hard   nofile    65536
oracle   soft   nproc    2047
oracle   hard   nproc    16384
oracle   soft   stack    10240
oracle   hard   stack    32768
[root@racnode1 ~]# 


>> Edit following file as listed in Installation doc for 12c RAC


[root@racnode1 ~]# cat /etc/security/limits.d/90-nproc.conf# Default limit for number of user's processes to prevent
# accidental fork bombs.
# See rhbz #432903 for reasoning.

#*          soft    nproc     1024
* 		- nproc 16384
[root@racnode1 ~]# 

>> configure /etc/hosts file as listed:


[root@racnode1 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
#RAC hosts IP addresses
#################PUBLIC################
192.168.1.41	racnode1.oralabs.com	racnode1 localhost
192.168.1.42	racnode2.oralabs.com	racnode2 
################PRIVATE###############
10.10.15.21	racnode1-priv.oralabs.com	racnode1-priv
10.10.15.22	racnode2-priv.oralabs.com	racnode2-priv
################VIP###################
192.168.1.47	racnode1-vip.oralabs.com	racnode1-vip
192.168.1.48	racnode2-vip.oralabs.com	racnode2-vip

>>  verify DNS is able to resolve scan name:


[root@racnode1 ~]# nslookup oralabs.com
Server:		192.168.1.1
Address:	192.168.1.1#53

Name:	oralabs.com
Address: 192.168.1.1

[root@racnode1 ~]# nslookup scan-rac12c
Server:		192.168.1.1
Address:	192.168.1.1#53

Name:	scan-rac12c.oralabs.com
Address: 192.168.1.46
Name:	scan-rac12c.oralabs.com
Address: 192.168.1.44
Name:	scan-rac12c.oralabs.com
Address: 192.168.1.45

[root@racnode1 ~]# 

>> disable NTP service:



[root@racnode1 ~]# chkconfig ntpd off
[root@racnode1 ~]# mv /etc/ntp.conf /etc/ntp.conf.orig
[root@racnode1 ~]# 

>>   Install all additional required rpms:



yum install binutils -y
yum install compat-libcap1 -y
yum install compat-libstdc++-33 -y
yum install compat-libstdc++-33.i686 -y
yum install gcc -y
yum install gcc-c++ -y
yum install glibc -y
yum install glibc.i686 -y
yum install glibc-devel -y
yum install glibc-devel.i686 -y
yum install ksh -y
yum install libgcc -y
yum install libgcc.i686 -y
yum install libstdc++ -y
yum install libstdc++.i686 -y
yum install libstdc++-devel -y
yum install libstdc++-devel.i686 -y
yum install libaio -y
yum install libaio.i686 -y
yum install libaio-devel -y
yum install libaio-devel.i686 -y
yum install libXext -y
yum install libXext.i686 -y
yum install libXtst -y
yum install libXtst.i686 -y
yum install libX11 -y
yum install libX11.i686 -y
yum install libXau -y
yum install libXau.i686 -y
yum install libxcb -y
yum install libxcb.i686 -y
yum install libXi -y
yum install libXi.i686 -y
yum install make -y
yum install sysstat -y
yum install unixODBC -y
yum install unixODBC-devel -y

- Install package for oracle ASM support.


[root@racnode1 Packages]# rpm -ivh oracleasm-support-2.1.5-1.el6.x86_64.rpm 
warning: oracleasm-support-2.1.5-1.el6.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY
Preparing...                ########################################### [100%]
   1:oracleasm-support      ########################################### [100%]
[root@racnode1 Packages]# rpm

>> Create user, groups and provide directory privileges:



>> Configure oracle asm:



[root@racnode1 ~]# oracleasm configure -i
Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver.  The following questions will determine whether the driver is
loaded on boot and what permissions it will have.  The current values
will be shown in brackets ('[]').  Hitting  without typing an
answer will keep that current value.  Ctrl-C will abort.

Default user to own the driver interface []: oracle
Default group to own the driver interface []: dbarac
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done
[root@racnode1 ~]# 


>> configure shared disks (label disks and create partitions)


[root@racnode1 ~]# fdisk -l

Disk /dev/sda: 65.0 GB, 65010663424 bytes
255 heads, 63 sectors/track, 7903 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000995f5

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1        1275    10240000   83  Linux
/dev/sda2            1275        2320     8388608   82  Linux swap / Solaris
/dev/sda3            2320        7904    44857344   83  Linux

Disk /dev/sdb: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/sdb doesn't contain a valid partition table

Disk /dev/sdc: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/sdc doesn't contain a valid partition table

Disk /dev/sdd: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/sdd doesn't contain a valid partition table

Disk /dev/sde: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/sde doesn't contain a valid partition table
[root@racnode1 ~]# 


====================


[root@racnode1 ~]# fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x86773cdb.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-130, default 1): 
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-130, default 130): 
Using default value 130

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
[root@racnode1 ~]# 



-------

[root@racnode1 ~]# fdik /dev/sdc
bash: fdik: command not found
[root@racnode1 ~]# fdisk /dev/sdc
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0xe2f263cb.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-130, default 1): 
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-130, default 130): 
Using default value 130

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
[root@racnode1 ~]# fdisk /dev/sdd
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x0ed3f58c.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-652, default 1): 
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-652, default 652): 
Using default value 652

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
[root@racnode1 ~]# fdisk /dev/sde
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0xb2444de9.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 
Value out of range.
Partition number (1-4): 1
First cylinder (1-652, default 1): 
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-652, default 652): 
Using default value 652

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
[root@racnode1 ~]# 




================================


[root@racnode1 ~]# fdisk  -l

Disk /dev/sda: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x86773cdb

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1         130     1044193+  83  Linux

Disk /dev/sdb: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xe2f263cb

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1         130     1044193+  83  Linux

Disk /dev/sdc: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0ed3f58c

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1         652     5237158+  83  Linux

Disk /dev/sdd: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xb2444de9

   Device Boot      Start         End      Blocks   Id  System
/dev/sdd1               1         652     5237158+  83  Linux

Disk /dev/sde: 65.0 GB, 65010663424 bytes
255 heads, 63 sectors/track, 7903 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000995f5

   Device Boot      Start         End      Blocks   Id  System
/dev/sde1   *           1        1275    10240000   83  Linux
/dev/sde2            1275        2320     8388608   82  Linux swap / Solaris
/dev/sde3            2320        7904    44857344   83  Linux
[root@racnode1 ~]# 



>> creation of asm disks:



[root@racnode1 ~]# oracleasm createdisk crsdisk1 /dev/sda1
Writing disk header: done
Instantiating disk: done
[root@racnode1 ~]# oracleasm createdisk crsdisk2 /dev/sdb1
Writing disk header: done
Instantiating disk: done
[root@racnode1 ~]# oracleasm createdisk datadisk1 /dev/sdc1
Writing disk header: done
Instantiating disk: done
[root@racnode1 ~]# oracleasm createdisk datadisk2 /dev/sdd1
Writing disk header: done
Instantiating disk: done
[root@racnode1 ~]# 

[root@racnode1 ~]# oracleasm listdisks
CRSDISK1
CRSDISK2
DATADISK1
DATADISK2
[root@racnode1 ~]# 

>> make sure you are able to ping all IP addresses configured.


4-CLONE VIRTUAL MACHINE TO RACNODE2

>> Clone virtual machine hard disks to racnode2


>> create another virtual machine and add cloned HDD to virtual machine









- Here select use existing HDD.




>> add share-able HDD to racnode2 virtualmachine.





>> similarly add all disks  and make sure its configured on same scsi ports as on racnode1 vm settintgs.

>> shutdown racnode1

>> poweron racnode2 

>> chnage IP addresses using network manager utility for public and private interfaces:





- After startup of virtual machine you may notice Interface names will be different for configured IP's. Please follow steps for changing the interface name:

- Incorrect interface names:



[root@racnode2 default]# ifconfig -a
eth2      Link encap:Ethernet  HWaddr 08:00:27:70:43:A5  
          inet addr:192.168.1.42  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::a00:27ff:fe70:43a5/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:156 errors:0 dropped:0 overruns:0 frame:0
          TX packets:105 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:17694 (17.2 KiB)  TX bytes:16071 (15.6 KiB)

eth3      Link encap:Ethernet  HWaddr 08:00:27:F6:68:BD  
          inet addr:10.10.15.22  Bcast:10.255.255.255  Mask:255.0.0.0
          inet6 addr: fe80::a00:27ff:fef6:68bd/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:126 errors:0 dropped:0 overruns:0 frame:0
          TX packets:41 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:15843 (15.4 KiB)  TX bytes:8431 (8.2 KiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:24 errors:0 dropped:0 overruns:0 frame:0
          TX packets:24 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:1632 (1.5 KiB)  TX bytes:1632 (1.5 KiB)

virbr0    Link encap:Ethernet  HWaddr 52:54:00:E4:D7:42  
          inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:12 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 b)  TX bytes:2730 (2.6 KiB)

virbr0-nic Link encap:Ethernet  HWaddr 52:54:00:E4:D7:42  
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

[root@racnode2 default]# 


>> check HW MAC address for configured interfaces:


[root@racnode2 network-scripts]# ifconfig -a | grep eth
eth2      Link encap:Ethernet  HWaddr 08:00:27:70:43:A5  
eth3      Link encap:Ethernet  HWaddr 08:00:27:F6:68:BD  
[root@racnode2 network-scripts]# 
>>  Edit this address in files ifcfg-eth0 and ifcfg-eth1 file under "/etc/sysconfig/network-scripts" directory:


root@racnode2 network-scripts]# cat ifcfg-eth0
DEVICE=eth0
NM_CONTROLLED=yes
ONBOOT=yes
TYPE=Ethernet
BOOTPROTO=none
#IPADDR=192.168.1.42
PREFIX=24
GATEWAY=192.168.1.1
DNS1=192.168.1.1
DOMAIN=oralabs.com
DEFROUTE=yes
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
NAME="System eth0"
#UUID=5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03
USERCTL=no
#HWADDR=08:00:27:BF:9D:56
HWADDR=08:00:27:70:43:A5
IPADDR=192.168.1.42
[root@racnode2 network-scripts]# 

----
[root@racnode2 network-scripts]# cat ifcfg-eth1
DEVICE=eth1
NM_CONTROLLED=yes
ONBOOT=yes
TYPE=Ethernet
BOOTPROTO=none
#IPADDR=10.10.15.22
PREFIX=8
DEFROUTE=yes
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
NAME="System eth1"
#UUID=9c92fad9-6ecb-3e6c-eb4d-8a47c6f50c04
NETMASK=255.0.0.0
USERCTL=no
#HWADDR=08:00:27:6E:E6:AA
HWADDR=08:00:27:F6:68:BD
IPADDR=10.10.15.22
[root@racnode2 network-scripts]# 
>> update mac address in below file


root@racnode2 default]# cat /etc/udev/rules.d/70-persistent-net.rules
# This file was automatically generated by the /lib/udev/write_net_rules
# program, run by the persistent-net-generator.rules rules file.
#
# You can modify it, as long as you keep each rule on a single
# line, and change only the value of the NAME= key.

# PCI device 0x8086:0x100e (e1000)
#SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="08:00:27:bf:9d:56", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

# PCI device 0x8086:0x100e (e1000)
#SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="08:00:27:6e:e6:aa", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"

# PCI device 0x8086:0x100e (e1000)
#SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="08:00:27:70:43:a5", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth2"

# PCI device 0x8086:0x100e (e1000)
#SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="08:00:27:f6:68:bd", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth3"

# PCI device 0x8086:0x100e (e1000) (custom name provided by external tool)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="08:00:27:f6:68:bd", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"

# PCI device 0x8086:0x100e (e1000) (custom name provided by external tool)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="08:00:27:70:43:a5", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

>> Verify the network interfaces again:


[root@racnode2 ~]# ifconfig -a
eth0      Link encap:Ethernet  HWaddr 08:00:27:70:43:A5  
          inet addr:192.168.1.42  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::a00:27ff:fe70:43a5/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:48 errors:0 dropped:0 overruns:0 frame:0
          TX packets:39 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:6656 (6.5 KiB)  TX bytes:5654 (5.5 KiB)

eth1      Link encap:Ethernet  HWaddr 08:00:27:F6:68:BD  
          inet addr:10.10.15.22  Bcast:10.255.255.255  Mask:255.0.0.0
          inet6 addr: fe80::a00:27ff:fef6:68bd/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:50 errors:0 dropped:0 overruns:0 frame:0
          TX packets:31 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:6782 (6.6 KiB)  TX bytes:5148 (5.0 KiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:16 errors:0 dropped:0 overruns:0 frame:0
          TX packets:16 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:960 (960.0 b)  TX bytes:960 (960.0 b)


>> Start virtual machine 1 and verify communication between nodes:

[root@racnode2 ~]# ping racnode1
PING racnode1.oralabs.com (192.168.1.41) 56(84) bytes of data.
64 bytes from racnode1.oralabs.com (192.168.1.41): icmp_seq=1 ttl=64 time=1.62 ms
^C
--- racnode1.oralabs.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 695ms
rtt min/avg/max/mdev = 1.628/1.628/1.628/0.000 ms
[root@racnode2 ~]# ping racnode2
PING racnode2.oralabs.com (192.168.1.42) 56(84) bytes of data.
64 bytes from racnode2.oralabs.com (192.168.1.42): icmp_seq=1 ttl=64 time=0.050 ms
^C
--- racnode2.oralabs.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 576ms
rtt min/avg/max/mdev = 0.050/0.050/0.050/0.000 ms
[root@racnode2 ~]# ping racnode2-priv
PING racnode2-priv.oralabs.com (10.10.15.22) 56(84) bytes of data.
64 bytes from racnode2-priv.oralabs.com (10.10.15.22): icmp_seq=1 ttl=64 time=0.047 ms
^C
--- racnode2-priv.oralabs.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 928ms
rtt min/avg/max/mdev = 0.047/0.047/0.047/0.000 ms
[root@racnode2 ~]# ping racnode1-priv
PING racnode1-priv.oralabs.com (10.10.15.21) 56(84) bytes of data.
64 bytes from racnode1-priv.oralabs.com (10.10.15.21): icmp_seq=1 ttl=64 time=1.34 ms
^C
--- racnode1-priv.oralabs.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 935ms
rtt min/avg/max/mdev = 1.346/1.346/1.346/0.000 ms
[root@racnode2 ~]# 

>> SCAN asm dfisks


[root@racnode2 u01]# oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
Instantiating disk "CRSDISK1"
Instantiating disk "CRSDISK2"
Instantiating disk "DATADISK1"
Instantiating disk "DATADISK2"
[root@racnode2 u01]# 

6-INSTALL ORACLE 12C GRID INFRASTRUCTURE:


>> Copy media to loca disk

>> run xhost + as root user

>> runInstaller

>> Follow screenshots










>> Verify Interfaces




>> change discovery path













>> This package already exists on system











>> Execute scripts orainstRoot.sh and root.sh in sequence as listed


[root@racnode1 ~]# /u01/oraInventory/orainstRoot.sh
Changing permissions of /u01/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /u01/oraInventory to dbarac.
The execution of the script is complete.
[root@racnode1 ~]# 


[root@racnode2 ~]# /u01/oraInventory/orainstRoot.sh
Changing permissions of /u01/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /u01/oraInventory to dbarac.
The execution of the script is complete.
[root@racnode2 ~]# 



[root@racnode1 ~]# /u01/grid_12c/root.sh
Changing permissions of /u01/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /u01/oraInventory to dbarac.
The execution of the script is complete.
[root@racnode1 ~]# /u01/grid_12c/root.sh
Performing root user operation for Oracle 12c 

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/grid_12c

Enter the full pathname of the local bin directory: [/usr/local/bin]: 
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...


Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/grid_12c/crs/install/crsconfig_params
2014/04/29 15:19:46 CLSRSC-363: User ignored prerequisites during installation

OLR initialization - successful
  root wallet
  root wallet cert
  root cert export
  peer wallet
  profile reader wallet
  pa wallet
  peer wallet keys
  pa wallet keys
  peer cert request
  pa cert request
  peer cert
  pa cert
  peer root cert TP
  profile reader root cert TP
  pa root cert TP
  peer pa cert TP
  pa peer cert TP
  profile reader pa cert TP
  profile reader peer cert TP
  peer user cert
  pa user cert
2014/04/29 15:20:57 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.conf'

CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'racnode1'
CRS-2677: Stop of 'ora.drivers.acfs' on 'racnode1' succeeded
CRS-2672: Attempting to start 'ora.evmd' on 'racnode1'
CRS-2672: Attempting to start 'ora.mdnsd' on 'racnode1'
CRS-2676: Start of 'ora.mdnsd' on 'racnode1' succeeded
CRS-2676: Start of 'ora.evmd' on 'racnode1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'racnode1'
CRS-2676: Start of 'ora.gpnpd' on 'racnode1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'racnode1'
CRS-2672: Attempting to start 'ora.gipcd' on 'racnode1'
CRS-2676: Start of 'ora.cssdmonitor' on 'racnode1' succeeded
CRS-2676: Start of 'ora.gipcd' on 'racnode1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'racnode1'
CRS-2672: Attempting to start 'ora.diskmon' on 'racnode1'
CRS-2676: Start of 'ora.diskmon' on 'racnode1' succeeded
CRS-2676: Start of 'ora.cssd' on 'racnode1' succeeded

ASM created and started successfully.

Disk Group GDATA created successfully.

CRS-2672: Attempting to start 'ora.storage' on 'racnode1'
CRS-2676: Start of 'ora.storage' on 'racnode1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'racnode1'
CRS-2676: Start of 'ora.crsd' on 'racnode1' succeeded
CRS-4256: Updating the profile
Successful addition of voting disk 6ca78369dffd4f37bf04f8a1b6dbb9b9.
Successfully replaced voting disk group with +GDATA.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   6ca78369dffd4f37bf04f8a1b6dbb9b9 (/dev/oracleasm/disks/CRSDISK1) [GDATA]
Located 1 voting disk(s).
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'racnode1'
CRS-2673: Attempting to stop 'ora.crsd' on 'racnode1'
CRS-2677: Stop of 'ora.crsd' on 'racnode1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'racnode1'
CRS-2673: Attempting to stop 'ora.storage' on 'racnode1'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'racnode1'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'racnode1'
CRS-2677: Stop of 'ora.storage' on 'racnode1' succeeded
CRS-2677: Stop of 'ora.drivers.acfs' on 'racnode1' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'racnode1'
CRS-2673: Attempting to stop 'ora.evmd' on 'racnode1'
CRS-2673: Attempting to stop 'ora.asm' on 'racnode1'
CRS-2677: Stop of 'ora.gpnpd' on 'racnode1' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'racnode1' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'racnode1' succeeded
CRS-2677: Stop of 'ora.evmd' on 'racnode1' succeeded
CRS-2677: Stop of 'ora.asm' on 'racnode1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'racnode1'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'racnode1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'racnode1'
CRS-2677: Stop of 'ora.cssd' on 'racnode1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'racnode1'
CRS-2677: Stop of 'ora.gipcd' on 'racnode1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'racnode1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-2672: Attempting to start 'ora.mdnsd' on 'racnode1'
CRS-2672: Attempting to start 'ora.evmd' on 'racnode1'
CRS-2676: Start of 'ora.mdnsd' on 'racnode1' succeeded
CRS-2676: Start of 'ora.evmd' on 'racnode1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'racnode1'
CRS-2676: Start of 'ora.gpnpd' on 'racnode1' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'racnode1'
CRS-2676: Start of 'ora.gipcd' on 'racnode1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'racnode1'
CRS-2676: Start of 'ora.cssdmonitor' on 'racnode1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'racnode1'
CRS-2672: Attempting to start 'ora.diskmon' on 'racnode1'
CRS-2676: Start of 'ora.diskmon' on 'racnode1' succeeded
CRS-2789: Cannot stop resource 'ora.diskmon' as it is not running on server 'racnode1'
CRS-2676: Start of 'ora.cssd' on 'racnode1' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'racnode1'
CRS-2672: Attempting to start 'ora.ctssd' on 'racnode1'
CRS-2676: Start of 'ora.ctssd' on 'racnode1' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'racnode1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'racnode1'
CRS-2676: Start of 'ora.asm' on 'racnode1' succeeded
CRS-2672: Attempting to start 'ora.storage' on 'racnode1'
CRS-2676: Start of 'ora.storage' on 'racnode1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'racnode1'
CRS-2676: Start of 'ora.crsd' on 'racnode1' succeeded
CRS-6023: Starting Oracle Cluster Ready Services-managed resources
CRS-6017: Processing resource auto-start for servers: racnode1
CRS-6016: Resource auto-start has completed for server racnode1
CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources
CRS-4123: Oracle High Availability Services has been started.
2014/04/29 15:28:16 CLSRSC-343: Successfully started Oracle clusterware stack

CRS-2672: Attempting to start 'ora.asm' on 'racnode1'
CRS-2676: Start of 'ora.asm' on 'racnode1' succeeded
CRS-2672: Attempting to start 'ora.GDATA.dg' on 'racnode1'
CRS-2676: Start of 'ora.GDATA.dg' on 'racnode1' succeeded
2014/04/29 15:30:22 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded

[root@racnode1 ~]# 

On Node2:


[root@racnode2 ~]# /u01/grid_12c/root.sh
Performing root user operation for Oracle 12c 

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/grid_12c

Enter the full pathname of the local bin directory: [/usr/local/bin]: 
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...


Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/grid_12c/crs/install/crsconfig_params
2014/04/29 15:31:26 CLSRSC-363: User ignored prerequisites during installation

OLR initialization - successful
2014/04/29 15:31:51 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.conf'

CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'racnode2'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'racnode2'
CRS-2677: Stop of 'ora.drivers.acfs' on 'racnode2' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'racnode2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-2672: Attempting to start 'ora.mdnsd' on 'racnode2'
CRS-2672: Attempting to start 'ora.evmd' on 'racnode2'
CRS-2676: Start of 'ora.mdnsd' on 'racnode2' succeeded
CRS-2676: Start of 'ora.evmd' on 'racnode2' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'racnode2'
CRS-2676: Start of 'ora.gpnpd' on 'racnode2' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'racnode2'
CRS-2676: Start of 'ora.gipcd' on 'racnode2' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'racnode2'
CRS-2676: Start of 'ora.cssdmonitor' on 'racnode2' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'racnode2'
CRS-2672: Attempting to start 'ora.diskmon' on 'racnode2'
CRS-2676: Start of 'ora.diskmon' on 'racnode2' succeeded
CRS-2789: Cannot stop resource 'ora.diskmon' as it is not running on server 'racnode2'
CRS-2676: Start of 'ora.cssd' on 'racnode2' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'racnode2'
CRS-2672: Attempting to start 'ora.ctssd' on 'racnode2'
CRS-2676: Start of 'ora.ctssd' on 'racnode2' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'racnode2' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'racnode2'
CRS-2676: Start of 'ora.asm' on 'racnode2' succeeded
CRS-2672: Attempting to start 'ora.storage' on 'racnode2'
CRS-2676: Start of 'ora.storage' on 'racnode2' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'racnode2'
CRS-2676: Start of 'ora.crsd' on 'racnode2' succeeded
CRS-6017: Processing resource auto-start for servers: racnode2
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'racnode1'
CRS-2672: Attempting to start 'ora.ons' on 'racnode2'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'racnode1' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'racnode1'
CRS-2677: Stop of 'ora.scan1.vip' on 'racnode1' succeeded
CRS-2672: Attempting to start 'ora.scan1.vip' on 'racnode2'
CRS-2676: Start of 'ora.scan1.vip' on 'racnode2' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'racnode2'
CRS-2676: Start of 'ora.ons' on 'racnode2' succeeded
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'racnode2' succeeded
CRS-6016: Resource auto-start has completed for server racnode2
CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources
CRS-4123: Oracle High Availability Services has been started.
2014/04/29 15:36:18 CLSRSC-343: Successfully started Oracle clusterware stack

2014/04/29 15:36:36 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded

[root@racnode2 ~]# 

>> Installation of Grid Infrastructure completed.


7-INSTALL ORACLE 12C RDBMS HOME (RAC):

>> copy rdbms software in loca directory

>> runInstaller


























>> Execute scripts in sequence as listed.







>> Installation of RDBMS software completed successfully.

8-CREATE RAC DATABASE:

>> Creation of ASM Disk group "DATA" for database

>> use "asmca" for creation of diskgroup













>> Creation of RAC database:

- set RDBMS ORACLE_HOME and execute dbca:




















Successfully configured 2 node RAC database on Oracle linux 6 using virtual box.

Hope this post will be helpful for all who are trying to Install and configure 12c RAC sw on personal computers and production deployments.

Thanks for reading. Any comments/suggestion are highly appreciated.

regards,
X A H E E R


2 comments:

Blogger said...

Teeth Night Guard is selling precise fitting and high quality custom made dental guards.

Blogger said...

Looking for the Ultimate Dating Site? Create an account to find your perfect match.