Oracle RAC (Real Application Cluster) is used for complex application to increase database performance, high availability and scalability.
Below high level steps will perform to install/configure Oracle RAC:
- Configure 2 Virtual Machine (VM) for 2 Nodes RAC.
- Install Oracle Linux 7.1.
- Oracle Installation Prerequisites.
- Additional Setup.
- Create Shared Disk.
- Clone the Virtual Machine for Node2.
- Install GI (Grid Infrastructure).
- Configure ASM.
- Install Oracle Database Binary and Create Database on RAC.
- Issues.
1. Configure VM for 2 Nodes RAC. Create VM
2. Install Oracle Linux 7.1. OEL 7.1
3. Oracle Installation Prerequisites:
Perform either the Automatic Setup or the Manual Setup to complete the basic prerequisites. The Additional Setup is required for all installations.
3.1. Automatic Setup:
All necessary prerequisites will be performed automatically.
[root@racnode1 ~]# yum install oracle-rdbms-server-11gR2-preinstall Loaded plugins: langpacks Resolving Dependencies --> Running transaction check ---> Package oracle-rdbms-server-11gR2-preinstall.x86_64 0:1.0-6.el7 will be installed --> Processing Dependency: gcc-c++ for package: oracle-rdbms-server-11gR2-preinstall-1.0-6.el7.x86_64 --> Processing Dependency: compat-libcap1 for package: oracle-rdbms-server-11gR2-preinstall-1.0-6.el7.x86_64 --> Processing Dependency: ksh for package: oracle-rdbms-server-11gR2-preinstall-1.0-6.el7.x86_64 --> Processing Dependency: libaio-devel for package: oracle-rdbms-server-11gR2-preinstall-1.0-6.el7.x86_64 --> Processing Dependency: compat-libstdc++-33 for package: oracle-rdbms-server-11gR2-preinstall-1.0-6.el7.x86_64 --> Processing Dependency: libstdc++-devel for package: oracle-rdbms-server-11gR2-preinstall-1.0-6.el7.x86_64 --> Running transaction check ---> Package compat-libcap1.x86_64 0:1.10-7.el7 will be installed ---> Package compat-libstdc++-33.x86_64 0:3.2.3-72.el7 will be installed ---> Package gcc-c++.x86_64 0:4.8.5-44.0.3.el7 will be installed --> Processing Dependency: libstdc++ = 4.8.5-44.0.3.el7 for package: gcc-c++-4.8.5-44.0.3.el7.x86_64 --> Processing Dependency: gcc = 4.8.5-44.0.3.el7 for package: gcc-c++-4.8.5-44.0.3.el7.x86_64 ---> Package ksh.x86_64 0:20120801-142.0.1.el7 will be installed ---> Package libaio-devel.x86_64 0:0.3.109-13.el7 will be installed --> Processing Dependency: libaio(x86-64) = 0.3.109-13.el7 for package: libaio-devel-0.3.109-13.el7.x86_64 ---> Package libstdc++-devel.x86_64 0:4.8.5-44.0.3.el7 will be installed --> Running transaction check ---> Package gcc.x86_64 0:4.8.3-9.el7 will be updated ---> Package gcc.x86_64 0:4.8.5-44.0.3.el7 will be an update --> Processing Dependency: libgomp = 4.8.5-44.0.3.el7 for package: gcc-4.8.5-44.0.3.el7.x86_64 --> Processing Dependency: cpp = 4.8.5-44.0.3.el7 for package: gcc-4.8.5-44.0.3.el7.x86_64 --> Processing Dependency: libgcc >= 4.8.5-44.0.3.el7 for package: gcc-4.8.5-44.0.3.el7.x86_64 ---> Package libaio.x86_64 0:0.3.109-12.el7 will be updated ---> Package libaio.x86_64 0:0.3.109-13.el7 will be an update ---> Package libstdc++.x86_64 0:4.8.3-9.el7 will be updated ---> Package libstdc++.x86_64 0:4.8.5-44.0.3.el7 will be an update --> Running transaction check ---> Package cpp.x86_64 0:4.8.3-9.el7 will be updated ---> Package cpp.x86_64 0:4.8.5-44.0.3.el7 will be an update ---> Package libgcc.x86_64 0:4.8.3-9.el7 will be updated ---> Package libgcc.x86_64 0:4.8.5-44.0.3.el7 will be an update ---> Package libgomp.x86_64 0:4.8.3-9.el7 will be updated ---> Package libgomp.x86_64 0:4.8.5-44.0.3.el7 will be an update --> Finished Dependency Resolution Dependencies Resolved ========================================================================================================================================================== Package Arch Version Repository Size ========================================================================================================================================================== Installing: oracle-rdbms-server-11gR2-preinstall x86_64 1.0-6.el7 ol7_latest 22 k Installing for dependencies: compat-libcap1 x86_64 1.10-7.el7 ol7_latest 17 k compat-libstdc++-33 x86_64 3.2.3-72.el7 ol7_latest 190 k gcc-c++ x86_64 4.8.5-44.0.3.el7 ol7_latest 7.2 M ksh x86_64 20120801-142.0.1.el7 ol7_latest 882 k libaio-devel x86_64 0.3.109-13.el7 ol7_latest 12 k libstdc++-devel x86_64 4.8.5-44.0.3.el7 ol7_latest 1.5 M Updating for dependencies: cpp x86_64 4.8.5-44.0.3.el7 ol7_latest 5.9 M gcc x86_64 4.8.5-44.0.3.el7 ol7_latest 16 M libaio x86_64 0.3.109-13.el7 ol7_latest 24 k libgcc x86_64 4.8.5-44.0.3.el7 ol7_latest 103 k libgomp x86_64 4.8.5-44.0.3.el7 ol7_latest 159 k libstdc++ x86_64 4.8.5-44.0.3.el7 ol7_latest 306 k Transaction Summary ========================================================================================================================================================== Install 1 Package (+6 Dependent packages) Upgrade ( 6 Dependent packages) Total download size: 32 M Is this ok [y/d/N]: y Downloading packages: No Presto metadata available for ol7_latest warning: /var/cache/yum/x86_64/7Server/ol7_latest/packages/compat-libcap1-1.10-7.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEYTA Public key for compat-libcap1-1.10-7.el7.x86_64.rpm is not installed (1/13): compat-libcap1-1.10-7.el7.x86_64.rpm | 17 kB 00:00:00 (2/13): cpp-4.8.5-44.0.3.el7.x86_64.rpm | 5.9 MB 00:00:01 (3/13): compat-libstdc++-33-3.2.3-72.el7.x86_64.rpm | 190 kB 00:00:05 (4/13): gcc-4.8.5-44.0.3.el7.x86_64.rpm | 16 MB 00:00:04 (5/13): ksh-20120801-142.0.1.el7.x86_64.rpm | 882 kB 00:00:00 (6/13): libaio-0.3.109-13.el7.x86_64.rpm | 24 kB 00:00:00 (7/13): libaio-devel-0.3.109-13.el7.x86_64.rpm | 12 kB 00:00:00 (8/13): libgcc-4.8.5-44.0.3.el7.x86_64.rpm | 103 kB 00:00:00 (9/13): libgomp-4.8.5-44.0.3.el7.x86_64.rpm | 159 kB 00:00:00 (10/13): libstdc++-4.8.5-44.0.3.el7.x86_64.rpm | 306 kB 00:00:00 (11/13): libstdc++-devel-4.8.5-44.0.3.el7.x86_64.rpm | 1.5 MB 00:00:00 (12/13): oracle-rdbms-server-11gR2-preinstall-1.0-6.el7.x86_64.rpm | 22 kB 00:00:01 (13/13): gcc-c++-4.8.5-44.0.3.el7.x86_64.rpm | 7.2 MB 00:00:04 ---------------------------------------------------------------------------------------------------------------------------------------------------------- Total 3.3 MB/s | 32 MB 00:00:09 Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle Importing GPG key 0xEC551F03: Userid : "Oracle OSS group (Open Source Software group) <[email protected]>" Fingerprint: 4214 4123 fecf c55b 9086 313d 72f9 7b74 ec55 1f03 Package : 7:oraclelinux-release-7.1-1.0.5.el7.x86_64 (@anaconda/7.1) From : /etc/pki/rpm-gpg/RPM-GPG-KEY-oracle Is this ok [y/N]: y Running transaction check Running transaction test Transaction test succeeded Running transaction Updating : libgcc-4.8.5-44.0.3.el7.x86_64 1/19 Updating : libstdc++-4.8.5-44.0.3.el7.x86_64 2/19 Installing : libstdc++-devel-4.8.5-44.0.3.el7.x86_64 3/19 Updating : libaio-0.3.109-13.el7.x86_64 4/19 Installing : libaio-devel-0.3.109-13.el7.x86_64 5/19 Installing : compat-libstdc++-33-3.2.3-72.el7.x86_64 6/19 Updating : cpp-4.8.5-44.0.3.el7.x86_64 7/19 Installing : ksh-20120801-142.0.1.el7.x86_64 8/19 Installing : compat-libcap1-1.10-7.el7.x86_64 9/19 Updating : libgomp-4.8.5-44.0.3.el7.x86_64 10/19 Updating : gcc-4.8.5-44.0.3.el7.x86_64 11/19 Installing : gcc-c++-4.8.5-44.0.3.el7.x86_64 12/19 Installing : oracle-rdbms-server-11gR2-preinstall-1.0-6.el7.x86_64 13/19 Cleanup : gcc-4.8.3-9.el7.x86_64 14/19 Cleanup : libstdc++-4.8.3-9.el7.x86_64 15/19 Cleanup : libgcc-4.8.3-9.el7.x86_64 16/19 Cleanup : cpp-4.8.3-9.el7.x86_64 17/19 Cleanup : libgomp-4.8.3-9.el7.x86_64 18/19 Cleanup : libaio-0.3.109-12.el7.x86_64 19/19 Verifying : gcc-4.8.5-44.0.3.el7.x86_64 1/19 Verifying : libaio-0.3.109-13.el7.x86_64 2/19 Verifying : oracle-rdbms-server-11gR2-preinstall-1.0-6.el7.x86_64 3/19 Verifying : libgomp-4.8.5-44.0.3.el7.x86_64 4/19 Verifying : compat-libstdc++-33-3.2.3-72.el7.x86_64 5/19 Verifying : gcc-c++-4.8.5-44.0.3.el7.x86_64 6/19 Verifying : compat-libcap1-1.10-7.el7.x86_64 7/19 Verifying : libaio-devel-0.3.109-13.el7.x86_64 8/19 Verifying : libstdc++-devel-4.8.5-44.0.3.el7.x86_64 9/19 Verifying : libgcc-4.8.5-44.0.3.el7.x86_64 10/19 Verifying : libstdc++-4.8.5-44.0.3.el7.x86_64 11/19 Verifying : ksh-20120801-142.0.1.el7.x86_64 12/19 Verifying : cpp-4.8.5-44.0.3.el7.x86_64 13/19 Verifying : gcc-4.8.3-9.el7.x86_64 14/19 Verifying : cpp-4.8.3-9.el7.x86_64 15/19 Verifying : libgomp-4.8.3-9.el7.x86_64 16/19 Verifying : libstdc++-4.8.3-9.el7.x86_64 17/19 Verifying : libgcc-4.8.3-9.el7.x86_64 18/19 Verifying : libaio-0.3.109-12.el7.x86_64 19/19 Installed: oracle-rdbms-server-11gR2-preinstall.x86_64 0:1.0-6.el7 Dependency Installed: compat-libcap1.x86_64 0:1.10-7.el7 compat-libstdc++-33.x86_64 0:3.2.3-72.el7 gcc-c++.x86_64 0:4.8.5-44.0.3.el7 ksh.x86_64 0:20120801-142.0.1.el7 libaio-devel.x86_64 0:0.3.109-13.el7 libstdc++-devel.x86_64 0:4.8.5-44.0.3.el7 Dependency Updated: cpp.x86_64 0:4.8.5-44.0.3.el7 gcc.x86_64 0:4.8.5-44.0.3.el7 libaio.x86_64 0:0.3.109-13.el7 libgcc.x86_64 0:4.8.5-44.0.3.el7 libgomp.x86_64 0:4.8.5-44.0.3.el7 libstdc++.x86_64 0:4.8.5-44.0.3.el7 Complete!
3.2. Manual Setup:
4. Additional Setup:
Hosts File:
[root@racnode1 ~]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 #::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 #RAC # Public IP 192.168.56.110 racnode1.localdomain racnode1 192.168.56.111 racnode2.localdomain racnode2 #Private IP 192.168.10.110 racnode1-priv.localdomain racnode1-priv 192.168.10.111 racnode2-priv.localdomain racnode2-priv #VIP 192.168.56.112 racnode1-vip.localdomain racnode1-vip 192.168.56.113 racnode2-vip.localdomain racnode2-vip #SCAN IP 192.168.56.114 racnode-scan.localdomain racnode-scan 192.168.56.115 racnode-scan.localdomain racnode-scan 192.168.56.116 racnode-scan.localdomain racnode-scan
Set password for oracle:
[root@racnode1 ~]# id oracle uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),54322(dba) [root@racnode1 ~]# passwd oracle Changing password for user oracle. New password: BAD PASSWORD: The password is shorter than 8 characters Retype new password: passwd: all authentication tokens updated successfully.
Set secure Linux to permissive:
[root@racnode1 ~]# cat /etc/selinux/config | grep permissive SELINUX=permissive
Disable Firewall:
[root@racnode1 ~]# systemctl stop firewalld [root@racnode1 ~]# systemctl disable firewalld
To create home directory for GI and Oracle DB, enter the following commands:
[root@racnode1 ~]# mkdir -p /u01/app/11.2.0/grid [root@racnode1 ~]# chown -R oracle:oinstall /u01/app/11.2.0/grid [root@racnode1 ~]# chmod -R 775 /u01/app/11.2.0/grid [root@racnode1 ~]# mkdir -p /u01/app/oracle [root@racnode1 ~]# chown -R oracle:oinstall /u01/app/oracle [root@racnode1 ~]# mkdir /u01/app/oracle/cfgtoollogs [root@racnode1 ~]# chown -R oracle:oinstall /u01/app/oracle/cfgtoollogs [root@racnode1 ~]# chmod -R 775 /u01/app/oracle [root@racnode1 ~]# mkdir -p /u01/app/oraInventory [root@racnode1 ~]# chown -R oracle:oinstall /u01/app/oraInventory [root@racnode1 ~]# chmod -R 775 /u01/app/oraInventory [root@racnode1 ~]# mkdir -p /u01/app/oracle/product/11.2.0/dbhome_1 [root@racnode1 ~]# chown -R oracle:oinstall /u01/app/oracle/product/11.2.0/dbhome_1 [root@racnode1 ~]# chmod -R 775 /u01/app/oracle/product/11.2.0/dbhome_1
Install Oracle ASMLib:
[root@racnode1 software]# rpm -Uvh oracleasmlib-2.0.12-1.el7.x86_64.rpm Preparing... ################################# [100%] Updating / installing... 1:oracleasmlib-2.0.12-1.el7 ################################# [100%] [root@racnode1 software]# rpm -Uvh oracleasm-support-2.1.8-3.el7.x86_64.rpm Preparing... ################################# [100%] Updating / installing... 1:oracleasm-support-2.1.8-3.el7 ################################# [100%] Note: Forwarding request to 'systemctl enable oracleasm.service'. ln -s '/usr/lib/systemd/system/oracleasm.service' '/etc/systemd/system/multi-user.target.wants/oracleasm.service'
5.1. Create the disks and associate them to VM as virtual media:
VBoxManage createhd --filename "C:\VM\racnode1\asmdisk01.vdi" --size 7168 --format VDI --variant Fixed VBoxManage createhd --filename "C:\VM\racnode1\asmdisk02.vdi" --size 5120 --format VDI --variant Fixed VBoxManage createhd --filename "C:\VM\racnode1\asmdisk03.vdi" --size 5120 --format VDI --variant Fixed VBoxManage createhd --filename "C:\VM\racnode1\asmdisk04.vdi" --size 5120 --format VDI --variant Fixed
5.2. Convert Disk to Shareable:
VBoxManage modifyhd "C:\VM\racnode1\asmdisk01.vdi" --type shareable VBoxManage modifyhd "C:\VM\racnode1\asmdisk02.vdi" --type shareable VBoxManage modifyhd "C:\VM\racnode1\asmdisk03.vdi" --type shareable VBoxManage modifyhd "C:\VM\racnode1\asmdisk04.vdi" --type shareable
5.3. Connect Disk to VM:
VBoxManage storageattach racnode1 --storagectl "SATA" --port 1 --device 0 --type hdd --medium "C:\VM\racnode1\asmdisk01.vdi" --mtype shareable VBoxManage storageattach racnode1 --storagectl "SATA" --port 2 --device 0 --type hdd --medium "C:\VM\racnode1\asmdisk02.vdi" --mtype shareable VBoxManage storageattach racnode1 --storagectl "SATA" --port 3 --device 0 --type hdd --medium "C:\VM\racnode1\asmdisk03.vdi" --mtype shareable VBoxManage storageattach racnode1 --storagectl "SATA" --port 4 --device 0 --type hdd --medium "C:\VM\racnode1\asmdisk04.vdi" --mtype shareable
5.4. Format disk for Creating ASM:
[root@racnode1 dev]# ls sd* sda sda1 sda2 sdb sdc sdd sde [root@racnode1 dev]# fdisk /dev/sdb Welcome to fdisk (util-linux 2.23.2). Changes will remain in memory only, until you decide to write them. Be careful before using the write command. Device does not contain a recognized partition table Building a new DOS disklabel with disk identifier 0xeb05d744. Command (m for help): n Partition type: p primary (0 primary, 0 extended, 4 free) e extended Select (default p): p Partition number (1-4, default 1): First sector (2048-14680063, default 2048): Using default value 2048 Last sector, +sectors or +size{K,M,G} (2048-14680063, default 14680063): Using default value 14680063 Partition 1 of type Linux and of size 7 GiB is set Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks.
[root@racnode1 dev]# ls sd* sda sda1 sda2 sdb sdb1 sdc sdc1 sdd sdd1 sde sde1
5.5. Configure ASM Library Drive:
[root@racnode1 dev]# id oracle uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),54322(dba) [root@racnode1 dev]# oracleasm configure -i Configuring the Oracle ASM library driver. This will configure the on-boot properties of the Oracle ASM library driver. The following questions will determine whether the driver is loaded on boot and what permissions it will have. The current values will be shown in brackets ('[]'). Hitting <ENTER> without typing an answer will keep that current value. Ctrl-C will abort. Default user to own the driver interface []: oracle Default group to own the driver interface []: dba Scan for Oracle ASM disks on boot (y/n) [y]: y Writing Oracle ASM library driver configuration: done
5.6. Create ASM Disks
[root@racnode1 dev]# /usr/sbin/oracleasm init [root@racnode1 dev]# /usr/sbin/oracleasm createdisk DISK_OCR /dev/sdb1 Writing disk header: done Instantiating disk: done [root@racnode1 dev]# /usr/sbin/oracleasm createdisk DISK_DATA01 /dev/sdc1 Writing disk header: done Instantiating disk: done [root@racnode1 dev]# /usr/sbin/oracleasm createdisk DISK_DATA02 /dev/sdd1 Writing disk header: done Instantiating disk: done [root@racnode1 dev]# /usr/sbin/oracleasm createdisk DISK_DATA03 /dev/sde1 Writing disk header: done Instantiating disk: done
5.7. Validate ASM Disks:
[root@racnode1 dev]# /usr/sbin/oracleasm scandisks Reloading disk partitions: done Cleaning any stale ASM disks... Scanning system for ASM disks... [root@racnode1 dev]# /usr/sbin/oracleasm listdisks DISK_DATA01 DISK_DATA02 DISK_DATA03 DISK_OCR
6. Clone the Virtual Machine for Node2:
6.1. You can clone VM using command prompt or GUI.
C:\Users\samad>cd "c:\Program Files\Oracle\VirtualBox" VBoxManage clonehd "C:\VM\racnode1\racnode1.vdi" "C:\VM\racnode2\racnode2.vdi"
6.2. Add all shareable disks to Node 2.
VBoxManage storageattach racnode2 --storagectl "SATA" --port 1 --device 0 --type hdd --medium "C:\VM\racnode1\asmdisk01.vdi" --mtype shareable VBoxManage storageattach racnode2 --storagectl "SATA" --port 2 --device 0 --type hdd --medium "C:\VM\racnode1\asmdisk02.vdi" --mtype shareable VBoxManage storageattach racnode2 --storagectl "SATA" --port 3 --device 0 --type hdd --medium "C:\VM\racnode1\asmdisk03.vdi" --mtype shareable VBoxManage storageattach racnode2 --storagectl "SATA" --port 4 --device 0 --type hdd --medium "C:\VM\racnode1\asmdisk04.vdi" --mtype shareable
6.3. Change hostname and IP on Node2:
[root@racnode2 ~]# cat /etc/hostname
racnode2.localdomain
[root@racnode2 ~]# ifconfig
enp0s3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.56.111 netmask 255.255.255.0 broadcast 192.168.56.255
inet6 fe80::a00:27ff:fe66:f6f6 prefixlen 64 scopeid 0x20<link>
ether 08:00:27:66:f6:f6 txqueuelen 1000 (Ethernet)
RX packets 1854 bytes 124671 (121.7 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 214 bytes 32521 (31.7 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
enp0s8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.10.111 netmask 255.255.255.0 broadcast 192.168.10.255
inet6 fe80::a00:27ff:fed9:b5f5 prefixlen 64 scopeid 0x20<link>
ether 08:00:27:d9:b5:f5 txqueuelen 1000 (Ethernet)
RX packets 73 bytes 8901 (8.6 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 79 bytes 10209 (9.9 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
enp0s9: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.242 netmask 255.255.255.0 broadcast 192.168.1.255
inet6 fe80::a00:27ff:fe53:3e76 prefixlen 64 scopeid 0x20<link>
inet6 2600:1700:8930:4760:a00:27ff:fe53:3e76 prefixlen 128 scopeid 0x0
6.3. Verify ASM Disks on Node2:
[root@racnode2 ~]# oracleasm configure -i Configuring the Oracle ASM library driver. This will configure the on-boot properties of the Oracle ASM library driver. The following questions will determine whether the driver is loaded on boot and what permissions it will have. The current values will be shown in brackets ('[]'). Hitting <ENTER> without typing an answer will keep that current value. Ctrl-C will abort. Default user to own the driver interface []: oracle Default group to own the driver interface []: dba Scan for Oracle ASM disks on boot (y/n) [y]: Writing Oracle ASM library driver configuration: done
6.5. ASM Disks Validation:
[root@racnode2 ~]# oracleasm scandisks Reloading disk partitions: done Cleaning any stale ASM disks... Scanning system for ASM disks... Instantiating disk "DISK_OCR" Instantiating disk "DISK_DATA02" Instantiating disk "DISK_DATA03" Instantiating disk "DISK_DATA01" [root@racnode2 ~]# oracleasm listdisks DISK_DATA01 DISK_DATA02 DISK_DATA03 DISK_OCR
7. Install GI (Grid Infrastructure):
7.1 Trigger runInstaller from GUI.
[oracle@racnode1 software]$ ls -lrt total 1177108 -rw-r--r--. 1 oracle oinstall 1205251894 Nov 12 2018 p13390677_112040_Linux-x86-64_3of7.zip [oracle@racnode1 software]$ unzip p13390677_112040_Linux-x86-64_3of7.zip [oracle@racnode1 grid]$ pwd /u01/software/grid [oracle@racnode1 grid]$ ls install readme.html response rpm runcluvfy.sh runInstaller sshsetup stage welcome.html [oracle@racnode1 grid]$ ./runInstaller Starting Oracle Universal Installer... Checking Temp space: must be greater than 120 MB. Actual 29069 MB Passed Checking swap space: must be greater than 150 MB. Actual 3069 MB Passed Checking monitor: must be configured to display at least 256 colors
7.2. Installation Software show all option one by one.
Choose “Skip software updates” and Click Next…
Choose “Install and Configure Oracle Grid Infrastructure for a Cluster” and Click Next…
Choose “Advanced Installation” and Click Next…
Choose “Product Languages -> English” and Click on Next….
Provide Cluster Name and Scan Name with port. Click Next..
Click on ADD button and provide the Node2’s hostname and Virtual Hostname.
If already setup the SSH connectivity for oracle user then Click on Test otherwise Click on Setup. Once password less SSH configured or tested then click on Next…
Select Interface for public and Private. Click on Next..
Choose ASM option for OCR Storage and Click on Next…
Provide Disk Group Name for OCR and Voting Disk and Select Redundancy and Disk. Click Next…
Provide Password for sys and asmsnmp users. Click Next…
Choose don’t use IPMI and Click Next…
Provide Group for OSASM, OSDBA and OSOPER. Click Next…
Click Yes to confirm. Click Next…
Provide Oracle Base and Home Location for GRID. Click Next…
Click Next…
Prerequisite checks. Recommended to install all required packages and meet all recommendation for configured on real life database. Click on “Fix & Check Again”.
Generated fixup script.
Execute Script on all recommended nodes from root user. Then Click Ok.
[root@racnode2 cfgtoollogs]# /tmp/CVU_11.2.0.4.0_oracle/runfixup.sh Response file being used is :/tmp/CVU_11.2.0.4.0_oracle/fixup.response Enable file being used is :/tmp/CVU_11.2.0.4.0_oracle/fixup.enable Log file location: /tmp/CVU_11.2.0.4.0_oracle/orarun.log Installing Package /tmp/CVU_11.2.0.4.0_oracle//cvuqdisk-1.0.9-1.rpm Preparing... ################################# [100%] Updating / installing... 1:cvuqdisk-1.0.9-1 ################################# [100%] [root@racnode2 cfgtoollogs]# exit logout Connection to racnode2 closed. [root@racnode1 ~]# /tmp/CVU_11.2.0.4.0_oracle/runfixup.sh Response file being used is :/tmp/CVU_11.2.0.4.0_oracle/fixup.response Enable file being used is :/tmp/CVU_11.2.0.4.0_oracle/fixup.enable Log file location: /tmp/CVU_11.2.0.4.0_oracle/orarun.log Installing Package /tmp/CVU_11.2.0.4.0_oracle//cvuqdisk-1.0.9-1.rpm Preparing... ################################# [100%] Updating / installing... 1:cvuqdisk-1.0.9-1 ################################# [100%]
If you want to ignore, Checked “Ignore All” check box and Click Next…
Summary of all selected options. Click on Install to start installation otherwise Click on Back and Change the option.
Installation is progressing.
Execute below recommended script from root user on recommended nodes one by one.
Execute orainstRoot.sh script on Node1 & and Node2.
[root@racnode1 ~]# cd /u01/app/oraInventory/ [root@racnode1 oraInventory]# ./orainstRoot.sh Changing permissions of /u01/app/oraInventory. Adding read,write permissions for group. Removing read,write,execute permissions for world. Changing groupname of /u01/app/oraInventory to oinstall. The execution of the script is complete. [root@racnode2 ~]# cd /u01/app/oraInventory/ [root@racnode2 oraInventory]# ./orainstRoot.sh Changing permissions of /u01/app/oraInventory. Adding read,write permissions for group. Removing read,write,execute permissions for world. Changing groupname of /u01/app/oraInventory to oinstall. The execution of the script is complete.
Getting Error on root.sh:
Adding Clusterware entries to inittab ohasd failed to start Failed to start the Clusterware. Last 20 lines of the alert log follow: 2020-10-13 19:06:26.319: [client(12973)]CRS-2101:The OLR was formatted using version 3. /u01/app/11.2.0/grid/perl/bin/perl -I/u01/app/11.2.0/grid/perl/lib -I/u01/app/11.2.0/grid/crs/install /u01/app/11.2.0/grid/crs/install/rootcrs.pl execution failed
Bug for 11.2.0.4 on OEL 7:
This is the known issue for this bug need to apply patch before running root.sh script.
Installation walk-through – Oracle Grid/RAC 11.2.0.4 on Oracle Linux 7 (Doc ID 1951613.1)
Applying Patch (18370031) on all nodes before running the root.sh script:
[oracle@racnode2 OPatch]$ pwd /u01/app/11.2.0/grid/OPatch [oracle@racnode1 OPatch]$ ./opatch napply -local /u01/software/18370031 Oracle Interim Patch Installer version 11.2.0.3.4 Copyright (c) 2012, Oracle Corporation. All rights reserved. Oracle Home : /u01/app/11.2.0/grid Central Inventory : /u01/app/oraInventory from : /u01/app/11.2.0/grid/oraInst.loc OPatch version : 11.2.0.3.4 OUI version : 11.2.0.4.0 Log file location : /u01/app/11.2.0/grid/cfgtoollogs/opatch/opatch2020-10-13_22-37-14PM_1.log Verifying environment and performing prerequisite checks... OPatch continues with these patches: 18370031 Do you want to proceed? [y|n] y User Responded with: Y All checks passed. Please shutdown Oracle instances running out of this ORACLE_HOME on the local system. (Oracle Home = '/u01/app/11.2.0/grid') Is the local system ready for patching? [y|n] y User Responded with: Y Backing up files... Applying interim patch '18370031' to OH '/u01/app/11.2.0/grid' Patching component oracle.crs, 11.2.0.4.0... Verifying the update... Patch 18370031 successfully applied. Log file location: /u01/app/11.2.0/grid/cfgtoollogs/opatch/opatch2020-10-13_22-37-14PM_1.log OPatch succeeded. [oracle@racnode1 OPatch]$ ./opatch lsinventory Oracle Interim Patch Installer version 11.2.0.3.4 Copyright (c) 2012, Oracle Corporation. All rights reserved. Oracle Home : /u01/app/11.2.0/grid Central Inventory : /u01/app/oraInventory from : /u01/app/11.2.0/grid/oraInst.loc OPatch version : 11.2.0.3.4 OUI version : 11.2.0.4.0 Log file location : /u01/app/11.2.0/grid/cfgtoollogs/opatch/opatch2020-10-13_22-39-31PM_1.log Lsinventory Output file location : /u01/app/11.2.0/grid/cfgtoollogs/opatch/lsinv/lsinventory2020-10-13_22-39-31PM.txt -------------------------------------------------------------------------------- Installed Top-level Products (1): Oracle Grid Infrastructure 11g 11.2.0.4.0 There are 1 products installed in this Oracle Home. Interim patches (1) : Patch 18370031 : applied on Tue Oct 13 22:37:38 EDT 2020 Unique Patch ID: 17950825 Patch description: "Grid Infrastructure Patch Set Update : 11.2.0.4.x (gibugno)" Created on 15 Aug 2014, 16:14:40 hrs PST8PDT Bugs fixed: 18370031 Rac system comprising of multiple nodes Local node = racnode1 Remote node = racnode2 -------------------------------------------------------------------------------- OPatch succeeded.
Execute root.sh script on Node1 and Node2. Click Next…
[root@racnode1 ~]# cd /u01/app/11.2.0/grid [root@racnode1 grid]# ./root.sh Performing root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u01/app/11.2.0/grid Enter the full pathname of the local bin directory: [/usr/local/bin]: The contents of "dbhome" have not changed. No need to overwrite. The contents of "oraenv" have not changed. No need to overwrite. The contents of "coraenv" have not changed. No need to overwrite. Creating /etc/oratab file... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params Creating trace directory User ignored Prerequisites during installation Installing Trace File Analyzer OLR initialization - successful root wallet root wallet cert root cert export peer wallet profile reader wallet pa wallet peer wallet keys pa wallet keys peer cert request pa cert request peer cert pa cert peer root cert TP profile reader root cert TP pa root cert TP peer pa cert TP pa peer cert TP profile reader pa cert TP profile reader peer cert TP peer user cert pa user cert Adding Clusterware entries to oracle-ohasd.service CRS-2672: Attempting to start 'ora.mdnsd' on 'racnode1' CRS-2676: Start of 'ora.mdnsd' on 'racnode1' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'racnode1' CRS-2676: Start of 'ora.gpnpd' on 'racnode1' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'racnode1' CRS-2672: Attempting to start 'ora.gipcd' on 'racnode1' CRS-2676: Start of 'ora.cssdmonitor' on 'racnode1' succeeded CRS-2676: Start of 'ora.gipcd' on 'racnode1' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'racnode1' CRS-2672: Attempting to start 'ora.diskmon' on 'racnode1' CRS-2676: Start of 'ora.diskmon' on 'racnode1' succeeded CRS-2676: Start of 'ora.cssd' on 'racnode1' succeeded ASM created and started successfully. Disk Group OCR created successfully. clscfg: -install mode specified Successfully accumulated necessary OCR keys. Creating OCR keys for user 'root', privgrp 'root'.. Operation successful. CRS-4256: Updating the profile Successful addition of voting disk 5a3d357651d04fefbf9fe121a4a7b459. Successfully replaced voting disk group with +OCR. CRS-4256: Updating the profile CRS-4266: Voting file(s) successfully replaced ## STATE File Universal Id File Name Disk group -- ----- ----------------- --------- --------- 1. ONLINE 5a3d357651d04fefbf9fe121a4a7b459 (ORCL:DISK_OCR) [OCR] Located 1 voting disk(s). CRS-2672: Attempting to start 'ora.asm' on 'racnode1' CRS-2676: Start of 'ora.asm' on 'racnode1' succeeded CRS-2672: Attempting to start 'ora.OCR.dg' on 'racnode1' CRS-2676: Start of 'ora.OCR.dg' on 'racnode1' succeeded Configure Oracle Grid Infrastructure for a Cluster ... succeeded #On Node2 [root@racnode2 grid]# ./root.sh Performing root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u01/app/11.2.0/grid Enter the full pathname of the local bin directory: [/usr/local/bin]: The contents of "dbhome" have not changed. No need to overwrite. The contents of "oraenv" have not changed. No need to overwrite. The contents of "coraenv" have not changed. No need to overwrite. Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params Creating trace directory User ignored Prerequisites during installation Installing Trace File Analyzer OLR initialization - successful Adding Clusterware entries to oracle-ohasd.service CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node racnode1, number 1, and is terminating An active cluster was found during exclusive startup, restarting to join the cluster Configure Oracle Grid Infrastructure for a Cluster ... succeeded
Once Executed all script on all nodes, Click Ok on above window.
Monitor Installation process.
You may ignore below failed message. Click Ok.
Installation Processes are completed Successfully. Click On Close.
7.3. Clusterware Status:
[root@racnode1 grid]# crsctl stat res -t -------------------------------------------------------------------------------- NAME TARGET STATE SERVER STATE_DETAILS -------------------------------------------------------------------------------- Local Resources -------------------------------------------------------------------------------- ora.LISTENER.lsnr ONLINE ONLINE racnode1 ONLINE ONLINE racnode2 ora.OCR.dg ONLINE ONLINE racnode1 ONLINE ONLINE racnode2 ora.asm ONLINE ONLINE racnode1 Started ONLINE ONLINE racnode2 Started ora.gsd OFFLINE OFFLINE racnode1 OFFLINE OFFLINE racnode2 ora.net1.network ONLINE ONLINE racnode1 ONLINE ONLINE racnode2 ora.ons ONLINE ONLINE racnode1 ONLINE ONLINE racnode2 -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE racnode2 ora.LISTENER_SCAN2.lsnr 1 ONLINE ONLINE racnode1 ora.LISTENER_SCAN3.lsnr 1 ONLINE ONLINE racnode1 ora.cvu 1 ONLINE ONLINE racnode1 ora.oc4j 1 ONLINE ONLINE racnode1 ora.racnode1.vip 1 ONLINE ONLINE racnode1 ora.racnode2.vip 1 ONLINE ONLINE racnode2 ora.scan1.vip 1 ONLINE ONLINE racnode2 ora.scan2.vip 1 ONLINE ONLINE racnode1 ora.scan3.vip 1 ONLINE ONLINE racnode1
8. Configure ASM: Create ASM DiskGroup
8. Install Oracle Database Binary and Create Database on RAC:
Below links will show how to install Oracle Software on GI/RAC. This link is for another setup and processes are same. So you may see different host name on this example.
Oracle Database Software Installation on RAC: Installation Processes
Create Database using DBCA on RAC: Create Database
Or,
Follow following steps to install and create oracle database on RAC.
Unzip Oracle Software:
[oracle@racnode1 software]$ ls -lrt total 2658168 -rw-r--r-- 1 oracle oinstall 1395582860 Nov 12 2018 p13390677_112040_Linux-x86-64_1of7.zip -rw-r--r-- 1 oracle oinstall 1151304589 Nov 12 2018 p13390677_112040_Linux-x86-64_2of7.zip [oracle@racnode1 software]$ unzip p13390677_112040_Linux-x86-64_1of7.zip [oracle@racnode1 software]$ unzip p13390677_112040_Linux-x86-64_2of7.zip
Trigger runInstaller from GUI.
[oracle@racnode1 database]$ pwd /u01/software/database [oracle@racnode1 database]$ ls -lrt total 48 drwxr-xr-x 2 root root 28 Aug 27 2013 sshsetup -rwxr-xr-x 1 root root 3267 Aug 27 2013 runInstaller drwxr-xr-x 2 root root 33 Aug 27 2013 rpm drwxr-xr-x 2 root root 58 Aug 27 2013 response -rw-r--r-- 1 root root 30016 Aug 27 2013 readme.html drwxr-xr-x 14 root root 4096 Aug 27 2013 stage -rw-r--r-- 1 root root 500 Aug 27 2013 welcome.html drwxr-xr-x 4 root root 4096 Aug 27 2013 install [oracle@racnode1 database]$ ./runInstaller Starting Oracle Universal Installer... Checking Temp space: must be greater than 120 MB. Actual 18630 MB Passed Checking swap space: must be greater than 150 MB. Actual 3071 MB Passed Checking monitor: must be configured to display at least 256 colors Passed
Uncheck the receiving email option and Click Next…
Confirm by clicking Yes and Click Next…
Choose “Create and Configure a Database” and Click Next…
Choose “Server Class” and Click Next…
Choose “Oracle Real Application Cluster database installation” and Select all nodes. If Oracle Database user is different then GI (i.e. GI owner is grid and database owner is oracle) then Click on “SSH Connectivity…” and setup password less ssh.
Click on Next…
Choose Advanced install option and Click on Next…
Select Product Language (most of the cases it will be English). Click Next…
Choose EE and Click on Next…
Provide Oracle Base and Home location. Click Next…
Choose the configuration type and Click on Next…
Provide Oracle Global and SID name. Click Next…
Choose memory for the database instance. You may change later also but need downtime.
Choose correct Character Sets for your application. Once you created database, you will not able to change it later. Click Next…
Click Next…
Choose ASM for Storage and provide password for ASMSNMP. Click Next…
Usually DBA set cron job for backup.
Choose ASM DiskGroup for Database Creation. Click Next.
Provide password. Click Next..
Assign the OS Group and Click Next..
Prerequisites check. Try to resolve all prerequisites specially on real life database. As these were warning I ignored these by checking the “Ignore All” check box. Click Next…
Summary of all option chosen. If you are ok, then clink Install otherwise clink back and change the options.
Installation is progressing. You may monitor log by clicking on Details.
Following error for known bug and you can ignore it by Clicking on Continue.
From Log:
INFO: /u01/app/oracle/product/11.2.0/dbhome_1/sysman/lib/ins_emagent.mk:113: warning: overriding recipe for target `nmosudo' /u01/app/oracle/product/11.2.0/dbhome_1/sysman/lib/ins_emagent.mk:52: warning: ignoring old recipe for target `nmosudo' INFO: /usr/bin/ld: warning: -z lazyload ignored. /usr/bin/ld: warning: -z nolazyload ignored. /usr/bin/ld: /u01/app/oracle/product/11.2.0/dbhome_1/sysman/lib//libnmectl.a(nmectlt.o): undefined reference to symbol 'B_DestroyKeyObject' /usr/bin/ld: note: 'B_DestroyKeyObject' is defined in DSO /u01/app/oracle/product/11.2.0/dbhome_1/lib/libnnz11.so so try adding it to the linker command line /u01/app/oracle/product/11.2.0/dbhome_1/lib/libnnz11.so: could not read symbols: Invalid operation collect2: error: ld returne INFO: d 1 exit status INFO: make[1]: Leaving directory `/u01/app/oracle/product/11.2.0/dbhome_1/sysman/lib' INFO: make[1]: *** [/u01/app/oracle/product/11.2.0/dbhome_1/sysman/lib/emdctl] Error 1 make: *** [emdctl] Error 2 INFO: End output from spawned process. INFO: ---------------------------------- INFO: Exception thrown from action: make Exception Name: MakefileException Exception String: Error in invoking target 'agent nmhs' of makefile '/u01/app/oracle/product/11.2.0/dbhome_1/sysman/lib/ins_emagent.mk'. See '/u01/app/oraInventory/logs/installActions2020-10-15_12-13-05AM.log' for details. Exception Severity: 1
Details from Oracle Metalink: Doc ID 1965691.1
Installation is Progressing.
Database is creating.
EM can start by running following command manually.
Run following script from root user on all nodes. After Executed script, Click OK.
Database Creation is completed successfully. Click Close.
Database Validation:
[oracle@racnode1 ~]$ srvctl status database -d orcl Instance orcl1 is running on node racnode1 Instance orcl2 is running on node racnode2 SQL> SELECT NAME, OPEN_MODE, INSTANCE_NAME, HOST_NAME FROM GV$DATABASE D, GV$INSTANCE I WHERE D.INST_ID=I.INST_ID; NAME OPEN_MODE INSTANCE_NAME HOST_NAME --------- -------------------- ---------------- ------------------------------ ORCL READ WRITE orcl1 racnode1.localdomain ORCL READ WRITE orcl2 racnode2.localdomain
11. Issues:
Issue 1: [client(12973)]CRS-2101:The OLR was formatted using version 3.
This is the known issue for 11.2.0.4 for OEL 7 and need to apply patch before running root.sh script.
Adding Clusterware entries to inittab ohasd failed to start Failed to start the Clusterware. Last 20 lines of the alert log follow: 2020-10-13 19:06:26.319: [client(12973)]CRS-2101:The OLR was formatted using version 3.
Installation walk-through – Oracle Grid/RAC 11.2.0.4 on Oracle Linux 7 (Doc ID 1951613.1)
Applying Patch (18370031) on both nodes before running the root.sh script:
Issue 2:
On Node 2, got the following issue while running root.sh:
[root@racnode2 grid]# ./root.sh Performing root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u01/app/11.2.0/grid Enter the full pathname of the local bin directory: [/usr/local/bin]: The contents of "dbhome" have not changed. No need to overwrite. The contents of "oraenv" have not changed. No need to overwrite. The contents of "coraenv" have not changed. No need to overwrite. Creating /etc/oratab file... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params Creating trace directory User ignored Prerequisites during installation Installing Trace File Analyzer OLR initialization - successful Adding Clusterware entries to oracle-ohasd.service CRS-2672: Attempting to start 'ora.mdnsd' on 'racnode2' CRS-2676: Start of 'ora.mdnsd' on 'racnode2' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'racnode2' CRS-2676: Start of 'ora.gpnpd' on 'racnode2' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'racnode2' CRS-2672: Attempting to start 'ora.gipcd' on 'racnode2' CRS-2676: Start of 'ora.cssdmonitor' on 'racnode2' succeeded CRS-2676: Start of 'ora.gipcd' on 'racnode2' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'racnode2' CRS-2672: Attempting to start 'ora.diskmon' on 'racnode2' CRS-2676: Start of 'ora.diskmon' on 'racnode2' succeeded CRS-2676: Start of 'ora.cssd' on 'racnode2' succeeded Disk Group OCR creation failed with the following message: ORA-15018: diskgroup cannot be created ORA-15031: disk specification 'ORCL:DISK_OCR' matches no disks Configuration of ASM ... failed
Reason: ASMLib configuration was not correct on Node 2.
Solution:
[root@racnode2 ~]# oracleasm configure -i Configuring the Oracle ASM library driver. This will configure the on-boot properties of the Oracle ASM library driver. The following questions will determine whether the driver is loaded on boot and what permissions it will have. The current values will be shown in brackets ('[]'). Hitting <ENTER> without typing an answer will keep that current value. Ctrl-C will abort. Default user to own the driver interface [oracle]: Default group to own the driver interface [dba]: Scan for Oracle ASM disks on boot (y/n) [y]: Writing Oracle ASM library driver configuration: done
Issue 3:
During RDBMS software installation, got below error.
From Log:
INFO: /u01/app/oracle/product/11.2.0/dbhome_1/sysman/lib/ins_emagent.mk:113: warning: overriding recipe for target `nmosudo' /u01/app/oracle/product/11.2.0/dbhome_1/sysman/lib/ins_emagent.mk:52: warning: ignoring old recipe for target `nmosudo' INFO: /usr/bin/ld: warning: -z lazyload ignored. /usr/bin/ld: warning: -z nolazyload ignored. /usr/bin/ld: /u01/app/oracle/product/11.2.0/dbhome_1/sysman/lib//libnmectl.a(nmectlt.o): undefined reference to symbol 'B_DestroyKeyObject' /usr/bin/ld: note: 'B_DestroyKeyObject' is defined in DSO /u01/app/oracle/product/11.2.0/dbhome_1/lib/libnnz11.so so try adding it to the linker command line /u01/app/oracle/product/11.2.0/dbhome_1/lib/libnnz11.so: could not read symbols: Invalid operation collect2: error: ld returne INFO: d 1 exit status INFO: make[1]: Leaving directory `/u01/app/oracle/product/11.2.0/dbhome_1/sysman/lib' INFO: make[1]: *** [/u01/app/oracle/product/11.2.0/dbhome_1/sysman/lib/emdctl] Error 1 make: *** [emdctl] Error 2 INFO: End output from spawned process. INFO: ---------------------------------- INFO: Exception thrown from action: make Exception Name: MakefileException Exception String: Error in invoking target 'agent nmhs' of makefile '/u01/app/oracle/product/11.2.0/dbhome_1/sysman/lib/ins_emagent.mk'. See '/u01/app/oraInventory/logs/installActions2020-10-15_12-13-05AM.log' for details. Exception Severity: 1
Solution:
- 01) Ignore the undefined symbol error during Oracle 11.2.0.4 installation and proceed further. The software installation will succeed without any further errors.
- 02) Download and install patch 19692824
Details on Oracle Metalink Doc-> Doc ID 1965691.1