Oracle Database 11g Release 2 RAC On Oracle Linux 7.1

Oracle RAC (Real Application Cluster) is used for complex application to increase database performance, high availability and scalability.

Below high level steps will perform to install/configure Oracle RAC:

1. Configure VM for 2 Nodes RAC. Create VM

2. Install Oracle Linux 7.1. OEL 7.1

3. Oracle Installation Prerequisites:

Perform either the Automatic Setup or the Manual Setup to complete the basic prerequisites. The Additional Setup is required for all installations.

3.1. Automatic Setup:

All necessary prerequisites will be performed automatically.

[root@racnode1 ~]# yum install oracle-rdbms-server-11gR2-preinstall
Loaded plugins: langpacks
Resolving Dependencies
--> Running transaction check
---> Package oracle-rdbms-server-11gR2-preinstall.x86_64 0:1.0-6.el7 will be installed
--> Processing Dependency: gcc-c++ for package: oracle-rdbms-server-11gR2-preinstall-1.0-6.el7.x86_64
--> Processing Dependency: compat-libcap1 for package: oracle-rdbms-server-11gR2-preinstall-1.0-6.el7.x86_64
--> Processing Dependency: ksh for package: oracle-rdbms-server-11gR2-preinstall-1.0-6.el7.x86_64
--> Processing Dependency: libaio-devel for package: oracle-rdbms-server-11gR2-preinstall-1.0-6.el7.x86_64
--> Processing Dependency: compat-libstdc++-33 for package: oracle-rdbms-server-11gR2-preinstall-1.0-6.el7.x86_64
--> Processing Dependency: libstdc++-devel for package: oracle-rdbms-server-11gR2-preinstall-1.0-6.el7.x86_64
--> Running transaction check
---> Package compat-libcap1.x86_64 0:1.10-7.el7 will be installed
---> Package compat-libstdc++-33.x86_64 0:3.2.3-72.el7 will be installed
---> Package gcc-c++.x86_64 0:4.8.5-44.0.3.el7 will be installed
--> Processing Dependency: libstdc++ = 4.8.5-44.0.3.el7 for package: gcc-c++-4.8.5-44.0.3.el7.x86_64
--> Processing Dependency: gcc = 4.8.5-44.0.3.el7 for package: gcc-c++-4.8.5-44.0.3.el7.x86_64
---> Package ksh.x86_64 0:20120801-142.0.1.el7 will be installed
---> Package libaio-devel.x86_64 0:0.3.109-13.el7 will be installed
--> Processing Dependency: libaio(x86-64) = 0.3.109-13.el7 for package: libaio-devel-0.3.109-13.el7.x86_64
---> Package libstdc++-devel.x86_64 0:4.8.5-44.0.3.el7 will be installed
--> Running transaction check
---> Package gcc.x86_64 0:4.8.3-9.el7 will be updated
---> Package gcc.x86_64 0:4.8.5-44.0.3.el7 will be an update
--> Processing Dependency: libgomp = 4.8.5-44.0.3.el7 for package: gcc-4.8.5-44.0.3.el7.x86_64
--> Processing Dependency: cpp = 4.8.5-44.0.3.el7 for package: gcc-4.8.5-44.0.3.el7.x86_64
--> Processing Dependency: libgcc >= 4.8.5-44.0.3.el7 for package: gcc-4.8.5-44.0.3.el7.x86_64
---> Package libaio.x86_64 0:0.3.109-12.el7 will be updated
---> Package libaio.x86_64 0:0.3.109-13.el7 will be an update
---> Package libstdc++.x86_64 0:4.8.3-9.el7 will be updated
---> Package libstdc++.x86_64 0:4.8.5-44.0.3.el7 will be an update
--> Running transaction check
---> Package cpp.x86_64 0:4.8.3-9.el7 will be updated
---> Package cpp.x86_64 0:4.8.5-44.0.3.el7 will be an update
---> Package libgcc.x86_64 0:4.8.3-9.el7 will be updated
---> Package libgcc.x86_64 0:4.8.5-44.0.3.el7 will be an update
---> Package libgomp.x86_64 0:4.8.3-9.el7 will be updated
---> Package libgomp.x86_64 0:4.8.5-44.0.3.el7 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

==========================================================================================================================================================
 Package                                               Arch                    Version                                  Repository                   Size
==========================================================================================================================================================
Installing:
 oracle-rdbms-server-11gR2-preinstall                  x86_64                  1.0-6.el7                                ol7_latest                   22 k
Installing for dependencies:
 compat-libcap1                                        x86_64                  1.10-7.el7                               ol7_latest                   17 k
 compat-libstdc++-33                                   x86_64                  3.2.3-72.el7                             ol7_latest                  190 k
 gcc-c++                                               x86_64                  4.8.5-44.0.3.el7                         ol7_latest                  7.2 M
 ksh                                                   x86_64                  20120801-142.0.1.el7                     ol7_latest                  882 k
 libaio-devel                                          x86_64                  0.3.109-13.el7                           ol7_latest                   12 k
 libstdc++-devel                                       x86_64                  4.8.5-44.0.3.el7                         ol7_latest                  1.5 M
Updating for dependencies:
 cpp                                                   x86_64                  4.8.5-44.0.3.el7                         ol7_latest                  5.9 M
 gcc                                                   x86_64                  4.8.5-44.0.3.el7                         ol7_latest                   16 M
 libaio                                                x86_64                  0.3.109-13.el7                           ol7_latest                   24 k
 libgcc                                                x86_64                  4.8.5-44.0.3.el7                         ol7_latest                  103 k
 libgomp                                               x86_64                  4.8.5-44.0.3.el7                         ol7_latest                  159 k
 libstdc++                                             x86_64                  4.8.5-44.0.3.el7                         ol7_latest                  306 k

Transaction Summary
==========================================================================================================================================================
Install  1 Package  (+6 Dependent packages)
Upgrade             ( 6 Dependent packages)

Total download size: 32 M
Is this ok [y/d/N]: y
Downloading packages:
No Presto metadata available for ol7_latest
warning: /var/cache/yum/x86_64/7Server/ol7_latest/packages/compat-libcap1-1.10-7.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEYTA
Public key for compat-libcap1-1.10-7.el7.x86_64.rpm is not installed
(1/13): compat-libcap1-1.10-7.el7.x86_64.rpm                                                                                       |  17 kB  00:00:00
(2/13): cpp-4.8.5-44.0.3.el7.x86_64.rpm                                                                                            | 5.9 MB  00:00:01
(3/13): compat-libstdc++-33-3.2.3-72.el7.x86_64.rpm                                                                                | 190 kB  00:00:05
(4/13): gcc-4.8.5-44.0.3.el7.x86_64.rpm                                                                                            |  16 MB  00:00:04
(5/13): ksh-20120801-142.0.1.el7.x86_64.rpm                                                                                        | 882 kB  00:00:00
(6/13): libaio-0.3.109-13.el7.x86_64.rpm                                                                                           |  24 kB  00:00:00
(7/13): libaio-devel-0.3.109-13.el7.x86_64.rpm                                                                                     |  12 kB  00:00:00
(8/13): libgcc-4.8.5-44.0.3.el7.x86_64.rpm                                                                                         | 103 kB  00:00:00
(9/13): libgomp-4.8.5-44.0.3.el7.x86_64.rpm                                                                                        | 159 kB  00:00:00
(10/13): libstdc++-4.8.5-44.0.3.el7.x86_64.rpm                                                                                     | 306 kB  00:00:00
(11/13): libstdc++-devel-4.8.5-44.0.3.el7.x86_64.rpm                                                                               | 1.5 MB  00:00:00
(12/13): oracle-rdbms-server-11gR2-preinstall-1.0-6.el7.x86_64.rpm                                                                 |  22 kB  00:00:01
(13/13): gcc-c++-4.8.5-44.0.3.el7.x86_64.rpm                                                                                       | 7.2 MB  00:00:04
----------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                     3.3 MB/s |  32 MB  00:00:09
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
Importing GPG key 0xEC551F03:
 Userid     : "Oracle OSS group (Open Source Software group) <[email protected]>"
 Fingerprint: 4214 4123 fecf c55b 9086 313d 72f9 7b74 ec55 1f03
 Package    : 7:oraclelinux-release-7.1-1.0.5.el7.x86_64 (@anaconda/7.1)
 From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
Is this ok [y/N]: y
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Updating   : libgcc-4.8.5-44.0.3.el7.x86_64                                                                                                        1/19
  Updating   : libstdc++-4.8.5-44.0.3.el7.x86_64                                                                                                     2/19
  Installing : libstdc++-devel-4.8.5-44.0.3.el7.x86_64                                                                                               3/19
  Updating   : libaio-0.3.109-13.el7.x86_64                                                                                                          4/19
  Installing : libaio-devel-0.3.109-13.el7.x86_64                                                                                                    5/19
  Installing : compat-libstdc++-33-3.2.3-72.el7.x86_64                                                                                               6/19
  Updating   : cpp-4.8.5-44.0.3.el7.x86_64                                                                                                           7/19
  Installing : ksh-20120801-142.0.1.el7.x86_64                                                                                                       8/19
  Installing : compat-libcap1-1.10-7.el7.x86_64                                                                                                      9/19
  Updating   : libgomp-4.8.5-44.0.3.el7.x86_64                                                                                                      10/19
  Updating   : gcc-4.8.5-44.0.3.el7.x86_64                                                                                                          11/19
  Installing : gcc-c++-4.8.5-44.0.3.el7.x86_64                                                                                                      12/19
  Installing : oracle-rdbms-server-11gR2-preinstall-1.0-6.el7.x86_64                                                                                13/19
  Cleanup    : gcc-4.8.3-9.el7.x86_64                                                                                                               14/19
  Cleanup    : libstdc++-4.8.3-9.el7.x86_64                                                                                                         15/19
  Cleanup    : libgcc-4.8.3-9.el7.x86_64                                                                                                            16/19
  Cleanup    : cpp-4.8.3-9.el7.x86_64                                                                                                               17/19
  Cleanup    : libgomp-4.8.3-9.el7.x86_64                                                                                                           18/19
  Cleanup    : libaio-0.3.109-12.el7.x86_64                                                                                                         19/19
  Verifying  : gcc-4.8.5-44.0.3.el7.x86_64                                                                                                           1/19
  Verifying  : libaio-0.3.109-13.el7.x86_64                                                                                                          2/19
  Verifying  : oracle-rdbms-server-11gR2-preinstall-1.0-6.el7.x86_64                                                                                 3/19
  Verifying  : libgomp-4.8.5-44.0.3.el7.x86_64                                                                                                       4/19
  Verifying  : compat-libstdc++-33-3.2.3-72.el7.x86_64                                                                                               5/19
  Verifying  : gcc-c++-4.8.5-44.0.3.el7.x86_64                                                                                                       6/19
  Verifying  : compat-libcap1-1.10-7.el7.x86_64                                                                                                      7/19
  Verifying  : libaio-devel-0.3.109-13.el7.x86_64                                                                                                    8/19
  Verifying  : libstdc++-devel-4.8.5-44.0.3.el7.x86_64                                                                                               9/19
  Verifying  : libgcc-4.8.5-44.0.3.el7.x86_64                                                                                                       10/19
  Verifying  : libstdc++-4.8.5-44.0.3.el7.x86_64                                                                                                    11/19
  Verifying  : ksh-20120801-142.0.1.el7.x86_64                                                                                                      12/19
  Verifying  : cpp-4.8.5-44.0.3.el7.x86_64                                                                                                          13/19
  Verifying  : gcc-4.8.3-9.el7.x86_64                                                                                                               14/19
  Verifying  : cpp-4.8.3-9.el7.x86_64                                                                                                               15/19
  Verifying  : libgomp-4.8.3-9.el7.x86_64                                                                                                           16/19
  Verifying  : libstdc++-4.8.3-9.el7.x86_64                                                                                                         17/19
  Verifying  : libgcc-4.8.3-9.el7.x86_64                                                                                                            18/19
  Verifying  : libaio-0.3.109-12.el7.x86_64                                                                                                         19/19

Installed:
  oracle-rdbms-server-11gR2-preinstall.x86_64 0:1.0-6.el7

Dependency Installed:
  compat-libcap1.x86_64 0:1.10-7.el7    compat-libstdc++-33.x86_64 0:3.2.3-72.el7  gcc-c++.x86_64 0:4.8.5-44.0.3.el7  ksh.x86_64 0:20120801-142.0.1.el7
  libaio-devel.x86_64 0:0.3.109-13.el7  libstdc++-devel.x86_64 0:4.8.5-44.0.3.el7

Dependency Updated:
  cpp.x86_64 0:4.8.5-44.0.3.el7         gcc.x86_64 0:4.8.5-44.0.3.el7           libaio.x86_64 0:0.3.109-13.el7     libgcc.x86_64 0:4.8.5-44.0.3.el7
  libgomp.x86_64 0:4.8.5-44.0.3.el7     libstdc++.x86_64 0:4.8.5-44.0.3.el7

Complete!

3.2. Manual Setup:

4. Additional Setup:

Hosts File:

[root@racnode1 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
#::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

#RAC

# Public IP
192.168.56.110          racnode1.localdomain    racnode1
192.168.56.111          racnode2.localdomain    racnode2

#Private IP
192.168.10.110          racnode1-priv.localdomain    racnode1-priv
192.168.10.111          racnode2-priv.localdomain    racnode2-priv

#VIP
192.168.56.112          racnode1-vip.localdomain    racnode1-vip
192.168.56.113          racnode2-vip.localdomain    racnode2-vip

#SCAN IP
192.168.56.114          racnode-scan.localdomain    racnode-scan
192.168.56.115          racnode-scan.localdomain    racnode-scan
192.168.56.116          racnode-scan.localdomain    racnode-scan

Set password for oracle:

[root@racnode1 ~]# id oracle
uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),54322(dba)
[root@racnode1 ~]# passwd oracle
Changing password for user oracle.
New password:
BAD PASSWORD: The password is shorter than 8 characters
Retype new password:
passwd: all authentication tokens updated successfully.

Set secure Linux to permissive:

[root@racnode1 ~]# cat /etc/selinux/config | grep permissive
SELINUX=permissive

Disable Firewall:

[root@racnode1 ~]# systemctl stop firewalld
[root@racnode1 ~]# systemctl disable firewalld

To create home directory for GI and Oracle DB, enter the following commands:

[root@racnode1 ~]# mkdir -p /u01/app/11.2.0/grid
[root@racnode1 ~]# chown -R oracle:oinstall /u01/app/11.2.0/grid
[root@racnode1 ~]# chmod -R 775 /u01/app/11.2.0/grid

[root@racnode1 ~]# mkdir -p /u01/app/oracle
[root@racnode1 ~]# chown -R oracle:oinstall /u01/app/oracle

[root@racnode1 ~]# mkdir /u01/app/oracle/cfgtoollogs
[root@racnode1 ~]# chown -R oracle:oinstall /u01/app/oracle/cfgtoollogs

[root@racnode1 ~]# chmod -R 775 /u01/app/oracle
[root@racnode1 ~]# mkdir -p /u01/app/oraInventory
[root@racnode1 ~]# chown -R oracle:oinstall /u01/app/oraInventory
[root@racnode1 ~]# chmod -R 775 /u01/app/oraInventory

[root@racnode1 ~]# mkdir -p /u01/app/oracle/product/11.2.0/dbhome_1
[root@racnode1 ~]# chown -R oracle:oinstall /u01/app/oracle/product/11.2.0/dbhome_1
[root@racnode1 ~]# chmod -R 775 /u01/app/oracle/product/11.2.0/dbhome_1

Install Oracle ASMLib:

[root@racnode1 software]# rpm -Uvh oracleasmlib-2.0.12-1.el7.x86_64.rpm
Preparing...                          ################################# [100%]
Updating / installing...
   1:oracleasmlib-2.0.12-1.el7        ################################# [100%]

[root@racnode1 software]# rpm -Uvh oracleasm-support-2.1.8-3.el7.x86_64.rpm
Preparing...                          ################################# [100%]
Updating / installing...
   1:oracleasm-support-2.1.8-3.el7    ################################# [100%]
Note: Forwarding request to 'systemctl enable oracleasm.service'.
ln -s '/usr/lib/systemd/system/oracleasm.service' '/etc/systemd/system/multi-user.target.wants/oracleasm.service'

5. Create Shared Disk:

5.1. Create the disks and associate them to VM as virtual media:

VBoxManage createhd --filename "C:\VM\racnode1\asmdisk01.vdi" --size 7168 --format VDI --variant Fixed
VBoxManage createhd --filename "C:\VM\racnode1\asmdisk02.vdi" --size 5120 --format VDI --variant Fixed
VBoxManage createhd --filename "C:\VM\racnode1\asmdisk03.vdi" --size 5120 --format VDI --variant Fixed
VBoxManage createhd --filename "C:\VM\racnode1\asmdisk04.vdi" --size 5120 --format VDI --variant Fixed

5.2. Convert Disk to Shareable:

VBoxManage modifyhd "C:\VM\racnode1\asmdisk01.vdi" --type shareable
VBoxManage modifyhd "C:\VM\racnode1\asmdisk02.vdi" --type shareable
VBoxManage modifyhd "C:\VM\racnode1\asmdisk03.vdi" --type shareable
VBoxManage modifyhd "C:\VM\racnode1\asmdisk04.vdi" --type shareable

5.3. Connect Disk to VM:

VBoxManage storageattach racnode1 --storagectl "SATA" --port 1 --device 0 --type hdd --medium "C:\VM\racnode1\asmdisk01.vdi" --mtype shareable
VBoxManage storageattach racnode1 --storagectl "SATA" --port 2 --device 0 --type hdd --medium "C:\VM\racnode1\asmdisk02.vdi" --mtype shareable
VBoxManage storageattach racnode1 --storagectl "SATA" --port 3 --device 0 --type hdd --medium "C:\VM\racnode1\asmdisk03.vdi" --mtype shareable
VBoxManage storageattach racnode1 --storagectl "SATA" --port 4 --device 0 --type hdd --medium "C:\VM\racnode1\asmdisk04.vdi" --mtype shareable

5.4. Format disk for Creating ASM:

[root@racnode1 dev]# ls sd*
sda  sda1  sda2  sdb  sdc  sdd  sde

[root@racnode1 dev]# fdisk /dev/sdb
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0xeb05d744.

Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p): p
Partition number (1-4, default 1):
First sector (2048-14680063, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-14680063, default 14680063):
Using default value 14680063
Partition 1 of type Linux and of size 7 GiB is set

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
[root@racnode1 dev]# ls sd*
sda  sda1  sda2  sdb  sdb1  sdc  sdc1  sdd  sdd1  sde  sde1

5.5. Configure ASM Library Drive:

[root@racnode1 dev]# id oracle
uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),54322(dba)
[root@racnode1 dev]# oracleasm configure -i
Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver.  The following questions will determine whether the driver is
loaded on boot and what permissions it will have.  The current values
will be shown in brackets ('[]').  Hitting <ENTER> without typing an
answer will keep that current value.  Ctrl-C will abort.

Default user to own the driver interface []: oracle
Default group to own the driver interface []: dba
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done

5.6. Create ASM Disks

[root@racnode1 dev]# /usr/sbin/oracleasm init

[root@racnode1 dev]# /usr/sbin/oracleasm createdisk DISK_OCR /dev/sdb1
Writing disk header: done
Instantiating disk: done
[root@racnode1 dev]# /usr/sbin/oracleasm createdisk DISK_DATA01 /dev/sdc1
Writing disk header: done
Instantiating disk: done
[root@racnode1 dev]# /usr/sbin/oracleasm createdisk DISK_DATA02 /dev/sdd1
Writing disk header: done
Instantiating disk: done
[root@racnode1 dev]# /usr/sbin/oracleasm createdisk DISK_DATA03 /dev/sde1
Writing disk header: done
Instantiating disk: done

5.7. Validate ASM Disks:

[root@racnode1 dev]# /usr/sbin/oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
[root@racnode1 dev]# /usr/sbin/oracleasm listdisks
DISK_DATA01
DISK_DATA02
DISK_DATA03
DISK_OCR

6. Clone the Virtual Machine for Node2:

6.1. You can clone VM using command prompt or GUI.

C:\Users\samad>cd "c:\Program Files\Oracle\VirtualBox"
VBoxManage clonehd "C:\VM\racnode1\racnode1.vdi" "C:\VM\racnode2\racnode2.vdi"

6.2. Add all shareable disks to Node 2.

VBoxManage storageattach racnode2 --storagectl "SATA" --port 1 --device 0 --type hdd --medium "C:\VM\racnode1\asmdisk01.vdi" --mtype shareable
VBoxManage storageattach racnode2 --storagectl "SATA" --port 2 --device 0 --type hdd --medium "C:\VM\racnode1\asmdisk02.vdi" --mtype shareable
VBoxManage storageattach racnode2 --storagectl "SATA" --port 3 --device 0 --type hdd --medium "C:\VM\racnode1\asmdisk03.vdi" --mtype shareable
VBoxManage storageattach racnode2 --storagectl "SATA" --port 4 --device 0 --type hdd --medium "C:\VM\racnode1\asmdisk04.vdi" --mtype shareable

6.3. Change hostname and IP on Node2:

[root@racnode2 ~]# cat /etc/hostname
racnode2.localdomain

[root@racnode2 ~]# ifconfig
enp0s3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.56.111  netmask 255.255.255.0  broadcast 192.168.56.255
        inet6 fe80::a00:27ff:fe66:f6f6  prefixlen 64  scopeid 0x20<link>
        ether 08:00:27:66:f6:f6  txqueuelen 1000  (Ethernet)
        RX packets 1854  bytes 124671 (121.7 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 214  bytes 32521 (31.7 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

enp0s8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.10.111  netmask 255.255.255.0  broadcast 192.168.10.255
        inet6 fe80::a00:27ff:fed9:b5f5  prefixlen 64  scopeid 0x20<link>
        ether 08:00:27:d9:b5:f5  txqueuelen 1000  (Ethernet)
        RX packets 73  bytes 8901 (8.6 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 79  bytes 10209 (9.9 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

enp0s9: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.242  netmask 255.255.255.0  broadcast 192.168.1.255
        inet6 fe80::a00:27ff:fe53:3e76  prefixlen 64  scopeid 0x20<link>
        inet6 2600:1700:8930:4760:a00:27ff:fe53:3e76  prefixlen 128  scopeid 0x0     

6.3. Verify ASM Disks on Node2:

[root@racnode2 ~]# oracleasm configure -i
Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver.  The following questions will determine whether the driver is
loaded on boot and what permissions it will have.  The current values
will be shown in brackets ('[]').  Hitting <ENTER> without typing an
answer will keep that current value.  Ctrl-C will abort.

Default user to own the driver interface []: oracle
Default group to own the driver interface []: dba
Scan for Oracle ASM disks on boot (y/n) [y]:
Writing Oracle ASM library driver configuration: done

6.5. ASM Disks Validation:

[root@racnode2 ~]# oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
Instantiating disk "DISK_OCR"
Instantiating disk "DISK_DATA02"
Instantiating disk "DISK_DATA03"
Instantiating disk "DISK_DATA01"

[root@racnode2 ~]# oracleasm listdisks
DISK_DATA01
DISK_DATA02
DISK_DATA03
DISK_OCR

7. Install GI (Grid Infrastructure):

7.1 Trigger runInstaller from GUI.

[oracle@racnode1 software]$ ls -lrt
total 1177108
-rw-r--r--. 1 oracle oinstall 1205251894 Nov 12  2018 p13390677_112040_Linux-x86-64_3of7.zip

[oracle@racnode1 software]$ unzip p13390677_112040_Linux-x86-64_3of7.zip

[oracle@racnode1 grid]$ pwd
/u01/software/grid
[oracle@racnode1 grid]$ ls
install  readme.html  response  rpm  runcluvfy.sh  runInstaller  sshsetup  stage  welcome.html

[oracle@racnode1 grid]$ ./runInstaller
Starting Oracle Universal Installer...

Checking Temp space: must be greater than 120 MB.   Actual 29069 MB    Passed
Checking swap space: must be greater than 150 MB.   Actual 3069 MB    Passed
Checking monitor: must be configured to display at least 256 colors

7.2. Installation Software show all option one by one.

Choose “Skip software updates” and Click Next…

Choose “Install and Configure Oracle Grid Infrastructure for a Cluster” and Click Next…

Choose “Advanced Installation” and Click Next…

Choose “Product Languages -> English” and Click on Next….

Provide Cluster Name and Scan Name with port. Click Next..

Click on ADD button and provide the Node2’s hostname and Virtual Hostname.

If already setup the SSH connectivity for oracle user then Click on Test otherwise Click on Setup. Once password less SSH configured or tested then click on Next…

Select Interface for public and Private. Click on Next..

Choose ASM option for OCR Storage and Click on Next…

Provide Disk Group Name for OCR and Voting Disk and Select Redundancy and Disk. Click Next…

Provide Password for sys and asmsnmp users. Click Next…

Choose don’t use IPMI and Click Next…

Provide Group for OSASM, OSDBA and OSOPER. Click Next…

Click Yes to confirm. Click Next…

Provide Oracle Base and Home Location for GRID. Click Next…

Click Next…

Prerequisite checks. Recommended to install all required packages and meet all recommendation for configured on real life database. Click on “Fix & Check Again”.

Generated fixup script.

Execute Script on all recommended nodes from root user. Then Click Ok.

[root@racnode2 cfgtoollogs]# /tmp/CVU_11.2.0.4.0_oracle/runfixup.sh
Response file being used is :/tmp/CVU_11.2.0.4.0_oracle/fixup.response
Enable file being used is :/tmp/CVU_11.2.0.4.0_oracle/fixup.enable
Log file location: /tmp/CVU_11.2.0.4.0_oracle/orarun.log
Installing Package /tmp/CVU_11.2.0.4.0_oracle//cvuqdisk-1.0.9-1.rpm
Preparing...                          ################################# [100%]
Updating / installing...
   1:cvuqdisk-1.0.9-1                 ################################# [100%]
[root@racnode2 cfgtoollogs]# exit
logout
Connection to racnode2 closed.
[root@racnode1 ~]# /tmp/CVU_11.2.0.4.0_oracle/runfixup.sh
Response file being used is :/tmp/CVU_11.2.0.4.0_oracle/fixup.response
Enable file being used is :/tmp/CVU_11.2.0.4.0_oracle/fixup.enable
Log file location: /tmp/CVU_11.2.0.4.0_oracle/orarun.log
Installing Package /tmp/CVU_11.2.0.4.0_oracle//cvuqdisk-1.0.9-1.rpm
Preparing...                          ################################# [100%]
Updating / installing...
   1:cvuqdisk-1.0.9-1                 ################################# [100%]

If you want to ignore, Checked “Ignore All” check box and Click Next…

Summary of all selected options. Click on Install to start installation otherwise Click on Back and Change the option.

Installation is progressing.

Execute below recommended script from root user on recommended nodes one by one.

Execute orainstRoot.sh script on Node1 & and Node2.

[root@racnode1 ~]# cd /u01/app/oraInventory/
[root@racnode1 oraInventory]# ./orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.

[root@racnode2 ~]# cd /u01/app/oraInventory/
[root@racnode2 oraInventory]# ./orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.

Getting Error on root.sh:

Adding Clusterware entries to inittab
ohasd failed to start
Failed to start the Clusterware. Last 20 lines of the alert log follow:
2020-10-13 19:06:26.319:
[client(12973)]CRS-2101:The OLR was formatted using version 3.

/u01/app/11.2.0/grid/perl/bin/perl -I/u01/app/11.2.0/grid/perl/lib -I/u01/app/11.2.0/grid/crs/install /u01/app/11.2.0/grid/crs/install/rootcrs.pl execution failed

Bug for 11.2.0.4 on OEL 7:

This is the known issue for this bug need to apply patch before running root.sh script.

Installation walk-through – Oracle Grid/RAC 11.2.0.4 on Oracle Linux 7 (Doc ID 1951613.1)

Applying Patch (18370031) on all nodes before running the root.sh script:

[oracle@racnode2 OPatch]$ pwd
/u01/app/11.2.0/grid/OPatch

[oracle@racnode1 OPatch]$ ./opatch napply -local /u01/software/18370031
Oracle Interim Patch Installer version 11.2.0.3.4
Copyright (c) 2012, Oracle Corporation.  All rights reserved.


Oracle Home       : /u01/app/11.2.0/grid
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/11.2.0/grid/oraInst.loc
OPatch version    : 11.2.0.3.4
OUI version       : 11.2.0.4.0
Log file location : /u01/app/11.2.0/grid/cfgtoollogs/opatch/opatch2020-10-13_22-37-14PM_1.log

Verifying environment and performing prerequisite checks...
OPatch continues with these patches:   18370031

Do you want to proceed? [y|n]
y
User Responded with: Y
All checks passed.

Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
(Oracle Home = '/u01/app/11.2.0/grid')

Is the local system ready for patching? [y|n]
y
User Responded with: Y
Backing up files...
Applying interim patch '18370031' to OH '/u01/app/11.2.0/grid'

Patching component oracle.crs, 11.2.0.4.0...

Verifying the update...
Patch 18370031 successfully applied.
Log file location: /u01/app/11.2.0/grid/cfgtoollogs/opatch/opatch2020-10-13_22-37-14PM_1.log

OPatch succeeded.

[oracle@racnode1 OPatch]$ ./opatch lsinventory
Oracle Interim Patch Installer version 11.2.0.3.4
Copyright (c) 2012, Oracle Corporation.  All rights reserved.

Oracle Home       : /u01/app/11.2.0/grid
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/11.2.0/grid/oraInst.loc
OPatch version    : 11.2.0.3.4
OUI version       : 11.2.0.4.0
Log file location : /u01/app/11.2.0/grid/cfgtoollogs/opatch/opatch2020-10-13_22-39-31PM_1.log

Lsinventory Output file location : /u01/app/11.2.0/grid/cfgtoollogs/opatch/lsinv/lsinventory2020-10-13_22-39-31PM.txt

--------------------------------------------------------------------------------
Installed Top-level Products (1):

Oracle Grid Infrastructure 11g                                       11.2.0.4.0
There are 1 products installed in this Oracle Home.

Interim patches (1) :

Patch  18370031     : applied on Tue Oct 13 22:37:38 EDT 2020
Unique Patch ID:  17950825
Patch description:  "Grid Infrastructure Patch Set Update : 11.2.0.4.x (gibugno)"
   Created on 15 Aug 2014, 16:14:40 hrs PST8PDT
   Bugs fixed:
     18370031

Rac system comprising of multiple nodes
  Local node = racnode1
  Remote node = racnode2
--------------------------------------------------------------------------------
OPatch succeeded.

Execute root.sh script on Node1 and Node2. Click Next…

[root@racnode1 ~]# cd /u01/app/11.2.0/grid
[root@racnode1 grid]# ./root.sh
Performing root user operation for Oracle 11g

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/app/11.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.

Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
User ignored Prerequisites during installation
Installing Trace File Analyzer
OLR initialization - successful
  root wallet
  root wallet cert
  root cert export
  peer wallet
  profile reader wallet
  pa wallet
  peer wallet keys
  pa wallet keys
  peer cert request
  pa cert request
  peer cert
  pa cert
  peer root cert TP
  profile reader root cert TP
  pa root cert TP
  peer pa cert TP
  pa peer cert TP
  profile reader pa cert TP
  profile reader peer cert TP
  peer user cert
  pa user cert
Adding Clusterware entries to oracle-ohasd.service
CRS-2672: Attempting to start 'ora.mdnsd' on 'racnode1'
CRS-2676: Start of 'ora.mdnsd' on 'racnode1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'racnode1'
CRS-2676: Start of 'ora.gpnpd' on 'racnode1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'racnode1'
CRS-2672: Attempting to start 'ora.gipcd' on 'racnode1'
CRS-2676: Start of 'ora.cssdmonitor' on 'racnode1' succeeded
CRS-2676: Start of 'ora.gipcd' on 'racnode1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'racnode1'
CRS-2672: Attempting to start 'ora.diskmon' on 'racnode1'
CRS-2676: Start of 'ora.diskmon' on 'racnode1' succeeded
CRS-2676: Start of 'ora.cssd' on 'racnode1' succeeded

ASM created and started successfully.

Disk Group OCR created successfully.

clscfg: -install mode specified
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-4256: Updating the profile
Successful addition of voting disk 5a3d357651d04fefbf9fe121a4a7b459.
Successfully replaced voting disk group with +OCR.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   5a3d357651d04fefbf9fe121a4a7b459 (ORCL:DISK_OCR) [OCR]
Located 1 voting disk(s).
CRS-2672: Attempting to start 'ora.asm' on 'racnode1'
CRS-2676: Start of 'ora.asm' on 'racnode1' succeeded
CRS-2672: Attempting to start 'ora.OCR.dg' on 'racnode1'
CRS-2676: Start of 'ora.OCR.dg' on 'racnode1' succeeded
Configure Oracle Grid Infrastructure for a Cluster ... succeeded

#On Node2

[root@racnode2 grid]# ./root.sh
Performing root user operation for Oracle 11g

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/app/11.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
User ignored Prerequisites during installation
Installing Trace File Analyzer
OLR initialization - successful
Adding Clusterware entries to oracle-ohasd.service
CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node racnode1, number 1, and is terminating
An active cluster was found during exclusive startup, restarting to join the cluster
Configure Oracle Grid Infrastructure for a Cluster ... succeeded

Once Executed all script on all nodes, Click Ok on above window.

Monitor Installation process.

You may ignore below failed message. Click Ok.

Installation Processes are completed Successfully. Click On Close.

7.3. Clusterware Status:

[root@racnode1 grid]# crsctl stat res -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
               ONLINE  ONLINE       racnode1
               ONLINE  ONLINE       racnode2
ora.OCR.dg
               ONLINE  ONLINE       racnode1
               ONLINE  ONLINE       racnode2
ora.asm
               ONLINE  ONLINE       racnode1                 Started
               ONLINE  ONLINE       racnode2                 Started
ora.gsd
               OFFLINE OFFLINE      racnode1
               OFFLINE OFFLINE      racnode2
ora.net1.network
               ONLINE  ONLINE       racnode1
               ONLINE  ONLINE       racnode2
ora.ons
               ONLINE  ONLINE       racnode1
               ONLINE  ONLINE       racnode2
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       racnode2
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       racnode1
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       racnode1
ora.cvu
      1        ONLINE  ONLINE       racnode1
ora.oc4j
      1        ONLINE  ONLINE       racnode1
ora.racnode1.vip
      1        ONLINE  ONLINE       racnode1
ora.racnode2.vip
      1        ONLINE  ONLINE       racnode2
ora.scan1.vip
      1        ONLINE  ONLINE       racnode2
ora.scan2.vip
      1        ONLINE  ONLINE       racnode1
ora.scan3.vip
      1        ONLINE  ONLINE       racnode1

8. Configure ASM: Create ASM DiskGroup

8. Install Oracle Database Binary and Create Database on RAC:

Below links will show how to install Oracle Software on GI/RAC. This link is for another setup and processes are same. So you may see different host name on this example.

Oracle Database Software Installation on RAC: Installation Processes

Create Database using DBCA on RAC: Create Database

Or,

Follow following steps to install and create oracle database on RAC.

Unzip Oracle Software:

[oracle@racnode1 software]$ ls -lrt
total 2658168
-rw-r--r-- 1 oracle oinstall 1395582860 Nov 12  2018 p13390677_112040_Linux-x86-64_1of7.zip
-rw-r--r-- 1 oracle oinstall 1151304589 Nov 12  2018 p13390677_112040_Linux-x86-64_2of7.zip

[oracle@racnode1 software]$ unzip p13390677_112040_Linux-x86-64_1of7.zip
[oracle@racnode1 software]$ unzip p13390677_112040_Linux-x86-64_2of7.zip

Trigger runInstaller from GUI.

[oracle@racnode1 database]$ pwd
/u01/software/database
[oracle@racnode1 database]$ ls -lrt
total 48
drwxr-xr-x  2 root root    28 Aug 27  2013 sshsetup
-rwxr-xr-x  1 root root  3267 Aug 27  2013 runInstaller
drwxr-xr-x  2 root root    33 Aug 27  2013 rpm
drwxr-xr-x  2 root root    58 Aug 27  2013 response
-rw-r--r--  1 root root 30016 Aug 27  2013 readme.html
drwxr-xr-x 14 root root  4096 Aug 27  2013 stage
-rw-r--r--  1 root root   500 Aug 27  2013 welcome.html
drwxr-xr-x  4 root root  4096 Aug 27  2013 install
[oracle@racnode1 database]$ ./runInstaller
Starting Oracle Universal Installer...

Checking Temp space: must be greater than 120 MB.   Actual 18630 MB    Passed
Checking swap space: must be greater than 150 MB.   Actual 3071 MB    Passed
Checking monitor: must be configured to display at least 256 colors    Passed

Uncheck the receiving email option and Click Next…

Confirm by clicking Yes and Click Next…

Choose “Create and Configure a Database” and Click Next…

Choose “Server Class” and Click Next…

Choose “Oracle Real Application Cluster database installation” and Select all nodes. If Oracle Database user is different then GI (i.e. GI owner is grid and database owner is oracle) then Click on “SSH Connectivity…” and setup password less ssh.

Click on Next…

Choose Advanced install option and Click on Next…

Select Product Language (most of the cases it will be English). Click Next…

Choose EE and Click on Next…

Provide Oracle Base and Home location. Click Next…

Choose the configuration type and Click on Next…

Provide Oracle Global and SID name. Click Next…

Choose memory for the database instance. You may change later also but need downtime.

Choose correct Character Sets for your application. Once you created database, you will not able to change it later. Click Next…

Click Next…

Choose ASM for Storage and provide password for ASMSNMP. Click Next…

Usually DBA set cron job for backup.

Choose ASM DiskGroup for Database Creation. Click Next.

Provide password. Click Next..

Assign the OS Group and Click Next..

Prerequisites check. Try to resolve all prerequisites specially on real life database. As these were warning I ignored these by checking the “Ignore All” check box. Click Next…

Summary of all option chosen. If you are ok, then clink Install otherwise clink back and change the options.

Installation is progressing. You may monitor log by clicking on Details.

Following error for known bug and you can ignore it by Clicking on Continue.

From Log:

INFO: /u01/app/oracle/product/11.2.0/dbhome_1/sysman/lib/ins_emagent.mk:113: warning: overriding recipe for target `nmosudo'
/u01/app/oracle/product/11.2.0/dbhome_1/sysman/lib/ins_emagent.mk:52: warning: ignoring old recipe for target `nmosudo'

INFO: /usr/bin/ld: warning: -z lazyload ignored.
/usr/bin/ld: warning: -z nolazyload ignored.
/usr/bin/ld: /u01/app/oracle/product/11.2.0/dbhome_1/sysman/lib//libnmectl.a(nmectlt.o): undefined reference to symbol 'B_DestroyKeyObject'
/usr/bin/ld: note: 'B_DestroyKeyObject' is defined in DSO /u01/app/oracle/product/11.2.0/dbhome_1/lib/libnnz11.so so try adding it to the linker command line
/u01/app/oracle/product/11.2.0/dbhome_1/lib/libnnz11.so: could not read symbols: Invalid operation
collect2: error: ld returne
INFO: d 1 exit status

INFO: make[1]: Leaving directory `/u01/app/oracle/product/11.2.0/dbhome_1/sysman/lib'

INFO: make[1]: *** [/u01/app/oracle/product/11.2.0/dbhome_1/sysman/lib/emdctl] Error 1
make: *** [emdctl] Error 2

INFO: End output from spawned process.
INFO: ----------------------------------
INFO: Exception thrown from action: make
Exception Name: MakefileException
Exception String: Error in invoking target 'agent nmhs' of makefile '/u01/app/oracle/product/11.2.0/dbhome_1/sysman/lib/ins_emagent.mk'. See '/u01/app/oraInventory/logs/installActions2020-10-15_12-13-05AM.log' for details.
Exception Severity: 1

Details from Oracle Metalink: Doc ID 1965691.1

Installation is Progressing.

Database is creating.

EM can start by running following command manually.

Run following script from root user on all nodes. After Executed script, Click OK.

Database Creation is completed successfully. Click Close.

Database Validation:

[oracle@racnode1 ~]$ srvctl status database -d orcl
Instance orcl1 is running on node racnode1
Instance orcl2 is running on node racnode2

SQL> SELECT NAME, OPEN_MODE, INSTANCE_NAME, HOST_NAME FROM GV$DATABASE D, GV$INSTANCE I 
WHERE D.INST_ID=I.INST_ID;

NAME      OPEN_MODE            INSTANCE_NAME    HOST_NAME
--------- -------------------- ---------------- ------------------------------
ORCL      READ WRITE           orcl1            racnode1.localdomain
ORCL      READ WRITE           orcl2            racnode2.localdomain

11. Issues:

Issue 1: [client(12973)]CRS-2101:The OLR was formatted using version 3.

This is the known issue for 11.2.0.4 for OEL 7 and need to apply patch before running root.sh script.

Adding Clusterware entries to inittab
ohasd failed to start
Failed to start the Clusterware. Last 20 lines of the alert log follow:
2020-10-13 19:06:26.319:
[client(12973)]CRS-2101:The OLR was formatted using version 3.

Installation walk-through – Oracle Grid/RAC 11.2.0.4 on Oracle Linux 7 (Doc ID 1951613.1)

Applying Patch (18370031) on both nodes before running the root.sh script:

Issue 2:

On Node 2, got the following issue while running root.sh:

[root@racnode2 grid]# ./root.sh
Performing root user operation for Oracle 11g

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/app/11.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.


Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
User ignored Prerequisites during installation
Installing Trace File Analyzer
OLR initialization - successful
Adding Clusterware entries to oracle-ohasd.service
CRS-2672: Attempting to start 'ora.mdnsd' on 'racnode2'
CRS-2676: Start of 'ora.mdnsd' on 'racnode2' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'racnode2'
CRS-2676: Start of 'ora.gpnpd' on 'racnode2' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'racnode2'
CRS-2672: Attempting to start 'ora.gipcd' on 'racnode2'
CRS-2676: Start of 'ora.cssdmonitor' on 'racnode2' succeeded
CRS-2676: Start of 'ora.gipcd' on 'racnode2' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'racnode2'
CRS-2672: Attempting to start 'ora.diskmon' on 'racnode2'
CRS-2676: Start of 'ora.diskmon' on 'racnode2' succeeded
CRS-2676: Start of 'ora.cssd' on 'racnode2' succeeded

Disk Group OCR creation failed with the following message:
ORA-15018: diskgroup cannot be created
ORA-15031: disk specification 'ORCL:DISK_OCR' matches no disks


Configuration of ASM ... failed

Reason: ASMLib configuration was not correct on Node 2.

Solution:

[root@racnode2 ~]# oracleasm configure -i
Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver.  The following questions will determine whether the driver is
loaded on boot and what permissions it will have.  The current values
will be shown in brackets ('[]').  Hitting <ENTER> without typing an
answer will keep that current value.  Ctrl-C will abort.

Default user to own the driver interface [oracle]:
Default group to own the driver interface [dba]:
Scan for Oracle ASM disks on boot (y/n) [y]:
Writing Oracle ASM library driver configuration: done

Issue 3:

During RDBMS software installation, got below error.

From Log:

INFO: /u01/app/oracle/product/11.2.0/dbhome_1/sysman/lib/ins_emagent.mk:113: warning: overriding recipe for target `nmosudo'
/u01/app/oracle/product/11.2.0/dbhome_1/sysman/lib/ins_emagent.mk:52: warning: ignoring old recipe for target `nmosudo'

INFO: /usr/bin/ld: warning: -z lazyload ignored.
/usr/bin/ld: warning: -z nolazyload ignored.
/usr/bin/ld: /u01/app/oracle/product/11.2.0/dbhome_1/sysman/lib//libnmectl.a(nmectlt.o): undefined reference to symbol 'B_DestroyKeyObject'
/usr/bin/ld: note: 'B_DestroyKeyObject' is defined in DSO /u01/app/oracle/product/11.2.0/dbhome_1/lib/libnnz11.so so try adding it to the linker command line
/u01/app/oracle/product/11.2.0/dbhome_1/lib/libnnz11.so: could not read symbols: Invalid operation
collect2: error: ld returne
INFO: d 1 exit status

INFO: make[1]: Leaving directory `/u01/app/oracle/product/11.2.0/dbhome_1/sysman/lib'

INFO: make[1]: *** [/u01/app/oracle/product/11.2.0/dbhome_1/sysman/lib/emdctl] Error 1
make: *** [emdctl] Error 2

INFO: End output from spawned process.
INFO: ----------------------------------
INFO: Exception thrown from action: make
Exception Name: MakefileException
Exception String: Error in invoking target 'agent nmhs' of makefile '/u01/app/oracle/product/11.2.0/dbhome_1/sysman/lib/ins_emagent.mk'. See '/u01/app/oraInventory/logs/installActions2020-10-15_12-13-05AM.log' for details.
Exception Severity: 1

Solution:

  • 01) Ignore the undefined symbol error during Oracle 11.2.0.4 installation and proceed further. The software installation will succeed without any further errors.
  • 02) Download and install patch 19692824

Details on Oracle Metalink Doc-> Doc ID 1965691.1