Oracle RAC (Real Application Cluster) is used for complex application to increase database performance, high availability and scalability.
Below high level steps will perform to configure / install Oracle RAC:
- Configure 2 Virtual Machine (VM) for 2 Nodes RAC
- Install Oracle Linux 5.7
- Install GI (Grid Infrastructure)
- Configure ASM
- Install Oracle Database Binary
- Create Database on RAC
Configure 2 Virtual Machine (VM) for 2 Nodes RAC
Install GI (Grid Infrastructure)
Oracle GI and Database Installation Prerequisites:
Create Group and User – Oracle for DB (For RAC – recommended to use GRID user for GI)
Note::: But with this setup, we will use Oracle user for Database as well as GI
# From root user: /usr/sbin/groupadd -g 503 oinstall /usr/sbin/groupadd -g 504 asmadmin /usr/sbin/groupadd -g 505 asmdba /usr/sbin/groupadd -g 506 asmoper /usr/sbin/groupadd -g 507 dba /usr/sbin/groupadd -g 508 oper
[root@racdr1 ~]# /usr/sbin/useradd -u 501 -g oinstall -G dba,oper,asmdba,asmadmin,asmoper oracle [root@racdr1 ~]# id oracle uid=501(oracle) gid=503(oinstall) groups=503(oinstall),504(asmadmin),505(asmdba),506(asmoper),507(dba),508(oper) context=root:system_r:unconfined_t:SystemLow-SystemHigh
To create home directory for GI and Oracle DB, enter the following commands:
#From root user mkdir -p /u01/app/12.1.0/grid chown -R oracle:oinstall /u01/app/12.1.0/grid chmod -R 775 /u01/app/12.1.0/grid # To create the Oracle Base directory, enter the following commands: mkdir -p /u01/app/oracle mkdir /u01/app/oracle/cfgtoollogs chown -R oracle:oinstall /u01/app/oracle/cfgtoollogs chown -R oracle:oinstall /u01/app/oracle chmod -R 775 /u01/app/oracle # To create the Oracle Inventory directory, enter the following commands: mkdir -p /u01/app/oraInventory chown -R oracle:oinstall /u01/app/oraInventory chmod -R 775 /u01/app/oraInventory #To create the Oracle RDBMS Home directory, enter the following commands: mkdir -p /u01/app/oracle/product/12.1.0/dbhome_1 chown -R oracle:oinstall /u01/app/oracle/product/12.1.0/dbhome_1 chmod -R 775 /u01/app/oracle/product/12.1.0/dbhome_1
Network Setting:
- eth0 -> will be public IP
- eth1 -> will be private IP.
IP Setting on Ethernet:
eth0: Static IP: 192.168.56.220 Network: 255.255.255.0 eth1: Static IP: 192.168.2.220 Network: 255.255.254.0
/etc/hosts file should be same for both nodes.
[root@racdr1 software]# cat /etc/hosts # Do not remove the following line, or various programs # that require network functionality will fail. 127.0.0.1 localhost.localdomain localhost ::1 localhost6.localdomain6 localhost6 #DR #Public IP 192.168.56.220 racdr1.localdomain racdr1 192.168.56.221 racdr2.localdomain racdr2 #Private IP 192.168.2.220 racdr1-priv.localdomain racdr1-priv 192.168.2.221 racdr2-priv.localdomain racdr2-priv #VIP 192.168.56.222 racdr1-vip.localdomain racdr1-vip 192.168.56.223 racdr2-vip.localdomain racdr2-vip #SCAN IP 192.168.56.224 racdr-scan.localdomain racdr-scan 192.168.56.225 racdr-scan.localdomain racdr-scan 192.168.56.226 racdr-scan.localdomain racdr-scan #PROD # Public IP 192.168.56.120 ocmnode1.localdomain ocmnode1 192.168.56.121 ocmnode2.localdomain ocmnode2 #Private IP 192.168.2.120 ocmnode1-priv.localdomain ocmnode1-priv 192.168.2.121 ocmnode2-priv.localdomain ocmnode2-priv #VIP 192.168.56.122 ocmnode1-vip.localdomain ocmnode1-vip 192.168.56.123 ocmnode2-vip.localdomain ocmnode2-vip #SCAN IP 192.168.56.124 ocmnode-scan.localdomain ocmnode-scan 192.168.56.125 ocmnode-scan.localdomain ocmnode-scan 192.168.56.126 ocmnode-scan.localdomain ocmnode-scan
Ping on both nodes:
[root@racdr2 ~]# ping racdr1 [root@racdr1 ~]# ping racdr2 [root@racdr1 ~]# ping racdr2-vip [root@racdr2 ~]# ping racdr1-vip [root@racdr1 ~]# ping racdr-scan [root@racdr2 ~]# ping racdr-scan
Set Environment Variable for Oracle user on both Nodes:
$ vi /home/oracle/.bash_profile
# Oracle Settings
TMP=/tmp; export TMP
TMPDIR=$TMP; export TMPDIR
ORACLE_HOSTNAME=racdr1; export ORACLE_HOSTNAME
ORACLE_UNQNAME=ORCL; export ORACLE_UNQNAME
ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE
GRID_HOME=/u01/app/12.1.0/grid; export GRID_HOME
DB_HOME=$ORACLE_BASE/product/12.1.0/dbhome_1; export DB_HOME
ORACLE_HOME=$DB_HOME; export ORACLE_HOME
ORACLE_SID=ORCL1; export ORACLE_SID
ORACLE_TERM=xterm; export ORACLE_TERM
BASE_PATH=/usr/sbin:$PATH; export BASE_PATH
PATH=$ORACLE_HOME/bin:$BASE_PATH; export PATH
LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH
CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPATH
if [ $USER = "oracle" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
fi
alias grid_env='. /home/oracle/grid_env'
alias db_env='. /home/oracle/db_env'
Note::: For Node 2 : All will be same except – ORACLE_SID=ORCL2; export ORACLE_SID
$ vim /home/oracle/grid_env ORACLE_SID=+ASM1; export ORACLE_SID ORACLE_HOME=$GRID_HOME; export ORACLE_HOME PATH=$ORACLE_HOME/bin:$BASE_PATH; export PATH LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPATH
Note::: For Node 2 : All will be same except – ORACLE_SID=+ASM2; export ORACLE_SID
$ vim /home/oracle/db_env ORACLE_SID=ORCL1; export ORACLE_SID ORACLE_HOME=$DB_HOME; export ORACLE_HOME PATH=$ORACLE_HOME/bin:$BASE_PATH; export PATH LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPATH
Note::: For Node 2 : All will be same except – ORACLE_SID=ORCL2; export ORACLE_SID
OS Kernel Parameter Setting for Oracle:
Add following line to /etc/sysctl.conf
# Oracle kernel.shmmni = 4096 kernel.sem = 250 32000 100 128 fs.file-max = 6815744 net.ipv4.ip_local_port_range = 9000 65500 net.core.rmem_default = 262144 net.core.rmem_max = 4194304 net.core.wmem_default = 262144 net.core.wmem_max = 1048576
Verify Kernel Parameter:
$ /sbin/sysctl -p
Add the following lines to the /etc/security/limits.conf file:
grid soft nproc 2047 grid hard nproc 16384 grid soft nofile 1024 grid hard nofile 65536 oracle soft nproc 2047 oracle hard nproc 16384 oracle soft nofile 1024 oracle hard nofile 65536
Below mentioned packages need to install. You can install package either using yum (if you have installed) otherwise manually.
Package Installation by using yum repository:
yum install binutils-2.15.92.0.2 yum install compat-libstdc++-33-3.2.3 yum install elfutils-libelf-0.97 yum install elfutils-libelf-devel-0.97 yum install expat-1.95.7 yum install gcc-3.4.6 yum install gcc-c++-3.4.6 yum install glibc-2.3.4-2.41 yum install glibc-common-2.3.4 yum install glibc-devel-2.3.4 yum install glibc-headers-2.3.4 yum install libaio-0.3.105 yum install libaio-devel-0.3.105 yum install libgcc-3.4.6 yum install libstdc++-3.4.6 yum install libstdc++-devel 3.4.6 yum install make-3.80 yum install pdksh-5.2.14 yum install sysstat-5.0.5 yum install unixODBC-2.2.11 yum install unixODBC-devel-2.2.11 yum install compat-libstdc++-33-3.2.3 yum install glibc-2.3.4-2.41 yum install libaio-0.3.105 yum install libaio-devel-0.3.105 yum install libgcc-3.4.6 yum install libstdc++-3.4.6 yum install unixODBC-2.2.11
Install package by using rmp command:
ISO Disk Mount:
[root@racdr1 ~]# mkdir -p /mnt/disk [root@racdr1 ~]# mount -o loop /u01/software/V31120-01.iso /mnt/disk
Use below command if yum repository doesn’t setup.
$cd /mnt/disk/Server $ rpm -Uvh binutils-2.* rpm -Uvh compat-libstdc++-33* rpm -Uvh elfutils-libelf-0.* rpm -Uvh elfutils-libelf-devel-* rpm -Uvh gcc-4.* rpm -Uvh gcc-c++-4.* rpm -Uvh glibc-2.* rpm -Uvh glibc-common-2.* rpm -Uvh glibc-devel-2.* rpm -Uvh glibc-headers-2.* rpm -Uvh ksh-2* rpm -Uvh libaio-0.* rpm -Uvh libaio-devel-0.* rpm -Uvh libgcc-4.* rpm -Uvh libstdc++-4.* rpm -Uvh libstdc++-devel-4.* rpm -Uvh make-3.* rpm -Uvh sysstat-7.* rpm -Uvh unixODBC-2.* rpm -Uvh unixODBC-devel-2.*
The following command can be run on the system to list the currently installed packages:
Command to verify installed package list:
[root@racdr1 Server]# rpm -q --qf '%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n' binutils \
compat-libstdc++-33 \
elfutils-libelf \
elfutils-libelf-devel \
gcc \
gcc-c++ \
glibc \
glibc-common \
glibc-devel \
glibc-headers \
ksh \
libaio \
libaio-devel \
libgcc \
libstdc++ \
libstdc++-devel \
make \
sysstat \
unixODBC \
unixODBC-devel
OUTPUT: (Show all list and find which packages are not installed. binutils-2.17.50.0.6-20.el5 (x86_64) compat-libstdc++-33-3.2.3-61 (x86_64) compat-libstdc++-33-3.2.3-61 (i386) elfutils-libelf-0.137-3.el5 (x86_64) package elfutils-libelf-devel is not installed gcc-4.1.2-52.el5 (x86_64) package gcc-c++ is not installed glibc-2.5-81 (x86_64) glibc-2.5-81 (i686) glibc-common-2.5-81 (x86_64) glibc-devel-2.5-81 (x86_64) glibc-headers-2.5-81 (x86_64) ksh-20100621-5.el5 (x86_64) libaio-0.3.106-5 (x86_64) libaio-0.3.106-5 (i386) libaio-devel-0.3.106-5 (x86_64) libaio-devel-0.3.106-5 (i386) libgcc-4.1.2-52.el5 (x86_64) libgcc-4.1.2-52.el5 (i386) libstdc++-4.1.2-52.el5 (x86_64) libstdc++-4.1.2-52.el5 (i386) libstdc++-devel-4.1.2-52.el5 (x86_64) libstdc++-devel-4.1.2-52.el5 (i386) make-3.81-3.el5 (x86_64) sysstat-7.0.2-11.el5 (x86_64) package unixODBC is not installed package unixODBC-devel is not installed
Install Oracle ASM related package:
[root@racdr1 software]# rpm -Uvh oracleasmlib-2.0.4-1.el5.x86_64.rpm warning: oracleasmlib-2.0.4-1.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159 Preparing... ########################################### [100%] 1:oracleasmlib ########################################### [100%] [root@racdr1 software]# rpm -Uvh oracleasm-support-2.1.3-1.el5.x86_64.rpm warning: oracleasm-support-2.1.3-1.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159 Preparing... ########################################### [100%] 1:oracleasm-support ########################################### [100%] [root@racdr1 software]# rpm -Uvh oracleasm-2.6.18-164.el5-2.0.5-1.el5.x86_64* warning: oracleasm-2.6.18-164.el5-2.0.5-1.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159 Preparing... ########################################### [100%] 1:oracleasm-2.6.18-164.el########################################### [100%]
Configure Disk For ASM on RAC
Create necessary shareable disks for ASM. How to create shareable disk on VM
Create Partition on ROW Disk:
[root@racdr1 ~]# cd /dev/ [root@racdr1 dev]# ls sd* [root@racdr1 dev]# ls sd* sda sda2 sda4 sda6 sdb sdd sdf sdh sda1 sda3 sda5 sda7 sdc sde sdg sdi
To verify which ROW disk are already mapping with filesystem: use ($ mount or $df -h) command.
[root@racdr1 dev]# mount
[root@racdr1 dev]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda5 2.0G 486M 1.4G 26% /
/dev/mapper/VolGroup00-LogVol00
70G 3.8G 62G 6% /u01
/dev/sda6 494M 11M 458M 3% /home
/dev/sda3 2.0G 68M 1.9G 4% /tmp
/dev/sda2 3.9G 1.9G 1.9G 51% /usr
/dev/sda1 99M 22M 73M 23% /boot
tmpfs 1.5G 0 1.5G 0% /dev/shm
/dev/sr0 3.7G 3.7G 0 100% /media/OL5.8 x86_64 dvd 20120229
User fdisk command to format a row disk. Follow below steps for each shareable disks (sdb to sdi).
[root@racdr1 dev]# fdisk /dev/sdb Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won't be recoverable. Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-522, default 1): Using default value 1 Last cylinder or +size or +sizeM or +sizeK (1-522, default 522): Using default value 522 Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks.
After partitioning all disk it will show like below:
[root@racdr1 dev]# ls /dev/sd* /dev/sda /dev/sda2 /dev/sda4 /dev/sda6 /dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf /dev/sdg /dev/sdh /dev/sdi /dev/sda1 /dev/sda3 /dev/sda5 /dev/sda7 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdh1 /dev/sdi1
Create ASM Disk:
[root@racdr1 dev]# oracleasm configure -i
Configuring the Oracle ASM library driver.
This will configure the on-boot properties of the Oracle ASM library driver. The following questions will determine whether the driver is loaded on boot and what permissions it will have. The current values will be shown in brackets ('[]'). Hitting <ENTER> without typing an
answer will keep that current value. Ctrl-C will abort.
Default user to own the driver interface []: oracle
Default group to own the driver interface []: dba
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done
[root@racdr1 dev]# /usr/sbin/oracleasm init Creating /dev/oracleasm mount point: /dev/oracleasm Loading module "oracleasm": oracleasm Mounting ASMlib driver filesystem: /dev/oracleasm
Run below command for all disks [sdb1 – sdi1]
[root@racdr1 dev]# oracleasm createdisk OCRDISK1 /dev/sdb1 Writing disk header: done Instantiating disk: done [root@racdr1 dev]# oracleasm createdisk OCRDISK2 /dev/sdc1 Writing disk header: done Instantiating disk: done [root@racdr1 dev]# oracleasm createdisk OCRDISK3 /dev/sdd1 Writing disk header: done Instantiating disk: done [root@racdr1 dev]# oracleasm createdisk DATADISK1 /dev/sde1 Writing disk header: done Instantiating disk: done [root@racdr1 dev]# oracleasm createdisk DATADISK2 /dev/sdf1 Writing disk header: done Instantiating disk: done [root@racdr1 dev]# oracleasm createdisk DATADISK3 /dev/sdg1 Writing disk header: done Instantiating disk: done [root@racdr1 dev]# oracleasm createdisk DATADISK4 /dev/sdh1 Writing disk header: done Instantiating disk: done [root@racdr1 dev]# oracleasm createdisk DATADISK5 /dev/sdi1 Writing disk header: done Instantiating disk: done
If need to delete any ASM disks, use below command:
Delete existing Disk: [root@racdr1 dev]# oracleasm deletedisk OCRDISK1 Clearing disk header: done Dropping disk: done [root@racdr1 dev]# oracleasm deletedisk OCRDISK2 Clearing disk header: done Dropping disk: done
Verify Disks:
[root@racdr1 dev]# /usr/sbin/oracleasm scandisks Reloading disk partitions: done Cleaning any stale ASM disks... Scanning system for ASM disks... [root@racdr1 dev]# /usr/sbin/oracleasm listdisks DATADISK1 DATADISK2 DATADISK3 DATADISK4 DATADISK5 OCRDISK1 OCRDISK2 OCRDISK3
This VM is almost ready. We can use this VM for Node 1 and Clone this VM for Node 2.
Note:::Once we have Node 2, We will check some prerequisites.
On Node 2:
Verify Disks:
[root@racdr1 dev]# /usr/sbin/oracleasm scandisks Reloading disk partitions: done Cleaning any stale ASM disks... Scanning system for ASM disks... [root@racdr1 dev]# /usr/sbin/oracleasm listdisks DATADISK1 DATADISK2 DATADISK3 DATADISK4 DATADISK5 OCRDISK1 OCRDISK2 OCRDISK3
Step 1: unzip GI binary file on one node
# From Oracle USER. If you have GRID user for GI, then run below command from GRID [oracle@racdr1 ~]$ cd /u01/software/ [oracle@racdr1 software]$ ls -lrt *grid* -rw-r--r-- 1 oracle oinstall 646972897 Feb 7 2019 linuxamd64_12102_grid_2of2.zip -rw-r--r-- 1 oracle oinstall 1747043545 Feb 7 2019 linuxamd64_12102_grid_1of2.zip [oracle@racdr1 software]$ unzip linuxamd64_12102_grid_1of2.zip [oracle@racdr1 software]$ unzip linuxamd64_12102_grid_2of2.zip
Step 2: run below command (runcluvfy.sh) for prerequisite check before final installation process start.
[oracle@racdr1 grid]$ pwd /u01/software/grid [oracle@racdr1 grid]$ ls install response rpm runcluvfy.sh runInstaller sshsetup stage welcome.html [oracle@racdr1 grid]$ ./runcluvfy.sh stage -pre crsinst -n racdr1,racdr2 -verbose
Prerequisites Check (runcluvfy.sh) Log:
[oracle@racdr1 grid]$ ./runcluvfy.sh stage -pre crsinst -n racdr1,racdr2 -verbose Performing pre-checks for cluster services setup Checking node reachability... Check: Node reachability from node "racdr1" Destination Node Reachable? ------------------------------------ ------------------------ racdr1 yes racdr2 yes Result: Node reachability check passed from node "racdr1" Checking user equivalence... Check: User equivalence for user "oracle" Node Name Status ------------------------------------ ------------------------ racdr2 passed racdr1 passed Result: User equivalence check passed for user "oracle" Checking node connectivity... Checking hosts config file... Node Name Status ------------------------------------ ------------------------ racdr1 passed racdr2 passed Verification of the hosts config file successful Interface information for node "racdr1" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0 192.168.56.220 192.168.56.0 0.0.0.0 UNKNOWN 08:00:27:0D:F4:E3 1500 Interface information for node "racdr2" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0 192.168.56.221 192.168.56.0 0.0.0.0 UNKNOWN 08:00:27:46:D3:12 1500 Check: Node connectivity of subnet "192.168.56.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- racdr1[192.168.56.220] racdr2[192.168.56.221] yes Result: Node connectivity passed for subnet "192.168.56.0" with node(s) racdr1,racdr2 Check: TCP connectivity of subnet "192.168.56.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- racdr1 : 192.168.56.220 racdr1 : 192.168.56.220 passed racdr2 : 192.168.56.221 racdr1 : 192.168.56.220 passed racdr1 : 192.168.56.220 racdr2 : 192.168.56.221 passed racdr2 : 192.168.56.221 racdr2 : 192.168.56.221 passed Result: TCP connectivity check passed for subnet "192.168.56.0" Interfaces found on subnet "192.168.56.0" that are likely candidates for a private interconnect are: racdr1 eth0:192.168.56.220 racdr2 eth0:192.168.56.221 WARNING: Could not find a suitable set of interfaces for VIPs Checking subnet mask consistency... Subnet mask consistency check passed for subnet "192.168.56.0". Subnet mask consistency check passed. Result: Node connectivity check passed Checking multicast communication... Checking subnet "192.168.56.0" for multicast communication with multicast group "224.0.0.251"... Check of subnet "192.168.56.0" for multicast communication with multicast group "224.0.0.251" passed. Check of multicast communication passed. Checking ASMLib configuration. Node Name Status ------------------------------------ ------------------------ racdr1 passed racdr2 passed Result: Check for ASMLib configuration passed. Check: Total memory Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- racdr2 2.9634GB (3107372.0KB) 4GB (4194304.0KB) failed racdr1 2.9634GB (3107372.0KB) 4GB (4194304.0KB) failed Result: Total memory check failed Check: Available memory Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- racdr2 2.5547GB (2678772.0KB) 50MB (51200.0KB) passed racdr1 2.4122GB (2529368.0KB) 50MB (51200.0KB) passed Result: Available memory check passed Check: Swap space Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- racdr2 0.0 bytes 2.9634GB (3107372.0KB) failed racdr1 0.0 bytes 2.9634GB (3107372.0KB) failed Result: Swap space check failed Check: Free disk space for "racdr2:/usr" Path Node Name Mount point Available Required Status ---------------- ------------ ------------ ------------ ------------ ------------ /usr racdr2 /usr 1.9043GB 25MB passed Result: Free disk space check passed for "racdr2:/usr" Check: Free disk space for "racdr1:/usr" Path Node Name Mount point Available Required Status ---------------- ------------ ------------ ------------ ------------ ------------ /usr racdr1 /usr 1.9043GB 25MB passed Result: Free disk space check passed for "racdr1:/usr" Check: Free disk space for "racdr2:/var,racdr2:/etc,racdr2:/sbin" Path Node Name Mount point Available Required Status ---------------- ------------ ------------ ------------ ------------ ------------ /var racdr2 / 1.4277GB 40MB passed /etc racdr2 / 1.4277GB 40MB passed /sbin racdr2 / 1.4277GB 40MB passed Result: Free disk space check passed for "racdr2:/var,racdr2:/etc,racdr2:/sbin" Check: Free disk space for "racdr1:/var,racdr1:/etc,racdr1:/sbin" Path Node Name Mount point Available Required Status ---------------- ------------ ------------ ------------ ------------ ------------ /var racdr1 / 1.4287GB 40MB passed /etc racdr1 / 1.4287GB 40MB passed /sbin racdr1 / 1.4287GB 40MB passed Result: Free disk space check passed for "racdr1:/var,racdr1:/etc,racdr1:/sbin" Check: Free disk space for "racdr2:/tmp" Path Node Name Mount point Available Required Status ---------------- ------------ ------------ ------------ ------------ ------------ /tmp racdr2 /tmp 1.8926GB 1GB passed Result: Free disk space check passed for "racdr2:/tmp" Check: Free disk space for "racdr1:/tmp" Path Node Name Mount point Available Required Status ---------------- ------------ ------------ ------------ ------------ ------------ /tmp racdr1 /tmp 1.1335GB 1GB passed Result: Free disk space check passed for "racdr1:/tmp" Check: User existence for "oracle" Node Name Status Comment ------------ ------------------------ ------------------------ racdr2 passed exists(501) racdr1 passed exists(501) Checking for multiple users with UID value 501 Result: Check for multiple users with UID value 501 passed Result: User existence check passed for "oracle" Check: Group existence for "oinstall" Node Name Status Comment ------------ ------------------------ ------------------------ racdr2 passed exists racdr1 passed exists Result: Group existence check passed for "oinstall" Check: Group existence for "dba" Node Name Status Comment ------------ ------------------------ ------------------------ racdr2 passed exists racdr1 passed exists Result: Group existence check passed for "dba" Check: Membership of user "oracle" in group "oinstall" [as Primary] Node Name User Exists Group Exists User in Group Primary Status ---------------- ------------ ------------ ------------ ------------ ------------ racdr2 yes yes yes yes passed racdr1 yes yes yes yes passed Result: Membership check for user "oracle" in group "oinstall" [as Primary] passed Check: Membership of user "oracle" in group "dba" Node Name User Exists Group Exists User in Group Status ---------------- ------------ ------------ ------------ ---------------- racdr2 yes yes yes passed racdr1 yes yes yes passed Result: Membership check for user "oracle" in group "dba" passed Check: Run level Node Name run level Required Status ------------ ------------------------ ------------------------ ---------- racdr2 5 3,5 passed racdr1 5 3,5 passed Result: Run level check passed Check: Hard limits for "maximum open file descriptors" Node Name Type Available Required Status ---------------- ------------ ------------ ------------ ---------------- racdr2 hard 65536 65536 passed racdr1 hard 65536 65536 passed Result: Hard limits check passed for "maximum open file descriptors" Check: Soft limits for "maximum open file descriptors" Node Name Type Available Required Status ---------------- ------------ ------------ ------------ ---------------- racdr2 soft 1024 1024 passed racdr1 soft 1024 1024 passed Result: Soft limits check passed for "maximum open file descriptors" Check: Hard limits for "maximum user processes" Node Name Type Available Required Status ---------------- ------------ ------------ ------------ ---------------- racdr2 hard 16384 16384 passed racdr1 hard 16384 16384 passed Result: Hard limits check passed for "maximum user processes" Check: Soft limits for "maximum user processes" Node Name Type Available Required Status ---------------- ------------ ------------ ------------ ---------------- racdr2 soft 2047 2047 passed racdr1 soft 2047 2047 passed Result: Soft limits check passed for "maximum user processes" Check: System architecture Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- racdr2 x86_64 x86_64 passed racdr1 x86_64 x86_64 passed Result: System architecture check passed Check: Kernel version Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- racdr2 2.6.32-300.10.1.el5uek 2.6.18 passed racdr1 2.6.32-300.10.1.el5uek 2.6.18 passed Result: Kernel version check passed Check: Kernel parameter for "semmsl" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ racdr1 250 250 250 passed racdr2 250 250 250 passed Result: Kernel parameter check passed for "semmsl" Check: Kernel parameter for "semmns" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ racdr1 32000 32000 32000 passed racdr2 32000 32000 32000 passed Result: Kernel parameter check passed for "semmns" Check: Kernel parameter for "semopm" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ racdr1 100 100 100 passed racdr2 100 100 100 passed Result: Kernel parameter check passed for "semopm" Check: Kernel parameter for "semmni" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ racdr1 128 128 128 passed racdr2 128 128 128 passed Result: Kernel parameter check passed for "semmni" Check: Kernel parameter for "shmmax" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ racdr1 68719476736 68719476736 1590974464 passed racdr2 68719476736 68719476736 1590974464 passed Result: Kernel parameter check passed for "shmmax" Check: Kernel parameter for "shmmni" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ racdr1 4096 4096 4096 passed racdr2 4096 4096 4096 passed Result: Kernel parameter check passed for "shmmni" Check: Kernel parameter for "shmall" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ racdr1 4294967296 4294967296 310737 passed racdr2 4294967296 4294967296 310737 passed Result: Kernel parameter check passed for "shmall" Check: Kernel parameter for "file-max" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ racdr1 6815744 6815744 6815744 passed racdr2 6815744 6815744 6815744 passed Result: Kernel parameter check passed for "file-max" Check: Kernel parameter for "ip_local_port_range" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ racdr1 between 9000 & 65500 between 9000 & 65500 between 9000 & 65535 passed racdr2 between 9000 & 65500 between 9000 & 65500 between 9000 & 65535 passed Result: Kernel parameter check passed for "ip_local_port_range" Check: Kernel parameter for "rmem_default" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ racdr1 262144 262144 262144 passed racdr2 262144 262144 262144 passed Result: Kernel parameter check passed for "rmem_default" Check: Kernel parameter for "rmem_max" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ racdr1 4194304 4194304 4194304 passed racdr2 4194304 4194304 4194304 passed Result: Kernel parameter check passed for "rmem_max" Check: Kernel parameter for "wmem_default" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ racdr1 262144 262144 262144 passed racdr2 262144 262144 262144 passed Result: Kernel parameter check passed for "wmem_default" Check: Kernel parameter for "wmem_max" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ racdr1 1048576 1048576 1048576 passed racdr2 1048576 1048576 1048576 passed Result: Kernel parameter check passed for "wmem_max" Check: Kernel parameter for "aio-max-nr" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ racdr1 65536 unknown 1048576 failed Current value incorrect. Configured value incorrect. racdr2 65536 unknown 1048576 failed Current value incorrect. Configured value incorrect. PRVG-1206 : Check cannot be performed for configured value of kernel parameter "aio-max-nr" on node "racdr1" PRVG-1206 : Check cannot be performed for configured value of kernel parameter "aio-max-nr" on node "racdr2" Result: Kernel parameter check failed for "aio-max-nr" Check: Kernel parameter for "panic_on_oops" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ racdr1 1 unknown 1 failed (ignorable) Configured value incorrect. racdr2 1 unknown 1 failed (ignorable) Configured value incorrect. PRVG-1206 : Check cannot be performed for configured value of kernel parameter "panic_on_oops" on node "racdr1" PRVG-1206 : Check cannot be performed for configured value of kernel parameter "panic_on_oops" on node "racdr2" Result: Kernel parameter check passed for "panic_on_oops" Check: Package existence for "make" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- racdr2 make-3.81-3.el5 make-3.81 passed racdr1 make-3.81-3.el5 make-3.81 passed Result: Package existence check passed for "make" Check: Package existence for "binutils" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- racdr2 binutils-2.17.50.0.6-20.el5 binutils-2.17.50.0.6 passed racdr1 binutils-2.17.50.0.6-20.el5 binutils-2.17.50.0.6 passed Result: Package existence check passed for "binutils" Check: Package existence for "gcc(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- racdr2 gcc(x86_64)-4.1.2-52.el5 gcc(x86_64)-4.1.2 passed racdr1 gcc(x86_64)-4.1.2-52.el5 gcc(x86_64)-4.1.2 passed Result: Package existence check passed for "gcc(x86_64)" Check: Package existence for "libaio(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- racdr2 libaio(x86_64)-0.3.106-5 libaio(x86_64)-0.3.106 passed racdr1 libaio(x86_64)-0.3.106-5 libaio(x86_64)-0.3.106 passed Result: Package existence check passed for "libaio(x86_64)" Check: Package existence for "glibc(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- racdr2 glibc(x86_64)-2.5-81 glibc(x86_64)-2.5-58 passed racdr1 glibc(x86_64)-2.5-81 glibc(x86_64)-2.5-58 passed Result: Package existence check passed for "glibc(x86_64)" Check: Package existence for "compat-libstdc++-33(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- racdr2 compat-libstdc++-33(x86_64)-3.2.3-61 compat-libstdc++-33(x86_64)-3.2.3 passed racdr1 compat-libstdc++-33(x86_64)-3.2.3-61 compat-libstdc++-33(x86_64)-3.2.3 passed Result: Package existence check passed for "compat-libstdc++-33(x86_64)" Check: Package existence for "glibc-devel(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- racdr2 glibc-devel(x86_64)-2.5-81 glibc-devel(x86_64)-2.5 passed racdr1 glibc-devel(x86_64)-2.5-81 glibc-devel(x86_64)-2.5 passed Result: Package existence check passed for "glibc-devel(x86_64)" Check: Package existence for "gcc-c++(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- racdr2 missing gcc-c++(x86_64)-4.1.2 failed racdr1 missing gcc-c++(x86_64)-4.1.2 failed Result: Package existence check failed for "gcc-c++(x86_64)" Check: Package existence for "libaio-devel(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- racdr2 libaio-devel(x86_64)-0.3.106-5 libaio-devel(x86_64)-0.3.106 passed racdr1 libaio-devel(x86_64)-0.3.106-5 libaio-devel(x86_64)-0.3.106 passed Result: Package existence check passed for "libaio-devel(x86_64)" Check: Package existence for "libgcc(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- racdr2 libgcc(x86_64)-4.1.2-52.el5 libgcc(x86_64)-4.1.2 passed racdr1 libgcc(x86_64)-4.1.2-52.el5 libgcc(x86_64)-4.1.2 passed Result: Package existence check passed for "libgcc(x86_64)" Check: Package existence for "libstdc++(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- racdr2 libstdc++(x86_64)-4.1.2-52.el5 libstdc++(x86_64)-4.1.2 passed racdr1 libstdc++(x86_64)-4.1.2-52.el5 libstdc++(x86_64)-4.1.2 passed Result: Package existence check passed for "libstdc++(x86_64)" Check: Package existence for "libstdc++-devel(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- racdr2 libstdc++-devel(x86_64)-4.1.2-52.el5 libstdc++-devel(x86_64)-4.1.2 passed racdr1 libstdc++-devel(x86_64)-4.1.2-52.el5 libstdc++-devel(x86_64)-4.1.2 passed Result: Package existence check passed for "libstdc++-devel(x86_64)" Check: Package existence for "sysstat" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- racdr2 sysstat-7.0.2-11.el5 sysstat-7.0.2 passed racdr1 sysstat-7.0.2-11.el5 sysstat-7.0.2 passed Result: Package existence check passed for "sysstat" Check: Package existence for "ksh" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- racdr2 ksh ksh passed racdr1 ksh ksh passed Result: Package existence check passed for "ksh" Check: Package existence for "nfs-utils" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- racdr2 nfs-utils-1.0.9-60.0.1.el5 nfs-utils-1.0.9-60 passed racdr1 nfs-utils-1.0.9-60.0.1.el5 nfs-utils-1.0.9-60 passed Result: Package existence check passed for "nfs-utils" Check: Package existence for "coreutils" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- racdr2 coreutils-5.97-34.el5 coreutils-5.97-23 passed racdr1 coreutils-5.97-34.el5 coreutils-5.97-23 passed Result: Package existence check passed for "coreutils" Checking availability of ports "6200,6100" required for component "Oracle Notification Service (ONS)" Node Name Port Number Protocol Available Status ---------------- ------------ ------------ ------------ ---------------- racdr2 6200 TCP yes successful racdr1 6200 TCP yes successful racdr2 6100 TCP yes successful racdr1 6100 TCP yes successful Result: Port availability check passed for ports "6200,6100" Checking availability of ports "42424" required for component "Oracle Cluster Synchronization Services (CSSD)" Node Name Port Number Protocol Available Status ---------------- ------------ ------------ ------------ ---------------- racdr2 42424 TCP yes successful racdr1 42424 TCP yes successful Result: Port availability check passed for ports "42424" Checking for multiple users with UID value 0 Result: Check for multiple users with UID value 0 passed Check: Current group ID Result: Current group ID check passed Starting check for consistency of primary group of root user Node Name Status ------------------------------------ ------------------------ racdr2 passed racdr1 passed Check for consistency of root user's primary group passed Starting Clock synchronization checks using Network Time Protocol(NTP)... Checking existence of NTP configuration file "/etc/ntp.conf" across nodes Node Name File exists? ------------------------------------ ------------------------ racdr2 yes racdr1 yes The NTP configuration file "/etc/ntp.conf" is available on all nodes NTP configuration file "/etc/ntp.conf" existence check passed No NTP Daemons or Services were found to be running PRVF-5507 : NTP daemon or service is not running on any node but NTP configuration file exists on the following node(s): racdr2,racdr1 Result: Clock synchronization check using Network Time Protocol(NTP) failed Checking Core file name pattern consistency... Core file name pattern consistency check passed. Checking to make sure user "oracle" is not in "root" group Node Name Status Comment ------------ ------------------------ ------------------------ racdr2 passed does not exist racdr1 passed does not exist Result: User "oracle" is not part of "root" group. Check passed Check default user file creation mask Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- racdr2 0022 0022 passed racdr1 0022 0022 passed Result: Default user file creation mask check passed Checking integrity of file "/etc/resolv.conf" across nodes PRVF-5600 : On node "racdr1" The following lines in file "/etc/resolv.conf" could not be parsed as they are not in proper format: racdr1 PRVF-5600 : On node "racdr2" The following lines in file "/etc/resolv.conf" could not be parsed as they are not in proper format: racdr2 Check for integrity of file "/etc/resolv.conf" failed Check: Time zone consistency Result: Time zone consistency check passed Checking integrity of name service switch configuration file "/etc/nsswitch.conf" ... Checking if "hosts" entry in file "/etc/nsswitch.conf" is consistent across nodes... Checking file "/etc/nsswitch.conf" to make sure that only one "hosts" entry is defined More than one "hosts" entry does not exist in any "/etc/nsswitch.conf" file All nodes have same "hosts" entry defined in file "/etc/nsswitch.conf" Check for integrity of name service switch configuration file "/etc/nsswitch.conf" passed Checking daemon "avahi-daemon" is not configured and running Check: Daemon "avahi-daemon" not configured Node Name Configured Status ------------ ------------------------ ------------------------ racdr2 yes failed racdr1 yes failed Daemon not configured check failed for process "avahi-daemon" Check: Daemon "avahi-daemon" not running Node Name Running? Status ------------ ------------------------ ------------------------ racdr2 yes failed racdr1 yes failed Daemon not running check failed for process "avahi-daemon" Starting check for /dev/shm mounted as temporary file system ... Check for /dev/shm mounted as temporary file system passed Starting check for /boot mount ... Check for /boot mount passed Starting check for zeroconf check ... ERROR: PRVE-10077 : NOZEROCONF parameter was not specified or was not set to 'yes' in file "/etc/sysconfig/network" on node "racdr2" PRVE-10077 : NOZEROCONF parameter was not specified or was not set to 'yes' in file "/etc/sysconfig/network" on node "racdr1" Check for zeroconf check failed Pre-check for cluster services setup was unsuccessful on all the nodes. [oracle@racdr1 grid]$
Note:::If you are happy with prerequisites validation, then Install GI.
Step 3: Install GI
Run runInstaller to start installation process.
[oracle@racdr1 grid]$ pwd /u01/software/grid [oracle@racdr1 grid]$ ls install response rpm runcluvfy.sh runInstaller sshsetup stage welcome.html [oracle@racdr1 grid]$ ./runInstaller Starting Oracle Universal Installer...
Select “Install and Configure Oracle Grid Infrastructure for a Cluster” option and Click Next…
Choose Configure a Standard cluster option and Click Next…
Select Installation Type (Advanced Installation) and Click Next…
Language will be default (English) and Click Next…
Enter Cluster Name, Scan Name and port and Click Next…
Default it shows local Node information but need to add all Node information by Clicking Add Button.
Node list will be like below… You may configure SSH connectivity by Click SSH connectivity button or Manually set SSH for Oracle (Or, if you use GRID user for GI).
Select appropriate Interface for Public and Private and Click Next…
For this practice, I used “Standard ASM for storage” but you may explore other option or select appropriate one for you environment. Click Next….
Enter the DISK group name for VotingDisk/OCR. Highly recommend to use Normal or High Redundant for VotingDisk. Click Next…
Provide the password for sys and asmsnp users. If don’t meet the recommendation then it will show below message
Highly recommended to use the strong password.
Choose Do not use IPMI and Click Next…
You may leave it unchecked and install OEM agent later. Click Next..
Select appropriate group for each categorizes and Click Next…
Provide the Software location for GI Home and Click Next…
Click Next…
You may select this option but I think better to run script manually… Click Next…
Prerequisites check
It will show list of warning or failed from minimum acquirement list. Click “Fix & Check Again”.
Run below script on both nodes to fix the requirement.
Run above recommended script on both nodes.
[root@racdr1 ~]# cd /tmp/CVU_12.1.0.2.0_oracle/ [root@racdr1 CVU_12.1.0.2.0_oracle]# ./runfixup.sh All Fix-up operations were completed successfully. [root@racdr2 ~]# cd /tmp/CVU_12.1.0.2.0_oracle/ [root@racdr2 CVU_12.1.0.2.0_oracle]# ./runfixup.sh All Fix-up operations were completed successfully.
Highly recommended to meet all requirements otherwise installation may failed. For VM setup, NTP and DNS option may ignore but in really life it should be resolved.
Click Yes for confirmation to proceed the installation process.
Summary report will show all Settings. If you are happy, Click Install otherwise may go back to change the options.
Monitor the Installation Process
Below two scripts need to run on all nodes one by one.
Need to run orainstRoot.sh script on both nodes:
#Node 1: [root@racdr1 CVU_12.1.0.2.0_oracle]# cd /u01/app/oraInventory/ [root@racdr1 oraInventory]# ls backup ContentsXML logs oraInst.loc orainstRoot.sh [root@racdr1 oraInventory]# ./orainstRoot.sh Changing permissions of /u01/app/oraInventory. Adding read,write permissions for group. Removing read,write,execute permissions for world. Changing groupname of /u01/app/oraInventory to oinstall. The execution of the script is complete. #Note 2: [root@racdr1 oraInventory]# ssh racdr2 root@racdr2's password: Last login: Tue Jun 23 13:24:35 2020 from racdr1.localdomain [root@racdr2 ~]# cd /u01/app/oraInventory/ [root@racdr2 oraInventory]# ls backup ContentsXML logs oraInst.loc orainstRoot.sh [root@racdr2 oraInventory]# ./orainstRoot.sh Changing permissions of /u01/app/oraInventory. Adding read,write permissions for group. Removing read,write,execute permissions for world. Changing groupname of /u01/app/oraInventory to oinstall. The execution of the script is complete.
Need to run root.sh script on both node one by one.
[root@racdr1 grid]# pwd /u01/app/12.1.0/grid [root@racdr1 grid]# ls -lrt root.sh -rwxr-x--- 1 oracle oinstall 471 Jun 23 13:39 root.sh [root@racdr1 grid]# ./root.sh
Log for root.sh script on Node 1
Performing root user operation.
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/12.1.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/12.1.0/grid/crs/install/crsconfig_params
2020/06/23 17:55:23 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector.
2020/06/23 17:55:49 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
2020/06/23 17:55:50 CLSRSC-363: User ignored prerequisites during installation
OLR initialization - successful
root wallet
root wallet cert
root cert export
peer wallet
profile reader wallet
pa wallet
peer wallet keys
pa wallet keys
peer cert request
pa cert request
peer cert
pa cert
peer root cert TP
profile reader root cert TP
pa root cert TP
peer pa cert TP
pa peer cert TP
profile reader pa cert TP
profile reader peer cert TP
peer user cert
pa user cert
2020/06/23 17:56:40 CLSRSC-330: Adding Clusterware entries to file '/etc/inittab'
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start 'ora.evmd' on 'racdr1'
CRS-2672: Attempting to start 'ora.mdnsd' on 'racdr1'
CRS-2676: Start of 'ora.mdnsd' on 'racdr1' succeeded
CRS-2676: Start of 'ora.evmd' on 'racdr1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'racdr1'
CRS-2676: Start of 'ora.gpnpd' on 'racdr1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'racdr1'
CRS-2672: Attempting to start 'ora.gipcd' on 'racdr1'
CRS-2676: Start of 'ora.cssdmonitor' on 'racdr1' succeeded
CRS-2676: Start of 'ora.gipcd' on 'racdr1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'racdr1'
CRS-2672: Attempting to start 'ora.diskmon' on 'racdr1'
CRS-2676: Start of 'ora.diskmon' on 'racdr1' succeeded
CRS-2676: Start of 'ora.cssd' on 'racdr1' succeeded
ASM created and started successfully.
Disk Group OCR created successfully.
CRS-2672: Attempting to start 'ora.crf' on 'racdr1'
CRS-2672: Attempting to start 'ora.storage' on 'racdr1'
CRS-2676: Start of 'ora.storage' on 'racdr1' succeeded
CRS-2676: Start of 'ora.crf' on 'racdr1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'racdr1'
CRS-2676: Start of 'ora.crsd' on 'racdr1' succeeded
CRS-4256: Updating the profile
Successful addition of voting disk 583fa0bac67c4fb2bf8c7573c78c9ac5.
Successfully replaced voting disk group with +OCR.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 1245fd1fdf6e4f0dbf507f6e6abb1061 (ORCL:OCR_DISK01) [OCR]
2. ONLINE a8594f4803f64ff5bff9114decda1b01 (ORCL:OCR_DISK02) [OCR]
3. ONLINE 98084ddf39354f19bfa787e6c55f3214 (ORCL:OCR_DISK03) [OCR]
Located 1 voting disk(s).
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'racdr1'
CRS-2673: Attempting to stop 'ora.crsd' on 'racdr1'
CRS-2677: Stop of 'ora.crsd' on 'racdr1' succeeded
CRS-2673: Attempting to stop 'ora.storage' on 'racdr1'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'racdr1'
CRS-2673: Attempting to stop 'ora.crf' on 'racdr1'
CRS-2673: Attempting to stop 'ora.ctssd' on 'racdr1'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'racdr1'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'racdr1'
CRS-2677: Stop of 'ora.storage' on 'racdr1' succeeded
CRS-2677: Stop of 'ora.drivers.acfs' on 'racdr1' succeeded
CRS-2677: Stop of 'ora.crf' on 'racdr1' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'racdr1' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'racdr1' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'racdr1' succeeded
CRS-2673: Attempting to stop 'ora.evmd' on 'racdr1'
CRS-2673: Attempting to stop 'ora.asm' on 'racdr1'
CRS-2677: Stop of 'ora.evmd' on 'racdr1' succeeded
CRS-2677: Stop of 'ora.asm' on 'racdr1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'racdr1'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'racdr1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'racdr1'
CRS-2677: Stop of 'ora.cssd' on 'racdr1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'racdr1'
CRS-2677: Stop of 'ora.gipcd' on 'racdr1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'racdr1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-2672: Attempting to start 'ora.mdnsd' on 'racdr1'
CRS-2672: Attempting to start 'ora.evmd' on 'racdr1'
CRS-2676: Start of 'ora.mdnsd' on 'racdr1' succeeded
CRS-2676: Start of 'ora.evmd' on 'racdr1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'racdr1'
CRS-2676: Start of 'ora.gpnpd' on 'racdr1' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'racdr1'
CRS-2676: Start of 'ora.gipcd' on 'racdr1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'racdr1'
CRS-2676: Start of 'ora.cssdmonitor' on 'racdr1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'racdr1'
CRS-2672: Attempting to start 'ora.diskmon' on 'racdr1'
CRS-2676: Start of 'ora.diskmon' on 'racdr1' succeeded
CRS-2676: Start of 'ora.cssd' on 'racdr1' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'racdr1'
CRS-2672: Attempting to start 'ora.ctssd' on 'racdr1'
CRS-2676: Start of 'ora.ctssd' on 'racdr1' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'racdr1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'racdr1'
CRS-2676: Start of 'ora.asm' on 'racdr1' succeeded
CRS-2672: Attempting to start 'ora.storage' on 'racdr1'
CRS-2676: Start of 'ora.storage' on 'racdr1' succeeded
CRS-2672: Attempting to start 'ora.crf' on 'racdr1'
CRS-2676: Start of 'ora.crf' on 'racdr1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'racdr1'
CRS-2676: Start of 'ora.crsd' on 'racdr1' succeeded
CRS-6023: Starting Oracle Cluster Ready Services-managed resources
CRS-6017: Processing resource auto-start for servers: racdr1
CRS-6016: Resource auto-start has completed for server racdr1
CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources
CRS-4123: Oracle High Availability Services has been started.
2020/06/23 18:01:00 CLSRSC-343: Successfully started Oracle Clusterware stack
CRS-2672: Attempting to start 'ora.asm' on 'racdr1'
CRS-2676: Start of 'ora.asm' on 'racdr1' succeeded
CRS-2672: Attempting to start 'ora.OCR.dg' on 'racdr1'
CRS-2676: Start of 'ora.OCR.dg' on 'racdr1' succeeded
2020/06/23 18:02:01 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded
Log for root.sh script on Node 2
[root@racdr2 ~]# cd /u01/app/12.1.0/grid/
[root@racdr2 grid]# ./root.sh
Performing root user operation.
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/12.1.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/12.1.0/grid/crs/install/crsconfig_params
2020/06/23 18:05:36 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector.
2020/06/23 18:06:02 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
2020/06/23 18:06:03 CLSRSC-363: User ignored prerequisites during installation
OLR initialization - successful
2020/06/23 18:07:15 CLSRSC-330: Adding Clusterware entries to file '/etc/inittab'
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'racdr2'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'racdr2'
CRS-2677: Stop of 'ora.drivers.acfs' on 'racdr2' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'racdr2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-2672: Attempting to start 'ora.mdnsd' on 'racdr2'
CRS-2672: Attempting to start 'ora.evmd' on 'racdr2'
CRS-2676: Start of 'ora.mdnsd' on 'racdr2' succeeded
CRS-2676: Start of 'ora.evmd' on 'racdr2' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'racdr2'
CRS-2676: Start of 'ora.gpnpd' on 'racdr2' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'racdr2'
CRS-2676: Start of 'ora.gipcd' on 'racdr2' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'racdr2'
CRS-2676: Start of 'ora.cssdmonitor' on 'racdr2' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'racdr2'
CRS-2672: Attempting to start 'ora.diskmon' on 'racdr2'
CRS-2676: Start of 'ora.diskmon' on 'racdr2' succeeded
CRS-2676: Start of 'ora.cssd' on 'racdr2' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'racdr2'
CRS-2672: Attempting to start 'ora.ctssd' on 'racdr2'
CRS-2676: Start of 'ora.ctssd' on 'racdr2' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'racdr2' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'racdr2'
CRS-2676: Start of 'ora.asm' on 'racdr2' succeeded
CRS-2672: Attempting to start 'ora.storage' on 'racdr2'
CRS-2676: Start of 'ora.storage' on 'racdr2' succeeded
CRS-2672: Attempting to start 'ora.crf' on 'racdr2'
CRS-2676: Start of 'ora.crf' on 'racdr2' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'racdr2'
CRS-2676: Start of 'ora.crsd' on 'racdr2' succeeded
CRS-6017: Processing resource auto-start for servers: racdr2
CRS-2672: Attempting to start 'ora.net1.network' on 'racdr2'
CRS-2676: Start of 'ora.net1.network' on 'racdr2' succeeded
CRS-2672: Attempting to start 'ora.ons' on 'racdr2'
CRS-2676: Start of 'ora.ons' on 'racdr2' succeeded
CRS-6016: Resource auto-start has completed for server racdr2
CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources
CRS-4123: Oracle High Availability Services has been started.
2020/06/23 18:10:04 CLSRSC-343: Successfully started Oracle Clusterware stack
2020/06/23 18:10:18 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded
After executed above scripts in all Nodes, Click OK. Monitor the process.
Ignore this error and Click OK to continue Install process.
Ignore this error.
Click Yes to confirm. Installation process are done.
Installation completed successfully.
Verify all services:
[root@racdr1 grid]# . oraenv
ORACLE_SID = [root] ? +ASM1
The Oracle base has been set to /u01/app/oracle
[root@racdr1 grid]# crsctl stat res -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
ONLINE ONLINE racdr1 STABLE
ONLINE ONLINE racdr2 STABLE
ora.OCR.dg
ONLINE ONLINE racdr1 STABLE
ONLINE ONLINE racdr2 STABLE
ora.asm
ONLINE ONLINE racdr1 STABLE
ONLINE ONLINE racdr2 Started,STABLE
ora.net1.network
ONLINE ONLINE racdr1 STABLE
ONLINE ONLINE racdr2 STABLE
ora.ons
ONLINE ONLINE racdr1 STABLE
ONLINE ONLINE racdr2 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE racdr1 STABLE
ora.MGMTLSNR
1 ONLINE ONLINE racdr1 169.254.9.200 192.16
8.2.220,STABLE
ora.cvu
1 ONLINE ONLINE racdr1 STABLE
ora.mgmtdb
1 ONLINE ONLINE racdr1 Open,STABLE
ora.oc4j
1 ONLINE ONLINE racdr1 STABLE
ora.racdr1.vip
1 ONLINE ONLINE racdr1 STABLE
ora.racdr2.vip
1 ONLINE ONLINE racdr2 STABLE
ora.scan1.vip
1 ONLINE ONLINE racdr1 STABLE
--------------------------------------------------------------------------------
Configure ASM
To Configure ASM Disk Group using asmca utility, set GI environment and run asmca command
You may check all ASM instances are up and running or not. Select Disk Groups Tab
Click Create Button
Enter Disk Group Name (FRA) and Choose Redundancy and Select Disks. Click OK
Confirmation message
If Disk Group is not mounted in all Instances, then Click Mount All Button
Install Oracle Database Software and Configure the Database
Unzip Oracle Database Binary
[oracle@racdr1 software]$ ls -lrt -rw-r--r-- 1 oracle oinstall 1014530602 Feb 7 2019 linuxamd64_12102_database_2of2.zip -rw-r--r-- 1 oracle oinstall 1673544724 Feb 7 2019 linuxamd64_12102_database_1of2.zip [oracle@racdr1 software]$ unzip linuxamd64_12102_database_1of2.zip [oracle@racdr1 software]$ unzip linuxamd64_12102_database_2of2.zip [oracle@racdr1 software]$ ls database linuxamd64_12102_database_1of2.zip linuxamd64_12102_database_2of2.zip
Start Installation
run runInstaller to start installation process.
[oracle@racdr1 database]$ pwd /u01/software/database [oracle@racdr1 database]$ ls install response rpm runInstaller sshsetup stage welcome.html [oracle@racdr1 database]$ ./runInstaller Starting Oracle Universal Installer... Checking Temp space: must be greater than 500 MB. Actual 1341 MB Passed Checking swap space: must be greater than 150 MB. Actual 4491 MB Passed
You can unchecked for receiving the security update via MOS. Click Next…
Select Installation Option- Create and configure a database and Click Next…
Select Server Class and Click Next…
Choose GI option – It should be Oracle Real Application Cluster database installation. Click Next…
Select RAC Database Configuration Type and Click Next…
Select all Nodes.
Select Languages.. Click Next…
Select the Database Edition. Based on your license, select the Edition and Click Next…
Provide Oracle Base and Home Location. Click Next…
Select the Database Configuration Type. Very important to choose right one based on your purpose. Click Next…
Provide the Database Name. Click Next…
Check “Enable Automatic Memory Management” and assign the total memory for Oracle.
Select Character Sets based on your requirement. This is very important for your environment otherwise you may face issue later. Click Next…
Select Storage Options. For RAC, It should be ASM. Click Next.
You may provide OMS details during installation and configure agent later. Click Next…
Select the Recovery Options and Click Next…
Select the ASM Group for database. Click Next.
Enter the password for sys, system and dbsnmp users. Click Next…
Assign the OS GROUP for each database group and click Next…
[root@racdr1 CVU_12.1.0.2.0_oracle]# ./runfixup.sh All Fix-up operations were completed successfully.
Below list of prerequisite did not resolved by running the fixup script. Recommended to resolve. Below prerequisite I ignored for this setup.
Summary of all setting. If you are good with all setting then Click Install
Installation is in progress
Open a new terminal or connect from putty and execute the root.sh script on both nodes from root user and once execution is done, Click OK.
Execute the root.sh script
[root@racdr1 CVU_12.1.0.2.0_oracle]# cd /u01/app/oracle/product/12.1.0/dbhome_1/
[root@racdr1 dbhome_1]# ./root.sh
Performing root user operation.
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/oracle/product/12.1.0/dbhome_1
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
[root@racdr1 dbhome_1]# pwd
/u01/app/oracle/product/12.1.0/dbhome_1
Database creation is progressing
Installation completed successfully.
[oracle@racdr1 ~]$ srvctl status database -d ORCL Instance ORCL1 is running on node racdr1 Instance ORCL2 is running on node racdr2
[oracle@racdr1 ~]$ srvctl config database -db ORCL Database unique name: ORCL Database name: ORCL Oracle home: /u01/app/oracle/product/12.1.0/dbhome_1 Oracle user: oracle Spfile: +DATA/ORCL/PARAMETERFILE/spfile.273.1043890263 Password file: +DATA/ORCL/PASSWORD/pwdorcl.256.1043889945 Domain: Start options: open Stop options: immediate Database role: PRIMARY Management policy: AUTOMATIC Server pools: Disk Groups: DATA Mount point paths: Services: Type: RAC Start concurrency: Stop concurrency: OSDBA group: dba OSOPER group: oinstall Database instances: ORCL1,ORCL2 Configured nodes: racdr1,racdr2 Database is administrator managed
Clusterware Control (CRSCTL) & Server Control (SRVCTL) to mange Grid and DB:
NOTE::: You don’t need to memorize these command. If you know what command does then can find the required command by your self.
Clusterware Control (CRSCTL) Utility:
[oracle@racdr1 ~]$ crsctl -help
Usage: crsctl add - add a resource, type or other entity
crsctl check - check the state or operating status of a service, resource, or other entity
crsctl config - display automatic startup configuration
crsctl debug - display or modify debug state
crsctl delete - delete a resource, type or other entity
crsctl disable - disable automatic startup
crsctl discover - discover DHCP server
crsctl enable - enable automatic startup
crsctl eval - evaluate operations on resource or other entity without performing them
crsctl get - get an entity value
crsctl getperm - get entity permissions
crsctl lsmodules - list debug modules
crsctl modify - modify a resource, type or other entity
crsctl query - query service state
crsctl pin - make the leases of specified nodes immutable
crsctl relocate - relocate a resource, server or other entity
crsctl replace - change the location of voting files
crsctl release - release a DHCP lease
crsctl request - request a DHCP lease or an action entry point
crsctl setperm - set entity permissions
crsctl set - set an entity value
crsctl start - start a resource, server or other entity
crsctl status - get status of a resource or other entity
crsctl stop - stop a resource, server or other entity
crsctl unpin - make the leases of previously pinned nodes mutable
crsctl unset - unset an entity value, restoring its default
More Specific:
[oracle@racdr1 ~]$ crsctl start -help
Usage:
crsctl start resource {<resName> [...]|-w <filter>]|-all} [-n <server> | -s <server_pools>] [-k <cid>] [-d <did>] [-env "env1=val1,env2=val2,..."] [-begin|-end] [-f] [-i] [-l]
Start designated resources
more....
Server Control (SRVCTL):
[oracle@racdr1 ~]$ srvctl start -help
The SRVCTL start command starts, Oracle Clusterware enabled, non-running objects.
Usage: srvctl start database -db <db_unique_name> [-startoption <start_options>] [-startconcurrency <start_concurrency>] [-node <node>] [-eval] [-verbose]
Usage: srvctl start instance -db <db_unique_name> {-node <node_name> [-instance <inst_name>] | -instance <inst_name_list>} [-startoption <start_options>]
Usage: srvctl start service -db <db_unique_name> [-service "<service_name_list>"] [-serverpool <pool_name>] [-node <node_name> | -instance <inst_name>] [-pq] [-global_override] [-startoption <start_options>] [-eval] [-verbose]
more....
1 pings
[…] Install Oracle Grid Infrastructure and Oracle Software: Step by Step […]