ORACLE RAC Sandbox Installation

Create 2 Standard Build Virtual Machines with Raw Storage devices and Oracle system settings

Create mapped physical storage for the VMWare hosts on the SAN according to the size requirements listed below in the raw mapping table.

Configure raw devices in /etc/sysconfig/rawdevices and restart the rawdevices service

/dev/raw/raw1 /dev/sdb1 # Voting 300M

/dev/raw/raw2 /dev/sdc1 # OCR 400M

/dev/raw/raw10 /dev/sdd1 #DATA1 4G

/dev/raw/raw11 /dev/sde1 # DATA2 4G

/dev/raw/raw12 /dev/sdf1 # DATA3 4G

Assign 2 network cards in VMWare to each host - 1 private and 1 public

Change owner on all raw devices to oracle.oinstall

Zero out the raw devices:

dd if=/dev/zero of=/dev/raw/raw1

dd if=/dev/zero of=/dev/raw/raw2

dd if=/dev/zero of=/dev/raw/raw10

dd if=/dev/zero of=/dev/raw/raw11

dd if=/dev/zero of=/dev/raw/raw12

Create 1 public and 1 private ip on each server

Create Oracle user and groups:

groupadd -g 200 oinstall

groupadd -g 201 dba

useradd -u 200 -g oinstall -G dba oracle

Create host entries for the public and private ip's on both servers

Example IP scheme:

Node Interface Name Type IP Address Registered In

rac1 rac1 Public 143.46.43.100 DNS (if available, else the hosts file)

rac1 rac1-vip Virtual 143.46.43.104 DNS (if available, else the hosts file)

rac1 rac1-priv Private 10.0.0.1 Hosts file

rac2 rac2 Public 143.46.43.101 DNS (if available, else the hosts file)

rac2 rac2-vip Virtual 143.46.43.105 DNS (if available, else the hosts file)

rac2 rac2-priv Private 10.0.0.2 Hosts file

Create additional host entries for the VIP addresses, but DO NOT create the actual ip interfaces for these VIPs - oracle installer handles the VIP addresses

create /etc/udev/permissions.d/40-udev.permissions file with the following:

#ocr file

raw/raw2:root:oinstall:0640

#votedisk file

raw/raw1:oracle:oinstall:0660

#If using Device Mapper for ASM, then use the dm devices

#ASM disks

#dm-[1-9]:oracle:oinstall:0640

/sbin/service nscd start

chkconfig nscd on

Validate software requirements:

Red Hat Enterprise Linux 4

(Update 1 or later):Packages

The following packages (or later versions) must be installed:

binutils-2.15.92.0.2-10.EL4

binutils-2.15.92.0.2-13.0.0.0.2.x86_64

compat-db-4.1.25-9

control-center-2.8.0-12

gcc-3.4.3-9.EL4

gcc-c++-3.4.3-9.EL4

glibc-2.3.4-2

glibc-common-2.3.4-2

gnome-libs-1.4.1.2.90-44.1

libstdc++-3.4.3-9.EL4

libstdc++-devel-3.4.3-9.EL4

make-3.80-5

Note: XDK is not supported with gcc on Red Hat Enterprise

Linux 4

Run commands to setup user equivalence:

Starting on racdev1 as the oracle user -

mkdir /home/oracle/.ssh; chmod 755 .ssh

ssh-keygen -t rsa # leave password blank

ssh-keygen -t dsa # leave password blank

cd .ssh

ssh racdev2 mkdir .ssh

ssh racdev2 chmod 755 .ssh

ssh racdev2 cat /home/oracle/.ssh/id_rsa.pub >> authorized_keys

ssh racdev2 cat /home/oracle/.ssh/id_dsa.pub >> authorized_keys

ssh racdev1 cat /home/oracle/.ssh/id_dsa.pub >> authorized_keys

ssh racdev1 cat /home/oracle/.ssh/id_rsa.pub >> authorized_keys

ssh racdev1

scp authorized_keys racdev2:/home/oracle/.ssh

ssh racdev2 uname -a

Downloaded 10G R2 for Linux x86_64:

http://www.oracle.com/technology/software/products/database/oracle10g/htdocs/10201linx8664soft.html

and Clusterware 10G R2:

http://download.oracle.com/otn/linux/oracle10g/10201/10201_clusterware_linux_x86_64.cpio.gz

Locate the cvuqdisk RPM package, which is in the directory clusterware/rpm

on the installation media. If you have already installed Oracle Clusterware, then it

is located in the directory CRS_home/rpm. Install on all nodes in cluster.

Set the environment variable CVUQDISK_GRP to point to the group that will own

cvuqdisk, typically oinstall.

Use the following command to install the cvuqdisk package:

rpm -iv cvuqdisk-1.0.1-1.rpm

Add to /etc/profile

if [ $USER = "oracle" ]; then

if [ $SHELL = "/bin/ksh" ]; then

ulimit -p 16384

ulimit -n 65536

else

ulimit -u 16384 -n 65536

fi

fi

/home/oracle/clusterware/cluvfy/runcluvfy.sh stage -post hwos -n racdev1,racdev2 -verbose

gzip -d 10201_database_linux_x86_64.cpio.gz

cpio -idmv<10201_database_linux_x86_64.cpio

vncserver

mkisofs -JR -l -o 10.2.0.1.0ora.iso database/

gunzip ship.db.cpio.gz

cpio -idmv<ship.db.cpio

Create iso for burning a CD

mkisofs -JR -l -o ship.db.iso Disk1/

unzip p5337014_10203_Linux-x86-64.zip

./runcluvfy.sh stage -pre crsinst -n racdev1,racdev2

su - oracle -c "/home/oracle/clusterware/cluvfy/runcluvfy.sh stage -pre crsinst -n racdev1,racdev2"

IGNORE THE ERROR REGARDING VIP ADDRESSES FOR NOW

Run the following command and also add it to /etc/rc.local:

insmod /lib/modules/2.6.9-67.0.4.ELsmp/kernel/drivers/char/hangcheck-timer.ko hangcheck_tick=30 hangcheck_margin=180

To avoid this error below:

/u01/crs1020/bin/crsctl.bin: error while loading shared libraries: libstdc++.so.5: cannot open shared object file: No such file or directory

Failure initializing entries in /etc/oracle/scls_scr/racdev1.

Do this:

rpm -Uvh http://santa/repository/patch/current/rhel4-x86_64/compat-libstdc++-33-3.2.3-47.3.x86_64.rpm

ln -s /usr/lib64/libstdc++.so.5.0.7 /lib/libstdc++.so.5

ln -s /usr/lib64/libstdc++.so.5.0.7 /usr/lib/libstdc++.so.5

Run

/home/oracle/clusterware/runInstaller from a vnc session and follow oracle instructions to the letter.

Log is in:

/u01/app/oracle/oraInventory/logs/installActions2008-02-22_04-25-47AM.log

Edit rootconfig

chmod +w /u01/crs1020/install/rootconfig

vi /u01/crs1020/install/rootconfig

change MISCNT value to 300 and save

when prompted run:

/u01/crs1020/root.sh

create /etc/udev/permissions.d/40-udev.permissions with the following:

#ocr file

raw/raw2:root:oinstall:0640

#votedisk file

raw/raw1:oracle:oinstall:0660

raw/raw10:oracle:oinstall:0660

raw/raw11:oracle:oinstall:0660

raw/raw12:oracle:oinstall:0660

/u01/crs1020/bin/racgons add_config racdev1:6200 racdev2:6200

/u01/crs1020/bin/oifcfg setif -global eth0/10.3.17.0:public eth1/192.168.10.0:cluster_interconnect

PRIF-12: failed to initialize cluster support services # Still stuck on this error

/u01/crs1020/bin/cluvfy stage -post crsinst -n racdev1,racdev2

############# Ran these for troubleshooting purposes only ##################

Ran these manually from the rootconfig script while troubleshooting errors

/u01/crs1020/css/admin/init.cssd activatevg

/u01/crs1020/bin/crsctl create scr oracle

/u01/crs1020/bin/clscfg -install -nn racdev1,1,racdev2,2 -pn racdev1-priv,1,racdev2-priv,2 -hn racdev1,1,racdev2,2 -o /u01/crs1020 -c crsdev -l 'AMERICAN_AMERICA.WE8ISO8859P1' -q /dev/raw/raw1 "-misscount 300"

[root@racdev2 bin]# mkdir ~oracle/.proxysrv

[root@racdev2 bin]# chown -R oracle ~oracle/.proxysrv

[root@racdev2 bin]# chmod 755 ~oracle/.proxysrv

[root@racdev2 bin]# cp evm/admin/init.evmd /etc/init.d/init.evmd

cp: cannot stat `evm/admin/init.evmd': No such file or directory

[root@racdev2 bin]# cd /u01/crs1020/

[root@racdev2 crs1020]# cp evm/admin/init.evmd /etc/init.d/init.evmd

[root@racdev2 crs1020]# cp css/admin/init.cssd /etc/init.d/init.cssd

[root@racdev2 crs1020]# cp crs/admin/init.crsd /etc/init.d/init.crsd

[root@racdev2 crs1020]# cp crs/admin/init.crs /etc/init.d/init.crs

/etc/init.d/init.crs start

/etc/init.d/init.cssd startcheck CSS

/u01/crs1020/bin/crsctl check install -wait 600

################# END of troubleshooting #########################