Oracle RAC 10g and 11g on CentOS 5.2

ORACLE RAC LAB BUILD

The purpose of the RAC Lab is to:

a) build a similar system to the work RAC systems and to go through the build installation to identify configuration and installation issues before attempting the same work on the systems at work.

b) become familiar with RAC configuration and installation

c) create a sandbox platform to experiment with different configurations and functional testing

LAB Equipment:

Servers:
2 Dual AMD Opteron 64bit w/4Gb RAM and 2 gigabit ethernet ports
1 120Gb internal HD

Operating system:
CentOS 5.2 (Red Hat 5.2 binary equiv)

Oracle Software:
Cluster file system OCFS2 1.4.1
ASMLib 2.0
11g Oracle Clusterware
11g RAC ASM
11g Database using 11g RAC ASM
10g Database using 11g RAC ASM

Shared Storage:
Openfiler 2.3
Installed on AMD PC with 4Gb RAM single 640Gb HD and single gigabit
ethernet port

Network Equipment:
1 8 port gigabit switch (Private Interconnect and Storage Connection)
1 4 port 100mb/s hub (Public)

Staging server:
Apple Macbook

SIGNIFICANT DIFFERENCES BETWEEN LAB SETUP AND WORK SERVERS
=======================================================================================
ITEM LAB WORK
-----------------------------------------------------------------------------------------------------------------------------
Hardware 2 Rackable Systems 2 HP DL585 G5 s
Dual AMD Opteron 4 Quad Core AMD Opterons
single core CPU s w/128Gb RAM
w/4Gb RAM & Dual 6 Gigabit Ethernet w/4
Gigabit Ethernet bonded in pairs for private
and public networks 3 HBA s for SAN connections

Disk iSCSI disks from EMC SAN Disks Multipathed
dedicated PC with & connected through HBA s
openfiler 2.3
connected to the
RAC nodes via the
private network

OS CentOS 5.2 Red Hat 5.2

Security No special add ons Keon ESM

Monitoring No special add ons Tivoli netsnmp/snmp-DC
BMC Perform&Predict

Backup No special add ons Veritas Netbackup

DB Monitoring Local DB Console Oracle GridControl (Centralized)

INSTALLATION ACTIONS
=======================================================================================
1)INSTALL OS ON EACH NODE

choose CentOS 5.2 x86_64
select base install
when it reboots for the first time login:
disable firewall and SELinux
skip creating a user
log in as root
Do NOT run automatic updates when complete!!!

2)SETUP NETWORK

Open network admin utility and set up Network

Set hostname
disable DHCP

port eth0 = 10.0.1.6X
port eth1 = 10.0.2.6X

restart network services

Open terminal and edit /etc/hosts

Remove hostname from 127.0.0.1

add the following:

10.0.1.69 racbaby1
10.0.1.68 racbaby2
10.0.2.69 racbaby1-priv
10.0.2.68 racbaby2-priv
10.0.1.169 racbaby1-vip
10.0.1.168 racbaby2-vip
10.0.2.66 openfiler1

3)MODIFY KERNEL AND SETTINGS

use GUI "services" utility to disable updater card reader bluetooth etc

use GUI "Add Software" tool to install these items listed under "development"
development tools
development libraries
java development
legacy systems development

In a terminal window install the following additional packages:

yum install libaio-devel
yum install sysstat
yum install elfutils-libelf-devel
yum install iscsi-initiator-utils.x86_64
yum install lsscsi
yum install xorg-x11-deprecated-libs
yum install libXp.so.6

Set parameters using these commands:

sysctl -w net.core.rmem_default=4194304
sysctl -w net.core.wmem_default=262144
sysctl -w net.core.rmem_max=4194304
sysctl -w net.core.wmem_max=262144
echo "# Default setting in bytes of the socket receive buffer" >>/etc/sysctl.conf
echo "net.core.rmem_default= 4194304" >>/etc/sysctl.conf
echo "# Default setting in bytes of the socket send buffer" >>/etc/sysctl.conf
echo "net.core.wmem_default=262144" >>/etc/sysctl.conf
echo "# Maximum socket receive buffer size which may be set by using" >>/etc/sysctl.conf
echo "# the SO_RCVBUF socket option" >>/etc/sysctl.conf
echo "net.core.rmem_max= 4194304" >>/etc/sysctl.conf
echo "# Maximum socket send buffer size which may be set by using" >>/etc/sysctl.conf
echo "# the SO_SNDBUF socket option" >>/etc/sysctl.conf
echo "net.core.wmem_max=262144" >>/etc/sysctl.conf

sysctl -w kernel.shmmax=2147483648
echo "kernel.shmmax=2147483648" >> /etc/sysctl.conf
sysctl -w kernel.sem="250 32000 100 128"
echo "kernel.sem=250 32000 100 128" >> /etc/sysctl.conf
sysctl -w fs.file-max=65536
echo "fs.file-max=65536" >> /etc/sysctl.conf
sysctl -w net.ipv4.ip_local_port_range="1024 65000"
echo "net.ipv4.ip_local_port_range = 1024 65000" >> /etc/sysctl.conf

cat >> /etc/security/limits.conf <<EOF
oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
EOF

cat >> /etc/pam.d/login <<EOF
session required /lib/security/pam_limits.so
EOF

cat >> /etc/profile <<EOF
if [ $USER = "oracle" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
umask 022
fi
EOF

4)CREATE ORACLE USER

groupadd -g 115 oinstall
groupadd -g 116 dba
useradd -m -u 175 -g oinstall -G dba -d /home/oracle -s /bin/bash -c "Oracle Software Owner" oracle
passwd oracle

5)CREATE SPACE FOR HOMES

mkdir -p /u01/app/oracle
chown -R oracle:oinstall /u01/app/oracle
chmod -R 775 /u01/app/oracle
mkdir -p /u01/app/11.1.0
chown -R oracle:oinstall /u01/app/11.1.0
chmod -R 775 /u01/app/11.1.0

6)CREATE ISCSI DISKS

Use openfiler to set up disks and map luns to the two nodes. I m not going to
listout how I setup Openfiler as it doesn t pertain to the purpose of the LAB
build.

For simplicity I named the luns according to function so that when they appear on the nodes you ll see something like the following in /var/log/messages:

[root@racbaby2 log]# cat messages.1|grep openf|sort
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:asm10 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:asm1 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:asm2 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:asm3 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:asm4 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:asm5 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:asm6 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:asm7 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:asm8 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:asm9 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:ocfs 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:ocr1 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:ocr2 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:spfile 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:vote1 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:vote2 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:vote3 10.0.2.66 3260]

7)COPY OVER SOFTWARE FILES FROM STAGING SERVER

Hangcheck Timer
hangcheck-0.8.0.tar.gz

Oracle CLUSTER and DB Software packages:
linux.x64_11gR1_clusterware.zip
linux.x64_11gR1_database_1013.zip
10201_database_linux_x86_64.cpio.gz

Oracle OCFS RPM s
ocfs2-2.6.18-92.el5-1.4.1-1.el5.x86_64.rpm
ocfs2-tools-1.4.1-1.el5.x86_64.rpm
ocfs2-tools-debuginfo-1.4.1-1.el5.x86_64.rpm
ocfs2-tools-devel-1.4.1-1.el5.x86_64.rpm
ocfs2console-1.4.1-1.el5.x86_64.rpm

ASMLib RPM s
oracleasm-2.6.18-92.1.22.el5-2.0.5-1.el5.x86_64.rpm
oracleasm-2.6.18-92.el5-2.0.5-1.el5.x86_64.rpm
oracleasm-support-2.1.2-1.el5.x86_64.rpm
oracleasmlib-2.0.3-1.el5.x86_64.rpm

homemade env variable switcher script
oradb

Oracle patchsets:
p6810189_10204_Linux-x86-64.zip
p6890831_111070_Linux-x86-64.zip

8)SET UP ISCSI ON RAC NODES

as root on each node:

service iscsi restart
chkconfig --level 345 iscsi on
iscsiadm -m discovery -t sendtargets -p openfiler1
service iscsi restart

verify disks are there

fdisk -l

run fdisk on each device to create a primary partition

cheap canada goose sale

If some one wishes to be updated with hottest technologies therefore he must be pay a visit this site and be up to date all the time.

If some one wishes to be updated with hottest technologies therefore he must be pay a visit this site and be up to date all the time.

unbelievablebeautytips.yolasite.com

Howdy just wanted to give you a brief heads up and let you know a few of the pictures aren't loading correctly. I'm not sure why but I think its a linking issue. I've tried it in two different browsers and both show the same results.

Howdy just wanted to give you a brief heads up and let you know a few of the pictures aren't loading correctly. I'm not sure why but I think its a linking issue. I've tried it in two different browsers and both show the same results.

Bala

Excellent

there is a lot of useful information here.

fnaf

fnaf

Thank you for posting such a great article! I found your website perfect for my needs. It contains wonderful and helpful posts. Keep up the good work!. Thank you for this wonderful Article!

schhathu sharma

jhbbjhb

cialis

Hey I know this is off topic but I was wondering if you knew of any widgets I could add to my blog that automatically tweet my newest twitter updates. I've been looking for a plug-in like this for quite some time and was hoping maybe you would have some experience with something like this. Please let me know if you run into anything. I truly enjoy reading your blog and I look forward to your new updates.

Hey I know this is off topic but I was wondering if you knew of any widgets I could add to my blog that automatically tweet my newest twitter updates. I've been looking for a plug-in like this for quite some time and was hoping maybe you would have some experience with something like this. Please let me know if you run into anything. I truly enjoy reading your blog and I look forward to your new updates.

Billig protein

For newest information you have to pay a quick visit world-wide-web and on internet I found this web site as a most excellent web page for latest updates.

For newest information you have to pay a quick visit world-wide-web and on internet I found this web site as a most excellent web page for latest updates.

Läkemedel vid förmaksflimmer

What a material of un-ambiguity and preserveness of valuable know-how on the topic of unexpected emotions.

What a material of un-ambiguity and preserveness of valuable know-how on the topic of unexpected emotions.

http://www.generiqueviagrafr.fr/viagra-prix-et-remboursement

Like Coke just keeps its formula as a trade secret. Things changed when Cialis made its appearance in the market. These products usually contain natural aphrodisiacs that have proven their efficacy in raising the sexual appetite and Vig - Rx Plus improving the sexual intercourse.

Like Coke just keeps its formula as a trade secret. Things changed when Cialis made its appearance in the market. These products usually contain natural aphrodisiacs that have proven their efficacy in raising the sexual appetite and Vig - Rx Plus improving the sexual intercourse.

Ansiktskräm känslig hud

We stumbled over here from a different page and thought I might check things out. I like what I see so now i'm following you. Look forward to exploring your web page for a second time.

We stumbled over here from a different page and thought I might check things out. I like what I see so now i'm following you. Look forward to exploring your web page for a second time.