Oracle RAC 10g and 11g on CentOS 5.2

ORACLE RAC LAB BUILD

The purpose of the RAC Lab is to:

a) build a similar system to the work RAC systems and to go through the build installation to identify configuration and installation issues before attempting the same work on the systems at work.

b) become familiar with RAC configuration and installation

c) create a sandbox platform to experiment with different configurations and functional testing

LAB Equipment:

Servers:
2 Dual AMD Opteron 64bit w/4Gb RAM and 2 gigabit ethernet ports
1 120Gb internal HD

Operating system:
CentOS 5.2 (Red Hat 5.2 binary equiv)

Oracle Software:
Cluster file system OCFS2 1.4.1
ASMLib 2.0
11g Oracle Clusterware
11g RAC ASM
11g Database using 11g RAC ASM
10g Database using 11g RAC ASM

Shared Storage:
Openfiler 2.3
Installed on AMD PC with 4Gb RAM single 640Gb HD and single gigabit
ethernet port

Network Equipment:
1 8 port gigabit switch (Private Interconnect and Storage Connection)
1 4 port 100mb/s hub (Public)

Staging server:
Apple Macbook

SIGNIFICANT DIFFERENCES BETWEEN LAB SETUP AND WORK SERVERS
=======================================================================================
ITEM LAB WORK
-----------------------------------------------------------------------------------------------------------------------------
Hardware 2 Rackable Systems 2 HP DL585 G5 s
Dual AMD Opteron 4 Quad Core AMD Opterons
single core CPU s w/128Gb RAM
w/4Gb RAM & Dual 6 Gigabit Ethernet w/4
Gigabit Ethernet bonded in pairs for private
and public networks 3 HBA s for SAN connections

Disk iSCSI disks from EMC SAN Disks Multipathed
dedicated PC with & connected through HBA s
openfiler 2.3
connected to the
RAC nodes via the
private network

OS CentOS 5.2 Red Hat 5.2

Security No special add ons Keon ESM

Monitoring No special add ons Tivoli netsnmp/snmp-DC
BMC Perform&Predict

Backup No special add ons Veritas Netbackup

DB Monitoring Local DB Console Oracle GridControl (Centralized)

INSTALLATION ACTIONS
=======================================================================================
1)INSTALL OS ON EACH NODE

choose CentOS 5.2 x86_64
select base install
when it reboots for the first time login:
disable firewall and SELinux
skip creating a user
log in as root
Do NOT run automatic updates when complete!!!

2)SETUP NETWORK

Open network admin utility and set up Network

Set hostname
disable DHCP

port eth0 = 10.0.1.6X
port eth1 = 10.0.2.6X

restart network services

Open terminal and edit /etc/hosts

Remove hostname from 127.0.0.1

add the following:

10.0.1.69 racbaby1
10.0.1.68 racbaby2
10.0.2.69 racbaby1-priv
10.0.2.68 racbaby2-priv
10.0.1.169 racbaby1-vip
10.0.1.168 racbaby2-vip
10.0.2.66 openfiler1

3)MODIFY KERNEL AND SETTINGS

use GUI "services" utility to disable updater card reader bluetooth etc

use GUI "Add Software" tool to install these items listed under "development"
development tools
development libraries
java development
legacy systems development

In a terminal window install the following additional packages:

yum install libaio-devel
yum install sysstat
yum install elfutils-libelf-devel
yum install iscsi-initiator-utils.x86_64
yum install lsscsi
yum install xorg-x11-deprecated-libs
yum install libXp.so.6

Set parameters using these commands:

sysctl -w net.core.rmem_default=4194304
sysctl -w net.core.wmem_default=262144
sysctl -w net.core.rmem_max=4194304
sysctl -w net.core.wmem_max=262144
echo "# Default setting in bytes of the socket receive buffer" >>/etc/sysctl.conf
echo "net.core.rmem_default= 4194304" >>/etc/sysctl.conf
echo "# Default setting in bytes of the socket send buffer" >>/etc/sysctl.conf
echo "net.core.wmem_default=262144" >>/etc/sysctl.conf
echo "# Maximum socket receive buffer size which may be set by using" >>/etc/sysctl.conf
echo "# the SO_RCVBUF socket option" >>/etc/sysctl.conf
echo "net.core.rmem_max= 4194304" >>/etc/sysctl.conf
echo "# Maximum socket send buffer size which may be set by using" >>/etc/sysctl.conf
echo "# the SO_SNDBUF socket option" >>/etc/sysctl.conf
echo "net.core.wmem_max=262144" >>/etc/sysctl.conf

sysctl -w kernel.shmmax=2147483648
echo "kernel.shmmax=2147483648" >> /etc/sysctl.conf
sysctl -w kernel.sem="250 32000 100 128"
echo "kernel.sem=250 32000 100 128" >> /etc/sysctl.conf
sysctl -w fs.file-max=65536
echo "fs.file-max=65536" >> /etc/sysctl.conf
sysctl -w net.ipv4.ip_local_port_range="1024 65000"
echo "net.ipv4.ip_local_port_range = 1024 65000" >> /etc/sysctl.conf

cat >> /etc/security/limits.conf <<EOF
oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
EOF

cat >> /etc/pam.d/login <<EOF
session required /lib/security/pam_limits.so
EOF

cat >> /etc/profile <<EOF
if [ $USER = "oracle" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
umask 022
fi
EOF

4)CREATE ORACLE USER

groupadd -g 115 oinstall
groupadd -g 116 dba
useradd -m -u 175 -g oinstall -G dba -d /home/oracle -s /bin/bash -c "Oracle Software Owner" oracle
passwd oracle

5)CREATE SPACE FOR HOMES

mkdir -p /u01/app/oracle
chown -R oracle:oinstall /u01/app/oracle
chmod -R 775 /u01/app/oracle
mkdir -p /u01/app/11.1.0
chown -R oracle:oinstall /u01/app/11.1.0
chmod -R 775 /u01/app/11.1.0

6)CREATE ISCSI DISKS

Use openfiler to set up disks and map luns to the two nodes. I m not going to
listout how I setup Openfiler as it doesn t pertain to the purpose of the LAB
build.

For simplicity I named the luns according to function so that when they appear on the nodes you ll see something like the following in /var/log/messages:

[root@racbaby2 log]# cat messages.1|grep openf|sort
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:asm10 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:asm1 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:asm2 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:asm3 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:asm4 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:asm5 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:asm6 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:asm7 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:asm8 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:asm9 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:ocfs 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:ocr1 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:ocr2 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:spfile 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:vote1 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:vote2 10.0.2.66 3260]
Mar 4 09:47:15 racbaby2 iscsid: session [iqn.2006-01.com.openfiler:vote3 10.0.2.66 3260]

7)COPY OVER SOFTWARE FILES FROM STAGING SERVER

Hangcheck Timer
hangcheck-0.8.0.tar.gz

Oracle CLUSTER and DB Software packages:
linux.x64_11gR1_clusterware.zip
linux.x64_11gR1_database_1013.zip
10201_database_linux_x86_64.cpio.gz

Oracle OCFS RPM s
ocfs2-2.6.18-92.el5-1.4.1-1.el5.x86_64.rpm
ocfs2-tools-1.4.1-1.el5.x86_64.rpm
ocfs2-tools-debuginfo-1.4.1-1.el5.x86_64.rpm
ocfs2-tools-devel-1.4.1-1.el5.x86_64.rpm
ocfs2console-1.4.1-1.el5.x86_64.rpm

ASMLib RPM s
oracleasm-2.6.18-92.1.22.el5-2.0.5-1.el5.x86_64.rpm
oracleasm-2.6.18-92.el5-2.0.5-1.el5.x86_64.rpm
oracleasm-support-2.1.2-1.el5.x86_64.rpm
oracleasmlib-2.0.3-1.el5.x86_64.rpm

homemade env variable switcher script
oradb

Oracle patchsets:
p6810189_10204_Linux-x86-64.zip
p6890831_111070_Linux-x86-64.zip

8)SET UP ISCSI ON RAC NODES

as root on each node:

service iscsi restart
chkconfig --level 345 iscsi on
iscsiadm -m discovery -t sendtargets -p openfiler1
service iscsi restart

verify disks are there

fdisk -l

run fdisk on each device to create a primary partition

Julie Watson

AntiRevoke

chip satış

Greetings! Very helpful advice within this post! It is the little changes that will make the most significant changes. Thanks for sharing!

Greetings! Very helpful advice within this post! It is the little changes that will make the most significant changes. Thanks for sharing!

unblocked games

unblocked games

Unlocked games at school and enjoy a lot. we share all not blocked games from Unlocked games 66, beast, 77 for school kids. unblocked games 500

คาสิโนออนไลน์

I was wondering if you ever thought of changing the layout of your website? Its very well written; I love what youve got to say. But maybe you could a little more in the way of content so people could connect with it better. Youve got an awful lot of text for only having 1 or 2 pictures. Maybe you could space it out better?

I was wondering if you ever thought of changing the layout of your website? Its very well written; I love what youve got to say. But maybe you could a little more in the way of content so people could connect with it better. Youve got an awful lot of text for only having 1 or 2 pictures. Maybe you could space it out better?

best forex trading robot

Thanks for any other wonderful post. Where else could anybody get that type of information in such a perfect approach of writing? I have a presentation next week, and I am at the search for such info.

Thanks for any other wonderful post. Where else could anybody get that type of information in such a perfect approach of writing? I have a presentation next week, and I am at the search for such info.

cheap jordans

Post writing is also a excitement, if you be acquainted with after that you can write if not it is difficult to write.

Post writing is also a excitement, if you be acquainted with after that you can write if not it is difficult to write.

anime

This website was... how do I say it? Relevant!! Finally I have found something that helped me. Thank you!

This website was... how do I say it? Relevant!! Finally I have found something that helped me. Thank you!

residential vpn

Hello, the whole thing is going fine here and ofcourse every one is sharing data, that's really excellent, keep up writing.

Hello, the whole thing is going fine here and ofcourse every one is sharing data, that's really excellent, keep up writing.

digital marketing manager

Accordingly, the mechanics in connection with promos in Facebook are generally easily manipulated or cheated by people who have a large network of friends. No one should go into a business not understanding what they are getting into. If bid costs are high for a particular keyword, simply could be unable to have the clicks you may need inside your limited budget.

Accordingly, the mechanics in connection with promos in Facebook are generally easily manipulated or cheated by people who have a large network of friends. No one should go into a business not understanding what they are getting into. If bid costs are high for a particular keyword, simply could be unable to have the clicks you may need inside your limited budget.

games

Because the admin of this web site is working, no hesitation very rapidly it will be famous, due to its feature contents.

Because the admin of this web site is working, no hesitation very rapidly it will be famous, due to its feature contents.