Cluster Database Setup
Setting up a clustered database is a lot like setting up an normal Oracle database. You have datafiles, controlfiles, redologs, rollback segments, and so on. With a clustered database you have a few new settings in your init.ora, and an second undo tablespace.
init.ora + config.ora setup
In a RAC environement, we finally see while Oracle has been recommending a separate config.ora and init.ora file all these years. config.ora contains instance specific parameters, such as the dump directories, name of the undo tablespace (there is one for each instance), and the instance and thread number. init.ora contains all common parameters two the database.
# config.ora for WEST instance
background_dump_dest=/home/oracle/admin/WEST/bdump
core_dump_dest=/home/oracle/admin/WEST/cdump
user_dump_dest=/home/oracle/admin/WEST/udump
undo_tablespace=UNDO_WEST
instance_name=WEST
instance_number=1
thread=1
# config.ora for EAST instance
background_dump_dest=/home/oracle/admin/EAST/bdump
core_dump_dest=/home/oracle/admin/EAST/cdump
user_dump_dest=/home/oracle/admin/EAST/udump
undo_tablespace=UNDO_EAST
instance_name=EAST
instance_number=2
thread=2
Notice that their are *TWO* undo tablespaces. In previous versions of Oracle this was rollback segment tablespace. At any rate each instance needs one. In the creating a RAC database section below, you’ll learn when and how these are created.
— initWEST.ora (on node 2 it’s initEAST.ora) —
# this is the only line that changes for each instance
ifile = /home/oracle/admin/WEST/pfile/configWEST.ora
control_files=
(/ocfs/oradata/EASTWEST/cntlEASTWEST01.ctl,
/ocfs/oradata/EASTWEST/cntlEASTWEST02.ctl,
/ocfs/oradata/EASTWEST/cntlEASTWEST03.ctl)
db_block_size=8192
# new Oracle9i parameter to set buffer cache size
db_cache_size=37108864
# if you have more instances, this number will be higher
cluster_database_instances=2
# see below for details
filesystemio_options=”directIO”
open_cursors=300
timed_statistics=TRUE
db_domain=localdomain
remote_login_passwordfile=EXCLUSIVE
# some stuff for Java
dispatchers=”(PROTOCOL=TCP)(SER=MODOSE)”, “(PROTOCOL=TCP)(PRE=Oracle.aurora.server.GiopServer)”, “(PROTOCOL=TCP)(PRE=Oracle.aurora.server.SGiopServer)”, “(PROTOCOL=TCP)”
compatible=9.0.0
# notice db name is different than instance names
db_name=EASTWEST
java_pool_size=12428800
large_pool_size=10485760
shared_pool_size=47440512
processes=150
fast_start_mttr_target=300
resource_manager_plan=SYSTEM_PLAN
sort_area_size=524288
undo_management=AUTO
cluster_database=true
That should do it. You may have more or less memory so adjust these values accordingly. Many of them are standard for non-RAC databases, so you’ll already be familiar with them. The Oracle docs are decent on explaining these in more detail, so check them for more info.
The init.ora parameter filesystemio_options is no longer a hidden parameter as of Oracle 9.2. The setting I use above is from Wim Coekaerts documentation. Arup Nanda says in the OPS days, “setall” was the setting he usually used. Your mileage may vary.
Steve Adam’s recommenations with respect to this parameter:
http://www.ixora.com.au/notes/filesystemio_options.htm
17. Creating the RAC database
This is much like creating a normal database. Most of the special stuff is in the init.ora and config.ora. The only new stuff is creating and enabling a separate undo tablespace, as well as second sets of redologs. Well you’re probably used to mirroring these anyway. Run this from node1.
— crEASTWEST.sql —
— send output to this logfile
spool crEASTWEST.log
startup nomount
— the big step, creates initial datafiles
create database EASTWEST
maxinstances 5
maxlogfiles 10
character set “we8iso8859p1”
datafile
‘/ocfs/oradata/EASTWEST/sysEASTWEST01.dbf’ size 500m reuse
default temporary tablespace tempts tempfile ‘/ocfs/oradata/EASTWEST/tmpEASTWEST01.dbf’ size 50m reuse
undo tablespace UNDO_WEST datafile ‘/ocfs/oradata/EASTWEST/undEASTWEST01.dbf’ size 50m reuse
logfile
‘/ocfs/oradata/EASTWEST/logEASTWEST01a.dbf’ size 25m reuse,
‘/ocfs/oradata/EASTWEST/logEASTWEST01b.dbf’ size 25m reuse;
— create the data dictionary
@?/rdbms/admin/catalog.sql
@?/rdbms/admin/catproc.sql
— create the second undo tablespace
create undo tablespace UNDO_EAST datafile
‘/ocfs/oradata/EASTWEST/undEASTWEST02.dbf’ size 50m reuse;
— create a second set of redologs
alter database add logfile thread 2 ‘/ocfs/oradata/EASTWEST/logEASTWEST02a.dbf’ size 25m reuse;
alter database add logfile thread 2 ‘/ocfs/oradata/EASTWEST/logEASTWEST02b.dbf’ size 25m reuse;
alter database enable thread 2;
shutdown immediate;
18. Startup of all instances
The magic step. Not a lot to it if all the above steps went
properly, but exciting none the less.
First on node1
$ sqlplus /nolog
SQL> connect / as sysdba
SQL> startup
Then the same thing on node2
$ sqlplus /nolog
SQL> connect / as sysdba
SQL> startup
Voila! You should be up and running at this point.
Errors. If you’re getting ORA-32700 like this:
SQL> startup
ORACLE instance started.
Total System Global Area 93393188 bytes
Fixed Size 450852 bytes
Variable Size 88080384 bytes
Database Buffers 4194304 bytes
Redo Buffers 667648 bytes
ORA-32700: error occurred in DIAG Group Service
It probably means oracm didn’t start properly. This would probably
give you trouble *CREATING* a database as well.
Part 2 – Basic Costs + Hardware Platform Outline
Part 3 – Software Requirements, Versions, etc
Part 5 – Firewire + OCFS Setup
Part 6 – Cluster Manager Setup
Part 7 – Cluster Database Setup
Part 8 – Review of Clustered Features + Architecture