Cluster Database Setup

Setting up a clustered database is a lot like setting up an normal Oracle database. You have datafiles, controlfiles, redologs, rollback segments, and so on. With a clustered database you have a few new settings in your init.ora, and an second undo tablespace.

init.ora + config.ora setup

In a RAC environement, we finally see while Oracle has been recommending a separate config.ora and init.ora file all these years. config.ora contains instance specific parameters, such as the dump directories, name of the undo tablespace (there is one for each instance), and the instance and thread number. init.ora contains all common parameters two the database.

# config.ora for WEST instance

background_dump_dest=/home/oracle/admin/WEST/bdump

core_dump_dest=/home/oracle/admin/WEST/cdump

user_dump_dest=/home/oracle/admin/WEST/udump

undo_tablespace=UNDO_WEST

instance_name=WEST

instance_number=1

thread=1

# config.ora for EAST instance

background_dump_dest=/home/oracle/admin/EAST/bdump

core_dump_dest=/home/oracle/admin/EAST/cdump

user_dump_dest=/home/oracle/admin/EAST/udump

undo_tablespace=UNDO_EAST

instance_name=EAST

instance_number=2

thread=2

Notice that their are *TWO* undo tablespaces. In previous versions of Oracle this was rollback segment tablespace. At any rate each instance needs one. In the creating a RAC database section below, you’ll learn when and how these are created.

— initWEST.ora (on node 2 it’s initEAST.ora) —

# this is the only line that changes for each instance

ifile = /home/oracle/admin/WEST/pfile/configWEST.ora

control_files=
(/ocfs/oradata/EASTWEST/cntlEASTWEST01.ctl,

/ocfs/oradata/EASTWEST/cntlEASTWEST02.ctl,

/ocfs/oradata/EASTWEST/cntlEASTWEST03.ctl)

db_block_size=8192

# new Oracle9i parameter to set buffer cache size

db_cache_size=37108864

# if you have more instances, this number will be higher

cluster_database_instances=2

# see below for details

filesystemio_options=”directIO”

open_cursors=300

timed_statistics=TRUE

db_domain=localdomain

remote_login_passwordfile=EXCLUSIVE

# some stuff for Java

dispatchers=”(PROTOCOL=TCP)(SER=MODOSE)”, “(PROTOCOL=TCP)(PRE=Oracle.aurora.server.GiopServer)”, “(PROTOCOL=TCP)(PRE=Oracle.aurora.server.SGiopServer)”, “(PROTOCOL=TCP)”

compatible=9.0.0

# notice db name is different than instance names

db_name=EASTWEST

java_pool_size=12428800

large_pool_size=10485760

shared_pool_size=47440512

processes=150

fast_start_mttr_target=300

resource_manager_plan=SYSTEM_PLAN

sort_area_size=524288

undo_management=AUTO

cluster_database=true

That should do it. You may have more or less memory so adjust these values accordingly. Many of them are standard for non-RAC databases, so you’ll already be familiar with them. The Oracle docs are decent on explaining these in more detail, so check them for more info.

The init.ora parameter filesystemio_options is no longer a hidden parameter as of Oracle 9.2. The setting I use above is from Wim Coekaerts documentation. Arup Nanda says in the OPS days, “setall” was the setting he usually used. Your mileage may vary.

Steve Adam’s recommenations with respect to this parameter:

http://www.ixora.com.au/notes/filesystemio_options.htm

17. Creating the RAC database

This is much like creating a normal database. Most of the special stuff is in the init.ora and config.ora. The only new stuff is creating and enabling a separate undo tablespace, as well as second sets of redologs. Well you’re probably used to mirroring these anyway. Run this from node1.

— crEASTWEST.sql —

— send output to this logfile

spool crEASTWEST.log

startup nomount

— the big step, creates initial datafiles

create database EASTWEST

maxinstances 5

maxlogfiles 10

character set “we8iso8859p1”

datafile
‘/ocfs/oradata/EASTWEST/sysEASTWEST01.dbf’ size 500m reuse

default temporary tablespace tempts tempfile ‘/ocfs/oradata/EASTWEST/tmpEASTWEST01.dbf’ size 50m reuse

undo tablespace UNDO_WEST datafile ‘/ocfs/oradata/EASTWEST/undEASTWEST01.dbf’ size 50m reuse

logfile
‘/ocfs/oradata/EASTWEST/logEASTWEST01a.dbf’ size 25m reuse,

‘/ocfs/oradata/EASTWEST/logEASTWEST01b.dbf’ size 25m reuse;

— create the data dictionary

@?/rdbms/admin/catalog.sql

@?/rdbms/admin/catproc.sql

— create the second undo tablespace

create undo tablespace UNDO_EAST datafile
‘/ocfs/oradata/EASTWEST/undEASTWEST02.dbf’ size 50m reuse;

— create a second set of redologs

alter database add logfile thread 2 ‘/ocfs/oradata/EASTWEST/logEASTWEST02a.dbf’ size 25m reuse;

alter database add logfile thread 2 ‘/ocfs/oradata/EASTWEST/logEASTWEST02b.dbf’ size 25m reuse;

alter database enable thread 2;

shutdown immediate;

18. Startup of all instances

The magic step. Not a lot to it if all the above steps went

properly, but exciting none the less.

First on node1

$ sqlplus /nolog

SQL> connect / as sysdba

SQL> startup

Then the same thing on node2

$ sqlplus /nolog

SQL> connect / as sysdba

SQL> startup

Voila! You should be up and running at this point.

Errors. If you’re getting ORA-32700 like this:

SQL> startup

ORACLE instance started.

Total System Global Area 93393188 bytes

Fixed Size 450852 bytes

Variable Size 88080384 bytes

Database Buffers 4194304 bytes

Redo Buffers 667648 bytes

ORA-32700: error occurred in DIAG Group Service

It probably means oracm didn’t start properly. This would probably

give you trouble *CREATING* a database as well.

Part 1 – Introduction

Part 2 – Basic Costs + Hardware Platform Outline

Part 3 – Software Requirements, Versions, etc

Part 4 – Initial Oracle Setup

Part 5 – Firewire + OCFS Setup

Part 6 – Cluster Manager Setup

Part 7 – Cluster Database Setup

Part 8 – Review of Clustered Features + Architecture

Part 9 – A quick 9iRAC example

Part 10 – Summary