Part 7: RAC/Linux/Firewire – Cluster Database Setup

Cluster Database Setup


Setting up a clustered database is a lot like setting up an normal Oracle database. You have datafiles, controlfiles, redologs, rollback segments, and so on. With a clustered database you have a few new settings in your init.ora, and an second undo tablespace.

init.ora + config.ora setup

In a RAC environement, we finally see while Oracle has been recommending a separate config.ora and init.ora file all these years. config.ora contains instance specific parameters, such as the dump directories, name of the undo tablespace (there is one for each instance), and the instance and thread number. init.ora contains all common parameters two the database.

# config.ora for WEST instance

background_dump_dest=/home/oracle/admin/WEST/bdump

core_dump_dest=/home/oracle/admin/WEST/cdump

user_dump_dest=/home/oracle/admin/WEST/udump

undo_tablespace=UNDO_WEST

instance_name=WEST

instance_number=1

thread=1

# config.ora for EAST instance

background_dump_dest=/home/oracle/admin/EAST/bdump

core_dump_dest=/home/oracle/admin/EAST/cdump

user_dump_dest=/home/oracle/admin/EAST/udump

undo_tablespace=UNDO_EAST

instance_name=EAST

instance_number=2

thread=2

Notice that their are *TWO* undo tablespaces. In previous versions of Oracle this was rollback segment tablespace. At any rate each instance needs one. In the creating a RAC database section below, you’ll learn when and how these are created.

— initWEST.ora (on node 2 it’s initEAST.ora) —

# this is the only line that changes for each instance

ifile = /home/oracle/admin/WEST/pfile/configWEST.ora

control_files=
(/ocfs/oradata/EASTWEST/cntlEASTWEST01.ctl,

/ocfs/oradata/EASTWEST/cntlEASTWEST02.ctl,

/ocfs/oradata/EASTWEST/cntlEASTWEST03.ctl)

db_block_size=8192

# new Oracle9i parameter to set buffer cache size

db_cache_size=37108864

# if you have more instances, this number will be higher

cluster_database_instances=2

# see below for details

filesystemio_options=”directIO”

open_cursors=300

timed_statistics=TRUE

db_domain=localdomain

remote_login_passwordfile=EXCLUSIVE

# some stuff for Java

dispatchers=”(PROTOCOL=TCP)(SER=MODOSE)”, “(PROTOCOL=TCP)(PRE=Oracle.aurora.server.GiopServer)”, “(PROTOCOL=TCP)(PRE=Oracle.aurora.server.SGiopServer)”, “(PROTOCOL=TCP)”

compatible=9.0.0

# notice db name is different than instance names

db_name=EASTWEST

java_pool_size=12428800

large_pool_size=10485760

shared_pool_size=47440512

processes=150

fast_start_mttr_target=300

resource_manager_plan=SYSTEM_PLAN

sort_area_size=524288

undo_management=AUTO

cluster_database=true

That should do it. You may have more or less memory so adjust these values accordingly. Many of them are standard for non-RAC databases, so you’ll already be familiar with them. The Oracle docs are decent on explaining these in more detail, so check them for more info.

The init.ora parameter filesystemio_options is no longer a hidden parameter as of Oracle 9.2. The setting I use above is from Wim Coekaerts documentation. Arup Nanda says in the OPS days, “setall” was the setting he usually used. Your mileage may vary.

Steve Adam’s recommenations with respect to this parameter:

http://www.ixora.com.au/notes/filesystemio_options.htm

17. Creating the RAC database

This is much like creating a normal database. Most of the special stuff is in the init.ora and config.ora. The only new stuff is creating and enabling a separate undo tablespace, as well as second sets of redologs. Well you’re probably used to mirroring these anyway. Run this from node1.

— crEASTWEST.sql —

— send output to this logfile

spool crEASTWEST.log

startup nomount

— the big step, creates initial datafiles

create database EASTWEST

maxinstances 5

maxlogfiles 10

character set “we8iso8859p1”

datafile
‘/ocfs/oradata/EASTWEST/sysEASTWEST01.dbf’ size 500m reuse

default temporary tablespace tempts tempfile ‘/ocfs/oradata/EASTWEST/tmpEASTWEST01.dbf’ size 50m reuse

undo tablespace UNDO_WEST datafile ‘/ocfs/oradata/EASTWEST/undEASTWEST01.dbf’ size 50m reuse

logfile
‘/ocfs/oradata/EASTWEST/logEASTWEST01a.dbf’ size 25m reuse,

‘/ocfs/oradata/EASTWEST/logEASTWEST01b.dbf’ size 25m reuse;

— create the data dictionary

@?/rdbms/admin/catalog.sql

@?/rdbms/admin/catproc.sql

— create the second undo tablespace

create undo tablespace UNDO_EAST datafile
‘/ocfs/oradata/EASTWEST/undEASTWEST02.dbf’ size 50m reuse;

— create a second set of redologs

alter database add logfile thread 2 ‘/ocfs/oradata/EASTWEST/logEASTWEST02a.dbf’ size 25m reuse;

alter database add logfile thread 2 ‘/ocfs/oradata/EASTWEST/logEASTWEST02b.dbf’ size 25m reuse;

alter database enable thread 2;

shutdown immediate;

18. Startup of all instances

The magic step. Not a lot to it if all the above steps went

properly, but exciting none the less.

First on node1

$ sqlplus /nolog

SQL> connect / as sysdba

SQL> startup

Then the same thing on node2

$ sqlplus /nolog

SQL> connect / as sysdba

SQL> startup

Voila! You should be up and running at this point.

Errors. If you’re getting ORA-32700 like this:

SQL> startup

ORACLE instance started.

Total System Global Area 93393188 bytes

Fixed Size 450852 bytes

Variable Size 88080384 bytes

Database Buffers 4194304 bytes

Redo Buffers 667648 bytes

ORA-32700: error occurred in DIAG Group Service

It probably means oracm didn’t start properly. This would probably

give you trouble *CREATING* a database as well.


Part 1 – Introduction

Part 2 – Basic Costs + Hardware Platform Outline

Part 3 – Software Requirements, Versions, etc

Part 4 – Initial Oracle Setup

Part 5 – Firewire + OCFS Setup

Part 6 – Cluster Manager Setup

Part 7 – Cluster Database Setup

Part 8 – Review of Clustered Features + Architecture

Part 9 – A quick 9iRAC example

Part 10 – Summary

Part 6: RAC/Linux/Firewire – Cluster Manager Setup

Cluster Manager Setup


The cluster manager software is how the Oracle instances communicate their activities. Obviously this is an important piece to the puzzle as well. Here I review the configs, and then show how to get it up and running on each node. I *DID NOT* patch the cluster manager with the 9.2.0.2 db patch, but your mileage may vary.

Edit file $ORACLE_HOME/oracm/admin/cmcfg.ora

HeartBeat=10000

ClusterName=Oracle Cluster Manager, version 9i

PollInterval=300

PrivateNodeNames=zenith utopia

PublicNodeNames=zenith utopia

ServicePort=9998

HostName=zenith

#CmDiskFile=/ocfs/oradata/foo

MissCount=5

WatchdogSafetyMargin=3000

WatchdogTimerMargin=6000

Note, if you patch oracm to 9.2.0.2, remove the two Watchdog lines, and uncomment and use the CmDiskFile.

Edit file $ORACLE_HOME/oracm/admin/ocmargs.ora

watchdogd -d /dev/null -l 0

oracm /a:0

norestart 1800

Note, if you patch oracm to 9.2.0.2, comment out the watchdog line.

Now *AS ROOT* start up the cluster manager:

$ ./$ORACLE_HOME/oracm/bin/ocmstart.sh

You should see 8 processes with “ps -auxw | grep oracm”. Note that if you are running RH8, there’s a new ps which needs a special option “m” to notice threads. Apparently oracm is threaded (Thanks Wim). This had me pulling my hair out for weeks, and I’m bald! Anyway if that is the case, use “ps auxwm | grep oracm”. One more little recommendation. oracm is communicating via a port which you define. If you’re using iptables/ipchains, or some other firewall solution, I would recommend disabling it, at least temporarily, until you know you’ve configured everything right. Then reenable it, being sure you are good at configuring just the ports you need.

15. Perform step 14 on node 2.

Part 1 – Introduction

Part 2 – Basic Costs + Hardware Platform Outline

Part 3 – Software Requirements, Versions, etc

Part 4 – Initial Oracle Setup

Part 5 – Firewire + OCFS Setup

Part 6 – Cluster Manager Setup

Part 7 – Cluster Database Setup

Part 8 – Review of Clustered Features + Architecture

Part 9 – A quick 9iRAC example

Part 10 – Summary

Part 5: RAC/Linux/Firewire – Firewire + OCFS Setup

Firewire + OCFS Setup


In this installment, we’ll discuss how to get the Firewire drive shared between your two Linux boxes.

8. Test Firewire drive

At this point you can test the firewire drive if you like, with the standard Linux driver. You won’t be able to share the drive between the two nodes yet, however.

As root do the following:

$ modprobe ohci1394

$ modprobe ieee1394

$ modprobe sbp2

$ modprobe scsi_mod

Grab a copy of rescan-scsi-bus.sh from here and run it:
http://www.fifi.org/cgi-bin/man2html/usr/share/man/man8/rescan-scsi-bus.sh.8.gz

run rescan scsi bus

Now partition it with fdisk:

$ fdisk /dev/sda

Now try making an ext2 filesystem with mke2fs

$ mke2fs /dev/sda1

Now mount it

$ mount -t ext2 /dev/sda1 /mnt/test

Now unmount it

$ umount /mnt/test

9. Linux Kernel Setup w/Firewire patch

The Linux kernel is a complex beast, and compiling it can often be a challenge. Though I like rolling my own, I downloaded the patched firewire source distro off of OTN, and try as I might, I could not get those compiled kernels to work. If anyone *DOES* get it to work, please send me their “.config” from the kernel source directory. Also I’ve tried to encourage the Oracle/Linux Firewire team to build a patch-only distro which can be applied against a standard Linux source tree. No luck yet.

Assuming you’re not going to roll your own, just download linux-2.4.20rc2-orafw-up.tar.gz from here:

http://otn.oracle.com/tech/linux/open_source.html

Move to the “/” or root directory, and untar the file:

$ tar xvzf linux-2.4.20rc2-orafw-up.tar.gz

Edit your /etc/lilo.conf or /etc/grub.conf file to include the new kernel. Do *NOT* make it the default kernel, it may not boot.

Reboot. If you come up again, you’re in luck, the kernel works for your machine. Next you want to edit your /etc/modules.conf to include these lines:

# options for oracle firewire patched kernel

options sbp2 sbp2_exclusive_login=0

post-install sbp2 insmod sd_mod

post-remove sbp2 rmmod sd_mod

As root, load the modules like this:

$ modprobe ieee1394

$ modprobe ohci1394

$ modprobe ide-scsi

$ modprobe sbp2

$ modprobe scsi_mod

If you’re having trouble seeing the device, grab a copy of rescan-scsi-bus.sh from here:
http://www.fifi.org/cgi-bin/man2html/usr/share/man/man8/rescan-scsi-bus.sh.8.gz

If you want to partition, now is a good time. Use fdisk as root like this:

$ fdisk /dev/sda

If you have other SCSI devices, it may be /dev/sdb or dev/sdc and so on.

10. Go through steps 1-8 on node 2

11. Cluster Filesystem setup (OCFS)

If you wanna play around, use mke2fs on the one of the partitions you created with fdisk, and then mount the partiton on machine a. Then mount the partition again on machine b. Create a file on one of the two boxes. The other machine *WON’T* reflect it. This is equivalent to unplugging a disk which is mounted, such as a USB device, or some such. You can and probably *HAVE* corrupted the filesystem. That’s ok, because we don’t have anything important on the disk yet. Ok, unmount on both machines. If you have trouble, you may need to reboot.

Having gone through the above example, you know why OCFS is so important. Ok, now the fun part. Install OCFS. There are good docs to be found in the linux_ocfs.pdf file here:

http://download.oracle.com/otn/linux/code/ocfs/linux_ocfs.pdf

Without RedHat Advanced Server, the RPMs are *NOT* going to work. Just grab a copy of ocfs-1.0-up.o and put it in
/lib/modules/2.4.20-rc2-orafw/kernel/fs.

Use ocfstool to create the /etc/ocfs.conf file. The pdf doc listed above is pretty good at explaining this.

Load the ocfs kernel module with load_ocfs. If everything goes right it will tell you like this:

$ cd /lib/modules/2.4.20-rc2-orafw/kernel/fs

$ load_ocfs

/sbin/insmod ocfs node_name=zenith
ip_address=192.168.0.9 ip_port=7000 cs=1865 guid=72C2AF5CA29FA17CB9CB000AE6312F24

Using /lib/modules/2.4.20-rc2-orafw/kernel/fs/ocfs.o

Next make the filesystem. ocfstool can do this too.

$ mkfs.ocfs -F -b 128 -L /ocfs -m /ocfs -u 1001 -g 1001 -p 0775 /dev/sda1

And finally mount the filesystem!

$ mount -t ocfs /dev/sda1 /ocfs

$ df -k

Filesystem 1K-blocks Used Available Use% Mounted on

/dev/hda2 55439548 20835260 31788096 40% /

/dev/hda1 101089 18534 77336 20% /boot

none 112384 0 112384 0% /dev/shm

/dev/cdrom 122670 122670 0 100% /mnt/cdrom

/dev/sda1 60049024 30080 60018944 1% /ocfs

12. Perform step 10 on node 2.

13. Test ocfs

Here we quickly verify that a file created on one instance is viewable on another.

On node1 do:

$ cd /ocfs

$ touch mytestfile

On node2 do:

$ cd /ocfs

$ ls

mytestfile

$

You’ll see to your astonishment that the file is now visible on node 2!

Part 1 – Introduction

Part 2 – Basic Costs + Hardware Platform Outline

Part 3 – Software Requirements, Versions, etc

Part 4 – Initial Oracle Setup

Part 5 – Firewire + OCFS Setup

Part 6 – Cluster Manager Setup

Part 7 – Cluster Database Setup

Part 8 – Review of Clustered Features + Architecture

Part 9 – A quick 9iRAC example

Part 10 – Summary

Part 4: RAC/Linux/Firewire – Initial Oracle Setup

Initial Oracle Setup


Follow these instructions to get Oracle up and running on your new Linux box.

1. Setup the oracle account, and environment as follows.

Create an oracle user on your linux box:

$ adduser oracle

2. Login to the oracle account and edit .oraenv9i as follows (assumes bash):

# oracle environment variables

export ORACLE_BASE=”/home/oracle”

export ORACLE_HOME=”/home/oracle/product/9.2.0″

# EAST on machines #2

export ORACLE_SID=”WEST”

export PATH=$PATH:$ORACLE_HOME/bin

export LD_LIBRARY_PATH=”$ORACLE_HOME/lib”

export LD_ASSUME_KERNEL=2.2.5

# US7ASCII is the default, but WE8ISO8859P1 support more languages

export NLS_LANG=AMERICAN_AMERICA.WE8ISO8859P1

export ORA_NLS33=$ORACLE_HOME/ocommon/nls/admin/data

export ORACLE_TERM=xterm

export ORACLE_OWNER=oracle

export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib:/usr/local/lib

export TNS_ADMIN=$ORACLE_HOME/network/admin

# run the RH compatability stuff

. /usr/i386-glibc21-linux/bin/i386-glibc21-linux-env.sh

# setup Java

export JAVA_HOME=/usr/local/java

export CLASSPATH=$ORACLE_HOME/jdbc/lib/classes12.zip:$ORACLE_HOME/JRE:$ORACLE_HOM

E/jlib:$ORACLE_HOME/rdbms/jlib:$ORACLE_HOME/network/jlib:.

Add this line to your .bash_profile:

. /home/oracle/.oraenv9i

Next install the glibc backward compatability libs per

compat-

egcs-6.2-1.1.2.14.i386.rpm

compat-

glibc-6.2-2.1.3.2.i386.rpm

compat-libs-6.

2-3.i386.rpm

3. Next install JDK.

4. Edit the hosts file if necessary to include two machines on your local network

by name. I used “utopia” and “zenith”.

5. Next run the Oracle installer as follows (assuming you opened the packages in

/tmp). If you’re not on the X-window console, connect with SSH and you should ha

ve x tunneling by default, and can display remotely:

$ ./tmp/Disk1/install/linux/runInstaller

First install the cluster manager. You’ll be prompted for the local and remote h

ostname, a quorum disk, as well as some other things. Doesn’t matter what you en

ter right now, as we’re going to go back and edit those files and do things by ha

nd anyway.

6. Next go through the 9.2.0.1 software install. Here are some other notes with

various weblinks. Be sure to select “Enterprise Edition” and also “Software Only

Install”.

I got started with this Oracle 8 install doc which details which compatability li

braries you’ll need, how to setup the Oracle account, environment variables, and

Java.

link

I ran into some trouble with LD_ASSUME_KERNEL, and compat libs:

You’ll encounter problems with the context makefile, and get this error “Error in

invoking target install of makefile /opt/oracle/product/9.2.0/ctx/lib/ ins_ctx.m

k”. Sad but true, this is *NORMAL* behavior. I suspect making the installer run

flawlessly isn’t at the top of Oracle’s priority list.

link

I also encountered problems with Oracle Net Configuration Assistant “Can’t find $

ORACLE_HOME/jre/1.1.8/bin/../bin/i586/green_threads/jre” and I added this symlink

:

$ ln -s $ORACLE_HOME/jre/1.1.8/bin/i686
$ORACLE_HOME/jre/1.1.8/bin/i586

I got a simlar error for libjava.so:

$ ln -s $ORACLE_HOME/jre/1.1.8/lib/i686
$ORACLE_HOME/jre/1.1.8/lib/i586

Though you’ll probably want to create your own database from scratch later, it’s

sometimes instructive to let the Oracle Database Configuration Assistant create a

starter one for you, and look at what options they use. As with the rest, you’l

l run into an error. This time it is “ORA-27123 unable to attach to shared memor

y segment: Oracle Database Configuration Assistant”. Fix it by doing the followi

ng as root on your linux box:

$ cat /proc/sys/kernel/shmmax

33554432

$ echo `expr 1024 * 1024 * 1024` > /proc/sys/kernel/shmmax

$ cat /proc/sys/kernel/shmmax

1073741824

Obviously you’ll have to go through this process (living hell?) on both boxes you

‘ll be using in your cluster.

Next rerun the installer and specify the location of the itty bitty 235M patch, a

nd install that.

7. Enable RAC in Oracle9i

The Real Application Cluster feature is *NOT* enabled by default. Here’s how you

enable it:

As the oracle user:

$ cd $ORACLE_HOME/rdbms/lib

$ make -f ins_rdbms.mk rac_on

$ make -f ins_rdbms.mk ioracle

As root set permissions on rac_on

$ chown oracle /etc/rac_on

$ chgrp dba /etc/rac_on

Part 1 – Introduction

Part 2 – Basic Costs + Hardware Platform Outline

Part 3 – Software Requirements, Versions, etc

Part 4 – Initial Oracle Setup

Part 5 – Firewire + OCFS Setup

Part 6 – Cluster Manager Setup

Part 7 – Cluster Database Setup

Part 8 – Review of Clustered Features + Architecture

Part 9 – A quick 9iRAC example

Part 10 – Summary

Part 3: Rac/Linux/Firewire – Software Requirements, Versions, etc.

Software Requirements, versions, etc


Advance notice for beginners out there. I’ve installed Linux probably 20-30 times over the last 10 years, and it’s gotten a lot easier, however, you’ll need to have some decent skill at kernel installations. I’ll outline what I did that finally worked, but I’ll also say that I tried getting this setup to work on a couple of other boxes before I ended up using the emachines ones. I think the kernel which the Linux Firewire team has compiled and patched to their liking is pretty finicky, hardware-wise.

RedHat 8.0

I can’t go into the details of installing Linux in this short article, and there are plenty of resources on the web to help you. You’ll need to get that up and running first, with the 1394 card in the box.

Oracle 9.2.0.1

Oracle is a beast to install, and challenging for the beginner. Luckily there are a number of resources on the web.

patch 9.2.0.2

This patch is a *REQUIRED* patch. The guys on the Linux Firewire project assure me that the Oracle Cluster Manager has a lot of bugs and won’t work straight out of the box from the 9.2.0.1 release. Here’s a little twist. Although this is an *OPEN SOURCE* project, and the source is all free, and you can download Oracle and play with it to your heart’s content, to download a patch, you need access to Oracle’s Metalink, which requires you to have a software license. Not much I can say here folks. Use your imagination on getting ahold of this patch. Maybe if you place nice, someone on an email list will post it temporarily for you or something. It’s 235M. 🙂

Login to Metalink (some like to call it metastink)

Click on the “patches” link along the left.

Search for patch 2632931

Also, take a look at this note:

Doc ID: Note:217811.1

Subject: ALERT: 9.2.0.2 Patchset for Linux Does Not

Apply Correctly in the RAC Environment

Type: ALERT

Status: PUBLISHED

Content Type: TEXT/PLAIN

Creation Date: 07-NOV-2002

Last Revision Date: 08-JAN-2003

Part 1 – Introduction

Part 2 – Basic Costs + Hardware Platform Outline

Part 3 – Software Requirements, Versions, etc

Part 4 – Initial Oracle Setup

Part 5 – Firewire + OCFS Setup

Part 6 – Cluster Manager Setup

Part 7 – Cluster Database Setup

Part 8 – Review of Clustered Features + Architecture

Part 9 – A quick 9iRAC example

Part 10 – Summary

Asterisk Calling Card Applications

Asterisk is a powerful PBX solution, that we already know. But what else can it do. In this article we’ll explain how to setup Asterisk to handle Call Data Records (CDR data) in MySQL. Once you have that configured, there are a number of calling card applications which can be integrated with Asterisk to provide you with the makings of a serious calling gateway.


Setup Asterisk CDR with MySQL

By default Asterisk pumps all it’s call data information to text-based log files. That’s fine for normal use, but what if you want to put that data to use in a calling card application? First you have to get Asterisk to use a database. Luckily the support is already there, all you have to do is configure it.


Start by editing your cdr_manager.conf file as follows:


enabled = yes

Next edit your modules.conf file, and somewhere in the [modules] section, add:


load => cdr_addon_mysql.so

We’re going to compile this, don’t worry. Next edit your cdr_mysql.conf file in /etc/asterisk or create it if necessary:


[global]

hostname=localhost

dbname=asteriskcdrdb

user=astxuser

;user=

password=astxpass

;password=

port=3306

sock=/var/lib/mysql/mysql.sock

;sock=/tmp/mysql.sock

userfield=1

Next install MySQL. Luckily for all you lazy bums out there, this is the simplest of all. You’ll need to download three RPMs and install them. You’ll need the latest version of mysql-server, mysql-client and finally mysql-devel.


Next you’ll create a database called “asteriskcdrdb” with mysqladmin, create a table named “cdr” with the Asterisk provided script, and then set user grants.


Now it’s time to compile the asterisk-addons package. Be sure you have zlib-devel and mysql-devel packages installed on your system or you may get errors. Checkout the source from cvs. I got some strange errors which I had to track down on the email lists, and then edit the makefile as shown below:


CFLAGS+=-DMYSQL_LOGUNIQUEID

Now stop asterisk, and start it up again, and monitor the asterisk logfile for errors as follows:


tail -f /var/log/asterisk/messages

You can finally verify that you are dumping cdr information into mysql as follows:


$ mysql asteriskcdrdb

mysql> select uniqueid, src, calldate from cdr;



There should be one entry for every call. Make some calls to local

extensions and verify that records show up here. New cdr records

will still show up in the /var/log/asterisk/cdr-csv/Master.csv

file. Not sure if this can be disabled.


Calling Card Applications


ASTCC

Though the homepage is just a voip-info wiki page

and the download available through CVS, this calling card application was updated in late December 2004. This application seems to be the winner in terms of popularity on the voip-info wiki. It comes from Digium, it supports MySQL, and setup is pretty straightforward.

AreskiCC

With a strange name, it nevertheless seems a pretty complete system. Last updated end of December, 2004, it includes a web interface, though no support for MySQL. That’s fine, but my MySQL setup instructions will need to change slightly as you’ll need to configure Asterisk to dump CDR data into Postgres.


Asterisk Billing – Prepaid application

Last updated in July, I had trouble compiling this application. There is a basic sourceforge download page, but no real homepage. I’m guessing this one is still sort of in the development stages. Also, it doesn’t come with any sound files, so you’ll have to record your own, or *borrow* from some of these other applications.

Open Insights 26 – Logistical Fitness

OPEN INSIGHTS Newsletter
Issue 26 – Logistical Fitness
November 1, 2006

by Sean Hull

Founder and Senior Consultant
Heavyweight Internet Group

Happy Holidays to everyone and welcome to our free monthly newsletter, discussing news, developments and business best practices at the intersection of Oracle and Open Source software.


In This Issue:

1. Feature: Logistical Fitness
2. New Articles
3. Audio Interviews
4. Current Reading
5. Lightweight Humor
6. Past Issues
7. Technical Articles
8. About Heavyweight Internet Group


1. Feature: Logistical Fitness

A few months back, the Economist featured a special survey on “logistics” that emerging science of supply chain management. They talked about what they termed the “physical internet”, ie the delivery of products in the global marketplace.

More and more the winners in business are the ones who manage their supply chains best. Zara, an example in the fashion industry, is a company that has managed to deliver clothing inspired by the latest runway trends five weeks later to the checkout. This is absolutely astonishing, given that other companies take six to nine months to do the same. They manage this feat with a mix of close coordination with all of their stores, and a Dell computer-like supply chain coordinating material from Asia, and production at their base in North-west Spain.

Wal-mart has managed to grow into the giant they are today by similar management of it’s supply chain, avoiding too much stock, and attendant sales to trim down, and shortages on the other hand. The head office has practically instantaneous information about stocked items at their different stores, providing unprecedented logistical information.

These same efficiencies are what has driven Dell computer to the top, allowing it to build each computer to a customer’s own specification, and keeping little inventory in the process.

All these efficiencies drive down costs, and make these companies incredibly nimble, despite their size. And what makes all this supply chain magic work together? At the heart of all of these technologies are enterprise databases like Oracle, and suites of software like Oracle Applications, or SAP AG.

So the next time you order a package from Amazon.com, track it day-by-day or hour-by-hour on Fedex.com, and are only mildly impressed when it shows up from halfway around the world at your doorstep in only a couple of days, realize that is phenomenal logistics, and supply chain management at work that makes it all possible.

If you’d like to download the Economist section on Logistics from June 2006 go here.

Other articles can be found at Yossi Sheffi’s “Resilient Enterprise” website.

Also visit the Council of Supply Chain Management Professionals

2. New Articles

Oracle 10g RAC versus DataGuard for High Availability

3. Audio Interviews

This month we have the opportunity to talk with William Hurley aka Whurley, the Chairman of the Open Management Consortium.

In our interview, we discuss open-source, and it’s impact on commercial software and solutions, and wrestle head on with some of the concerns people have on both sides of the fence.

William Hurley is the CTO at Qlusters, where he launched the openQRM project. He has been awarded IBM’s Master Inventor title, multiple awards for innovation at Apple Computer. Prior to joining Qlusters he was CTO and founder at Symbiot. He holds 11 patents for research and development at IBM, Tivoli Systems, and Apple Computer. He was recently elected Chairman of the Open Management Consortium.

4. Current Reading

The Resilient Enterprise by Yossi Sheffi

Mr. Sheffi’s new book explains how supply chains are transforming companies, making them more competitive and nimble in the process. With plenty of success and failure stories, it is very readable material.

iWoz by Steve Wozniak and Gina Smith

The original innovator, and inventor of the personal computer, Steve Wozniak discusses the fascinating origins of Apple computer, and how it rose to be the incredible company it is today.

The New CIO Leader by Marianne Broadbent & Ellen Kitzis

This is really a how-to book, offering insights and advice to CIOs on how to transform their position and take on a more prominent influential role in the enterprise.

5. Lightweight Humor

Check out the funny fake walmart blog at Gaping Void.

6. Past Issues
Issue 25: Which Red Button
Issue 24: Consulting Conflicts of Interest
Issue 23: Devil In The Details
Issue 22: Beware of Software Fashion
Issue 21: Open Season, Open Sesame?
Issue 20: Better Web Better Business
Archive: Past Issues

7. Technical Articles

Oracle DBA Interview: click here
Tools for the Intrepid DBA: click here
Oracle9i + RAC on Linux/Firewire: click here
Migrating MySQL to Oracle: click here
MySQL Disaster Recovery: click here

8. About Heavyweight Internet Group

In a nutshell, Oracle. Everything related to and surrounding the database technology we specialize in, but specifically setup, admin and tuning of Oracle technology. I have 10 years experience with Oracle, wrote a book on the technology, and write and lecture frequently. I’m founder and senior consultant of the company. In capacities where your company might hire Deloitte, AIG, or Oracle Consulting we can bring the same level of service and experience, at about half the price. Simple equation.

Looking for top-flight a DBA? Visit us on the web at iheavy.com.

Open Insights Newsletter

Welcome to our free monthly newsletter, discussing news, developments and business best practices at the intersection of Oracle and Open Source software. Click the links along the right to view past newsletters.

Please forward to interested friends and colleagues. To subscribe to our email list, click here for the list homepage.

Older Issues

Issue 39 (December 2007) – Reputation Management

Issue 38 (November 2007) – Are You Fast Failing?

Issue 37 (October 2007) – A Real Open Book

Issue 36 (September 2007) – Rarity of Excellence

Issue 35 – not published

Issue 34 (August 2007) – Hindsight Is Always 20/20

Issue 33 (July 2007) – Market For Experts

Issue 32 (June 2007) – Different Heritages

Issue 31 (May 2007) – Auto or Traffic Engineer

Issue 30 (April 2007) – Crowdsourcing

Issue 29 (March 2007) – Mainroads or Sidestreets

Issue 28 (February 2007) – High Availability

Issue 27 (January 2007) – Fragile Foundations

Issue 26 (December 2006) – Logistical Fitness

Issue 25 (November 2006) – Which Red Button?

Issue 24 (October 2006) – Consulting Conflicts of Interest

Issue 23 (September 2006) – Devil In The Details

Issue 22 (August 2006) – Beware of Software Fashion

Issue 21 (July 2006) – Open Season, Open Sesame?

Issue 20 (June 2006) – Better Web, Better Business

Issue 19 (May 2006) – Avoiding a Fixed Fee Fix

Issue 18 (April 2006) – The Cost of Consulting

Issue 17 (March 2006) – Secrets of the Interview

Issue 16 (February 2006) – Success in Juggling

Issue 15 (January 2006) – Marketing About Technology

Issue 14 (December 2005) – The Tricky Database

Issue 13 (November 2005) – Oracle Heavy Lifting

Issue 12 (October 2005) – What the Geeks Mean

Issue 11 (September 2005) – Google Wave or Tsunami

Issue 10 (August 2005) – Do You Arbitrage?

Issue 09 (July 2005) – IT Certifications

Issue 08 (June 2005) – The Devil Is In The Details

Issue 07 (May 2005) – Open Source In The Enterprise

Issue 06 (April 2005) – Practice What You Preach

Issue 05 (March 2005) – Building on Success

Issue 04 (February 2005) – Managers: Databases 101

Issue 03 (January 2005) – The Business of Open Source

Issue 02 (December 2004) – Consulting Apples and Oranges

Issue 01 (November 2004) – Newsletter Introduction

Open Insights 25: Which Red Button


OPEN INSIGHTS Newsletter

Issue 25 – Which Red Button?

November 1, 2006
by Sean Hull
Founder and Senior Consultant
Heavyweight Internet Group


Welcome to our free monthly newsletter, discussing news, developments and business best practices at the intersection of Oracle and Open Source software.


In This Issue:

1. Feature: Which Red Button?
2. Upcoming Speaking Engagements
3. Published Articles
4. Audio Interviews
5. Current Reading
6. Lightweight Humor
7. Of Interest
8. Upcoming Events
9. Past Issues
10. Technical Articles
11. About Heavyweight Internet Group


1. Feature: Which Red Button?

Many past newsletters have been inspired by real experiences I’ve had at client sites, and this month’s story falls into that category.

A few weeks back I had the pleasure of bringing on a new hire at a client, and bringing them up to speed with the systems. After a couple of weeks of training, they began to accept heavier responsibilities. While I was at another site, they were asked to perform some maintenance on a production server. Through miscommunication, mislabeling, or other error that any of us could fall prey to, an entire production database was deleted.

Although backups were in order, it took more than 24 hours to recover the complete system. Although this sounds like a mistake made by a more junior DBA, it was actually because the enterprise lacked good checks and balances on the systems. There were development and production applications mixed together, with mislabeling and other problems besides. Given all that, a mistake was almost inevitable.

Here’s the scenario. Imagine you have three red buttons on your desk, one turns off your computer, one turns off all the power in New York City, and the third telephones the whitehouse telling them nukes are on the way. Now your boss comes in and says hit the red button, we need to restart your machine. So, happily you hit the first button you see. Bam, all the electricity in NYC goes out!

A funny analogy, but the point is if you have high profile systems, you have to take more precautions to protect them. That means more security, better naming, and configuration that discourages mistakes. Keep those red buttons out of reach. Label them clearly, and check and recheck, ten times if necessary, whenever you’re working on production systems. There should be high-level checks and balances put in place, and systems that reinforce correct usage and management, and discourage mistakes. These are not technical hurdles, they are hurdles that every part of a business must grapple with. In technology, unfortunately they are often lost in the shuffle of configuration & systems complexity.

2. Upcoming Speaking Engagements

November 11th – DBA Online, Edison NJ
Title: Otop – Diagnosing Bottlenecks In Your Database

December 14th – New York Oracle User Group, Manhattan
Title: Creating an Oracle Database – Manual Step-by-Step Guide

3. Published Articles

We’re proud to announce an article of ours was chosen as a feature on ChangeThis.com. Thanks to everyone who voted for us.

Ever wonder what those folks in IT actually do everyday? Most likely, they are busy defending your company against invaders, inefficiency and wasted resources. How? Here, Sean Hull, “self-appointed International Interpreter for Geek-to-Suit communications

Open Insights 24: Consulting Conflicts of Interest

OPEN INSIGHTS Newsletter
Issue 24 – Consulting Conflicts of Interest
October 1, 2006

by Sean Hull
<shull@iheavy.com>
Founder and Senior Consultant
Heavyweight Internet Group

Welcome to our free monthly newsletter, discussing news, developments and business best practices at the intersection of Oracle and Open Source software.


In This Issue:

1. Feature: Consulting Conflicts of Interest
2. Audio Interviews
3. Current Reading
4. Lightweight Humor
5. Past Issues
6. Technical Articles
7. About Heavyweight Internet Group


1. Feature: Consulting Conflicts of Interest

A. Where There’s Interest, There’s Conflict.

I enjoy putting together this newsletter every month. It’s a bit like blogging, except it goes out to my list of colleagues, friends, and company associates past and present.

I’ve touched upon rates, and hourly billing in the past, but not in the context of the ironic twist that is a conflict of interest. As we all know, a conflict of interest is whenever you are faced with two conflicting loyalties. If you were in charge of hiring at a company, and were also taking a fee from the recruiting company for sending business their way that would be one. If you are giving financial advice to consumers, and also getting paid a fee by the companies you favor, that would be another big one. There are conflicts of interest in business, legal matters, or whenever you loyalties are at odds.

In consulting though, I’m talking about a much subtler conflict. Whenever we discuss hourly rates, we are putting a dollar value on an hour of time, but we are sidestepping the whole discussion about length of time. How much time?

The whole nature of hourly billing presumes you don’t have a sense of the time a project will take, otherwise you could forecast, and offer a flat fee for the project. But where’s the conflict? The conflict is that your loyalty to the client, to minimize costs is at odds with your loyalty to your small business, to drive and increase profits. Yes it’s a dirty little secret, but there it is. Efficiency is a very malleable substance. Work in an office with lots of interruptions and people talking around you, or in a cafe for that matter, you might be less efficient. Or perhaps you haven’t gotten enough sleep, or are working overtime on one project before getting to another. Do I bill for a five minute call, or roll it into forty five minutes of work later in the day? Are you checking your email while working on client work, or checking the news between tasks? Are you answering calls from other clients, or making a quick five minute call to your dentist? Anyone who’s wanted to get out early on a Friday, or before a vacation knows how efficient one can be when they really want to be.

B. Striking A Balance

Ultimately there is an amount of good judgement involved. When a client begins working with a consultant, without a history, or a glowing recommendation from a very trusted colleague, there is an element of distrust. From past experience, and dealing with an unknown resource.

The truth is after all is said and done, the client will judge what you did overall, and what it cost them for you to bring those solutions to their business. They may find themselves micromanaging your time, but this just means they need further reassurances from you that you have things under control and the project won’t unravel with endless bills, and no results.

In truth working on a project is as much about making the client happy as anything. Get too caught up in the nitty gritty of security, tuning, and problem solving, and forget about the client’s specific concerns and you will surely not win the day.

In my experience negotiating rates up front has always been an exercise in assurances and reassurances that they are reasonable and will produce results. Again and again I find clients are overwhelmingly happy with my work after the fact, and found it cost them much less than they thought it would. In technology there is so much unknown, about what the problems are, and how to solve them, that it is difficult to discern which resources will solve those problems, and in a timely fashion.

C. Stay On Your Toes

Surely clients, you should stay on your toes, do your due diligence on backgrounds, and experience. But also pay attention to the intangibles, such as timeliness, honesty and character. In the end, the success of your project will rely on trust as much as anything. Beyond contracts, and promises the relationship is your strongest insurance.

2. Audio Interviews

Ingres Chief Technology Officer Dave Dargo joins us this month in another podcast interview. We talk with him about the open-source Ingres Database, and the economics driving open-source software today. Great insights, and plenty of food for thought, so have a listen.

Do you use Open-source technologies in your enterprise? Would you like to talk about your experiences, and business successes? We’d like to hear from you. Email me at shull@iheavy.com

3. Current Reading

The Little Money Book – David Boyle

Now here’s an interesting book that you won’t be able to put down. Detailing everything from our move away from the gold standard, and causes of the Great Depression to understanding currencies, the IMF, the World Bank, and much more. Did you know that in 1975 foreign exchange transactions amounted to 15 billion, and today amount to 2000 billion? Read on to learn more.

Inescapable Data: Harnessing the Power of Convergence – Stakutis & Webster

When terms like “knowledge is power” or “time is money” are bandied about we usually think of them as true, but mostly in the abstract. But arbitrage is a real-world example of how disparities in knowledge can be very important. These two authors discuss the convergence of “data-everywhere” devices, wireless networks, and the advanced software that we’re building to rewrite the book on almost everything from financial transactions, to product tracking, and consumer feedback. Prescient material.

Naked Conversations – How Blogs are Changing the Way Businesses Talk with Customers by Robert Scoble

Scoble’s blog gets 3.5 million readers per year. So clearly when he talks, people listen. He offers tips on how to blog, and has lots and lots of practical advice.

4. Lightweight Humor

Just found this amusing humor site blaugh.com which I think you’ll enjoy. In th is strip, the search engine optimization expert is pitted against the doom-and-gloom anti-Google pessimist!
http://blaugh.com/2006/09/14/which-one-are-you/


5. Past Issues

Issue 23: Devil In The Details
Issue 22: Beware of Software Fashion
Issue 21: Open Season, Open Sesame?
Issue 20: Better Web Better Business
Issue 19: Managing Fixed Fees
Issue 18: The Cost of Consulting
Issue 17: Secrets Of The Interview
Archive: Past Issues

6. Technical Articles

Oracle 10g RAC on a Laptop: click here
Oracle9i + RAC on Linux/Firewire: click here
Migrating MySQL to Oracle: click here
MySQL Disaster Recovery: click here

7. About Heavyweight Internet Group

In a nutshell, Oracle. Everything related to and surrounding the database technology we specialize in, but specifically setup, admin and tuning of Oracle technology. I have 10 years experience with Oracle, wrote a book on the technology, and write and lecture frequently. I’m founder and senior consultant of the company. In capacities where your company might hire Deloitte, AIG, or Oracle Consulting we can bring the same level of service and experience, at about half the price. Simple equation.

Visit us on the web at www.iheavy.com.