Part 6: RAC/Linux/Firewire – Cluster Manager Setup

Cluster Manager Setup


The cluster manager software is how the Oracle instances communicate their activities. Obviously this is an important piece to the puzzle as well. Here I review the configs, and then show how to get it up and running on each node. I *DID NOT* patch the cluster manager with the 9.2.0.2 db patch, but your mileage may vary.

Edit file $ORACLE_HOME/oracm/admin/cmcfg.ora

HeartBeat=10000

ClusterName=Oracle Cluster Manager, version 9i

PollInterval=300

PrivateNodeNames=zenith utopia

PublicNodeNames=zenith utopia

ServicePort=9998

HostName=zenith

#CmDiskFile=/ocfs/oradata/foo

MissCount=5

WatchdogSafetyMargin=3000

WatchdogTimerMargin=6000

Note, if you patch oracm to 9.2.0.2, remove the two Watchdog lines, and uncomment and use the CmDiskFile.

Edit file $ORACLE_HOME/oracm/admin/ocmargs.ora

watchdogd -d /dev/null -l 0

oracm /a:0

norestart 1800

Note, if you patch oracm to 9.2.0.2, comment out the watchdog line.

Now *AS ROOT* start up the cluster manager:

$ ./$ORACLE_HOME/oracm/bin/ocmstart.sh

You should see 8 processes with “ps -auxw | grep oracm”. Note that if you are running RH8, there’s a new ps which needs a special option “m” to notice threads. Apparently oracm is threaded (Thanks Wim). This had me pulling my hair out for weeks, and I’m bald! Anyway if that is the case, use “ps auxwm | grep oracm”. One more little recommendation. oracm is communicating via a port which you define. If you’re using iptables/ipchains, or some other firewall solution, I would recommend disabling it, at least temporarily, until you know you’ve configured everything right. Then reenable it, being sure you are good at configuring just the ports you need.

15. Perform step 14 on node 2.

Part 1 – Introduction

Part 2 – Basic Costs + Hardware Platform Outline

Part 3 – Software Requirements, Versions, etc

Part 4 – Initial Oracle Setup

Part 5 – Firewire + OCFS Setup

Part 6 – Cluster Manager Setup

Part 7 – Cluster Database Setup

Part 8 – Review of Clustered Features + Architecture

Part 9 – A quick 9iRAC example

Part 10 – Summary

Part 5: RAC/Linux/Firewire – Firewire + OCFS Setup

Firewire + OCFS Setup


In this installment, we’ll discuss how to get the Firewire drive shared between your two Linux boxes.

8. Test Firewire drive

At this point you can test the firewire drive if you like, with the standard Linux driver. You won’t be able to share the drive between the two nodes yet, however.

As root do the following:

$ modprobe ohci1394

$ modprobe ieee1394

$ modprobe sbp2

$ modprobe scsi_mod

Grab a copy of rescan-scsi-bus.sh from here and run it:
http://www.fifi.org/cgi-bin/man2html/usr/share/man/man8/rescan-scsi-bus.sh.8.gz

run rescan scsi bus

Now partition it with fdisk:

$ fdisk /dev/sda

Now try making an ext2 filesystem with mke2fs

$ mke2fs /dev/sda1

Now mount it

$ mount -t ext2 /dev/sda1 /mnt/test

Now unmount it

$ umount /mnt/test

9. Linux Kernel Setup w/Firewire patch

The Linux kernel is a complex beast, and compiling it can often be a challenge. Though I like rolling my own, I downloaded the patched firewire source distro off of OTN, and try as I might, I could not get those compiled kernels to work. If anyone *DOES* get it to work, please send me their “.config” from the kernel source directory. Also I’ve tried to encourage the Oracle/Linux Firewire team to build a patch-only distro which can be applied against a standard Linux source tree. No luck yet.

Assuming you’re not going to roll your own, just download linux-2.4.20rc2-orafw-up.tar.gz from here:

http://otn.oracle.com/tech/linux/open_source.html

Move to the “/” or root directory, and untar the file:

$ tar xvzf linux-2.4.20rc2-orafw-up.tar.gz

Edit your /etc/lilo.conf or /etc/grub.conf file to include the new kernel. Do *NOT* make it the default kernel, it may not boot.

Reboot. If you come up again, you’re in luck, the kernel works for your machine. Next you want to edit your /etc/modules.conf to include these lines:

# options for oracle firewire patched kernel

options sbp2 sbp2_exclusive_login=0

post-install sbp2 insmod sd_mod

post-remove sbp2 rmmod sd_mod

As root, load the modules like this:

$ modprobe ieee1394

$ modprobe ohci1394

$ modprobe ide-scsi

$ modprobe sbp2

$ modprobe scsi_mod

If you’re having trouble seeing the device, grab a copy of rescan-scsi-bus.sh from here:
http://www.fifi.org/cgi-bin/man2html/usr/share/man/man8/rescan-scsi-bus.sh.8.gz

If you want to partition, now is a good time. Use fdisk as root like this:

$ fdisk /dev/sda

If you have other SCSI devices, it may be /dev/sdb or dev/sdc and so on.

10. Go through steps 1-8 on node 2

11. Cluster Filesystem setup (OCFS)

If you wanna play around, use mke2fs on the one of the partitions you created with fdisk, and then mount the partiton on machine a. Then mount the partition again on machine b. Create a file on one of the two boxes. The other machine *WON’T* reflect it. This is equivalent to unplugging a disk which is mounted, such as a USB device, or some such. You can and probably *HAVE* corrupted the filesystem. That’s ok, because we don’t have anything important on the disk yet. Ok, unmount on both machines. If you have trouble, you may need to reboot.

Having gone through the above example, you know why OCFS is so important. Ok, now the fun part. Install OCFS. There are good docs to be found in the linux_ocfs.pdf file here:

http://download.oracle.com/otn/linux/code/ocfs/linux_ocfs.pdf

Without RedHat Advanced Server, the RPMs are *NOT* going to work. Just grab a copy of ocfs-1.0-up.o and put it in
/lib/modules/2.4.20-rc2-orafw/kernel/fs.

Use ocfstool to create the /etc/ocfs.conf file. The pdf doc listed above is pretty good at explaining this.

Load the ocfs kernel module with load_ocfs. If everything goes right it will tell you like this:

$ cd /lib/modules/2.4.20-rc2-orafw/kernel/fs

$ load_ocfs

/sbin/insmod ocfs node_name=zenith
ip_address=192.168.0.9 ip_port=7000 cs=1865 guid=72C2AF5CA29FA17CB9CB000AE6312F24

Using /lib/modules/2.4.20-rc2-orafw/kernel/fs/ocfs.o

Next make the filesystem. ocfstool can do this too.

$ mkfs.ocfs -F -b 128 -L /ocfs -m /ocfs -u 1001 -g 1001 -p 0775 /dev/sda1

And finally mount the filesystem!

$ mount -t ocfs /dev/sda1 /ocfs

$ df -k

Filesystem 1K-blocks Used Available Use% Mounted on

/dev/hda2 55439548 20835260 31788096 40% /

/dev/hda1 101089 18534 77336 20% /boot

none 112384 0 112384 0% /dev/shm

/dev/cdrom 122670 122670 0 100% /mnt/cdrom

/dev/sda1 60049024 30080 60018944 1% /ocfs

12. Perform step 10 on node 2.

13. Test ocfs

Here we quickly verify that a file created on one instance is viewable on another.

On node1 do:

$ cd /ocfs

$ touch mytestfile

On node2 do:

$ cd /ocfs

$ ls

mytestfile

$

You’ll see to your astonishment that the file is now visible on node 2!

Part 1 – Introduction

Part 2 – Basic Costs + Hardware Platform Outline

Part 3 – Software Requirements, Versions, etc

Part 4 – Initial Oracle Setup

Part 5 – Firewire + OCFS Setup

Part 6 – Cluster Manager Setup

Part 7 – Cluster Database Setup

Part 8 – Review of Clustered Features + Architecture

Part 9 – A quick 9iRAC example

Part 10 – Summary

Part 4: RAC/Linux/Firewire – Initial Oracle Setup

Initial Oracle Setup


Follow these instructions to get Oracle up and running on your new Linux box.

1. Setup the oracle account, and environment as follows.

Create an oracle user on your linux box:

$ adduser oracle

2. Login to the oracle account and edit .oraenv9i as follows (assumes bash):

# oracle environment variables

export ORACLE_BASE=”/home/oracle”

export ORACLE_HOME=”/home/oracle/product/9.2.0″

# EAST on machines #2

export ORACLE_SID=”WEST”

export PATH=$PATH:$ORACLE_HOME/bin

export LD_LIBRARY_PATH=”$ORACLE_HOME/lib”

export LD_ASSUME_KERNEL=2.2.5

# US7ASCII is the default, but WE8ISO8859P1 support more languages

export NLS_LANG=AMERICAN_AMERICA.WE8ISO8859P1

export ORA_NLS33=$ORACLE_HOME/ocommon/nls/admin/data

export ORACLE_TERM=xterm

export ORACLE_OWNER=oracle

export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib:/usr/local/lib

export TNS_ADMIN=$ORACLE_HOME/network/admin

# run the RH compatability stuff

. /usr/i386-glibc21-linux/bin/i386-glibc21-linux-env.sh

# setup Java

export JAVA_HOME=/usr/local/java

export CLASSPATH=$ORACLE_HOME/jdbc/lib/classes12.zip:$ORACLE_HOME/JRE:$ORACLE_HOM

E/jlib:$ORACLE_HOME/rdbms/jlib:$ORACLE_HOME/network/jlib:.

Add this line to your .bash_profile:

. /home/oracle/.oraenv9i

Next install the glibc backward compatability libs per

compat-

egcs-6.2-1.1.2.14.i386.rpm

compat-

glibc-6.2-2.1.3.2.i386.rpm

compat-libs-6.

2-3.i386.rpm

3. Next install JDK.

4. Edit the hosts file if necessary to include two machines on your local network

by name. I used “utopia” and “zenith”.

5. Next run the Oracle installer as follows (assuming you opened the packages in

/tmp). If you’re not on the X-window console, connect with SSH and you should ha

ve x tunneling by default, and can display remotely:

$ ./tmp/Disk1/install/linux/runInstaller

First install the cluster manager. You’ll be prompted for the local and remote h

ostname, a quorum disk, as well as some other things. Doesn’t matter what you en

ter right now, as we’re going to go back and edit those files and do things by ha

nd anyway.

6. Next go through the 9.2.0.1 software install. Here are some other notes with

various weblinks. Be sure to select “Enterprise Edition” and also “Software Only

Install”.

I got started with this Oracle 8 install doc which details which compatability li

braries you’ll need, how to setup the Oracle account, environment variables, and

Java.

link

I ran into some trouble with LD_ASSUME_KERNEL, and compat libs:

You’ll encounter problems with the context makefile, and get this error “Error in

invoking target install of makefile /opt/oracle/product/9.2.0/ctx/lib/ ins_ctx.m

k”. Sad but true, this is *NORMAL* behavior. I suspect making the installer run

flawlessly isn’t at the top of Oracle’s priority list.

link

I also encountered problems with Oracle Net Configuration Assistant “Can’t find $

ORACLE_HOME/jre/1.1.8/bin/../bin/i586/green_threads/jre” and I added this symlink

:

$ ln -s $ORACLE_HOME/jre/1.1.8/bin/i686
$ORACLE_HOME/jre/1.1.8/bin/i586

I got a simlar error for libjava.so:

$ ln -s $ORACLE_HOME/jre/1.1.8/lib/i686
$ORACLE_HOME/jre/1.1.8/lib/i586

Though you’ll probably want to create your own database from scratch later, it’s

sometimes instructive to let the Oracle Database Configuration Assistant create a

starter one for you, and look at what options they use. As with the rest, you’l

l run into an error. This time it is “ORA-27123 unable to attach to shared memor

y segment: Oracle Database Configuration Assistant”. Fix it by doing the followi

ng as root on your linux box:

$ cat /proc/sys/kernel/shmmax

33554432

$ echo `expr 1024 * 1024 * 1024` > /proc/sys/kernel/shmmax

$ cat /proc/sys/kernel/shmmax

1073741824

Obviously you’ll have to go through this process (living hell?) on both boxes you

‘ll be using in your cluster.

Next rerun the installer and specify the location of the itty bitty 235M patch, a

nd install that.

7. Enable RAC in Oracle9i

The Real Application Cluster feature is *NOT* enabled by default. Here’s how you

enable it:

As the oracle user:

$ cd $ORACLE_HOME/rdbms/lib

$ make -f ins_rdbms.mk rac_on

$ make -f ins_rdbms.mk ioracle

As root set permissions on rac_on

$ chown oracle /etc/rac_on

$ chgrp dba /etc/rac_on

Part 1 – Introduction

Part 2 – Basic Costs + Hardware Platform Outline

Part 3 – Software Requirements, Versions, etc

Part 4 – Initial Oracle Setup

Part 5 – Firewire + OCFS Setup

Part 6 – Cluster Manager Setup

Part 7 – Cluster Database Setup

Part 8 – Review of Clustered Features + Architecture

Part 9 – A quick 9iRAC example

Part 10 – Summary

Part 3: Rac/Linux/Firewire – Software Requirements, Versions, etc.

Software Requirements, versions, etc


Advance notice for beginners out there. I’ve installed Linux probably 20-30 times over the last 10 years, and it’s gotten a lot easier, however, you’ll need to have some decent skill at kernel installations. I’ll outline what I did that finally worked, but I’ll also say that I tried getting this setup to work on a couple of other boxes before I ended up using the emachines ones. I think the kernel which the Linux Firewire team has compiled and patched to their liking is pretty finicky, hardware-wise.

RedHat 8.0

I can’t go into the details of installing Linux in this short article, and there are plenty of resources on the web to help you. You’ll need to get that up and running first, with the 1394 card in the box.

Oracle 9.2.0.1

Oracle is a beast to install, and challenging for the beginner. Luckily there are a number of resources on the web.

patch 9.2.0.2

This patch is a *REQUIRED* patch. The guys on the Linux Firewire project assure me that the Oracle Cluster Manager has a lot of bugs and won’t work straight out of the box from the 9.2.0.1 release. Here’s a little twist. Although this is an *OPEN SOURCE* project, and the source is all free, and you can download Oracle and play with it to your heart’s content, to download a patch, you need access to Oracle’s Metalink, which requires you to have a software license. Not much I can say here folks. Use your imagination on getting ahold of this patch. Maybe if you place nice, someone on an email list will post it temporarily for you or something. It’s 235M. 🙂

Login to Metalink (some like to call it metastink)

Click on the “patches” link along the left.

Search for patch 2632931

Also, take a look at this note:

Doc ID: Note:217811.1

Subject: ALERT: 9.2.0.2 Patchset for Linux Does Not

Apply Correctly in the RAC Environment

Type: ALERT

Status: PUBLISHED

Content Type: TEXT/PLAIN

Creation Date: 07-NOV-2002

Last Revision Date: 08-JAN-2003

Part 1 – Introduction

Part 2 – Basic Costs + Hardware Platform Outline

Part 3 – Software Requirements, Versions, etc

Part 4 – Initial Oracle Setup

Part 5 – Firewire + OCFS Setup

Part 6 – Cluster Manager Setup

Part 7 – Cluster Database Setup

Part 8 – Review of Clustered Features + Architecture

Part 9 – A quick 9iRAC example

Part 10 – Summary

Asterisk Calling Card Applications

Asterisk is a powerful PBX solution, that we already know. But what else can it do. In this article we’ll explain how to setup Asterisk to handle Call Data Records (CDR data) in MySQL. Once you have that configured, there are a number of calling card applications which can be integrated with Asterisk to provide you with the makings of a serious calling gateway.


Setup Asterisk CDR with MySQL

By default Asterisk pumps all it’s call data information to text-based log files. That’s fine for normal use, but what if you want to put that data to use in a calling card application? First you have to get Asterisk to use a database. Luckily the support is already there, all you have to do is configure it.


Start by editing your cdr_manager.conf file as follows:


enabled = yes

Next edit your modules.conf file, and somewhere in the [modules] section, add:


load => cdr_addon_mysql.so

We’re going to compile this, don’t worry. Next edit your cdr_mysql.conf file in /etc/asterisk or create it if necessary:


[global]

hostname=localhost

dbname=asteriskcdrdb

user=astxuser

;user=

password=astxpass

;password=

port=3306

sock=/var/lib/mysql/mysql.sock

;sock=/tmp/mysql.sock

userfield=1

Next install MySQL. Luckily for all you lazy bums out there, this is the simplest of all. You’ll need to download three RPMs and install them. You’ll need the latest version of mysql-server, mysql-client and finally mysql-devel.


Next you’ll create a database called “asteriskcdrdb” with mysqladmin, create a table named “cdr” with the Asterisk provided script, and then set user grants.


Now it’s time to compile the asterisk-addons package. Be sure you have zlib-devel and mysql-devel packages installed on your system or you may get errors. Checkout the source from cvs. I got some strange errors which I had to track down on the email lists, and then edit the makefile as shown below:


CFLAGS+=-DMYSQL_LOGUNIQUEID

Now stop asterisk, and start it up again, and monitor the asterisk logfile for errors as follows:


tail -f /var/log/asterisk/messages

You can finally verify that you are dumping cdr information into mysql as follows:


$ mysql asteriskcdrdb

mysql> select uniqueid, src, calldate from cdr;



There should be one entry for every call. Make some calls to local

extensions and verify that records show up here. New cdr records

will still show up in the /var/log/asterisk/cdr-csv/Master.csv

file. Not sure if this can be disabled.


Calling Card Applications


ASTCC

Though the homepage is just a voip-info wiki page

and the download available through CVS, this calling card application was updated in late December 2004. This application seems to be the winner in terms of popularity on the voip-info wiki. It comes from Digium, it supports MySQL, and setup is pretty straightforward.

AreskiCC

With a strange name, it nevertheless seems a pretty complete system. Last updated end of December, 2004, it includes a web interface, though no support for MySQL. That’s fine, but my MySQL setup instructions will need to change slightly as you’ll need to configure Asterisk to dump CDR data into Postgres.


Asterisk Billing – Prepaid application

Last updated in July, I had trouble compiling this application. There is a basic sourceforge download page, but no real homepage. I’m guessing this one is still sort of in the development stages. Also, it doesn’t come with any sound files, so you’ll have to record your own, or *borrow* from some of these other applications.

Open Insights 26 – Logistical Fitness

OPEN INSIGHTS Newsletter
Issue 26 – Logistical Fitness
November 1, 2006

by Sean Hull

Founder and Senior Consultant
Heavyweight Internet Group

Happy Holidays to everyone and welcome to our free monthly newsletter, discussing news, developments and business best practices at the intersection of Oracle and Open Source software.


In This Issue:

1. Feature: Logistical Fitness
2. New Articles
3. Audio Interviews
4. Current Reading
5. Lightweight Humor
6. Past Issues
7. Technical Articles
8. About Heavyweight Internet Group


1. Feature: Logistical Fitness

A few months back, the Economist featured a special survey on “logistics” that emerging science of supply chain management. They talked about what they termed the “physical internet”, ie the delivery of products in the global marketplace.

More and more the winners in business are the ones who manage their supply chains best. Zara, an example in the fashion industry, is a company that has managed to deliver clothing inspired by the latest runway trends five weeks later to the checkout. This is absolutely astonishing, given that other companies take six to nine months to do the same. They manage this feat with a mix of close coordination with all of their stores, and a Dell computer-like supply chain coordinating material from Asia, and production at their base in North-west Spain.

Wal-mart has managed to grow into the giant they are today by similar management of it’s supply chain, avoiding too much stock, and attendant sales to trim down, and shortages on the other hand. The head office has practically instantaneous information about stocked items at their different stores, providing unprecedented logistical information.

These same efficiencies are what has driven Dell computer to the top, allowing it to build each computer to a customer’s own specification, and keeping little inventory in the process.

All these efficiencies drive down costs, and make these companies incredibly nimble, despite their size. And what makes all this supply chain magic work together? At the heart of all of these technologies are enterprise databases like Oracle, and suites of software like Oracle Applications, or SAP AG.

So the next time you order a package from Amazon.com, track it day-by-day or hour-by-hour on Fedex.com, and are only mildly impressed when it shows up from halfway around the world at your doorstep in only a couple of days, realize that is phenomenal logistics, and supply chain management at work that makes it all possible.

If you’d like to download the Economist section on Logistics from June 2006 go here.

Other articles can be found at Yossi Sheffi’s “Resilient Enterprise” website.

Also visit the Council of Supply Chain Management Professionals

2. New Articles

Oracle 10g RAC versus DataGuard for High Availability

3. Audio Interviews

This month we have the opportunity to talk with William Hurley aka Whurley, the Chairman of the Open Management Consortium.

In our interview, we discuss open-source, and it’s impact on commercial software and solutions, and wrestle head on with some of the concerns people have on both sides of the fence.

William Hurley is the CTO at Qlusters, where he launched the openQRM project. He has been awarded IBM’s Master Inventor title, multiple awards for innovation at Apple Computer. Prior to joining Qlusters he was CTO and founder at Symbiot. He holds 11 patents for research and development at IBM, Tivoli Systems, and Apple Computer. He was recently elected Chairman of the Open Management Consortium.

4. Current Reading

The Resilient Enterprise by Yossi Sheffi

Mr. Sheffi’s new book explains how supply chains are transforming companies, making them more competitive and nimble in the process. With plenty of success and failure stories, it is very readable material.

iWoz by Steve Wozniak and Gina Smith

The original innovator, and inventor of the personal computer, Steve Wozniak discusses the fascinating origins of Apple computer, and how it rose to be the incredible company it is today.

The New CIO Leader by Marianne Broadbent & Ellen Kitzis

This is really a how-to book, offering insights and advice to CIOs on how to transform their position and take on a more prominent influential role in the enterprise.

5. Lightweight Humor

Check out the funny fake walmart blog at Gaping Void.

6. Past Issues
Issue 25: Which Red Button
Issue 24: Consulting Conflicts of Interest
Issue 23: Devil In The Details
Issue 22: Beware of Software Fashion
Issue 21: Open Season, Open Sesame?
Issue 20: Better Web Better Business
Archive: Past Issues

7. Technical Articles

Oracle DBA Interview: click here
Tools for the Intrepid DBA: click here
Oracle9i + RAC on Linux/Firewire: click here
Migrating MySQL to Oracle: click here
MySQL Disaster Recovery: click here

8. About Heavyweight Internet Group

In a nutshell, Oracle. Everything related to and surrounding the database technology we specialize in, but specifically setup, admin and tuning of Oracle technology. I have 10 years experience with Oracle, wrote a book on the technology, and write and lecture frequently. I’m founder and senior consultant of the company. In capacities where your company might hire Deloitte, AIG, or Oracle Consulting we can bring the same level of service and experience, at about half the price. Simple equation.

Looking for top-flight a DBA? Visit us on the web at iheavy.com.

Open Insights Newsletter

Welcome to our free monthly newsletter, discussing news, developments and business best practices at the intersection of Oracle and Open Source software. Click the links along the right to view past newsletters.

Please forward to interested friends and colleagues. To subscribe to our email list, click here for the list homepage.

Older Issues

Issue 39 (December 2007) – Reputation Management

Issue 38 (November 2007) – Are You Fast Failing?

Issue 37 (October 2007) – A Real Open Book

Issue 36 (September 2007) – Rarity of Excellence

Issue 35 – not published

Issue 34 (August 2007) – Hindsight Is Always 20/20

Issue 33 (July 2007) – Market For Experts

Issue 32 (June 2007) – Different Heritages

Issue 31 (May 2007) – Auto or Traffic Engineer

Issue 30 (April 2007) – Crowdsourcing

Issue 29 (March 2007) – Mainroads or Sidestreets

Issue 28 (February 2007) – High Availability

Issue 27 (January 2007) – Fragile Foundations

Issue 26 (December 2006) – Logistical Fitness

Issue 25 (November 2006) – Which Red Button?

Issue 24 (October 2006) – Consulting Conflicts of Interest

Issue 23 (September 2006) – Devil In The Details

Issue 22 (August 2006) – Beware of Software Fashion

Issue 21 (July 2006) – Open Season, Open Sesame?

Issue 20 (June 2006) – Better Web, Better Business

Issue 19 (May 2006) – Avoiding a Fixed Fee Fix

Issue 18 (April 2006) – The Cost of Consulting

Issue 17 (March 2006) – Secrets of the Interview

Issue 16 (February 2006) – Success in Juggling

Issue 15 (January 2006) – Marketing About Technology

Issue 14 (December 2005) – The Tricky Database

Issue 13 (November 2005) – Oracle Heavy Lifting

Issue 12 (October 2005) – What the Geeks Mean

Issue 11 (September 2005) – Google Wave or Tsunami

Issue 10 (August 2005) – Do You Arbitrage?

Issue 09 (July 2005) – IT Certifications

Issue 08 (June 2005) – The Devil Is In The Details

Issue 07 (May 2005) – Open Source In The Enterprise

Issue 06 (April 2005) – Practice What You Preach

Issue 05 (March 2005) – Building on Success

Issue 04 (February 2005) – Managers: Databases 101

Issue 03 (January 2005) – The Business of Open Source

Issue 02 (December 2004) – Consulting Apples and Oranges

Issue 01 (November 2004) – Newsletter Introduction

Open Insights 25: Which Red Button


OPEN INSIGHTS Newsletter

Issue 25 – Which Red Button?

November 1, 2006
by Sean Hull
Founder and Senior Consultant
Heavyweight Internet Group


Welcome to our free monthly newsletter, discussing news, developments and business best practices at the intersection of Oracle and Open Source software.


In This Issue:

1. Feature: Which Red Button?
2. Upcoming Speaking Engagements
3. Published Articles
4. Audio Interviews
5. Current Reading
6. Lightweight Humor
7. Of Interest
8. Upcoming Events
9. Past Issues
10. Technical Articles
11. About Heavyweight Internet Group


1. Feature: Which Red Button?

Many past newsletters have been inspired by real experiences I’ve had at client sites, and this month’s story falls into that category.

A few weeks back I had the pleasure of bringing on a new hire at a client, and bringing them up to speed with the systems. After a couple of weeks of training, they began to accept heavier responsibilities. While I was at another site, they were asked to perform some maintenance on a production server. Through miscommunication, mislabeling, or other error that any of us could fall prey to, an entire production database was deleted.

Although backups were in order, it took more than 24 hours to recover the complete system. Although this sounds like a mistake made by a more junior DBA, it was actually because the enterprise lacked good checks and balances on the systems. There were development and production applications mixed together, with mislabeling and other problems besides. Given all that, a mistake was almost inevitable.

Here’s the scenario. Imagine you have three red buttons on your desk, one turns off your computer, one turns off all the power in New York City, and the third telephones the whitehouse telling them nukes are on the way. Now your boss comes in and says hit the red button, we need to restart your machine. So, happily you hit the first button you see. Bam, all the electricity in NYC goes out!

A funny analogy, but the point is if you have high profile systems, you have to take more precautions to protect them. That means more security, better naming, and configuration that discourages mistakes. Keep those red buttons out of reach. Label them clearly, and check and recheck, ten times if necessary, whenever you’re working on production systems. There should be high-level checks and balances put in place, and systems that reinforce correct usage and management, and discourage mistakes. These are not technical hurdles, they are hurdles that every part of a business must grapple with. In technology, unfortunately they are often lost in the shuffle of configuration & systems complexity.

2. Upcoming Speaking Engagements

November 11th – DBA Online, Edison NJ
Title: Otop – Diagnosing Bottlenecks In Your Database

December 14th – New York Oracle User Group, Manhattan
Title: Creating an Oracle Database – Manual Step-by-Step Guide

3. Published Articles

We’re proud to announce an article of ours was chosen as a feature on ChangeThis.com. Thanks to everyone who voted for us.

Ever wonder what those folks in IT actually do everyday? Most likely, they are busy defending your company against invaders, inefficiency and wasted resources. How? Here, Sean Hull, “self-appointed International Interpreter for Geek-to-Suit communications

Open Insights 24: Consulting Conflicts of Interest

OPEN INSIGHTS Newsletter
Issue 24 – Consulting Conflicts of Interest
October 1, 2006

by Sean Hull
<shull@iheavy.com>
Founder and Senior Consultant
Heavyweight Internet Group

Welcome to our free monthly newsletter, discussing news, developments and business best practices at the intersection of Oracle and Open Source software.


In This Issue:

1. Feature: Consulting Conflicts of Interest
2. Audio Interviews
3. Current Reading
4. Lightweight Humor
5. Past Issues
6. Technical Articles
7. About Heavyweight Internet Group


1. Feature: Consulting Conflicts of Interest

A. Where There’s Interest, There’s Conflict.

I enjoy putting together this newsletter every month. It’s a bit like blogging, except it goes out to my list of colleagues, friends, and company associates past and present.

I’ve touched upon rates, and hourly billing in the past, but not in the context of the ironic twist that is a conflict of interest. As we all know, a conflict of interest is whenever you are faced with two conflicting loyalties. If you were in charge of hiring at a company, and were also taking a fee from the recruiting company for sending business their way that would be one. If you are giving financial advice to consumers, and also getting paid a fee by the companies you favor, that would be another big one. There are conflicts of interest in business, legal matters, or whenever you loyalties are at odds.

In consulting though, I’m talking about a much subtler conflict. Whenever we discuss hourly rates, we are putting a dollar value on an hour of time, but we are sidestepping the whole discussion about length of time. How much time?

The whole nature of hourly billing presumes you don’t have a sense of the time a project will take, otherwise you could forecast, and offer a flat fee for the project. But where’s the conflict? The conflict is that your loyalty to the client, to minimize costs is at odds with your loyalty to your small business, to drive and increase profits. Yes it’s a dirty little secret, but there it is. Efficiency is a very malleable substance. Work in an office with lots of interruptions and people talking around you, or in a cafe for that matter, you might be less efficient. Or perhaps you haven’t gotten enough sleep, or are working overtime on one project before getting to another. Do I bill for a five minute call, or roll it into forty five minutes of work later in the day? Are you checking your email while working on client work, or checking the news between tasks? Are you answering calls from other clients, or making a quick five minute call to your dentist? Anyone who’s wanted to get out early on a Friday, or before a vacation knows how efficient one can be when they really want to be.

B. Striking A Balance

Ultimately there is an amount of good judgement involved. When a client begins working with a consultant, without a history, or a glowing recommendation from a very trusted colleague, there is an element of distrust. From past experience, and dealing with an unknown resource.

The truth is after all is said and done, the client will judge what you did overall, and what it cost them for you to bring those solutions to their business. They may find themselves micromanaging your time, but this just means they need further reassurances from you that you have things under control and the project won’t unravel with endless bills, and no results.

In truth working on a project is as much about making the client happy as anything. Get too caught up in the nitty gritty of security, tuning, and problem solving, and forget about the client’s specific concerns and you will surely not win the day.

In my experience negotiating rates up front has always been an exercise in assurances and reassurances that they are reasonable and will produce results. Again and again I find clients are overwhelmingly happy with my work after the fact, and found it cost them much less than they thought it would. In technology there is so much unknown, about what the problems are, and how to solve them, that it is difficult to discern which resources will solve those problems, and in a timely fashion.

C. Stay On Your Toes

Surely clients, you should stay on your toes, do your due diligence on backgrounds, and experience. But also pay attention to the intangibles, such as timeliness, honesty and character. In the end, the success of your project will rely on trust as much as anything. Beyond contracts, and promises the relationship is your strongest insurance.

2. Audio Interviews

Ingres Chief Technology Officer Dave Dargo joins us this month in another podcast interview. We talk with him about the open-source Ingres Database, and the economics driving open-source software today. Great insights, and plenty of food for thought, so have a listen.

Do you use Open-source technologies in your enterprise? Would you like to talk about your experiences, and business successes? We’d like to hear from you. Email me at shull@iheavy.com

3. Current Reading

The Little Money Book – David Boyle

Now here’s an interesting book that you won’t be able to put down. Detailing everything from our move away from the gold standard, and causes of the Great Depression to understanding currencies, the IMF, the World Bank, and much more. Did you know that in 1975 foreign exchange transactions amounted to 15 billion, and today amount to 2000 billion? Read on to learn more.

Inescapable Data: Harnessing the Power of Convergence – Stakutis & Webster

When terms like “knowledge is power” or “time is money” are bandied about we usually think of them as true, but mostly in the abstract. But arbitrage is a real-world example of how disparities in knowledge can be very important. These two authors discuss the convergence of “data-everywhere” devices, wireless networks, and the advanced software that we’re building to rewrite the book on almost everything from financial transactions, to product tracking, and consumer feedback. Prescient material.

Naked Conversations – How Blogs are Changing the Way Businesses Talk with Customers by Robert Scoble

Scoble’s blog gets 3.5 million readers per year. So clearly when he talks, people listen. He offers tips on how to blog, and has lots and lots of practical advice.

4. Lightweight Humor

Just found this amusing humor site blaugh.com which I think you’ll enjoy. In th is strip, the search engine optimization expert is pitted against the doom-and-gloom anti-Google pessimist!
http://blaugh.com/2006/09/14/which-one-are-you/


5. Past Issues

Issue 23: Devil In The Details
Issue 22: Beware of Software Fashion
Issue 21: Open Season, Open Sesame?
Issue 20: Better Web Better Business
Issue 19: Managing Fixed Fees
Issue 18: The Cost of Consulting
Issue 17: Secrets Of The Interview
Archive: Past Issues

6. Technical Articles

Oracle 10g RAC on a Laptop: click here
Oracle9i + RAC on Linux/Firewire: click here
Migrating MySQL to Oracle: click here
MySQL Disaster Recovery: click here

7. About Heavyweight Internet Group

In a nutshell, Oracle. Everything related to and surrounding the database technology we specialize in, but specifically setup, admin and tuning of Oracle technology. I have 10 years experience with Oracle, wrote a book on the technology, and write and lecture frequently. I’m founder and senior consultant of the company. In capacities where your company might hire Deloitte, AIG, or Oracle Consulting we can bring the same level of service and experience, at about half the price. Simple equation.

Visit us on the web at www.iheavy.com.

Open Insights 23 – Devil In The Details

OPEN INSIGHTS Newsletter
Issue 23 – Devil In The Details
September 1, 2006

by Sean Hull
<shull@iheavy.com>
Founder and Senior Consultant
Heavyweight Internet Group

Welcome to our free monthly newsletter, discussing news, developments and business best practices at the intersection of Oracle and Open Source software.


In This Issue:

1. Feature: Devil In The Details
2. Audio Interviews
3. Current Reading
4. Lightweight Humor
5. Miscellaneous
6. Past Issues
7. Technical Articles
8. About Heavyweight Internet Group


1. Feature: Devil In The Details

We all have heard the saying, “the devil is in the details” but I wonder how often we remind ourselves of that when we need to. I have seen this situation over and over in ten years of consulting, and it is one that continues to challenge with new projects.

In science we call this phenomenon emergent complexity. Imagine you’ve some wires, or strings or ropes. Put them all in a bag to store away, and when you try to take them out again, somehow you always have knots and tangles! We have all experienced this problem with the wires behind our TV/Stereo systems, I’m sure!

Ever seen that commercial on TV for the PaintCrew sprayer? First they show someone struggling to paint their house by hand, and then they switch you to the happy person using the magic spraying machine. How easy my life will be if I just get one of those. Unfortunately they don’t have a shot of someone trying to clean that machine. I’m sure cleaning brushes would be easier. What’s more 30% or more of the house you have to use brushes anyway, so you’ll still have to clean them. And besides all that, the real work of painting a house, inside or outside, is taping it all up. So much for technology saving the day!

I hope by now everyone can see the parallels in consulting. When specing projects up front, there is often an incredible pressure to include the kitchen sink to “get it right” the first time. However in my experience, building small, starting modest at the outset, and then growing and building off of that is a much safer way to stay within bounds, on-time and under budget.

Also, please bear all these factors in mind when putting together a contract. Clearly outline what items will be completed, and what is in and out of scope. And converse back and forth between client and consultant, verbally reiterating what will be done and how. That’s because despite all efforts to outline perfectly on paper, there are always details that the client and the contractor envision differently. The more conversations you have going over those details, the more likely you’ll be on the same page as far as the spirit of the contract, even if the letter of the contract misses some minutiae.

Lastly a word of warning. This goes equally to consultants, as it does to clients to try to have a sense of perspective. Beware conversations saying “that should be easy”, and things of that nature. You will inevitably want to see it that way, before a contract is signed, as you want to get the contract in the first place. But as we all know, once you start digging there is always more complexity hidden away. In my experience statements like these also throw up a red flag because it is a coded way of saying we think this part won’t cost that much.

Try to emphasize that you’re available, convenient, and an outside resource, and so on. Emphasize the VALUE of having you solve the problem, and complete the project as a whole, and how your timeliness, and delegation of the project is a win for them. I always emphasize that we’re not the cheapest solution in town, but we have a very good track record of delivering what we promise, when we promise it. And that is worth a heck of a lot.

2. Audio Interviews

Ingres Chief Technology Officer Dave Dargo joins us this month in another podcast interview. We talk with him about the open-source Ingres Database, and the economics driving open-source software today. Great insights, and plenty of food for thought, so have a listen.

Do you use Open-source technologies in your enterprise? Would you like to talk about your experiences, and business successes? We’d like to hear from you. Email me at shull@iheavy.com

3. Current Reading

Andy Wibbels – Blogwild!
If you’re interested in blogging as a way to build or grow your business, this book is one you’ll want to check out. It is chock full of practical advice on how to use tagging, and RSS, and other technologies to build word-of-mouth attraction to your business. Of course, he has a blog.

Jim Collins – Built To Last
Jim Collins is the author of “Good to Great”, that essential reading for understanding the anatomy of successful businesses. In Built To Last, he takes a look under the covers at 18 companies that have been around for at least fifty hears, and applies that same ruthless logic, and research to find out what really makes them tick, beyond the media glamour, and star CEOs.

David Allen – Getting Things Done
David Allen is one of the fifty people to know according to business 2.0. If you’re looking for a very good book on organizing your time, and making creative use of every last bit of it, this is a great place to start. David also has a blog.

4. Lightweight Humor

If you’ve never visited Overheard in New York, don’t waste anymore time. It is a funny side, full of little anecdotes, and quotes from funny new york situations, on trains and subways.

This one is particularly funny: overheard in NY tourist!

5. Miscellaneous

Next time you’re mulling over consulting fees, and considering the fulltime overhead of benefits, vacation time, 401k, and training, consider this article on office waste.

6. Past Issues
Issue 22: Beware of Software Fashion
Issue 21: Open Season, Open Sesame?
Issue 20: Better Web Better Business
Issue 19: Managing Fixed Fees
Issue 18: The Cost of Consulting
Issue 17: Secrets Of The Interview
Archive: Past Issues

7. Technical Articles

Oracle9i + RAC on Linux/Firewire: click here
Migrating MySQL to Oracle: click here
MySQL Disaster Recovery: click here

8. About Heavyweight Internet Group

In a nutshell, Oracle. Everything related to and surrounding the database technology we specialize in, but specifically setup, admin and tuning of Oracle technology. I have 10 years experience with Oracle, wrote a book on the technology, and write and lecture frequently. I’m founder and senior consultant of the company. In capacities where your company might hire Deloitte, AIG, or Oracle Consulting we can bring the same level of service and experience, at about half the price. Simple equation.

Visit us on the web at www.iheavy.com