Open Insights 13 – Oracle Heavy Lifting

Heavyweight Internet Group Newsletter

Issue 13 – Oracle Heavy Lifting

November 1, 2005

by Sean Hull

Founder and Senior Consultant

Heavyweight Internet Group

Welcome to our free monthly newsletter, discussing news, developments and business best practices at the intersection of Oracle and Open Source software.

Please forward to interested friends and colleagues. Subscription information can be found at the end.

We are now writing material regularly for our sister site Oracle + Open Source so check there for more frequent updates.

Oracle Heavy Lifting

In consulting, I have the unique opportunity to both think deeply about technology, how it works, and what it can do, and also about business questions, what does a business need to do, and how will technology help it achieve greater returns. I just finished pouring over Tom Kyte’s new Expert Oracle title. It is an excellent book, full of plenty of deep insights about Oracle technology. As I was reading the chapters, I noted in particular which features required enterprise edition, and which are included in standard edition. You can find my chapter by chapter review on Amazon.

As I’m engaged with many different types of clients and businesses, I have the opportunity to see Oracle in various real-world situations, solving real business problems. I came to a profound realization that most enterprises decide how much to spend on Oracle in proportion to (a) the business problem & use they are putting it to and (b) how expensive the hardware and consulting services are. In other words the expense is relative to the budget constraints, and in proportion to the problem it is solving. However, with my technology hat on I realized that though it might seem like a rational decision from the business side, you may not be doing all the heavy lifting you think you’re doing. Put another way Enterprise Edition may not be what you need, or at least more than you’re using.

Without going into a tremendous amount of technical detail, I just wanted to outline the situations where Enterprise Edition would likely be worth the money, and when it may be overkill. When Standard Edition sits at roughly 10-15% of Enterprise Edition, that’s certainly something a business should consider.

A traditional way to separate database types is into two categories, Data Warehouse and OLTP. A Data Warehouse is characterized by very few users, doing large long running queries. OLTP or OnLine Transaction Processing is characterized by hundreds of concurrent users, such as a database which serves as a backend datastore to a website. An Oracle database can be tuned to prioritize and use resources most efficiently for each type of database.

So what does this have to do with Enterprise or Standard Edition. Well as it turns out Enterprise Edition proves a very attractive option for Data Warehouses. There are features such as streams, transportable tablespaces, materialized views, advanced analytical functions, and compression to name a few. None of these features are included in Standard Edition. Additionally there are other EE options which are attractive for Data Warehouses including Partitioning, and various ways to parallelize operations to help a single session consume the resources of the entire machine. These parallel operations can speed up dataloads, rebuilds, batch jobs, and large queries against even terabytes of data.

But what about Transactional environments (OLTP)? Many businesses use Oracle as a backend datastore for their online presence, such as banks, bookstores, airlines, and just about any other dynamic website. Well there are some features such as Fine Grained Access, Virtual Private Database, and Secure Roles which aren’t available in Standard Edition. There are also the Tuning, Change Management, and Diagnostics packs that are add-on options only available with Enterprise Edition. But many folks are just looking for speed and high availability features. Will Enterprise Edition help in this area? Well you only have Data Guard in Enterprise Edition, but databases can be cloned, and that process can be scripted, and although not seemless, and an integrated feature as Data Guard is, something similar can be done in Standard Edition. So how about the Parallel Features, and the Partitioning option. Will those help with Transactional databases? Let’s start with partitioning, it is a feature which may or may not be heavily used in your environment, but if your application is OLTP, chances are you’re not getting dramatic query improvements, because your queries are characterized by small frequent accesses by lots of concurrent users. Now partitioning will help you with maintenance and availability, but in that sense it serves more as a DBA tool, than a performance boost overall. What about the parallel operations. Again mostly for DBA operations, one-off rebuilds and dataloads can be made to run quite a lot faster but the day to day operation of your database may not be dramatically impacted by this feature.

With Oracle’s recent announcement of it’s Express Edition of 10g as well as it’s purchase of Innobase a commercial component of MySQL providing solid transactional support, your options are wider and more complex than ever before. Your best bet navigating this complex landscape is to get an accessment of your current infrastructure, or do sufficient research before investing in that new project. Like buying a car, ask lots of questions, don’t believe everything you hear, and kick the tires before you buy. Both Standard and Enterprise Edition can be downloaded for single-instance development use & testing off of Oracle Technology Network.

Technical Articles at IHEAVY.COM

Coming soon… A Hitchhiker’s Guide to Oracle 10g RAC

Note: The following articles will be returning to soon. We’ve done some redesign and rebuilding, and have to add these again…

Tracking the Wily Proxy Hackers

Asterisk Calling Card Applications

MySQL Disaster Recovery

Dummy’s Guide to Linux firewalls

Wireless Truth or Dare

Mirroring for the Impatient

Open Insights 12 – What the Geeks Mean

Heavyweight Internet Group Newsletter
Issue 12 – What the Geeks Mean
October 4, 2005

by Sean Hull
Founder and Senior Consultant
Heavyweight Internet Group

Welcome to our free monthly newsletter, discussing news, developments and business best practices at the intersection of Oracle and Open Source software. 

Please forward to interested friends and colleagues.  Subscription information can be found at the end.

We’d also like you to take a look at our sister site where you’ll find other non-technical and related news and content.

What the Geeks Mean

I’m blessed for having gotten started with computers when I was very young.  Well that’s either blessed or cursed depending on how you look at it.  At any rate, having worked with computers for about 25 years, I know well what the geeks are talking about.  And having spent the last 10 years doing a lot of writing, and consulting for diverse businesses, I’ve learned to understand what the business folks are talking about.  I’ve learned business thinking, and learned about the concerns of business owners.  So this naturally leaves me in a comfortable position as International Interpreter for Geek to Suit communication. 

Well I’m just kidding of course, but seriously there is often a wide gap in understanding and I make it a point of constantly trying to bridge that gap.  So what I’m going to discuss this month are some of the big issues that come up on the technology end, and try to help you understand what your technology team is really talking about.  This will help you make better purchasing and budget allocation decisions, as some problems impact the business more seriously, or more immediately than others.  And don’t worry, we’re going to use a broad brush here, and not get too technical.  Those technical folks in the audience, be patient where analogies don’t precisely match the underlying issues.

The six sections below basically amount to areas where your systems may be encountering problems.  For each section I make a hopefully memorable and clear analogy, and then in parentheses I write the technical description of that problem that you may hear.

A. Our Disks Form a Single Lane Freeway
(Disk I/O Subsystem problems)

Disks, Disk Subsystems, or I/O Subsytems are basically the same thing.  On a PC that sits on your desk, or at home there is a normally a single harddrive which stores things while the computer is turned off.  When the system is booted, or a program is started, programs and data are loaded off that storage media into memory.  In server computing environments much the same thing happens, albeit there are many more parts.  It is often referred to as a storage subsystem because it may be composed of a stack of harddrives all managed together in one cabinet, and interfaced together in what is called a RAID.  Why should I be concerned as a manager you ask?  Well how these are configured (ie what RAID level) can dramatically impact the performance, and reliability of your SAN (Storage Area Network), effectively giving you a one lane highway in one direction (while writing to disk) while giving you a 5 lane highway in the other direction.  What’s more some configurations are more prone to crashing, and loss of a harddrive (crashing, and loss of a lane in our car analogy) than others.  

If your developers and system administrators are saying what they have is incorrect this may be because the particular server in question has a controller card which does not support the proper RAID type.  In that case new hardware would solve your problem.  External SAN/NAS solutions such as EMC or NetApp normally support all the various RAID types in whatever configuration and combinations you want.

Keep in mind that you want to keep the traffic flowing in both directions of your highway, and you want to keep the car crashes down.  Of course also want to get the most performance especially if you invested a lot on that expensive and fast car, Oracle.  Above all avoid the single lane freeway and its attendant accidents.

B. Our Network Is a Party Line
(Network + Database Security)

Some of us may recall back in the old days when you might have a party line running into your home for telephone calls.  When you picked up the telephone you might hear your next door neighbor on the telephone.  You would have to politely (or not so politely) ask them when they would be finished so you could make your call.  Beyond the inconvenience, the privacy concerns are obvious. 

In the technology world, much more of our communications are like a party line, or a postcard that you send through the mail, then you might imagine.  But how much?  And when and where should I be most concerned?

If you have a dedicated networking team, who manage the firewall, and keep patches up to date, you still may have to be concerned about wireless access to your network.  Believe it or not even encrypted WiFi connections can be breached within 10 to 15 minutes with the right tools.  You have no WiFi you say?  You only use a VPN to allow folks on the road to connect?  Well there can be vulnerabilities in all of that software, ones that can allow a hacker to breach the network, and steal valuable data. 

But even if you have all that covered, what about your Oracle database itself, the pot of gold hidden inside your network, and closely guarded.  Is it?  How safe is it from insiders?  Perhaps someone in the office wants to know the salaries of key people, or perhaps read and email or document about a potential merger or aquisition that may impact them.  There are many reasons you may want part or all of your database to be protected, even from certain internal folks. 

The best way to find out is to audit all of this.  Find out if the db has strong passwords, if data sent over the network is encrypted, and what if anything would or could be compromised, and how.

C. Trouble at the Interchange
(Middle-tier Problems)

Internet websites such as Amazon, Ebay, or your favorite bank as well as intranet applications (web based apps running in-house only) all use a three tier architecture today.  In simple terms the browser (Firefox, Internet Explorer, or Safari) running on your desktop or laptop is the client.  The middle tier is the webserver and/or application server such as Apache, JBoss, Websphere, and so on.  The backend piece is your database itself.  If your systems are sluggish it could very well point to problems in the middle tier or at the interchange.  If cars aren’t getting off at the right exit, you have problems.

These servers can often be replicated quite easily, and in fact that is what many companies like your favorite online bank or bookstore do.  They also add lots of memory, and fast harddrives for reading the data and returning it back to you.  All of these middle tier servers send their database requests to the same database on the backend. 

If you’re just starting to provision new servers for a project, and haven’t decided on platform (Windows, Linux, Solaris etc), memory, cpu and so on, be sure to run performance tests first.  If you’re choosing between Windows and Linux you’ll want to consider your IT expertise, but also seriously consider the performance and stability you can expect on each platform.

D. Alphabet Soup Tuning
(SQL Query Optimization)

Your developer, database administrator, or consultant just came up to you and told you that you are experiencing serious problems because of bad SQL.  What to do?  Sounds like something the doctor prescribed, and maybe you’re feeling like you’ll need to call one by the end of it.

Here’s some help.  It is difficult to find a page in a book without an index to what you’re searching for.  It would be difficult to find an address on Fifth Avenue if you don’t know what street it is near.  In fact, if they weren’t in order (which they are not in the database without sorting) you have to go through every address from Houston Street to 59th Street.  That’s going to take you a long time.  What’s more if someone comes later looking for the same or similar addresses you’ll have a hard time writing them down because you passed so many on your way up Fifth Avenue.  This is exactly what happens in a database when an index is missing.  Or when a developer asks the database for all the addresses in New York City, when they only wanted the first one, or one with the last name Smith. 

Essentially your Queries (SQL) are the key to these types of messes, and the key to cleaning them up as well.  Since your database is executing hundreds of requests from various middle-tier servers, you want it to return quickly, and efficiently only exactly what is necessary, and cache it for related queries in the future. 

This is very often a problem we identify in tuning applications, and database problems.  It is also one that often doesn’t show up in development, when you only have a couple of streets worth of addresses instead of the whole island of Manhattan!

E. We’ve Got A Leaning Tower of Pisa
(Architectural Problems)

When you’ve built a complex software system, sometimes problems and issues aren’t related to small fixes, those potholes in the road.  Sometimes issues are serious structural or architectural problems, which could not have been anticipated when the system was built.  Such a leaning tower can threaten to collapse, if you continue to patch, and provide short term fixes. 

What to do?  A system-wide assessment might be a good place to start.  This should definitely be by someone outside the organization who can provide you with feedback without threatening his or her own position.  Such an assessment can be like getting a second opinion from another doctor.  It can also identify serious bottlenecks possibly caused by platform decisions which though structural, can be remedied easier than rebuilding the tower.  It may be that such an assessment points to the need for a complete rewrite of the application, possibly porting it to another platform which is more scalable.  All of these conclusions though perhaps difficult to hear, are what you want to know as they are important for strategic decisions going forward.  You may hold back a launch, or upgrade to a new version until you’ve fixed the fundamental flaws.

Malcolm Gladwell, in his book “The Tipping Point” used this aphorism:  “People don’t change when you tell them there is a better option.  They change when they conclude that they have no other option.”  So make your own conclusions, but make them informed conclusions.

F. Have We Done A Firedrill?
Disaster Recovery, Backups etc

Disaster recovery has been on everyone’s mind since the September 11th tragedy, and even more so following hurricanes Katrina and Rita.  If your organization hasn’t run a firedrill, how can you be confident that you’re in good shape?  The answer is you can’t.

A firedrill much like the ones we all remember from elementary school, take an organization through EVERY STEP to recover the database, middle-tiers, and any other relevant systems from backups onto freshly setup servers.  If you don’t have enough beefy systems in-house to spare for such a firedrill, there are hosting services which can lease you usage of such servers on a short-term basis. 

In the process of this firedrill, you will learn some very important lessons.  First, you’ll learn if  you have all the pieces in place, or if you have to patch some things from the production systems to get it right.  That tells you if your backup is complete.  Second you will learn how long the process takes, and in running through it you document the process so you guarentee it’ll be faster in a real emergency.  Third and finally you will gain peace of mind because you can identify and fix the gaps before the impossible happens.

Oracle Open World – comments

We attended Oracle Open World this year, and have to say that it was one of the biggest in many years.  With all of Oracle’s recent acquisitions, many new vendors were there, and of course the media was there in attendance as well.

At our sister site Oracle and Open Source in a piece titled Restoring an Open World we discussed some of the announcements, and emphasis this year, on Open Source and Open Standards.  

Open Insights 11 – Google Wave or Tsunami

Heavyweight Internet Group Newsletter
Issue 11 – Google Wave or Tsunami?
September 1, 2005

by Sean Hull
Founder and Senior Consultant
Heavyweight Internet Group

Welcome to our free monthly newsletter, discussing news, developments and business best practices at the intersection of Oracle and Open Source software. 

Please forward to interested friends and colleagues.  Subscription information can be found at the end.

We are now writing material regularly for our sister site so check there for more frequent updates.

Google Wave or Tsunami?

With the recent announcement  of Google’s desktop messaging and VOIP client, the industry is full of analysis about how it will impact the playing field.  Some argue that it’s foray into VOIP will bring that much more attention to the amazing and destabalizing technology.  Some are also arguing interestingly that the new client could open up the instant messaging generally, which would be a big win for consumers.

Google’s new client Google Talk  is based on an Open Source application called Jabber  which is like a multi-lingual IM client, able to talk to AOL, ICQ, MSN and Yahoo, assuming you have  an account on all of those.  That they’ve embraced an Open platform, is further evidence of Google’s general trajectory of opening up areas of computing formerly held in rigid grip by commercial vendors, and proprietary protocols.

The discussion of Google Talk has also interestingly rekindled the rumors over a possible Google or Web OS.  Jason Kottke discusses this in depth as does Silicon Beat.  These discussions are no mere Slashdot scuttlebut, but ruminations by industry figures who likely have their finger on the Google pulse.

What would such an OS look like is anyone’s guess.  But if Tom Friedman has anything to say about it, the forces making the world a flatter playing field for business include work flow software, open protocols and Open Source software in general.   A flat playing field, like the internet, routes around monopolies, and finds ways to do business and computing cheaper, more efficiently and better.  

Open Source News

A comparison of TCO between Windows and Linux:

A Discussion of Open Source on Windows:

CTO of Athena Healthcare discusses open source integration

Success Story of Cendant Travel saves close to $100 million

A changing software landscape

Heavyweight Internet Group – Technical Articles

Tracking the Wily Proxy Hackers

Asterisk Calling Card Applications

MySQL Disaster Recovery

Dummy’s Guide to Linux Firewalls

Wireless Truth or Dare

Mirroring for the Impatient

Migrating a MySQL Database to Oracle 

Heavyweight Internet Group does Oracle integration with Open Source technologies, and has ten years experience in this space.  For more information visit our website at or call us toll free at 866 268-9448.


Open Insights 10 – Do You Arbitrage?

Heavyweight Internet Group Newsletter
Issue 10 – Do You Arbitrage?
August 1, 2005

by Sean Hull

Founder and Senior Consultant

Welcome to our free monthly newsletter, discussing news, developments and bus

iness best practices at the intersection of Oracle and Open Source software.

Subscriptions to this newsletter has been growing, we’re happy

to say, and we hope you all continue to forward on the email to anyone else you

think might benefit.  Subscription information can be found at the bottom.

We are now writing material regularly for our sister site so check there for more frequent updates.

Do You Arbitrage?

While traveling in Europe on business for the last couple of

weeks, I’ve been devouring Tom Friedman’s book The Lexus and the Olive Tree.  It’s subtitled “Understanding Globalization” but could have

just as well been “keeping up with the rush of technological change”.  Wha

tever your opinions on globalization & outsourcing, these forces of change,

along with disruptive technologies like the Internet and VOIP are flattening the

world and making it smaller and faster.  This both helps busines

s in providing a more efficient field on which to play, but also makes it more c

ompetitive as well.

In particular he discussed Arbitrage, that branch of finance about

profiting from disparities of information about markets.  Having

information or knowledge that some group of people or businesses don’t have isn

‘t enough, finding a way to put it to use helping people, that’s the tough part.

As technological change accelerates this becomes more and mo

re relevant in IT.  For example, companies are taking advantage of new tech

nologies such as Voice over IP to completely sidestep the traditional telephony

providers and saving a killing in the process.  Om Malik, the aut

hor of Broadbandits, discusses the rise of VOIP in this seminal blog entry

The Voice Over IP Insurrection.

Or take the example of Cendant Travel who has managed to save a bundle on an upgrage of their Orbitz and airline travel system by choosing not to go with a mainframe solution, but rather a 144 server Linux solution.
Of course these are two success stories, and just as with financial arbitrage, in

formation arbitrage can be risky.  Finding other businesses that can portray the trials and tribulations,  along with plenty of research, and testing

are crucial to avoiding a big waste of money.

Good consulting is all about this type of information arbitrage.  Knowledge of one area of IT is not enough.  You need experience with Operating Systems from Windows to Linux, MacOS to HP-UX.  You need a touch

of networking knowledge, and a good head about security.  And you need to have a sense of what troubles end users really struggle with.  On the other

hand you also need to juggle different disciplines, which gives you a very broa

d view of the business.  One day you are thinking about a proposal, and fra

ming and pricing that fits the clients budget.  The next day you are thinki

ng about technical problems and how to solve them within that structure you’ve p

ut together.  You’re thinking about promises, and timelines but also releva

nce and context.  There is always a bigger problem to  solve, always a

complex way to look at a problem.  The trick is often distilling a problem

down to it’s essence and focusing on that to bring out the solution as efficien

tly as possible.  And the truth is sometimes you make guesses.  Based

on all your years of experience, and diversity of encounters with problems at clients in various industries, you use your gut feeling to eliminate and focus on

the  relevant.  Of course there is plenty of science, and investigation, but the  efficiency is often a function of such intangibles.

Good consulting though is also about the story telling.  It is the non-technical side, conveying, distilling, and making analogies.  Information such as this helps business management make the right decisions based on budget, short and long-term expectations, and customer needs.

Consider all of these factors the next time you weigh and delibe

rate over outside resources for a project.  Although fixed fee projects sho

uld be fairly easy  to compare assuming you’ve spec’d out the bounds well,

hourly billing can be very misleading, a comparison of apples and oranges.

Also consider a wider net of  experience than just the particular need of

the moment.  Security vulnerabilities, or efficiencies of other components

of your infrastructure may be discovered, getting more for your money.

Technical Articles

Tracking the Wily Proxy Hackers

Asterisk Calling Card


MySQL Disaster Recovery

Dummy’s Guide to Linux Firewalls

Wireless Truth or Dare
Mirroring for the Impatient

Migrating a MySQL Database to Oracle

Open Insights 09 – IT Certifications

Heavyweight Internet Group Newsletter

Issue 09 – What About IT Certification?
July 2, 2005

by Sean Hull
Founder and Senior Consultant
Heavyweight Internet Group

Welcome to our free monthly newsletter, discussing news, developments and business best practices at the intersection of Oracle and Open Source software.

We are now writing material regularly for our sister site so check there for more frequent updates.

What About Certification?

From time to time the debate comes up again about certifications  in IT.  Are they important?  What do they indicate?  Should they be a deal breaker?  Which ones are relevant and which ones are less meaningful.  Depending on who you ask, you’ll get wildly  different answers.

Take for example a recent discussion over at Computerworld where Editor Don Tennant says he’s “Certifiably Concerned

Basically he covers a study in which it was found that folks WITH certifications were receiving smaller raises.  He found this very counterintuitive and managers may also.  But I’ll tell you with some confidence that most of your tech folks won’t be surprised.

There are a couple of different reasons for this.  The first one  sort of cuts to the heart of IT.  It is very much a moving target, that is to say that the set of skills, were they defined as a bunch of commands that work a certain way, are constantly in flux and changing.  Multiple choice tests especially focus on this type of wrote memorization of commands, not on problem  solving skills.  Due to the nature of technology being in constant flux, most good engineers don’t know all the commands.  It’s true ask one of your engineers the syntax for say doing an XOR with UNION in SQL.  The concepts are truely what is important, and that cannot be thoroughly tested in a certification exam.  It is more the job of a proper University education to teach that type of theory, and instill the problem solving skills to find the syntax and details when needed.

But we’re really hitting on a second point here, and that is that a Bachelor of Science degree from a good University is really the best certification you can get, besides real-world on-the-job  experience of course.  I’ve found in my experience that the ones most insistent on seeking out certifications are those who don’t  have Computer Science degrees in the first place but rather technical school training, and are looking to beef up the resume.  Barring perhaps the Cisco Certified Engineer which I understand is quite grueling with real-world problem solving with a real network, and real messes to cleanup, most certifications seem to be pretty  much a waste of time.

So where did this certification mania originate from then?  Well Law has the Bar Exam, Medical Doctors must go through residency, and get licensed to practice medicine, so why not apply the same rigors to the IT industry, albeit in more simplified form to provide the same sort of measures of skill and aptitude?  Good question, but as it turns out the IT industry is still too  disjointed, with no central authority governing it to provide  that type of certifications.  The best you have for such measures right now is the Bachelor of Science degree, hopefully from a  University which emphasizes engineering and has a good  reputation to go along with it.

And now you’re wondering, without certifications how do we find  good people?  Well I hope for one thing you take away from this that a certification might be misleading you in the first place on aptitude.  But really it means you have to do the legwork of finding the right people who have (a) some synergy with your company  (b) a degree from a good University (c) plenty of real-world  experience.  Run them by your best people to get their gut feel on the person, ask some tough questions, or how they might solve some problem in the enterprise. 

This type of selection process should go equally for full-timers as it does for the consultants you bring on board for your shorter term needs.  Don’t take the sales pitch, their parent company’s reputation, or other easy indicators at face value, and dig a  little deeper to find out if they’re worth your time, and money.

 Recent Technical Articles

Tracking the Wily Proxy Hackers

Asterisk Calling Card Applications

MySQL Disaster Recovery

Dummy’s Guide to Linux Firewalls

Wireless Truth or Dare

Mirroring for the Impatient

Migrating a MySQL Database to Oracle 

Open Insights 08 – The Devil Is In The Details

Heavyweight Internet Group Newsletter

Issue 08 – The Devil Is In The Details
June 1, 2005

by Sean Hull
Founder and Senior Consultant
Heavyweight Internet Group

Welcome to our free monthly newsletter, discussing news, developments and business best practices at the intersection of Oracle and Open Source software.

We are now writing material regularly for our sister site so check there for more frequent updates.

The Devil Is In The Details

Project planning has got to be one of the most difficult areas in computing. And yet often it is what is most overlooked. Popular insistence on hourly rates makes for easy comparisons of two resources, and allows management to avoid scope and specification, the exact opposite of what you want to do if you’re trying to watch costs. Also, without a fixed fee you cannot make an accurate ROI decision because you don’t know how long it will take to complete. Person A costs X, person B costs Y, the cheaper of the two is probably a better bet, even if they’re a little slower. Not only does this miss the impact of experience, which drives up rates naturally, but also an experienced technologist can make strategic and time-saving decisions up front if he knows what the project is comprised of. But if he is being directed this way and that he is more likely to waste time and therefore money on the wrong things.

Time and time again I’ve seen projects get started, with the most clear intentions in mind end up being a huge can of worms once they get underway. Take for example an upgrade we did for a large security firm in midtown Manhattan.

The project started out fairly straightforward. Migrate an existing Oracle 8i database running on a terribly slow single processor and single disk Sun server to a 4-processor Sun server with a 10 disk raid array. Since there was version compatability we had a good sense that (a) the application would continue to work the same way, (b) the optimizer would still work the same way, so performance of queries would be consistent, and related Operating System and backup scripts would continue to work as before with little change. Predictability is the key to scoping a project, which is of course key to coming up with a fixed cost at the outset.

As things unfolded, the management team decided that 9i was a key requirement, and that despite potential trouble along the way, the expected downtime was a sensible time to upgrade, that application functionality did not rely heavily on 8i features that might have changed, and cost-wise it would be better to do the two together. Nice on paper.

Of course a change like this completely eliminates the predictability for a project, quickly pushing us onto an hourly basis for work beyond a certain point. It is open ended because application changes are difficult to predict and changes in Oracle’s Cost Based Optimizer could impact performance as well.

After a period of almost 20 hours of downtime, and a lot of tired IT folks, we managed to get things running again. The biggest hurdle turned out to be getting the standby database working again as we encountered some Oracle bugs with the configuration, which were causing core dumps, and general panic as well.

All of this underlines the need for careful planning, testing, and then deployment. It’s like checking out track conditions and the route on the day of the race, it just makes good sense. Even if you have ten years experience running marathons as we do in Oracle IT, you still want to do your due diligence or the technology might bite back. Also try to lean towards well-scoped projects that are conducive to fixed fees, and avoid hasty comparisons of hourly rates between different resources.

Recent Technical Articles

Tracking the Wily Proxy Hackers

Asterisk Calling Card Applications

MySQL Disaster Recovery

Dummy’s Guide to Linux Firewalls

Wireless Truth or Dare

Mirroring for the Impatient

Migrating a MySQL Database to Oracle

Open Insights 07 – Open Source in the Enterprise

Open Source in the Enterprise

Heavyweight Internet Group is of course involved heavily in Oracle

database administration, setup, management and tuning. The other half

of our business involves Open Source integration, and development

using various technologies such as the LAMP platform, Linux, Apache,

MySQL and

PHP. We also make heavy use of Open Source technologies to run the

enterprise, and thought it might be illustrative to talk about that.

There are a whole host of technologies and applications we rely on

from day to day to run our business. When I actually sat down to

write this months newsletter, I was rather shocked at how long the

list was. Here’s a peek into what we use.

Sales + Marketing

For starters we use the spectacular SugarCRM (

{}) for sales and customer relationship

management. More than an addressbook, it manages opportunity lists,

open tasks, calls, notes, leads, and accounts. You can keep track of

prospects at every stage of the sales process from prospecting,

qualification, needs analysis, value proposition, and id-ing decision

makers to perception analysis, proposal, price quote, negotiation, and

deal closed. There is also a dashboard which displays excellent color

graphs of sales stage, opportunity size, and breakdowns by month and

lead source. For a business like ours which has run for years on

paper, and disjointed organizing methods, this application has had a

tremendous impact.


Invoicing we manage with an application called phpaga. Still under

development, it already offers a host of features such as HTML or PDF

invoices, quotes, varying billable rates by project, and by resource,

as well as various graph reports. Definitely worth a look.

( {})

Web Content Management

Next we use PostNuke ( {}) to

handle website content management automatically and easily. The

formating is not perfect, but it works quite well for our needs. We

have also started using Moveable Type (

{}) for managing our new weblog Oracle +

Open Source (

{}). All of these solutions use Apache

( {}) as a webserver, and

MySQL ( {}) as a database for basic

website needs. Though this tool is not strictly open source, it has

fairly loose restrictions on usage, and comes with source code, so we

include it here. Of course you need a browser to view all

this great content, and though we dabble with Galeon, Opera, and even

IE from time to time, we primarily use Firefox


{}) because it is fast, and

secure, saving all of our computing environment from the trouble of

spyware, adware, trojans, and other malware.

Desktop Publishing

Next we have started using OpenOffice (

{}) which provides fairly consistent

compatibility with Microsoft’s Office suite of tools. To be fair we

also occaisionally use the Windows versions too, as client comfort is

ultimately our goal. We’ve also begun using Scribus

( {}) for some

applications, which provides Quark-type document publishing, and can

generate nice PDF files as output.

Email, SPAM + List Management

Email lists we’ve just installed some new management software for

called phplist. ( {}) It provides

support for multiple lists, and easy subscribe/unsubscribe which we’ve

currently been doing manually! For email itself we use Postfix

( {}) Not email

solution would be complete if it couldn’t handle spam. For that we

use a whitelist solution called ASK (

{}) Not always perfect, it became an

absolute necessity when our junkmail started reaching into the

hundreds a day. Now we receive about one spam a week, from the odd

spammer that bothers to do an active reply.

Operating Systems

Of course no enterprise would be complete without an operating system

and we use Mandrake Linux 10.1 (

{}) in most cases, including the desktop (KDE

and GNOME) and on our mail + webservers. When we need to remotely

administer Windows machines, as is necessary from time to time, the

Open Source Cygwin tool comes in very handy. It provides a proper

command line interface, and openssh implementation for secure remote

logins. Most of the other standard Unix tools are there as well, such

as Emacs for editing.


If you think you’ve heard it all, think again. Our enterprise phone

PBX uses the superb Asterisk PBX and VOIP gateway solution

( {}). This allows integration

of traditional trunks from the phone company as well as VOIP lines

over our internet connection. It supports standard telephone

equipment, IP phones, and digital softphones all of which we use

actively. We also use Skype ( {}) to

coordinate with technologists, partners, and clients overseas.

Monitoring + Backups

No enterprise would be complete without monitoring and backup

solutions. For monitoring we use Nagios (

{}) for monitoring our own, and our client

systems, including web, mail, disk usage, load averages, Oracle

statistics and much more. It notifies us by email whenever there is a

problem. We manage our backups with rsync and rsnapshot

( {}) two great

tools that are fast and efficient whe combined with a secure shell

like OpenSSH ( {}). For

site statistics we have some integrated information which comes

directly from Postnuke mentioned above, but we also use Webalizer

which provides fancy graphs broken down by month and year for

detailed information directly from the webserver logfiles.


Lastly we use a few tools for project management. Since we coordinate

and manage the efforts of a number of developers we use CVS

( {}) for version

control. This allows everyone to be making changes to the code at the

same time without stepping on each others toes. We also have a trouble

ticket system called OTRS ( {}) which

helps us manage change requests, and bugs in these various

applications so the developers know what to work on and what has the

highest priority.

In an enterprise which handles 14 hosted domains, mostly for

non-profit organizations, as well as six active clients, we need to

be organized to remain efficient and effective. We use powerful

software to help us do that, and remain focused on pressing business


Open Insights 06 – Practice What You Preach

This month’s newsletter discusses a little server mishap which we

encountered here at Heavyweight Internet Group. I use the example to

illustrate a few lessons.

Practicing What You Preach


Despite great efforts, it’s sometimes difficult to not come off a bit

preachy when discussing Open Source technology. The solutions are often

so good, you miss the difficulties. To that end, here’s a bit of a

rundown on what technologies we run our business on.

For Customer Relations Management we use SugarCRM. For a webserver we use

Apache 2. For a database we use MySQL though of course for enterprise

applications Oracle obviously comes into play. Our site software is

PostNuke, and our new weblog uses MoveableType. The

server that hosts all this great software runs Mandrake 10.1 distribution

of Linux. For email we use Postfix, and ASK a whitelist system for

spam blocking. We use Mandrake on the client side and the Firefox

browser. Also we use OpenOffice for creating and viewing Word, Excel, and

Powerpoint documents. And on the development side PHP, Perl, and CVS for

source code versioning. Soon we’ll be implementing OTRS for trouble

tickets, and possibly some web-based solution for invoicing.

The point here, and it’s a dramatic one, is that we learn by doing, and

sometimes the hard way. And by learning these lessons, we know better

what works for our clients.

Not Just Windows Servers Get Hit


Here’s a great example of learning by doing. We host our own servers even

though it might be simpler to outsource this process to a hosting company

who dedicates themselves to this. Not that we’d recommend this to all our

clients, we probably wouldn’t. But time and again it teaches lessons

about what technologies work, and what can happen during the minute that

your head is turned.

Last month our server was hijacked to send spam. This wasn’t done in any

of the obvious ways of finding a way onto the machine by brute-force.

They did not get a login to the machine itself. They managed to exploit a

default feature of our webserver, Apache, to proxy requests to other

sites. Though the obvious hole of having your mail server configured for

an open mail relay had been closed long ago, these spam hackers had

managed to find a way to relay through Apache. In so doing, they slowed

down our server for a period, interrupted our network, but worse sent out

tons of spam from our host. This meant we got thrown on a spam-blocking

list, and many sites were bouncing emails from us. After tracking down

the problem, we patched the problem, and eliminated the unwanted traffic.

Once again we’re able to send out email.

A hard lesson to be sure, but one we are sure to see, and identify quickly

at client sites, so they don’t suffer the same troubles.

In the long run these types of lessons are what help your professional

services stand out, beyond the obvious problem solving, to identify and

manage difficult and complex enterprise computing environments.

Open Insights 05 – Building on Success

Building on Experience


Thinking about this month’s newsletter, I considered titling it

“Building on Success”. But despite the marketing and media

hype, good companies, build on experiences both good and bad,

learn from them, and repeat the successful formulas, and are

careful to avoid repeating the mistakes. In short, finding the

right consultants to hire for a project isn’t about finding

those with the proverbial midas touch, but rather in finding

those who understand your organization, have the patience to

work closely with you, and the perseverence to follow

through to success.

Aligning the right people & resources


Facilitating communication can often be the greatest challenge in

any consulting engagement. Organizations, even smaller ones,

often have their own unofficial hierarchy of how things get done.

Consultants are brought in, not just when the right expertise

can’t be found in-house, but also when the expertise already there

isn’t solving the right problems. So inevitably these cases require

cutting accross the usual channels, to put the right people

together with the right resources.

This can come in the form of training, facilitating, and

coordinating these new relationships that foster the organization

to solve it’s own problems.

Turning ambitious strategies into successful ones


So what’s next? Ambitious and lofty strategies can often remain

out of reach until the right workflow is established. If that’s

not happening right away with your technology rollout, don’t

let the problem languish. Identify it, and resolve to

straighten it out.

Heavyweight Internet Group has spent nine years in the business

working closely with technology companies, identifying the

proverbial DNA of the enterprise, and putting together

technology solutions which solve real problems. Call us for a

free evaluation today. (866) 268 9448

Open Insights 04 – Managers: Databases 101

Managers: Database 101


Managers and CEOs of companies which rely on databases for their

infrastructure, or build on them for their applications, no doubt

already have a handle on the basic “What is…” question.

In short, databases boil down to glorified filing cabinets, albeit

electronic ones which can hold miles of encyclopias worth of

information, and retrieve it in tiny fractions of a second.

More info:

Behind the Scenes: How You Use it is Everything


We know that databases are put to an endless list of business

uses, from storing credit card billing statements & indexing

web content on the internet, to storing mountains of information

on customer spending habits from larger retailers. Running a

business, one naturally focuses on the business uses. But in

order to understand things at least a little bit under the hood

we’ll show how your technology people look at things.

Principally, there are two types of database uses. One is called

datawarehousing, and the other online transaction processing, which

we’ll call transactional for short. What divides them is dramatic.

A datawarehouse is composed mainly of large mountains of historical

data, which can be used to predict future outcomes, or answer other

big business questions. Huge reports are run against such databases

to show trends, or plot trajectories. How many of our customers

that live in Asia, buy books about technology, and when? How do

they normally pay? How many have joint accounts with family members?

A transactional database is one that runs a website, or other

application which makes frequent small requests of the database.

Such a database is characterized by lots of small requests or


You can think of the two types like readers of the news. The

datawarehouse is like the history researcher, that is going to comb

through old issues on microfilm, to put together a work of great length

on some past story. The transactional version is like the reader

going to work, who skims headlines, looking for important bits of

news, and leaves the newspaper on the subway where someone else picks

it up and scans it again.

Understanding which type of usage you are putting your database to

can help you focus your business and your technology choices

properly. It can also help your technology team tune your systems

for your specific usage. Mixing the two types of uses can have some

negative impacts easily avoided by dividing the work into separate

systems as appropriate.

Which are some of the popular variants?


Some of the popular databases in use today are obviously Oracle, Sybase

and DB2. These are the big commercial solutions. You might include

Microsoft’s SQL Server in the list, though it doesn’t really compete

in terms of real server technology performance and security. On the

other hand there are some dramatic open source solutions to choose

from as well, such as MySQL and Postgres.

What are the tradeoffs pro + con?


There are dramatic differences between the various database technologies

as you might guess. The first obvious one is price. Speed, and

supported platforms are factors too. Some less obvious factors behind

the scenes are the feature sets that each database supports. What

programming languages are supported, how big can tables and data grow

to, how are backups handled, how secure is the database, and how

difficult is maintenance? These are the types of questions which you

will need to ask to differentiate the various systems.

Is there a risk-free way to go?


A lot of companies have already made an investment in one technology

such as Oracle. Others have invested in Open Source solutions such

as Linux and MySQL and have built up the staff to handle that and

are wondering if the jump to Oracle would be difficult. On the

consulting side of the fence, most companies have partnerships and

experience with one particular technology and will try to lead you

in that direction.

Consider these factors when talking with and outsourcing, or

consulting company about database technologies. At Heavyweight

Internet Group we believe there is a right technology for each

business use, and we’ll help you understand those tradeoffs before

biting off more than you can chew.

Remember too that you can prototype with one database backend, and

switch later on as your needs grow if your application has been

written with those eventualities in mind. Furthermore you can

prototype and develop to a limited extent with Oracle without

purchasing a license. While you are doing development you are ok

after which time doing Q/A or obviously moving to production

would require a license purchase.

How do I know if I’m under or over-gunned?


An assesment of the current technology is the best way to figure

out where you stand on the database usage horizon. It involves

reviewing the underlying hardware, and Operating System, evaluating

the database technology, looking at the application, and how

well it is taking advantage of that database architecture, and

features, and seeing where the real bottlenecks lay.

Heavyweight Internet Group can help your business with database

technology assessments. With the right information, and options

at your fingertips, you can make more informed decisions saving

you time and money in the long run.



Your database is likely your single most sensitive technology,

housing your business crown jewels. Consequently just like the type

of car you choose can mean a lot in terms of expenses you will

incur, whether you can haul lumber, ferry your kids to school, or

drive the autobahn, so too what type of database technology you

favor can affect your business tremendously.

Look before you leap, choose the right guide, and you can grow

your business right — from the start.