This month’s newsletter discusses a little server mishap which we
encountered here at Heavyweight Internet Group. I use the example to
illustrate a few lessons.
Practicing What You Preach
Despite great efforts, it’s sometimes difficult to not come off a bit
preachy when discussing Open Source technology. The solutions are often
so good, you miss the difficulties. To that end, here’s a bit of a
rundown on what technologies we run our business on.
For Customer Relations Management we use SugarCRM. For a webserver we use
Apache 2. For a database we use MySQL though of course for enterprise
applications Oracle obviously comes into play. Our site software is
PostNuke, and our new weblog OracleOpenSource.com uses MoveableType. The
server that hosts all this great software runs Mandrake 10.1 distribution
of Linux. For email we use Postfix, and ASK a whitelist system for
spam blocking. We use Mandrake on the client side and the Firefox
browser. Also we use OpenOffice for creating and viewing Word, Excel, and
Powerpoint documents. And on the development side PHP, Perl, and CVS for
source code versioning. Soon we’ll be implementing OTRS for trouble
tickets, and possibly some web-based solution for invoicing.
The point here, and it’s a dramatic one, is that we learn by doing, and
sometimes the hard way. And by learning these lessons, we know better
what works for our clients.
Not Just Windows Servers Get Hit
Here’s a great example of learning by doing. We host our own servers even
though it might be simpler to outsource this process to a hosting company
who dedicates themselves to this. Not that we’d recommend this to all our
clients, we probably wouldn’t. But time and again it teaches lessons
about what technologies work, and what can happen during the minute that
your head is turned.
Last month our server was hijacked to send spam. This wasn’t done in any
of the obvious ways of finding a way onto the machine by brute-force.
They did not get a login to the machine itself. They managed to exploit a
default feature of our webserver, Apache, to proxy requests to other
sites. Though the obvious hole of having your mail server configured for
an open mail relay had been closed long ago, these spam hackers had
managed to find a way to relay through Apache. In so doing, they slowed
down our server for a period, interrupted our network, but worse sent out
tons of spam from our host. This meant we got thrown on a spam-blocking
list, and many sites were bouncing emails from us. After tracking down
the problem, we patched the problem, and eliminated the unwanted traffic.
Once again we’re able to send out email.
A hard lesson to be sure, but one we are sure to see, and identify quickly
at client sites, so they don’t suffer the same troubles.
In the long run these types of lessons are what help your professional
services stand out, beyond the obvious problem solving, to identify and
manage difficult and complex enterprise computing environments.