I worked for a medium size internet startup called Method Five. When I came on board they were having a terrible time with their site performance.
When I first met the team, I was tasked with performance problems. After all their flagship web property kept crashing, and it didn’t look good to investors. As with most web properties in those days it was a home-grown datacenter in the back of the office, running on Sun Microsystems hardware, with Oracle on the backend and Apache serving webpages.
As it became clearer after day one, the project was particularly sensitive. They were negotiating a huge acquisition by a firm called Xceed Corp. The sticking point? Their crashing website did not sell their technology prowess in a particularly positive light. To say the least!
As it turns out the site had all the right players, from systems administrators to a DBA who sat watch over the Oracle systems.
As I dug into the systems, I found a serious smoking gun. It seems the Oracle software was configured to use just 5M of memory out of about 256M free. Just like MySQL, the server must be configured to use available memory upon startup. There are myriad caches and buffers which need to be attended to. By today’s standards these numbers probably sound absurd. Nevertheless the DBA wasn’t familiar with the basic memory settings, and so the system was terribly bottlenecked.
We then ordered some urgent changes to the system, configuring all of Oracle’s caches to use up the precious memory available.
Immediate the website unlocks, transactions begin flowing, and webpages are returning quickly. End users pull their noggins off their keyboards, and the executives begin breathing a sigh of relief. The site was literally 1000x faster during peak.
Shortly thereafter the acquisition goes through for a cool 5 million in cash and 80 million in stock.
Where’s my cut?! You might be asking that question. But my policy is almost always defer to something concrete and tangible, aka fees and real compensation. I did not negotiate any stock in the deal.
Another popular war story we wrote A CTO Must Never Do This….
o Don’t believe received wisdom. Check and double check what’s really happening.
o Use the memory and resources you have available.
o Measure capacity, and isolate bottlenecks in the system
o Decouple services wherever possible
o Problems are as often people and process as they are with technology
Make it this far? Grab our newsletter!