Category Archives: All

5 Things You Overlooked with MySQL Dumps

1. Point In Time Recovery

If you’ve never done point in time recovery, it’s time to take a second look. With a standard mysqldump you restore your database to the time when the backup happened. Only do them once a day, then you can lose as much as 24 hours of data.

Enter point-in-time recovery, and you have the option to restore all those transactions that occured since your backup last night. Those changes, INSERTs, UPDATEs, DELETEs & even ALTER TABLEs are all stored in the binary logs. That’s right, all those statements are preserved for you!

The tool to fetch those statements with is called mysqlbinlog. When you use it, you’ll need to specify the binlog file & offset to start at. Uh oh, you don’t have those?

That’s where the mysqldump option –master-data comes in. We recommend using –master-data=2 option. That includes this important nuggets of information in the dump file as a comment.

Now you’ll know where to start applying transactions. Simply dig this information out of the dump file.

What I like to do is dump the statements into a text file first, then apply them.

Here’s how you fetch all the transactions since your dump:

$ mysqlbinlog --offset=12345 /var/lib/mysql/binlog.000005 > my_point_in_time_data.mysql

Here’s how you would then apply them:

$ mysql < my_point_in_time_data.mysql

Also checkout: Replication setup with no downtime - using hotbackups!

2. Locking & Consistency

MySQL can be confusing when it comes to concurrency and locking. Do we need to lock tables during a backup, yes, no? Sometimes? The confusion is because MyISAM tables are non-transactional. So sometimes locking all tables is required to get a consistent dump.

--lock-all-tables

This option is only required if your application works on tables across schemas. So if any one application hits multiple schemas and there are MyISAM tables in both, you'll need to use this option. It's equivalent to FLUSH TABLES WITH REACH LOCK.

--lock-tables

This option locks all tables in one schema, performs backup, then moves on to the next schema. You only need it if you have MyISAM tables. It's on by default and with --opt flag.

--single-transaction

This is the least invasive way to do a dump of your data. However it only works if all your tables are InnoDB. Keep in mind it won't error out if you have MyISAM tables, but your backup will be inconsistent. The plus side is that from your applications perspective there is really no blocking locks happening. Very cool indeed!

For a more general guide see Accidental DBAs Guide to MySQL.

3. Backups of stored code

If you have stored procedures, triggers or functions, you're not capturing them in your dumps by default. Really! Use the --routines option to include them. Better yet do a dump with just your stored code. This line will do so, and add a date timestamp for you. Be sure to have the [client] or [mysql] section in your .my.cnf file with user & password specified. Then you don't need them on command line.

$ mysqldump --no-data --no-create-info --all-databases --routines > db_code_`/bin/date +%m%d%Y`.mysql

Are you verifying replication slaves? You should know they regularly drift out of sync without throwing errors. Read more to verify MySQL replication..

4. Separate schema & data dumps

Sometimes it's useful to have a schema only backup. That is a dump of the objects in your database (CREATE statements) without any of the data. Why you ask? For starters you could use the unix "diff" command between a schema dump today, and one you did last month. That's a cool use case, but there are others. Suppose you want to build a test server, without all the production data or some subset of it? Again, a schema only dump is your friend.

o Use unix diff to compare objects in two different databases
o Audit schema changes over time
o Build test servers without all the production data
o Parallel loading of data
- use multiple mysql processes to import data
- break up the data only portion of your backup

Here's the syntax to get the schema only dump:

$ mysqldump --no-data --all-databases > db_schema_`/bin/date +%m%d%Y`.mysql

Here's how you get the data only dump:

$ mysqldump --no-create-info --all-databases --complete-insert --skip-quote-names > db_data__`/bin/date +%m%d%Y`.mysql

Looking for a MySQL DBA position? Check our our interview questions guide.

5. A few more options

--complete-insert

By default mysqldump includes INSERT statements built to *follow* create statements. So the assume all the columns match. If you break up your schema & data you could potentially have a mismatch. This option makes sure the INSERT statemens included in the dump specify column names. So if they don't match, you'll get a proper error message.

--skip-quote-names

MySQL's default dump settings will quote everything under the sun. That's to protect if you use certain reserved words or worse if you have objects with spaces in the names. But of course best practices advice against *both* of those in your database, so be advised. This option will make your dump much more readable and easier to edit.

--hex-blob

We had a customer who was using the cross-platform media publishing system K4. When we moved data from one server to another, we had a terrible time at first. That's because this enterprise application stores data in MySQL blobs. Unfortunately the default mysqldump options didn't account for certain special characters which were getting munged. The way around that is to use this option. If you have blobs, you really need this.

Read this far? Grab our scalable startups newsletter.

Beware the sales wolf in sheep suits

Recently a colleague called me up to get my opinion.

[quote]We’re in the process of standardizing our systems on Red Hat Linux, but management and higher ups are convinced we should deploy Oracle on Oracle’s own Linux distribution. Which is better?[/quote]

Therein lies the eternal drama in organizations, the push & pull between dollars and technology best practices.

We had a similar experience with a MySQL deployment, and solution framed by Oracle sales.

Battle lines are drawn

Clearly the battle lines are drawn now. Between director of operations & team versus management & business stakeholders, between high level and the trenches, or between the systems that support your business and day-to-day running of them.

Business units & management are tasked with budgets, cost management, and long term thinking about trajectory and what’s best for the business. Operations teams are tasked with the day-to-day stability, the command line perspective.

What is the sales team’s position?

Sales guys at Oracle have a job to sell licenses. This isn’t good or bad, it’s their driver. Understanding all the drivers will help us align them.

Sales guys sell to management, so they will likely frame all their stories to management concerns. Also Oracle’s history here is fairly clear. Get customers locked into Oracle up and down the stack, and they become more and more beholden to you as their primary provider. As customers become more dependent, they will begin to squeeze more and more out of them.

Nothing personal, this is how money is made. But understand the goal.

How do OS choices affect the business bottom line?

Standardizing across the enterprise reduces costs & reduces operational complexity. This can reduce risks to operator error & other downtime that increase with more heterogeneous environment.

On the Oracle distribution side, you likely have tweaks to make Oracle run better. However don’t forget the profit motive. Some tweaks may be conveniently “overlooked” in favor of profit. For example for many years the Oracle installer would not complete without error on many Linux systems. Imagine all the professional services that are sold around running through a complex install. Streamlining such an install would *reduce* profits. Don’t laugh.

What happens on the front lines?

On the front lines of course are the ops teams & DBAs, actually installing and supporting enterprise software. Let’s not forget these guys are at the command line. They know inordinately more about what’s really happening down in the trenches. You may find them repeatedly rolling their eyes at salesmen claims.

However they are not the colorful storytellers or communicators that salesmen are, so they may

Want to hire a DBA? Here’s our MySQL interview hiring guide. We also wrote a similar one for Oracle DBA Interview questions.

Align each division’s interests

Despite cultural differences, business management & operations teams should work hard to connect, and align with one another.

Operations should make an effort to better understand the business bottom line. Money doesn’t grow on trees as they say, and choices have to be based on budget, and real-world needs. We’d all like to sit in a university and program or build things just to create something new, but in a business there are market pressures. All teams should reflect on those.

Management should also make an effort to understand ops teams needs. Why are my ops teams telling me a different story than they Oracle sales guys? Fight the urge to bond with the sales folks, despite their smooth delivery, great suits and peer positioning.

Weigh short and long term tradeoffs

List out advantages & tradeoffs on all sides. These should be technical and business bullet points. Brainstorming a full list like this, and having the whole team discuss the list openly will help the team together come up with a more realistic outcome. Some questions to ask…

1. What are the advantages & disadvantages of having multiple providers for your technology stack?
2. Which solutions are open and which are proprietary? What are the tradeoffs there?
3. What does your team have subject matter expertise in?
4. Are there real technical advantages to one solution or the other?
5. Are there real cost advantages to one solution or the other?
6. Are there expertise advantages & training savings to go one direction?
7. Is the technology widely used in your industry? Will additional or replacement operations experts be easy or hard to find?

Read this far? Grab our scalable startups newsletter.

Juggling apples & oranges in the datacenter


In which a few choice words become one serious accident…

The Backstory

More than five years ago now, I worked for a shop in the business of news & information around the legal and real estate sectors. It was a fairly large organization with a number of Oracle and MySQL backed applications. The whole place ran on Sun servers, with a team of systems administrators, developers, and of course editors & content folks.

I was the primary database administrator for almost an entire year back then. I reported directly to the CTO. She was bright, competent and great to work for.

Although she had a technical background, she often spoke about products and gave very high level directives when making requests. This was made more confusing as the environment lacked naming conventions. So often product names didn’t match server or database names.

I tended to take the very paranoid approach. I’d ask over and over for clarification, and let some time pass before actually executing on a request.

A Changing of the Guard

After many months as a contractor DBA, the firm finally located a fulltime guy to replace me. It’s no easy task finding a DBA these days, especially for MySQL.

He was a very bright guy with a lot of technical knowledge. A bit green behind the ears, but fully capable to manage an enterprise database shop.

Looking for a top-notch DBA? Here’s our MySQL interview questions & hiring guide. We also have one for hiring an Oracle DBA.

Nuking the database

After two weeks on the job, something unpleasant happened.

Imagine a chef working with cooks & confusing dishes with vegetables.

[quote]
Chef says, “Toss the avocado”
Cook throws the avocado salad in the trash thinking it’s rotten.
Chef comes back later asking quizzically, “I wanted you to mix it up!”.
[/quote]

In the datacenter the conversation went something like this…

[quote]
CTO: Drop the journal database & rebuild.
DBA: Ok. Give me a few minutes
CTO: What did you do? The whole application is offline now!
[/quote]

From there scrambling ensued. After nearly six hours of screaming, and firefighting, everything is finally restored from backups and the application brought back online.

Naming – product or components?

Semantics is very important. Those in the trenches tend to take requests word-for-word while those managing the troops tend to make requests in terms of products, divisions & the vantage point of the business.

That’s why naming conventions can be so important. Don’t want to be talking about apples when you really mean oranges.

Living with dysfunction

As environments grow over years and years, they tend to evolve into a spaghetti of confusing names & relationships. It’s the nature of enterprise environments.

– big confusion can mean big mistakes
– check & recheck – be risk averse and a bit paranoid
– check yourself, your shell, your hostname, your login
– ask questions & clarify repeatedly
– let some time pass before executing a destructive command

Made it this far? Grab our newsletter.

Powerful punchy prose, big hit blog

Word Hero sounds familiar doesn’t it? I had bumped into the title here and there. I finally picked up a copy when I realized it was by the same author as my recent favorite Thank You For Arguing – Jay Heinrichs. Turns out his title inspiration was Guitar Hero, and therein lies the secret to success.

Get Viral

Timing is critical as we learned writing the book on Oracle and Open Source. But just as important is pithy, punchy prose.

Word Hero takes you through chapter by chapter introducing you lightly to the wisdom of the ancient Greeks, but more importantly explains exactly how to get to these phrases that have a life of their own.


o take great phrases & dissect them to see how they work
o illustrate great rhyming & verbing
o focus on the sound to make things lyrical
o puns, mad libs & the square root of rainbows

He quotes from great pop cultural stuff, and even a bit of Warren Buffet & Yogi Berra. By far my favorites though are his quotes from the popular TV show Glee:

[quote]You think that is hard? I’m passing a gallstone as we speak. That is hard![/quote]

If you’re a freelance blogger or sometime free agent, you should also take a look at Startup of You by Hoffman & Casnocha.

Practice your techniques

I definitely like the way Heinrichs breaks down techniques & offers illustrates exactly how to give it all a try yourself. I tried it myself a bit with this blog post. What do you think?

I also tried to put it to use in this review of Startup of You. I titled it Opportunity a day – career risk at bay. Kinda catchy I thought.

Definitely grab a copy of Word Hero, it’ll help you with your blogging, sending a memorable email, or putting together presentations that resonate.

Don’t forget to work on your Klout score!

Read this far? Grab our newsletter.

Opportunity a day – career risk at bay

Free Agent. Stress Test. Avoid Sameness

As the globalization juggernaut rolls on, it continues to create more Detroits. Skills and perspectives quickly become obsolete.

What to do in the face of such change?

[quote]Small fires prevent the big burn[/quote]

So there’s your quick answer. Get the book if you want more!

Some related material: why is it so hard to find a mysql dba?.
Consulting 101 Guide – Finding Business :: Completing Engagements :: Growing business

Your Mentors

On this tour, a free agent needs mentors. Hoffman & Casnocha provide you with plenty from stories & lessons from some of the startup industry’s finest. Jack Dorsey, Mark Andreesen, Cheryl Sandberg, Rick Warren, Paul Graham, Jeff Bezos, Joi Ito and a few of their own running Paypal & Linkedin.

What you’ll love

Each chapter closes with concrete actionable advice. The authors carefully craft marching orders for you in the next day, next week and next month. Go ahead, give them a try.

[quote]Safe is the new risky – Phil Simon[/quote]

An executive summary of Startup of You

1. develop your strengths
– what do you find easy that others find difficult?
– diversify asset mix aka learn new skills

2. plan to be nimble
– pivot as you learn more
– always prepare a lifeboat contingency plan

3. work & develop your network
– hangout with those already on the road
– domain experts, people who know you & smart people

4. hustle for breakout opportunities

5. Embrace baby steps of risk
– bounds of unemployment – shocks that motivate
– adjust your strategy & pivot if need be

What’s next?

Had a taste and want more? If you’re a MySQL DBA we wrote an interview guide. Also check out our Oracle dba interview questions.

Want more? Check out our best of content compilation.

[quote]Only the paranoid survive. – Andy Grove[/quote]

Read this far? Grab our newsletter. We cover all sorts of great topics for free agents, consultants, and those who want to hire them.

Sometimes… let things break a little

Have you ever started a new project, just into it you realize that maybe there aren’t technical problems to solve? It starts to dawn on you the real crux of the problem boils down to people & processes?

It’s happened to me on a number of occasions, but once in particular really stands out for me.

I was working for a firm in the education space, in particular around test preparations.

Asked to automate a publish process

The environment had a mix of relational databases, from SQL*Server to MySQL for some applications. The web facing database however used Oracle on the backend.

Their career DBA was real old guard Oracle, he had his ways of doing things, and didn’t want to rock the boat. In particular he managed the process for publishing changes to the website. Publishing amounted to running a few hand rolled scripts and each step was a manual one.

With the process setup this way, the editors had to work closely with engineering each time they wanted to move content to the website. Slow, cumbersome, and not very workable.

The real problem, siloed departments & infighting

As I worked closely with the DBA, quite a few things became clearer. For one he was sometimes a grumpy fellow & he had a strong accent which was sometimes hard to cut through. Knowing the other team members, I knew this all contributed to the trouble. But he maintained quite a bit of resistance to automation of the process no matter what. His view was, if he hands over the reigns to editors who don’t understand the technology, they’ll screw something up, make a mess, and that would ultimately create more work for him. After all that he’d be doing it manually anyway!

Further attempts to communicate between teams or even between the managers and this guy went nowhere.

It’s not easy to find a good DBA. We wrote a MySQL interview guide to help and one for an Oracle interview too. The mythical dbas remain in rather short supply.

We weren’t trained for this in engineering school

When you’re looking for a technical solution and you realize the bigger issue is a people problem, what do you do? You can gently bring it up with the higher ups, but they may have a different style, or prefer the shout orders and bark mode of management.

Things continued to go around in circles, and attempts to get further information from this DBA didn’t prove fruitful. He was protective of his domain, and fought tooth & nail to open up.

Things come to a head

The bigger boss, the one above my direct report, one day called me into his office. Actually it was an off day I was just stopping by to check in on progress.

He pulls me into his office. Since I rarely interacted with the guy, my guard was very much down. I thought we’d have a chat about the weather or perhaps who was going to win the world cup.

He proceeds to tear into me without warning. Practically screaming, he’s giving me a piece of his mind and not stopping to hear what I have to say. Where is this project going, why is there no progress, we’ve got serious deadlines, you’re pushing us right up to the wall … that kind of thing.

As I listened to him fire away at me, I realized some of this had gotten filtered through some sources, who didn’t completely understand the blocking issues either. That it wasn’t technical challenges, but rather people and processes that weren’t working. As I began to explain this, he stopped me and said:

[quote]Sean, your job is to push, push, push and push us more. Rock the boat if necessary. When I was a constultant I was constantly running around making sure everyone was talking to eachother[/quote].

A few things ran through my mind at that point. One was well he’s not a consultant, so either he couldn’t last in it, or the life didn’t appeal to him. Or perhaps his skill sets ran truer to management in a large firm, a different albeit tougher role to master.

But it also occurred to me that different folks have very different styles, some like to push, and prefer confrontation & believe that leads to resolution. While others are more listeners and find there way around a problem by giving everyone a chance to voice their positions.

My style is the latter, while his was clearly the former.

The fallout

What ended up happening is a project manager also got assigned to the project, as well as another manager. The PM liked working with me very much, and as things unfolded, much of the departmental siloing began to dissolve, and the bigger communication problems began to surface. From there solutions followed.

Lessons learned

– beware the status quo – some don’t really want to rock the boat
– communicate your position, but beware that others may have a different style
– you may be asked to support a path you see as the wrong one
– let things “break a little bit” so everyone learns the hard lessons
– getting burned can be a lesson for the whole team, and lead to new solutions

Made it this far huh? Grab our newsletter.

Oracle DBAs… You Know You Want MySQL!

If you’re an Oracle DBA or developer and considering migrating an application over to MySQL you probably have a lot of questions. Here’s a five minute summary of what you should know.

Looking to hire a top flight MySQL DBA? Check out our MySQL interview guide.

What is truly delicious

o a command line that’s decades ahead of SQL*Plus
o fast & easy install via yum or apt-get package managers
– saves hassles, headaches, and standarded on linux distribution
o a very active open-source community with tons of roll-your-own solutions
o myriad of replication topologies with endless use cases

Flashback technology

Although MySQL doesn’t provide this by default, a slave set to lag behind by an hour or more can serve this function quite nicely. Perfect DIY solution.

Auditing

Out of the box, MySQL doesn’t have great auditing facilities. However the binary log can be of great assistance here. Use the mysqlbinlog tool to get human readable data and filter for the information you need with your favorite Unix tools such as sed, awk, sort and so forth.

MySQL also includes a general query log which may also be useful for such auditing jobs.

Users & Security

Logins are authenticated by hostname & username. So user@localhost has a distinct set of permissions from user@remoteserver. This can be very confusing so make every effort to create users & grants consistently. Also use –skip-name-resolve option to avoid DNS lookups. These can easily cause scalability problems.

Stored procedures

They’re there in MySQL, but not robust. Use sparingly. They can easily break replication, and cause serious performance and scalability problems. They also make your deployments more complicated – for example mysqldump requires the –routines flag to capture them.

SQL Queries & the cost based optimizer

MySQL uses a cost based optimizer like Oracle’s however it is much less mature. Don’t expect the same level of smarts. Some queries will need special attention.

o there’s no hash join, only nested loops
o you can use one index per table – no index merging
o correlated subqueries with IN() lists don’t optimize well
o not much control over stats collection & freshness

Partitioning

You can partition large tables by range, key or hash. However managing to get your queries just right to do partition pruning may be a trick. What’s more all the added operational benefits you get in Oracle may be limited. You can drop partitions quickly, however reorganizing them will be painfully slow.

Materialized views

MySQL doesn’t offer them per se, but check out Justin Swanhart’s Flexviews for a solution that may give you what you need.

Replication

Oracle standby database is a change based technology. Although MySQL is beginning to experiment with this using row-based replication, the vast majority of deployments are statement based.

Since slaves are hot or read-only, they can be used to horizontally scale your applications read capacity, serve as live backup, makeshift flashback or a logging apply server. Here’s how you can setup replication without downtime using hotbackups.

Be vigilant about performing checksums against your slave instances. This will alert you to any drift or other inconsistencies across your tables or data. We wrote a howto bulletproof replication with checksums.

Clustering

Don’t expect to find something like Oracle RAC. You won’t. For any typical web-facing database needs, MySQL NDB Cluster is most likely not what you’re looking for.

Consider MySQL master-master replication where one is active and one is passive. Configured as circular replication, all changes propagate to all nodes. It’s crucial you only ever write to one node, however. Use the inactive node to do online schema changes, test upgrades or other availability sensitive operations.

Lastly you may try DRBD which is a Linux filesystem level solution. Keep in mind your inactive database will be cold. You won’t experience any replication consistency issues, but upon promotion to master, you may experience slow performance until the caches are warmed up.

A few miscellaneous surprises

o many ALTER functions are blocking operations, so can’t be done online
o replication can be troublesome to manage & drift out of sync silently
o default MyISAM tables can corrupt easily and lose data – use Innodb instead
o complex many table joins & certain subqueries do not perform well
o there is a lot less instrumentation around the optimizer & db engine

Read this far, grab our newsletter!

Where’s my 80 million dollars?

Way back in the heydays of the dot-com boom, the year is 1999.

Join 12,100 others and follow Sean Hull on twitter @hullsean.

I worked for a medium size internet startup called Method Five. When I came on board they were having a terrible time with their site performance.

Website crashing

When I first met the team, I was tasked with performance problems. After all their flagship web property kept crashing, and it didn’t look good to investors. As with most web properties in those days it was a home-grown datacenter in the back of the office, running on Sun Microsystems hardware, with Oracle on the backend and Apache serving webpages.

Also: Why a killer title can make or break your content efforts

Negotiating an acquisition

As it became clearer after day one, the project was particularly sensitive. They were negotiating a huge acquisition by a firm called Xceed Corp. The sticking point? Their crashing website did not sell their technology prowess in a particularly positive light. To say the least!

Read: Why high availability is so very hard to deliver

Investigation

As it turns out the site had all the right players, from systems administrators to a DBA who sat watch over the Oracle systems.

As I dug into the systems, I found a serious smoking gun. It seems the Oracle software was configured to use just 5M of memory out of about 256M free. Just like MySQL, the server must be configured to use available memory upon startup. There are myriad caches and buffers which need to be attended to. By today’s standards these numbers probably sound absurd. Nevertheless the DBA wasn’t familiar with the basic memory settings, and so the system was terribly bottlenecked.

Read this: Why a four letter word divides dev and ops

Problem Solved

We then ordered some urgent changes to the system, configuring all of Oracle’s caches to use up the precious memory available.

Immediate the website unlocks, transactions begin flowing, and webpages are returning quickly. End users pull their noggins off their keyboards, and the executives begin breathing a sigh of relief. The site was literally 1000x faster during peak.

Related: MySQL interview guide for managers and candidates alike

Acquisition

Shortly thereafter the acquisition goes through for a cool 5 million in cash and 80 million in stock.

Where’s my cut?! You might be asking that question. But my policy is almost always defer to something concrete and tangible, aka fees and real compensation. I did not negotiate any stock in the deal.

Another popular war story we wrote A CTO Must Never Do This….

Read: Why devops talent is in short supply

Lesson’s Learned

o Don’t believe received wisdom. Check and double check what’s really happening.
o Use the memory and resources you have available.
o Measure capacity, and isolate bottlenecks in the system
o Decouple services wherever possible
o Problems are as often people and process as they are with technology

Also: 5 more things deadly to scalability

Make it this far? Grab our newsletter!

You're Too Young To Be My Boss

About a year ago I engaged with a firm to do some operations work on their site. They provided services to colleges and universities.

When they first reached out to me, they were rather quick to respond to my proposal. They seemed to think the quote was very reasonable. I also did some due diligence of my own, checking the guy’s profile on the about page. I noticed he was 25, rather young, but I didn’t think much else of it.

We discussed whether they wanted fixed hours. Since those would limit my availability we both agreed a more flexible approach made sense. This worked well for me as I tend to shift and schedule time liberally, so I can be efficient & flexible with clients, but still have a life too.

Trouble Brewing

As we began to interact the first week, I sensed something amiss. My thought was that the first week you work with a client, they feel you out. They see how you work, when you work, how much gets done and so forth. This provides a benchmark with which to measure you. If either party is unhappy with how things are going, they discuss and make adjustments accordingly.

What was happening in this case was the guy started pestering me. I began to get incessant messages on instant messenger asking for updates. I had none. I explained that I would contact him as things were completed, or if I had questions.

This was only two days into the project. I’d barely gained access to the servers!

The Fever Pitch

After discussing my concerns on the phone, the gentleman kind of glossed them over. From there the pestering continued. I explained that I could not be available to him any hour of the day, while the engagement only provided for one half of a week. This began to interrupt me from other client work, so I had to signoff of instant messenger. Not good.

The Pot Boils Over

We spoke again on Monday briefly, and decided to connect the following day. From there the pestering began anew, and I began to lose my patience. I insisted that we speak on the phone before work would continue. I felt the problem was deteriorating and discussing over text would only make things worse.

He emailed me back as I was then offline. In his email he ordered me to come online. While he sat in a meeting, he explained, he could not take a call! Nevertheless he insisted we resolve it during the meeting. Distracted no less.

[quote]It was then that I started receiving text messages on my personal mobile phone from the guy, pestering me to get online so we could resolve our communication problem! You can’t make this stuff up![/quote]

The Fallout

Eventually we did both get on the phone, and I explained I had reached wits end. After only ten short days of working together, we had both set strong precedents and they were obviously not compatible. He asked if I would stay on longer, and reconsider working together, and I said I would think about it.

I chose not to dig a deeper hole, and let him know I wouldn’t be invoicing for previous the weeks work.

The Lessons

o beware age differences – in our case an 18 year gap
o pay attention to management styles – self-starters don’t need micromanaging
o be patient & keep communicating
o allow for an exit strategy that is amenable to both parties

Read this far? You’ll love our newsletter. Get Scalable Startups. No Spam. No Selling..

Why I Wrote the Book – Oracle and Open Source

Back in the late 90’s New York City was deep in the dot-com boom. Silicon Alley was being born, and a thousand internet startups were sprouting. Everyone was hiring, it was an exciting time to work in technology!

Join 11,500 others and follow Sean Hull on twitter @hullsean.

Trend Spotting Circa 2000

As an independent consultant, I had the opportunity to work at quite a few startups. The technology stack was identical at almost all of them. Sun Microsystems hardware, Apache webservers, and Oracle on the backend. The database was always the sticking point, and developers struggled to get their queries right.

It was an interesting role to hold. Most career DBAs worked at large fortune 500 firms, the old stodgy kind where nothing ever changes. Few of the Oracle old guard, the kind you’d meet at User Groups or conferences, had much exposure to Linux, and they certainly didn’t trust it.

Also: Here’s how to do a scalability performance review

Meanwhile in the startup scene in NYC I was seeing the cutting edge uses of the technology, with more and more shops switching to Linux and commodity hardware. There was even talk of *gasp* Oracle porting to Linux. There was a real rumor mill around all of this.

Oracle and Open Source Published – 2001

Seeing this shift towards commodity hardware, and the tremendous demand for Oracle married with open source technologies, I pitched O’Reilly and Associates with a book idea. Let’s talk about what’s happening in the trenches. How and when does Oracle – the most commercial of relational databases, work with Open Source technologies? What is in the mix? What are real firms using it for? What tools and technologies can help firms grow faster?

Related: Oracle DBA Interview questions for managers, candidates & recruiters alike

These were the questions my co-author and I sought to answer, and to judge from the response I think we did a very good job. As that push continued, Oracle eventually ported it’s enterprise database to Linux. This was a seismic shift that meant existing Oracle customers would spend a lot less on hardware, and thus have more to spend on Oracle licenses. Win-win except for Sun. The trend continued with Oracle pushing Apache into the mix as well.

Fast Forward a Decade

Now a decade later, Oracle has bought it’s former partner Sun, and in so doing owns MySQL too.

Read this: Top MySQL Interview questions for Devops, managers & recruiters

What new trends are happening? We hear an incessant drum of hype around cloud computing. In many ways the trend parallels what happened a decade ago. See our related piece a history lesson for cloud detractors. How so?

[quote]Commoditization: push towards new platforms, driven by cost. [/quote]

But this is slowed by an equally large stumbling block.

[quote]Performance: new cloud servers can’t compete with their big iron cousins. Not yet at least.[/quote]

Interested in Amazon EC2? We wrote an Intro to EC2 Cloud Deployments article which digs in deeper.

What’s Next for Datacenters

Commiditization will continue, driving costs downward. This will provide more gravity to cloud migrations for firms big and small.

Performance will improve. Cloud services like Amazon EC2 will get bigger & better, as will the all important network & disk subsystems.

Also: 5 things toxic to scalability

Big enterprises are already dipping their feet in the water with VPC technology, tying their existing datacenter to a cloud. They can grow elastically while still having feet firmly planted on the ground.

As large enterprises begin to get experience behind the wheel, it’ll chip away at the stranglehold of Oracle and the huge taxation type licensing that firms struggle with today. Where salesforce.com had a huge impact, workday.com will be even bigger.

[quote]The cloud will finally disrupt the last old guard industry – enterprise software.[/quote]

Read this far? Get us monthly in your inbox. Grab our scalable startups newsletter!