In the last two posts on how to build a successful consulting business I shared advice and tips on closing deals and managing and completing your engagements.

This post will look at where to focus your efforts in order to sustain your consulting business, and build skills.

Focus on your subject matter expertise

Being a subject matter expert takes years of education, and professional experience to build. It’s your most valuable asset. Build it, and use it. This is not to say there isn’t great value in being a generalist (link) as I’ve emphasized in blog postings. But don’t get distracted trying to do things others can do better. Web-based newsletter managers can do it better than you can, professional invoicing solutions as well. Though you might be able to do a bit of HTML and graphic design, if that’s not your expertise, hire someone to do your website in WordPress. You’ll save time and money in the long run, and their professional experience will surely include a few surprising nuggets of value that you wouldn’t have known yourself.

Manage your business with online tools

There are a myriad of online tools to help you run your business better.

Google Analytics – Get insights on what parts of your site your customers find valuable. Learn to convert those visitors with downloads, newsletter signups and ebooks for sale.

Freshbooks, Zoho or Paymo – There are many online invoicing solutions, so best to evaluate them for your specific needs. The point is, using a solution will save you money in the end, and make you more professional in the process. It’ll help you track monthly & yearly, send reminders, and summarize things for you all in one place.

Mailchimp – Newsletter creation can’t get any more fun than with mailchimp. With endless video howtos, documents, and great support, plus templates for your newsletter that come in every shape and size, these guys have really thought of everything. What’s more the graphs and campaign reports will give you insight into what topics are popular, and what resonates with your audience.

Google Docs and Calendars allow you to share your schedule and work with a small team. So whether it’s communicating things with your clients, or your subcontractors, these tools are essential.

Dropbox – Another indispensable tool for collaboration, share documents easily across the web.

Don’t Be a Commodity

If your prospect is asking you for a resume, and comparing you to some outsourced resources provide, you’re not going to get very far as a consultant. Don’t be a generic resource that sits at the keyboard and churns out code.

What’s the alternative? Learn about business, talk to business stakeholders, listen to their concerns, and learn to speak their language. Use stories, and analogies. Reach across the table to speak in terms that everyone can understand. Your value is in symplifying complex technology.

Be Hungry & Don’t Give Up!

Being hungry means continuing to improve on your weaknesses, experimenting with different rates until you match the market demand, managing multiple projects and always networking widely. Spend less than you make and build up six to twelve months cushion. This will allow you much more freedom to take only the projects you want. Learn to pick and choose.

Further Reading…

There are lots of other areas you’ll lean about, running a successful consulting business. Here are a few:

• managing cash flow
• paying quarterly & yearly corp taxes
• paying subcontractors
• managing & accounting for expenses

For more articles like these go to iHeavy, Inc +1-212-533-6828

]]> http://www.iheavy.com/2012/05/17/consulting-essentials-building-your-business/feed/ 0 http://www.iheavy.com/2012/05/15/consulting-essentials-managing-completing-engagements/ http://www.iheavy.com/2012/05/15/consulting-essentials-managing-completing-engagements/#comments Tue, 15 May 2012 16:47:49 +0000 Sean Hull http://www.iheavy.com/?p=4358

Read the original article at Consulting essentials: Managing & Completing Engagements

This is the second in a series of three articles on Consulting Essentials. Read the previous post, Consulting essentials: Getting the business Communicating well and knowing when to step in or stand back is the linchpin of successful consulting. Some people have natural charm. If you’re one of these people you’ll find consulting is definitely [...]

For more articles like these go to iHeavy, Inc +1-212-533-6828

]]> Read the original article at Consulting essentials: Managing & Completing Engagements

This is the second in a series of three articles on Consulting Essentials.
Read the previous post, Consulting essentials: Getting the business

Communicating well and knowing when to step in or stand back is the linchpin of successful consulting.
Some people have natural charm. If you’re one of these people you’ll find consulting is definitely for you. You’ll use that skill all the time as each new client brings a half dozen or a dozen new people to interact with.

If it doesn’t come easily, practice practice practice. Try to get out of your own head space, and hear what troubles your client, and what big business challenges worry them.

Be ready to help but don’t try to be the hero

A decade ago I worked for an Internet startup. They were having serious performance problems which was slowing down the site, and turning users away. When digging into the systems I found serious security issues besides the performance ones, and got distracted trying to wrap up those lest someone break in and destroy or steal their business assets. Communicating the situation to the client, they looked aghast. After explaining the situation to them, they understood the risks and explained that the current priorities were to get users back online.

The technical problems I saw may not have been aligned with the business priorities. Your job is to make your client happy. Provide your professional opinion and advice whenever and wherever your skills come into play, but let them run their own business.

If you’re focusing on one area, and you discover other problems or things that may need resolving going forward, bring this to the attention of the client. Allow them to prioritize for themselves. It’s their business not yours. Your job is to give your professional opinion, raise concerns that you see, but most importantly solve problems they want you to solve.

Project Your Personality

Smile a lot and listen to people. Make sure you’re talking less than half the time. When you first engage with a client, they should be speaking more like two-thirds of the time. You want to get in the habit of listening, and stepping in your clients shoes. You want to understand their pain, their business concerns and how to satisfy them.

Manage Time Efficiently

Get things done. Everybody talks about it, but not everyone does it. I personally avoid all the faddish tools for this, and use a simple checklist. Focus on the task at hand. Give yourself a doable list of tasks each day, and check them off as you go. Try hard to avoid working on things not on that list. The last point relates back to the principle of solving only the problems that you’ve been asked to solve.

Communicate Successes & Progress

In many engagements you’ll come upon struggles and get blocked by situations that seem intransigent. I can’t emphasize enough how important it is to communicate with the client during these situations. Don’t get stuck thinking it will make you look weak. Communicating with the client has a number of surprising advantages.

For one sometimes they’ll have a solution, such as a different angle on the business problem, or insight and details that just simplify the problem you think you thought needed to be solved.

Second, it allows the client to adjust schedules in advance if something will take a little longer. You’d be surprised how often a client will sympathize with a difficult problem.

Lastly, involving the client intimately allows them to enjoy the triumph when you solve the problem. This helps morale, communicates more about what it is you do day-to-day and how you work through a problem. And overall it helps them appreciate the intrinsic value you’re providing.

For more articles like these go to iHeavy, Inc +1-212-533-6828

]]> http://www.iheavy.com/2012/05/15/consulting-essentials-managing-completing-engagements/feed/ 0 http://www.iheavy.com/2012/05/10/tech-consulting-essentials-getting-the-business/ http://www.iheavy.com/2012/05/10/tech-consulting-essentials-getting-the-business/#comments Thu, 10 May 2012 11:59:20 +0000 Sean Hull http://www.iheavy.com/?p=4312

Read the original article at Consulting essentials: Getting the business

Over the years, a lot of people have approached me asking how to become a tech consultant. What do I need to do to get started? How can I take my first step? I also hear from managers and CEOs that have asked how I got my start, and how I keep the business running. [...]

For more articles like these go to iHeavy, Inc +1-212-533-6828

]]> Read the original article at Consulting essentials: Getting the business

Over the years, a lot of people have approached me asking how to become a tech consultant. What do I need to do to get started? How can I take my first step?

I also hear from managers and CEOs that have asked how I got my start, and how I keep the business running. What lessons from consulting can be applied to startups and small businesses? Having worked independently for many years I’ve built up my own cache of strategies and methods which I hope can be helpful to anyone looking to strike it out on their own.

This is the first of a series of three articles on consulting essentials.

Networking to avoid the feast or famine cycle

If no one knows about your services, it doesn’t matter how good you are, you won’t have regular work. Some consultants get involved in a regular full-time engagement, and after a year or more experience a lot of down time when that gig ends.

Want to avoid the feast or famine cycle? Get out there and meet people. Go to meetups, and exchange cards. Go to startup and venture events such as Ultralight or The Hatchery. Participate in General Assembly  and related incubators. Start a newsletter, and say something interesting and useful every month. Add every professional colleague you encounter to your list. A newsletter is an invaluable way to keep in touch with folks in your professional world, and the value keeps growing over time.

Build your network, connect people together and keep in contact with people. This makes you a focal point, and that will bring business to you.

Proposals & Meetings – Always be Closing

1. Take many meetings and interviews

If you’re drawing a wide net, you should have tons of opportunities to meet prospects. Sometimes these may be framed as interviews, while other times a meeting of peers. Ultimately as a professional services provider, you should think of it as a partnership where each party contributes to solve a business problem. Hone your speaking skills, compliment people, smile, listen carefully to people’s concerns and try to frame answers in language that works uniquely for each person.

2. Talk with Economic Buyers

In order to get hired for your services, ultimately you’ll need to be talking with the person who manages the budget for such hiring. However often there are gatekeepers or even human relations folks between you and that person. If you’re being asked for a resume you’re probably going down the wrong road. Typically a stake holder will ask you about solutions you’ve provided for other customers, how you executed, and what were some of the highlights. They won’t be giving you a test, or asking you to solve a riddle or coding problem.

3. Send quotes

If you are active in business, networking regularly and reaching out to new contacts and colleagues then you should find many asking for advise and expertise. Always be sending out quotes. Sending out a quote is not a do or die arrangement. You should be sending out many many more than you expect to win. By continuing to pitch, you test the market for the demand of your skills, package your services in new and creative ways, and frame solutions that should hopefully resonate with your prospects.

4. Cherry Pick the Best Clients

All clients are not created equal. Some are trying to get a lot for less than market value, and will try hard to micromanage and squeeze value out of you. This isn’t personal. Business is about maximizing value for a given price. It’s important though to qualify the client, make sure they have the budget to be worth your while, and do your due diligence. Better to have a half dozen to a dozen projects in one year, than fifty tiny projects.

5. Give Audience to fulltime offers

If you’re good at consulting, you’ll gain a lot of experience, exposure to many types of businesses, many technologies, and many environments. This is valuable experience, and business owners and startups will try to hire you. As part of the *listen* part, it’s always ok to hear out a prospect. Leave the “door open” as they say. Provide real explanation for what appeals to you, from independence, freedom of schedule, choosing projects, and compensation. Creative discussion about budgets and pricing options can well turn such a discussion into how you might work together on a situational or project basis.

Although independent consulting isn’t for everyone for those who’ve made a career out of it, taking a fulltime opportunity is a step away from that freedom unless you’re making a deliberate career change. Doing this can be a signal you don’t have a full stream of work, that you’re experiencing the famine cycle, and you’re looking for safety and security.

Consulting services cost more hour per hour than a fulltime resource. But typically you work in shorter bursts or stints. What’s more your availability is always a business value. Lastly because you can ride the crest, stay in demand, stay relevant and stay fresh, you’ll continue to command top dollar.

Price Your Services Properly

I’ve met many freelancers over the years who struggle to get paid, or struggle with cash flow. If this is happening you’re either not keeping the pipeline full or not charging sufficiently for your services. What exact price is right for you of course will vary. My personal recommended method is to research the large firms in your services space. Find out what their junior and senior folks are billed out at. These firms of course have a lot of overhead, so 50% of those rates should be a good base number to work around. Furthermore these calculations can be used in discussions with prospects to sell your services. Remind prospects that they’ll be getting the same professional experience, more customer focus because you’re a one man shop, and pay half the price for it. A proverbial win-win.

1. Discount by Reducing Value

Beware taking too much pro-bono work, or giving discounts at the outset of an assignment. This signals to the savvy firm that you’re keen to close the deal, and will probably negotiate further. You’ll have little room to renegotiate rates higher, once the engagement has begun. You’ve already set a precedent.

So if a firm asks for a discount, be sure to reduce the entire package in some way. Total number of hours, on-call coverage, amount of software or widgets delivered etc.

2. Experiment with Different Billing Models

Hourly billing is not the only model in town. If you’re finding it’s a lot of work to keep track of hours, or you and your client are staying unduly focused on these details, consider other models. Pricing by the day may work, for example. Further options may include billing by the week or even by the month. Chances are you’ll want to discount somewhat for larger incremental steps in pricing. Keep in mind that with monthly billing for example, your project cannot end on June 10th or July 5th. It will always roll forward to the end of a month. So offering a discount for larger billing periods may even out in the end.

In the next post: How to manage and complete engagements.

For more articles like these go to iHeavy, Inc +1-212-533-6828

]]> http://www.iheavy.com/2012/05/10/tech-consulting-essentials-getting-the-business/feed/ 1 http://www.iheavy.com/2012/05/08/3-things-ceos-should-know-about-the-cloud-2/ http://www.iheavy.com/2012/05/08/3-things-ceos-should-know-about-the-cloud-2/#comments Tue, 08 May 2012 16:44:04 +0000 Sean Hull http://www.iheavy.com/?p=4302

Read the original article at 3 things CEOs should know about the Cloud

You’ve heard all the buzz and spiel about the cloud, and there’re good reasons to want to get there. On-demand compute power makes new levels of scalability possible. Low up front costs means moving capital expenditure to operating expenditure and saving a bundle in the process. We won’t give you anymore of the rah rah [...]

For more articles like these go to iHeavy, Inc +1-212-533-6828

]]> Read the original article at 3 things CEOs should know about the Cloud

You’ve heard all the buzz and spiel about the cloud, and there’re good reasons to want to get there. On-demand compute power makes new levels of scalability possible. Low up front costs means moving capital expenditure to operating expenditure and saving a bundle in the process. We won’t give you anymore of the rah rah marketing hoopla. You’ve heard enough of that. We’ll gently play devil’s advocate for a moment, and give you a few things to think about when deploying applications with a cloud provider. Our focus is mainly on Amazon EC2.

You might also be interested in a wide reaching introduction to deploying on Amazon EC2.

1. Funky Performance

One of the biggest hurdles we see clients struggle with on Amazon EC2 is performance. This is rooted in the nature of shared resources. Computer servers, just like desktops rely on CPUs, Memory, Network and Disk. In the virtual datacenter, you can be given more than your fair share without you even knowing it. More bandwidth, more CPU, more disk? Who would complain? Well if your application behaves erratically, while you suddenly compete for disk resources you’ll quickly feel the flip side of that coin. Stocks go up, and they can just as easily come right back down.

Variability around disk I/O seems to be the one that hits applications the hardest, especially the database tier of many web applications. If your application requires extremely high database transaction throughput, you would do well to consider physical servers and a real RAID array to host your database server. Read more about IOPs

2. Uncertain Reliability – A Loaded Gun

Everybody has heard the saying, don’t hand someone a loaded gun. In the case of Amazon servers, you really do load your applications onto fickle and neurotic servers.

Imagine you open a car rental business. You could have two brand new fully reliable cars to rent out to customers. Your customers would be very happy, but you’d have a very small business. Alternatively you could have twenty used Pintos. You’d have some breaking down a lot, but as long as you keep ten of them rented at a time, your business is booming.

In the Amazon world you have all the tools to keep your Ford Pintos running, but it’s important to think long and hard about reliability, redundancy, and automation. Read more about Failures, Lessons & the Chaos Monkey

3. Iffy Support

Managed hosting providers vary drastically in terms of the support you can expect. Companies like Rackspace, Servint or Datapipe have Support built into their DNA. They’ve grown up around having a support tech that your team can reach when they’re having trouble.

Amazon takes the opposite approach. They give you all the tools to do everything yourself. But in a crunch it can be great to have that service available to help troubleshoot and diagnose a problem. Although they’re now offering support contracts, it’s not how they started out.

If you have a crack operations team at your disposal, or you hire a third party provider like Heavyweight Internet Group Amazon Web Services gives you the flexibility and power to build phenomenal and scalable architectures. But if you’re a very small team without tons of technical know-how, you may well do better with a service-oriented provider like Rackspace et al.

A few more considerations…

• Will your cloud provider go out of business?
• Could a Subpoena against your provider draw you into the net?
• Since you don’t know where your sensitive data is, should you consider encryption?
• Should I keep additional backups outside of the cloud?
• Should I use multiple cloud providers?
• Should I be concerned about the lack of perimeter security?

For more articles like these go to iHeavy, Inc +1-212-533-6828

]]> http://www.iheavy.com/2012/05/08/3-things-ceos-should-know-about-the-cloud-2/feed/ 0 http://www.iheavy.com/2012/05/03/ten-things-remember-mysql-backups/ http://www.iheavy.com/2012/05/03/ten-things-remember-mysql-backups/#comments Thu, 03 May 2012 21:33:27 +0000 Sean Hull http://www.iheavy.com/?p=4260

Read the original article at Ten things to remember about MySQL backups

Use Hot Backups Hot backups are an excellent way to backup MySQL.  They can run without blocking your application, and save tons on restore time.  Percona’s xtrabackup tool is a great way to do this.  We wrote a how-to on using xtrabackup for hotbackups. Use Logical Backups Just because we love hot backups using xtrabackup [...]

For more articles like these go to iHeavy, Inc +1-212-533-6828

]]> Read the original article at Ten things to remember about MySQL backups

1. Use Hot Backups

Hot backups are an excellent way to backup MySQL.  They can run without blocking your application, and save tons on restore time.  Percona’s xtrabackup tool is a great way to do this.  We wrote a how-to on using xtrabackup for hotbackups.

2. Use Logical Backups

Just because we love hot backups using xtrabackup doesn’t mean mysqldump isn’t useful.  Want to load data into Amazon RDS?  Want to isolate and load only one schema, or just one table?  All these great uses make mysqldump indispensable.  Use it in combination with periodic hot backups to give you more recovery options.

3. Replication isn’t a backup

While replication provides a great way to keep a hot copy of your production database, it’s not the same as a backup.  Why?  Operator error, that’s why!  People make mistakes, drop tables and database schemas that later need to be restored.  This can and will happen, so head off the disaster by doing real backups.

As an additional note, if you’re using replication, you surely want to perform regular checksums of your data.  These ensure that the primary and secondary do indeed contain the same data.

4. Firedrills & Restore Time

The only way to be sure your backup is complete is to test restoring everything.  Yes it’s a pain, but it will inevitably be a learning experience.  You’ll document the process to speed it up in future tests, you’ll learn how long recovery takes, and find additional pieces to the pie that must be kept in place.  Doing this in advance of d-day is

Different backups have different recovery times.  In the industry vernacular, your RTO or recovery time objective should inform what will work for you.  Although a mysqldump may take 30 minutes to complete, your restore of that data might take 8 hours or more.  That’s due in part to rebuilding all those indexes.  When you perform the dump one create index statement is formulated from the data dictionary, but on import the data must be sorted and organized to rebuild the index from scratch.  Percona’s mysqldump utility will capitalize on MySQL’s fast index rebuild for Innodb tables.  According to the Percona guys this can bring a big improvement in import time.  Yet another great reason to use the Percona distro!

5. Transaction Logs

If you want to be able to do point in time recovery, you’ll need all the binlog files as well.  These are being created all the time, while new transactions are completed in your database. If your last backup was last night at 3am, and you want to recovery today until 3pm, you’ll need all the binary logs from the intervening hours to apply to that backup.  This process is called point-in-time recovery, and can bring your database restore up to the current commited transactions.

6. Backup Config Files

Don’t forget that lonely /etc/my.cnf file.  That’s an important part of a backup if you’re rebuilding on a newly built server.  It may not need to be backed up with the same frequency, but should be included.

7. Stored Code & Grants

Stored procedures, triggers and functions are all stored in the mysql database schema.  If you are doing a restore of just one database schema, you may not have this, or it may make the restore more complicated.  So it can be a good idea to backup code separately.  mysqldump can do this with the –routines option.  Hot backups by their nature, will capture everything in the entire instance – that is all database schemas including the system ones.

Grants are another thing you may want to backup separately.  For the same reasons as stored code, grants are stored in the system tables.  Percona toolkit includes a nice tool for this called pt-show-grants.  We recommend running this periodically anyway, as it’ll give you some perspective on permissions granted in your database.  You’re reviewing those right?

8. Events & Cronjobs

MySQL allows the running of events inside the database.  SHOW EVENTS or SHOW EVENTS schema_name will display the events scheduled.

You may also have cronjobs enabled.  Use crontab -l to display those for specific users.  Be sure to check at least “mysql” and “root” users as well as other possible application users on the server.

9. Monitoring

Backups are a nit picky job, and often you don’t know if they’re complete until it’s time to restore.  That’s why we recommend firedrills above, and they’re very important.  You can also monitor the backups themselves.  Use an error log with mysqldump or xtrabackup, and check that logfile for new messages.  In addition you can check the size of the resulting backup file.  If it has changed measurably from the recent backup sizes, it may indicate problems.  Is your backup size 0, something serious is wrong.  Half the size of recent ones, it may have failed halfway through, or the filesystem filled up.

10. Security

This is often overlooked area, but may be a concern for some environments.  Is the data contained in your backup sensitive?  Consider where the backups are stored and retained for long term.  Reason who has access to those files, and make use of the least privileges rule.

For more articles like these go to iHeavy, Inc +1-212-533-6828

]]> http://www.iheavy.com/2012/05/03/ten-things-remember-mysql-backups/feed/ 7 http://www.iheavy.com/2012/04/30/tyranny-of-a-google-vote-2/ http://www.iheavy.com/2012/04/30/tyranny-of-a-google-vote-2/#comments Mon, 30 Apr 2012 11:15:11 +0000 Sean Hull http://www.iheavy.com/?p=4166

Read the original article at Tyranny of a Google vote

For the past year I’ve been seeing headline blogs analyzing the effect of Google’s last algorithm update, dubbed the Panda. There was much talk of unfair relegation from the first page of Google search results, and general indignance by the SEO community. As with any subject in which I only have cursory knowledge I didn’t [...]

For more articles like these go to iHeavy, Inc +1-212-533-6828

]]> Read the original article at Tyranny of a Google vote

Image by Hajo de Reijger, politicallyillustrated.com

For the past year I’ve been seeing headline blogs analyzing the effect of Google’s last algorithm update, dubbed the Panda. There was much talk of unfair relegation from the first page of Google search results, and general indignance by the SEO community.

As with any subject in which I only have cursory knowledge I didn’t think much of it. I thought that as long as I didn’t engage in link-buying and whatever is known as “black hat” tactics, the search engines would be fair. What I didn’t realise with Google was how subjective it has become in ranking websites. I was particularly tripped up in the area of duplicate content.

Some of my articles are syndicated to DZone.com, a hub for tech bloggers. They’d approached me about a year ago asking if I’d like them to carry our content. It seemed like a good way to gain visibility so I agreed.

Recently, I ran a Google search on the actual content that was syndicated and found the following results:

Title	Dzone rank	iheavy rank
Zero Downtime – What is it?	#21	(not in first 5 pages)
Deploying MySQL on EC2	#1	(not in first 5 pages)
Cloud Computing Use Cases	#5	(not in first 5 pages)

Now DZone.com’s pagerank is a 6 while iheavy.com is a 3. Google’s algorithm is probably weighing the pagerank of Dzone higher, and serving up those results at the expense of the original. It could be that the algorithms can’t determine which is original but Google’s bots know full well when content is published, so it knows the iheavy.com content was created before.

More likely it cannot supersede the ranking algorithm. It is one based on popularity.

What’s Original Content?

All of this raises the question of the value of original content. In this case we’ve given this site permission to carry our content, not knowing whether that would be good or bad for us. Nevertheless, it does seem to go against common sense, and perhaps what Internet users intuition might tell them, that they were not clicking through to the original creator of some content.

Can Google’s Algorithm Discern Original Work? Is there an incentive to do so?

The update brings up interesting questions about Google’s ranking algorithm. In a world where the popularity indicator is given the highest weight, will we find what we are looking for? In my case, I write articles related to my area of expertise, which is on web architecture, scalability and general tech consulting matters. Of course I want people to find my site when they’re looking for solutions to problems relevant to them. Yet in Google’s calculations, popularity trumps provenance.

Many bigger sites are experiencing the same thing and at an even larger scale. According to SEO Moz, Panda is forcing a change onto the role of SEOs, turning them into that of web strategists. While traditional SEO methods of optimizing for keywords, and putting out quality content still count, design and user experience, shareability, likeability; what’s known as “signals” that could predict the site’s popularity, are influencing the overall results of your site.

As an independent business owner with limited resources my time off hours is invested in writing better articles that appeal to people searching for MySQL or scalability consulting and less about putting in the bells and whistles to raise site popularity. But if that’s what Google favors then I’ll probably have to rethink my approach.

As much as I can squeeze out of a busy schedule, there’s not a high chance that this website can surpass a giant such as Dzone in popularity.

The Internet is meant to be a place where the pint-sized can have a fair chance at making an impression. With the way search algorithms have evolved, things are looking more like a reality TV talent contest where skill alone without good looks and a nice smile are just not enough to win the popular vote.

For more articles like these go to iHeavy, Inc +1-212-533-6828

]]> http://www.iheavy.com/2012/04/30/tyranny-of-a-google-vote-2/feed/ 5 http://www.iheavy.com/2012/04/27/the-power-of-habit/ http://www.iheavy.com/2012/04/27/the-power-of-habit/#comments Fri, 27 Apr 2012 08:01:49 +0000 Sean Hull http://www.iheavy.com/?p=4151

Read the original article at The Power of Habit by Charles Duhigg

Habits. We all have them. The good ones we celebrate, but the bad ones we struggle with. Duhigg’s book may introduce some ideas to those of us less familiar with behavioral sciences but it fails to effectively teach us how to form good habits and break the bad ones. Filled with pages of stories from [...]

For more articles like these go to iHeavy, Inc +1-212-533-6828

]]> Read the original article at The Power of Habit by Charles Duhigg

Habits. We all have them. The good ones we celebrate, but the bad ones we struggle with. Duhigg’s book may introduce some ideas to those of us less familiar with behavioral sciences but it fails to effectively teach us how to form good habits and break the bad ones.

Filled with pages of stories from successful brands such as Pepsodent which Duhigg credits for turning the brushing of teeth into a daily routine; and perhaps more tenuous ones about leaders such as Paul O’ Neill, the CEO of Alcoa who purportedly turned around the fortunes of an ailing organisation by changing its safety practices.

From cue, routine to reward we must first identify the habit, then in a way that parallels the success of Alcoholic Anonymous, you replace the routine, keeping the cue and reward. In discussing the success of AA and others, he brings up the importance of belief in long term success of habit change. He references William James’ famous quote “Believe that life is worth living, and your belief will help create that fact”.

Still, I couldn’t help thinking that for the average business manager it lacked actionable advice of the kind you might find in a Jim Collins Good to Great or Chip Conley’s Peak. These books also have excellent story telling, but break things down in a very specific set of steps and attributes that an organization or individual can apply today.

Duhigg’s writing is easy to read and that’s probably the book’s greatest strength. Yet with most of it grounded more in interesting anecdotes than credible research, the examples unfortunately give for more entertaining reading than any deep insight.

For more articles like these go to iHeavy, Inc +1-212-533-6828

]]> http://www.iheavy.com/2012/04/27/the-power-of-habit/feed/ 0 http://www.iheavy.com/2012/04/26/bulletproofing-mysql-replications-with-checksums/ http://www.iheavy.com/2012/04/26/bulletproofing-mysql-replications-with-checksums/#comments Thu, 26 Apr 2012 10:33:03 +0000 Sean Hull http://www.iheavy.com/?p=4140

Read the original article at Bulletproofing MySQL replication with checksums

Your MySQL replications running well? You might not even know if they aren’t. One of the scariest things about MySQL replication is that it can drift out of sync with the master “silently”. No errors, no warnings. What and Why? MySQL’s replication solution evolved as a statement based technology. Instead of sending actual block changes, [...]

For more articles like these go to iHeavy, Inc +1-212-533-6828

]]> Read the original article at Bulletproofing MySQL replication with checksums

Your MySQL replications running well? You might not even know if they aren’t. One of the scariest things about MySQL replication is that it can drift out of sync with the master “silently”. No errors, no warnings.

1. What and Why?

MySQL’s replication solution evolved as a statement based technology. Instead of sending actual block changes, MySQL just has to log committed transactions, and reapply those on the slave side. This affords a wonderful array of topologies and different uses, but has it’s drawbacks. The biggest occur when data does not get updated or changed in the same way on the slave. If you’re new to MySQL or coming from the Oracle world you might expect that this would flag an error. But there are many scenarios in which MySQL will not flag an error:

• mixed transactional and non-transactional tables
• use of non-deterministic functions such as uuid()
• stored procedures and functions
• update with LIMIT clause

There are others but suffice it to say if you want to rely on your slave being consistent, you need to check it!

2. The solution – mathematical checksums

If you’re a seasoned Linux user, you’re probably familiar with the md5sum command. It creates a checksum on a file. You can do so on different servers to compare a file in a mathematically exact way. In fact rsync uses this technique to efficiently determine what files or pieces of files need to be copied across a network. That’s what makes it so fast!

It turns out that MySQL can checksum tables too. However were we to build our own solution, we might have trouble doing so manually as table data is constantly in a state of flux.

Enter Percona’s pt-table-checksum tool formerly part of Maatkit. Run it periodically against your master schemas or the entire instance if you like. It will store checksums of all of your tables in a special checksum table. The data from this table then will propagate through replication to all of your connected slaves.

The tool then has a check mode, which allows you to verify all the connected slaves are ok, or report the differences if it finds any.

3. Step-by-step Setup

First you’ll need to grab a copy of the percona toolkit. Note that if you previously installed maatkit then you may want to delete those old scripts to avoid confusion. mk-table-checksum if you used maatkit, or pt-table-checksum if you have 1.0 versions. You likely installed using wget or perl Makefile, so you may need to go and remove those manually.

Assuming you’ve already got the percona repository installed issue:

$ yum install -y percona-toolkit

I’ve found some of the maatkit tools to be rather fussy about getting all the options right. The first thing to do which will help simplify this is to add a section in your local user’s “.my.cnf” file like this:

[client]

user=root

password=myrootpassword


That way the percona tools will look for this whenever it needs authentication credentials. Otherwise we assume localhost for this example, so you should verify you can connect with the mysql client as root from localhost.

Now let's checksum the "mysql" system schema.

$ pt-table-checksum --replicate=test.checksum --create-replicate-table --databases=mysql localhost

Note the --create-replicate-table option. You only need this option the first time. From there the test.checksum table will exist.

You should see some output that looks like this:

TS ERRORS DIFFS ROWS CHUNKS SKIPPED TIME TABLE

04-24T16:06:45 0 0 0 1 0 0.099 mysql.columns_priv

04-24T16:06:45 0 0 32 1 0 0.100 mysql.db

04-24T16:06:45 0 0 0 1 0 0.096 mysql.event

04-24T16:06:45 0 0 0 1 0 0.096 mysql.func

04-24T16:06:45 0 0 38 1 0 0.102 mysql.help_category

04-24T16:06:45 0 0 452 1 0 0.106 mysql.help_keyword

04-24T16:06:46 0 0 993 1 0 0.096 mysql.help_relation

04-24T16:06:46 0 0 506 1 0 0.100 mysql.help_topic

04-24T16:06:46 0 0 0 1 0 0.099 mysql.host

04-24T16:06:46 0 0 0 1 0 0.104 mysql.ndb_binlog_index

04-24T16:06:46 0 0 0 1 0 0.107 mysql.plugin

04-24T16:06:46 0 1 1 1 0 0.115 mysql.proc

04-24T16:06:46 0 0 0 1 0 0.186 mysql.procs_priv

04-24T16:06:46 0 1 1 1 0 0.097 mysql.proxies_priv

04-24T16:06:47 0 0 0 1 0 0.097 mysql.servers

04-24T16:06:47 0 0 0 1 0 0.096 mysql.tables_priv

04-24T16:06:47 0 0 0 1 0 0.098 mysql.time_zone

04-24T16:06:47 0 0 0 1 0 0.097 mysql.time_zone_leap_second

04-24T16:06:47 0 0 0 1 0 0.100 mysql.time_zone_name

04-24T16:06:47 0 0 0 1 0 0.100 mysql.time_zone_transition

04-24T16:06:47 0 0 0 1 0 0.095 mysql.time_zone_transition_type

04-24T16:06:47 0 1 38 1 0 0.100 mysql.user

4. How to check slaves

Once you've collected all those fancy checksums for your tables, nicely timestamped, you'll want to verify that your slaves are happily in sync. You can do that with the following command, also on the master:

$ pt-table-checksum --replicate=test.checksum --replicate-check-only --databases=mysql localhost

If there's no differences you'll see no output. If you have a difference it'll look something like this:

Differences on ip-10-15-27-19

TABLE CHUNK CNT_DIFF CRC_DIFF CHUNK_INDEX LOWER_BOUNDARY UPPER_BOUNDARY

mysql.user 1 1 1

In our case you can see we created some users on the slaves accidentally, hence the differences. It illustrates how easy it is for differences to creep into your environment and also how easy it now is to find them!

5. Special Cases

Since one of my clients uses Drupal, they've had trouble replicating the semaphore table. This tables is a MyISAM table, and unfortunately no one dares convert it to InnoDB. So from time to time some gunk builds up in there, and it fails on the slave. We could clean out the table, but we decided to just filter out this one table. Since Drupal doesn't use fully qualified schema.table names in it's code, only "use" we have found this to be safe.

However the percona toolkit explicitely checks for replication filters and will not run. It'll stop with an error as follows:

$ pt-table-checksum --replicate=test.checksum --databases=sean --ignore-tables=semaphore localhost

04-24T15:59:29 Replication filters are set on these hosts:

ip-10.15.27.19

replicate_ignore_table = sean.semaphore

ip-10-15-27-72

replicate_ignore_table = sean.semaphore

ip-10-15-27-18

replicate_ignore_table = sean.semaphore

Please read the --check-replication-filters documentation to learn how to solve this problem. at /usr/bin/pt-table-checksum line 6166.

The solution is the --nocheck-replication-filters option. Keep in mind that this sanity check is there for a reason, so be sure to skip the relevant tables in your checksum building, and checksum checks.

To build checksums skipping the semaphore table use this command:

$ pt-table-checksum --replicate=test.checksum --ignore-tables=prod.semaphore --nocheck-replication-filters localhost

Now you can check your slaves but ignore the semaphore table:


$ pt-table-checksum --replicate=test.checksum --replicate-check-only --ignore-tables=prod.semaphore --nocheck-replication-filters localhost

We also found a bug which preventing us from specifying multiple tabes on the ignore-tables line. So we used multiple invocations to do different schemas like this:

$ pt-table-checksum --replicate=test.checksum --replicate-check-only --databases=prod1 --ignore-tables=prod1.semaphore --nocheck-replication-filters localhost

$ pt-table-checksum --replicate=test.checksum --replicate-check-only --databases=prod2 --ignore-tables=prod2.semaphore --nocheck-replication-filters localhost

Crash Protection

If you've used MySQL replication for any length of time, you've probably seen a server crash. MySQL replication can have trouble restarting if you're using temporary tables, as they'll be missing upon restart. Also MySQL before 5.5 leaves syncing the info files to the operating system. So they may be incorrect after a crash.

1. Use MySQL 5.5 if possible

There are some new parameters in 5.5 that protect the info files from a crash. These are a great addition, and will make your slave databases more bulletproof.

sync_master_info = 1

sync_relay_log = 1

sync_relay_log_info = 1

2. Don't use temporary tables

After a restart they'll simply be gone, so queries requiring or running against them will fail.

The Percona guys' new book High Performance MySQL third edition, suggests an alternate solution to using temporary tables. Use a special schema to hold your temp data, but create them as normal permanent tables. Be sure your application creates them with unique names, using the connection_id() for example. Lastly have a cleanup process drop tables periodically, based on closed connection_ids.

For more articles like these go to iHeavy, Inc +1-212-533-6828

As a tech consultant one of the most interesting parts of the job is being able to observe human relations at work. I’ve learned through the years that because tech people and non-tech people speak different ‘languages’, bridging the communication gap is a critical part of my role as a consultant.

Sometimes the relationship between tech and other business units is less sweet. The typical complaints are that IT guys are always denying requests, aloof and even downright unhelpful.

On the other side, the geeks feel frustrated that people “just don’t listen”. We remind people to always use strong passwords, and people still make “password” their password. We train end-users and give specific guidance and instructions but they still commit the fundamental mistakes. Meanwhile, the managers expect IT staff to perform miracles.

So who can be blamed when this animosity exists? Geeks for their arrogance? Or end-users for not making an effort to improve their understanding of tech concepts? Perhaps both sides can share the blame. But as tech folks we can try to make things better by working on our communication skills. Sliding into aloofness will not only make people resentful but suspicious of our motives too. Don’t be that guy!

If you’ve found yourself slipping up in the people-skills department, here are a few tips that can help you along. Think of understanding and persuading other people as a puzzle equally complex as the biggest engineering challenge. In that light you can look at it as an ongoing project to improve your communication and charm.

For more articles like these go to iHeavy, Inc +1-212-533-6828

MySQL replication technology is powerful and flexible. But it doesn’t do everything perfectly all the time. You may experience trouble with the slaves falling behind the master, or want to scale horizontally by building new slaves automatically. Or you might need to build a slave without blocking the master database.

All of these goals can be achieved using some powerful tools. Here’s a quick guide to those tools and how to use them.

For more articles like these go to iHeavy, Inc +1-212-533-6828

Setting up replication in MySQL is something we need to do quite often. Slaves die, replication fails, or tables and data get out of sync. Whenever we build a slave, we must start with a snapshot of all the data from the master database.

MySQLdump is the tried and true method of doing this, however it requires that we lock all the tables in the database. If we’re dumping a large database, this could be a significant period, where no writing can happen to our database for the duration of the backup. For many environments read-only is still an outage.

Enter hotbackups to the rescue. Percona comes with a tool that allows you to perform hotbackups of a running MySQL database, with no blocking. It’s able to do this because of Innodb & multi-version concurrency control (MVCC). Luckily we don’t need to dig into the guts to enjoy the benefits of this great technology.

Here’s a quick step-by-step guide to using xtrabackup to create a slave.

For more articles like these go to iHeavy, Inc +1-212-533-6828

I first read about Heinrichs in a Bloomberg Businessweek piece on him. He’s quite a character, with high profile clients like Ogilvy & Mather and the Pentagon. Struck by some of the ideas in there, I decided to pickup Thank You for Arguing.

48 laws of soft power

Compiled into 25 very readable chapters, he persuades the reader by illustrating how to win trust through managing your own volume control for positive affect, verbal jousting and calling fouls, and mastering timing. Sure in the real world this is all going to require a lot of trial and error, and practice in the trenches. But his book serves as a very good guide along the way.

Don’t worry too much about Aristotle, Cicero or the classics you never learned in school. If anything they serve as a colorful highlight to his useful everyday illustrations.

Some examples worth recalling are:

He has one whole chapter on Bushisms, which I found intriguing. He explains that Bush uses code grooming to very strong effect. When speaking to different groups, he emphasizes these code words in his sentences. With women, words like “I understand”, “peace”, “security” and “protecting”. With a military group words such as “never relent”, “we must not waver” and “not on my watch” were common. For religious audiences, “I believe” resonated strongly. He quotes a superb Bushism which in this light suddenly begins to sound powerful:

“I know what I believe. I will continue to articulate what I believe and what I believe–I believe what I believe is right.”

Rhetoric indeed. I’ll be studying this book for months to come!

For more articles like these go to iHeavy, Inc +1-212-533-6828

Autoscaling your webserver tier is typically straightforward. Image your apache server with source code or without, then sync down files from S3 upon spinup. Roll that image into the autoscale configuration and you’re all set.


With the database tier though, things can be a bit tricky. The typical configuration we see is to have a single master database where your application writes. But scaling out or horizontally on Amazon EC2 should be as easy as adding more slaves, right? Why not automate that process?

Below we’ve set out to answer some of the questions you’re likely to face when setting up slaves against your master. We’ve included instructions on building an AMI that automatically spins up as a slave. Fancy!

For more articles like these go to iHeavy, Inc +1-212-533-6828

In the Internet world 24×7 has become the defacto standard. Websites must be always on, available 24 hours a day, 365 days a year. In our pursuit of perfection, performance is being measured down to three decimal places, that is being up 99.999% of the time; in short, five-nines

Just like a mantra, when repeated enough it becomes second nature and we don’t give the idea a second thought. We don’t stop to consider that while it may be generally a good thing to have, is five-nines necessary and is it realistic for the business?

In my dealings with small businesses, I’ve found that the ones that have been around longer, and with more seasoned managers tend to take a more flexible and pragmatic view of the five-nines standard. Some even feel that periods of outages during off hours as – *gasp* – no problem at all! On the other hand it is a universal truth held by the next-big-idea startups that 24×7 is do or die. To them, a slight interruption in service will send the wrong signal to customers.

The sense I get is that businesses that have been around longer have more faith in their customers and are confident about what their customers want and how to deliver it.  Meanwhile startups who are building a customer base feel the need to make an impression and are thus more sensitive to perceived limitations in their service.

Of course the type of business you run might well inform your policy here. Short outages in payments and e-commerce sites could translate into lost revenue while perhaps a mobile game company might have a little more room to breathe.

Sustaining five nines is too expensive for some

The truth is sustaining high availability at the standard of five-nines costs a lot of money. These costs are incurred from buying more servers, whether as physical infrastructure or in the cloud. In addition you’ll likely involve more software components and configuration complexity. And here’s a hard truth, with all that complexity also comes more risk.  More moving parts means more components that can fail. Those additional components can fail from bugs, misconfiguration, or interoperability issues.

What’s more, pushing for that marginal 0.009% increase in high availability means you’ll require more people and create more processes.

Complex architecture downtime

In a client engagement back in 2011, I worked with a firm in the online education space.  Their architecture was quite complex.  Although they had web servers and database servers—the standard internet stack—they did not have standardized operations.  So they had the Apache web server on some boxes, and Nginx on others.  What’s more they had different versions of each as well as different distributions of Linux, from Ubuntu to RedHat Enterprise Edition.  On the database side they had instances on various boxes, and since they weren’t all centralized they were not all being backed up.  During one simple maintenance operation, a couple of configurations were rearranged, bringing the site down and blocking e-commerce transactions for over an hour.  It wasn’t a failure of technology but a failure of people and processes made worse by the hazard of an overly complex infrastructure.

In another engagement at a financial media firm, I worked closely with the CTO outlining how we could architect an absolutely zero downtime infrastructure.  When he warned that “We have no room for *ANY* downtime,” alarm bells were ringing in my head already.

When I hear talk of five-nines, I hear marketing rhetoric, not real-world risk reduction.   Take for example the power grid outage that hit the Northeast in 2003.  That took out power from large swaths of the country for over 24 hours.  In real terms that means anyone hosted in the Northeast failed five-nines miserably because downtime for 24 hours would be almost 300 years of downtime at the five-nines standard!

For true High Availability look at better management of processes

So what can we do in the real-world to improve availability?  Some of the biggest impacts will come from reducing so-called operator error, and mistakes of people and processes.

Before you think of aiming for five-nines,  first ask some of these questions:

If you’re thinking about and answering these questions you’re well on your way to improving availability and uptime.

For more articles like these go to iHeavy, Inc +1-212-533-6828

I’ve been looking for this book for a long time. Or maybe I should rephrase that. I was pondering this topic and had been looking for a book which covered it for a while so I was pleased to come across The Filter Bubble.

Digging into search engine optimization and analytics while building my own website, I was often confused by inconsistent Google search results. Realizing I was on a different computer, or I was logged into Google services I would logout to see the untainted results, the results everyone else was seeing. Or was I?

As Google+ personalization launched the topic of search really piqued my interest. Why had I been given different results at different times? The coverage on Gigaom, AllThingsD, TechCrunch and ReadWriteWeb cautioned that this could be the turning point for Google, in a bad way.

The filter

It’s true Google takes signals from many different sources. With the launch of Google+ they now incorporate additional social signals. As Facebook becomes the default dashboard for more and more internet users, the means of finding content is shifting from Search to Social. So Google is responding to this by making their overall service more social as well.

The impact though for users of the service could be confusion. Many users I’ve spoken to, working in tech or otherwise think the results they see on Google are unbiased and the same for each user. Google’s secret sauce has always been its algorithm that returned the best results. Now that social signals are mixed into the page rank brew, will users continue to value Google results?

A cause for concern

Pariser’s illustrates the difference in Google search today with great examples. After the gulf oil spill, he asked two friends to search for “BP”. One saw breaking news on the topic, the other got investment information about BP. Filter bubble, indeed.

Behavior Targeting, as it’s termed in the industry, is all about figuring out what you want before you ask. But sociologist Danah Boyd argued in a Web 2.0 Expo speech in 2009, that with all this personalization giving us exactly what we *want* that
“If we’re not careful, we’re going to develop the psychological equivalent of obesity”.
Even foundres, Sergey Brin and Larry Page in the early days apparently thought that this bias might turn out to be a problem

Firms like Recorded Future promise to “unlock the predictive power of the web” and lesser known but formidable Acxiom which specializes in marketing and personalization, combing through mountains of data to figure out what coast you live on, how you vote, what you eat and drink, and almost what you’re going to do before you do it.

Pariser touches on everything in this book from present bias to meaningful threats, the priming effect, accessibility bias and warns of getting trapped in what he terms a “you loop” where you continue to see things framed and personalized by what you’ve viewed and reacted to before, ultimately narrowing your view, and limiting your exposure to new information.

Perhaps the biggest problem with these opaque transformations applied to your data is that they play judge and jury with no appeal; sometimes without your knowledge that you were in a courtroom being judged. Programmers write algorithms and code to perform these transformations, sorting people into groups. If they put people in a group that doesn’t match them they call it “overfitting”. In society we might call this some sort of stereotyping.

One chapter titled the Adderall Society, asks if this filter bubble isn’t part of a larger transformation that’s pushing our thinking towards the conservative and calculative, limiting creativity or foreign ideas that can break categories, and encouraging us to ignore or steer around serendipity.

The book bumps into a great spectrum of thinkers on this topic, from Danny Sullivan of SearchEngineLand, to Amit Singhal an engineer on Google’s team, and John Battelle’s SearchBlog. He speaks to Chris Coyne – okcupid.com, David Shields on what he calls truthiness and former CIA consultant John Rendon who says “filter bubbles provide new ways of managing perceptions” and suggests the garden variety Thesaurus as a powerful tool that nudges debates with new language.

Be aware but don’t be paranoid

Although I think the book gives valuable insight, I was a little dismayed by the mood of paranoia in its title. With a subtitle like What the Internet is Hiding from You, it suggests a conspiracy or hidden agenda. Now obviously these large corps have a motive to make money, but I don’t think anyone is surprised at that. To some, Pariser’s views may appear somewhat left leaning but the issues raised in his book transcends political boundaries. They are matters that concern society at large.

In the end I think I’m probably more optimistic about these things. With a long view, society tends to work out these issues, through public pressure or simply buying differently. As Google is quick to remind us, we can easily choose an alternate search engine. In the future perhaps public pressure will push firms to provide more transparency about these filtering mechanisms allowing end users to manage their own filter settings.

I’ll leave you with a few ideas to chew on. Can code and algorithms curate properly? Should there be another button alongside the Like button such as “Important”?

Pariser quoted the folks at the New York Times: “We don’t let metrics dictate our assignments and play, because we believe readers come to us for our judgement, not the judgement of the crowd”. Indeed. But in the internet age, is that what they *buy* or *click*?

For more articles like these go to iHeavy, Inc +1-212-533-6828

An article I read on Tech Crunch recently got me thinking about startup culture. In Are You Building A Company, Or Just Your Credentials Geoff Lewis, expressed his distaste for a friend’s plan to get on Y Combinator’s ‘no idea’ startup incubation program. In this experimental approach, groups or individuals with a desire to be part of a startup but who have no product or business idea to begin with, can apply.

The thinking I believe, is that since brilliant ideas aren’t the only factor for startup success (many other factors like organisation skills, business savvy and tenacity matter too) YC will dig into their vault of ideas and match one that’s most suitable to these idea-starved groups.

Firstly, reading this I could sympathize with the discontent.  Venture capitalists exist for people who have an idea and want to realise it. There are already programs for people who don’t have an idea but want to achieve some success. They are called “careers”.

However, if you look at it from an incubator’s perspective, it’s a pretty clever and measured approach. YC knows what sort of group dynamic in startups have a higher chance of success and it is casting its net wide to find them.

But Lewis took issue with the fact that his MBA-qualified, credential-seeking buddy was signing up with YC presumably just to add to his blue ribbon collection.

With such articles what’s usually more interesting is the comments they elicit. Many who reacted responded with the same feelings of contempt; calling paper-chasers ‘hucksters’ and scoffing at their lack of passion and sincerity.

The tone among some carried this notion that startups were a special breed of entrepreneurs burdened with some sort of higher calling to liberate the world; money and honour being an afterthought.

Not being directly part of the startup circle, so-to-speak, I found the reaction amusing and frankly, rather foolish. Are people involved in startups turning into a sort of in-group? Do they really think of themselves as some kind of mutant-strain of businesses that are different from ‘regular’ enterprises?

If we look at the richest Internet companies today, once startups themselves, they are no less motivated by avarice and the bottomline, so why be so judgmental of Mr MBA treating the YC program as a way to gain credentials? I bet many talented individuals are going to have a go at it for the same reason if not a variety of reasons. As the program draws out, I suspect the number of participants will peter out by attrition anyway.

And who’s to say not coming up with a disruptive idea makes you less enterprising? Think of startups as team sports. There are some teams that innovate with the most creative gameplay that catch their opponents off-guard. There are teams that win by consistency and endurance, doing the same thing well over many years to pull ahead of the pack.

To cast aspersions on the motivations of others and to let silly prejudices limit participation seems discordant with the spirit of the Internet itself, where challenging convention is the order of the day and everyone is entitled to a shot at success.

For more articles like these go to iHeavy, Inc +1-212-533-6828

To The Cloud: Powering An Enterprise introduces the concepts of cloud computing from a high-level and strategic standpoint. I’ve read quite a few tomes on cloud computing and I was interested to see how this one would stack up against the others.

The book is not too weighty in technical language so as not to be overwhelming and intimidating. However at ninety five pages, one might argue it is a bit sparse for a $30 book, if you purchase it at full price.

It is organized nicely around initiatives to get you moving with the cloud.

Chapter 1, Explore takes you through the process of understanding what the cloud is and what it has to offer.

Chapter 2, Envision puts you in the drivers seat, looking at the opportunities the cloud can offer in terms of solutions to current business problems.

Chapter 3, Enable discusses specifics of getting there, such as selecting a vendor or provider, training your team, and establishing new processes in your organization.

Finally in Chapter 4, we hit on real details of adopting the cloud in your organization. Will you move applications wholesale, or will you adopt a hybrid model? How will you redesign your applications to take care of automated scaling? What new security practices and processes will you put in place. The authors offer practical answers to these questions. At the end there is also an epilogue discussing emerging market opportunities for cloud computing, such as those in India.

One of the problems I had with the book is that although it doesn’t really position itself as a Microsoft Cloud book per se, that is really what the book aims at.

For example, Microsoft Azure is sort of the default platform throughout the book, whereas in reality most folks think of Amazon Web Services to be the sort of default when talking about cloud computing. Although specifically, Azure is really a platform, while AWS is Infrastructure or raw iron, that can run Linux based Operating Systems, or Windows Azure stuff.

Of course having a trio of Microsoft executives as authors gives a strong hint to readers to expect some plugging but a rewrite of the title would probably manage readers’ expectations better.

The other missing piece with this book is a chapter on tackling new challenges in the cloud. Cloud Computing – Azure or otherwise, brings challenges with respect to hardware as using the cloud means deploying across shared resources. For example it’s hard to deploy a high-performance RAID array or SAN solution devoted to one server in the cloud. This is a challenge on AWS as well, and continues to be a major adoption hurdle. It’s part of the commoditization puzzle, but it’s as yet not completely solved. Such a chapter to discuss mitigating against virtual server failures, using redundancy, and cloud components to increase availability would be useful.

Lastly, I found it a bit disconcerting that all of the testimonials were from fellow CTOs and CIOs of big firms, not independents or other industry experts. For example I would have liked to see George Reese of Enstratus, Thorsten von Eicken from Rightscale or John Engates from Rackspace provide a comment or two on the book.

Overall the book is a decent primer if you’re looking for some guidance on Microsoft Azure Cloud. It is not a comprehensive introduction to cloud computing and you’d definitely need other resources to get the full picture. At such a hefty sticker price, my advice is to pick this one up at the bargain bin.

For more articles like these go to iHeavy, Inc +1-212-533-6828

I was recently approached by a healthcare company for advice on suitable database solutions capable of executing its new initiative. The company was primarily an Oracle shop so naturally, they began by shopping for possible Oracle solutions.

The CTO relayed his conversation with the Oracle sales rep, who at first recommended an Oracle solution that, expensive as it may have been, ultimately aligned with the company’s existing technology and experience. Unfortunately this didn’t match their budget and so predictably, the Oracle sales rep whipped out a MySQL-based solution as an alternative.

Having worked as an Oracle DBA throughout the dot-com years, I know the technology well. I also know the cultural differences between enterprises that choose Oracle solutions and those that choose open-source ones.

This encounter with the healthcare firm struck me as a classic conundrum for today’s companies who are under pressure to meet business targets under a tight budget, and in a very short time.

Can an open-source solution like MySQL be the answer to such huge demands?

The Oracle sales rep will likely nod excitedly and say no sweat. But as a consultant I could only manage an equivocal yes.

As the healthcare CTO rattled off the list of products he wanted to use, specific RTOs and RPOs (recovery time objective + recovery point objective – all I could think was to react with concern.

In my experience with startup after startup I’ve seen plenty of different MySQL installations but I’d never heard of one with the technology stack he described. What’s more I’d never heard of these solutions described with the Oracle Corp titles.

On one hand I wanted to discuss the merits of the solution he was keen to implement, while on the other, I was expressing concern over possible directions and paths we might take.

An Oracle cluster is not a MySQL cluster

The solution Oracle suggested was a MySQL Cluster. The term cluster unfortunately means different things to different people. Such loose usage of the word dilutes its meaning. In particular a lot of Oracle technologists expect that this solution might be similar to Oracle’s Real Application Cluster technology. It’s not. There are a lot of limitations, and frankly it’s really just a different beast.

The list also included various management dashboards which Oracle likes to push, but which I rarely see in my consulting assignments. What’s more I heard nothing about replication integrity considering that replication problems are an ongoing concern for real-world MySQL installations due to the particular technology used under the hood. There are reliable solutions to this problem but none yet available from Oracle. In fact, this is a big problem but one that may be completely off the sales guys’ radar.

Don’t let sales frame your architecture

Honestly, I don’t have a particularly large axe to grind with the sales guys. They have a job to do, and providing solutions which bring revenue to their firm and commissions for themselves is what puts food on their tables. Each party is motivated in different ways. But as a company shopping for solutions, this should be kept clearly in mind when starting down that road.

Beware prescribed architectural frameworks that appear too easy because they almost always don’t do what they say on the tin. Unfortunately sales folks don’t have experiencing designing architectures in the real world, so they can’t really know how the technologies work beyond the data sheet with feature bullet points.

As we all know in the technology space, all software come with bugs and real-world experience does not match the feature lists in the brochures. In law they have de jure and de facto. The former describes what is written and the latter, what’s practiced. For technology solutions, its never just adding water for something to work.

Do your homework

Before you embark on a new trip through the open source technology jungle, do some due diligence. Read up on real-world solutions, and how other large firms are using the technology. What configurations are they having success with? Which are causing trouble for a lot of people.

One of the great advantages of open-source are the very vibrant communities, forums and discussion groups where people are glad to share their experiences and offer advice.

Allow sufficient time to test and
bring your team up to speed

This is very important one. Shifting from an enterprise that relies primarily on Oracle for it’s relational database solution over to one that relies on open source technologies is a very big step indeed. Open-source technologies tend to be much more do-it-yourself and roll your own. Oracle solutions tend much more toward predefined paths and solutions and prescriptions for customers.

There are merits to each of these paths, with attendant pros and cons. But they are decidedly different. It’s likely that your team will also require time to get up to speed, not just with the particular software components, but with the new process by which things happen in the open-source space. Allow sufficient time for this shift to take place, lest you create more problems than solutions.

For more articles like these go to iHeavy, Inc +1-212-533-6828

What makes a beginner’s guide handy is when it speaks to your intuition. It anticipates the burning questions that follow from a newbie trying to grasp new concepts and it quickly answers them. PHP and MongoDB Web Development – Beginner’s Guide is one such guide.

I hadn’t heard of Packt Publishing or Rubayeet Islam before picking up this title and I must say I’m impressed. Based in Birmingham, with offices in Mumbai, part of Packt’s business model is to give part of the royalties earned from its books to the open source projects they cover.

I already had a working knowledge of MongoDB, mostly from an operational perspective. If you are new to MongoDB you’ll certainly appreciate how this book is structured. it cuts to the chase, diving right into the nuts and bolts of installing the pieces you’ll need such as database and drivers, and getting your first application running.

From there they take you through a basic web application step-by-step, with chapters on session management, MapReduce and GridFS. EVery time I flip through the pages of a technical book, I find I always have questions in the background; ‘what about performance?’ or ‘How do I troubleshoot these pieces as I’m building them?’

What I liked about this book is that almost as quickly as I’d formulate some question about performance, I’d happen upon answers in the book, as if it knew what would come to my mind at each point of the

I was thinking about tuning and application performance and then found chapter 9 which discusses MongoDB’s explain facility, similar to that of MySQL. From there they cover index creation, hints, and finally profiling. These are all important topics for a developer, ones that he or she should have in mind while building their applications. So I was happy to see good coverage of that even in a self-avowed beginners’ guide.

Building apps that talk to both MySQL and MongoDB

Another interesting chapter was one introducing the idea of building an application that can talk to both MySQL and MongoDB and using those two datastores for different purposes. Again while I’m reading it I start thinking about operational concerns, and I start asking how one would support such an architecture. And then just like clockwork, Islam answers that very question.

He explains the challenges around data consistency and operational support in detail. It’s a great way to introduce a topic without necessarily pushing that adoption per se. Islam is clearly an experienced programmer, with much reasoned advice to share.

The book had great utility but I do have a few complaints.

First off the font is a little funky, and hard to read after a while. In that same vein, some of the screenshots are very wide and as such were zoomed down. This made those tiny and not very readable. Also the screenshots aren’t really consistent, some are black on white and some white text on black terminal which ended up being impossible to read.

Lastly I would have liked to see more use case discussions. Particularly, when should I consider a NoSQL database like MongoDB over a relational database? Which types of applications are really well suited? Which aren’t? What about versus other NoSQL’s? The same with GridFS. There was some caution there after the material was introduced but more discussion about what applications it is well suited for would be useful.

Those few complaints aside, the book is overall very good and perhaps the publishers will consider improving the type and diagrams in the next edition. It definitely sticks to it’s cover page motto “Learn by doing: less theory, more results”.

For more articles like these go to iHeavy, Inc +1-212-533-6828

Continuing from our Top MySQL DBA interview questions (Part 1) here are five more questions that test a MySQL DBA’s knowledge, with two that will help suss out some personality traits.

Personality Questions

In my experience, some of the most important traits of a new hire center around personality traits, and how they might mix with your existing team.  Being punctual for an interview, for instance sets a precedent for many things.  But that door swings both ways, so if you want to hire these types of folks, don’t keep them waiting either!

Pay attention to whether or not the candidate takes some lead in the conversation at all.  This can indicate the person is a self starter.  Obviously a great candidate will also listen carefully and patiently to what you have to say, but may then take the ball and run with it somewhat.

Listen for signals that the person is active in the field, posting on forums, and attending conferences, meetups and forums on technology topics.  You might also ask them if they blog, and what topics interest them.

For more articles like these go to iHeavy, Inc +1-212-533-6828

MySQL DBAs are in greater demand now than they’ve ever been. While some firms are losing the fight for talent, promising startups with a progressive bent are getting first dibs with the best applicants. Whatever the case, interviewing for a MySQL DBA is a skill in itself so I thought I’d share a guide of top MySQL DBA interview questions to help with your screening process.
It’s long and detailed with some background to give context so I will be publishing this in two parts.

The history of the DBA as a career

In the Oracle world of enterprise applications, the DBA has long been a strong career path. Companies building their sales staff required Peoplesoft or SAP, and those deploying the financial applications or e-business suite needed operations teams to manage those systems. At the heart of that operations team were database administrators or DBAs, a catchall title that included the responsibility of guarding your businesses crown jewels. Security of those data assets, backups, management and performance were all entrusted to the DBA.

In the world of web applications, things have evolved a bit differently. Many a startup are driven only by developers. In those smaller shops, Operations tasks are designated to one developer who takes on the additional responsibility of managing systems. In that scenario, Operations or DBA duties becomes a sort of secondary role to the primary one of building the application. Even in cases where the startup creates a specific operations role with one person managing systems administration, chances are they don’t also have DBA experience. Instead, these startups are more likely to manage the database as a typical Linux application.

When I grow up I (don’t) want to be a MySQL DBA

Where do they come from, and why don’t a lot of computer science folks gravitate towards operations, and DBA? This may be in part due to and the romance of certain job roles which we discussed in a past article, The Mythical MySQL DBA. This pattern appeared a lot in the Oracle world as well. Many folks who were career DBAs actually moved to that role from the business side. In fact you’d find that many didn’t have a computer science or engineering background in the first place. In my experience I saw many Linux and Unix administrators with a stronger foundation who would fit into the DBA role but were simply not interested in it. The same can be said of the MySQL side of the house. Computer science grads don’t get out of school aiming for a career in ops or as a DBA because it has never been regarded as the pinnacle. It’s typically the PROGRAMMERS who become the rockstars in a cool startup.

But as the Internet grows into a richer and more complex medium, things are changing. People talk about scalability, high availability, zero downtime and performance tuning. When brief outages cost millions in losses expectations are very high and that requires skilled, experienced DBAs.

We’ve made a list of comprised of skill questions, general questions and ‘good-to-know’ questions. Have fun grilling your candidate with them, although bear in mind that with interviews it’s not about knowing it all, rather how the person demonstrates critical thinking skills.

Skills Questions

Feeling like a MySQL expert yet? In Part 2 of Top MySQL DBA Interview Questions we’ll walkthrough four more questions plus a bonus.

For more articles like these go to iHeavy, Inc +1-212-533-6828

I picked up Phil Simon’s The Age of the Platform after running into his blog, and some of his writing online. Simon is an interesting guy with an obvious strong technical background. He’s also an accomplished speaker and you can find several videos of his speaking online.

The first thing that struck me about this book was how it came to be. The book was funded through Kickstarter, an online platform for people to fund their creative projects. Perhaps it was Simon trying to drive home the point of his book. But it gets better, he self-published the book through Motion Publishing. Furthermore the book isn’t cheap for a paperback at $20. That said I admire that he has obviously eaten his own dog food, as the proverbial saying goes, and done it himself.

The premise of the book is that we’re entering a new age exemplified by four companies, namely Google, Facebook, Amazon and Apple. He takes us through a quick history of each company, then illustrates their successes and how each of them have successfully created platforms to extend their reach.

Apple did something amazing when they created the Appstore, which boosted demand for their products just a bit more each time a new app is developed for their platform. Google has strong gravitational pull with their platform by building endless free services from email to instant message and voice, to the news, maps & shopping. And let’s not forget Amazon who are putting a storefront in your hands with their Kindle, and expanding in other directions with their datacenter technologies. And last but surely not least comes Facebook, who’ve also obviously gained much by allowing third parties to build apps and extend their own platform.

Simon describes these ‘platforms’ as a new economic phenomenon. I think large firms have done this in the past, such as Microsoft, Oracle or Kodak. Further one might argue car makers, with the endless variety of third party suppliers & accessories support a huge ecosystem as well.

Another argument that seemed to prevail in the book is that the gang of four embrace openness and that this is something new. Certainly these four embrace openness in certain areas. But overall I would argue that they like many firms that have come before them, embrace openness when they are being disruptive to existing businesses, while they exhibit the same territorial monopolistic tendencies when they are protecting their own established business. Google has shown this in dealing with affiliates dismissively, Apple has shown this by not allowing certain apps into the app store which pose a competitive risk, Facebook exhibits this tendency when trying to export your data, profile, events or photo albums out of the service. And even Amazon has been tempted by and accused of using old economy tactics to pressure publishers for bigger discounts.

Should we all jump on the platform?

Simon’s premise also seems to include the idea that you and me, small business owners or sole proprietors can use these same concepts, to build our own businesses in new and exciting directions. In fact he says that we must do this or face obsolesence. Now while I admit that it is certainly important to watch trends, and not allow the changing winds of business and technology to leave us without customers or clients, it’s not at all clear to me that focusing on one thing and doing it well is a recipe for failure. The folks at 37 Signals have illustrated this so well with Basecamp. So has Chip Conley has with his chain of Joie de Vivre Hospitality hotels. Read more about his business philosophies in his great book Peak – How Great Companies Get Their Mojo from Maslow.

What’s more I definitely think great companies have failed in the past by trying to innovate in areas not close to core business.

So overall I think Simon has some good ideas, I think he’s stretched it a bit in trying to point out an underlying trend calling this an age of the platform. If we must call it something it’s simply the age of the Internet, the age of disruption.

For more articles like these go to iHeavy, Inc +1-212-533-6828

We’ve all seen cloud computing discussed ad nauseam on blogs, on Twitter, Quora, Stack Exchange, your mom’s Facebook page… you get the idea. The tech bloggers and performance experts often pipe in with their graphs and statistics showing clearly that dollar-for-dollar, cloud hosted virtual servers can’t compete with physical servers in performance, so why is everyone pushing them? It’s just foolhardy, they say.

On the other end, management and their bean counters would simply roll their eyes saying this is why the tech guys aren’t running the business.

Seriously, why the disconnect?

Quite simply, they have very different agendas. The operations team is trying to deliver application speed, reliability, and scalability. Meanwhile the management team is trying to balance cash flow, capital investments and operating expenses, all this while retaining as much flexibility in decision making as possible.

Sound familiar? Yes, the bottom line for both are the same, but they approach the problem from two different sides.

After hearing this refrain from enough customers, I realized the question we were asking in operations was the wrong question. It is true that cloud servers do not perform as well as traditional servers — at least not yet. The question that one should ask is, given this performance constraint with virtual servers, how can I alleviate the performance limitations of virtual servers and storage but still deploy into the cloud?

A little bit of history

Anyone who’s been around computing since the mid 1990’s will see where I’m heading with this. Commoditization of computing, that’s where. In the late nineties there was a big shift happening. In the 1990’s as Linux was maturing, commodity servers became all the rage. A big shift began to happen which drove cheaper PC hardware into the datacenter. Where once Sun stood supreme, suddenly all these upstarts were pushing crappy commodity hardware at 1/10th of the cost.

Old-guard systems administrators  at that time would balk, telling you how the stuff wasn’t reliable, performance was worse, and they just failed too often. And you know what? They were right! Nevertheless look where we are today. Linux barreled through the datacenter because it lowered costs and introduced flexibility by giving you more choices and more redundancy.

All this rings very true today with the push to the cloud. Startups are the biggest adopters of cloud computing because the future is mostly unknown to them. Investing a quarter million dollars in hardware today when you don’t know where you’ll be in six months – that’s a very hard prospect to entertain.

But cloud computing more than just lowers cost. Don’t have the money for redundant servers to support Disaster Recovery? With Amazon EC2 you simply write scripts to rebuild your infrastructure and keep them handy when such an ill-fated day arrives. Flexibility, scalability and easier cost management. That’s what the cloud delivers.

For more articles like these go to iHeavy, Inc +1-212-533-6828

For more articles like these go to iHeavy, Inc +1-212-533-6828

Scanning Crains NY Business recently, I saw an article on ‘starting up’ in 54 hours.  It’s the brainchild of Marc Nager, Clint Nelsen and Franck Nouyrigat called Startup Weekend. Startup bootcamps seem to be the current extra-curricular activity of choice these days. Wharton is also getting in on it with their Innovation Tournament. Then there is the 48 Hour Startup  and of course let’s not forget the 3 Day Startup.

So what’s my beef?  Truth be told I admire the ambition, the optimism, and the openness of these efforts.  And for sure these bootstrapping marathons do introduce entrepreneurs to future colleagues and partners, get them asking the right questions about financing, customers, revenue, competition and so forth.

My problem with these events is they frame startups as something you *can* do quickly. As if it were a Lego set or pop-up book that gives instant results and gratification. Sure startups are 21st century tech-driven business that provide innovative products in a very short development cycle but a lot of the day-to-day running of the business are still very mundane 20th century sensibilities; not unlike running a mom and pop store, a laundromat, deli or sandwich shop.

No shortcuts to startup success

You have to learn how to ask your customers what they want, learn how to sell them something they need, and be prepared to be surprised in the process. I’ve worked with many startups and have seen often how many have had to throw out an idea that didn’t work and re-strategize. This takes courage an conviction.

One startup client was building iPhone apps for event management. After testing it on the market it turned out nobody really wanted an event management app.  But there was a market for professional and consulting services that can help build technology platforms, and architect the right solutions.  So they shifted gears and moved into that space.  They’ve grown tenfold since.

Another startup client’s change in direction was so radical that it even warranted a name change. Buzzd, started out as a city guide for consumers based on aggregating Facebook, FourSquare and Gowalla check-ins. Last October Buzzd became LocalResponse and turned its focus onto advertisers instead.

This kind of honest self-analysis and business acumen so crucial for survival can only be acquired through experience. For example managing overheads or being aware that profitability does not always mean solvency. And then there are decision taking and setting priorities. Repair that wall or build a new display case? Use that money on product and inventory?  Attract customers to your store?  What’s the right price point to settle on?  How do you avoid legal entanglements, and who are some of the predators to watch out for?

Startup crash courses are probably a great way for  aspiring entrepreneurs to network, and for VCs to spot the next killer investment. But at the same time, it feels suspiciously like a diet fad that promises to change 30+ years of behavior overnight, and get you looking like a bikini model in no time.  Fitness and health are lifestyle choices, as is building a successful business. As much as we like to think otherwise, there are simply no shortcuts.

For more articles like these go to iHeavy, Inc +1-212-533-6828