When You're Hired to Solve a People Problem

Also find Sean Hull’s ramblings on twitter @hullsean.

A good five years ago I worked for a firm in online education. Among various products they provided through their website, they were struggling with a process to get content churned out more quickly. The bottleneck was slowing down their business, and limiting the new products they could offer.

Help Us Publish, Pleaseā€¦

Among a number of things I was asked to look at, one particularly vexing problem surrounded publishing. Adding new products had become a cumbersome & difficult process. It took days sometimes weeks. For obvious reasons the stake holders wanted to wrestle this process out of the hands of engineering, and place it were it arguably belonged in the hands of the business units.

[quote]
When you’re hired to solve specific technical problems it only figures that you go looking for software solutions. But sometimes the problems turn out to involve the people and processes of an organization. Getting them unstuck is one of the biggest challenges an professional services consultant can face. But it is also one of the most rewarding to solve.
[/quote]

Bumping into Fear, Uncertainty & Doubt

As I dug into the meat of the problem I began to work closely with the database administrator. He was a very smart gentleman & friendly in his own way. But he also spoke with a very thick accent and brusqueness about his manner that proved difficult at times. After working together for some awhile, however I began to win him over, and he started to trust me.

Looking for a top-flight database administrator? Here’s our interview guide for recruiters, managers and candidates alike

It became apparent that he was rather resistant to handing over the keys to the publish process to non-technical folks in other departments. Having handled his share of outages, and bungling screw ups, which sometimes fall on operations during some of the least hospitable hours on the dial, I could understand his concern. What’s more he knew the code which had grown unwieldy.

If I were to use a polite euphemism I would call it spaghetti code.

Management, Managers & Trouble Brewing

Around then the CTO decided to send a manager to sniff around. Unfortunately the manager in question was a very hands off type. His edict was simply to get this done in two weeks, and proceeded to go on vacation. Upon his return when things were still hitting snags, things started to go south.

Despite AWS failures firms like AirBNB and Reddit didn’t have to go down.

Though some of the process had been automated, I refused to move the changes into full push-button automation without first testing on dev environments. Of course those requests had fallen on deaf ears.

Problem comes to a head

Next the hands off manager escalated things upstream, of course adding his own spin on the situation. Shortly thereafter I’m called into the CTOs office only to get royally chewed out. A serious smack down which I’ll admit came almost out of nowhere.

A related article which readers also found quite popular: A CTO Must Never Do This

Oh, honestly I’m not complaining. On some level this is the job of the consultant. To act as the third party, wise or unbiased second opinion, and even punching bag at times.

Once things calmed down, I explained the situation from top to bottom. Yes there was messy code, and yes the process was complex, but it could of course be automated. What really stood in the way was a very resistant engineer who currently owned the process.

As much of the Sandy recovery continues, Devops can learn real lessons from the hurricane.

The CTO for his part concurred, having had trouble communicating with the engineer himself, and really not liking him much. He then appointed a proper project manager to oversee redoing the publish process from scratch.

A Plea for Cooperation

If I were to do it all again, for my part I’d sniff out the people dynamics more carefully. It’s often the case the companies have the engineering talent in house to solve a particular problem, but not the will or knowledge to put it into play.

Is your business growing? Having trouble scaling? Here’s how we do a performance review. It’s the first step on your way to hyper growth.

To managers & CTOs I’d encourage where possible to look for people, process and communication issues. Try to ferret out when something is an engineering problem, or whether it is one of people, silos and territory.

Want more? Grab our Scalable Startups monthly for more tips and special content. Here’s a sample

Switch to the cloud – shift of a century

The switch to cloud is way bigger than you think

A Review of Nicholas Carr’s book “The Big Switch”, available on Amazon here.

Also find Sean Hull’s ramblings on twitter @hullsean.

Do you work in devops or as a performance consultant? Do you manage web applications backed by databases? If so you probably love high performance beefy iron, big servers with equally fast RAID arrays that deliver lightening fast response to an entire application and ultimately your customers.

A related article, Devops can learn from Sandy, serious and very real disaster recovery lessons.

So if you’re like me you may feel a little leery about the cloud. On AWS for example, server commoditization has taken infrastructure for a sharp turn south. We struggle with unreliable disk performance & shared network bandwidth, while our applications compete with other customers in the so-called multi-tenant environment. Even the servers themselves drop like flies. Something’s got to give!

[quote]
All of them would be wise to study the examples of General Electric and Westinghouse. A hundred years ago, both these companies were making a lot of money selling electricity-production components and systems to individual companies. That business disappeared as big utilities took over electricity supply. But GE and Westinghouse were able to reinvent themselves. They became leading suppliers of generators and other equipment to the new utilities, and they also operated or invested in utilities themselves. Most important of all, they built vast new businesses supplying electric appliances to consumers — businesses that only became possible after the arrival of large-scale electric utilities. Sometimes a company can discover an even better business if it’s willing to abandon an old one.
[/quote]

Shopping for a smartphone? Find out why the Android platform is broken.

That’s why Carr’s book offers an eerie and uncanny read. What we’re seeing today in infrastructure very closely parallels what happened to electricity before it. Turns out at the turn of the century electricity production was not centralized and no electric grid was yet criss-crossing the country. Big companies actually built and managed their own power plants.

What happened?

Through the efforts of great entrepreneurs like Thomas Edison & Samuel Insull, the electricity production machinery were centralized and eventually incorporated and run by government managed utilities. All of this drove costs of electricity precipitously downward.

Looking for a database expert? Take a look at our MySQL Interview guide for candidates, hiring managers and recruiters alike.

How did we benefit? Oh can you count the ways?

Now households could afford electricity too. Next we saw consumer appliances and automation begin. Vacuum cleaners to washing machines flourished, bringing social change with it.

[quote]
What the fiber-optic internet does for computing is exactly what the alternating-current network did for electricity: it makes the location of the equipment unimportant to the user. But it does more than that. Because the internet has been designed to accomodate any type of computer and any form of digital information, it also plays the role of Insull’s rotary converter: it allows disparate and formerly incompatible machines to operate together as a single system. It creates harmony out of cacophony.
[/quote]

Cloud providers fail, components fail, datacenters fail. Find out why AirBNB & Reddit didn’t have to fail even while it’s AWS cloud went down.

The takeaway

The shift to cloud computing is way bigger than one application or one business. And the gains and momentum are way larger than we in devops may realize. With that it’s an inexorable shift, and one we would do well to embrace. Like all shifts we need to learn to adopt our technologies, as the benefits to business are incalculable.

Want more? Grab our Scalable Startups monthly for more tips and special content. Here’s a sample