Is there a serious skills shortage around devops space?

via GIPHY

As devops adoption picks up pace, the signs are everywhere. Infrastructure as code once a backwater concept, and a hoped for ideal, has become an essential to many startups.

Why might that be?

Join 37,000 others and follow Sean Hull on twitter @hullsean.

My theory is that devops enables the business in a lot of profound ways. Sure it means one sysadmin can do much more, manage a fleet of servers, and support a large user base. But it goes much deeper than that.





Being able to standup your entire dev, qa, or production environment at the click of the button transforms software delivery dramatically. It means it can happen more often, more easily, and with less risk to the business. It means you can do things like blue/green deployments, rolling out featues without any risk to the production environment running in parallel.

What kind of chops does it take?

Strong generalist skills

For starters you’ll need a pragmatist mindset. Not fanatical about one technology, but open to the many choices available. And as a generalist, you start with a familiarity with a broad spectrum of skills, from coding, troubleshooting & debugging, to performance tuning & integration testing.

Stir into the mix good operating system fundamentals, top to bottom knowledge of Unix & Linux, networking, configuration and more. Maybe you’ve built kernels, compiled packages by hand, or better yet contributed to a few open source projects yourself.

You’ll be comfortable with databases, frontend frameworks, backend technologies & APIs. But that’s not all. You’ll need a broad understanding of cloud technologies, from GCP to AWS. S3, EC2, VPCs, EBS, webservers, caching servers, load balancing, Route53 DNS, serverless lambda. Add to all of that programmable infrastructure through CloudFormation or Terraform.

Related: 30 questions to ask a serverless fanboy

Competent programmer

Although as a devop you probably won’t be doing frontend dev, you’ll need some cursory understanding of those. You should be competent at Python and perhaps Nodejs. Maybe Ruby & bash scripts. You’ll need to understand JSON & Yaml, CloudFormation & Terraform if you want to deliver IAC.

Related: Does a 4-letter-word divide dev & ops?

Strong sysadmin with ops mindset

These are fundamental. But what does that mean? Ops mindset is born out of necessity. Having seen failures & outages, you prioritize around uptime. A simpler stack means fewer moving parts & less to manage. Do as Martin Weiner would suggest & use boring tech.

But you’ll also need to reason about all these components. That’ll come from dozens of debug & troubleshooting sessions you’ll do through years of practice.

Related: How to hire a developer that doesn’t suck

Understand build systems & deployment models

Build systems like CircleCI, Jenkins or Gitlab offer a way to automate code delivery. And as their use becomes more widespread knowing them becomes de rigueur. But it doesn’t end there.

With deployments you’ll have a lot to choose from. At the very simplest a single target deploy, to all-at-once, minimum in service and rolling upgrades. But if you have completely automated your dev, qa & prod infra buildout, you can dive into blue/green deployments, where you make a completely knew infra for each deploy, test, then tear down the old.

Related: Is AWS too complex for small dev teams?

Personality to communicate across organization

I think if you’ve made it this far you will agree that the technical know-how is a broad spectrum of modern computing expertise. But you’ll also need excellent people skills to put all this into practice.

That’s because devops is also about organizational transformation. Yes devs & ops have to get up to speed on the tech, but the organization has to get on board too. Many entrenched orgs pay lip service to devops, but still do a lot of things manually. This is out of fear as much as it stands as technical debt.

But getting past that requires evangelizing, and advocating. For that a leader in the devops department will need superb people skills. They’ll communicate concepts broadly across the organization to win hearts and minds.

Related: Will Microservices just die already?

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

5 ways to level up as cloud expert

aws certified

Cloud computing is blowing up! But don’t take my word for it, read this recent NY Times piece: Tech companies clamor to entice cloud computing experts.

Join 32,000 others and follow Sean Hull on twitter @hullsean.

Still don’t believe me? Get on the phone with a recruiter or two. They’ll convince you because they’ve got companies banging down the door looking for talent that is plainly in SHORT SUPPLY. And that’s the supply *you* want to be. 🙂

Check Gary’s Guide Jobs, or the ever popular Angel List Jobs. There’s also Stack Overflow jobs and many more.

1. Become a book reviewer

You’ve already got a technical background, and want to hone those skills. Take a look at technical book reviewing.

Manning is putting out some excellent technical books these days. Apply here to be a reviewer.

Also take a look at Pragmatic Bookshelf. They are are looking for reviewers too.

In either case you can expect to spend time reading a book chapter by chapter, as it’s written, offer strategic or layout advice, feedback on presentation, comprehension, and edits.

Also: When hosting data on Amazon turns bloodsport

2. Join an Open Source project

There are millions. Flip through github to some that you’re interested in. Contribute a bug fix or comment, reach out to the project leaders.

Afraid to dive in? Join one of the forums or google discussion groups, and lurk for a while. Ask questions, offer a helping hand!

Related: Is Amazon too big to fail?

3. Self-paced labs

Online education is blowing up, and for good reason. They get the job done & for the right price!

One of my favorites for AWS Certification is the A Cloud Guru courses. These offer lecture style introduction to all levels of AWS from Sysops Administration, Developer & Solutions Architect to Devops, Lambda & CodeDeploy.

The courses are priced right, and geared directly towards Amazon’s certifications. That helps you focus on the right things.

Amazon also partners with qwiklabs to offer courses geared towards getting certified. There are specific ones for the associate & professional certification, and many others besides.

You’ll need to signup for AWS Activate first, before you can use these qwiklabs. They offer you 80 credits right out of the gate.

For the next two weeks many of the courses are free! One thing I really like is they include a free temporary aws login for the students. That way there’s no risk of deploying infrastructure, and accidentally getting a big bill at the end of the month.

The labs though are more like reading documentation versus a nice video course lecture. So you the student have to do a lot more to get through it.

Read: Are we fast approaching cloud-mageddon?

4. Coursera, Khanacademy & Udemy

There’s a free class on Coursera called Startup Engineering by Balaji Srinivasan & Vijay Pande. Some pretty amazon material & lectures in here, and if you’re determined, it’s 12 weeks that will get you going on the right foot!

KhanAcademy has a great many courses on computer programming. Awesome and free stuff here. One particularly interesting is their hour of code. For those hesitant, that’s an easy way to jump in!

There is also udemy, which offers some great material on cloud computing. Notice that the certification courses are the same ones from A-Cloud Guru!

Also: Are SQL databases dead?

5. Interview tests

Apply to jobs. Even if you’re unsure if that is your dream job. Why? Because they often include a test to find out about your technical chops. Diving into these tests is a great way to push your own edge. You may do well, you may not. Learn where your weaknesses are.

I especially like the ones where you’re asked to login to a server, configure some things, write some code, and solve a real problem. Nothing beats a real-world example!

Also: Why dropbox didn’t have to fail?

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Why the Twitter IPO mentions scalability

ShannaBanan-o-rama

Join 12,000 others and follow Sean Hull on twitter @hullsean.

1. High availability is hard

After seven years in the business you might think Twitter has operations and scalability nailed. I wouldn’t blame you for hoping, but here’s one thing they said in their IPO filing:

“we are not currently serving traffic equally through our co-located data centers”

What does this mean exactly? Let’s think of your drive to work everyday. Remember that one intersection that’s always congested? Could the city designers have envisioned that 50 or 100 years ago? Probably not. In the present day, with all the buildings & roads, can we redesign around it? Not easily. So we adapt, and evolve and deal with the day-to-day realities of an evolving city.

James Urquhart says these are complex systems. The internet, the cloud and your startup infrastructure are by nature brittle.

Also: Why generalists are better at scaling the web.

2. Fail whale is part of the DNA

The graphic above is a whimsical remake of Twitter’s own by Shanna Banan. Consider though, someone at twitter was tasked with designing a graphic for when the site fails. The devops team then built a page for failure, and have itat the ready, for when there’s an outage, not if. It’s symbolic of the many other things your operations team does behind the scenes in expectation of that fateful day.

As Eric Ries argues, design for failure. Then manage it.

Related: 5 reasons why scalability is a process.

3. Investors, wall street: we’re working on it

What Twitter is really saying is, hey investors, we understand that five nines is extremely difficult, we’re vulnerable in certain ways and want to disclose that.

ReadWrite argues Twitter has not banished the fail whale and is “surprisingly vulnerable”. Readwrite, I ask you… who has? Google? Nope. Facebook? Nope. Not AirBNB or Reddit either.

These are world class firms. They’ve got the deep pockets to do it right, and the engineering talent to match. They still have failures.

Read this: How to hire a developer that doesn’t suck.

Get more in your inbox: Exclusive monthly Scalable Startups. We share tips and special content. Here’s a sample

Why I Wrote the Book – Oracle and Open Source

Back in the late 90’s New York City was deep in the dot-com boom. Silicon Alley was being born, and a thousand internet startups were sprouting. Everyone was hiring, it was an exciting time to work in technology!

Join 11,500 others and follow Sean Hull on twitter @hullsean.

Trend Spotting Circa 2000

As an independent consultant, I had the opportunity to work at quite a few startups. The technology stack was identical at almost all of them. Sun Microsystems hardware, Apache webservers, and Oracle on the backend. The database was always the sticking point, and developers struggled to get their queries right.

It was an interesting role to hold. Most career DBAs worked at large fortune 500 firms, the old stodgy kind where nothing ever changes. Few of the Oracle old guard, the kind you’d meet at User Groups or conferences, had much exposure to Linux, and they certainly didn’t trust it.

Also: Here’s how to do a scalability performance review

Meanwhile in the startup scene in NYC I was seeing the cutting edge uses of the technology, with more and more shops switching to Linux and commodity hardware. There was even talk of *gasp* Oracle porting to Linux. There was a real rumor mill around all of this.

Oracle and Open Source Published – 2001

Seeing this shift towards commodity hardware, and the tremendous demand for Oracle married with open source technologies, I pitched O’Reilly and Associates with a book idea. Let’s talk about what’s happening in the trenches. How and when does Oracle – the most commercial of relational databases, work with Open Source technologies? What is in the mix? What are real firms using it for? What tools and technologies can help firms grow faster?

Related: Oracle DBA Interview questions for managers, candidates & recruiters alike

These were the questions my co-author and I sought to answer, and to judge from the response I think we did a very good job. As that push continued, Oracle eventually ported it’s enterprise database to Linux. This was a seismic shift that meant existing Oracle customers would spend a lot less on hardware, and thus have more to spend on Oracle licenses. Win-win except for Sun. The trend continued with Oracle pushing Apache into the mix as well.

Fast Forward a Decade

Now a decade later, Oracle has bought it’s former partner Sun, and in so doing owns MySQL too.

Read this: Top MySQL Interview questions for Devops, managers & recruiters

What new trends are happening? We hear an incessant drum of hype around cloud computing. In many ways the trend parallels what happened a decade ago. See our related piece a history lesson for cloud detractors. How so?

[quote]Commoditization: push towards new platforms, driven by cost. [/quote]

But this is slowed by an equally large stumbling block.

[quote]Performance: new cloud servers can’t compete with their big iron cousins. Not yet at least.[/quote]

Interested in Amazon EC2? We wrote an Intro to EC2 Cloud Deployments article which digs in deeper.

What’s Next for Datacenters

Commiditization will continue, driving costs downward. This will provide more gravity to cloud migrations for firms big and small.

Performance will improve. Cloud services like Amazon EC2 will get bigger & better, as will the all important network & disk subsystems.

Also: 5 things toxic to scalability

Big enterprises are already dipping their feet in the water with VPC technology, tying their existing datacenter to a cloud. They can grow elastically while still having feet firmly planted on the ground.

As large enterprises begin to get experience behind the wheel, it’ll chip away at the stranglehold of Oracle and the huge taxation type licensing that firms struggle with today. Where salesforce.com had a huge impact, workday.com will be even bigger.

[quote]The cloud will finally disrupt the last old guard industry – enterprise software.[/quote]

Read this far? Get us monthly in your inbox. Grab our scalable startups newsletter!