Categories
All Blogging Consulting Hiring

5 data points I track for reputation & career building

When I tell people I’ve been independent for two decades, they often look at me surprised. How do you do that? How do you keep business coming in?

recent linkedin views

Join 32,000 others and follow Sean Hull on twitter @hullsean.

As a freelancer you surely have to be on top of changing trends, and where the wind is blowing. But whether you’re a CEO or CTO of a larger firm, or a developer, HR or marketing director, you can also benefit by actively tracking yourself. Career building never ends…

1. Real Leads

This is probably the hardest metric to track, but the most important. A lead is anyone who may potentially hire my services. These can come from Linkedin, newsletter subscribers, or via a Google search. I track how they reached me, and how warm the lead is.

I do also track when recruiters reach out, as I think this can serve as a useful barometer as well. Also as my blog has grown, I get a lot of SEO bloggers, fishing for sites they can post backlinks on. Although I rarely entertain them, it is a useful reflection of how popular your site is getting.

Also: Are we fast approaching cloud-mageddon?

2. Newsletter signups

I think of the newsletter as an extension of my blog. I invite everyone I’ve ever touched in business. This includes coworkers, to colleagues at meetups & conferences. I invite recruiters & headhunters as well, because name recognition & reputation building is also important.

The newsletter is a way to show up in the inbox of everybody you’ve ever worked with. Month after month, year in and year out, you’re plodding away & doing your thing. It’s a reminder that you’re out there, and colleagues, CEOs & CTOs refer me all the time. It’s been very valuable over ten years.

newsletter signups

I also track email opens & email clicks. Those range around 25% and 10% respectively. I know when I’ve hit a topic that resonates & try to have that inform future content direction.

Related: The Myth of Five Nines

3. Linkedin Views

Linkedin is super valuable too. They provide a nice graph of how many times your profile was viewed weekly through to the last 90 days. This is super useful to find out if your resume & profile is keyword rich.

I like to actively tweak my profile, for the latest trending terminology. For example in the 90’s Unix Administrator or Systems Administrator was common, but nowadays everyone likes to say SRE. What’s that? Site Reliability Engineer. Yes it’s a buzzword, and as it turns out people use trending terms & buzzwords to search for people with your skills.

So get on it, and edit those terms!

Read: Is Amazon too big to fail?

4. Website Visitors

In a services business you don’t usually sell widgets on your website. However, I like to think of a web presense as my business card. So in that light, more visitors means more renown. That projects your personal brand, and builds it long term.

website visitors

Also: When hosting data on Amazon turns bloodsport

5. Klout Score

Klout score is a rough measure of how active you are across social media. Twitter is a big one, but it also finds you on Linkedin & other platforms as well. Although the score is far from perfect, it does give you a sense of reputation & noteriety, which do ultimately translate to business.

Also: 5 Things Toxic To Scalability

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Categories
All Cloud Computing CTO/CIO Hiring Trends

5 ways to level up as cloud expert

aws certified

Cloud computing is blowing up! But don’t take my word for it, read this recent NY Times piece: Tech companies clamor to entice cloud computing experts.

Join 32,000 others and follow Sean Hull on twitter @hullsean.

Still don’t believe me? Get on the phone with a recruiter or two. They’ll convince you because they’ve got companies banging down the door looking for talent that is plainly in SHORT SUPPLY. And that’s the supply *you* want to be. 🙂

Check Gary’s Guide Jobs, or the ever popular Angel List Jobs. There’s also Stack Overflow jobs and many more.

1. Become a book reviewer

You’ve already got a technical background, and want to hone those skills. Take a look at technical book reviewing.

Manning is putting out some excellent technical books these days. Apply here to be a reviewer.

Also take a look at Pragmatic Bookshelf. They are are looking for reviewers too.

In either case you can expect to spend time reading a book chapter by chapter, as it’s written, offer strategic or layout advice, feedback on presentation, comprehension, and edits.

Also: When hosting data on Amazon turns bloodsport

2. Join an Open Source project

There are millions. Flip through github to some that you’re interested in. Contribute a bug fix or comment, reach out to the project leaders.

Afraid to dive in? Join one of the forums or google discussion groups, and lurk for a while. Ask questions, offer a helping hand!

Related: Is Amazon too big to fail?

3. Self-paced labs

Online education is blowing up, and for good reason. They get the job done & for the right price!

One of my favorites for AWS Certification is the A Cloud Guru courses. These offer lecture style introduction to all levels of AWS from Sysops Administration, Developer & Solutions Architect to Devops, Lambda & CodeDeploy.

The courses are priced right, and geared directly towards Amazon’s certifications. That helps you focus on the right things.

Amazon also partners with qwiklabs to offer courses geared towards getting certified. There are specific ones for the associate & professional certification, and many others besides.

You’ll need to signup for AWS Activate first, before you can use these qwiklabs. They offer you 80 credits right out of the gate.

For the next two weeks many of the courses are free! One thing I really like is they include a free temporary aws login for the students. That way there’s no risk of deploying infrastructure, and accidentally getting a big bill at the end of the month.

The labs though are more like reading documentation versus a nice video course lecture. So you the student have to do a lot more to get through it.

Read: Are we fast approaching cloud-mageddon?

4. Coursera, Khanacademy & Udemy

There’s a free class on Coursera called Startup Engineering by Balaji Srinivasan & Vijay Pande. Some pretty amazon material & lectures in here, and if you’re determined, it’s 12 weeks that will get you going on the right foot!

KhanAcademy has a great many courses on computer programming. Awesome and free stuff here. One particularly interesting is their hour of code. For those hesitant, that’s an easy way to jump in!

There is also udemy, which offers some great material on cloud computing. Notice that the certification courses are the same ones from A-Cloud Guru!

Also: Are SQL databases dead?

5. Interview tests

Apply to jobs. Even if you’re unsure if that is your dream job. Why? Because they often include a test to find out about your technical chops. Diving into these tests is a great way to push your own edge. You may do well, you may not. Learn where your weaknesses are.

I especially like the ones where you’re asked to login to a server, configure some things, write some code, and solve a real problem. Nothing beats a real-world example!

Also: Why dropbox didn’t have to fail?

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Categories
All Hiring

Are top candidates evaluating your startup?

Editor & writer in friendly dialog

I work for a lot of startups. Many ask me for referrals. I play matchmaker when I can. But as the market continues to heat up, the demand for top talent is reaching a boiling point.

Join 29,000 others and follow Sean Hull on twitter @hullsean.

1. It’s a sellers market

That means folks with technical skills across the spectrum are very indemand. How in demand? Check Angellist, Made In NY or Indeed.com. From SRE’s to full stack developers, devops & automation experts to DBAs. Java, Ruby, Python, PHP, node.js, and of course design skills too.

I was speaking with a recruiter just today, and heard the same refrain…

Top candidates are evaluating us just as we are evaluating you.

That means firms must go the extra mile to stand out, and draw in the best talent.

Also: 5 Things toxic to scalability

2. Open the glassdoor

That’s right, manage your social media presence. Sites like Glass Door provide forums where employees past & present can discuss the day-to-day work environment. This gives prospects a chance to peer behind the curtain.

Other social media can be avenues too, from Facebook to Twitter. Having someone on staff that monitors online reputation can be crucial.

Related: Are SQL Databases dead?

3. Host a tech blog & meetups

A lot of top firms have great tech blogs. Truth be told many are dormant as demands of the day trump these outward facing initiatives. But they also put a face on the technical side of working for a firm. What problems are they solving? How cutting edge is their team?

Meetups are also a limitless forum. Smart minds will be mixing, your company brand will be spreading. Hosting technical discussions brings your firm front & center in multiple ways. It also brings possible new hires to your living room.

Read: Is high availability a myth?

4. Show warmth & transparency

I know everybody loves to grill candidates at interviews. But interviewees should be schooled on politeness & how to give a pleasant interview.

I remember one interview where I faced off with four other engineers at a round table. As the discussion unfolded, each aimed shots in succession, almost rapid fire at me. It was not only intimidating, but frustrating. Needless to say it made me a stronger more resilient interviewer, but it’s not a great way to welcome great talent. Buyer beware!

Also: The chaos theory of cloud scalability

5. Show me the money

I know I know, for engineers it’s not all about the money. Or is it? Truth be told compensation is always something prospects will weigh. Equity is fine, for what it is. But it’s a promise into the future.

More senior talent who have been through a few startups or even dot-com 1.0, may be a bit more dubious of abstract compensation. In the end competitive real dollars will speak volumes.

Also: Is upgrading Amazon RDS like a shit-storm that will not end?

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Categories
All Hiring

Big data scientist interview questions

Screen-shot-2012-08-02-at-1.28.35-PM

Everybody wants to hire a data scientist these days. According to read write the role is overhyped and overpaid. Hype aside, what’s a good approach to interviewing these hard to find people?

Here’s Read Write’s guide. Also Hilary Mason has an interview guide. Also take a look at Chris Pearson’s data scientist hiring guide.

Join 28,000 others and follow Sean Hull on twitter @hullsean.

While you’ll surely have technical questions to ask, we figure you may already have a handle on those. They’ll vary from business to business.

What we’ve put together is a series of questions that we hope will tease out some good stories, and underscore a candidates real-world experience. These are are also great for the cross section of folks involved in the hiring process, higher level managers, HR & recruiters, plus technical folks that may have data near & dear to them.

1. What’s common?

What key metrics do you see firms repeatedly missing? Why are they important?

You’ve worked as data scientist before, and run into a lot of problems at different firms. Inevitably, some of those repeat themselves. Give an example of a metric you see over-and-over, that’s essential, but often missing attention.

Also: Is the difference between dev & ops a four-letter word?

2. What’s your favorite?

What is your favorite KPI and why?

As a data scientist, you’ve probably approached different companies, and found a couple of indicators that you particularly like. Maybe they highlight potential for growth? Or lead to other interesting discoveries about the business?

Related: Is automation killing old-school operations?

3. Let’s talk dollars

Give an example of a financial benefit you brought to a firm. How & how much?

Give an example where a measurement you made, and a business change it informed had real ROI for the business. What was that discovery? How did the business make the change? What was the financial benefit to the bottom line?

Read: Do managers underestimate operational cost?

4. Business data discovery

Give an example where you discovered data the business didn’t know it had. What & How?

Sometimes businesses have assets stored, that have been forgotten. Perhaps they’ve been archived, or a collection job has been forgotten. Perhaps it’s a corner of salesforce that hasn’t been evaluated. How did you bring the new data to light, and make use of it?

Also: Is the difference between dev & ops a four-letter word?

5. Why do you love data?

Why is data scientist your chosen career path?

This is an open ended question, but should spark some stories. Perhaps the candidate enjoys working with tech, product & biz-ops equally? Why are your skills uniquely suited to the role over other technical careers?

Also: Is the difference between dev & ops a four-letter word?

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Categories
All Hiring Web Operations

Is there a devops talent gap?

jenkins_docker

New programming languages & services are being invented at a staggering pace. Hosting is changing, networking is changing, race to market is quickening.

But what does all of this mean to the search for talent? Who understands all these components? Who is an expert in any one?

Join 28,000 others and follow Sean Hull on twitter @hullsean.

1. That new car smell

We all remember that time. You know when you drove out of the dealership with your brand new wheels. Driving down the road, you feel on top of the world. You start dreaming of all the fun times you’ll have in your new car. For days and weeks afterward you walk out to your car, open the door & sit inside. It all feels special. You kind of hang out there for a few minutes enjoying the smell before you drive off. Right?

Let’s be vigilant to remember the same thing happens, or rather is happeing in technology all the time. As we automate our infrastructures with Ansible, Puppet & Chef, deploy continuous integration with Jenkins, Travis or Codeship, we should give pause. Each of these tools has it’s own syntax, it’s own bugs, it’s own community, it’s own speed of development & change, it’s own life.

Also: Does a four letter word divide dev & ops?

2. A lot of rushing

Google tells me the synonyms of agile are, nimble, lithe, supple & acrobatic. So in a fast moving world it’s no wonder agile is so big. Anything that allows us to respond to customers quicker & evolve our product faster is a good thing. Yes it is.

Over the years I’ve worked with a lot of clients & customers. Some right out of the gates are in a hurry. There is a sense of urgency even from the initial meeting. Although not in every case, sometimes these are the sign of the perpetually late. They end up throwing money around, throwing technology around, and all in a desperate attempt to plug a leaking ship.

In our race to automate & remain agile & nimble, we should also consider the future. Lets attempt to find a balance & consider future implications of technology decisions & choices.

Read: Is automation killing old-school operations?

3. Hosting, what’s that?

For many of the startups I work with today, they’ve never deployed on anything but Amazon. There was no rack of computers in a closet & a T1 line, circa 1997. There was no rackspace hosted servers or a colo in New Jersey circa 2005. Right from the beginning it was all on-demand computing.

This shift has surely brought a lot of benefit. But no one can argue it isn’t still very new. And with newness there is a learning curve. And bugs & surprises.

Related: Does a devop need to practice the art of resistance?

4. More complexity in troubleshooting

The wild ride really begins when you’re troubleshooting performance problems. Running your database on RDS you say? How the heck do I get to the terminal and run “top”? Can I do an iostat?

And what does iostat output really mean in multi-tenant Amazon, where your disk is an EBS volume across an unknown & unfriendly network. Who knows why it just slowed to a crawl, then sped up dramatically a few minutes later.

Even fetching the relevant logfile can be complicated. For all the problems the cloud eliminates, it sure introduces a few of it’s own. And who is the expert, and how to find them?

Read this: When fat fingers take down your database

5. More tech, fewer experts

I asked the question a few weeks back Do todays startups require assembly of a lot of parts that no one really understands?.

I’ve taken to browsing the stacks at the lovely StackShare site lately. There you can see what some of the top startups are using for their technology stacks. Docker, Yammer, Yelp, Stripe, Vine, Spotify & Stack Overflow are all there today.

There are new message queues like NSQ & programming languages like Markdown, Coffeescript & Clojure. Even Java. Are people still building web apps in Java. No please no!

While it’s wonderful to see such an explosion of innovation, I look at this from an operations perspective. In five years, when the first & second wave of developers at your startup have left, picture yourself trying to find talent in a long since out-of-fashion language like Dart or Swift. What’s more how do you untangle the mess you’ve now built?

Check this: Is the SQL database dead?

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Categories
All Hiring

Best of hiring posts on scalable startups

strawberries

Join 28,000 others and follow Sean Hull on twitter @hullsean.

Why I write about hiring

I’ve worked as a consultant for almost twenty years. Technology & professional services are pretty far removed from hiring, so why would I write about it?

As it turns out, finding projects, working with clients, and selling your skills & solutions has quite a lot in common to do with hiring.

As a services consultant, you’re more often a peer to technology directors & CTOs, while hiring for traditional roles is more of a boss employee relationship.

Recruiters

I’ve run into a lot of recruiters & hr folks over the years. Usually it means I’m talking to the wrong folks, as they’re gatekeepers & not decision makers. I wrote Why I don’t work with recruiters after some ups & downs.

Still they’re all a fact of life, and each of us has a role to play. So let’s play fair!

Games

I’ve always wondered, Is Hiring a numbers game? That is does it bend more to persistence & throwing spagetti at the wall, or deliberate, precision searches?

MySQL interview

If you’re looking for a database expert, I put together
Top MySQL DBA interview questions and then another one
Advanced MySQL DBA Interview questions.

These are helpful not just to candidates, but to hiring managers, hr, recruiters & everyone in between.

Mythical talent

Since as far back as I can remember, DBAs have been in short supply. In the 90’s I was doing primarily Oracle work. There were never enough technical dbas. Many came from business backgrounds, and didn’t have operating system & hardware fundamentals.

As startups shifted to open source databases in droves during the 00’s, the situation became even worse. I wrote about
The mythical mysql dba – where can we find one?

Will NoSQL databases continue the same trend?

Hire a developer

With a little light humor, we throw some opinions into the fray around hiring devs with How to hire a developer that doesn’t suck.

As devops gains momentum, some see peace between the old-school silos of developers & operations. Some see the need for ops being supplanted by developers. We have some opinions too.

AWS Interview

Are you looking for an Amazon Web Services expert, who knows how to scale in the cloud? Devops & automation also on your mind? Check out
8 Questions to ask an amazon ec2 expert.

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Categories
All Hiring

Why I don’t work with recruiters, but I learn from them

via GIPHY

Join 11,000 others and follow Sean Hull on twitter @hullsean.

Bring me clients, please… pretty please!

When you first start out as a freelancer, your network is small. Without a steady stream of projects, the tendency is to reach for whatever you can find. Head shops & agencies build a brand, and ongoing relationships with firms. However a project with a middleman is a relationship with him or her, not the client directly. You lose control of a few very key things, such as fees, testimonials and payment terms.

Read: NYC technology startups are hiring

It’s all about the relationship, Luke

With independent consulting, your relationship with clients is key. Fostering that relationship, builds trust, communication, and confidence in you as a service provider. Doing operations and database management, a CTO, VP or Director of engineering needs to be confident entrusting enterprise systems to you. Security of assets, reliability that things won’t break, and consistency are all crucial.

Working with a recruiter, agent or head shop the client then may feel a stronger relationship with that firm. Testimonials and due credit for successful completion of a project may go to them rather than to you directly.

It also means you lose control of the conversation about fees. Want to do project or week-based fees, your suggestion may fall on deaf ears. What’s more you will share large margins that could amount to 25% or even 50% of the overall fee.

Read This: Why hiring is a numbers game

Headhunters have the pulse of the market

With all those complaints, you might think I don’t like recruiters much, but you’d be sadly mistaken. It turns out I learn a ton from recruiters, and almost always take their calls.

o those conversations are good practice for talking shop
o they provide good feedback & ask questions about confusing areas in conversation
o buzzwords will pop up prominently, helping you understand what their clients needs are
o gives you a bit of the pulse of the market

I learn to speak in broad terms, in a language managers and folks at all levels of an organization can understand, and I learn patience too.

Check out: Top MySQL DBA Interview questions for candidates, managers & recruiters

Recruiter pings – a key performance indicator

Over a ten year period you start to notice trends. Certain times of the year I get more calls & more pings from talent agencies. Here’s what I monitor:

o recruiter views of my linkedin profile
o recruiters email me on linkedin
o recruiters call me
o recruiters signup for my newsletter

Also: Why operations & MySQL DBA talent is hard to find

Learn from people whose business is communication

Although I don’t have referrals or connections for the HR or search consultant that’s reaching out to me there is still lots I can learn. At the end of the day, recruiters are in the business of relationships, and that’s where I become the humble student.

Also: 8 Questions to ask an AWS expert

Get some in your inbox: Exclusive monthly Scalable Startups. We share tips and special content. Here’s a sample

Categories
All CTO/CIO Hiring Startups

NYC Tech Firms Are Hiring – Map

Made In NY - Startups Hiring

If you haven’t noticed how much the NYC tech scene has grown recently, I’m afraid you’ve been hiding under a rock. It’s simply incredible.

Take a look at Mapped In NY a google maps mashup of the growing list popularized by the NY Tech Meetup called Made In New York.

Join 5000 others and follow Sean Hull on twitter @hullsean.

[mytweetlinks]

Having been around during the first dot-com boom back in the late 1990’s this is even more exciting to see. Despite the recession, New York’s economy is truly thriving!

[quote]
New York’s Startup scene is truly thriving with a whopping 1263 firms, many of which are hiring.
[/quote]

Why is database administration talent in short supply? They are the Mythical MySQL DBAs

Also take a look at: Why Generalists are Better at Scaling the Web

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Here’s a sample

Categories
All CTO/CIO Hiring Startups

Hacking Job Search – Three Meaty Ideas

Also find the author on twitter @hullsean.

Demand for talented engineers has never been higher. It is in fact the dirty little secret of the startup industry, that there are simply not enough qualified folks to fill the positions.

What this means for you is that you have a lot of options. What it means for a hiring manager is that you will have to work even harder to find the right candidate. Just going to a recruiter isn’t enough. Use your network, go to meetups, follow Gary’s Guide daily.

Also check out our Mythical MySQL DBA piece where we talk about the shortage of DBAs and operations folks.

Further if you’ve dabbled in freelance or independent consulting, I wrote an interesting an in depth look at Why do people leave consulting. Understanding this can help avoid it in your own career, or avoid your resources leaving for better shores.

Find us on twitter @hullsean and linkedin where we share content and ideas everyday!

1. Build your reputation

As they say, your reputation precedes you. So start building it now. Fulltime or freelance, you want to be known.

Speaking, yes you can do it. Start with some small meetups, volunteer to speak on a topic. A ten person room is easier than 30, 50 or 100. Once you have a couple under your belt, fill out a CFP for Velocity, OSCon or some software developers conference. There are many.

Blog – if you’re not already doing so you should. Start with once a week. Comment on industry topics, controversial ideas, or engineering know-how. Prospects can look at this and learn a lot more than from a business card.

Write a book, yes you can. It may sound impossible, but the truth is that publishers are always looking for technical writers. Pick a topic near and dear to you. It’ll also give you endless material for your blog.

Go to meetups, you really need to be getting out there and networking. Get some Moo Business Cards and start working on your elevator pitch!

Social media – being active here helps your blog, and helps people find you. Twitter is a great place to do this. Interact with colleagues and startup founders, VCs and more. If you’re a hiring manager or CTO, you may find great programmers and devops this way.

We also wrote a more in-depth article Consulting and Freelance 101. It’s a three part guide with a lot of useful nugets.

Also take a look at our MySQL DBA Interview Guide which is as helpful to devops and DBAs as it is to managers hiring them!

[quote]
Above all else, build your network & your reputation. It will put you in front of more people as a person, not a commodity or a resume in a pile of hundreds.
[/quote]

2. Qualify prospects

You definitely don’t want to take the first offer you get, and managers don’t want to hire the first candidate that comes along. You want two or three to choose from. Best way to do this is to have options.

If you’re a candidate, network or work through your colleagues. When you do get a lead, be sure you’re speaking to an economic buyer. If you’re not you’ll need to try to find that person who actually signs the checks. They are the ones who ultimately make the decision, so you want to sell yourself to them.

Get a Deposit – I know I know, if it’s your first freelance job, you don’t want to scare them off. Or maybe you do? The only prospects that would be scared off by this are ones who may not pay down the line. Dragging their feet with a deposit can also mean bureaucratic red tap, so be patient too.

Sara Horowitz has an excellent book Freelancers bible, we recommend you grab a copy right now!

Commodity You Are Not so don’t sell yourself as one. What do I mean? You are not an interchangeable part. You have special skills, you have personality, you have things that you’re particularly good at. These traits are what you need to focus on. The dime-a-dozen skills should sit more in the background.

You’ll also need to price and package your services. We talked about this in-depth in Consulting Essentials – Getting the Business.

We also think there is a reason Why Generalists are better at scaling the web.

3. Play the numbers game

For hiring managers this doesn’t mean working through recruiters that might be bringing subpar talent, it means networking through industry events, meetups, startup pitch and venture capital events. There are a few every single day in NYC and there’s no reason not to go to some of them.

For candidates, be eyeing a few different companies, and following up on more than one prospect. You should really think of this process as an integral and enjoyable part of your career, not a temporary in between stage. Networking doesn’t happen overnight, but from a regular process of meeting and engaging with colleagues over years and years in an industry.

At the end of the day hiring is a numbers game so you should play it as such. Keep searching, and always be watching the horizon.

Read this far? Grab our Scalable Startups for more tips and special content.

Categories
All Cloud Computing Database Management Hiring MySQL Oracle Startups

Cloud DBA and Management Interview

What does a cloud computing expert need to know? This is the last of a three part guide to interviewing for a cloud operations position. You can find them here – part one Operations Interview and part two Deployment Interview.

Here’s my guide to do just that.

1. Database administration experience

Although in some shops the DBA role is a completely separate one, there are many others where the Linux and Operations teams manage these services as well. We do have a some other material Oracle DBA Interview questions and MySQL DBA Interview Guide. Here’s a taste of what to expect.

o What is RAID? Which type is best?

RAID is a way to share a whole bunch of disks on one server. Databases like Oracle or MySQL do a lot of writing and reading from disk. If there are more disks sharing this work, it’s like you have more waiters in your restaurant. Faster serivce.

Although some folks still hang onto RAID 5 as an option, it’s generally a very bad one. It has a serious write penalty because of parity checking it must perform. Most databases do a lot of writing, even when user transactions are not doing INSERT or UPDATE. What’s more if a disk fails, RAID 5 although technically online, will be so slow as to be effectively unusable while the long slow rebuild happens.

What’s the answer then? RAID 10! It mirrors each volume, and then stripes across those mirrored sets. Fast I/O, fast recovery. Done & done.

o What are the tradeoffs with more indexes versus fewer?

In all relational databases, you build indexes on data. Indexes are just like the ones you think of in the yellow pages, phonebooks of yore. An index on first name means you can look up Obama by Barack as well. Index on street addresses means you can lookup on the White House. So the more indexes you have, the more different ways you can search for & fetch what you want.

On the other hand the penalty here, is that whenever you add new data & records to this database, all those indexes must be updated. That’s overhead, which slows down writes.

So the tradeoff is more indexes – faster fetching, slower writing. Fewer indexes slower fetching, faster writing.

o What do NoSQL databases eliminate? How do they achieve great speed?

There are quite a few different types of NoSQL databases. So I’m generalizing quite a lot here. One thing NoSQL databases eliminate is the ability to JOIN data across different columns. By removing this great feature of relational databases, they dramatically simplify the underlying implementation. No free lunch!

What else? Many of these databases cut corners on what’s called durability. What is durability? Imagine you are in a lecture hall and bring your notebook or are waiting tables, and taking orders. It might be quicker to do so without writing things down. You keep it all in your head. Great, but what if you forget something? You have to go ask for the order again! It may be faster, but more prone to error. Losing data is not something to be taken lightly. NoSQL databases don’t always flush data to permanent storage.

[quote]
Whether or not an web operations candidate uses command line may seem like a small issue. But it speaks to what their DNA is, and the strength of their foundation. Strength and comfort on the command line is key.
[/quote]

o What is Amazon RDS? When should I use it?

Amazon has a managed relational database solution called RDS. It’s basically MySQL, Oracle or SQL Server, but modified so you can’t shoot yourself in the foot. Administrative tasks are simplified, but so are your configuration options.

I wrote an in-depth Amazon RDS use cases article. It mostly covers MySQL, but the general rules apply to Oracle & SQL Server. At the end of the data RDS is a lot less configurable and flexible. But if you don’t have a regular DBA on staff, it will probably simplify your administration of these servers.

o What are read-replicas? What about Multi-az?

Read-replicas are read-only copies of your data. Using MySQL these are fairly stock master-slave configurations. Note since they’re the standard technology, they’re still asyncronous. So yes the read-replica can lag behind.

Multi-az is a proprietary technology, and Amazon doesn’t disclose what’s under the hood. However it’s likely running on top of something like DRBD which is a distributed filesystem. This allows the underlying disk I/O to be mirrored across the internet, and to another availability zone. You’ll enjoy syncronous copies of your data, and no data consistency problems. Keep in mind those that the alternate server is offline or cold and can take time to come online.

o What is the primary bottleneck of hosting databases in the cloud? How has Amazon recently addressed this?

As I explained above disk I/O remains the largest bottleneck for relational databases, even if the entire dataset fits in memory. Why? Because sorting, joining, and rearranging data can take orders of magnitude more memory to magically do in memory. And that’s not even talking about durability guarentees.

The cloud has traditionally lagged quite a lot behind physical servers in terms of disk I/O so some internet firms have shyed away from moving to the cloud. EBS volumes were typically limited to a few hundred IOPs.

Amazon’s recently announced Provisioned IOPs. It’s a mouthful of a name for a very big development. It means you can provision how fast you want those virtual disks to be. For individual volumes the limit seems to be 2000 IOPs but you can also software raid across many of those virtual disks. For Amazon RDS the limit is reportedly 10,000 IOPs. This new feature will make a huge difference for hosting large high I/O databases in Amazon’s cloud.

2. Architecture & Management Questions

o Why does the API battle between Amazon & Eucalyptus (FOSS) matter?

As large applications are architected to build hardware components, and resources in the cloud, the API they work through becomes key. Sticking to an open standard for this API means you can change cloud vendors and/or build on multiple ones. We talked about this multi-cloud solution as a key way to avoid outages like AirBNB and Reddit experienced when AWS had an outage.

Following on the heels of that article, we were quoted about multi-cloud by Brandon Butler in his Network World piece .

o Do you use command line tools? Why?

A good web operations candidate should be very comfortable with command line tools. Everything in Linux is command line. It’s like broadway acting to movie acting, or literature to books. It’s the original source, much more powerful, what’s more it indicates and requires much stronger theoretical knowledge of the underlying systems being managed.

o What can go wrong with backups? How do we test them?

Everything can go wrong with them. They can fail to complete. Be backups of the wrong service or resource. Even the backup software itself can have bugs. The only way to sleep well at night is if you run firedrills and restore your application and data top to bottom.

o Should we encrypt filesystems in the cloud? What are the risks?

This depends on your environment and how sensitive your data is. If you’re collecting credit card data for instance, it may be key. However some surprising blips may push other applications to encrypt as well. Bugs in the hypervisor could potentially make your data vulnerable. What’s more if the cloud provider gets subpeonaed, it may well capture your server and data into the net. Better safe than sorry. Remember you don’t know where your data actually resides, but you do control who has access if you’re encrypted.

We wrote a very in-depth piece on Deploying on Amazon EC2 where we discuss questions such as encryption in more depth.
o Should we use offsite backups?

It’s definitely worth doing this. One more layer of insurance.

o What is load balancing? Why is it difficult with databases?


Load balancing puts a digital traffic circle into your infrastructure, giving you two roads or paths to resources. However those resources have to be exactly the same. With databases you are constantly writing to tables, and updating records. When you scale those horizontally, it becomes impossible to keep track of changes.

[quote]
Relational databases are inherently difficult to scale. Most environments scale a single authoritative master vertically, and add multiple read-only slaves horizontally to allow the appplication to serve more customers.
[/quote]


o Why use a package manager? Can we install from source?

Package managers simplify the installation of software components. A team such as Redhat, Ubuntu or Debian builds a distribution, and compiles all components storing them in a repository. Installing packages this way allows your setup to be standard across servers. This allows more automation, and is simpler for another admin to figure out what you have, down the line when it passes to someone elses shoulders.

Installing from source is generally a bad idea. Although it allows you to tweak and configure each piece of software the way you want, tightly and efficiently, it also means everything is custom. No commoditization advantages.

o What is horizontal scalability?

This involves adding more hardware, more individual servers to service the same application and users.

o What is vertical scalability?

This means scaling up or growing your existing single server, so it is larger, has more memory, cpu or faster disk.

o What can go wrong with automatic failover?

Just about everything. Applications and services can stall, disks can fail, servers can hang. What’s more networks can exhibit latency. Automatic failover is ultimately a piece of software or algorithm trying to diagnose and handle situations. And it does so based on a very small list of rules or heuristics. The real world is messy, so this can often lead to false failure detection, and potentially loss of data.

o How do cloud vendors implement vertical scalability?

This may vary dramatically between cloud providers. Ultimately, however since virtualization allows you to boot a disk image onto any hardware, you can snapshot your current root volume or disk and then boot it on another server, one that is larger, smaller and so forth. About the only thing you need to watch out for is 32 versus 64 bit questions.

If you haven’t already, don’t forget to checkout the rest of this series – part one Operations Interview and part two Deployment Interview.

Read this far? Grab our newsletter – startup scalability.