Category Archives: CTO/CIO

NYC Tech Firms Are Hiring – Map

Made In NY - Startups Hiring

If you haven’t noticed how much the NYC tech scene has grown recently, I’m afraid you’ve been hiding under a rock. It’s simply incredible.

Take a look at Mapped In NY a google maps mashup of the growing list popularized by the NY Tech Meetup called Made In New York.

Join 5000 others and follow Sean Hull on twitter @hullsean.

[mytweetlinks]

Having been around during the first dot-com boom back in the late 1990′s this is even more exciting to see. Despite the recession, New York’s economy is truly thriving!

[quote]
New York’s Startup scene is truly thriving with a whopping 1263 firms, many of which are hiring.
[/quote]

Why is database administration talent in short supply? They are the Mythical MySQL DBAs

Also take a look at: Why Generalists are Better at Scaling the Web

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Here’s a sample

Don't Miss Percona Live 2013

The biggest event on the MySQL calendar is the yearly Percona Live and it’s just around the corner.

This year you’ll be able to pick from a whopping 110 technical sessions by 90 different speakers from companies like Facebook, Amazon, Google and Linkedin. Learn what’s happening at the cutting edge of open source database deployments. Besides the technical sessions, there’s plenty of hobnobbing with fellow DBAs, developers, and industry folks. A great opportunity all around.

It’s just a couple of weeks away, so don’t waste any time. You can register here and enjoy a 20% discount code “SH-Live”!

Get to it!

Want more? Grab our Scalable Startups monthly for more tips and special content. Here’s a sample

When You Have to Take the Fall

Also find Sean Hull’s ramblings on twitter @hullsean.

One of the biggest jobs in operations is monitoring. There are so many servers, databases, webservers, search servers, backup servers. Each has lots of moving parts, lots that can go wrong. Typically if you have monitoring, and react to that monitoring, you’ll head off bigger problems later.

A problem is brewing

We, myself & the operations team started receiving alerts for one server. Space was filling up. Anyone can relate to this problem. You fill up your dropbox, or the drive on your laptop and all sorts of problems will quickly bubble to the surface.

Also check out – Why generalists are better at scaling the web.

As we investigated over the coming days, a complicated chain of processes and backups were using space on this server. Space that didn’t belong to them.

Dinner boils over

What happened next was inevitable. The weekly batch jobs kicked off and failed for lack of space. Those processes were not being monitored. Business units then discovered missing data in their reports and a firestorm of emails ensued.

Hiring? Get our MySQL DBA Interview Guide for managers, recruiters and candidates alike.

Why weren’t these services being monitored, they wanted to know.

Time to shoot the messenger

Having recently seen a changing of the guard, and a couple of key positions left vacant, it was clear that the root problem was communication.

Looking for talent? Why is it so hard to find a mythical MySQL DBA or devops expert these days?

I followed up the group emails, explaining in polite tone that we do in fact have monitoring in place, but that it seemed a clear chain of command was missing, and this process fell through the cracks.

I quickly received a response from the CTO requesting that I not send “these types of emails” to the team and to direct issues directly to him.

You might also like: A CTO Must Never Do This

A consultants job

As the sands continued to shift, a lead architect did emerge, one who took ownership of the products overall. Acting as a sort of life guard with a higher perch from which to watch, we were able to escalate important issues & he would then prioritize the team accordingly.

Are you a startup grappling with scalability? Keep in mind these 5 things toxic to scalability

Sometimes things have to break a little first.

What’s more a consultants job isn’t necessarily to lead the pack, nor to force management to act. A consultant’s job is to provide the best advice possible & to raise issues to the decision makers. And yes sometimes it means being a bit of a fall guy.

Those are the breaks of the game.

Want more? Grab our Scalable Startups monthly for more tips and special content. Here’s a sample

When Clients Don't Pay – Consulting War Stories

Also find Sean Hull’s ramblings on twitter @hullsean.

It’s a cold cold winter out there, so as they say you need money to keep the lights on? Yep, that’s true whether you’re a small business or a consultant. Everyone has to pay their bills. Or do they?

It’s an unfortunate fact of life in business, but sometimes there are differences. Disagreements about deliverables, timelines, milestones, and deadlines. But when all the work is done, there are still sometimes differences over dollars.

Over the years most of those I’ve managed to work out with clients, but there were a few that went sour. One case was with a large entertainment firm. The music business is one that I hadn’t interacted with much before. For these guys I’d done a few days work in the past, and was paid promptly. Now they were in a bit of a jam. They called me up and asked if I could help.

Related article: A CTO Must Never Do This.

These types of emergencies often come at the worst times, and I explained that I was already juggling a few other things. They pleaded for help, and I relented. I carved out a full day of time for them, explaining the day rate and so forth. While on the phone though, I expressed caution.

[quote]I understand that your issue is urgent now, what happens if your needs change in the next 48 hours, I asked? Not a problem they replied, we can use your help anyway, so we’ll book you either way. Fast forward 48 hours, work canceled & client won’t take my call.[/quote]

Great I thought, verbal assurances. That works for me, I thought.

Fast forward 32 hours, and I receive an email saying the problem is resolved, and offering a “kill fee” of which I knew nothing, and which was never negotiated or discussed.

Along comes the day of reconning, and I call the client. They don’t take my call. Shortly there after I receive additional emails. I reply and explain we should talk on the phone. Still the client can’t find the time to pickup the call.

So I judiciously put together an invoice for the work. I email it directly to finance, and CC all parties. From there I get responses ranging from disinterest to denial. Over the coming months I periodically resend the invoice, but to no avail.

Or was it? I actually feel that this experience is to great avail.

1. Small disagreements foreshadow larger ones down the road
2. A relationship between client and vendor is a mutual one. If parties can only pickup the phone when they need something, then things are out of balance already.

Sure I lost a day & the fee associated. But I gained a lesson.

Patience and polite persistence

I firmly believe that being patient and persistent wins in the end. Sometimes clients have hiccups in payroll or budgets. Keep communication lines open.

Freelancers & consultants: Grab my Consulting 101 Guide.

Appealing to fairness

If you maintain a healthy relationship with your client, then appeals to fairness are normally heeded.

Appealing to promises

Emails are important to keep a paper trail of agreements. Communicate clearly and often so you know when you get derailed, and can refer back to what was agreed previously.

Popular: AirBNB didn’t have to failed – AWS hosting outages.

Setting precedents & expectations up front

This is an important one, that freelancers and consultants alike sometimes forget. Setting and agreeing on expectations is key. Often details are in the fine print or left out completely. So ongoing communication can iron out those differences or bring them to light.

Want to hire the best? Read our DBA Interview Guide.

Getting a protective deposit

If you haven’t worked together before, a deposit makes a lot of sense. Executing on this is more than a show of faith. It underlines that accounts payable is on board with your hiring, and you are now in the payments system.

Related: Hiring is a numbers game

Sizing each other up

Websites provide the first representation of you and your client to each other. How you carry yourself and how they feel meeting them face to face is important as well. Ideally you’ll meet each other at the client’s offices, where you may meet others on the team, and put names to faces. At the very least a skype call will go a long way as well.

Managing spend – communication along the way

Keep a close eye on invoices. If a client is getting behind, resolve it before continuing to work. Deadlines are mutual in a business relationship. Yours to complete work by a certain date, and theirs to pay by an equally agreed upon time.

Read also: Real Disaster Recovery Lessons from hurricane Sandy.

Never go to court – defer to the handshake & gentleman’s agreement

I know the lawyers out there may think I’m naive. But I’ve been in business a long time, and I believe a handshake means and says a lot. Also common sense language & contracts, in the form of emails and so forth are better than heavily legalese ones that no one but a lawyer can understand.

What’s more going to court has a huge cost in time & stress. Don’t go that route. I also think it appeals to clients knowing that you’re not the litigious sort.

Want more? Grab our Scalable Startups monthly for more tips and special content. Here’s a sample

A master isn't born but made

A review of Mastery by Robert Greene.

Also find Sean Hull’s ramblings on twitter @hullsean.

Robert Greene’s 48 Laws of Power was a great read, offering endless lessons for business and personal dealings. When I saw he just published a new book, I was quick to grab a copy.

What I like about his writing is that he’s replete with counterintuitive bits of wisdom, that really offer new perspectives on old topics.

[quote]
Many people might find the notion of an apprenticeship and skill acquisition as quaint relics of bygone eras when work meant making things. After all, we have entered the information and computer age, in which technology makes it so we can di without the kinds of menial tasks that require practice and repetition; so many things have become virtual in our lives making the craftsman model obsolete. Or so the argument goes.
[/quote]

You might also enjoy our 3 part consulting 101 guide and our very popular DBA hiring guide.

He goes on to elaborate on this idea…

[quote]
In truth, however, this idea of the nature of the times we are living in is completely incorrect, even dangerous. The era we have entered is not one in which technology will make everything easier, but rather a time of increased complexity that affects every field. In business, competition has become globalized and more intense. A business person must have a command of a much larger picture than in the past, which means more knowledge and skills. The future in science does not lie in specialization but rather in combining and cross-fertilization of knowledge in various fields. In the arts, tastes and styles are changing at an accelerated rate. An artist must be on top of this and capable of creating new forms, always remaining ahead of the curve. This often requires having more than just a specialized knowledge of that particular art form — it requires knowing other arts, even the sciences, and what is happening in the world.
[/quote]

I couldn’t agree more. We wrote a piece a while back called Why generalists are better at scaling the web and that aligns nicely with what Greene is getting at here.

He begins with insight on finding ones life task, then apprenticeship & mentoring then working through the social challenges that are always present and finally ways to stimulate the creative-active impulses.

I really like that he emphasizes it as a process and one of life-long hard work. This resonates a lot for me, as that’s how I’ve found success doing independent consulting over the years. There have been a lot of ups and downs, wrong turns, and missteps, but tenacity wins out in the end. He even dispells the myth of the naturally gifted, such as Mozart or Einstein, arguing that in fact they did put in the requisite 10,000 hours of study and were not born with mastery as such.

Greene’s lastest book is a pleasure to read, and full of insight for startups, programmers, designers and business people alike. I highly recommend it.

Want more? Grab our Scalable Startups monthly for more tips and special content. Here’s a sample

Business Agility at AWS re:Invent

Also find Sean Hull’s ramblings on twitter @hullsean.

Although I couldn’t be in Vegas to attend re:Invent, there is so much online it’s almost better than being at the conference. From an ongoing live stream of keynotes and sessions, to an archived collection on Youtube.

The big wins

You may have heard of all the great things that Amazon or cloud computing can do, but I thought Andy Jassy summarized these nicely in these six points.

1. Replace capex with opex
2. lower total costs of ownership
3. no guessing about capacity
4. encourage agility & innovation
5. differentiation
6. global from the start

Redshift

By far the biggest announcement at the show is Amazon’s new Redshift product. It is a fully managed datawarehouse solution that scales to petabytes in it’s cloud. Currently there are two business intelligence tools that are supported namely Jaspersoft and Microstrategy.

[quote]
In 2003 Amazon was a 5 billion dollar company. Today AWS adds the same infrastructure capacity everyday to it’s availability zones!
[/quote]

Reduced prices by 25% for S3

As a lot of folks know, Amazon has always been about cheaper prices. That model has been disruptive in the book selling industry, and in a huge way in the infrastructure and datacenter industry. As more customers signup, economies of scale mean they can offer the same hardware & services for lower prices.

With that they’re announcing lower prices for S3 by a whopping 25%. To me this speaks to their continuing push to dominate the market by driving prices downward.

Amazon’s Channel on Youtube

If you weren’t able to attend the conference, or want to recap some highlights you might have missed, they have put up a great AWS Channel on Youtube.

Some of the speakers include Sharon Chiarella VP Mechanical Turk, Glenn Hazard, CEO, Xceedium, Todd Barr CMO of Alfresco talks, Bright Fulton, Operations for Swipely, Colin Percival, FreeBSD Developer, Ted Dunning, Chief Application Architect of MapR Technologies, James Broberg, CTO & Founder of MetaCDN, Mitchell Garnaat, Sr. Engineer, David Etue, Vice President, SafeNet, and Mike Culver, Sr. Consultant to name just a few.

Read this far? Grab our Scalable Startups for more tips and special content.

Hacking Job Search – Three Meaty Ideas

Also find the author on twitter @hullsean.

Demand for talented engineers has never been higher. It is in fact the dirty little secret of the startup industry, that there are simply not enough qualified folks to fill the positions.

What this means for you is that you have a lot of options. What it means for a hiring manager is that you will have to work even harder to find the right candidate. Just going to a recruiter isn’t enough. Use your network, go to meetups, follow Gary’s Guide daily.

Also check out our Mythical MySQL DBA piece where we talk about the shortage of DBAs and operations folks.

Further if you’ve dabbled in freelance or independent consulting, I wrote an interesting an in depth look at Why do people leave consulting. Understanding this can help avoid it in your own career, or avoid your resources leaving for better shores.

Find us on twitter @hullsean and linkedin where we share content and ideas everyday!

1. Build your reputation

As they say, your reputation precedes you. So start building it now. Fulltime or freelance, you want to be known.

Speaking, yes you can do it. Start with some small meetups, volunteer to speak on a topic. A ten person room is easier than 30, 50 or 100. Once you have a couple under your belt, fill out a CFP for Velocity, OSCon or some software developers conference. There are many.

Blog – if you’re not already doing so you should. Start with once a week. Comment on industry topics, controversial ideas, or engineering know-how. Prospects can look at this and learn a lot more than from a business card.

Write a book, yes you can. It may sound impossible, but the truth is that publishers are always looking for technical writers. Pick a topic near and dear to you. It’ll also give you endless material for your blog.

Go to meetups, you really need to be getting out there and networking. Get some Moo Business Cards and start working on your elevator pitch!

Social media – being active here helps your blog, and helps people find you. Twitter is a great place to do this. Interact with colleagues and startup founders, VCs and more. If you’re a hiring manager or CTO, you may find great programmers and devops this way.

We also wrote a more in-depth article Consulting and Freelance 101. It’s a three part guide with a lot of useful nugets.

Also take a look at our MySQL DBA Interview Guide which is as helpful to devops and DBAs as it is to managers hiring them!

[quote]
Above all else, build your network & your reputation. It will put you in front of more people as a person, not a commodity or a resume in a pile of hundreds.
[/quote]

2. Qualify prospects

You definitely don’t want to take the first offer you get, and managers don’t want to hire the first candidate that comes along. You want two or three to choose from. Best way to do this is to have options.

If you’re a candidate, network or work through your colleagues. When you do get a lead, be sure you’re speaking to an economic buyer. If you’re not you’ll need to try to find that person who actually signs the checks. They are the ones who ultimately make the decision, so you want to sell yourself to them.

Get a Deposit – I know I know, if it’s your first freelance job, you don’t want to scare them off. Or maybe you do? The only prospects that would be scared off by this are ones who may not pay down the line. Dragging their feet with a deposit can also mean bureaucratic red tap, so be patient too.

Sara Horowitz has an excellent book Freelancers bible, we recommend you grab a copy right now!

Commodity You Are Not so don’t sell yourself as one. What do I mean? You are not an interchangeable part. You have special skills, you have personality, you have things that you’re particularly good at. These traits are what you need to focus on. The dime-a-dozen skills should sit more in the background.

You’ll also need to price and package your services. We talked about this in-depth in Consulting Essentials – Getting the Business.

We also think there is a reason Why Generalists are better at scaling the web.

3. Play the numbers game

For hiring managers this doesn’t mean working through recruiters that might be bringing subpar talent, it means networking through industry events, meetups, startup pitch and venture capital events. There are a few every single day in NYC and there’s no reason not to go to some of them.

For candidates, be eyeing a few different companies, and following up on more than one prospect. You should really think of this process as an integral and enjoyable part of your career, not a temporary in between stage. Networking doesn’t happen overnight, but from a regular process of meeting and engaging with colleagues over years and years in an industry.

At the end of the day hiring is a numbers game so you should play it as such. Keep searching, and always be watching the horizon.

Read this far? Grab our Scalable Startups for more tips and special content.

No iPhones Were Harmed in the Creation of this Outage

Apple’s recent iMessage outage had some users confused. What do you mean I can’t text my favorite cat photos?? How can Apple do this to me!?!?

What happened?

Apple provides services to everyone who uses it’s platform. iCloud for example stores your contacts, calendar, photos, apps and documents in the cloud. No more syncing to itunes to make sure all your stuff is backed up. It’s automatic in the cloud. Yes or course unless iCloud is down.

Same goes for iMessage. Apple has quietly introduced this, as a more feature rich version of text messaging. It’s great until the service isn’t available. What gives?

All these services are backed magically or not so magically by computer servers. These computers sit in datacenters, managed by operations teams, and to some degree with automation. All the things that brought down AWS & AirBNB & Reddit with it could also take out Apple. A serious storm like Sandy also presents real risks.

[quote]
iMessage is a text and SMS replacement service for iPhones & iPads. It is more feature rich, offering device synchronization, group texting & return receipt. But in a very big way it is also an attempt for Apple to muscle into the market and further extend it’s platform reach.
[/quote]

100% uptime ain’t easy

Even for firms that promise insanely good uptime, five nines remains very very hard to achieve in practice.

For starters all the components behind your service, need to be redundant. Multiple load balancers, webservers, caching servers, and of course databases that hold all your business assets.

But as the repeated AWS outages attest, even redundancy here isn’t enough. You also need to use multiple cloud providers. Here you can mirror across clouds so even an outage in one won’t bring down your business.

What about in the world of messaging? Well you can bet your customers don’t likely know or care about high availability, uptime, or any of these other web operations buzzwords. But they sure understand when they can’t use their service. It may give companies like Apple pause as they try to stretch themselves into areas outside their core business of iphones, ipads, and the IOS platform itself.

iMessage – messaging standards power play

When I first upgraded to an iPhone 4S, the first thing I noticed was the light blue bubbles when texting certain people. Why was that, I wondered? I quickly found out about iMessage, which was conveniently configured, to replace my old and trusty text messaging.

Texts or SMS work across all phones, smartphone or not, and apple or not. But open standards don’t lend themselves well to market muscle and dominance. So it makes sense that Apple would be pushing into this space. I met more than one blackberry owner who loved using bbm to keep in touch with colleagues. It’s like your own private club. And that muscle further strengthens Apple’s platform overall. Just take a look at how the Android Ecosystem is broken if you need an example of what not to do.

The flip side is it means you have more to manage. More servers, more services, more dimensions to your business. More frequent outages that can tarnish your reputation.

[quote]
A lot complaining and publicity like the iMessage outage received, may just be an indication that you’re big enough for people to care.
[/quote]

Alternatives abound…

There is huge competition in the messaging space. The outage and it’s publicity further underline this fact.

For example on the iPhone for messaging there is ChatOn, Whatsapp, LINE, SKYPE & wechat just to name a few.

Interestingly, while researching this article, I downloaded WhatsApp to give it a try. Only 99 cents, why not. Turns out that they had not one, but two outages, just a week ago. Seems Apple isn’t the only one experiencing growing pains.

A lot of complaining and publicity could be a sign that you’re big enough for people to care!

Read this far? Grab our Scalable Startups monthly.

Crisis Management in the Crosshairs – Sandy

Crisis Management During Sandy

The news this past week has brought endless images of devastation. All metropolitan region, the damage is apparent.

More than once in conversation I’ve commented “That’s similar to what I do.” The response is often one of confusion. So I go on to clarify. Web operations is every bit about disaster recovery and crisis management in the datacenter. If you saw Con Edison down in the trenches you might not know how that power gets to your building, or what all those pipes down there do, but you know when it’s out! You know when something is out of order.

That’s why datacenter operations can learn so much about crisis management from the handling of Hurricane Sandy.

This is a followup to our popular article last week Real Disaster Recovery Lessons from Sandy.

1. Run Fire Drills

Nothing can substitute for real world testing. Run your application through it’s paces, pull the plugs, pull the power. You need to know what’s going to go wrong before it happens. Put your application on life support, and see how it handles. Failover to backup servers, restore the entire application stack and components from backups.

2. Let the Pros Handle Cleanup

This week Fred Wilson blogged about a small data room his family managed, for their personal photos, videos, music and so forth. He ruminated on what would have happened to that home datacenter, were he living there today when Sandy struck.

It’s a story many of us can related to, and points to obvious advantages of moving to the cloud. Handing things over to the pros means basic best practices will be followed. EBS storage, for example is redundant, so a single harddrive failure won’t take you out. What’s more S3 offers geographically distributed redundant copies of your data.

After last week’s AWS outage I wrote that AirBNB & Reddit didn’t have to fail. What’s more in the cloud, disaster recovery is also left to the professionals.

[quote]
Web Operations teams do what Con Edison does, but for the interwebs. We drill down into the bowels of our digital city, find the wires that are crossed, and repair them. Crisis management rules the day. I can admire how quickly they’ve brought NYC back up and running after the wrath of storm Sandy.
[/quote]

3. Have a few different backup plans

Watching New Yorkers find alternate means of transportation into the city has been nothing short of inspirational. Trains not running? A bus services takes it’s place. L trains not crossing the river? A huge stream of bikes takes to the williamsburg bridge to get workers to where they need to go.

Deploying on Amazon can be a great cloud option, but consider using multiple cloud providers to give you even more redundancy. Don’t put all your eggs in one basket.

Some very important things to remember about MySQL backups.

4. Keep Open Lines of Communication

While recovery continued apace, city dwellers below 34th street looked to text messages, and old school radios to get news and updates. When would power be restored? Does my building use gas or steam to heat? Why are certain streets coming back online, while others remain dark?

During an emergency like this one, it becomes obvious how important lines of communication are. So to in datacenter crisis management, key people from business units, operations teams, and dev all must coordinate. Orchestrating that is and art all by itself. A great CTO knows how to do this.

Read this far? Grab our monthly scalable startups.

Cloud Operations Interview

What does a cloud computing expert need to know? How do you hire a cloud computing expert? Competition for operations & DBAs is fierce, so you’ll want to know how to find the best.

If you’re a systems administrator or ops guy, you may want to prepare for an interview for such a position. Meanwhile, if you’re a director of it or operations, a recruiter or manager in HR, you’ll want to have some idea how to find the right candidate.

Here’s my guide to do just that. You may also jump to part two Cloud Deployment Interview or the last part three Cloud DBA, Architecture and Management Interview.

1. Solid unix systems administrator

At the top of the list, a cloud operations expert needs to understand Unix and more importantly Linux. Here are some sample questions to get the conversation moving:

o What is web operations and what have you done day-to-day?

Prepare some stories.

o What’s your favorite feature of the linux kernel?

This is an open ended question, but a systems administrator should have some knowledge here. The kernel is the most basic piece of software that runs when a computer boots up, whether it is a desktop or a server. This piece of software coordinates everything, manages resources, and directs traffic.

o Name some distributions of linux. What is a distro?

Linux is built by a collaborative team of thousands on the internet. That’s what makes it open source. The distributions, include the operating system, along with a collection of software to go along with it. All the supporting utilities, libraries and servers must be compiled and held in a repository. That’s what makes up a distribution. Debian, Redhat and Ubuntu are a few popular ones.

[quote]
A cloud operations expert needs to have a wide ranging skillset, from unix administration, architecture, scalability, database & webserver administration, troubleshooting & performance, load & stress testing. You’ll also want someone who has learned hard lessons from some failures, has some war stories to tell and has a hard nose for stability.
[/quote]

o What’s the difference between apache and nginx?

These two pieces of software are both webservers, that is they respond to the HTTP protocol, and can serve HTML pages. They also have a myriad of plugins to support different languages and features. The difference? Nginx (pronounced engine-X) is a newer incarnation. It’s been rearchitected from the ground up, building on all the things learned from Apache over the years. Its tighter, more efficient code, and easier to configure.

You might also enjoy our Intro to EC2 Cloud Deployments Guide.

o What is a key value store? examples?

There are lots of examples of these types of databases. They are a very simple memory cache that can interface with most applications. Memcache is a popular example of a key value store. Redis, CouchDB and Voldemort can also do this.

o What is a page cache? Reverse proxy cache? examples?

These are all the same thing. They are basically a very minimal webserver without all the plugins or bells and whistles. You put one of these in front of your webserver to handle all the easy stuff, and speed up overall throughput. Varnish is a popular example.

o What filesystem do you prefer?

This is a bit arcane, but one should have some opinions here. xfs is a popular filesystem, though ext3 and ext4 are also common. Emphasize the journaling aspect here. Journaling means that if you pull the cord or your server crashes, the filesystem can recover upon reboot. It does this by journaling changes, much how a database keeps a redolog cache of recent changes to database tables.

o Command line tools

There are lots of commands in the day-to-day toolbox of a web ops expert. Here are some examples:
rsync (pronounced our-sync) – sync files between servers & do checksums to allow easy restarts
scp (pronounced s-c-p) – secure copy, similar to rsync but no checksums, so less reliable
curl (pronounced kurl) – diagnose & test urls and HTTP from the command line
cron (pronounced cron) – run commands at scheduled times
ssh (pronounced s-s-h) – secure shell, the most basic tool to reach a cloud server
ifconfig (pronounced if-config) – check the network interfaces on the server
vi/emacs (pronounced v-i and e-macks) – terminal editors, to modify config files
uptime (pronounced up-time) – display the current load average of the server
top (pronounced top) – interactive display of system metrics like memory, load, swap & processes
ps (pronounced p-s) – shows running processes on the server
/var/log/messages – essential system logfile

o What are application servers? How are they different from webservers?

Tomcat & Glassfish are two examples of application servers. These handle heavier weight languages & applications like Java. Application server on some level is just a more heavyduty webserver and these days Apache can be thought of as an application server also.


2. Cloud concepts

o What is virtualization? What is a hypervisor?

Virtualization allows you to run one or more computers within a computer. You can do virtualization on a desktop, sharing network, memory, cpu and disk resources among a number of virtual servers. But more importantly in cloud computing or IaaS offerings you can do virtualization at the datacenter level. The hypervisor layer is a datacenter virtualization technology that provisions server resources, and balances shared network and disk resources.

o What is an image?

In Amazon the world, the AMI or amazon machine image is a snapshot of a server state at one moment in time. This image is take at the block level, and includes the master block record, the first block on disk that a server boots from. All that is the state of a server, when it is shutdown, is what is stored on disk or in this image. All config files, logfiles, and anything else writing to disk.

o What is multi-tenant?

This means that there are multiple servers sharing resources. The tenants are the customers who each want to get the server, cpu, memory, network and disk that they paid for.

o What is the downside to shared resources?

Contention for resources is always the challenge. If your fellow tenants are not very thirsty, this can work to your advantage. But if they’re also heavy users, the hypervisor layer has manage the balancing act. You may get a spike of disk I/O at one point, but later get a dearth. This can cause a relational database like MySQL or Oracle to suddenly look stalled.

o What is instance-store? What is ebs?

Instance store servers were Amazon’s original offering, where servers had their own local (and slow) storage. This storage was ephemeral, so all machine state was lost on reboot. These servers also boot slowly. EBS also known as elastic block storage is a virtualized storage option, similar to NAS or NFS. You can create arbitrary chunks of storage, and attach them to servers, all from command line APIs. Cool!

o What is virtual private cloud?

With the VPC offering, Amazon drops a router into your existing datacenter. You can then provision virtual servers to your hearts content, and they all appear to be servers in your existing datacenter. Elastically scale, within the network and security model you’re already using.

o What is a hybrid approach to cloud adoption?

Keeping your investments in hardware and datacenter is obviously an appealing option for firms that have large existing environment. A hybrid approach with a VPC allows you to get your feet wet, but still keep essential applications on physical servers.

o What is Amazon EC2?

Elastic Compute Cloud refers to the virtual servers you spinup in Amazon Web Services.

o What is Amazon RDS, Oracle RDS, Mysql RDS?

Amazon has various relational and non-relational database offerings. RDS stands for relational database service.

RDS or roll your own – which is better? Here are some use cases to help you decide.

o What is multi-az?

Amazon’s infrastructure offering isn’t just a single datacenter with servers. The beauty of what they’ve built is that they offer a number of datacenters (called availability zones) in each of many regions such as Northern Virginia, Oregon and Singapore.

Incidentally multi-az is a key feature to how businesses can protect themselves from failure. Amazon recently had an outage, but AirBNB, Reddit & Foursquare didn’t have to fail.

o What does a CDN do? How does it work? examples?

A CDN is a content delivery network. Remember all those files that make up a webpage? Images, video, css files? Turns out serving these components from servers *closer* to your customer, make their webpages load much faster. CDNs are networks of servers that hold the content of your pages, and serve them faster.

It works by replacing content paths with a special one from your provider. A simple change in your code will allow content to dynamically load from across the web. Cool!

CloudFront is Amazon’s offering coupled with S3 for file storage. Akamai is another big provider.

We’re not done yet. In part two on deployments and http://www.iheavy.com/2012/11/01/cloud-deployment-interview/”>part three of this series, we’ll hit on other important skills a cloud ops expert should have including scripting, database administration (Our MySQL Interview Guide), scalability, performance, configuration management, metrics, monitoring, and some all important war stories!

Here are some questions to pique your interest:

o Why does the API battle between Amazon & Eucalyptus (FOSS) matter?
o Do you use command line tools? why?
o What can go wrong with backups? how do we test them?
o Should we encrypt filesystems in the cloud? what are the risks?
o Should we use offsite backups?
o What is DRBD?
o Why is auditing important? access control?
o What is load balancing? why is it difficult with databases?
o How do you perform a benchmark? perform load testing?
o Why use a package manager? can we install from source?

Our Deploying MySQL on Amazon EC2 Guide is also related to this interview process.

You may also jump to part two Cloud Deployment Interview or the last part three Cloud DBA, Architecture and Management Interview.

Read this far? Grab our newsletter – startup scalability.