Is AWS too complex for small dev teams & startups?


I was discussing a server outage with a colleague recently. AWS had done some confusing things, and the team was rallying to troublehsoot & fix.

He made an offhand comment that caught my attention…

AWS is too complex for small dev teams. I’d recommend we host in a traditional datacenter.

Join 32,000 others and follow Sean Hull on twitter @hullsean.

It’s an interesting point. For all the fanfare over Amazon, lost in the shuffle is the staggering complexity that we’re taking on. For small firms, this is a cost that’s often forgotten when we smell the on-demand cool-aid that is EC2.

Here are my thoughts…

1. Over 70 services offered

Everytime I login to the AWS console there’s a new service offering. Lambda & serverless computing. CodeDeploy, Redshift, EMR, VPC’s, developer tools, IOT, the list goes on. If you haven’t enabled MFA on your IAM accounts you’re not alone!

Also: Is Amazon too big to fail?

2. Still complex to build high availability

The song I hear out of Amazon is, we offer all the components for a high availability infrastructure. multiple availability zones, regions, load balancers, autoscaling, geo & latency dns routing. What’s more companies like Netflix have open sourced tools to help.

But at a lot of startups that I see, all these components are not in use, nor are they well understood. Many admins are still using Amazon like an old-school datacenter. And that’s not good.

Sometimes it seems that AWS is a patient in need of constant medication.

Related: Are we fast approaching cloud-mageddon?

3. Need a dedicated devops

As AWS becomes more complex, and the offering more robust, so too the need for dedicated ops. If you’re devs are already out of bandwidth, but you don’t quite have so much need for a fulltime resource a consultant may be an option. Round out the team & keep costs manageable.

If you’re looking for an aws solutions architect, we can help!

Check out: Does Amazon eat it’s own dogfood?

4. Orchestration involves many moving parts

Infrastructure as code offers the promise of completely versioning all your servers, configurations and changes. From there we can apply test driven development & bring a more professional level of service to our business. That’s the theory anyway.

In practice it brings an incredible number of new toolsets to master and a more complex stack besides. All those components can have bugs, need troubleshooting. This sometimes just kicks the can down the road, moving the complexity elsewhere.

It’s not clear that for smaller shops, all this complexity is manageable.

Also: 5 things toxic to scalability

5. Troubleshooting failed deployments

I was looking at a problem with a broken deploy recently. Turns out a developer had copy & pasted some code solution off the internet, possibly from a tutorial, and broke deployments to staging.

Yes perhaps this was avoidable, and more checks & balances can fix. But my thought is continuous integration & continuous deployments are not a panacea. More complexity brings a more complex web to unweave.

I sometimes wonder if we aren’t fast approaching cloud-mageddon?

Read: Why Airbnb didn’t have to fail?

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

  • lawrencekrubner

    I remember this conversation. I’ve just written an expanded version in which I add more details:

    “AWS is inappropriate for small startups because its complexity demands a specialist”

    • Sean Hull

      Good stuff!

    • Sergey Novgorodsky

      What’s the alternative? IMO, if you are creating a SAAS, you have no choice – it has to be cloud-native from the very beginning. Everything you do at a tech startup demands a specialist ๐Ÿ™‚ So one of your ‘specialists’ has to know how to use AWS too, or you learn AWS as you go. I’ve been in this situation when I was a CTO/co-founder, and I just had to figure out AWS by myself

      • Sean Hull

        I hear you Sergey. I mean there are alternatives outside AWS. Though these days few are discussing those.

      • lawrencekrubner

        Sergey, I would say that if a small startup only needs EC2, Route53, S3 and RDS, then it should not be on AWS at all, because dealing with AWS means dealing with all of AWS, and that is a lot of cognitive overload when you only need the basics. There are a lot of other companies (other than Amazon) that give you just the basics, without having to worry about all the other stuff. If you saw the conversation on Hacker News, then you probably saw a lot of people replied to me with some version of “AWS can be very simple if you just use the basics and ignore everything else.” But “ignore everything else” has a mental cost. You have to first learn what “the basics” of AWS is. You have to learn that there are 5 services that you can use, while ignoring the other 70 services.

        I am not sure what kinds of startups you’ve worked at, but I’ll point out your sentence “Everything you do at a tech startup demands a specialist” can not be true. People at startups wear many hats, which is to say they are forced to operate as generalists. While a startup might have one or two domain specific experts, it’s impossible to hire a specialist for everything, when the team is small. Only a large corporation, with a huge team, could possibly hire a specialist for each task.

        Last year I worked at a startup that used Natural Language Processing to allow salespeople to write plain English sentences that we then parsed into the API of Salesforce, thus allowing salespeople to use English to talk to their Salesforce databases. We had a team of 3 people: an iPhone developer, an NLP developer, and me to do everything else. I had to build the API that connected the iPhone to the NLP engine, and I had to build the Salesforce integration. I also had to handle devops. And this is a common situation in every startup that I’ve worked: there is no devops guy. There is simply a lead engineer who knows basic Linux devops. And AWS is inappropriate, because AWS devops is not standard Linux devops. Therefore, as I said in the title of my essay “AWS is inappropriate for small startups because its complexity demands a specialist”.

        AWS becomes appropriate as soon as a company is big enough to hire a devops specialist.

        • Sergey Novgorodsky

          I’ve worked in three different startups over the past six years, all startups initially were 3-4 person teams. In every case we used AWS from the very beginning. There was no a full-time devops guy, as you noted, in small startups people have to wear many hats and so we either shared AWS management responsibilities among the team members or one person would spend more time dealing with devops. I do think that for someone with general knowledge of Linux devops, networking, etc it wouldn’t be too overwhelming to start with ELB, EC2, Route53, S3,RDS on AWS. Yes, of course you need to know that those are the basic services that you have to use, but I think you can figure it out spending a couple of days going through the AWS tutorials and docs. I can see, though, how in a situation when you are working on a POC or just want to roll out MVP as quickly as possible, you may want to start with PAAS, like Heroku – this makes sense to me. I think that basic AWS knowledge will soon become a required skill for any startup generalist

          • Sean Hull

            I think Serverless technologies like Webtask & Lambda are falling nicely into this sweet spot as well. For MVP building, it really simplifies things.

            At the Serverlessconf a few weeks back I heard the guys from say their ENTIRE site is on lambda now.

          • Sergey Novgorodsky

            Yes, Lambda is probably one of the most exciting services added to AWS recently. I think it may eventually replace most of the current EC2 + ELB deployments. And it’s cheaper than running EC2 instances too ๐Ÿ™‚

          • Taylor Kaplan

            In trying to work with lambda, which sounds great on paper… but my initial work with it feels like it’s still too limited beyond simple crud apps or small tasks.

            Debugging lambdas imo is not straight forward. Its easier to just manage the servers and develop with docker. + I hate vendor lock.

          • Sean Hull

            Agreed. It’s a bear to debug right now. Have you tried working with serverless framework?

            I suspect that Amazon is putting a lot of R&D into tools & instrumentation to allow better debugging & troubleshooting of lambda.

          • Taylor Kaplan

            If you mean aws-lambda-express as a serverless framework, ya I work with that. If that weren’t available I wouldn’t bother with it at all.

            Keep in mind that cost is the only reason I would ever consider lambda. Learning a proprietary framework just for serverless for one vendor makes me nervous and adds more unnecessary tools to the stack.

            Looking back at software dev. introducing these new concepts makes development 10x more difficult and has not made operations or dev any easier imo. Kubernetes is really the way to go.

            Furthermore, I’m incredibly uneasy about amazon’s predatory business model that competes with every customer and will consider alternative cloud or hybrid cloud options in the future.

          • Sean Hull

            Serverless Framework is here:

            There’s a channel:

            I wrote a 5min howto:

  • Ron Barak

    If youโ€™re devs are already out of bandwidth -> If your devs are already out of bandwidth

    • Sean Hull

      Thx Ron!

  • Ariel Perez

    This is why I’ve opted to go down the Heroku route for my applications. I have two junior engineers and myself, and the level of complexity in AWS can become unmanageable without a dedicated DevOps person. I may end up paying more for PaaS and the plug-and-play Heroku provides, but there’s quite a bit of peace-of-mind involved with understanding our environment and architecture.