Categories
Cloud Computing Cloud Migrations CTO/CIO

Should we right size instances in the public cloud?

via GIPHY

I’m a big fan of Corey Quinn’s Last Week in AWS newsletter.

Recently he wrote a piece titled Right Sizing your instances is nonesense

Not to be outdone, a blogger Joe at Sunshower.io wrote a counterpoint piece Why Right Sizing instances is not nonsense.

Join 35,000 others and follow Sean Hull on twitter @hullsean.

So what’s the verdict here? Is Corey wrong and Joe right? Or is Joe right and Corey wrong?

I would argue it depends. Corey’s piece emphasizes the big picture, essentially that technical changes can buy more trouble than the money they save. While

1. Corey emphasizes the 300 foot view

Corey’s article uses a broad brush, and in doing so some of his specifics are incorrect. For example the point about older versions of Operating Systems and hypervisor. In most cases your OS won’t know it’s running on a hypbervisor at all, so this seems a very rare edge case indeed.

That said his big picture conclusion seems spot on. Changing instance sizes can be a huge risk if you don’t do so regularly. With legacy apps, who knows how they might behave. In this you carry the same burden as changing instances at all. Your AMI may not be ready, or you may have had some manual steps to build the box again.

Related: How can we keep cloud architectures simple

2. Joe gets down in the weeds of technical specifics

The first thing about Joe’s post that caught my attention was to use the console to change the instance size. You shouldn’t be manually changing instances through the console to start with. What, everything is code, all the way down? Hehe…

Also his points about cost savings seemed cherry picked. If you can save that much money by changing instance sizes, it is well worth the cost to do regression, integration & disaster recovery tests to make sure it will all work. Get on it!

Also: What hidden things does a deposit reveal?

3. Use infrastructure as code, test & retest it

IAC is really the way to go. But that still doesn’t mean you’re out of the woods. Be cautious when changing instance sizes!

With code, there is testing. So change your Terraform code and then verify it works. Now just because you have a variable indicating the instance size, doesn’t mean changing it won’t break something.

Read: Can communication mixups sour an engagement?

4. Weigh the cost savings with the risk of breaking things

Changing an instance size and redeploying could break all manner of things. It’s possible you used a variable for the instance size in some places and hard coded it in others. Or made some weird reference in an autoscaling group.

It may be the AMI you’ve built works on one type of instance but not another. Or that your AMI is deployed in one region but not another. Or that your old instance size is available in us-east-2, but your new instance size is not yet available there. Yes the console wouldn’t have offered it, but your Terraform code didn’t know.

Check out: How I use 5 daily habits to stay on track

5. Put up some guiderails

As you’ll see further down in the comments, Joe suggests to put limits around instance size changes. That makes sense. After you’ve done testing, you’ll have an idea of what these limits should be. No instance size less than 30GB memory? No instance size greater than X. None in region Y. Etc.

It’ll require tweaking your terraform infrastructure code, so it’s not a free change. But it’ll pay dividends if your costs savings are in the thousands.

Also: Can daily notes help you work better with clients?

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Categories
All Cloud Computing CTO/CIO High Availability Startups

Are you as good as the public cloud?

via GIPHY

According to Lyft’s recent public filing, they plan to spend 300 million buckaroos in the next 2.5 years on AWS.

Did I hear that right?

Join 38,000 others and follow Sean Hull on twitter @hullsean.

Perhaps that is their estimate, or the maximum amount they want to budget for. Regardless that’s a lot of money any way you slice it. A lot of folks are commenting about how crazy that is, and how much datacenter you could build yourself with that much money.

What do you think? Is it foolhardy? Or is there a hidden wisdom here?

Here’s my take.

1. Do you have one million customers testing your datacenter?

If you’re comparing the cost of the cloud to the raw numbers of running your own datacenter, the hardware costs are not enough. You’ll need to include the ops teams & other engineers. Right, you probably guessed that.

But did you factor in the costs of a legion of testers. This is the hidden cost that commercial software carries, even while open source software gets this benefit for free.

With a public cloud like AWS you have millions of customers testing the product everyday, and running into edge cases long before you do. So you get a better service, that’s more reliable, all invisibly for free.

Related: How can we keep cloud architectures simple

2. Do you have 66 datacenters spread across 21 regions and a free network between them?

Anybody who was building web applications in the year 2000 will remember how websites didn’t load the same for different customers. Depending on where in the world they were located, they could experience a very different user experience.

These days we assume that we can be global from day one. But how exactly do we achieve this? Remember with a public cloud, you’re getting tons of things for free, without knowing it. Moving data between AZs or regions? That’s all going across a private interconnect.

And that’s not even including the 180 nodes inside cloudfront that give you a global CDN footprint too!

Also: What hidden things does a deposit reveal?

3. Do you have an engineering team automating away job roles?

I remember the days of DBA job role, do you? Probably not. I specialized in this for years, and there were tons of companies hiring me to help them with it. First Oracle, then MySQL, then Postgres.

Then along came Amazon RDS. Guess what, companies don’t really hire for that role anymore. They do need help with it from time to time, but not as a primary specialization.

What do I mean? Well by hosting your application on AWS, you’re benefiting from the work of teams of engineers in different departments, all expanding on APIs and automating things that those one million customers are asking for.

You’re not going to be able to innovate that well and that quickly in your own datacenter. So you’ll pay more!

Read: Can communication mixups sour an engagement?

4. Do you have APIs that tons of engineers have already written code for?

A quick peek at Terraform’s community modules on Github and you’ll probably blush. From VPCs to bastion boxes, key management to load balancers, lots of code has been written and open sourced.

By deploying on a platform that a lot of other devs are using, you’ll benefit from all this open source code. That means you won’t have to write that stuff yourself.

Sure you’ll have integration work to do, but the hidden benefit of being on a popular platform saves you money.

Check out: How I use 5 daily habits to stay on track

5. Can you do disaster recovery for free?

If you build your own datacenter, you have to buy all your capacity. So there are no spare servers sitting around waiting for your use. In the public cloud there is always spare capacity.

What that means is you can write automation code to spinup copies of your application stack in alternate regions, at the push of a button. Thus you effectively get disaster recovery for free!

Also: Can daily notes help you work better with clients?

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Categories
All Consulting CTO/CIO

How can communication mixups sour an enagement?

via GIPHY

I recently had some communications mixups with a customer. It reminded me how delicate, communications are between customers & vendors. What’s more they can be challenging between developers & managers. It highlighted for me these challenges, and the strategies I’ve learned over the years.

Join 38,000 others and follow Sean Hull on twitter @hullsean.

While I didn’t lose the project, the initial misunderstandings continued to eclipse the project, long after they were cleared up.

1. First a missed conference call

Early on, we setup a call to discuss the challenges. The time of the conference call had been agreed to, but somehow it didn’t make it into my calendar. So when the appointed day & time came, i missed the call. This was before any contract was signed, or even the engagement had gotten started.

Needless to say this is a very delicate moment, as everything we do sets precedents about our personality and working style.

While we were able to reschedule, it added some initial strain to the relationship. As you’ll see that compounded more later.

Related: Walking the delicate balance of transparency

2. Next arriving late to the kickoff meeting

I always pride myself on timeliness. I think it communicates all sorts of things to customers. First it shows you’re serious and will manage the project carefully. Next it shows you respect for others time.

As usual, I left plenty of extra time, so I would arrive well before the meeting. Arriving at the building 20 minutes early, I searched but could not find the entrance. Neither could google as it turns out. Strange I thought, what could be wrong? I walked into the building where the address should be, and asked the doorman. He explained that the company didn’t reside there. Perhaps they’re not located at Park Avenue, but rather Park Avenue South, he suggested. And then the lightbulb goes off. Of course!

Realizing I now have 5 minutes to arrive on time, I’m going to be late. So I attempt to call the manager leading the meeting. I get his voicemail, and leave a message. I then jump in a taxi, and head to the Park Avenue South address. Arriving 10 minutes late, I quickly head upstairs. I’m greeted by some grumbling, and frustrated looks.

Despite this being an understandable mistake, it comes on the heels of another mixup. So now I’ve set a precedent of lateness. Despite being a timely person, it’s hard to erase the stamp that is there now.

We continued to have strained relations through the engagement. While it did finish to completion, I believe it would have gotten extended were I not to have stumbled early on.

Also: When you have to take the fall

3. What can a mixup indicate?

There are many questions it may raise. Possible ones include:

o Is candidate too busy with other tasks?
o Is the person forgetful?
o Is one party bullying on their perspective?
o Is there finger pointing & blame game in the org?
o What is the culture of the organization?
o Is it one of understanding & working together or blame game?
o Is the person uninterested?
o Is the project not a priority?
o Is the company disorganized
o Is miscommunication endemic?

Some of these thoughts may bubble up consciously, and some may linger as a bad taste in your mouth. Regardless, they should be faced head on, with understanding and humility on both sides.

Read: Why i ask for a deposit

4. The weight of first impressions

Inevitably, when there is a mixup, of lateness or missed meeting, there is a technical explanation. In my story above, the *reason* is Park Avenue and Park Avenue South are completely different addresses.

o First impressions are KEY

Even with a reasonable explanation, there is a reaction that is felt.

o There is a visceral emotional reaction we all have anyway

Such a reaction is easy to cause, but hard to patch up. It will take time, and multiple interactions to set a new impression to people.

o Reactions can be incorrect & irrational sometimes
o They can color further interactions

With time impressions can be adjusted, but it takes much more work after an initial mistake.

Check out: How to hire a developer that doesn’t suck

5. Possible solutions

While there is no sure fire way to avoid mixups like these, there are some things that can work in your favor.

o maintain flexibility

That means accepting blame, and mutual responsibility in reaching the goal posts.

o maintain a sense of I *can* be wrong

Everyone can be wrong, and everyone makes mistakes. So don’t try to avoid blame. That said emphasize that everyone must work together. On communicating engagement details, on mutual agreed times, and time zones.

o look for a sense of we *can* be wrong

I think these types of mixups can also be beneficial. For they underscore the customers management style. Do they point fingers, or acknowledge reasonable mistakes. Both parties will make mistakes eventually, and understanding of this builds good faith down the road.

o “let’s work together to improve communication”

Framing the mixup as a shared problem is important. Although the address mixup above is technically my fault, it’s probably a common one. Park Avenue South confuses everyone in New York. So an understanding customer might offer to share a bit in this with you.

o hold frame of mutual responsibility and working together using the word “we”

The frame is key. It’s not *all* your fault, nor is it the customers if they mixup. We all need to be understanding, to a point.

Also: Can daily notes help you work better with clients?

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters