Can misfits teach us a thing or two about innovation?

I just finished reading Alexa Clay & Kyra Maya Phillips tour de force, The Misfit Economy.

(Yes that’s an affiliate link. The first one I’ve ever posted on this blog. If you like the book, please 🙂 buy through my link. )
I have to admit I was surprised & delighted by the book.

Join 38,000 others and follow Sean Hull on twitter @hullsean.

Alexa & Kyra offer us a tantalizing question. Could it be that we could learn a lot from oddball innovators at the edge of the economy? When I say edge, I really mean it. She interviews Sam Hostetler who is building a business around milking camels, and then there’s Abdi Hasan a pirate from Galkayo northern Somalia. Yeah really! Or what about the German copycats Wimdu who built a complete replica of Airbnb by reverse engineering it.

1. Hack the cold call

Take the example of Lance Weiler. Early on the industry was very against digital. They didn’t see it as really making films.

“Part of [Lance] Weiler’s success was due to his ability to work the system. He wrote letters to major production companies telling them he wanted to make the first digital motion picture. After he didn’t hear back, he took a page from the con man’s handbook and wrote the same letters but intentionally misaddressed them so they were sent to the wrong companies. Sony for example would get a letter intended for Barco.”

He was later able to bring digital projection to Cannes & Sundance!

“For Weiler his big epiphany was when he realized he could be creative across all of it [the business]. Not just in the art product, but in financing, distribution, and business aspects of artistic production.”

Related: The art of resistence or when you have to be the bad guy

2. Copy the product

The german brothers Oliver, Marc & Alexander Samwer make a superb example of how copying can bring building prowess to compete against innovators that were first to market.

“in 1998 Marc Samwer had an instinct that eBay would thrive in the German market… his brothers agreed… they contacted eBay via email numerous times, recommending that the company replicate it’s platformin Germany. Claiming that eBay failed to respond, the brother’s started their own German-language auction site, Alando, which was then purchased by eBay for 38 million euros (over $50 million) only 100 days after it’s debut. Had the Samwers not copied, eBay might have remained complacent, not realizing its potential within the german market.”

Although not mentioned in the book, Inditex the wildly successful firm behind fashion brand Zara did much the same thing to the fashion industry. By mastering the supply chain, they enabled their company to take designs from the runway & replicate them, turning designs into real clothing in stores, in just two weeks! And indeed they really do replicate, borrow & straight copy those designs from what they see at fashion week. Sad & brilliant at the same time.

Related: When you have to take the fall

3. Don’t forget to hustle


“In the lexicon of the Misfit Economy, we define “hustle” as making something out of nothing. To move fast, to trade one thing for another, and to proactively create your own opportunities rather than waiting for opportunity to come your way. To hustle means getting your hands dirty, being lean and facile, working hard, being resourceful and resilient, and showing or having gumption, chutzpah, or mojo.”

And after all, isn’t that everything the startup industry aspires to? Agile teams? Growth hackers? Scrappy startups & innovation?

Related: When clients don’t pay

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Can daily notes help you work better with clients?

via GIPHY

Years ago I was working at a customer for a few weeks. There was some confusion as to what was going on, in terms of progress. Things weren’t moving as quickly as they expected.

After a lot of back and forth, I suggested I could provide detailed notes of what I had done.

Join 38,000 others and follow Sean Hull on twitter @hullsean.

After I put together my in-depth notes the customer was really happy. It seems these notes had highlighted a few problems that they didn’t know about. What’s more they even highlighted some people issues, where communicate was blocked. Whats more the notes underlined what I was doing, and this really improved the customers confidence in the work product.

1. Visibility

Keeping daily notes is a habit I found useful over and over again. If your client or customer comes to you and says, why are we paying $X, you can provide the notes as a detailed explanation of what they have gotten for their money.

Related: Are generalists better at scaling the web?

2. Transparency

Transparency is a door that swings both ways. As I mentioned above it can be great when the customer is not sure how much work was done, or what the bill is for. But it can also highlight things they may not want done. For instance perhaps you were investigating a problem authenticating to a server. You determined that it was an important piece.

When the customer sees this in your notes they may say “Oh we don’t need to deal with that system. Please leave it alone” or they may say “We actually have Rakesh available to help us with that piece, so please communicate with him and he can resolve that”.

Related: When clients don’t pay

3. Trust

Most important of all, keeping detailed notes helps build trust. Many customers, hiring managers & CTOs are not command-line technical. And that’s perfectly normal. However looking over a long list of notes like these provides great insight to them as to what you do from day-to-day.

Do they need to know what every line means? No. But the visibility goes a long long way toward building trust in the consultant client relationship.

Related: 5 conversational ways to evaluate great consultants

Week 1 April 1 – April 10

Here’s a sample of the kind notes I keep. Actually they cover a ten day period, but that’s because the initial day was towards the end of a week.

Friday April 1st
o coord with Jake on getting started
o dropbox for password, creds & server docs
o reviewing system network diagram
o reviewing techlist excel doc
– techlist
– server list & access
– database access
– projects -old
o reviewing systems access.docx
o testing AWS login credentials
– issue with permissions
– coordinating with Jake on Admin access
o testing AWS creds again
– access to all AWS services
– IAM for seanhull user
– enabling MFA for user
o questions for outgoing op Roger

Sat April 2nd (no hours)

Sun April 3rd (no hours)

Mon April 4th
o coord with Jake to get onboarded
o sending W9 form to Acme Inc.
o setup slack
o plan for today
– review aws servers
– review dg servers
– questions for Roger
– review docs
o coord with Roger on VPN access
– reach out to Larry
– emailed Larry CC Jake
– Larry requests Acme access CC to mgmt
– turns out VPN access isn’t required
– can just whitelist IP inside the relevant security groups
– coord with J, going ahead to add whitelist 1.2.3.4/32
o updating Acmemedia-sandbox security group
– trying to reach host, coord with Roger
– asked to drop ssh key onto servers
– asked about .ssh/config file – Did you get from Jake?
– found the AWS PEM folder that I overlooked 🙂
o configuring .ssh/config file
– copying up to iheavy.com
– setting permissions 600 on pem files
– ssh to sandbox successful!!
o adding whitelist to Acmemedia-prod security group
o updating Jake – access is working

Tue April 5th
o coord with Jake on todo list for today
o verifying mysql access
– review security groups
– no whitelisted IPs
– can reach from webserver?
– test db1 MySQL access via webserver, OK
– test db2 MySQL access via webserver OK
o reviewing monitoring system
– testing nagios access
– locating configurations
– reviewing dashboard
– understanding tests
– down system db1 – 108 days – why?
– down system p1 – LB1 sailthru check down for 85 days why?
– down staging – 174 days why?
– emailed nagios questions to Roger
– request to add me to nagios notifications group
o coord with Roger on questions
– nagios setup & stopped checks
– add to admin group
o github access for sandbox details doc
o login to Acmemedia wp
– check list of 25 plugins
– review recent backups on abc (8)
o login to DDD wordpress
– check list of 33 plugins
– review recent backups in abc (8)
o login to EEE wordpress
– check list of 31 plugins
– review recent backups in abc (37)
o login to FFF wordpress
– check list of 35 plugins
– review recent backups in abc (8)
o login to DDD
o login to EEE
o login to FFF
o emailed Roger – request details about Glasgow server
o review various Acme github pages

Also: The art of resistance or when you have to be the bad guy

Wed April 6th
o coord with Jake on todos for today
o reviewing github pages docs on various system processes
– git deployment server page
– git deployment process
– new deploy process Nov 2015
– wiki pages are a bit sparse overall
o tested jenkins login
– found API cache clear
– found varnish cache clear
o understand separation of dev & production
o digging into Jenkins docs
o understanding build process
o tried login to EEEv2 wp login, don’t have pass
– coordinating with Jake on that login
o checking on nfs disk full nagios alert
– can’t reach box
– notified Jake & Roger via slack
– slack with Lester
– yes nfs01 space 90% is normal
– new launch of EEE tomorrow & old stuff will be deleted then
o updating nfs security group
– ssh login working now.
o getting diskspace error on prod04
– messaged Lester, related to EEE launch tonight
o email from Jake – local dev & test environment setups are slow
– very overengineered for simple wordpress site
– not using multisites, so have FOUR SEPARATE setups
– different plugins on each install
– four sets of logins
– four places to update
– four places to test/qa
– migration may be complex based on custom Acme plugins
– shortcodes compatability across four sites
– not using ithemes security plugin
o discuss with Lester on slack
– API is hosted on datagram
– single point of failure for the site currently
– outage there would take the site down
– migrate to AWS using internal loadbalancer & webservers in 2 AZs

Thu April 7th
o call with Jake on EEEv2 launch today
– general observations of Acme sites & architecture
o reviewing access.Acmemedia.com
o discuss with Jake
– hosting media files on S3 vs nfs
– using multisite
– using wordpress through API only
– javascript based static site builder
– moving API to amazon EC2
– create slave MySQL db of master MySQL currently in datadotnet
o discuss with Roger
– launch plan
– two vhosts new.EEE.com
– old.EEE.com
– simply restart apache to enable switch
– refresh maxCDN after launch
o review EEEv2 deploy steps
– pre-deploy steps
– DNS for old.EEE.com
– add vhosts EEEv2.conf
– restart apache
– restart varnish
– clear maxcdn
o verified login to access.Acmemedia.com
– API log is in /var/log/httpd/production-access.log
– login as sandy & root
o not able to login to dashboard.Acmemedia.com
– tried admin & pass in datagram docs

o meeting onsite with Jake & Roger
– discuss deployment process
– discuss legacy systems
– discuss NFS vs S3 for media files
– discuss plugins & management
– discuss wordpress version upgrade process
– discuss plugin version upgrade process
– discuss Jenkins access, configs, success & error logs
– discuss managing secrets file
– script that takes webserver out of load balancer while apache restarting
o met Rachel, Louis, Lester, Rick, Stuart, Jack

Fri April 8th
o testing Acme stage build
o emailed Roger further questions
– where is secrets file configuration & process
– composer is PHP dependency management
– what are the steps to upgrade plugin only
o summarizing & notes on Acme
o put together steps for complete firedrill
– questions for Roger, requesting help with process
– build webserver with varnish & apache
– should setup separate NFS server
– should use Acmemedia.com bc it uses API heavily
– setup copy of API server & db
– setup mysql instance for wordpress
– setup amazon cloudfront for content
o outline additional questions for Roger
– how to upgrade plugin only
– composer for php dependency management
– how are secrets files managed & deployed outside developer access
o secrets management
– asked Roger for clarificaiton
o plugin-only installs
– reviewed jenkins configs
– various questions to Roger
– composer:install seems to be the key change (not just deploy which does all?)
– why is STAGING PLUGIN DEPLOY for ORIM different?
o what happens when github account is disabled!!
– jenkins changes for new github deploy account
– THIS WOULD BREAK ALL DEPLOYS & CI/CD pipeline
– capistrano changes?
– any other changes on sandbox
– any other dependencies for Roger github?
o email step-by-step outline to add a plugin
– reviewing steps with Roger
– making sure no missing pieces

Sat April 9th (1 off-hour)
o receiving nagios alert for p1
o emailed Roger, Jake about issue
o slack messaged Jake
o raises question about off-hours coverage

Sun April 10th (2 off-hours)

o p1 still throwing errors
o coordinating with Lester & Ralph on Slack
– reiterated this is *not* an issue with NFS
– because of large number of nagios alerts, p1 lost in the shuffle
– p1 is new error, 97% so more dire than the NFS issue
– Lester attempting to login, fails because of AWS security group
– adding his *own* home IP as whitelist (devs have access to AWS console)
– first time logging in from home?
– Lester deleted old DDD logfiles to clear up 1.2G
– plan to touch base again tomorrow about issue
o emailing Jake about status
o questions for Jake
– how to manage on-call & alerts
– how to manage developer access
– Roger mentioned secrets files are not shared with devs
o Lester questions, comments on servers & diskspace

Related: When you have to take the fall

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Can i get more done by taking some dream time?

via GIPHY

When I have a long todo list and a million things on my plate, my usual tactic is to just plow through it. Take short break to eat, but then get right back to work. My feeling is, if it’s weighing on the back of my mind, I won’t enjoy downtime anyway.

Join 38,000 others and follow Sean Hull on twitter @hullsean.

Recently though I had a very different experience. And it surprised me.

1. Too much to do

A colleague of mine asked me to meetup for beers. We planned to talk technology, and to catchup on what we were both working on.

As the night rolled on he had some delays, and I wanted to cancel too. After all I had a ton of work to do, and didn’t think I would enjoy myself. I really felt like I’d be worrying about all this work on my plate. It’s like taking a vacation when you have a deadline. It doesn’t feel quite right.

Related: Can a growth mindset help you recover from setbacks?

2. My surprise

We ended up meeting anyway. At first I wasn’t totally relaxed, but then we started talking.
Our conversation turned to the evolution of datacenters. How they used to be on premise, then there were lots of hosting companies. And then Amazon changed everything!

We talked about evolution of tooling & automation. Although system administrators of old have been writing bash scripts forever to make their jobs easier, the proliferation of tools for deployment has allowed smaller ops teams to control fleets of servers. As my friend & colleague was newly starting a job on Amazon Web Services, a lot of this cloud stuff was new to him. So talking about it from a teaching vantage point, made me realize how strong I was in a lot of this stuff.

We talked about docker & containerization too. Even the origins back in the late 70’s with Unix chroot all the way up to Docker today. I explained to him that he could think of a container almost like a unix user, but with a more self-contained view of the whole system. In many ways a container acts like a vm, with it’s own filesystem and processes.

We talked a lot about aws, how S3 was an evolution of FTP in the old days, but much much better, how VPCs worked and the virtualization of networking, how VMs in the AWS world match with bare metal or not, how they share EBS storage. How Amazon has built a database service RDS around popular platforms like Oracle, MySQL & Postgres.

We shared a lot of ideas & brainstorming. About coding, C versus Java versus Python, package management, dependencies and on and on. He also mentioned he needed to build a test script to talk to an Amazon queue. I explained that it should be quite easy, and which libraries to look for.

Related: How I use terraform & composer to automate wordpress on AWS

3. Breaking through hurdles

It’s funny how dramatically different I felt after we got together. I all of a sudden had tons of new ideas bouncing around in my head.

Instead of waiting for the next day, after our get together, I went straight to the terminal. I quickly finished a coding challenge I was working on and struggling with. Easy peasy!

After that I felt inspired further. I created an Amazon SQS queue with the dashboard, and then wrote some python code to talk to an Amazon sqs

I created a git repo & checked in my code. All within a couple of hours!

I was just sitting there laughing. Because I felt such relief that I’ve made progress.

It was a big surprise that such a circuitous route got me there.

I guess the takeaway is that mental play or dream time is important to making progress. Otherwise you’re just working in a vacuum!

Related: What I’ve learned from 10 years of blogging

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Can a growth mindset help you recover from setbacks?

I just finished reading Carol Dweck’s tour de force, Mindset: The New Psychology of Success.

Join 38,000 others and follow Sean Hull on twitter @hullsean.

In a word, her book is mind blowing. How can a little thing like mindset make a difference? Can changes in thinking & attitude really have a profound impact on success?

I think it most definitely can. Here’s my story & more excerpts from the book.

1. My own story with the growth mindset

If your memory stretches back to 2008, you will like recall when the market took a dive. Everybody was nervous about default. Big banks were failing. The hiring climate became like a nuclear winter. For a good six to twelve months, things were frozen.

As an independent consultant, that felt like a real shock. Where once there were a lot of firms hiring me on projects, suddenly everything was quiet.

At first I thought of different options. I could weather the storm from many months, but then what? I decided I was commited to consulting, and didn’t want to take a fulltime role. So what did I do next?

Well I picked up a copy of Alan Weiss’ guide, Million Dollar Consulting. I read it cover to cover in a day, then I took a look at the business.

When the market is climbing, and demand is surging, I experimented with increasing rates. Sometimes they were too high, but often I would read the demand right, and turn a bigger fee. Now it was the reverse. Time to swallow your pride & drop those rates! And so I did. This helped close more deals.

From there I just dig in my heels. I worked my network as best I could. Having worked at hundreds of startups, I’ve met hundreds more colleagues over the years. I started reconnecting with them in emails, at meetups, and over a beer or coffee. I took more calls with recruiters to feel out the market, and keep my ear to the ground.

All of this paid off. Within a year I was rolling again, when for many the market still seemed frigid. A learning experience indeed, about business, but also about the growth mindset. It works!

Related: Why does Reddit CTO Martin Weiner advocate boring tech?

Jack Welsh, Michael Jordan & Setya Nadella vs Lee Iococca & John McEnroe

Dweck overs numerous examples of great personalities, who exhibited different mindsets.

For example take a look at a quote from Jack Welsh. He approached things with a growth mindset. Failures are only an opportunity to learn, not a description of your character…

“He [Welsh] learned to select people: for their mindset not their pedigrees. Originally, academic pedigrees impressed him. He hired engineers from MIT, Princeton, and Caltech. But after a while, he realized that wasn’t what counted. ‘Eventually I learned that I was really looking for people who were filled with passion and a desire to get things done. A resume didn’t tell me much about that inner hunger.'”

Or Michael Jordan. You think he never failed until you look at his own words. We forget how much practice day in and day out, it took to create his mastery.

“Michael Jordan embraced his failures. In fact in one of his favorite ads for Nike, he says ‘I’ve missed more than nine thousand shots. I’ve lost almost three hundred games. Twenty six time I’ve been trusted to take the game winning shot, and missed.’ You can be sure that each time, he went back and practiced the shot a hundred times”.

Lately I’ve been seeing Carol Dweck everywhere. Take a recent Bloomberg Businessweek interview where Satya Nadella credit’s Dweck’s ideas for the culture he’s created at Microsoft.


Culture is something that needs to adapt and change, and you’ve got to be able to have a learning culture. The intuition I got was from observing what happens in schools. I read a book called Mindset. In there there’s this very simple concept that Carol Dweck talks about, which is if you take two people, one of them is a learn-it-all and the other one is a know-it-all, the learn-it-all will always trump the know-it-all in the long run, even if they start with less innate capability.

On the flip side, here are two examples where the fixed mindset takes hold. In Lee Iococca’s case, it drove him to harm others, and the company he was charged with driving…

“He [Iococca] also looked to history, to how he would be judge and remembered. But he did not address this concern by building the company. Quite the contrary. According to one of his biographers, he worried that his underlings might get credit for successful new designs so he balked at approving them. He worried as Chrysler faltered, that his underlings might be seen as the new saviors, so he tried to get rid of them. He worried that he could be written out of Chrysler history, so he desperately hung on as CEO long after he had lost his effectiveness.”

And another example of John McEnroe. A loss for him wasn’t a chance to learn something. He believed he had innate talent. He was special. So for this fixed mindset, a loss damages his character, and makes him feel humiliated.


“Here’s how failure motivated him. In 1979, he played mixed doubles at Wimbledon. He didn’t play mixed doubles again for twenty years. Why? He and his partner lost in three straight sets. Plus, McEnroe lost his serve twice, while no one else lost theirs even once. ‘That was the ultimate embarrassment, I said. That’s it. I’m never playing again. I can’t handle this.'”

Related: How I use terraform & composer to automate wordpress on AWS

Learning the growth mindset

Carol’s book provides example after example of the mindsets in action, in real people. Her chapters cover sports, business, and even love & relationships. Towards the final section of the book she talks about how to learn the growth mindset.

Catch yourself and your negative self-talk. Turn things towards a learning opportunity. Don’t allow failures to define you or your character. Always be growing!

Here’s a great page, summarizing the mindsets & how to get there!

Related: 30 questions to ask a serverless fanboy

Get more. Get monthly insight on startups & innovation. Straight to your inbox! Our latest Should SQL be retired?

What have I learned in 10 years of blogging?

via GIPHY

I was just reading Andrew Chen’s latest posting, where he distills many of the things he’s learned from blogging over a decade.

Join 38,000 others and follow Sean Hull on twitter @hullsean.

This reminded me that I’ve been blogging that long as well. And to be sure it has brought great benefits. In the way that public speaking gives you visibility, but also forces you to communicate better, form your voice, and so on.

All the great things you gain by talking to other people, and getting into the conversation.

1. Understand your audience

I struggled with this when I first started blogging. As any engineer might approach things, I thought I should publish technical material. What better way to show what I know. And further how I can help a customer.

What I didn’t realize is that all of your readers aren’t technical. So it goes a long way if you can appeal to a broader audience.

I found that my readers fell into a few big categories.

1. Fellow engineers & peers
2. Hiring managers & startup CTOs
3. Recruiters & other publishers

This really helped me divide up the types of content I would write, some directed towards each of the different audiences.

Related: Why does Reddit CTO Martin Weiner advocate boring tech?

2. Tell your story

I’ve written often about why I wrote the book on Oracle. In it I outlined a long arc of datacenter evolution which started with the maturity of Linux, and today provides the bedrock of the cloud that is Amazon Web Services among others.

What this also allowed me to do is tell my own history.

Related: 5 reasons devops should blog

3. Form your voice

Forming your voice is different than speaking to specific audiences. It’s about having opinions & getting into the line of fire. Being passionate about a subject, you’re sure to care & sit on one side or the other of a particular argument.

For example I argued the Android ecosystem was broken. Although Google has fixed some of these problems, many remain as a symptom of the platform itself.

I also argued with Fred Wilson’s estimation of Apple being overvalued. At the time in May 2014 the price was at $85. Now it sits comfortably at $177.

Related: How to hire a developer that doesn’t suck

4. Put yourself out there

Putting yourself out there isn’t easy. You’ll be open to criticism. And sometimes you’ll be wrong. But by challenging yourself in this way you’ll grow too. And prospects will notice this. More than engineering might, and power at the keyboard, your perspective of what’s happening in computing generally, and what is on the horizon is invaluable to customers.

Related: 30 questions to ask a serverless fanboy

5. Learn & Share

Writing howtos is a great challenge too. By forcing yourself to teach something, you in turn learn the material better. You become better at executing, and formulating solutions.

As you share knowledge, you’ll also learn from others. As the disqus.com comments on my site can attest. Sure you get much of this same value from having an active account on Reddit.com, but your own real estate carries even more weight for your personal brand.

Related: Why you should always be publishing

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

READ: Lanier’s Dawn of the New Everything

Jaron Lanier was the founder of VPL in the 90’s, a pioneering company around virtual reality.

I stumbled onto a recent Interview with Lanier about his new book.

Join 38,000 others and follow Sean Hull on twitter @hullsean.

Lanier is a creative thinker, one who has grappled with ideas like Ray Kurzweil’s singularity. He has even provided a counterpoint against cybernetic totalism.

His original perspectives come at the same time he sits within the circles of esteemed computer science researchers, providing a rare critical perspective from within the adademia.

1. Playing virtual instruments

Lanier wasn’t just a pioneer around virtual reality, but was also a musician. In the 80’s & 90’s he built & even *played* virtual instruments in VR, on stage.

I managed to catch one of these shows in the late 90’s at Knitting Factory back when it was at 66 Leonard street & Tribeca was still a rough area.

Related: Why does Reddit CTO Martin Weiner advocate boring tech?

2. A VR sport for the Olympics

Apparently the Olympics Committee commissioned him to build a sport in VR. Complete with simulated antigravity, and some other neat features.

Related: How I use terraform & composer to automate wordpress on AWS

3. An antidote to hysteria & pessimism

With all the hype & hysteria around fake news & backlash against the Facebooks & Googles of the world it’s easy to feel like maybe social media is tearing apart society.

With that I would heartily recommend Lanier’s new book, as an antidote to all of that. His writing is & ideas have always been well ahead of their time, and I expect this to be no exception.

I’ll followup with a review in the coming months.

Related: 30 questions to ask a serverless fanboy

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Is Alex Hudson right that software architecture is failing?

via GIPHY

I read Hacker News aka Ycombinator’s popular top 100. I never fail to find useful, surprising & stimulating reading there.

I recently stumbled on Alex Hudson’s software architecture is failing.

It’s very good, I recommend reading it.

Join 38,000 others and follow Sean Hull on twitter @hullsean.

But why did it grab my attention, you might ask? Perhaps I’m a naysayer. But I do find there is a lot of hype, and a lot of sex in software today. It’s as though the shiniest, newest, coolest toys are the ones getting the spotlight.

So when I find an alternative view, I sit up and take notice.

1. Are we making systems too complex?

Right out of the gates, Alex makes a great point:

“We’re not delivering quickly enough!”. “Our systems are too complex to maintain!”. “The application we delivered last year is completely legacy now but it’s too difficult to replace!”.

Our industry’s obsession with the newest & coolest toys, means we’re building things that don’t last very long. A real & ongoing problem.

Related: Why does Reddit CTO Martin Weiner advocate boring tech?

2. Smaller enterprises

One thing Alex pointed out that really struck a nerve was this:

For those in tech who are not working at Facebook/Google/Amazon, we’re simply not talking enough about what systems at smaller enterprises look like.

I couldn’t agree more. As a profession, we watch closely at what the big guys are doing. And that’s useful to a point. But for many smaller companies, to use such architectures would be over engineering in the extreme. Not to mention extremely costly!

Related: How I use terraform & composer to automate wordpress on AWS

3. Not bleeding & far from the edge

Another choice quote from Alex’s piece:


“It’s totally legacy, and no-one maintains it – it just sits there working, except for the occasions it doesn’t. The problem is replacing it is so hard, it’s got great performance, and the business doesn’t want to spend time replacing something working”. This is the problem being ahead of the curve – the definition of “success” (it works great, it’s reliable, it’s performant, we don’t need to think about it) looks a hell of a lot like the definition of “legacy”.

We know the term bleeding edge because it’s tough being out there trail blazing. Here I agree that sometimes legacy is also boring, yet eminently reliable.

Related: 30 questions to ask a serverless fanboy

4. Reduce, reuse, recycle

Should we build it or should we buy it? Here’s what Alex says:


I think we’re often getting the build/buy decision wrong. Software development should be the tool of last resort: “we’re building this because it doesn’t exist in the form we need it”.

Well said. Sure we should consider integration costs & testing. And using a service brings other things to balance. But it means we don’t have to own that code.

Better to focus on our business core competency.

Related: Is Amazon about to disrupt your data warehouse?

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Why I didn’t go postal at the postal service

I’ve had a mailbox with the postal service for over a decade. It’s been great. It’s a convenient address for a business, and it doesn’t change. Great for receiving bills, tax refunds, contracts, pay checks and everything else.

Join 38,000 others and follow Sean Hull on twitter @hullsean.

I picked the postal service years back because I figured they are solid. A corner deli or other mailbox service might go out of business. And the idea was to have a box that doesn’t have to change often.

In all those years I’ve always been timely with paying the rent, which can be every six months or twelve months as you choose. Recently I mixed up the renewal. When I went in to pickup my mail, the lock had been changed. I went to the customer service counter to ask about it. Amanda came to help me. She was cordial.

She explained however that the box had been closed. Since it was the day before the holiday, she wouldn’t be able to help with this, and suggested I come back on Monday.

As I like to be agreeable, I agreed to this request. In the meantime she was able to give me my mail. It was nice to see they had been keeping it. After all I was only a few weeks overdue.

Bring me the documents!

On Monday I had planned to return right away. But as my day got going I wondered if I would have enough time. This probably won’t be just a quick stop in. For some reason I sensed the process would be complicated.

Tuesday I set my alarm clock to wake up two hours early. I got dressed & headed straight to the post office. I wanted to get an early start. I went to the customer service desk again, and met Michael. I explained the mixup. Yes I was late playing my rent. Could I pay a penalty of some kind and then renew for another year?

“No I’m sorry, your box is closed now. You’ll have to reopen it.”, he explained.

“That’s no trouble let’s do that. What do we need to do?”, I asked.

“Well, you’ll need to bring all the documents that you opened the box with originally. Driver’s license, second form of ID, and the business document.”, Michael explained.

“Sure I said. I have my driver’s license right here. I can bring the business certificate. Great. And second form of ID?”, I asked.

“A passport would work.”

“That’ll be fine.”, I said. “So if I bring the business certificate & passport, and my driver’s license here, we’ll be able to open the box up again.”, I asked.

“Yes. If you’re able to bring those today or tomorrow, since nobody has opened another box, you can get that same box number again.”

“Wonderful. Thank you. :)”

Related: Is Amazon about to disrupt your data warehouse?

The endless lunchtime

From there I took another train trip home. The passport was easy enough to find. However the business certificate I had misplaced. It took me quite a while rifling through folders, but eventually I found it. Great I thought, I’ll just return to the post office, and have this task done for the day!

One thought occurred to me, I wonder if they might need a proof of address. He didn’t mention it, but I’ll just bring a recent gas bill just in case. Little did I know what would follow…

As I returned to the post office, Amanda greeted me. Hi there. Yes, I remember you from last week she said. Thx! Michael is at lunch now. He will be back in ten minutes.

So I waited. And waited. Twenty minutes go by. I go back to the counter, and Amanda says she’s not sure where he is, but she is going to lunch now too. When you see him return, just buzz the bell and he can help you.

Another twenty minutes go by and I see Michael return. I buzz the door, and he comes out.

Hi there Michael. I’ve managed to track down the documents you asked for, and I’ve got them all right here. Hopefully everything looks right and we can get this taken care of.

He asks me to fill out another form. On it includes my current address. Which has changed & is different from the original business address. Now I had thought to just write the same address down, as that might “simplify” things. But then I thought, that’s probably not prudent. Better to be honest. Right? I mean honesty is rewarded eh?

He sees the address & explains, I’m sorry but you’ll need a proof of address.

“Umm. Yes of course. I was careful to be very attentive to your requests earlier. Did I understand you correctly that you needed two items, the passport & business certificate.”, I asked?

“Yes but I didn’t see that you didn’t have your address here. You’ll need that too”

“No problem sir. Although it isn’t something we discussed, I did also bring my gas bill, because I thought that might be helpful here.”

“No we don’t accept that. Don’t you have a lease? Do you rent an apartment? You’ll need to bring me the right documentation!”

At this point it really does feel like a scene out of Terry Gilliam’s famous 1985 classic Brazil, where beaurocrat’s control your life. We must have form 27b-6 or else!

Related: How I use terraform & composer to automate wordpress on AWS

Customer thine enemy

At this point I’m getting internally quite furious. It is now my third trip to the post office, and still I don’t have the right forms. Remember I’ve also been a customer for over ten years! But that seems to have no bearing. What’s more I have all the documents required, even a proof of address, but it is the wrong one!

My agitation is increasing, and I’m kind of shaking with frustration. I want to scream or yell at this point. But I realize that will only make things worse.

“With all do respect sir, I’ve returned here three times already. It’s quite a trip back and forth. I’ve brought the documents you requested. “, I explain

It seems his tension is rising too. I don’t know if this is the usual day but he is not bending.

“I’ve explained what you need to do. That’s it!”

“Can you please explain to me again sir.”, I ask

“One more question. That’s all I’ll take from you!”, he says holding up his finger menacingly to me.

“How do I know if I return with one more document, we won’t get further through this process and find something else missing. Then I may have to return again.”

I am despondent at this point. Close to giving up. I have less confidence now that I’ve had throughout. I think he sees my pain at this point. I’m practically crawling on the ground beaten. That must have been enough for pity.

“Well you could talk to Miss Adams. Maybe she can help you.”, he says

“Yes miss Adams you say? Sure let’s talk to miss Adams.”

Related: 30 questions to ask a serverless fanboy

The magical Miss Adams

I go around to the main post office and ask after Miss Adams. I meet Kelly at the front desk. She explains that miss Adams is at lunch, and will be back soon. But asks what is the problem.

I explain my mixup. About paying the rent late, and how the box must be reopen. I further explain that I’ve returned three times and show her the documents I have. She says they look fine, what’s the problem. Michael explained that a National Grid bill is not sufficient proof of address.

“Why not”, she asks?

It is at this point I have a revelation. There is no rhyme or reason to any of this madness. There may be a rule, it may or may not be followed. Depending on the day, the moon, the alignment of planets. Who knows.

“I’m sure that should be fine. Let’s ask miss Adams”, she says.

And then as if by magic, Miss Adams materializes out of nowhere. After such a trial, I imagine I felt much like fraternity pleges feel as they’ve been beaten and abused for days. Miss Adams is like a saint arriving from the heavens.

“Michael doesn’t take the gas bill as proof of address? Well I do.”, she says.

And with one simple wave of the wand, everything is resolved. Just like that.

Related: Is Amazon about to disrupt your data warehouse?

Innovation, Customers First & Startups

When I think of this experience, yes I’m frustrated as anyone would be.

But it also really stands out for me in stark contrast. For I have worked with innovation, entrepreneurs & startups for many years. We all approach business from the perspective of solving problems. There understanding your customers, helping them, and simplifying processes is the rule of the day.

When I think of a government agency like the Post Office, I think of FedEx. Their market cap is 60 billion dollars. They exist with the sole purpose of moving packages. Customers will pay an incredible amount of money to avoid every having to deal with the post office.

This is a testament to innovation. And to startups. It’s why I’ve enjoyed working in the startup space all these years. The hard work & the creative problem solving. I live for that.

Related: How do I migrate my skills to the cloud?

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

How do I migrate my skills to the cloud?

via GIPHY


Hi, I’m currently an IT professional and I’m training for AWS Solutions Architect – Associate exam. My question is how to gain some valuable hands-on experience without quitting my well-paying consulting gig I currently have which is not cloud based. I was thinking, perhaps I could do some cloud work part time after I get certified.

Join 38,000 others and follow Sean Hull on twitter @hullsean.


I work in the public sector and the IT contract prohibits the agency from engaging any cloud solutions until the current contract expires in 2019. But I can’t just sit there without using these new skills – I’ll lose it. And if I jump ship I’ll loose $$$ because I don’t have the cloud experience.


Hi George,

Here’s what I’d suggest:

1. Setup your AWS account

A. open aws account, secure with 2FA & create IAM roles

First things first, if you don’t already have one, go signup. Takes 5 minutes & a credit card.

From there be sure to enable two factor authentication. Then stop using your root account! Create a new IAM user with permissions to command line & API. Then use that to authenticate. You’ll be using the awscli python package.

Also: Is Amazon too big to fail?

2. Automatic deployments

B. plugin a github project
C. setup CI & deployment
D. get comfy with Ansible

Got a pet project on github? If not it’s time to start one. 🙂

You can also alternatively use Amazon’s own CodeCommit which is a drop-in replacement for github and works fine too. Get your code in there.

Next setup codedeploy so that you can deploy that application to your EC2 instance with one command.

But you’re not done yet. Now automate the spinup of the EC2 instance itself with Ansible. If you’re comfortable with shell scripts, or other operational tools, the learning curve should be pretty easy for you.

Read: Is AWS too complex for small dev teams? The growing demand for Cloud SRE

3. Clusters

E. play around with kubernetes or docker swarm

Both of these technologies allow you to spinup & control a fleet of containers that are running on a fixed set of EC2 instances. You may also use Amazon ECS which is a similar type of offering.

Related: How to deploy on EC2 with Vagrant

4. Version your infrastructure

F. use terraform or cloudformation to manage your aws objects
G. put your terraform code into version control
H. test rollback & roll foward infrastructure changes

Amazon provides CloudFormation as it’s foundational templating system. You can use JSON or YAML. Basically you can describe every object in your account, from IAM users, to VPCs, RDS instances to EC2, lambda code & on & on all inside of a template file written in JSON.

Terraform is a sort of cloud-agnostic version of the same thing. It’s also more feature rich & has got a huge following. All reasons to consider it.

Once you’ve got all your objects in templates, you can checkin these files into your git or CodeCommit repository. Then updating infrastructure is like updating any other pieces of code. Now you’re self-documenting, and you can roll-forward & backward if you make a mistake!

Related: How I use terraform & composer to automate wordpress on AWS

5. Learn serverless

I. get familiar with lambda & use serverless framework

Building applications & deploying only code is the newest paradigm shift happening in cloud computing. On Amazon you have Lambda, on Google you have Cloud Functions.

Related: 30 questions to ask a serverless fanboy

6. Bonus: database skills

J. Learn RDS – MySQL, Postgres, Aurora, Oracle, SQLServer etc

For a bonus page on your resume, dig into Amazon Relational Database Service or RDS. The platform supports various databases, so try out the ones you know already first. You’ll find that there are a few surprises. I wrote Is upgrading RDS like a sh*t storm that will not end?. That was after a very frustrating weekend upgrading a customers production RDS instance. 🙂

Related: Is Amazon about to disrupt your data warehouse?

7. Bonus: Data warehousing

K. Redshift, Spectrum, Glue, Quicksight etc

If you’re interested in the data side of the house, there is a *LOT* happening at AWS. From their spectrum technology which allows you to keep most of your data in S3 and still query it, to Glue which provides an ETL as a service offering.

You can also use a world-class columnar storage database called Redshift. This is purpose built for reporting & batch jobs. It’s not going to meet your transactional web-backend needs, but it will bring up those Tableau reports blazingly fast!

Related: Is Amazon about to disrupt your data warehouse?

8. Now go find that cloud deployment job!


With the above under your belt there’s plenty of work for you. There is tons of demand right now for this stuff.

Did you do learn all that? You’ve now got very very in-demand skills. The recruiters will be chomping at the bit. Update those buzzwords (I mean keywords). This will help match you with folks looking for someone just like you!

Related: Why I don’t work with recruiters

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

What’s the *real* way to deploy on Google Cloud?

via GIPHY

I was talking to a customer recently and they asked about deployments. They wanted to do things the real way. Here’s a snippet…

I’m helping out a company called Blue Marble and they are getting ready to deploy a new POS system. The app has been built using a Node.js back-end and Google Cloud Datastore for storage. The current dev build is hosted on AWS and connects to Google for the data bits.

Join 38,000 others and follow Sean Hull on twitter @hullsean.


For prod launch, they are interested in migrating to the “real” way of deployment on Google for everything.

They are pressed on time and looking for someone who can jump in quickly. Are you available? Do you have Google Cloud expertise?

Here’s what I said.

Cultural hurdles


Yep, I’ve have used Bigquery & GCE.

What are they looking for specifically? Full deployment automation? Multiple deploys per day?

I’ve found that sometimes the biggest hurdle to fully automated deploys can be cultural issues.

In other words yes you can automate your deployment so it is push button, get all the artifacts & moving parts automated. Then deploy without much intervention. But to go from that to the team having *faith* in the system, that is a challenge.

Also: Why would I help a customer that’s not paying?

Unit testing


Once the process has been streamlined, a lot often still needs to happen around unit & smoke tests.

If the team isn’t already in the habit of building tests for each bit of code, this may take some time. Also building tests can be an art in itself. What are the edge cases? What values are out of bounds?

Consider for example odd vulnerabilities that show up when hackers type SQL code into fields that devs were expecting. Sanity checking anyone?

Read: Is AWS too complex for small dev teams? The growing demand for Cloud SRE

Integraton testing

What makes this all even more complicated is integration testing. Today many application use various third party APIs, service-based authentication, and even web-based databases like Firebase. So these things can complicate testing.

Related: How to build an operational datastore on Amazon Redshift with S3

Getting there

Although your project, startup or business may be pressed for time, that may not change the realities of development. Your team has to become culturally ready to be completely agile. Many teams choose a middle ground of automating much of the deployment process, but still having a person in the loop just in case.

Same with testing. Sure automating can make you more agile & more efficient. But you’ll never automate out creative thinking, problem solving & ownership of the product.

Related: Why did Flatiron School fail?

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters