Just to give some context, I’ll start by a quick walk through compute history. From the server cabinet in the back office, to the early managed hosting providers and then on to today’s modern cloud offerings, I’ll explain how we got here.
With that new context in mind, I’ll talk about that evolution one step further, to managed functions. What’s that you ask? Just hand over your code to the cloud, and let them handle running the servers, provisioning load balancers, and reacting to your customers when they hit the endpoint.
No presentation is complete without a proper diagram. My reference architecture makes use of Amazon’s many cloud services, including API endpoint, cognito for user authentication, lambda for serverless functions, dynamodb to store state information, S3 for storing objects, CloudFront for the edge caching network, and Route53 for the domain name.
Each of the components I mention above, requires some explanation. I’ll talk about how to setup a serverless project, how to define and manage your API endpoint. This is where users first touch your application. I’ll introduce user authentication with Amazon’s own service or a third party like OneLogin or Auth0. From there you’ll see how Amazon’s nosql database Dynamodb works, and how you can store your original & edited images in S3. And no site would be complete without an edge cache, and we’ll have that setup too. Then store your domain name in Route53 and point it to your API.
I have to admit, though I’m not a management consultant, I do pickup the big ones from time to time. Good to Great, How to Make Friends & Influence People, The Lean Startup, Innovators Dillemma & Who Moved My Cheese among my favorites.
I’d seen Ben Horowitz’s book “The Hard Thing about Hard Things” in the news. But I also really love the a16z podcast, and although I don’t know a ton about the VC business, I thought it would be a good read.
Boy is that an understatement! The book is so readable & so accessible, there are nuggets of value in there for anyone in the startup world, or building their career, CEO or not!
On the efficient market hypothesis
I had always assumed Adam Smith’s invisible hand was a good theory, almost as scripture. So to see a different perspective on this, and one backed up by real experience. That’s cool.
“No, markets weren’t “efficient” at finding the truth; they were just very efficient at converging on a conclusion — often the wrong conclusion”. p52
What’s more for investors out there, it means good old fashioned investigative work can still turn up gems, that are worth investing in. Word to the wise.
Freaky Friday was a movie way back in the 80’s. In it a mother & daughter are at each others throats, frustrated with the each other. They end up switching places, and quickly learn to sympathize with the other’s plight in life.
Horowitz decided to put this method to use between two of his managers. Pretty ingenious.
“After just one week walking in the other’s moccasins, both executives quickly diagnosed the core issues causing the conflict. They then swiftly acted to implement a simple set of processes that cleared up the combat and got the teams working harmoniously. p253
“In life, everybody faces choices between doing what’s popular, easy, and wrong versus doing what’s lonely, difficult, and right.” p212
Every time you make the hard, correct decision you become a bit more courageous and every time you make the easy, wrong decision you become a bit more cowardly. p213
Boy, can I relate to these bits of wisdom. Running my own business all these years, it hasn’t been easy. There have been ups and downs, and times when people told me to take a different road. But that courage. When you find it, it can be real fuel for you moving forward.
I realized that embracing the unusual parts of my background would be the key to making it through. It would be those things that would give me unique perspectives and approaches to the business. p276
When I work with entrepreneurs today, this is the main thing that I try to convey. Embrace your weirdness, your background, your instinct. p276
I hadn’t really thought of this, and it’s an interesting point. It may be one of the reasons why customers hire me, that I hadn’t realized. Certainly I can give an original viewpoint. But I think I will try to put this to work in the future.
This is a curious & fascinating point about the history of venture capital. Ripe for disruption indeed!
Marc discovered that the original venture capital firms in the late 1940s and early ’50s were modeled after the original investment banks such as J.P. Morgan and Rothschild. Those banks also did not do PR for a very specific reason: The banks funded wars—and sometimes both sides of the same war—so publicity was not a good idea. p271
Alexa & Kyra offer us a tantalizing question. Could it be that we could learn a lot from oddball innovators at the edge of the economy? When I say edge, I really mean it. She interviews Sam Hostetler who is building a business around milking camels, and then there’s Abdi Hasan a pirate from Galkayo northern Somalia. Yeah really! Or what about the German copycats Wimdu who built a complete replica of Airbnb by reverse engineering it.
1. Hack the cold call
Take the example of Lance Weiler. Early on the industry was very against digital. They didn’t see it as really making films.
“Part of [Lance] Weiler’s success was due to his ability to work the system. He wrote letters to major production companies telling them he wanted to make the first digital motion picture. After he didn’t hear back, he took a page from the con man’s handbook and wrote the same letters but intentionally misaddressed them so they were sent to the wrong companies. Sony for example would get a letter intended for Barco.”
He was later able to bring digital projection to Cannes & Sundance!
“For Weiler his big epiphany was when he realized he could be creative across all of it [the business]. Not just in the art product, but in financing, distribution, and business aspects of artistic production.”
The german brothers Oliver, Marc & Alexander Samwer make a superb example of how copying can bring building prowess to compete against innovators that were first to market.
“in 1998 Marc Samwer had an instinct that eBay would thrive in the German market… his brothers agreed… they contacted eBay via email numerous times, recommending that the company replicate it’s platformin Germany. Claiming that eBay failed to respond, the brother’s started their own German-language auction site, Alando, which was then purchased by eBay for 38 million euros (over $50 million) only 100 days after it’s debut. Had the Samwers not copied, eBay might have remained complacent, not realizing its potential within the german market.”
Although not mentioned in the book, Inditex the wildly successful firm behind fashion brand Zara did much the same thing to the fashion industry. By mastering the supply chain, they enabled their company to take designs from the runway & replicate them, turning designs into real clothing in stores, in just two weeks! And indeed they really do replicate, borrow & straight copy those designs from what they see at fashion week. Sad & brilliant at the same time.
“In the lexicon of the Misfit Economy, we define “hustle” as making something out of nothing. To move fast, to trade one thing for another, and to proactively create your own opportunities rather than waiting for opportunity to come your way. To hustle means getting your hands dirty, being lean and facile, working hard, being resourceful and resilient, and showing or having gumption, chutzpah, or mojo.”
And after all, isn’t that everything the startup industry aspires to? Agile teams? Growth hackers? Scrappy startups & innovation?
When I have a long todo list and a million things on my plate, my usual tactic is to just plow through it. Take short break to eat, but then get right back to work. My feeling is, if it’s weighing on the back of my mind, I won’t enjoy downtime anyway.
Recently though I had a very different experience. And it surprised me.
1. Too much to do
A colleague of mine asked me to meetup for beers. We planned to talk technology, and to catchup on what we were both working on.
As the night rolled on he had some delays, and I wanted to cancel too. After all I had a ton of work to do, and didn’t think I would enjoy myself. I really felt like I’d be worrying about all this work on my plate. It’s like taking a vacation when you have a deadline. It doesn’t feel quite right.
We ended up meeting anyway. At first I wasn’t totally relaxed, but then we started talking.
Our conversation turned to the evolution of datacenters. How they used to be on premise, then there were lots of hosting companies. And then Amazon changed everything!
We talked about evolution of tooling & automation. Although system administrators of old have been writing bash scripts forever to make their jobs easier, the proliferation of tools for deployment has allowed smaller ops teams to control fleets of servers. As my friend & colleague was newly starting a job on Amazon Web Services, a lot of this cloud stuff was new to him. So talking about it from a teaching vantage point, made me realize how strong I was in a lot of this stuff.
We talked about docker & containerization too. Even the origins back in the late 70’s with Unix chroot all the way up to Docker today. I explained to him that he could think of a container almost like a unix user, but with a more self-contained view of the whole system. In many ways a container acts like a vm, with it’s own filesystem and processes.
We talked a lot about aws, how S3 was an evolution of FTP in the old days, but much much better, how VPCs worked and the virtualization of networking, how VMs in the AWS world match with bare metal or not, how they share EBS storage. How Amazon has built a database service RDS around popular platforms like Oracle, MySQL & Postgres.
We shared a lot of ideas & brainstorming. About coding, C versus Java versus Python, package management, dependencies and on and on. He also mentioned he needed to build a test script to talk to an Amazon queue. I explained that it should be quite easy, and which libraries to look for.
I’ve had a mailbox with the postal service for over a decade. It’s been great. It’s a convenient address for a business, and it doesn’t change. Great for receiving bills, tax refunds, contracts, pay checks and everything else.
I picked the postal service years back because I figured they are solid. A corner deli or other mailbox service might go out of business. And the idea was to have a box that doesn’t have to change often.
In all those years I’ve always been timely with paying the rent, which can be every six months or twelve months as you choose. Recently I mixed up the renewal. When I went in to pickup my mail, the lock had been changed. I went to the customer service counter to ask about it. Amanda came to help me. She was cordial.
She explained however that the box had been closed. Since it was the day before the holiday, she wouldn’t be able to help with this, and suggested I come back on Monday.
As I like to be agreeable, I agreed to this request. In the meantime she was able to give me my mail. It was nice to see they had been keeping it. After all I was only a few weeks overdue.
Bring me the documents!
On Monday I had planned to return right away. But as my day got going I wondered if I would have enough time. This probably won’t be just a quick stop in. For some reason I sensed the process would be complicated.
Tuesday I set my alarm clock to wake up two hours early. I got dressed & headed straight to the post office. I wanted to get an early start. I went to the customer service desk again, and met Michael. I explained the mixup. Yes I was late playing my rent. Could I pay a penalty of some kind and then renew for another year?
“No I’m sorry, your box is closed now. You’ll have to reopen it.”, he explained.
“That’s no trouble let’s do that. What do we need to do?”, I asked.
“Well, you’ll need to bring all the documents that you opened the box with originally. Driver’s license, second form of ID, and the business document.”, Michael explained.
“Sure I said. I have my driver’s license right here. I can bring the business certificate. Great. And second form of ID?”, I asked.
“A passport would work.”
“That’ll be fine.”, I said. “So if I bring the business certificate & passport, and my driver’s license here, we’ll be able to open the box up again.”, I asked.
“Yes. If you’re able to bring those today or tomorrow, since nobody has opened another box, you can get that same box number again.”
From there I took another train trip home. The passport was easy enough to find. However the business certificate I had misplaced. It took me quite a while rifling through folders, but eventually I found it. Great I thought, I’ll just return to the post office, and have this task done for the day!
One thought occurred to me, I wonder if they might need a proof of address. He didn’t mention it, but I’ll just bring a recent gas bill just in case. Little did I know what would follow…
As I returned to the post office, Amanda greeted me. Hi there. Yes, I remember you from last week she said. Thx! Michael is at lunch now. He will be back in ten minutes.
So I waited. And waited. Twenty minutes go by. I go back to the counter, and Amanda says she’s not sure where he is, but she is going to lunch now too. When you see him return, just buzz the bell and he can help you.
Another twenty minutes go by and I see Michael return. I buzz the door, and he comes out.
Hi there Michael. I’ve managed to track down the documents you asked for, and I’ve got them all right here. Hopefully everything looks right and we can get this taken care of.
He asks me to fill out another form. On it includes my current address. Which has changed & is different from the original business address. Now I had thought to just write the same address down, as that might “simplify” things. But then I thought, that’s probably not prudent. Better to be honest. Right? I mean honesty is rewarded eh?
He sees the address & explains, I’m sorry but you’ll need a proof of address.
“Umm. Yes of course. I was careful to be very attentive to your requests earlier. Did I understand you correctly that you needed two items, the passport & business certificate.”, I asked?
“Yes but I didn’t see that you didn’t have your address here. You’ll need that too”
“No problem sir. Although it isn’t something we discussed, I did also bring my gas bill, because I thought that might be helpful here.”
“No we don’t accept that. Don’t you have a lease? Do you rent an apartment? You’ll need to bring me the right documentation!”
At this point it really does feel like a scene out of Terry Gilliam’s famous 1985 classic Brazil, where beaurocrat’s control your life. We must have form 27b-6 or else!
At this point I’m getting internally quite furious. It is now my third trip to the post office, and still I don’t have the right forms. Remember I’ve also been a customer for over ten years! But that seems to have no bearing. What’s more I have all the documents required, even a proof of address, but it is the wrong one!
My agitation is increasing, and I’m kind of shaking with frustration. I want to scream or yell at this point. But I realize that will only make things worse.
“With all do respect sir, I’ve returned here three times already. It’s quite a trip back and forth. I’ve brought the documents you requested. “, I explain
It seems his tension is rising too. I don’t know if this is the usual day but he is not bending.
“I’ve explained what you need to do. That’s it!”
“Can you please explain to me again sir.”, I ask
“One more question. That’s all I’ll take from you!”, he says holding up his finger menacingly to me.
“How do I know if I return with one more document, we won’t get further through this process and find something else missing. Then I may have to return again.”
I am despondent at this point. Close to giving up. I have less confidence now that I’ve had throughout. I think he sees my pain at this point. I’m practically crawling on the ground beaten. That must have been enough for pity.
“Well you could talk to Miss Adams. Maybe she can help you.”, he says
“Yes miss Adams you say? Sure let’s talk to miss Adams.”
I go around to the main post office and ask after Miss Adams. I meet Kelly at the front desk. She explains that miss Adams is at lunch, and will be back soon. But asks what is the problem.
I explain my mixup. About paying the rent late, and how the box must be reopen. I further explain that I’ve returned three times and show her the documents I have. She says they look fine, what’s the problem. Michael explained that a National Grid bill is not sufficient proof of address.
“Why not”, she asks?
It is at this point I have a revelation. There is no rhyme or reason to any of this madness. There may be a rule, it may or may not be followed. Depending on the day, the moon, the alignment of planets. Who knows.
“I’m sure that should be fine. Let’s ask miss Adams”, she says.
And then as if by magic, Miss Adams materializes out of nowhere. After such a trial, I imagine I felt much like fraternity pleges feel as they’ve been beaten and abused for days. Miss Adams is like a saint arriving from the heavens.
“Michael doesn’t take the gas bill as proof of address? Well I do.”, she says.
And with one simple wave of the wand, everything is resolved. Just like that.
When I think of this experience, yes I’m frustrated as anyone would be.
But it also really stands out for me in stark contrast. For I have worked with innovation, entrepreneurs & startups for many years. We all approach business from the perspective of solving problems. There understanding your customers, helping them, and simplifying processes is the rule of the day.
When I think of a government agency like the Post Office, I think of FedEx. Their market cap is 60 billion dollars. They exist with the sole purpose of moving packages. Customers will pay an incredible amount of money to avoid every having to deal with the post office.
This is a testament to innovation. And to startups. It’s why I’ve enjoyed working in the startup space all these years. The hard work & the creative problem solving. I live for that.
You’ve heard it before. Automate your builds. That means putting everything in version control, from environment building scripts, to configs, artifacts & reference data. Once you can do that, you’re on your way to automating production deploys completely.
While everyone is scrambling to figure out why part of the internet went down … wait is S3 is part of the internet, really? While I’m figuring out if it is a service of Amazon, or if Amazon is so big that Amazon *is* the internet now…
I loved Drew Bell’s story of stumbling into home ownership, attempting to fix a doorbell, and falling down a familiar rabbit hole. With parallels to legacy software systems… aka any older then oh say five years?
If you grew up on the virtual world of the cloud, you may have never touched hardware besides your own laptop. Developing in this world may completely remove us from understanding those pesky underlying physical layers. Yes indeed folks containers do run in “virtual” machines, but those themselves are running on metal, somewhere down the stack.
Amazon is building out it’s serverless offering at a rapid clip. Lambda makes a great solution for a lot of different use cases including:
o a hybrid approach, building lambda functions for small pieces of your application, sitting along side your full application, working in concert with it
o working with Kinesis firehose to add ETL functionality into your pipeline. Extract Transform & Load is a method of transforming data from a relational or backend transactional databases, into one better fit for reporting & analytics.
o retrofitting your API? Layer Lambda functions in front, to allow you to rebuild in a managed way.
o a natural way to build microservices, with each function as it’s own little universe
Great, tons of ways to put serverless to use. What’s Amazon doing to make it even better? Here are some of the features you’ll find indispensible in building with Lambda.
1. Versioned functions
As your serverless functions get more sophisticated, you’ll want to control & deploy different versions. Lambda supports this, allowing you to upload multiple copies of the same function. Coupled with Aliases below, this becomes a very powerful feature.
As you deploy multiple versions of your functions in AWS, you don’t want to recreate the API endpoints each time. That’s where aliases come in. Create one alias for dev, another for test, and a third for production. That way when new versions of those are deployed, all you have to do is change the alias & QA or customers will be hitting the new code. Cool!
Using the API gateway, we can do some fancy footwork with Lambda. First we can enabling caching to speedup access to our endpoint. Control the time-to-live, capacity of the cache easily. We’ll also need to invalidate the cache when we make changes & redeploy our functions.
Throttling is another useful feature, allowing you to control the maximum number of times your function can be called per second on average (the rate) and maximum number of times (burst limit). These can be set at both the stage & method levels.
Creating multiple stages, for dev, test & production means you can separate out and control environment variables with more granular control. For example suppose you have access & secret keys to reach S3. You can set environment variables for these to avoid committing any credentials or secrets in your code. Definitely don’t do that!
Allowing multiple copies of stage variables, means you can set them separately for dev, test & production.
I’m choosing a fictional name here, but the meeting was real. We met over lunch to discuss how we might work together. Their org has been around for years, has a phenomenal track record, and they are strongly sales oriented.
o They’re hungry. They pushed for client lists & sniffed for leads.
o They’re margin oriented, they had a clear idea of where their strong suit was, and what types of customers they wanted to work with. That’s because they had a clear idea of their margins.
o They understand the industry well, much better than I did.
o They could certainly talk circles around me in terms of industry categories & verticals.
o They glossed over technical details
o They made broad generalizations & mixed up facts at times
Here again I’m choosing a fictional name. We met over dinner to discuss my opinions of the market and also if I might have any venture leads or could make introductions.
Some Observations I came away with:
o Their company is all engineering.
o They’re intimately focused on coding & building the product.
o They downplayed product limitations & somewhat out of touch with customer.
o They seemed to be feeling around in the dark for investors
o They seemed to have a weak network
One of my past customers, also a fictional name here, they were also an incredibly sales heavy organization.
o Their monthly standups felt like a sporting huddle.
o Lots of ra ra ra & high fives
o They were extremely sales driven, growing rapidly
o They had tremendous problems around engineering.
o They seemed to be boxing wayyy above their weight class.
As a consultant I find this all fascinating. It often seems like this cultural style is driven from the top. The big movers are the ones who shape the organization.
I think of Google as an incredible example of an engineering driven organization. Finding top people is always about math & problem solving, but short on personality emphasis. Meanwhile their products lack the UI polish, but are functionally accurate & always fast.
Contrast that with Oracle, which send in a heavy armament of perfect suits to close a deal, negotiate soft until you’re firm is locked in, then jack up the license fees until you bleed. Meanwhile although the product is a sturdy technical construction, it’s every bit the marketing that is smooth & polished.