Category Archives: Startups

Is Amazon too big to fail?

aws fault tolerance

Amazon is the huge online retailer everyone knows well. However there is another side of Amazon, namely Amazon Web Services that hosts many of the internets largest websites.

Join 28,000 others and follow Sean Hull on twitter @hullsean.

In the infrastructure & operations world, Amazon is the Citibank, JP Morgan or Goldman Sachs of cloud providers.

1. Outage takes down Yelp & Netflix

As reported on Thousand Eyes among other places, Amazon had a major outage yesterday.

Amazon experienced a problem with how they route data over the network. Routing is the technical term for how the internet moves data around. When routing goes wrong at a provider like Amazon, the websites they host will go down too.

Also: Are we fast approaching cloud-mageddon?

2. Automation can’t save you

Netflix is famous for their great streaming service, and shows like House of Cards.

On the technology side they’re also pretty famous. They deploy legions of Amazon servers to stream movies using Chaos Monkey. This open source suite allows them to remain resilient even if individual servers or components go offline.

Yet a heavy reliance on Amazon itself, meant a wider outage for them was also an outage for Netflix.

Related: What tech do startups use most?

3. Of cloud monopolies

Amazon’s dominance in the cloud hosting space is incredible. There are providers that can beat them in compute power, speed & price. But with their incredible reach of global datacenters & relentless growth they are still the first choice for most internet shops.

What is the downside of such dominance? What happened yesterday illustrates it clearly. When Amazon goes down, so do financial companies like Experian,

Read: Do managers underestimate operational cost?

4. Diversify your data portfolio

In the banking world we can put together legislation, regulating banks. We can enact capital requirements or consider breaking up the largest ones. For investors & consumers you can diversify your portfolio, putting money in different asset classes & institutions. If one fund fails, others will balance it out.

We can do the same with cloud hosting. For larger internet applications, deploying on multiple clouds can be very beneficial. In that case an outage at Amazon, would merely mean your global load balancer kicks in, sending traffic to your plan B servers.

Also: Replicate big data to Amazon Redshift with Tungsten

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Are we fast approaching cloud-mageddon ?

storm coming

One look at StackShare’s trending technologies, and you’ll discover the exploding growth of languages, webservers, load balancers, databases, caching servers, automation & monitoring tools, continuous integration suites & a broad spectrum of Software as a service solutions.

The choices today boggles the mind. Choice is good, but too much choice can mean trouble too.

Join 30,000 others and follow Sean Hull on twitter @hullsean.

1. What am I actually using?

Erich Schubert wrote a superb piece about the sad state of the sysadmin in the age of containers. Here’s what caught my eye:

Stack is the new term for “I have no idea what I’m actually using”.

That definitely rings true for me. The customers I’m seeing these days have such complicated stacks, that nobody really knows what’s installed. That’s dangerous!

Also: Do today’s startups assemble at their own risk?

2. Embrace failure more broadly

Recently I wrote a blog post asking Is AWS the patient that needs constant medication?. It got a lot of traction, and here’s why I think that happened.

AWS uses very commodity, cheapo components. The assumption is, with an infinitely redundant datacenter, component failure is ok. It’s ordinary & everyday.

Unfortunately most startups, even ones that employ some Ansible & devops, still don’t have Netflix grade automation.

Those regularly everyday failures are still getting detected by old-school manual monitoring. And that’s a recipe for trouble

Also: 5 Things toxic to scalability

3. What are complex systems?

In this excellent deck, James Urquhart talks about emergent behavior in complex systems. It’s worth a quick read.

***

Read: How I find entrepreneurial focus

4. What to do? Do you like boring?

Dan McKinley formerly principal engineer at Etsy & now with Stripe wrote a brilliant essay arguing for boring technology.

This comes as a shock to many in the startup world. It sort of smacks in the face of open source, or does it?

I worked in the enterprise space as an Oracle DBA for a decade starting in the mid-nineties. Among DBAs there was always a chuckle when a new version of Oracle came out. No one wanted to touch it with a ten foot pole. Sure we’d install it on test boxes, start learning the new features and so forth. But deploy it? No way, wait a good 2 or even 3 years before upgrading.

Meanwhile management was eager for the latest software. Don’t we want the newest? The Oracle sales guys would be selling the virtues of all sorts of features that nobody needed right away anyway.

Choosing boring components takes discipline to fight sexy new technologies & bleeding edge versions. But staid & stodgy wins you everyday in operations uptime.

Related: Is automation killing old-school operations?

5. Use tried & tested components

Do you find your application or stack contains java, ruby, python & PHP? Choose one.

One webserver like nginx, one caching server like memcache or redis, one search server like solr or elasticsearch, one database like MySQL or postgres. Standardize all your components on one image, so you can use that for all your servers, regardless of which you use.

Fewer components will mean fewer interdependencies, less maintenance, & less chaos.

Also: What’s the luckiest thing that’s happened in your career?

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

How I find entrepreneurial focus

Brian K

Relentless focus. This is surely a key to entrepreneurial success. But how to find it? And how to maintain that focus through the ups & downs?

Join 28,000 others and follow Sean Hull on twitter @hullsean.

I’ve found a simple system with a few rules has brought me success. I’ve used it for two decades of as entrepreneurial, and still do.

Here’s how it works…

1. keep a list of small tasks

This is the number one thing I do daily. I start out early in the morning, over coffee. Anything that “needs to get done” goes on the todo list, but I also separate out the things I’ll do today.

Todo list items are not big ones, like “save the world” or “get new job”. They are small nuggets of work that take 15 to 60 minutes. If they’re larger, they need to be broken down.

A typical day covers five major tasks. You will get sidetracked. You’ll need to answer calls & emails that aren’t captured on the list.

All this sounds simple, but it actually requires a discipline. Both to keep tasks actionable & small. You’ll learn your work pace with practice. But there’s one more thing to remember.

Also: Are generalists better at scaling the web?

2. trust the list

One thing I find myself doing is pushing just because something is on the list.

I don’t do list based on feelings or moods

This requires a lot of habit building, but it becomes valuable. By doing this over time, you begin to trust the list. You know things on it will get done. So you can safely “add it to the list” and forget about it for the moment.

This lets your mind relax and bcomes a real godsend when you have a mountain of work to do.

Just work list because it’s there. And trust that things that need to get done simply go on the list.

Related: Why airbnb didn’t have to fail

3. done with list, done for day

On days where things get hairy, and you work more, you’ll have to slog through to get everything on the list done. But sticking to it will build a habit that’s valuable.

At the same time some days will be easier. Avoid the temptation to add more work to fill the day.

When you’re done with the list, you’re done for the day

This is a discipline too. Pat yourself on the back, and give yourself a break. You did what you said you would do. Time for a beer. :)

Read: Which tech do startups use most?

4. big projects require faith

Anothe lesson I’ve learned is that really large projects, or ones bringing you into new areas, require a lot of faith. For me, with an engineering background, I don’t have an easy time finding that. I want to measure, and dice up everything from the start.

When I was embarking on a project to buy real estate in Brooklyn, I really learned this lesson. There was so much unknown. How do I work with real estate brokers that have a different style of communicating than engineers & corporate professionals? How do I negotiate? What’s the right price? What about mortgages & their brokers? Architect inspections, land surveys, flood zones, crime maps, loans & assets, legal & closing costs. The list of unknown & nebulous areas of expertise was staggering.

Hang around edges to get lay of land

If I would distill this faith idea down for someone embarking on a new career or diving into a pool of unknown depth, I would say start by hanging around the edges. Pick off pieces that you can, and add them to the todo list.

I went to open houses. I asked questions. I researched online, and I always made sure I understood how agents & players were incentivized. That means believing none of the words people speak, but rather, look behind the curtain and make educated guesses about those realities.

Also: Do todays startups assemble at their own risk?

5. break down & do

It is inevitable that you will experience writers block. Or any other kind of block that manifests as procrastination. Don’t over think it.

Continue to break down to smallest viable unit that you *can* do.

get started on anything to get inertia

With writing I find blocks where I don’t have a solid idea formulated. Maybe you have a topic? So then my todo list for the day is “write five bullet points”. This by itself will take some time, but you know you can write something. By moving past your block, I sometimes find I wanna keep writing and finish the piece. This is the kind of habit you want to form.

Also: Is Fred Wilson right to say speed is a feature?

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

5 Things I learned from Fred Wilson & Mark Suster

I was recently flipping through AVC.com and saw this interview by Mark Suster of Upfront Ventures. He talks in depth with veteran in the VC world, Fred Wilson.

Join 30,000 others and follow Sean Hull on twitter @hullsean.

Here’s a more in-depth blog post on Mark’s interview with Fred Wilson from Union Square Ventures.

1. I’m not into debt

Around the 20:25 in the interview, Fred is discussing a period in his career before some of his first big investments, where things were financially challenging. He makes a rather candid comment about personal debt:

“I’m not that kinda person. I don’t like debt. I’m not into debt”

I think this is key. I also think it frames the whole way people approach business & career.

Also: Are generalists better at scaling the web?

2. Brains & hustle is key

Among the most successful entrepreneurs there are certainly many who are very intellectually astute. Meanwhile there are others who are great speakers, who can sell an idea, and persuade, but perhaps not as deep product wise or deeply technical.

The very best though, tend to have both brains & hustle.

Related: 8 questions to ask an AWS expert

3. Best technology doesn’t win markets

Around 11:45 in the interview, Mark & Fred are discussing Novell & Banyan.

“That was when I learned that best technology doesn’t win markets”

t’s interesting because as you hear the story of how Banyan lost out to Novell, it resonates today with companies that often have the best tech, but don’t win in markets. Interesting.

Read: Why Airbnb didn’t have to fail

4. Find answers through blogging

“It’s like Venus Fly Paper. When I write about topics that are relevant, suddenly anybody with a startup solution in that field will approach us. This works brilliantly.”

Indeed, I’ve found blogging to be crucial myself to career building. It helps in a myriad of ways.

Blogging brings visibility, as your blog gains in popularity. That is certainly big. But also it helps you craft & formalize your voice & your vision. Blogging asks you everyday to think about your perspective, and share it in a way that appeals to a broad audience. And analytics give you real feedback that you are saying something of value to people.

Also: Are SQL databases dead?

5. Listen to the younger generation

Around 1:11:15 in the interview, there’s an interesting point where Mark asks Fred if there were any deals that they regret not getting into. Fred responds that AirBNB was such a deal, as it was a quintessential Union Square ventures company.

As it turns out they didn’t invest because they couldn’t imagine using the service. Meanwhile the younger members of their team had a different perspective.

“We’re not gonna reject anything that we wouldn’t do and the younger team would.”

Interesting point. I think of Venmo as another example of this. I personally wouldn’t use the service, meanwhile it is clearly very popular among teen & twenty something demographic.

Also: 5 Things toxic to scalability

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

What’s the luckiest thing that’s happened in your career?

sashi [via Flickr]

I was browsing through Career Dean recently, a site that facilitates professionals to share knowledge & experience with more junior & recent college grads about the work world. It’s a great site. I saw the question What’s the luckiest thing that’s ever happened for your career?

I read in John Adam’s AMA his “million dollar piss” (www.careerdean.com/q/howd-you-get-the-job-twitter), which he sowed the seeds of his success basically during a piss. That’s a 1 in a million kind of story I know. I’d like to hear if anyone else has ever experienced anything remotely lucky in that way? =) something fun to come back and read if anyone answers.

Join 28,000 others and follow Sean Hull on twitter @hullsean.

Here’s how I responded…

I moved to NYC & worked at a tiny startup in the mid-nineties. Got to do Mac stuff, windows & Sun Solaris unix as well. Jumped on an Oracle project where I was a bit underwater. The firm hired a consultant to assist me for a few days. I watched what he did and learned like a sponge. Within a few months I dove into Oracle consulting and never looked back.

I felt this was an amazingly lucky opportunity to for a few reasons.

1. DIY

I’ve been consulting for almost twenty years now. And I get asked all the time how to get into freelance or independent consulting. For me the jumping off point was working for a really small ten person startup.

An environment like this is very different from a large corporation where you do one thing. At a tiny shop, everything is very do-it-yourself. You have to be self-serve & lean. It’s a constant challenge to teach yourself what you don’t already know. It’s a very vibrant environment as you enter your career.

Also: 5 Things toxic to scalability

2. Generalist

I also found that I had the chance to really apply everything I learned in computer science. It’s a hardware problem? It’s a software problem? These kind of silos that you experience at university don’t apply. One day you can be doing windows, mac, or Unix operating system configuration, the next you can be writing code. And on the third day you can be doing dba work.

In today’s terminology, this role was site reliability engineer or SRE, fullstack developer, tech support, evangelist, CTO, DBA, scalability & performance lead and more.

Related: Are generalists better at scaling the web?

3. Cutting edge

Startups to be sure are on the bleeding edge. They’re constrained by budgets, and through sheer will & experimentation, are cutting their teeth on the newest technologies out there.

These days that might be Cassandra & Kafka, Docker, MongoDB, hdfs, Redshift and so on.

Read: Do managers underestimate operational cost?

4. Ok to Fail

In larger enterprises, a lot of politics weigh on decisions, and exotic technologies are risky. When you’re at a startup, and by design you are entering uncharted waters, it’s sort of a given that it is ok to fail. This encourages learning, as there is less risk of failure.

Also: Is the difference between dev & ops a four-letter word?

5. Iterative & Agile

We talk about being agile, and lean at startups. At a very small place like this, you have one or two developers, and you deploy code constantly. It’s agile by default. And that’s a good thing.

Also: Is high availability overrated? The myth of five nines.

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

The chaos theory of cloud scalability

The.Rohit - Flickr

The.Rohit – Flickr

Reading Benedict Evans weekly newsletter, you’re bound to bump into something new & useful. His newsletter covers Mobile, but that also means it touches on a lot of other areas of tech, innovation & startups.

This week he pointed me to A Weissman’s The Chaos Theory of Startups. He argues a VC’s job is to help a startup identify the right framework. It’s about finding the signal in the noise.

Join 29,000 others and follow Sean Hull on twitter @hullsean.

I think you can carry this idea over to technical operations today. There are a few key maxims I follow to keep you on the scalable track.

1. Degrade gracefully

You’ve heard it before, but have you done anything about it?

Build a read-only or browse only mode into your application. Do it now. You will thank me. When your database goes down unexpectedly (with RDS this might happen sooner than you think), you want to be able to use your lovely read-only slave database. Browse only mode, forces developers to add read-only support in most application functions, keeping the site up and running, without a full visible and ugly outage.

Which brings me to point two, be sure to have copies of your production database. Real live, only read-only copies. In Amazon speak, this is a read-replica, in MySQL this is a slave database. Most startups I see these days have this, but if you’re one of the ones dragging your feet, do this now.

Also: Is the difference between dev & ops a four-letter word?

2. Monitor & measure

Amazon’s cloudwatch is fine for what it is, and so is New Relic. But employing a dedicated tool just for monitoring, such a Nagios & cacti can give you much more granular intelligence about what’s happening with your infrastructure. Nagios gives you the monitoring & alerting, Cacti gives you the history. It’s like a BI reporting tool for infrastructure.

Related: Is automation killing old-school operations?

3. Keep components simple

Keep it simple stupid? Don’t adopt new technologies, languages, or versions of software, without first vetting them. Ask questions:

o Is there an existing piece of software or service that can overlap this new one, killing two birds with one stone?
o Does everybody know this new technology?
o Does this choice of technology solve any other broad problems we have?
o Is there a large community around the project?
o Are there a lot of engineers with experience in this chosen technology?

Tellingly, many startups don’t have an operations person to start with. In those, the danger is developers choose new solutions, with no push back.

I asked… Does a four letter word divide dev and ops?

Read: Do managers underestimate operational cost?

4. Don’t force database abstraction

Object Relational Modelers, aka database middleware, are great in theory. We want a library that takes database & SQL drudgery away from developers. Why reinvent the wheel?

The trouble is database independent code doesn’t work, and never has. ORMs are painfully inefficient, selecting all columns, or repeatedly reading rows from tables. This causes serious traffic jams inside your database.

They come in various guises, Cake PHP, Active Record for Ruby, Hibernate for Java, SQL Alchemy for Python.

Also: Is the difference between dev & ops a four-letter word?

5. Be asynchronous

This means don’t make your application code wait. Make asynchronous calls to APIs & check back later, use software queues so traffic backups don’t clog your components & communication.

Avoid any type of two-phase or multi-phase commit. These are common in clustered databases, forcing a serialization point so nodes can agree on what data looks like. However they’re toxic to scalability. Look for technologies that use eventually consistent algorithms.

Also: Is the difference between dev & ops a four-letter word?

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

If you’re building a startup tech blog you need to ask yourself this question

Editor & writer in friendly dialog

Join 28,000 others and follow Sean Hull on twitter @hullsean.

I work at a lot of startups, and these days more and more are building tech blogs. With titles like labs or engineering at acme inc, these can be great ways to build your brand, and bring in strong talent.

So how do we make them succeed? It turns out many of the techniques that work for other blogs apply here, and regular attention can yield big gains.

1. Am I using snappy headlines?

Like it or not we live in a news world dominated by sites like Upworthy, Business Insider, Gawker & Huffpo. Ryan Holiday gained fame using a gonzo style as director of marketing at American Apparel. Ryan argues that old-style yellow journalism is back with a vengence.

Click bait asside, you *do* still need to write headlines that will click. What works often is for your title to be a little sound bite, encapsulating the gist of your post, but leaving enough hook that people need to click. Don’t be afraid to push the envelope a bit.

Also: Which tech do startups use most?

2. Line up those share buttons & feedburner

Of course you want to make the posts easy as hell to share. Cross posting on twitter, linkedin, facebook and whereever else your audience hangs out is a must. Use tools like hootsuite & buffer to line up a pipeline of content, and try different titles to see which are working.

You’ll also want to enable feedburner. Some folks will add your blog to feedly. Subscriber counts there can be a good indication of how it is growing in popularity too.

Related: Do today’s startups assemble software at their own risk?

3. Watch & listen to google analytics

You’re going to keep an eye on traffic by installing a beacon into your page header. There are lots of solutions, GA being the obvious one because it’s free. But how to use it?

Ask yourself questions. Who are my readers? Where are they coming from? How long do they spend on average? Do some pages spur readers to read more? Is there copy that works better for readers? Are my readers converting?

It’ll take time if you’re new to the tool, but start with questions like those.

Read: Is automation killing old-school operations?

4. Optimize your SEO a little bit

Although you don’t want to go overboard here, you do want to pay some attention. Using keyword rich titles, and < h2 > tags, along with wordpress SEO plugins that support other meta html tags means you’ll be speaking the language search engines understand. Add tags & categories that are relevant to your content.

Don’t overdo it though. Stick to a handful of tags per post. If you add zillions with lots of word order combinations & so forth, this kind of stuff may tip of the search engines in ways that work against you.

Check out: How to hire a developer that doesn’t suck

5. Search for untapped keywords

When I first started getting serious about blogging, I had an intern helping me with SEO. She did some searching with the moz keyword research tools and found some gems. These are searches that internet users are doing, but for which there still is not great content for.

For example if results showed “cool tech startups in gowanus brooklyn” had no strong results, then writing an article that covered this topic would be a winner right away.

These are big opportunities, because it means if you write directly for that search, you’ll rank highly for all those readers, and quickly grow traffic.

Read also: 5 things toxic to scalability

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Best of Startup Content on Scalable Startups

strawberries

Join 28,000 others and follow Sean Hull on twitter @hullsean.

Costs

Costs of techops can involve short-term architectural, decisions, but what about the longer term affects of choices? Do cto’s underestimate operational costs?

A stack of…

These days the full stack of a internet or mobile startup involves a lot of varied components, from Chef, Puppet & Ansible, to Nginx, haproxy, redis, solr and some database like MySQL or Postgres on the relational end of the spectrum, or Mongodb, Hbase or Cassandra on the NoSQL side. What type of challenges does this pose to a team? I’m curious,
Do startups assemble at their own risk?

Most used tech

Leo Polovets ran some stats over the Angellist data of startups. He wanted to know Which tech do startups use most? and I summarized the results.

Death of ops?

These days with all the talk of automation, I’ve heard heard developers & even CTO’s argue of a diminishing need for backend administrators. Do startups still need techops?

Speed as a feature

Is Fred Wilson right to say speed is a feature? What does this mean for those migrating or already running in the cloud? How does scalability come into play?

Avoiding outages

Are many outages avoidable? Did Airbnb have to fail?

Performance Review

Reviewing architecture & site speed is a type of engagement that a lot of startups can benefit from. Here’s my Anatomy of a performance review.

Let things fail

Does it sometimes make sense to let things break a little? A tale of managed failure.

Young founders

I worked at one startup with a CTO just out of college. Although they were flush with cash & had real problems scaling, communication problems ultimately soured the engagement. Are you too young to be a boss?

80 million fix

Sometimes fixing serious performance bottlenecks can get a site back up on it’s feet. In this success story they went on to get acquired weeks after the fix. In tongue in cheek fashion I askWhere’s my 80 million dollars?

CTO’s should never do

There are times to get into the trenches. But what if it sacrifices leadership?What should a CTO never do?

Startups too cool for school

Joining YC but have no ideas? No problem. Is my startup too cool for your business school?

Instant business, just add water

Can a business be built in just a weekend? Is there a problem with startup bootcamps?

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Today’s startups: assemble at your own risk

devops divide

I was talking with Todd Hoff recently over at High Scalability about a trend I’ve seen of late.

ME: I really liked this post by Zoli Kahan from Clay.io.  AWS, cloudflare, docker, haproxy, mysql, mongo, memcache, ansible.  They use just about every technology being talked about these days.  

Todd: Yah, that’s why I asked to republish it. I thought it was a good updated sampler stack.

ME: That said I defy you to find a team that actually *KNOWS* all those technologies.  

Todd: Agreed. Systems are a lot of assembly these days, which doesn’t mean we know how to build the parts being assembled.

Join 28,000 others and follow Sean Hull on twitter @hullsean.

The article I was referring to was: How Clay.io Built their 10x Arch Using AWS, Docker, HAProx & Lots More

1. Dizzying array of technologies in use

I’ve been working with startups since the mid-nineties. In those days most application stacks consisted of a PHP application running on Apache, with Oracle on the backend. Both webserver & db ran on Sun Solaris. Hardware was reliable. Most attention was focused on fitting everything in memory, and monitoring the servers for swapping, and disk failure. Boy have those days changed.

I see dozens of startups each year, so I see a lot of very cutting edge environments. Here’s a peak at what I’m seeing these days:

Database: MySQL, Postgres & Oracle, to Mongodb, Cassandra & Couchbase

Caching: Memcache or Redis

Search: Solr

Webservers: Apache, Nginx, Lighttpd

Load balancers: haproxy, Zen

Languages: PHP, Python & Ruby

Publishing: Drupal, WordPress, Joomla

Continuous Integration: Jenkins

Metrics: Cacti, collectd, NewRelic

Monitoring: Nagios, Ganglia, Munin, OpenNMS

Automation: Ancible, Chef, Puppet, Docker & Vagrant

Logs: Logstash

DDOS & CDN: Cloudflare, Ultradns

Whew… That’s a long list!! And we’re not even considering the API’s that many applications are now building on.

Also: Are generalists better at scaling the web?

2. Shortcuts abound

Startups early on, don’t have enough working capital to hire a huge engineering team. So that means everyone is stretched. With a list of technologies that is ever growing, something’s gotta give.

These may cut corners by handing the web & technical operations work to a developer who has some skills. But I continue to ask… Does a four-letter word divide dev & ops?

Read: Which tech do startups use most?

3. More things to break & master

Ownership of a software stack, such as a database means mastery of…

o features in current versions
o bugs of current versions
o vulnerabilities of various versions
o troubleshooting
o best practices
o backup & reliability

For example a lot of shops where I dig into the database, I find low hanging fruit, such as misconfigured startup settings, table layout or index usage.

I see similar things when a networking expert pours over the haproxy configuration, or runs ping tests across the network. Most of these components are setup with fairly vanilla configurations, leaving loose ends and frayed threads.

Check out: Why I can’t raise the bar at every firm

4. Many startups carrying technical debt

I’ve seen a growing reliance on ORM’s which is worrying. Build your foundation on a crutch, and it gets very hard to eliminate down the line. Here are Ward Cunningham’s warnings on technical debt.

Related: Are SQL Databases Dead?

5. Long term support & viability

At one five year old firm, I was brought in to address scalability problems. I met with the team and was asked to provide a comprehensive review. The first thing I found was all the original engineers had long since left, so the code was new for everyone. As I dug my heels in, I found multiple versions of Apache along with Nginx on some other servers. Their stack was built on a patchwork of Python, Ruby & PHP. Then digging in further, we found a complicated web of dependencies for digital assets, mounted across servers & unmonitored.

Lack of standards is common in environments like these. Without an operational or architectural lead, developers are left to make decisions with what is directly in front of them. Though a decision of what language to use may appear simple at the outset, it carries long term consequences.

Will that language or technology be supported in five years? Will the community survive? Will your firm be able to hire people with that skill set? Will engineers still be excited about it?

See also: Is high availability overrated? Is five nines a myth?

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

When prospects mislead

MUHAMMAD ALI ROCKS GEORGE FOREMAN ON THE JAW

While a story is fresh in ones mind, it’s a great time to tell it. And so I set out to putting pen to paper about a recent consulting war story.

Join 28,000 others and follow Sean Hull on twitter @hullsean.

A financial services firm reached out to me, asking about services. We discussed the project plan, and the day after the call I sent along a quote. I suggested three options, a weekly fee, a monthly one, or monthly with advance payment.

They decided to go with option C, and we arranged a kickoff meeting.

1. Level setting on trust

I’ve done this kind of work for so long, and worked with so many clients over the years, that it sometimes becomes second nature. I arrived, and we chatted amicably. I asked him about his wikipedia page, which he seemed excited to talk about.

I was surprised that there wasn’t a check ready, as we had decided on advanced payment in full, but didn’t make a mention right away. He then tried to dial in his partner, but that just went to voicemail. So we continued the meeting without him.

I don’t know how important the meeting was to both team members, but they were both on the invite & emails. His partner never called back through the meeting either.

Read this: When migrating from Oracle to MySQL Prepare to Bushwack

2. Negotiations is part art & dance

Interestingly I had met up with some colleagues the night before over italian food. I mentioned I was meeting a new prospect the next day, but had reservations about whether they had really decided to hire me, or were just still prospecting.

So during the meeting I was somewhat conscious of that question. Are we already in exploratory, discovery mode? Has the project even begun? That’s a question, and from what I sensed it was still an open one.

As the meeting wore on, questions about oracle licenses, versions, and EC2 configurations came up. Furious note taking continues.

Related: Which tech do startups use most?

3. Time & mismanagement

One thing that comes up for me in these situations is questions of time management. In order to work with a new client, I must clear my schedule, and make time available. That has a value to start with. When it turns out a project isn’t actually ready yet, it becomes an awkward stumble out of the gates.

Also: Is automation killing the sysadmin job role?

4. Can you research this one thing

As I raised various concerns about Oracle, the data loader portion, and unknowns around how that software worked, the prospect asked if I could do a little research for them.

This is where things started to crack. Rather than answer the question, I made a more aggressive nod to the question on my mind: Have we really started on this project yet? I explained that I was confused, and gathered from our email this this was a kickoff meeting. The tension in the air rose noticeably.

He then explained “Well we’re still waiting to hear back from a vendor about XYZ”. From there I began to gather up my things.

Check this: What can fashion week teach Chad Dickerson about Net Neutrality?

5. Watch out for those Rothkos

As I stand up I comment on the digs. “Is this shared office space, those look like Rothkos?” I ask. “Nope this is all ours, my wife is a collector & art dealer. We have some real Warhol’s too”. “Wow…”, I respond, “tough business to be in!”. With that he says “Well it is very volatile, we can be out of business in a month.”

My take away here isn’t to be wary of all new prospects. Each person or business has their own *style* of doing business. Rather, until you’ve established trust with a new client, consider that you may not yet be working on the project at all.

And with that the dance continues. While you may wish to demonstrate and illustrate your knowledge, and the solutions you’d recommend, beware of solving the problem before you’re even hired!

Read: Are SQL Databases Dead?

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters