Category Archives: CTO/CIO

Locking down cloud systems from disgruntled engineers

medieval gate fortified aws

I worked at a customer last year, on a short term assignment. A brilliant engineer had built their infrastructure, automated deployments, and managed all the systems. Sadly despite all the sleepless nights, and dedication, they hadn’t managed to build up good report with management.

Join 32,000 others and follow Sean Hull on twitter @hullsean.

I’ve seen this happen so many times, and I do find it a bit sad. Here’s an engineer who’s working his butt off, really wants the company to succeed. Really cares about the systems. But doesn’t connect well with people, often is dismissive, disrespectful or talks down to people like they’re stupid. All burns bridges, and there’s a lot of bad feelings between all parties.

How to manage the exit process. Here’s a battery of recommendations for changing credentials & logins so that systems can’t be accessed anymore.

1. Lock out API access

You can do this by removing the administrator role or any other role their IAM user might have. That way you keep the account around *just in case*. This will also prevent them from doing anything on the console, but you can see if they attempt any logins.

Also: Is AWS too complex for small dev teams?

2. Lock out of servers

They may have the private keys for various serves in your environment. So to lock them out, scan through all the security groups, and make sure their whitelisted IPs are gone.

Are you using a bastion box for access? That’s ideal because then you only have one accesspoint. Eliminate their login and audit access there. Then you’ve covered your bases.

Related: Does Amazon eat it’s own dogfood?

3. Update deployment keys

At one of my customers the outgoing op had setup many moving parts & automated & orchestrated all the deployment processes beautifully. However he also used his personal github key inside jenkins. So when it went to deploy, it used those credentials to get the code from github. Oops.

We ended up creating a company github account, then updating jenkins with those credentials. There were of course other places in the capistrano bits that also needed to be reviewed.

Read: Is aws a patient that needs constant medication?

4. Dashboard logins

Monitoring with NewRelic or Nagios? Perhaps you have a centralized dashboard for your internal apps? Or you’re using Slack?

Also: Is Amazon too big to fail?

5. Non-key based logins

Have some servers outside of AWS in a traditional datacenter? Or even servers in AWS that are using usernames & passwords? Be sure to audit the full list of systems, and change passwords or disable accounts for the outgoing sysop.

Also: When hosting data on Amazon turns bloodsport?

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Why is Reddit’s CTO Martin Weiner special?

reddit cto martin weiner

I was reading the New Stack recently, and stumbled on Joab Jackson’s article about Reddit CTO Martin Weiner.

Join 32,000 others and follow Sean Hull on twitter @hullsean.

He had some pretty on point observations about stable applications & predictability.

1. Because he should know

He was technical lead at Pinterest & now he’s CTO at Reddit. Those are pretty serious creds. But why wouldn’t he advocate the coolest new language, or baddest new NoSQL database?

Also: Does Amazon eat it’s own dogfood?

2. Because you can google boring tech

That’s right, picture yourself the ops team or developer who’s gotten paged in the middle of the night. You rub your eyes and look at the computer screen. You’re getting an error on MySQL. You dial up google & find the answer. You fix it & fall back to sleep!

“If it is 3 A.M., and your site is broken, because it will break, whatever the problem is with MySQL, the answer will up on Google”

Related: Did Dropbox have to fail?

3. Because you want predictability

New unproven technologies may solve old problems, but they’re also unpredictable. They break in new ways. They’re still immature. That’s dangerous.

What you really want is predictability & you get that from boring tech.

Read: Is Amazon too big to fail?

4. Because you can hire for it!

There are lots of technologies that have been around for a while, that are stable, reliable & *gasp* you can find people who know them!

“Python is a really mature tech. Everyone knows how to use it, and you can hire for it”

Also: Is Amazon Redshift a game changer?

5. Because everything breaks

While you’re discovering the coolest bleeding edge technology, and imaging the castles you can build, don’t forget that it will break at some point.

“If it breaks in the middle of the night, they wake up and fix it”

With boring tech, the fix is within reach.

Also: Is data your dirty little secret?

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

5 ways to level up as cloud expert

aws certified

Cloud computing is blowing up! But don’t take my word for it, read this recent NY Times piece: Tech companies clamor to entice cloud computing experts.

Join 32,000 others and follow Sean Hull on twitter @hullsean.

Still don’t believe me? Get on the phone with a recruiter or two. They’ll convince you because they’ve got companies banging down the door looking for talent that is plainly in SHORT SUPPLY. And that’s the supply *you* want to be. :)

Check Gary’s Guide Jobs, or the ever popular Angel List Jobs. There’s also Stack Overflow jobs and many more.

1. Become a book reviewer

You’ve already got a technical background, and want to hone those skills. Take a look at technical book reviewing.

Manning is putting out some excellent technical books these days. Apply here to be a reviewer.

Also take a look at Pragmatic Bookshelf. They are are looking for reviewers too.

In either case you can expect to spend time reading a book chapter by chapter, as it’s written, offer strategic or layout advice, feedback on presentation, comprehension, and edits.

Also: When hosting data on Amazon turns bloodsport

2. Join an Open Source project

There are millions. Flip through github to some that you’re interested in. Contribute a bug fix or comment, reach out to the project leaders.

Afraid to dive in? Join one of the forums or google discussion groups, and lurk for a while. Ask questions, offer a helping hand!

Related: Is Amazon too big to fail?

3. Self-paced labs

Online education is blowing up, and for good reason. They get the job done & for the right price!

One of my favorites for AWS Certification is the A Cloud Guru courses. These offer lecture style introduction to all levels of AWS from Sysops Administration, Developer & Solutions Architect to Devops, Lambda & CodeDeploy.

The courses are priced right, and geared directly towards Amazon’s certifications. That helps you focus on the right things.

Amazon also partners with qwiklabs to offer courses geared towards getting certified. There are specific ones for the associate & professional certification, and many others besides.

You’ll need to signup for AWS Activate first, before you can use these qwiklabs. They offer you 80 credits right out of the gate.

For the next two weeks many of the courses are free! One thing I really like is they include a free temporary aws login for the students. That way there’s no risk of deploying infrastructure, and accidentally getting a big bill at the end of the month.

The labs though are more like reading documentation versus a nice video course lecture. So you the student have to do a lot more to get through it.

Read: Are we fast approaching cloud-mageddon?

4. Coursera, Khanacademy & Udemy

There’s a free class on Coursera called Startup Engineering by Balaji Srinivasan & Vijay Pande. Some pretty amazon material & lectures in here, and if you’re determined, it’s 12 weeks that will get you going on the right foot!

KhanAcademy has a great many courses on computer programming. Awesome and free stuff here. One particularly interesting is their hour of code. For those hesitant, that’s an easy way to jump in!

There is also udemy, which offers some great material on cloud computing. Notice that the certification courses are the same ones from A-Cloud Guru!

Also: Are SQL databases dead?

5. Interview tests

Apply to jobs. Even if you’re unsure if that is your dream job. Why? Because they often include a test to find out about your technical chops. Diving into these tests is a great way to push your own edge. You may do well, you may not. Learn where your weaknesses are.

I especially like the ones where you’re asked to login to a server, configure some things, write some code, and solve a real problem. Nothing beats a real-world example!

Also: Why dropbox didn’t have to fail?

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Sean Hull interviewed on the Doppler Cloud podcast

I recently got a chance to talk with Mike Kavis over at Cloud Technology Partners. It was fun to get away from the keyboard, and in front of the microphone for a change.

Join 32,000 others and follow Sean Hull on twitter @hullsean.

1. Docker

Docker is making deployments easier & easier. But as the pace accelerates, are we introducing vulnerabilities & scalability problems faster than we can fix them?

Also: Are generalists better at scaling the web?

2. Redshift

I’ve blogged that I don’t work with recruiters but I do chat with them regularly.

In a recent conversation a recruiter asked me:

“Why is it that suddenly everyone is looking for Redshift?”

I’m seeing the same trend. And if you look at Hadoop you might see why. Writing SQL queries against Redshift data is wildly simpler than writing EMR jobs for Hadoop.

Related: Why Dropbox didn’t have to fail?

3. Devops automation

These days I hear a lot of talk that all operations is software development. Are you still SSHing into boxes. You’re doing it wrong!

Read: When hosting data on Amazon turns bloodsport

4. Hardware solves all speed problems

Having performance problems? Scale out! Database slow, scale up! These days it seems the old short sighted way of thinking is back with a vengence. Throw hardware at the problem and kick the can down the road.

Also: How to hire a developer that doesn’t suck?

5. Amazon disrupting VC

During dot-com version one-point-oh, you’d need hundreds of thousands to buy hardware & software licenses to get an idea off the ground. That necessarily meant real VC money to get off the ground.

Amazon web services & on-demand computing has brought world class infrastructure to even the smallest startups. For just dollars, they can get started.

Now we’re seeing startups get going with micro investments from the likes of Angel List syndicates. Cutting traditional VCs right out of the equation.

Also: Is Amazon too big to fail?

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Does FedRAMP formalize what good devops already do?

fedramp-logo

amazon-govcloud

Amazon’s GovCloud provides a specialized region within Amazon’s global footprint of datacenters. These are hosted within the United States, and provide a subset of the full Amazon cloud functionality.

Join 32,000 others and follow Sean Hull on twitter @hullsean.

However, hosting within GovCloud is not the whole story. Beyond this, you’ll want to implement FedRAMP compliant procedures & policies.

Are these policies new? As a seasoned systems administrator of Unix & Linux networks, you’ll likely find these very familiar best practices. What they do however, is formalize those into a set of procedures for testing compliance. And that’s a good thing.

1. Use a bastion box

A bastion box is a single point of entry for all your SSH activity. Instead of allowing SSH access to any of your servers from *anywhere* on the internet, you limit it to one box. This box is hardened with multi-factor authentication for security, only opens port 22, monitors & logs access, and funnels movement to all your other boxes. Thus you gain a virtual perimeter that you’re already familiar with in more traditional firewall setups.

Also: Ward Cunningham explains the high cost of technical debt (video)

2. Monitor & scan for vulnerabilities

Monitoring, scanning & logging are all key facilities for security management. Regular patch management of each of your servers, is essential to protect from newly discovered vulnerabilities. FedRAMP also requires scanning by tools such as Nessus or Retina.

Also centralizing your authorization, access & error logs allows easy monitoring & alerting of threats & improper access attempts.

Related: Do managers underestimate the cost of operations?

3. Policy of least privilege

The policy of least privilege is an old friend in computing & managing unix systems. It means first to eliminate all privileges (default to none) and then grant only those a user requires to do his or her work.

In Amazon it means not using the root account for provisioning infrastructure, it means a clear separation of dev, test & production environments. It limits who can access production & especially make changes there. It limits who can see sensitive data.

As well, you’ll use Access Control Lists (ACL’s) and security groups to control which servers can reach which other servers, whom on the internet can touch specific servers & ports, and so forth. These are the Amazon Cloud equivalent of perimeter security you may be familiar with in more traditional firewalls.

Read: When hosting data on Amazon turns bloodsport

4. Encrypt your data

If you want to be truly secure, you’ll want to encrypt your data at rest. You can do this by using encrypted filesystems in Linux. That way data is in a digital envelope, even on disk. Only when data is read into memory is it unencrypted. This provides additional insurance, because your EBS snapshots, backups & so forth are all hidden from prying eyes.

Also: Why dropbox didn’t have to fail

5. Conclusion

Amazon’s GovCloud provides access to a subset of their cloud offerings including EC2 their elastic compute cloud virtual servers, EBS the elastic block storage their own storage area network, S3 for file storage, VPC, IAM, RDS, Elasticache & Redshift.

FedRAMP formalizes what good systems administrators do already. Secure systems, deliver reliability & high availability & protect from unauthorized entry.

Also: Is Amazon too big to fail?

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Why Dropbox didn’t have to fail

dropbox outage dec 2015

Dropbox is currently experiencing a *major* outage. See the dropbox status page to get an update.

Join 32,000 others and follow Sean Hull on twitter @hullsean.

I’ve written about outages a lot before. Are these types of major failures avoidable? Can we build better, with redundant services so everything doesn’t fall over at once?

Here’s my take.

1. Browse only mode

The first thing Dropbox can do to be more resilient is to build a browsing only mode into the application. Often we hear about this option for performaing maintenance without downtime. But it’s even more important during a real outage like Dropbox is currently experiencing.

Not if but *when* it happens, you don’t have control over how long it lasts. So browsing only can provide you with real insurance.

For a site like Dropbox it would mean that the entire website is still up and operating. Customers can browse their documents, view listings of files & download those files. However they would not be able to *add* or change files during the outage. Thus only a very small segment of customers is interrupted, and it becomes a much smaller PR problem to manage.

Facebook has experienced outages of service. People hardly notice because they’ll often only see a message when they try to comment on someone’s wall post, send a message or upload a photo. The site is still operating, but not allowing changes. That’s what a browsing only mode affords you.

A browsing only mode can make a big difference, keeping most of the site up even when transactions or publish are blocked.

Drupal is an open source platform that powers big publishing sites like Adweek, hollywoodreporter.com & economist.com. It supports a browsing only mode out of the box. An outage like this one would only stop editors from publishing new stories temporarily. It would be a huge win to sites that get 50 to 100 million with-an-m visitors per month.

Also: Is Amazon too big to fail

2. Redundancy

There are lots of components to a web infrastructure. Two big ones are webservers & databases. Turns out Dropbox could make both tiers redundant. How do we do it?

On the database side, you can take advantage of Amazon’s RDS & either read-replicas or Multi-AZ. Each have different service characteristics, so you’ll need to evaluate your app to figure out what works best.

You can also host MySQL, Percona or Mariadb direclty on Amazon instances yourself & then use replication.


Using redundant components like placing webservers and databases in multiple regions, Dropbox could avoid a major outage like they’re experiencing this weekend.

Wondering about MySQL versus RDS? Here are some uses cases.

Now that you’re using multiple zones & regions for your database the hard work is completed. Webservers can be hosted in different regions easily, and don’t require complicated replication to do it.

Related: Are SQL databases dead?

3. Feature flags

On/off switches are something we’re all familiar with. We have them in the fuse box in our house or apartment. And you’ll also find a bigger larger shutoff in the basement.

Individual on/off switches are valuable because they allow us to disable inessential features. We can build them into heavier parts of a website, allowing us to shutdown features in an emergency. Host components in multiple availability zones for extra piece of mind.

Read: 5 Things toxic to scalability

4. Simian armies

Netflix has taken a more progressive & proactive approach to outages. They introduce their own! Yes that’s right they bake redundancy & automation right into all of their infrastructure, then have a loose canon piece of software called Chaos Monkey that periodically kills servers. Did I hear that right? Yep it actually nocks components offline, to actively test the system for resiliency.

Take a look at the Netflix blog for details on intentional load & stress testing.

Also: When hosting data on Amazon turns bloodsport

5. Multiple clouds

If all these suggestions aren’t enough for you, taking it further you could do what George Reese of enstratus recommends and use multiple cloud providers. Not being dependant on one company could help in many situations, not just the ones described here.

Basic Amazon EC2 best practices require building redundancy into your infrastructure. Virtual servers & on-demand components are even less reliable than commodity hardware we’re familiar with. Because of that, we must use Amazon’s automation to insure us against expected failure.

Also: Why I like Etsy’s site performance report

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Five things I learned at NY CTO Summit 2015

cto summit 2015

Enjoyed attending the New York CTO Summit yesterday with a notable list of presenters. Looking forward to the slides. Links to follow.

Join 32,000 others and follow Sean Hull on twitter @hullsean.

1. Product is a reflection of teams

Conway’s law was repeated by three different presenters!

Also: Is the difference between dev & ops a four-letter word?

2. Agile government

Government efficiency can be tackled with startup efficiencies!

Related: Is AWS enabling Angellist to boil the VC business?

3. Learning culture

There are lots of benefits to building a learning culture, not least is making the business succeed.

Read: Do managers underestimate operational cost?

4. Don’t report to finance

Let’s remember how important which teams report to whom is. It can make or break your technology initiatives.

Also: Is Amazon too big to fail?

5. Course correction & size

The cost of changing course gets bigger as your org does.

Also: Airbnb didn’t have to fail

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Are software benchmarks to blame for Volkswagens woes?

volkswagen emissions

With the recent media attention Volkswagen has gotten, a lot of folks are wondering, how could that happen? Aren’t there checks & balances?

Join 32,000 others and follow Sean Hull on twitter @hullsean.

Then I ran across this observation on Todd Hoff’s brilliant blog High Scalability


Is what Volkswagen did really any different that what happens on benchmarks all the time? Cheating and benchmarks go together like a clear conscience and rationalization. Clever subterfuge is part of the software ethos. There are many many (search google) examples. Cars are now software is a slick meme, but that transformation has deep implications. The software culture and the manufacturing culture are radically different.

What exactly does all of this mean?

1. MySQL & Aurora

I was recently chatting with a colleague of mine Bret Miller who runs DeepSQL an adaptive database platform compatible with MySQL. He said:

“We’re actually doing testing against Aurora, but we recently had a couple customers do it independently with more challenging loads.   Didn’t see the performance stated in the marketing stuff. ”

My response was… “Yeah.  Aurora looks to be a win on the HIGH AVAILABILITY front. 

On the scalability front, MySQL has certain limitations in it’s core.  So i’m not surprised that the marketing material was grandiose in it’s promises.  

The best way to improve mysql performance is to tune queries.  As you’re writing your application, and when you want to boost performance.  ”

And so it goes.

Also: Can hosting data on Amazon turn bloodsport?

2. Redis & Memcached

Then I stumbled upon Salvatore Sanfilippo. He is the author of the brilliant & phenomenally successful NoSQL database called Redis. Turns out that another famous blogger was making some sweeping statements about Memcached & Redis and Salvatore ended up defending Redis in a blog post titled Clarifications about Redis.

The topic turned to benchmarks. Which lead me to another post titled
Why we don’t have benchmarks.

Heard this before?

Related: Did Airbnb have to fail?

3. Is Mongo webscale?

When Mongo was first releasing it’s benchmarks, the media went wild. And DBAs were scratching their heads. This fabulous video captures the sentiment of the time. :)

Read: Is Amazon too big to fail?

4. Oracle meets David DeWitt

In the 80’s Oracle began to forbid publishing benchmarks. After seeing a research paper by David DeWitt, Larry Ellison amended the End-user-license-agreement to include the DeWitt Clause. Later other database vendors followed.

It’s easy to see why. Benchmarks by their very nature depend on so many factors. It’s inevitable that those factors will be carefully picked by each platform to highlight it’s strengths.

Also: Are SQL databases dead?

5. Product versus disks

It is inevitable that all of this continues. When we reside at the level of the business, we perceive the product & its performance through that lens.

When we dive down to the level of disks, buses, cpus, network latency, multi-tenant clouds and a myriad of other factors, the waters are never so clear.

So remember your mileage may vary and buyer beware are as true today as they ever were.

Also: 5 Things are toxic to scalability

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

When hosting data on Amazon turns bloodsport

reddit aws outage

There’s a strong trend to automation across the cloud. That’s a great thing for startups because it reduces operational headaches & lets them focus on building products.

Join 31,000 others and follow Sean Hull on twitter @hullsean.

But as that trend begins to touch the database tier, all sorts of complications emerge. Let’s take a look at some of the tradeoffs.

1. Database as a service trend

I was recently reading Baron Schwartz’s article on the trend to database as a service.

I work with a lot of venture backed startups & pay close attention to what’s happening in New York & SF. From where I’m standing I see a similar trend. As automation simplifies management across the application stack, from load balancers to web & search servers, the same advantages are moving to database management.

Also: How to automate MySQL analysis on Amazon RDS

2. How Amazon RDS helps

Amazon’s RDS offers firms a data solution for Oracle & SQL Server as well as MySQL. For those just starting, it offers a long list of advantages.

o quick push-button deployment in minutes
o standardized parameters settings that just work
o ability to scale up or down from the dashboard
o automated backups
o multi-az so you can sleep at night

This brings a huge advantage to startups. Many have a team of developers but aren’t large enough to need an operations team and can’t afford a dedicated database administrator.

Amazon is obviously helping these firms raise the bar. And that’s a good thing.

Related: RDS or MySQL 10 use cases

3. How Amazon RDS hurts

As you get bigger, your needs will grow too. You’ll have tens of millions of customers, and with more customers comes an even higher bar. Zero downtime becomes critical. It’s then that Amazon’s solution starts to become frustrating.

Unpredictable upgrades

MySQL upgrades on RDS are a messy activity. Amazon will restart the instance, backup the instance, perform the upgrade then restart again. Each of these restarts takes a few minutes. The whole operation may have you down for ten minutes. This becomes more frustrating when your hands are completely tied. You don’t know when or what will happen!

When you roll-your-own instance, an upgrade can be performed in a matter of seconds. No instance restarts are necessary and you can monitor the process to know exactly where you are. This is the kind of control you’re going to want if you have millions of customers relying on your site & uptime.

Unnecessary slow restarts

When you apply parameter changes on RDS, some require a MySQL restart. Amazon forces the whole server to restart, increasing this downtime from a few seconds (when you roll your own) to many minutes. And while some parameters can be changed online, Amazon can provoke some strange behavior that is not always predictable.

With the frequency of these types of changes, you’ll quickly grow tired and frustrated with RDS.

EBS Snapshots are not portable

As mentioned above Amazon uses it’s standard filesystem snapshot technology to perform backups. While this works well, it can be slow & unpredictable in a multi-tenant environment.

When you roll your own, you can take advantage of xtrabackup, and perform hot backups against your database with zero downtime. This is a real godsend. What’s more they are portable, and can be moved to any other server even ones not hosted in Amazon’s cloud!

Promoting a read-replica is slow too!

One feature that Amazon touts is creating copies or “read replicas” of your data. These are great and can facilitate easy copying of data. However promoting these again brings unnecessary restarts which are slow.

When you roll your own, you can promote a read-replica or read-only slave in seconds. A few seconds can seem invisible to end users, while minutes will be perceived as a real outage or downtime.

Read: Is zero downtime even possible with RDS?

4. Is migration an option?

So what to do? As I mentioned above, there are real advantages to startups deploying their first database. It really does help. I would argue for many it can be a good place to start.

If you’re starting to outgrow RDS and frustrated with the limitations, performance tuning headaches & unneeded downtime, luckily you have options.

Migrating off of RDS onto a physical server can be done in a number of ways.

o slave off of the master

Here you build a MySQL slave on a standard EC2 instance, with your RDS instance as the master. When you’re caught up, bring your site down temporarily. Reset the slave & set to read-write mode. Then point your webservers at your new EC2 instance and bring the site back up. If done carefully 10 to 20 seconds of downtime should be plenty.

Don’t forget to run through the process with a firedrill first!

o dump & import

Another way to move your data may be MySQLdump. This option would be slower & bring a lot more downtime, but possibly necessary in some cases.

Also: 5 Reasons to move data to Amazon Redshift

5. Speed: It’s the database

Fred Wilson says speed is the number one feature of a web application. If customers are frustrated & waiting, they may leave & not come back. On the web it can be everything.

Many firms are rushing to database as a service to simplify administration. While that’s wonderful at the beginning, as you grow performance will become more of a day-to-day concern. And when it does, the database is going to be big on your list of headaches.

Web application performance inevitably involves the database and while it does, your decision to choose database as a service may come into question. Don’t be afraid to bite the bullet and manage things yourself when that time comes.

Also: Is upgrading RDS like a shit-storm that will not end?

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

What Deborah Tannen taught me about conversation & interruption

tannen you just dont understand

I was recently invited to attend a charity event in Washington DC. Dinner was a catered affair of 300 with a few senators & Muhammad Yunus there to talk about micro financing.

After dinner we broke up into some smaller groups, and had great conversations into the night. It was interesting to me as I don’t often rub elbows with lobbyists & political animals. While we were all talking, the subject of language came up, and in particular how different people’s styles affect how they come off.

Join 30,000 others and follow Sean Hull on twitter @hullsean.

I became really engaged, as this topic has always interested me. I was introduced to the ideas of Deborah Tannen. She’s a professor of linguistics from Georgetown University, and an expert on the topic.

Afterward, I went straight to my kindle & bought here seminal book “You Just Don’t Understand”.

Boy do I understand a lot more now.

1. Conversational style varies by culture & gender

Across cultures, from europeans to Asians, North to South Americans, conversational styles vary. Some pause longer between breathes, while others make briefer pauses. Some deem conversation more like judge & jury, where each should be afforded carefully the chance to take stage, while others prefer the casual chance to jump in, and constant overlap.

These differences lead to the sense of pushiness versus interest, interruption versus dominance. Interest versus boredom. Since all these cultures have a different style, it can get rather complicated interpreting someone’s intentions if you’re not from that culture.

What’s more these vary quite a bit between men & women.

Also: What I learned from Jay Heinrichs about click worthy blog titles

2. Report & rapport talk are different

Report talk is in public, perhaps at a lecture, or out with a large group of friends around the dinner table. There stories & conversation revolves around a larger group.

Rapport talk on the other hand is at home, among intimates.

She says that women tend to prefer the latter while men prefer the former. So in different circumstances it can appear that one or the other has “nothing to say”, when it actually revolves around their preferences of when to speak.

Related: Is automation killing old-school operations?

3. Like & respect

Women’s behavior & style of speaking is rooted in the goal of being liked. So there are many cases where they may downplay themselves, to reach a more equal state with those around them.

Men’s behavior & conversational style is based around seeking respect. This can often mean emphasizing differences, and not parity.

Read: Do managers underestimate operational cost?

4. Contest or connection?

Men often see the world through the lens of contest, especially in relationships with others. Women on the other hand tend to see it as an interconnected network. By building bonds you strengthen that network.

These two styles inform dramatically different behaviors in similar situations.

Also: Is Reid Hoffman right about career risk?

5. Interest or independence

Here’s another example of how men & women may see things differently.


When men change the subject, women think they are showing a lack of sympathy — a failure of intimacy. But the failure to ask probing questions could just as well be a way of respecting the other’s need for independence.

So it seems styles & priorities inform intention & interpretation of a lot in conversations.

Although all of this doesn’t resolve or put to rest these differences, being informed can certainly help a lot towards understanding.

Also: What I learned from the 37 Signals team about work & startups

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters