Category Archives: All

5 reasons to move data to Amazon Redshift

redshift amazon

Join 28,000 others and follow Sean Hull on twitter @hullsean.

Amazon is rolling out new database offerings at a rapid clip. I wondered Did MySQL and Mongodb just have a beautiful baby called Aurora? That was last month.

Another that’s been out for a while is the data warehouse offering called RedShift.

1. old-fashioned SQL interface

Ok, yes Redshift can support petabyte databases and this in itself is staggering to consider. But just after you digest that little fact, you’ll probably discover that it’s SQL compatible.

This is a godsend. It means the platform can leverage all of the analytical tools already in the marketplace, ones that your organization is already familiar. Many are already certified on RedShift such as Looker and Chart IO.

Also: Are SQL Databases Dead?

2. Lots of ways to load data

After you build your first cluster, the first question on your mind will be, “How do I get my data into RedShift?” Fortunately there are lots of ways.

Stage in S3 & use COPY

Everyone using AWS is already familiar with S3, and RedShift uses this as a staging ground. Create a bucket for your csv or other datafiles, then parallel load them with the special COPY command.

For those coming from the Oracle world, this is like SQL*Loader, which doesn’t go through the SQL engine, but directly loads data as blocks into datafiles. Very fast, very parallel.

AWS Data Pipeline

Some folks are leveraging the AWS Data Pipeline to copy MySQL tables straight into RedShift.

FlyData for Amazon MySQL RDS

I’m in the process of evaluating FlyData sync. This is a service based solution which connects to your Amazon RDS for MySQL instance, capturing binload data much like Oracle’s GoldenGate does, and ships it across to RedShift for you.

If you have constantly changing data, this may be ideal as you don’t have a one-shot dataload option, implied by the basic COPY command solution.

Read: What is ETL and why is it important?

3. Very fast or very big nodes

There are essentially two types of compute nodes for RedShift, DW2 are dense compute running on SSD. As we all know, these are very fast solid state memory drives, and bring huge disk I/O benefits. Perfect for a data warehouse. They cost about $1.50/Tb per hour.

The second type is DW1 or so-called dense storage nodes. These can scale up to a petabyte of storage. They are running on traditional storage disks so aren’t SSD fast. They’re also around $0.50/Tb per year. So a lot cheaper.

Amazon recommends if you’re less than 1Tb of data, go with Dense Compute or DW2. That makes sense as you get SSD speed right out of the gates.

Related: What is a data warehouse?

4. distkeys, sortkeys & compression

The nice thing about NoSQL databases is you don’t have to jump through all the hoops trying to shard your data with a traditional database like MySQL. That’s because distribution is supported right out of the box.

When you create tables you’ll choose a distkey. You can only have one on a table, so be sure it’s the column you join on most often. A timestamp field, or user_id, perhaps would make sense. You’ll choose diststyle as well. ALL means keep an entire copy of the table on each node, key means organize based on this distkey, and EVEN the default means let Amazon try to figure it out.

RedShift also has sortkeys. You can have more than one of these on your table, and they are something like b-tree indexes. They order values, and speed up sorting.

Check: 8 Questions to ask an AWS expert

5. Compression, defragmentation & constraints

Being a columnar database, Redshift also supports collumn encodings or compression. There is LZO often used for varchar columns, bytedict and runlength are also common. One way to determine these is to load a sample of data, say 100,000 rows. From there you can ANALYZE COMPRESSION on the table, and RedShift will make recommendations.

A much easier way however, is to use the COPY command with COMPUPDATE ON. During the initial load, this will tell RedShift to analyze data as it is loaded and set the column compression types. This is by far the most streamlined approach.

RedShift also supports Table constraints, however they don’t restrict data. Sounds useless right? Execept they do inform the optimizer. What’s that mean? If you know you have a primary key id column, tell RedShift about it. No it won’t enforce that but since your source database is, you’re able to pass along that information to RedShift for optimizing queries.

You’ll also find some of the defragmentation options from Oracle & MySQL present in Redshift. There is vacuum which reorganizes the table & resets the high water mark, while it is still online for updates. And then there is Deep Copy which is more thorough, but takes the table offline to do it. It’s faster, but locks the table.
o deep copy

Related: Is Oracle killing MySQL?

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Did MySQL & Mongo have a beautiful baby called Aurora?

amazon aurora slide

Amazon recently announced RDS Aurora a new addition to their database as a service offerings.

Here’s Mark Callaghan’s take on what’s happening under the hood and thoughts from Fusheng Han.

Amazon is uniquely positioned with RDS to take on offerings like Clustrix. So it’s definitely worth reading Dave Anselmi’s take on Aurora.

Join 28,000 others and follow Sean Hull on twitter @hullsean.

1. Big availability gains

One of the big improvements that Aurora seems to offer is around availability. You can replicate with aurora, or alternatively with MySQL binlog type replication as well. They’re also duplicating data two times in three different availability zones for six copies of data.

All this is done over their SSD storage network which means it’ll be very fast indeed.

Read: What’s best RDS or MySQL? 10 Use Cases

2. SSD means 5x faster

The Amazon RDS Aurora FAQ claims it’ll be 5x faster than equivalent hardware, but making use of it’s proprietary SSD storage network. This will be a welcome feature to anyone already running on MySQL or MySQL for RDS.

Also: Is MySQL talent in short supply?

3. Failover automation

Unplanned failover takes just a few minutes. Here customers will really be benefiting from the automation that Amazon has built around this process. Existing customers can do all of this of course, but typically require operations teams to anticipate & script the necessary steps.

Related: Will Oracle Kill MySQL?

4. Incremental backups & recovery

The new Aurora supports incremental backups & point-in-time recovery. This is traditionally a fairly manual process. In my experience MySQL customers are either unaware of the feature, or not interested in using it due to complexity. Restore last nights backup and we avoid the hassle.

I predict automation around this will be a big win for customers.

Check out: Are SQL Databases dead?

5. Warm restarts

RDS Aurora separates the buffer cache from the MySQL process. Amazon has probably accomplished this by some recoding of the stock MySQL kernel. What that means is this cache can survive a restart. Your database will then start with a warm cache, avoiding any service brownout.

I would expect this is a feature that looks great on paper, but one customers will rarely benefit from.

See also: The Myth of Five Nines – Is high availability overrated?

Unanswered questions

The FAQ says point-in-time recovery up to the last five minutes. What happens to data in those five minutes?

Presumably aurora duplication & read-replicas provide this additional protection.

If Amazon implemented Aurora as a new storage engine, doesn’t that mean new code?

As with anything your mileage may vary, but Innodb has been in the wild for many years. It is widely deployed, and thus tested in a variety of environments. Aurora may be a very new experiment.

Will real-world customers actually see 500% speedup?

Again your mileage may vary. Lets wait & see!

Related: 5 Things toxic to scalability

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

How do you get prepared for Infrastructure Engineering jobs?

datacenter-rack

I just started contributing to a great site called Career Dean. It offers a forum where students and new college graduates can learn from those with established careers in industry.

Join 28,000 others and follow Sean Hull on twitter @hullsean.

A recent question…

Application infrastructure is not something we learned in my college, and it’s definitely not something I will learn anytime soon in my current job (I work as a mobile developer for a mid-sized startup). I also think it’s not something you can just goof around with in your own computer. 

Do companies prepare their software engineers when hiring infrastructure engineers, or do they all expect you to know your skills and tools? 

Also: Is automation killing old-school operations

For example, My guess is that Facebook has a huge infrastructure team making the site usable and fast for as many people as possible. Where can you learn that skills, or get prepared for that time of job? Do you think it is possible to self-learn those skills?

Here’s my take on some of this. Since the invention of Linux, experimenting with infrastructure has been within reach. In the present day there are some even better reasons to experiment & teach yourself about this important aspect of devops & backend server management.

Early Linux circa 1992

Before Linux (in the 80′s we’re talking about) it was a lot harder. Into the 90′s Linux came on the scene and you could cobble together parts, video, motherboard, memory, ide or scsi bus & disks & build a 486 tower. You could then start building linux. I mean because of course everything had to be hand rolled (compiled by hand & debugged usually)!

Also: Is five nines availability a myth in todays datacenters?

Present day virtualization

Fast forward 20 years, and it’s an incredible time to be messing with infrastructure. Why? Because virtualization means you can do it all right on your laptop.

Also: Are SQL databases dead?

What to learn

Start learning Vagrant. It automates the provisioning of virtual machines on your own desktop. You can boot those linux boxes to your hearts content, network between them, hack them, run services on them, build your skills.

I’d also recommend digging into docker. It is the lightening fast younger brother to Virtualization.

Also: Is Oracle trying to kill MySQL?

Fundamentals

You really need those fundamentals. Build some 1.x Linux kernels and see if you can get ‘em running. That’ll teach you some hacking & troubleshooting skills. Find forums to get answers.

Also take a look at CoreOS. It has some really cool stuff around infrastructure management & automation.

Also: Is the art of resistance important to devops success?

After all of that, you might want to play around with puppet or chef. Learn how to setup continuous integration, jenkins etc.

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Best of hiring posts on scalable startups

strawberries

Join 28,000 others and follow Sean Hull on twitter @hullsean.

Why I write about hiring

I’ve worked as a consultant for almost twenty years. Technology & professional services are pretty far removed from hiring, so why would I write about it?

As it turns out, finding projects, working with clients, and selling your skills & solutions has quite a lot in common to do with hiring.

As a services consultant, you’re more often a peer to technology directors & CTOs, while hiring for traditional roles is more of a boss employee relationship.

Recruiters

I’ve run into a lot of recruiters & hr folks over the years. Usually it means I’m talking to the wrong folks, as they’re gatekeepers & not decision makers. I wrote Why I don’t work with recruiters after some ups & downs.

Still they’re all a fact of life, and each of us has a role to play. So let’s play fair!

Games

I’ve always wondered, Is Hiring a numbers game? That is does it bend more to persistence & throwing spagetti at the wall, or deliberate, precision searches?

MySQL interview

If you’re looking for a database expert, I put together
Top MySQL DBA interview questions and then another one
Advanced MySQL DBA Interview questions.

These are helpful not just to candidates, but to hiring managers, hr, recruiters & everyone in between.

Mythical talent

Since as far back as I can remember, DBAs have been in short supply. In the 90′s I was doing primarily Oracle work. There were never enough technical dbas. Many came from business backgrounds, and didn’t have operating system & hardware fundamentals.

As startups shifted to open source databases in droves during the 00′s, the situation became even worse. I wrote about
The mythical mysql dba – where can we find one?

Will NoSQL databases continue the same trend?

Hire a developer

With a little light humor, we throw some opinions into the fray around hiring devs with How to hire a developer that doesn’t suck.

As devops gains momentum, some see peace between the old-school silos of developers & operations. Some see the need for ops being supplanted by developers. We have some opinions too.

AWS Interview

Are you looking for an Amazon Web Services expert, who knows how to scale in the cloud? Devops & automation also on your mind? Check out
8 Questions to ask an amazon ec2 expert.

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Best of Startup Content on Scalable Startups

strawberries

Join 28,000 others and follow Sean Hull on twitter @hullsean.

Costs

Costs of techops can involve short-term architectural, decisions, but what about the longer term affects of choices? Do cto’s underestimate operational costs?

A stack of…

These days the full stack of a internet or mobile startup involves a lot of varied components, from Chef, Puppet & Ansible, to Nginx, haproxy, redis, solr and some database like MySQL or Postgres on the relational end of the spectrum, or Mongodb, Hbase or Cassandra on the NoSQL side. What type of challenges does this pose to a team? I’m curious,
Do startups assemble at their own risk?

Most used tech

Leo Polovets ran some stats over the Angellist data of startups. He wanted to know Which tech do startups use most? and I summarized the results.

Death of ops?

These days with all the talk of automation, I’ve heard heard developers & even CTO’s argue of a diminishing need for backend administrators. Do startups still need techops?

Speed as a feature

Is Fred Wilson right to say speed is a feature? What does this mean for those migrating or already running in the cloud? How does scalability come into play?

Avoiding outages

Are many outages avoidable? Did Airbnb have to fail?

Performance Review

Reviewing architecture & site speed is a type of engagement that a lot of startups can benefit from. Here’s my Anatomy of a performance review.

Let things fail

Does it sometimes make sense to let things break a little? A tale of managed failure.

Young founders

I worked at one startup with a CTO just out of college. Although they were flush with cash & had real problems scaling, communication problems ultimately soured the engagement. Are you too young to be a boss?

80 million fix

Sometimes fixing serious performance bottlenecks can get a site back up on it’s feet. In this success story they went on to get acquired weeks after the fix. In tongue in cheek fashion I askWhere’s my 80 million dollars?

CTO’s should never do

There are times to get into the trenches. But what if it sacrifices leadership?What should a CTO never do?

Startups too cool for school

Joining YC but have no ideas? No problem. Is my startup too cool for your business school?

Instant business, just add water

Can a business be built in just a weekend? Is there a problem with startup bootcamps?

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Best of Scalable Startups Consulting Content

strawberries

I’ve been blogging very regularly for the past four years. In fact the blog itself has been around for over ten years! Time flies!

In that time I’ve posted a lot of evergreen content, some that google finds, and some that could be dusted off.

Join 28,000 others and follow Sean Hull on twitter @hullsean.

So here’s a peak into the archives, of some of the very best of scalable startups. Enjoy!

1. I blog about consulting

When you spend years doing consulting, professional services & freelance work, you learn all sorts of things. You stumble, you find yourself in unfamiliar territory, you learn. All that makes great fodder for blogging about business, and war stories. So here’s some of my best writing on the topic.

I had one experience where a prospect was still on the fence. That may be positive spin, as the title was
When prospects mislead. It turned out to be more a case of free consulting advice than anything else.

At networking events, I meet other freelancers, and consultants. There’s always debate about this topic, so I wrote
Why i ask clients for a deposit. There are reasons for both client and consultant, and I touch on the lessons i’ve learned.

It might seem strange that I’d write a post titled
Why I can’t raise the bar at every firm but there are prospects that aren’t the right fit for me. Here are some of the pre-qualifying questions on both sides of the fence.

Sad to say, but every client & consultant relationship isn’t a love story. So I wrote
When a client takes a swing at you about one such relationship and how I handled it.

I ask the question,
Does weekly billing increase time pressure? I think it does change the dynamic in some positive ways and I discuss those.

You’re ready to hire a consultant. What’s next? As it turns out, professional services is more a peer relationship with CEO’s, CTO’s & managers. So the typical, “send me your resume” and so forth may not be best. Here’s
5 conversational ways to evaluate consultants that provide an alternate approach to finding the best services.

One of the hardest things for engineers can be sales. Along the way to consulting success, I wrote
Can an engineer learn to love sales? Eventually it’s a skill that you have to improve at, if you want to stay in business for yourself.

Ever consulting engagement is not about your own triumphs. The conclusion isn’t always the wonderful things you’ve done for the firm. I wrote
When you have to take the fall after an engagement where it wasn’t a celebration at the end.

Sometimes in consulting, there’s what you’re hired to do on paper, and then what the real challenges are.
When you’re hired to solve a people problem addresses one such engagement, and how I handled it.

Believe it or not folks, sometimes there is a disconnect between management, and accounts payable. So I wrote
When clients don’t pay as a lesson & how I handled it.

Consulting is decidedly not the career path for everyone.
Why do people leave consulting
is my attempt to explain why some I’ve seen have left the business.

Everybody doesn’t love consultants. So
Do you heed John Greathouse – beware the consultant? That’s a question I attempted to answer.

Are you talking to Oracle or other technology sales teams about what solutions are right for your business?
Beware a wolf in sheeps clothing as it can be surprisingly dangerous field.

Another war story I wrote,
When apples & oranges bring down your business. Here a misunderstanding of semantics, means manager & dba make a severe misjudgement, and both pay the price.

After twenty years in the business, here are the
Top 3 questions I get from clients.

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Today’s startups: assemble at your own risk

devops divide

I was talking with Todd Hoff recently over at High Scalability about a trend I’ve seen of late.

ME: I really liked this post by Zoli Kahan from Clay.io.  AWS, cloudflare, docker, haproxy, mysql, mongo, memcache, ansible.  They use just about every technology being talked about these days.  

Todd: Yah, that’s why I asked to republish it. I thought it was a good updated sampler stack.

ME: That said I defy you to find a team that actually *KNOWS* all those technologies.  

Todd: Agreed. Systems are a lot of assembly these days, which doesn’t mean we know how to build the parts being assembled.

Join 28,000 others and follow Sean Hull on twitter @hullsean.

The article I was referring to was: How Clay.io Built their 10x Arch Using AWS, Docker, HAProx & Lots More

1. Dizzying array of technologies in use

I’ve been working with startups since the mid-nineties. In those days most application stacks consisted of a PHP application running on Apache, with Oracle on the backend. Both webserver & db ran on Sun Solaris. Hardware was reliable. Most attention was focused on fitting everything in memory, and monitoring the servers for swapping, and disk failure. Boy have those days changed.

I see dozens of startups each year, so I see a lot of very cutting edge environments. Here’s a peak at what I’m seeing these days:

Database: MySQL, Postgres & Oracle, to Mongodb, Cassandra & Couchbase

Caching: Memcache or Redis

Search: Solr

Webservers: Apache, Nginx, Lighttpd

Load balancers: haproxy, Zen

Languages: PHP, Python & Ruby

Publishing: Drupal, WordPress, Joomla

Continuous Integration: Jenkins

Metrics: Cacti, collectd, NewRelic

Monitoring: Nagios, Ganglia, Munin, OpenNMS

Automation: Ancible, Chef, Puppet, Docker & Vagrant

Logs: Logstash

DDOS & CDN: Cloudflare, Ultradns

Whew… That’s a long list!! And we’re not even considering the API’s that many applications are now building on.

Also: Are generalists better at scaling the web?

2. Shortcuts abound

Startups early on, don’t have enough working capital to hire a huge engineering team. So that means everyone is stretched. With a list of technologies that is ever growing, something’s gotta give.

These may cut corners by handing the web & technical operations work to a developer who has some skills. But I continue to ask… Does a four-letter word divide dev & ops?

Read: Which tech do startups use most?

3. More things to break & master

Ownership of a software stack, such as a database means mastery of…

o features in current versions
o bugs of current versions
o vulnerabilities of various versions
o troubleshooting
o best practices
o backup & reliability

For example a lot of shops where I dig into the database, I find low hanging fruit, such as misconfigured startup settings, table layout or index usage.

I see similar things when a networking expert pours over the haproxy configuration, or runs ping tests across the network. Most of these components are setup with fairly vanilla configurations, leaving loose ends and frayed threads.

Check out: Why I can’t raise the bar at every firm

4. Many startups carrying technical debt

I’ve seen a growing reliance on ORM’s which is worrying. Build your foundation on a crutch, and it gets very hard to eliminate down the line. Here are Ward Cunningham’s warnings on technical debt.

Related: Are SQL Databases Dead?

5. Long term support & viability

At one five year old firm, I was brought in to address scalability problems. I met with the team and was asked to provide a comprehensive review. The first thing I found was all the original engineers had long since left, so the code was new for everyone. As I dug my heels in, I found multiple versions of Apache along with Nginx on some other servers. Their stack was built on a patchwork of Python, Ruby & PHP. Then digging in further, we found a complicated web of dependencies for digital assets, mounted across servers & unmonitored.

Lack of standards is common in environments like these. Without an operational or architectural lead, developers are left to make decisions with what is directly in front of them. Though a decision of what language to use may appear simple at the outset, it carries long term consequences.

Will that language or technology be supported in five years? Will the community survive? Will your firm be able to hire people with that skill set? Will engineers still be excited about it?

See also: Is high availability overrated? Is five nines a myth?

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

5 things I learned from Gif Constable about Talking to Humans

talking to humans

I was just scanning through AVC.com, Fred Wilson’s popular blog, and hit on a post about great reading material. In it was mentioned a free e-book by Gif Constable called Talking to Humans.

Gif developed the Lean Launchpad curriculum, taught at Stanford, UC Berkeley, Columbia, UCSF, NYU & now hundreds of other universities worldwide.

Join 28,000 others and follow Sean Hull on twitter @hullsean.

One of the main takeaways from that work is the idea of “getting out of the building”. It means essentially that before you get to far along with your idea, building your product, and too heavily vested and invested in one direction, go do real research with real potential customers.

Right from the beginning test your ideas, and talk to customers. It’s not easy, but if done right will be very revealing.

The book can be had for free at Talking To Humans as an e-book, or send it straight to your kindle for $0.99 cents! With a forward by Steve Blank & Tom Fishburne’s funny cartoons and at only 98 pages, it’s well worth an hour or two of your time.

More details on Gif’s blog.

1. How to be a detective

Getting out of the building and talking to people is hard. It’s messy. It’s going into the real world where customers may not understand or care about your product.

But that’s also exactly why you want to talk to people. You’ll get real raw perspectives.

Also be wary of talking to friends & family. They may have biases, and want to tell you what you want to hear.

While interviewing, beware of speculation in your own ideas and what your interview subjects are telling you. Ask for stories instead and tease out real behavior.

Read this: When clients don’t pay

2. Fight cognitive biases with metrics

We all have biases. We think are customers are soccer moms, or 20-somethings who like lattes. By calculating metrics, we find out which market segments actually want our product and why. Keep calculating metrics, and make conclusions from real data.

At the same time beware the dynamic of mistaking statistics for facts. Remain skeptical!

Check out: 5 ways startups misstep on scalability

GIFF CONSTABLE 03-SD from The GovLab on Vimeo.

3. Map out your business

There are a few models mentioned in the book for mapping your business. Choose your favorite:

Alexander Osterwalder’s business model canvas

Ash Maurya’s lean canvas

Also: Are SQL Databases Dead?

4. How a startup can fail

Startups can fail in a myriad of different ways. This business is not for the weary or faint of heart! Here are some of the land mines in the road ahead.

o wrong customer or market
o wrong revenue model
o wrong cost structure
o wrong customer acquisition
o wrong product
o wrong team
o wrong timing

Related: Will Oracle kill MySQL?

5. How to screw up customer discovery

Interviewing real customers in the field requires a lot of balance. Here are a few things you should avoid:

o let speculation equal confirmation
o lead the witness to your conclusion
o talk over them
o selective hearing
o weigh one conversation heavily
o let fear of rejection win
o talking to anyone
o be unprepared for the interview
o try to learn everything at once
o only do customer dev first week
o ask customer to design the product

See also: How can Vagrant be used to deploy on Amazon EC2

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

5 Things I just learned from James Turnbull about Docker

docker containers

Join 28,000 others and follow Sean Hull on twitter @hullsean.

I just got my hands on a copy of James Turnbull’s new book The Docker Book. It’s an excellent introduction to Linux containers & the powerful things you can do with them. It’s 335 pages covering all the introductory topics to get you up and running and then more advanced topics like working with the docker API, building services & extending docker.

Here’s what I learned…

1. Containers aren’t new

The technology today we call containers in Unix is based on chroot mechanism which was introduced way back in the 80′s.

With traditional virtualization, we use a hypervisor layer, so we emulate hardware. The virtual machine running on top, can run anything, from Windows, to different flavors & versions of unix. It appears to be a completely separate piece of hardware.

With containers we move up to the operating system level, and we create isolation between users. These users all share the same parent operating system. This means it requires dramatically less overhead. That means speed!

Docker is an automation layer built on Lightweight Linux Containers or LXC. To applications it looks like they have their own machine, their own userspace, their own filesystem, their own network.

Also: Is Apple betting against big data?

2. No more VirtualBoxes

Are you tired of waiting for your VMs to spinup? Building dev & test environments becomes lightening fast with Docker. This accelerates software development, and makes a lot of things easier.

Also: When prospects mislead

3. Images, registries & containers

Images share some of the properties of images in hypervisor virtualization. However they are implemented with union file systems. While VirtualBox images take some time to boot, as the entire filesystem must be read & code executed anew, docker images are more like source code to the LXC subsystem.

Registries store your public and private images. The Docker Hub is one popular one. You can also host & deploy your own docker registry as your needs dictate.

Like VMs, containers can be started & stopped at will, albeit at lightening fast speed. They can also be deleted much as a VM can be.

Also: What can new york fashion week teach Chad Dickerson about Net Neutrality?

4. Lightning fast sandboxes

As we mentioned containers are fast. Did we mention really fast?

This can facilitate unit testing & continuous integration. A lot of shops are starting to use Jenkins for continuous integration, and fast testing is key to this process.

Also: Is automation killing old-school technical operations?

5. They work with Vagrant

Are you already using Vagrant to automate deployment of virtual environments. If so the transition is easy. Here Docker becomes your provisioner.

Mark Stratmann put together a great how to, Implementing a Vagrant / Docker Dev environment which we’d recommend you take a look at. You can also head over to the Vagrant docs themselves.

Also: Which tech do startups use most?

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Is Apple betting against big data?

apple_android

Join 28,000 others and follow Sean Hull on twitter @hullsean.

1. Pushing privacy

Apple has been pushing it’s privacy policy of late, in much of it’s marketing around the new iOS 8 and iPhone 6.

In particular Tim Cook takes direct aim at Google’s collection of user data:

“A few years ago, users of Internet services began to realize that when an online service is free, you’re not the customer. You’re the product. But at Apple, we believe a great customer experience shouldn’t come at the expense of your privacy.”

Read: Is Fred Wilson wrong about Apple?

2. Weak in cloud

It’s been quoted in various news that Apple is rather “weak” in the cloud. But digging a little deeper, this appears to be a deliberate strategy, a bet against using customer data in ways those end users may grow to resent.

Also: Is the Android ecosystem still broken?

3. The bet against open worked

Recall that Apple has had a fairly closed ecosystem since the beginning. This has kept their AppStore much cleaner, and free of malware. Reference the terrible problems that still plague the Android Play Store, from lack of policing.

Open also works as an iron fist on UI & UX, enforcing a consistency across apps and developers. This is a clear win for consumers and end users, even if they don’t understand the hows, whys and wherefores.

Related: No iPhones were harmed in the creation of this outage

4. Don’t monetize what you store in iCloud

Apple doesn’t directly monetize what is stored in iCloud. That means there’s no business imperative to make *use* of your data. They’re just storing it. This means they can also push encryption, a win for consumers, as it doesn’t bump heads with their business in any way.

Check this: What is mobile scalability & why is it important?

5. iAd has real privacy limits

Apple does have a platform called iAd. But even that has in-built limitations.

“iAd sticks to the same privacy policy that applies to every other Apple product. It doesn’t get data from Health and HomeKit, Maps, Siri, iMessage, your call history, or any iCloud service like Contacts or Mail, and you can always just opt out altogether.”

It’s unclear if all of these moves will help Apple in the marketplace. It remains to be seen if consumers will choose technology based on privacy concerns and fears.

Read this: How to increase newsletter signups with nifty iphone trick

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters