Join 34,000 others and follow Sean Hull on twitter @hullsean.
A reader pointed out that some No-SQL databases do support joins. Huh? My face contorts in total confusion. How? Why?
For years SQL was misunderstood & unloved
Relational databases have been around for decades. 43 years according to the StackExchange article. That’s a lot of years. I’ve spent a few years as a dba, aka database administrator. The role can be distilled down as a herder of sorts. Keep all the data bits in the right boxes with the right labels. A digital librarian, that makes sure the books don’t get lost.
Of course patrons don’t always put books back where they should, and strict rules get put in place to avoid losing that one volume of shakespear in miles of shelves.
In the fast moving world of web + mobile, product is king, and agile rules the day. And anything that can make us more agile also wins. SQL, much maligned & misunderstood, was not one of those things.
NoSQL burst on the scene with much fanfare
With all that pressure, it was no wonder engineers thought, there must be a better way. Then along comes the No SQL database. I mean just the name speaks volumes about the design goal.
We’ll sacrifice anything, just please don’t make me write SQL!
1. Never have to deal with pesky SQL that we don’t understand!
2. Interact with the database like any other data structure in our code!
3. Be schemaless! Crotchety Database Administrators be damned!
4. Be distributed. Be everywhere consistent! Be indistinguishable from magic!
5. Always be fast.
In that rush into the abyss, we lost track of durability. And down the rabbithole we went!
Relational databases tried to be key-value
Then I started hearing about crazy things, like MySQL providing a memcache plugin, so you could use it albeit lightening fast, as a key-value store. You could sidestep that pesky SQL engine, and get right down to the bare metal. But why? Memcache & Redis were already doing that & purpose built. Why indeed?
I started to argue maybe we shouldn’t be muddying the waters. I mean stick with what you know!
War was won, success declared
Around this I think was when Mongodb was declaring the war won. We had finally left SQL databases in the dustheap of history. It may or may not have inspired this popular youtube skit…
Meanwhile hadoop is losing ground. Bigquery & Redshift both speak SQL
But then something funny started to happen. It seemed there was a backlash against Mongodb. A lot of customers were losing data. (Yep that’s what durability means guys…) And the hype started reversing. Even the mighty hadoop has been losing popularity of late. How long does it take to write an EMR job versus an SQL query. Let’s be honest?
I asked myself, Is Hadoop losing ground to SQL warehouses like Redshift & Bigquery?. I wonder.
NoSQL databases are looking for JOINs?
As certain relational databases try to become No SQL databases, other No SQL databases are trying to add more complex SQL, because well somehow their product is missing something.
Engineering truth versus fashion
43 years is a lot of years. And when we drop all the fashion trends in tech, and the new database du jour, what do we find?
There is room for No SQL databases. Yep. And the do certain things, and solve certain types of problems well. But their not general workhorses, nor can they slice and dice your data however you like. And when you get to that point in your project, you’re going to want to ask interesting questions of your data.
And surely that’s where SQL excels. It ain’t going anywhere, folks!