Comments on: WhySQL?

By: Troy Stephen Augustine slc

Troy Stephen Augustine slc — Fri, 12 Apr 2013 05:11:17 +0000

An interesting discussion is definitely worth comment. I do believe that you ought to write more about this topic, it may not be a taboo subject but typically people do not speak about these issues. To the next! All the best!!

By: Chris Traganos

Chris Traganos — Tue, 05 Feb 2013 17:11:13 +0000

The photo sharing app you mention is a part of the company that owns some of the largest data centers in the world. They manage many of their own cloud services in house.

Our in house ops team manages our cloud service and we have full control of the setup.

Hope this helps.

By: Cosmo

Cosmo — Wed, 23 Jan 2013 06:11:05 +0000

I still want to know Why Evernote running your own hardware instead of hosting Evernote in [cloud service provider Y].
As we know, Instagram cloud keep they server very well when they have only ten peoples by use Amazon cloud service.

By: Joe

Joe — Tue, 06 Mar 2012 21:44:55 +0000

Thanks very much for sharing Dave! It really helps to have access to real examples such as Evernote.

By: Dave Engberg

Dave Engberg — Tue, 06 Mar 2012 04:32:11 +0000

Yes, we have one small master “account” database that we use for simple things like username, password, etc.
http://blog.evernote.com/tech/2011/05/17/architectural-digest/

By: Joe

Joe — Tue, 06 Mar 2012 02:46:40 +0000

Do you have a master database for authentication, and for looking up the shard for a given user? Or some other mechanism?

By: Rob

Rob — Tue, 28 Feb 2012 19:41:33 +0000

Dave-

Great points. The consistency is key and I think a major reason why NoSQL has been limited to big-scale, web-facing problems to a certain extent.

I wrote up some more detailed thoughts about it here:
http://www.cambridgesemantics.com/blog/-/blogs/whysql-evernote%E2%80%99s-boring-old-reliable-architectu-1

cheers!
-Rob

By: Dave Engberg

Dave Engberg — Mon, 27 Feb 2012 22:06:08 +0000

For reporting, we track relevant activity (“created a note”, “logged in”) using one flat “event log” file on each server each day. These files are pushed into a reporting server that does loads them into a single database where we can perform aggregate queries to make pretty graphs and charts for Phil.

On rare occasions, we’ll want to do an ad hoc aggregate query that touches all of the actual shard databases to find out some information that wasn’t in the event logs. E.g. something like “how many notes contain images that are more than 5MB” or something like that. For that, we have a simple script that runs the query in parallel on the different shards and then just displays the individual and aggregate results.

But neither of those are used for any “real time” operations for end users … they’re back-end activities for our own internal metrics which don’t need to finish quickly for any end-user UI operation.

By: Brian Tarbox

Brian Tarbox — Mon, 27 Feb 2012 17:22:01 +0000

How do you do aggregated queries across all shards (e.g. for reporting?) In other words how do you handle the cases where the users are not actually isolated?

By: Jon Clemens

Jon Clemens — Mon, 27 Feb 2012 11:44:10 +0000

Very interesting post. I like the balanced, pragmatic point of view. I grow so tired of techie diatribes on the latest fad!