Morgan Tocker ([info]mtocker) wrote,
@ 2007-10-27 15:35:00
Previous Entry  Add to memories!  Tell a Friend  Next Entry
Researching Amazon EC2 for MySQL Usage
I'm a little curious to know how MySQL will run on what is essentially a virtualized environment. Normally a database server would be disk-bound, and it seems that for at least the interim, Amazon is not giving you much choice in the way of disk subsystems, since essentially shutting down a node means you loose all data. I have an EC2 account, so I plan to begin running sysbench tests shortly.

I was pointed to this link via one of the attendees of the MySQL User group in Montreal.

The concept is nice - and it's true that it's good to be able to fire up new instances so you can get your system up and running while you audit through what happened. I'm not sure I would use someone else's scripts though, since it doesn't look like they're either (a) open source or (b) remotely describe how they are written/work, so there's going to be zero room for customization.

What has been other peoples experiences?

(3 comments) - (Post a new comment)

MySQL on EC2
[info]http://getopenid.com/roobaron
2007-10-28 12:59 am UTC (link)
Hey mate,

Not have persistent storage on any one EC2 is just something you need to work around. Given Google and Amazon themselves design for failure, do the same.

Please post your sysbench results, I have done some IO benchmarks using bonnie and IOZone. The more the better.

Most articles I have seen use MySQL replication with backups of binlogs and databases pushed to S3 for long term backups.

Have Fun
Paul

(Reply to this)

EC2 is a great platform for mysql!
(Anonymous)
2007-10-29 05:38 pm UTC (link)
Morgan, thanks for your interest in RightScale and EC2. We've been working hard to make EC2 a fabulous platform for running mysql and while we're happy how far we've gotten, we know we still have a ways to go. Here are some things to consider with respect to what you wrote:

* EC2 now offers large and x-large instances, the latter come with 4 cores, 4x 420GB disks, and 15GB memory so that provides some real oomph for a database

* the flexibility of launching new instances more than makes-up for the fact that disk storage is not persistent

* the scripts in RightScale are all open, in that you can inspect and customize to your hearts content. if you look at the public rails all-in-one server template you can see what I mean (the mysql ones are not "open source" as in "free")

We currently have a number of customers using our mysql master slave set-up with one-click slave-to-master upgrades, one-click launch new slave, etc. It works nicely but we also know there is a lot more we can (and will) do.

Regarding the potential for data loss, I often compare the situation to the SAN storage arrays I used to work with. They too failed, not because they broke, but because the complexity of the overall system caused interesting and unexpected failure modes. In those cases we had to switch to a backup system that was a replication slave across the continent. Using Oracle physical replication it received log files as they were "cut", approx once every 5-10 minutes, so we'd loose up to that much in data (only had to do this very rarely). When we'd get the main system back up of course the data was still there on the SAN, but by that time (typically hours later) the operational system had advanced so much that someone would have had to pay us real big bucks to extract the "lost" transactions and somehow "insert" them into the live system. Of course the risk & cost tradeoff changes with the business, so your mileage may vary.

What really has me excited are the new possibilities. Like firing up a new slave to run some consistency checks or reporting jobs, and shutting it back down when done. Or cloning the database for a staging system, or perhaps a quick scalability test. Want to roll back to yesterday's state? No problem, fire-up a fresh instance and load the backup from S3. The sky's the limit...

Thorsten - CTO, RightScale

(Reply to this) (Thread)
Re: EC2 is a great platform for mysql!
[info]mtocker
2007-10-29 05:54 pm UTC (link)
Thorsten wrote:
> * EC2 now offers large and x-large instances, the latter come with 4 cores, 4x 420GB disks, and 15GB memory so > that provides some real oomph for a database

I wasn't sure if they were actually disks or just partitioned that way? It should be interesting to see if amazon offers add-ons or instance types with guaranteed I/O or similar. I can see a real demand.

Your notes in your blog about master-slave replication with a single thread causing possible delays were quite accurate - I was impressed by the research you have invested, and I would be interested to try it for myself ;)

- Morgan

(Reply to this) (Parent)

(3 comments) - (Post a new comment)

Create an Account
Forgot your login or password?
Login w/ OpenID
English • Español • Deutsch • Русский…