Morgan Tocker ([info]mtocker) wrote,
@ 2009-08-07 14:21:00
Previous Entry  Add to memories!  Tell a Friend  Next Entry
Entry tags:itsnotaforkitsaspoon, mysql

Understanding the MySQL forks

I put together the following diagram to explain what the origin of the current MySQL forks and deltas looks like:

But in the distributed revision control world we live in, it's never really that simple. Here are some other notes:

  • XtraDB is an InnoDB fork and "mostly" only a storage engine. It has a future in MariaDB and Drizzle, but may not make it into MySQL (about to be owned by Oracle, who also owns InnoDB).
  • OurDelta collects more patches than just the Percona patches, and enables the PBXT storage engine. There's also a repository of OurDelta for 5.1 - binaries just aren't out yet.
  • Like XtraDB, PBXT has a future in Drizzle, MariaDB and *maybe* also MySQL (Oracle may be more friendly towards PBXT than XtraDB, but Oracle is not known to be friendly so who knows!). I left PBXT absent from the diagram for not having much of a lineage with previous MySQL releases (flames welcome if you disagree).
  • XtraDB draws a lot from the Google patches for MySQL/InnoDB (not present on the diagram either).
  • Drizzle is now incompatible with everything else in MySQL-land terms (replication, partitioning, etc), and they're happy to be. In storage engine terms, they're still compatible though, which leads me to describe Drizzle as a completely new "userland", with storage engine options still remaining very similar.
  • MariaDB is both a new storage engine (Maria) and a userland "delta" of MySQL. I call it a delta since it is trying to keep binary-format compatibility with MySQL wherever it can. This means that it makes a good retrofit, but it limits what they can do without changing things like the .frm format, etc.
  • The InnoDB plugin has a weird history. Oracle announced it as an optional replacement for the 5.1 InnoDB but not much has become of it since introduction. I would have expected 5.4 to be based off the plugin, but it's not (at least at this point).
  • MySQL is the real loser in the direction the patches are moving. While all other forks are free to share amongst themselves (where compatible) and take from MySQL, MySQL will only accept a patch if the author signs a Contributor's License Agreement. They need to do this - otherwise they can't sell OEM copies, which still make a large chunk of sales. Until recently, MySQL didn't accept any InnoDB patches unless they came downstream from Oracle, and Oracle keeps InnoDB development as a closely guarded secret - which makes influencing it very difficult.

Comments welcome.




(8 comments) - (Post a new comment)

InnoDB plugin
(Anonymous)
2009-08-07 08:06 pm UTC (link)
Can InnoDB plugin be used as a plugin for 5.4?

When 5.4 comes out, will there be XtraDB based on that?

(Reply to this) (Thread)

Re: InnoDB plugin
[info]mtocker
2009-08-07 08:49 pm UTC (link)
"Can InnoDB plugin be used as a plugin for 5.4?"

In theory, yes. According to the MySQL manual it is planned - " a future version of MySQL 5.4, the InnoDB Plugin will replace the (built-in) version of InnoDB currently distributed with MySQL."

http://dev.mysql.com/doc/refman/5.4/en/innodb.html

"When 5.4 comes out, will there be XtraDB based on that?"

5.4 and XtraDB have a strange relationship, since they are both (in many ways) the downstream product of the Google patches. So, to put it another way "XtraDB already has 5.4 in it." There were a few cases where the Sun 5.4 implementation showed better performance than XtraDB, but Percona started backporting straight away:
http://www.mysqlperformanceblog.com/2009/05/12/read_ahead-as-steroi/

There are some diagnostic tools like slow query log enhancements and INDEX_STATISTICS that won't make it into 5.4 - which means 'no deal' from my perspective.

When customers demand it though, I'd expect Percona will upgrade to 5.4 since it will contain other userland fixes (like RESIGNAL and subquery optimizations).

(Disclosure: I work for Percona, but this blog reflects my personal opinions, not their's).

(Reply to this) (Parent)

libdrizzle is extra compatible..
(Anonymous)
2009-08-07 09:12 pm UTC (link)
Great write-up Morgan.

I just wanted to follow-up to "Drizzle is now incompatible with everything else in MySQL-land terms (replication, partitioning, etc), and they're happy to be." That statement is pretty darned true from a systems level, but I'd like to point out for folks that our new client library, libdrizzle, speaks MySQL as well. This means that things written with it, or its wrappers (such as DBD::drizzle) can happily talk to both MySQL and to Drizzle.

Just didn't want folks to think we don't care... :)

(Reply to this) (Thread)

Re: libdrizzle is extra compatible..
[info]mtocker
2009-08-07 10:53 pm UTC (link)
Thanks... for some reason thinking about drivers and protocols escaped me when I wrote this!

(Reply to this) (Parent)


[info]krow
2009-08-13 04:04 pm UTC (link)
Hi!

Cool Article. A couple of points:
1) We find that most MySQL applications "just work" with Drizzle. We are not 100% compatible, but if you are just using ANSI SQL there shouldn't be an issue.

2) BlitzDB is a fork from Drizzle (and we have distributions which are are for PBXT as well).

3) I would put up "distributions" and "forks". Many of the above are just repackaging (though I believe long term they will all be forks).

4) I would list "InfoBright" as a fork of MySQL as well. I am not sure how compatible they are, but they deserve some mention because of their own adoption. I believe it was forked from 5.0.

5) MySQL Carrier Grade is a fork as well (I think from 5.1... they are now on 7?).


This would make for a great Wikipedia article :)

Cheers,
-Brian

Edited at 2009-08-13 04:06 pm UTC

(Reply to this) (Thread)

Thanks!
[info]mtocker
2009-08-13 06:18 pm UTC (link)
You gave quite a lot of feedback there, so I just wanted to comment on it:

"1) We find that most MySQL applications "just work" with Drizzle. We are not 100% compatible, but if you are just using ANSI SQL there shouldn't be an issue."

Point taken. From my perspective I drew the picture to explain to users that it isn't forking in every direction with a lot of lost development effort - I completely omitted user experience.

"2) BlitzDB is a fork from Drizzle (and we have distributions which are are for PBXT as well)."

Great!

"3) I would put up "distributions" and "forks". Many of the above are just repackaging (though I believe long term they will all be forks)."

How much has to change to be a fork is always a sensitive issue. I'm not sure if you meant to include Linux distributions as "distributions", but that's how IceWeasel started, etc.

I think distributions is a better word than Deltas.

"4) I would list "InfoBright" as a fork of MySQL as well. I am not sure how compatible they are, but they deserve some mention because of their own adoption. I believe it was forked from 5.0."

Acknowledged. Kickfire probably counts here too - I know a lot of the data warehouse SEs all had to change quite a few things.

"5) MySQL Carrier Grade is a fork as well (I think from 5.1... they are now on 7?)."

I think MySQL would like to think of it as a Delta, but sure ;)

"This would make for a great Wikipedia article :)"

I had no intention of it - but maybe that's not a bad idea. If anyone reading this blog wants to build off my work, then I grant permission to do so.

(Reply to this) (Parent)(Thread)

Re: Thanks!
[info]krow
2009-08-13 06:38 pm UTC (link)
The difference between distribution and fork I believe is intention. For Drizzle, we are a fork. We keep some compatibility, but far from all. MariaDB/Percona are shooting for a high degree of compatibility and I don't believe they would consider themselves to be a fork (and they shy away from incompatible changes).

The word "delta" I think is just an attempt to soften the term (I think...).

The reason I would consider the telecom version a fork is that it has several features which are not found in the main branch, and from external observation it does not look like these will go back in. This is speculation though on my part, I have no direct knowledge of what they do/etc.

One other thought about your tree, I would put in the ancestors to MySQL, aka Unireg and MiniSQL. That piece of history that tends to be forgotten (though how much code/design was taken from MiniSQL... I doubt we will ever know).

(Reply to this) (Parent)


[info]dledwards
2009-08-17 04:16 pm UTC (link)
"Let’s go back to Morgan Tocker for a moment, and his post helping everyone in understanding the MySQL forks, complete with a handy family tree. Thank you, Morgan. [...]"

Log Buffer #158

(Reply to this)


(8 comments) - (Post a new comment)

Create an Account
Forgot your login or password?
Login w/ OpenID
English • Español • Deutsch • Русский…