Posts Tagged ‘MySQL’

FOSDEM 2016 – See you in Brussels

Over the weekend I read in the FT (paywall): Is Brussels safe? Ring a local resident to find out. I’m sure it will be fine, and you will want to be there for FOSDEM, happening 30-31 January 2016. 

There is the excellent one day track, that is the MySQL & Friends Devroom (site). Talks hail from Oracle, MariaDB Corporation, Percona and more. We don’t have a booth this year, but we do have amazingly good content on Saturday. I’m happy to have been part of the committee that chose the talks, but you know that this is a labour of love put on by Frédéric Descamps, Liz van Dijk, Dimitri Vanoverbeke, and Kenny Gryp. I’m sure the party will be awesome.

But that is not all! In the distributions devroom, you can see me give a talk at 11:00-11:20 titled Distributions from the view of a package. This is an important topic, because you start seeing MariaDB Server becoming the default in many distributions with the last holdout being Debian. But there is a lot of discussion, especially from the security standpoint there now, about MySQL overall. But that’s not the focus of my talk — I’m going to talk to you about how we, as upstream, have had to deal with distributions, changing requirements, etc. overall. I’ve done this since the MySQL days, so have quite a bit of experience dealing with it. 

If you are making software and want to be included and supported across all distributions, I highly recommend you coming to my talk. If you happen to decide to live in an ecosystem where there are forks, I also promise to make it useful for you.

And on Sunday, you will want to go visit the RocksDB Storage Engine for MySQL talk by none other than Yoshinori Matsunobu of Facebook. This will be at the main track and I highly recommend you visit it — I’m sure Sergei Petrunia will also make an appearance as he spends a lot of time on this too.

All in, I’m extremely excited to be at FOSDEM 2016. And you don’t need to ring a local resident to find out if its going to be safe/fun — come for the learning, stay for the beer ;-)

SCALE14x – lots of MySQL content there

One of my favourite events run by a grassroots organisation is SCALE, and they are now doing their 14th edition, SCALE14x. If you’re into opensource software as well as all things open, this is the place to be from January 21-24 2016. It is at a new location in Pasadena (so not quite next to LAX as it was previously), but this is due to growth — so kudos to the team.

From MariaDB Corporation you get to see Max Mether (Scaling MySQL & MariaDB – I’m extremely interested in seeing what he has to say and will likely blog the session) and me (The MySQL Server Ecosystem in 2016).

One thing is for sure is that the topic I plan to present on will surely come under contention since I also represent a server maker — however I believe I will be extremely objective and will put up blog posts before/after the event as well as slides, because it is clear that MySQL is now going to be 21 years old and the ecosystem has grown tremendously. Let me reiterate my main thesis: MySQL server development has been at its most vibrant since the Oracle acquisition — the ecosystem is flourishing, and Oracle is doing a great job with MySQL, Percona with Percona Server, MariaDB Corporation/MariaDB Foundation with MariaDB Server, and lets not forget the wonderful work from the WebScaleSQL Consortium, Facebook’s MySQL tree and even Alibaba’s tree (the Twitter tree seems to be sadly not really maintained much these days, but there was innovation coming out of it in the past).

There are also going to be many other great talks at the MySQL track on Friday, from Peter Zaitsev, Dave Stokes (I’m excited about the JSON support in MySQL 5.7), Ovais Tariq/Aleksandr Kuzminsky on indexes, and Janis Griffin on query tuning. There’s also an excellent PostgreSQL track and I think one of the highlights should also be the keynote from Mark Shuttleworth at UbuCon on Thursday.

See you at SCALE14x? Oh, before I forget, MariaDB Corporation also has a booth, so you will get to see Rod Allen manning it and I’m sure there will be giveaways of some sort. 

If you have any feedback about the MySQL Server ecosystem and its developments, please feel free to leave a comment here or send an email to me. Thanks!

Learnings from Swift becoming opensource

Swift is now opensource, and it’s interesting to see Craig Federighi talk about it. This is Apple doing right, considering FaceTime is long overdue to being an open standard. People are nitpicking on Apple’s Open Source tagline, but really, this is akin to nitpicking on Mark Zuckerberg donating 99% of his Facebook stock to his new limited liability corporation charity (key: don’t look a gift horse in the mouth).

Apple has chosen to put Swift on Github, and they’ve ensured that it wasn’t just an initial commit, but you’re seeing a lot of history. And it’s the right choice clearly, for engagement — 1,275 watching, 18,884 stars, 2,139 forks, 51 pull requests currently, but most interestingly a lot of accepted code. Even simple things along the lines of “fixing typos” (see commits, eg. d029f7e5ae84cf8f6c12907f9ed0ac0a694881aa, e8b06575d26a684f415af95143ec576a6aa5168d, etc.). 

Swift has open source documentation — like all good open source projects are supposed to have. They use Sphinx and its in the source tree. This is something I’d wish to see from MySQL (docs copyright Oracle, online, but you can take it offline too via PDF) or MariaDB (friendly licensed Knowledge Base), but so far only Percona Server has gotten this right.

What else did Swift do right? Focus on user contributions — the Contributing page is a breath of fresh air. And don’t forget the code of conduct for participating in the project.

But besides just the documentation and contribution pages, I learned something new from one commit in particular — the existence of nproc, part of coreutils. I immediately hopped onto IRC to chat with Nirbhay (our resident MariaDB Galera Cluster expert), because in scripts/wsrep_sst_xtrabackup.sh, we do this via a get_proc() function. We should be focusing on modernising/standardising our codebase, shouldn’t we?

There is a lot we at MariaDB Corporation and the MariaDB Foundation can learn from Swift being opensource and how Apple deals with the community at large. Here’s hoping we get the best practices from it and implement it in due time.

Voting for talks at the Percona Live Data Performance Conference 2016

So this year the Percona Live conference has a new name — it is the “Data Performance Conference” (presumably for a much broader appeal and the fact that Percona is now in the MongoDB world as well). And the next new thing to note? You have to go through a process of “community voting”, i.e. the speaker has to promote their talks before via their own channels to see how many votes they can get (we tried this before at the MySQL & Friends Devroom at FOSDEM; in this case, please remember you also need to create a new account and actually vote while logged in).

I hope you vote for Sergei, Monty and my proposals!

  1. Using and Managing MariaDB – a tutorial, which has been referred to as The Complete MariaDB Server tutorial, I thought I will change the name up a little, in addition to the content. The most recent version of this tutorial was given at the Percona Live Conference in Santa Clara in 2015 (slides). Since then we’ve released MariaDB Server 10.1, and there’s much more new things to talk about!
  2. MariaDB 10.1 – What’s New? – a talk that would have Michael “Monty” Widenius (creator of MySQL and MariaDB) and me give it together. I’ve described this as a dance, and the last time we did this was at Percona Live Amsterdam. The content will of course be new, and I am creating the slide deck this time around.
  3. Databases in the Hosted Cloud – this is a pet talk. It costs some money to make, and if accepted I plan to also showcase who has better performing hosted databases. I did this at Percona Live Amsterdam 2015 (slides), but since then we’ve seen Amazon offering MariaDB Server as part of RDS, HPCloud being sunset, and also Rackspace upping their offering with High Availability Databases. More research to be done from now till then!
  4. Best Practices for MySQL High Availability – this would be another tutorial, and at Percona Live Amsterdam 2015 it had the highest registered attendance (Kortney told me the day before and I removed all practicals, since 100+ people with practicals is impossible for one person to manage – slides). I think with the changes in NDBCLUSTER (recently announced at OpenWorld), the addition of tools in the MHA world (mha-helper), this should have a lot of new information (and more importantly a lot of new things to play with).
  5. Choosing a MySQL HA solution today – a talk based on the above tutorial, cut short, to ensure people whom are not at tutorial day, will have solutions to think about and take home for implementation in the future.
  6. MariaDB/MySQL security essentials – a talk which focuses on improvements in MariaDB Server 10.1, and MySQL 5.6/5.7, including encryption at rest, easier SSL setup for replication topologies, and even external authentication plugins (eg. Kerberos is almost ready – see MDEV-4691).
  7. The MySQL Server Ecosystem in 2016 – a talk about MySQL and the forks around it, including the private trees that exist (some like the Twitter tree haven’t been updated in a while, but clearly have made inroads in giving us new features). Learn what to use, and what is the best one for your use case. 
  8. MariaDB Connectors: Fast and Smart with the new protocol optimizations – a talk from Sergei Golubchik, about new protocol optimisations in MariaDB Server as well as how we optimise this from the connectors as well.
  9. MariaDB 10.1 Security: Validation, Authentication, Encryption – a talk from Sergei Golubchik focusing on MariaDB 10.1 security improvements; he’s got some amazing slides on encryption that I saw at Percona Live Amsterdam, and you can see a five-minute lightning version from the meetup.

Here’s to happy voting and I hope to give at least some of these talks (if not all!).

Ubuntu Online Summit: MySQL & Variants in 16.04

I personally have always enjoyed the Ubuntu Developer Summits (UDS), but nowadays they have been converted to the Ubuntu Online Summits (UOS). Attending them is not always convenient (timezone issues, might be travelling, etc.) so I watched the recorded video of a session I was interested in: MySQL & Variants in 16.04.

My key takeaways

  1. Ubuntu 16.04 Xenial Xerus is an LTS release.
  2. The term “cross-grade” is used a lot (it is not about downgrading/upgrading, but being able to use MySQL or MariaDB or Percona Server interchangeably)
  3. It would be nice to see MySQL 5.7 in this release (for Xenial as well as Debian Stretch). From Oracle there is a new packager taking over the task (Lars)
  4. MySQL 5.5 is still the default in Debian, and there needs to be upgrades tested between 5.5 to 5.7 (it looks like the ideal jump is that Ubuntu will not be seeing MySQL 5.6)
  5. Percona Server 5.7 is 60-90 days out; xtrabackup has had some new modifications and deserves an upgrade
  6. Boost is a new requirement for MySQL 5.7 & Percona Server 5.7; some old TokuDB problems in the builds are likely already fixed in MariaDB Server so this can be inherited
  7. MariaDB is waiting to iron out the bugs in 10.0, and may stick to that

My “raw” transcribed notes

  • Attendees:

    • Jon Grimm (Engineering Director for Ubuntu)
    • Robie Basak (Ubuntu)
    • Otto Kekäläinen (MariaDB Foundation)
    • Lars Tangvald, Norvald H. Ryeng (Oracle)
    • George Ormond Lorch III (Percona)
  • Robie: Waiting in Debian for a transition slot from MySQL 5.5 to MySQL 5.6. There’s some discussion with bugs, re: Akonadi, need to also resolve ABI issues with MySQL 5.6. Not really discussed MySQL 5.7 yet.

  • Norvald: 5.7, changes to installation. Client library ABI cleaned up. There may be some clients breaking because of that. No more exported symbols. See: The Client Library, Part 1: The API, the Whole API and Nothing but the API & The Client Library, Part 2: The Version Number
  • mysql_install_db is now replaced by --initialize in the server, so have to rewrite the post-install scripts. Might also have some AppArmour changes. Spoke to people @ DebConf (so best place is to put AppArmour profiles upstream (i.e. in mysql) and Debian and other distros will get it from there). AppArmour profile is in the MySQL source package now. Probably can get away with doing everything as cmake variables.
  • MySQL 5.7 has disabled the old password hashing algorithm, so if people haven’t upgraded they might have problems; so a manual intervention to fix their accounts.
  • Going from MySQL 5.7 to MySQL 5.6? It is done by dump and restore. There is no testing automated downgrades. Are there disk format changes? Norvald is not aware of any. If you use virtual columns in 5.7, you can’t downgrade easily to 5.6.
  • Robie would prefer to not release 5.6 and 5.7 concurrently. During Trusty, there was some level of user confusion. Debian – release team would prefer to see one transfer than two, so is it better to just do a single transition to 5.7?
  • Norvald says there hasn’t been testing from 5.5 -> 5.7. They only support upgrades from 5.5 -> 5.6 -> 5.7. For Ubuntu the choice can be to have 5.6 and then later do 5.7, but Jessie only just released with 5.5, so Stretch with 5.6 might not be a great idea (so users migrating from Jessie to Stretch will go from 5.5 to 5.7). Could also have 5.7 depend on a stripped 5.6 binary (like the embedded server; this is for localhost and the security team shouldn’t be too annoyed) for people to do an upgrade. Norvald says this has not been tried and there needs to be a migration path tested from 5.5 -> 5.7.
  • Conclusion: 5.7 in Stretch. Xenial is an LTS release, and 5.7 should be targeted for that.
  • If the maintainer script fails (postinstall script fails – don’t leave apt in a weird state). If it fails then upgrades, leave a debconf critical notice to say that the service is disabled and then fix it manually. Otto says that leaving /etc in a broken state is terrible, so we should avoid it.
  • Do we (Oracle) have the resources for 5.7 packaging and how soon can it be done in time for Xenial? There were patches from Lars in the git tree, but there haven’t been more recently. Lars will take over the 5.7 transition so if there is a list of work items, this will be settled (Lars will take over from Norvald).
  • There will be a separate session with Norvald/Lars/Robie outside of UOS about 5.7. Defer the Boost conversation after the session as well.
  • George: Percona is mainly looking out towards the 5.7 work and what kind of resources that will be put to that. There are new folk @ Percona to help with this. Percona inherits so much from the upstream codebase, it just works for Percona Server. There is Percona XtraDB Cluster and Percona xtrabackup, and xtrabackup has moved on quite a bit since the last upload (since last November 2014). So might be good idea to look at a refresh. There has also been a lot of work done on Percona XtraDB Cluster and there are some developments with Codership, so they are unsure if they will have their own Percona XtraDB Cluster 5.7 by the time Ubuntu is supposed to ship. When Percona is ready for something, just give Robie a shout to ensure that things happen. 60-90 days before a Percona Server 5.7 release. Just be aware of feature freeze for Xenial.
  • Norvald mentions that Percona Server 5.7 will also depend on Boost and there needs to be a decision on this. George mentions that TokuDB is now part of Percona Server, and it has some of its own requirements as well. Do we include TokuDB? It has requirements like it will only run on 64-bit platforms. Things to figure out going forward? MariaDB has been carrying TokuDB last November, but Robie remembers disabling it in Ubuntu. George says there were some licensing issues back then but they seem to be taken care of.
  • Otto says the builds for TokuDB was failing. It has a dependency on jemalloc, and that might have been the reason there were failures (says George). There may be something else where it doesn’t build on Ubuntu builders. But Otto says that there was a commit where this got fixed about last month. George will follow on, just to absorb it, since the legwork is already complete.
  • Otto: Trusty has 5.5, and Jessie and all other Ubuntu releases have 10.0, and 10.1 was released last month and I’m not quite pushing it to Debian quite yet. Fix 10.0 build fixes, upstream them, then only focus on 10.1. Blocking? (last summer) 5.6 is not in testing, so could not depend on it/changes done in 5.6 mysql-common. Here’s hoping that mysql-common going forward will be generated separately.
  • Robie will take an action to resolve the delta (probably just drop it). To sync MariaDB 10.0 to Xenial.
  • Discussion on /var/lib/mysql/*.flag thing on the list — conclusion at: mailing list — goal: within a single Ubuntu release, people can “cross-grade” between MySQL variants. The goal is to support all 3, and users want to try them, and thats when the bug reports come. Robie’s goal: move to a per-variant data directory. Otto says that once directory names change, 3rd party tools might have breakage. So a working prototype. Migration path is difficult. Maybe the best is to turn /var/lib/mysql into a symlink and store the data elsewhere. PostgreSQL does per version directories today; so studying that is going to happen.

Rackspace Cloud High Availability Databases for MariaDB, MySQL, Percona Server

Continuing on with the cloud theme, I think its worth noting that since mid-2014, Rackspace has offered MariaDB (as well as MySQL and Percona Server) in the cloud, as part of their Cloud Databases offering. It’s powered by OpenStack.

Now there is an additional “High Availability instance” being offered — this gives you up to two replicas per database instance, you have the ability to load balance reads across all replicas (pretty standard), but the cool thing to try out: failover is automatic. It’s not just that if the master fails, you get a new slave being the master; you get a replacement node being added, so as to ensure that your load keeps up with the traffic. These instances don’t cost much more (the higher the memory size, the cheaper it gets — 1.5% extra for something production ready, down to 7.7% more expensive for something to kick around the tires with)

There is also scheduled backups (daily incremental, weekly full) and you can specify the backup window.

Previously on Rackspace, you not only had to spin up a cloud database, but also a compute instance to access your databases. Now, they’re allowing you to get a public IP address, via an ACL.

In another post, I’ll go thru these services with the intention to update my deck and also share the results here. Have you tried or do you use Rackspace Cloud Databases?


i