Posts Tagged ‘Ubuntu’

Ubuntu Online Summit: MySQL & Variants in 16.04

I personally have always enjoyed the Ubuntu Developer Summits (UDS), but nowadays they have been converted to the Ubuntu Online Summits (UOS). Attending them is not always convenient (timezone issues, might be travelling, etc.) so I watched the recorded video of a session I was interested in: MySQL & Variants in 16.04.

My key takeaways

  1. Ubuntu 16.04 Xenial Xerus is an LTS release.
  2. The term “cross-grade” is used a lot (it is not about downgrading/upgrading, but being able to use MySQL or MariaDB or Percona Server interchangeably)
  3. It would be nice to see MySQL 5.7 in this release (for Xenial as well as Debian Stretch). From Oracle there is a new packager taking over the task (Lars)
  4. MySQL 5.5 is still the default in Debian, and there needs to be upgrades tested between 5.5 to 5.7 (it looks like the ideal jump is that Ubuntu will not be seeing MySQL 5.6)
  5. Percona Server 5.7 is 60-90 days out; xtrabackup has had some new modifications and deserves an upgrade
  6. Boost is a new requirement for MySQL 5.7 & Percona Server 5.7; some old TokuDB problems in the builds are likely already fixed in MariaDB Server so this can be inherited
  7. MariaDB is waiting to iron out the bugs in 10.0, and may stick to that

My “raw” transcribed notes

  • Attendees:

    • Jon Grimm (Engineering Director for Ubuntu)
    • Robie Basak (Ubuntu)
    • Otto Kekäläinen (MariaDB Foundation)
    • Lars Tangvald, Norvald H. Ryeng (Oracle)
    • George Ormond Lorch III (Percona)
  • Robie: Waiting in Debian for a transition slot from MySQL 5.5 to MySQL 5.6. There’s some discussion with bugs, re: Akonadi, need to also resolve ABI issues with MySQL 5.6. Not really discussed MySQL 5.7 yet.

  • Norvald: 5.7, changes to installation. Client library ABI cleaned up. There may be some clients breaking because of that. No more exported symbols. See: The Client Library, Part 1: The API, the Whole API and Nothing but the API & The Client Library, Part 2: The Version Number
  • mysql_install_db is now replaced by --initialize in the server, so have to rewrite the post-install scripts. Might also have some AppArmour changes. Spoke to people @ DebConf (so best place is to put AppArmour profiles upstream (i.e. in mysql) and Debian and other distros will get it from there). AppArmour profile is in the MySQL source package now. Probably can get away with doing everything as cmake variables.
  • MySQL 5.7 has disabled the old password hashing algorithm, so if people haven’t upgraded they might have problems; so a manual intervention to fix their accounts.
  • Going from MySQL 5.7 to MySQL 5.6? It is done by dump and restore. There is no testing automated downgrades. Are there disk format changes? Norvald is not aware of any. If you use virtual columns in 5.7, you can’t downgrade easily to 5.6.
  • Robie would prefer to not release 5.6 and 5.7 concurrently. During Trusty, there was some level of user confusion. Debian – release team would prefer to see one transfer than two, so is it better to just do a single transition to 5.7?
  • Norvald says there hasn’t been testing from 5.5 -> 5.7. They only support upgrades from 5.5 -> 5.6 -> 5.7. For Ubuntu the choice can be to have 5.6 and then later do 5.7, but Jessie only just released with 5.5, so Stretch with 5.6 might not be a great idea (so users migrating from Jessie to Stretch will go from 5.5 to 5.7). Could also have 5.7 depend on a stripped 5.6 binary (like the embedded server; this is for localhost and the security team shouldn’t be too annoyed) for people to do an upgrade. Norvald says this has not been tried and there needs to be a migration path tested from 5.5 -> 5.7.
  • Conclusion: 5.7 in Stretch. Xenial is an LTS release, and 5.7 should be targeted for that.
  • If the maintainer script fails (postinstall script fails – don’t leave apt in a weird state). If it fails then upgrades, leave a debconf critical notice to say that the service is disabled and then fix it manually. Otto says that leaving /etc in a broken state is terrible, so we should avoid it.
  • Do we (Oracle) have the resources for 5.7 packaging and how soon can it be done in time for Xenial? There were patches from Lars in the git tree, but there haven’t been more recently. Lars will take over the 5.7 transition so if there is a list of work items, this will be settled (Lars will take over from Norvald).
  • There will be a separate session with Norvald/Lars/Robie outside of UOS about 5.7. Defer the Boost conversation after the session as well.
  • George: Percona is mainly looking out towards the 5.7 work and what kind of resources that will be put to that. There are new folk @ Percona to help with this. Percona inherits so much from the upstream codebase, it just works for Percona Server. There is Percona XtraDB Cluster and Percona xtrabackup, and xtrabackup has moved on quite a bit since the last upload (since last November 2014). So might be good idea to look at a refresh. There has also been a lot of work done on Percona XtraDB Cluster and there are some developments with Codership, so they are unsure if they will have their own Percona XtraDB Cluster 5.7 by the time Ubuntu is supposed to ship. When Percona is ready for something, just give Robie a shout to ensure that things happen. 60-90 days before a Percona Server 5.7 release. Just be aware of feature freeze for Xenial.
  • Norvald mentions that Percona Server 5.7 will also depend on Boost and there needs to be a decision on this. George mentions that TokuDB is now part of Percona Server, and it has some of its own requirements as well. Do we include TokuDB? It has requirements like it will only run on 64-bit platforms. Things to figure out going forward? MariaDB has been carrying TokuDB last November, but Robie remembers disabling it in Ubuntu. George says there were some licensing issues back then but they seem to be taken care of.
  • Otto says the builds for TokuDB was failing. It has a dependency on jemalloc, and that might have been the reason there were failures (says George). There may be something else where it doesn’t build on Ubuntu builders. But Otto says that there was a commit where this got fixed about last month. George will follow on, just to absorb it, since the legwork is already complete.
  • Otto: Trusty has 5.5, and Jessie and all other Ubuntu releases have 10.0, and 10.1 was released last month and I’m not quite pushing it to Debian quite yet. Fix 10.0 build fixes, upstream them, then only focus on 10.1. Blocking? (last summer) 5.6 is not in testing, so could not depend on it/changes done in 5.6 mysql-common. Here’s hoping that mysql-common going forward will be generated separately.
  • Robie will take an action to resolve the delta (probably just drop it). To sync MariaDB 10.0 to Xenial.
  • Discussion on /var/lib/mysql/*.flag thing on the list — conclusion at: mailing list — goal: within a single Ubuntu release, people can “cross-grade” between MySQL variants. The goal is to support all 3, and users want to try them, and thats when the bug reports come. Robie’s goal: move to a per-variant data directory. Otto says that once directory names change, 3rd party tools might have breakage. So a working prototype. Migration path is difficult. Maybe the best is to turn /var/lib/mysql into a symlink and store the data elsewhere. PostgreSQL does per version directories today; so studying that is going to happen.

Ubuntu 14.04 – some MySQL ecosystem notes

Following my previous post on the launch, I just rolled Ubuntu 14.04 LTS on an Amazon EC2 t1.micro instance (not something you expect to run a database server on, for sure – 1 vCPU, 0.613GiB RAM). If you do an apt-cache search mysql you get 435 return result sets with the default configuration (trusty: main & universe).

If you do apt-get install mysql-server, you get MySQL 5.5. You enter the password of choice, and before you know it, MySQL is installed (a SELECT VERSION() will return 5.5.35-1ubuntu1).

Next you decide to install MariaDB. I run an apt-get install mariadb-server. It pulls in libjemalloc (for TokuDB) and I expect future releases to ship this engine by default. You enter the password, and you get a new message (as pictured).

MariaDB Ubuntu 14.04 LTS

I verify my test database that I created exists. It does. A SELECT VERSION() returns 5.5.36-MariaDB-1. The innodb_version returns 5.5.36-MariaDB-33.0.

I’m curious about MySQL 5.6 now. So I run apt-get install mysql-server-5.6. Not so straightforward. 

start: Job failed to start
invoke-rc.d: initscript mysql, action "start" failed.
dpkg: error processing package mysql-server-5.6 (--configure):
 subprocess installed post-installation script returned error exit status 1
Setting up mysql-common-5.6 (5.6.16-1~exp1) ...
Processing triggers for libc-bin (2.19-0ubuntu6) ...
Errors were encountered while processing:
E: Sub-process /usr/bin/dpkg returned an error code (1)

Looks like MySQL 5.6 is more memory hungry… I edited /etc/mysql/my.cnf to ensure that innodb_buffer_pool_size = 64M (I increased this to 128M and it worked too) was set (there was nothing in the default config) and re-ran apt-get install mysql-server-5.6 and it started. My test database was still around ;-)

I wanted to make sure that MySQL 5.6 isn’t more memory hungry just on that instance so I created yet another clean t1.micro instance and did an apt-get install mysql-server-5.6. Same error. Reported lp#1311387.

Nothing to report in particular about Percona – 5.5.34 Percona XtraDB Cluster (GPL), Release 31.1 (Ubuntu), wsrep_25.9.rXXXX. One thing is for sure – if you’re playing around with the ecosystem, installs and upgrades aren’t exactly straightforward.

Congratulations Ubuntu, for the wide choice!

Inspired by Yngve Svendsen’s post, I too think it makes absolute sense to congratulate Ubuntu on the 14.04 LTS release (some server notes – MySQL has a section dedicated to it). Ubuntu users have a lot of server choice today (that’s from all major MySQL ecosystem vendors):

  • MySQL 5.5.35 ships in main. It is the default MySQL. Oracle has committed to providing updates to 5.5 throughout the LTS release cycle of Ubuntu (which is longer than the planned EOL for 5.5). This is why the grant of a Micro Release Exception (MRE).
  • MySQL 5.6.16 ships in universe
  • MariaDB 5.5.36 ships in universe.
  • Percona XtraDB Cluster 5.5.34 ships in universe

Ubuntu’s pitch is being the cloud platform of choice, with OpenStack support. This explains why Percona XtraDB Cluster (the only shipping Galera Cluster variant — no upstream Codership release, and no MariaDB Galera Cluster) is critical infrastructure as its used widely in OpenStack deployments. 451Research estimates that the OpenStack distributions market is worth $82 million in 2014 and $119 million in 2015.

Press release had a choice quote from Percona CEO, Peter Zaitsev:

“We are very pleased that Percona XtraDB Cluster is included in Ubuntu 14.04 LTS. Many organisations that use MySQL need high availability solutions to ensure that their applications meet the expectations of their users. Percona XtraDB Cluster is an easy to use, open source solution for MySQL clustering which addresses these high availability needs. We continue to see growth in Ubuntu usage by our customers and our open source software users so we are confident that the inclusion of Percona XtraDB Cluster in Ubuntu 14.04 will help spread the adoption of cost-effective, high availability MySQL.” Peter Zaitsev, Co-Founder and CEO at Percona


MariaDB & distributions update, Dec 2013

A few things to note recently, amongst MariaDB in distributions. 

  1. Ubuntu keeps MySQL 5.5 despite MariaDB’s success. There’s a lot of reasons for this, but remember the key takeaway here is MySQL 5.5 & the fact that MariaDB wasn’t even in Debian yet when the decision was made.
  2. MariaDB is now inside of Debian/sid – check out the packages.
  3. RHEL 7 comes with MariaDB 5.5 as a default; this is a good thing.

Now, from a distribution standpoint, we’re looking at starting to ship 10.0 as well. Distro maintainers don’t want one-way streets (i.e. an upgrade to MariaDB prevents you from going back to MySQL). This is something we have to deal with as more start looking at MySQL 5.6 & MariaDB 10 (think temporal literals as an example).

Ubuntu Edge failure and what it means to me

A few days ago I received a credit from Indiegogo, because the USD$600 that I pledged for the Ubuntu Edge didn’t work out (I pledged on day one not because it was cheap but because I felt I needed the device and thank Canonical for the wonderful work they’ve done in addition to being brave about going into new markets; I would have paid $895 if need be – we don’t get heavily subsidised phones where I come from). There was a lot of buzz about how this is the largest crowdfunding experience ever, and so on, but to me, as a believer in opensource, I feel this failure to get an Ubuntu Edge more than ever.

It was by no means a shoddy amount that was pledged, in the sense that it raised USD$12,813,501 out of the USD$32,000,000 goal. I was curious with who pledged, and this is quite public as well – see the pledges list. But what you see is that a lot of people pledged not for the phone but smaller amounts which I guess is a huge problem.

Simply put, you need about 50,000 people (community members/Ubuntu users/etc.) to pledge to buy the phone (at an average sale price of USD$695). A mere 50,000. I planned to analyze the data, but its great that The Guardian did most of the work for me, so read: Ubuntu Edge: how many phones were really ordered – and the mistakes.

14,577 individuals pledged to order the phone. Enterprises were shy by the looks of it.

Out of the 14,577 individuals, I expect many of them to be Ubuntu users to some extent (if not lovers of opensource). Where are the rest of the Ubuntu users?

The public stats for Ubuntu are quite impressive – generally it is the most popular desktop Linux distribution out there. Just look at the adoption & reception: in June 2009, it was estimated that there are 13 million active users; in fall 2011 Canonical itself estimated more than 20 million users worldwide. This number must have grown tremendously, but even at a 20 million base, you’re looking at 0.073% conversion rate to buy an Ubuntu Edge.

I know people that are Ubuntu users and wanted to buy it, but not at the price point. Over $600 for a phone with a computer that docks just isn’t feasible as a cost in many parts of the developing world. Without user registration, we can’t tell where Ubuntu users are located, but I’m willing to bet it’s a good mix between the developed/developing world, right?

I was hoping to hold an Edge in my hand come May 2014. I’m still hoping to hold an Ubuntu mobile device in my hand. While I am disappointed, I can imagine Mark Shuttleworth asking himself a lot of questions. He’s spent millions developing Ubuntu, the community that surrounds it and the commercial aspects around it. Apparently monetizing the userbase is harder than it looks.

Facebook Home

Colored houseI happened to be awake last night so I caught the announcement via livestream for Facebook Home. I’m glad its just a system launcher. There are many (I myself on my phone use Nova Launcher), but from the demo, this is beautifully designed with a new take on the interface. The demo showed it being smoother than butter ;)

Chat heads look interesting. Some may claim it being bothersome or unintuitive, but most iOS users have this already turned on via accessibility settings assistive touch since the home button breaks far too easily. Why a little white dot when you can now make it do things for you?

I was impressed with the amount of partners on launch day. Buy-in from manufacturers like HTC, Samsung, Sony, Huawei, Lenovo, ZTE, Alcatel. Chipmaker like Qualcomm. Telcos like AT&T, Orange, EE. I can only expect this to grow of course. Gives great competition in the mobile landscape for 2013.

You see, FirefoxOS has a huge amount of partners & buy-in. I continue to be surprised that Ubuntu doesn’t have a similar page.

Am I switching from iOS as my main phone? Unlikely. I’m almost certain that many at Facebook, including Zuckerberg runs on IOS. But I will be playing with this on my secondary device (the Galaxy S3). I’m a little surprised that the April 12 launch isn’t available for the Nexus set of phones… and in Asia, the Galaxy Note form factor is popular, where did that go?

HTC First will be the first device to come with the Facebook Home system launcher as a default. I’m not sure how this is different to them applying skins and admitting that Facebook does it better. This isn’t the first time they’re playing around with a Facebook phone though.

Interesting times as Facebook has confirmed that their strategy is clearly mobile first. The fact that they built this on top of Android can’t really impress Google very much ;)

Others have also covered this well, i.e a strategy for Facebook, how this isn’t good for privacy.