Colin Charles Agenda

Red Hat releases Spacewalk. It is described as: “the upstream community project from which the Red Hat Network Satellite product is derived“. Congratulations to all whom have worked on it, especially my friends who tired endlessly over it in the past.

Red Hat, is sticking true to its promise, of open sourcing everything they make. Best of all, they recognise Fedora (they always did, since say, Fedora Core 2 or 3), CentOS (a direct “competitor”/rebuild of RHEL), and Scientific Linux (I know of a certain university’s sysadmin who will be blessing Spacewalk, as her life will now be a lot easier).

There have been a few blogs about it… Matt Asay asks about a community (Red Hat traditionally wasn’t good at this, but with Fedora, I believe they’ve learned, and I’m happy to say I think, I helped in the education process). No one however, focused on the technical aspects around Spacewalk/RHN.

Case in point: Oracle is at the heart of it. RHN was designed almost seven years ago, and I’ve heard amazing stories from Gafton, Greg, and Peter. How Gafton found hidden “secrets” inside Oracle to boost performance, and a whole bunch of interesting things, best to talk about over a beer (the irony? When I first met these folk, I couldn’t even legally drink a beer in the US…)

Read the Developer Documentation, note that they use Perl, Python and Java in the current code base (but only Perl and Java is the way forward). There’s a DB Schema available… and I wonder when someone will port this to MySQL?

The Spacewalk FAQ mentions the lack of resources in the past to add an open source database, but would want to do so soon. There’s even help on getting Oracle XE running. The glimmer that there is to be an open source database behind Spacewalk, is what tells me that the MySQL community, that benefit from such a tool (so you’re a DBA and a sysadmin at a fairly largeish installation), should port this to run on MySQL.

What else can we take away from Spacewalk? The excellent positioning. A community project from which the RHN product is derived. This is similar to what Fedora is positioned as: Another striking difference of Fedora is our goal to empower others to pursue their vision of what a free operating system should be like. Fedora now forms the basis for derivative distributions such as Red Hat Enterprise Linux , the One Laptop Per Child XO and Creative Commons’ Live Content DVDs.

Distinctive naming. Helps create a lack of confusion (at the price of an ubiquitous name? Sure, you just have two ubiquitous names now). MySQL Enterprise vs. MySQL Community. They’re both MySQL (don’t even get started on the odd/even numbering scheme…). I dream the day, when we have MySQL Enterprise and Sakila (formerly known as MySQL Community).

Dear (mac/bsd) Lazyweb,

Any idea how I can do the following, on Mac OS X?

iptables -t nat -A PREROUTING -p tcp -d my.pop.server –dport 110 -j DNAT –to-destination 127.0.0.1:1235
iptables -t nat -A OUTPUT -p tcp -d my.pop.server –dport 110 -j DNAT –to-destination 127.0.0.1:1235

Does ipfw(8) work in OS X? What magic will I have to use for this to work? All I really want is for my.pop.server:110 to point to localhost:1235, so if the SSH tunnel isn’t up, I can’t POP my mail. I wrote about this in Secure travelling tips with iptables and SSH port forwarding, as that’s what I do on Linux.

Why do I ask? I’m thinking that my personal laptop (currently a Dell Inspiron 640m running Linux) might actually be replaced with a MacBook Air in the near future (lighter, easier to carry, etc.). Oh, if you have comments about the Air, don’t hesitate to tell them to me as well.

As some of you would have recently noticed, I’ve started playing around (I can’t really say using, seeing that its still not a daily basis kind of thing for me yet) with OpenSolaris and NetBeans. Now, Sun is encouraging students to use, review, and blog about these two great products - check out the student reviews contests.

If you’ve not tried OpenSolaris 2008.05, you should probably give it a twirl. Sure, I’ve not installed it on bare metal hardware yet (I’ve been travelling so much, I’m not anywhere near machines), but it works fine inside VirtualBox. NetBeans is great if you’re writing Java, Ruby, PHP, and are connecting to a database - Connector/J for MySQL is shipped with it. If you were trying it out, take a look at the NetBeans+GlassFish+MySQL bundle.

So, students out there, take a look at the Student Reviews Contest. You have till June 6, 2008. There’s your chance to win anywhere between $100-$250 (USD?), which really means a lot of beer ;)

(and here’s hoping we run something similar when MySQL 5.1 is GA… we used to run bug finding contests where we used to give away iPod Nano’s, though I think Visa Debit Card’s are so much more useful than iPod’s…)

VirtualBox 1.6 is out. Note that now you can use Mac OS X and Solaris as a host platform. Naturally, having Mac OS X support excites me.

I tried installing a Ubuntu 8.04 server guest. Found a tiny issue - 64-bit guests aren’t supported yet :( So I pulled in the 32-bit ISO, and that installed just fine. Note that PAE support for guests exist now, and this is a good step in the right direction.

Sun’s building an OpenxVM community, which currently focus on xVM and xVM VirtualBox. It also harnesses technologies like Open Service Tag. All in all, I think a lot of MySQL users should be interested in virtualization, as there is a growing amount of hardware out there with many, many cores available for use.

Installing Ubuntu 8.04 server and getting past the kernel not booting
This is more of an Ubuntu problem, than a VirtualBox problem, but I faced an issue:

The kernel requires the following features not present on the CPU
0:6
Unable to boot - please use a kernel appropriate for your CPU

Turns out, the problem was the wrong kernel was installed. Rescue Ubuntu, and install linux-generic. For reference, look at Unable to boot 8.04 Alpha 3 Server install on laptop and also the fix.

I paid great attention to Ian Murdock’s talk at LugRadio Live USA 2008, as he’s an important person at Sun in terms of the open source community, and I’m community facing as well. It was also the first time I got to meet Ian (after his talk), and we hit off a conversation really quickly. I look forward to working alongside Ian more… Now to the talk notes.

A Bit About Ian
- Linux user, developer, and advocate since 1993
- Founder of Debian, co-founder of Progeny
- Joined Sun in March 2007, Chief OS strategist (launched Project Indiana), and now since February 2008 is VP Developer and Community Marketing

What’s a Linux guy doing at Sun?
- When people say they want Linux… they don’t actually mean that they want Linux. You don’t only want the kernel, but you want the userspace as well. You really want a distribution, not Linux itself. Ian is more of an open source guy. So create a business model behind OpenSolaris?
- Solaris ships all userland utilities you’d expect from Linux. GNU Utilities vs. UNIX Utilities. Where the real differentiation is, is Linux Kernel vs. Solaris Kernel.
- Its the kernel where you drive change for the OS. Look at ZFS, Dtrace, containers, etc.
- Linux has multiple configurable platforms with expanding ecosystems. OpenSolaris, is an integrated operating system, with binary compatibility.
- Where Sun wants to go with OpenSolaris? Have binary compatibility in the core platform, and have the expanding ecosystem in it.
- Project Indiana has Solaris innovation with the “distribution” model of the Linux world, i.e. have package repositories. Also, to close the “familiarity gap” so that there is a GNU userland, a package management system, etc. And of course, focus on the unique Solaris capabilities - upgrade rollback via ZFS snapshots, AMP stack with integrated DTrace probes (MySQL in OpenSolaris Developer Preview 2 already has this capability), binary compatibility, etc.
- Remember, people deploy what’s accessible to them. This is a whole new deployment model. This is how Linux got into the Enterprise. This is the reasoning behind OpenSolaris… Solaris itself is great in the Enterprise, but lacks a user community. Follow the Linux (and largely, Microsoft) model…
- One Solaris Platform, two delivery mechanisms - Solaris: enterprise class, support, long release cycle (3-5 years); OpenSolaris: latest Solaris innovations, short release cycle (6 months), much easier to use, network economy, support from Sun also
- Adoption-Led Market is what its at now. Users become customers. Give users something of value. That’s the basic business model at Sun, around open source.
- Volume drives value! New consumers/communities on the network will drive infrastructure demand. Reach the developer community through massive volume, get it in use as many places as possible, then sell software/storage/servers/services/etc.
- Just spent $1 billion on MySQL. Databases are often deployed on other middleware, say identity management. Sun sells identity management software.
- Developer platforms of choice are moving up the stack. Traditional Sun platforms are increasingly invisible infrastructure. New platforms remove barriers to entry and hide complexity so developers can focus on getting to market quickly.
- Remember, no one “owns” the web. The Web 2.0 world is like open source in the 90s.
- Hot new platforms are silos - you write your application to Google APIs or Amazon APIs and there’s only one place to deploy it. Sun is a leader in open standards for 25 years, how do you take this leadership into the new platform world? One needs the ability to deploy on a number of different platforms, and take your application from one to another. Sun strategy?
- Where do down the stack technologies, like operating systems, fit? Do they still matter? Yes.
- Solaris 10 is free today (you just have to register to get it)

On the packaging system
- OpenSolaris package system is new. Needed to build own for a few reasons, despite studying apt/yum/conary.
1) platform is very different from Linux; use ZFS and you get rollback, and this is Solaris specific
2) zones/containers, single shared kernel, multiple application environments, which is intelligent for resource sharing, and the package system is fundamentally aware of zones
3) an opportunity to innovate; Sun’s all about innovation. Package managers haven’t changed much in over 10 years (apt, yum)
4) the notion of customisation, i.e. have a version control like facility for patches, etc. so that changes can be reconciled when you rebase with upstream, is something the new package management system can use. This is kind of like Conary?
5) All customers tend to customise, even (especially) in Enterprises. IPS helps them (refer to 4)

Questions & Answers
- OpenSolaris and Solaris, how is it in sync? Codebase is in sync, generally. Solaris 10 is one codebase, and there’s a development version (i.e. the next release). There is a Solaris “train” and an OpenSolaris train now. How will these two trains come together? Solaris Next based on OpenSolaris? OpenSolaris with Enterprise support? Questions that are being asked now, and there’s no real decision yet. Wait till 2-3 OpenSolaris releases first, before finding out more.
- What is Sun doing to make OpenSolaris work better in a virtualized environment? I.e. working better in Xen, QEMU, VMWare, etc. Linux naturally has better device support than OpenSolaris, so virtualization is very important. Mentions the innoTek purchase, so that VirtualBox could be used for bundle delivery.
- Is Linux and OpenSolaris in par, security wise? Solaris has a slight edge at this point… They’re fairly comparable, but there’s more maturity in the Solaris product.
- OpenSolaris on PowerPC - unknown status, there’s been some work at Sun Labs to port it to PPC, you can boot it and do some basic things, but unsure if its ready for users or not.

Technorati Tags: lugradio, lugradio live usa, ian murdock, sun, opensolaris, linux, distributions, IPS, packaging system

No, its not alphabet soup. Just some notes from the session at the Sun Tech Days. I’ve not looked at DTrace much (my only look into instrumentation, has been from SystemTap, which doesn’t deal with applications), but plan on doing so soon… I’ve managed to get OpenSolaris Developer Preview 2 installed in VirtualBox, so it can only start being more fun from here…

Want to learn more about DTrace and MySQL? Then come to the MySQL Conference & Expo 2008, in Santa Clara, California, because on Thursday, Ben Rockwood, from Joyent, will be presenting a session on DTrace and MySQL (read the abstract, its good). The talk covers the fact that you can get useful information currently, even without the embedded probes in current versions of MySQL. For more DTrace and MySQL tips, don’t hesitate to read Joyeur, Joyent’s weblog.

DtTace, Web 2.0, Java, AJAX, PHP and the rest (notes from the talk)
by Peter Karlsson, Solaris Technology Evangelist

DTrace now has providers for a large number of languages: JavaScript, Java, PHP, Python, Ruby. Perl is on the way.

MySQL 6.0 will have DTrace support; PostgreSQL already has this in Solaris currently. If building from source, there’s a flag that needs to be enabled. A lot of work was done thanks to a community member.

You need a Solaris kernel. Ported to OS X and FreeBSD. Supports “dynamic instrumentation”. D is the dynamic language, used to script instrumentation

Very common request? Find how much time is spent in a given function. The thread local variable (self->variable = expression;) - nowadays, you can be running two threads coming down in the same function call. DTrace - so this is great for multi-threaded debugging.

PHP doesn’t have DTrace integrated, so, get the Coolstack PHP.

DTrace probes have been added to Mozilla to help debug JavaScript applications. This is available in Firefox 3 (in beta now). There is also a generic DTrace framework, that isn’t just JavaScript only - the networking parts of Firefox, to look at how DNS lookups work, etc. all can be instrumented via DTrace.

Further reading? DTrace and PHP, demonstrated.

Technorati Tags: dtrace, php, javascript, mysql

Being the long-time Linux user that I am, there was no way I was going to be satisfied with Xandros, which is the stock Linux that ASUS ships with the Eee PC. I was stuck for choice between Ubuntu and Fedora, and after some careful evaluation, I decided that Fedora, was right for me.

Getting Eeedora, was pretty straight-forward. The installation wiki is pretty accurate when it comes to the “how” of installing Eeedora. For me, it was made easier that I had a Fedora system already, so I could run the tool to create Live CDs from the livecd-tool package.

One snag I noticed, and this is more with the Live CD script, is that when you don’t have a bootable USB thumb drive, it tells you that, but doesn’t quite tell you it didn’t make an installation on the thumb drive. I guess the script could be more idiot-proof. Anyways, making the drive bootable is easy.

Now, once that was done, it was on to installing on the Eee. I ensured that in the BIOS (accessed via pressing F2), the first boot device was the USB thumb drive attached to the system at boot-up. However, it was never booting into Eeedora, and I only managed to see the Xandros start up. Highly disappointing.

Turns out, that hitting the Esc key was the magic sauce, during boot-up. Only when doing that, was I given a boot menu, and then I could choose if I’d like to boot off the internal SSD, or the USB thumb drive. Once that was sorted, I was pretty happy to see a familiar Fedora-looking screen (sure, it said Eeedora, for legal reasons, but I think its a pretty darn good spin :P).

The installation process went on pretty smoothly. During partitioning, I was pretty much rid of Xandros - the chosen default was actually the most sensible. Just one / partition, filling up the entire disk, with no swap. As usual, anaconda (the Fedora installer) will warn you that not having a swap partition will be detrimental to performance - I wonder how many newbies might decide to create a swap partition (kind of a big no-no, on these SSD based devices).

One thing I did notice was the use of ext2 partitioning as opposed to a journalled filesystem like ext3. The natural question then comes to mind, as to why not just use jffs2?

Once the filesystem partitioning was sorted, and GRUB was chosen as the default bootloader, anaconda proceeded to install packages. This was a fairly speedy process, and once that was complete, it was a simple reboot, and I was booting off the SSD before I knew it.

You’re logged in by default as the eeedora user. You get a stock XFCE4 environment (or stock, that I would think anyway), and it comes with the basic utilities you need to get working (Firefox, wireless, etc.).

There were some things I obviously did not like, so decided to poke a little further. First up, was creating a user, that was not”eeedora”.

Dissecting the .bash_profile of the eeedora user, I noticed some things:

• pulseaudio -D - pulseaudio as a daemon for the user? Why, sound seems to work just fine on my Eee without it
• xsetroot -solid steelblue - seems OK, but not actually required anyway
• startxfce4 - if you want a GUI the moment you login, this is useful. But it only really works out well, because of the hack in /etc/inittab that says c7:12345:respawn:/sbin/mingetty –autologin=eeedora tty7. Seeing that I disabled autologin for the eeedora user, I pretty much see a login screen, and startxfce4 manually. Sometimes, I can actually get away with working in just a shell… However, I can see this from a usability point of view, I guess. Of course, the other issue is that the eeedora user, does not actually have a password!
• Logging out of XFCE4 and going back to the console, immediately triggers a machine shutdown. This seemed counterintuitive.

Needless to say, I disabled all this, in my new user account, as well as disabled autologin in the inittab.

Why XCFE4? Its a 900MHz Mobile Celeron processor, deliberately underclocked to 600MHz, with 512MB of RAM. This is the kind of environment, that can run GNOME. And when we notice problems with GNOME, its time we fixed it. Heck, I have an almost similarly configured IBM laptop, without PAE, that runs GNOME just fine!

I must commend WiFi Radar - this is something that pretty much, just works. And its small. It looks unmaintained, sadly, so its probably time to take an interest in it. Fn+F2 enables and disables wireless very well. I do however notice that even with the wireless disabled, the WiFi light (in blue), remains turned on. I find this to be rather quaint, and must prod it further.

Brightness control (Fn+F3/Fn+F4) seem to work well in Xandros, but I can’t seem to replicate such joy in Fedora. There is absolutely no reason why this shouldn’t work. It does annoy me, because its sucking precious battery life, from my usage of the Eee, by keeping it nice and brightly lit, when I’m on the battery.

Suspend and resume, just work. And because we’re dealing with an SSD, it just works, really fast as well. Sure, when I open up the screen of my Eee, I actually need to press the power button to get it to resume, but this kind of behaviour is perfect for me.

Do I like recompiling all the magic in /root/eee-setup, everytime I get a new kernel? No. Speaking with Dave Jones though, there is an expectation that all the drivers will be available in F9. Kudos.

Webcam? Works fine when I run lucview. Doesn’t work with the Skype beta that I get from the Skype download site. No idea what is missing, I just feel a bit bummed that I’ve got to find the solution to this issue at some stage soon.

As an install report, I think this does, just fine. Its now just time to hack on the distribution, to get it to the way I want it to work.

Peter posted that CentOS comes with a build of MySQL Enterprise. It should really be clarified that CentOS itself, comes with MySQL Community, as does Red Hat Enterprise Linux. On RHEL5/CentOS5, you’ll see:

mysql-5.0.22-2.2.el5_1.1
mysql-server-5.0.22-2.2.el5_1.1

The above are the default packages that CentOS provides. However, what Peter really is referring to is the CentOSPlus Repository, which by their own admittance is “not part of the upstream distribution and extend CentOS’s functionality at the expense of upstream compatibility. Enabling this repository makes CentOS different from upstream.”

The idea behind providing Enterprise builds, largely came from RHBZ#230412: No src.rpm available for mysql. Red Hat provides something called the Web Application Stacks product (RHWAS), in where they include MySQL Enterprise, amongst other software packages, and they charge for support (that differs from buying just a RHEL license). Max Spevack answers it pretty well, as this is the response he came to, after talking to folk at MySQL.

Its worth noting that CentOSPlus does not use Enterprise tarballs/SRPMS, but use sources from the BitKeeper tree. Its worth noting, that at the time of this writing, CentOSPlus does not include Enterprise RPMs for CentOS 5, just CentOS 4. Its also worth noting that patches are being applied, that are out of tree, to the RPMs. A snippet from the RPM changelog (rpm -q –changelog <packagename> if you must):

* Mon Dec 24 2007 Johnny Hughes <johnny@centos.org> 5.0.54-1.el4.centos
- upgraded to the 5.0.54 Enterprise BK tree. Removed patches that are
already incorporated into 5.0.54.

- added mysql-5.0.52-mysqldump-hang-33057.patch for mysql bug #33057

- added mysql-5.0.50-openssl-handshake-33050.patch fo mysql bug #33050

* Sun Dec 23 2007 Johnny Hughes <johnny@centos.org> 5.0.48-3.el4.centos
- modified the process of obtaining the Enterprise Sources to using bkf and
downloading directly from the MySQL Enterprise BitKeeper tree.

So, definitely interesting times. If you hit a bug, you might be wondering if its an upstream bug, or something that was provided by your packager. Of course, this is what is so great about the MySQL community - the ability to do just this! Currently, against 5.0.54, CentOSPlus is shipping 2 patches, with a few extra source packages that don’t come with regular tarballs. Will this number grow? Just remember, the “Enterprise” version you get in CentOSPlus is not “MySQL Enterprise”.

Technorati Tags: centos, mysql, distributions shipping mysql, patches