After three years it was time to move the server (under which this blog is hosted) again. Now we get eight times the memory for the same monthly price, not bad indeed. The newly setup environment makes heavy use of virtual machines and this is the topic of this post.

A friend of mine and I once already managed a server that consisted of virtual machines handled through Xen. Setting up the base hypervisor (the operating system under which the guest virtual machines run) was a major PITA, especially on a remote machine where you do not have access to the boot manager.

Now fast forward to today, I’m using Ubuntu 9.04. Most of the work was done by just installing ubuntu-virtual-server and ubuntu-vm-builder. The former setups a virtual environment (using KVM) with a virtual network hub where the new virtual machines will be connected on a private subnet. Adding a new virtual machine was more or less invoking ubuntu-vm-builder which states the name of the vm, its distribution and size and then starting it with virsh start name-of-the-vm. Afterwards you’re already able to login to the virtual machine through SSH, add some routing rules on the host machine so that the virtual machines are accessable from the internet and you’re done.

Later on I’ve found some other capabilties of the virsh tool: pool and volume management. Alas while it was able to detect and view my LVM partitions I was not able to dynamically attach one volume to a VM, but maybe this will function soon.

Another nice tool is virt-manager. This gtk-application allows to connect to a remote host running ubuntu-virt-machine. You’re then able to monitor and alter the settings of the network, guest virtual machines and volumes from a graphical tool, also there’s a VNC/console forwarding for accessing the remote machine even if the network dies or is misconfigured.

All after all I must say that I’m impressed with the usability progress..

Over the last one and a half years I’ve been involved with the SECOQC project. It’s goal was to provide a prototype of a quantum key distribution network. Such a system would provide unconditional security, thus wouldn’t be isn’t vulnerable to improvements in computing power as traditional cryptography.

The final presentation of the prototype happened this Wednesday. The last days and nights before that were filled with applying the last fixed but finally it was worth the time. But let the newspapers do the talking: orf, heise, der standard, sueddeutsche, Austrian Telekom News. There was quite good news coverage in german-speaking Europe (and some eastern europe countries) but sadly the news didn’t seem to have jumped over the pond (at least some American physicists were at the presentation so it got noticed anyway).

Feels strange to know that something that big and cutting-edge is finally successfully finished.. and that I’m an unemployed student agai

One of the components of the system developed here at Blackwhale is an fairly advanced web mail/campaigning and analytics system. The first iteration of that component was fully implemented in Ruby on Rails. Writing the front end was fairly easy and fast, a perfect opportunity for Rails to display its strengths. On the back end and communication part on the other hand I stumbled into various problems:

• Compared to PHP, Python or Java there are just too few communication libraries. And even the libraries that exist are lacking fundamental features and almost all of them come without useable documentation. That there is no encryption (SSL or TLS) support in the whole Ruby 1.8 mail libraries is an outright shame. The IMAP library is so clumsy that a web company sells their own (still not perfect library) through their online store and they seem to make a good buck with it.

  On the Java side is just a completly different picture: Lots of libraries, even documented ones are available. After having to touch the TMail (rail’s MIME mail handler library) API using javax.mail is a heavenly gift. And it seems that the TMail generated MIME messages were invalid in a couple of cases. Not the best feature of a support library. Also the chance of finding unsolved known bugs and errors seems to be lot smaller in Java.

• This brings me to another point: I might change my opinion on Java’s constraints on its users. Java tries hard to prevent errors (i.e. the forced exception catching). I always thought that that took too many stylish possibilities away from the user, but by now I must confess that I think that this is exactly what I want from something that I’m using on the network side. I’m a lazy programmer, I want to be reminded and forced to write secure and stable code. This is the quite different to Ruby and Rails ‘make it easy for the programmer’ attitude.
• Background processing is hard. As Ruby on Rails is not multi-thread safe you can’t just spawn a thread if you need to perform some longer running task. Another disadvantage of using a single-process model is that the long running request will occupy one rails worker (i.e. rails cluster process) until it has finished – in our case that costs us around 60MB of memory per long running request, even if it is just waiting for some simple SMTP feedback. If you can find a situation where you can delay the execution of a network related task (and if you don’t you’re not thinking) for two seconds, 6 requests per seconds will DoS a standard rails cluster.

  The only solution that’s actually usable is BackgroundRb. But projects that just change their background communication system just don’t sound to production-grade ready for me. Also the admin start/stop scripts for their background server didn’t work too well for me.

• For a language that interferes it’s object’s attribute types directly from the database the ActiveRecord layer is weak. Don’t get me wrong, I understand that the Simplicity is needed to make it easily usable but I ran into various situations where I’d love to have a full blown ORM behind me. One feature that is needed quite often by our application is inheritance. ActiveRecord only offers single table inheritance, and even there you have to make sure that each row is valid (ActiveRecord should have all needed information to do that by itself BTW) or you will run into problems later on. One problem is, that it tries to abstract too much functionality away from the database while not provided as advanced interfaces by itself. Data integrity handling? Abstracted away by rails, so all databases can be used the same. The drawback is, that the data constraints and relations are fully handled by rails and not passed on to the database. Any process that might produce invalid data (e.g. a faulty rails component) might corrupt the data. Rails is able to handle that cases by itself (due to ducktyping and very few default checks), but access that data with any other framework and it blows up directly into your face.
• Transaction handling. Just try it. Then cry. Also I’m not sure if Transaction handling is even done on database level or in Rails (as done with constraints). If the later is true, it’s acutally not worth anything as soon as more processes try to access the database.

The library and documentation problems where the main reason for me to reimplement the mailing and campaigning backend in Java. The front end is still a Rails application — which is exactly what Rails is for. As I’m no friend of blown-up EJB based solutions I’ve choosen a simple Spring and JPA based solution for that problem.

I’m currently testing the last features and replacing the Ruby code part by part. As soon as I’ve done that another blog post will examine the two implementations, how much time was spent on coding them and how they perform when compared to each other.

So I’ve finally got my Dell D630 three months after I ordered it. Everything was shiny, even Ubuntu 7.10 worked out of the box except the sound card (as long as you use the safe video mode for the initial install).

But I noticed a small scratch directly on the display panel last Sunday. Not nice. As I’ve enrolled almost every support update that is possible with Dell I contacted their support line, a technician came two days later and installed a new panel. I was quite happy for the quick support.

Two hours later I recognized that my former glossy display had turned into a reflective one. While this is advantageous as I do not own a mirror in my flat I decided to contact Dell’s service again as it is just impossible to work longer while staring into a kinda mirror. Also some pixel errors appeared (around half a dozen always white or black pixel).

After half an hour in the waiting queue the helpful support tech told me that the display will be replaced with a glossy one and the local technician will contact me on Monday. This time I also reported my defective Dell D610 display where the lower left side of the panel seems to be folding backwards (which should be physical impossible). The support guy asked me to send in some pictures to better understand the problem, I did it and just received a mail notification that the panel will be replaced.

So lots of bonus points for Dell’s intent of changing every defective part, some negative points for their execution. There are still worlds between Dell’s support division and that hellhole named Apple Support.

The new laptop included a nice nVidia NVS 135 discrete graphics card so I finally took the time to try Cedega/wine (Cedega is the commerical version of wine that should provide better (3D) support). Through that emulator I should be able to play most (older) Windows-based games under Linux. The only two important games that came to my mind were Starcraft and Civilization IV: Beyond the Sword. The former already worked with my old laptop and the later one produces obscure error messages. While wine just tells me about some “shader error” (for which I tried the various hints from the web, but none of them worked), cedega does the right thing ™ of just segfaulting the background process and giving the user no feedback about the execution status whatsoever. What was even more curious is that the installation through wine seemed a lot faster than the cedega one.

I spent the last two days trying to setup two wireless access points. The first one should be used as an access point (with internet uplink) and the second one should be used as an wireless bridging client (i.e. there can be normal computers connected to its lan ports).

I used LinkSys linux-enabled hardware and tried most firmwares available for this product. While doing the tests I bricked my device twice, but was able to recover. At least it works by now.

More…

Everyone who has been using a computer for a while now has probably noticed the use of asterisks ******* when typing in a password. New in Kubuntu 7.04 is the use of solid dots when typing in a password. This new feature is part of the overall future in KDE usability.

Wow! what improvement! What about resolving some of the performance problems or bugs? This came from the official release announcement.

While there are lots of tutorials how to get those nifty compiz effects working, I never got my notebook playing nice and fast along it. As I prefer OSS drivers the only viable way wa AIGLX which never did work as well as XGL on my hardware.
This changed today. After playing around with my graphics card config, those were the settings that made a compiz-enabled gnome as fast as the normal one.

Edit the graphics card section in your /etc/X11/xorg.conf:

Section “Device”
Identifier “ATI Radeon R250 Lf [Radeon Mobility    9000 M9]”
Driver “radeon”
Option “XAANoOffscreenPixmaps”
Option “backingstore” “true”
Option “EnablePageFlip” “true”
Option “RenderAccel” “true”
Option “AGPMode” “4″
Option “AGPSize” “64″
Option “DynamicClocks” “on”
Option “mtrr” “on”
Option “EnablePageFlip” “true”
Option “ColorTiling” “true”
BusID “PCI:1:0:0″
EndSection

Get the whole xorg.conf here.

The only thing that makes my computer unstable was enabling “AGPFastWrite”, so I let it turned off. If this still is to slow for you, disable “subpixel rendering” or turn the color depth back to 16bit.
Another annoying thing is the gnome-menu popup speed. I always thought that it’s slowness was caused by a slow icon cache, but alas! it was caused by the gnome developers inserting a .25 second pause before opening windows. I do not know why.

To turn it off, just add the following to your ~/.gtkrc-2.0:

gtk-menu-popup-delay = 0

In your face, Vista.

Through this guide you will be able to up/download files from your Nokia N70 throuth a bluetooth connection. Also we will setup gammu to access more of your phone’s data (i.e. calendar entries).

First of all identify your bluetooth device:

$ hcitool scan
Scanning …
00:17:4B:12:2A:BD Nightcrawler

I’ll use 00:17:4B:12:2A:BD as bluetooth ID through the rest of this post, exchange it for yours.

Now you should be possible to query your phone’s characteristics via “sdptool browse $PHONE“. To transfer files, we will use the OBEX File Transport service which is rep rted as listening on channel 10 – but alas that’s a just another firmware error: rightfully it’s channel 12.

So just enter: “obex://[00:17:4B:12:2A:BD]:12” into your Konqueror address bar and you’ll be able to browse and up/download files on your phone.

Now download the gammu sources (make sure that bluetooth support is enabled) or install your distribution packages. Use konqueror to copy gammu.sis (it’s in the /other/symbian directory of the gammu sources) on your phone. On the phone use the file manager to install it.

Create a gammu configuration file in your home directory (~/.gammurc):

[gammu]
connection = bluerfgnapbus
model = gnap
port = 00:17:4B:12:2A:BD

Exchange my phone’s bluetooth ID (port) with yours. Start gnapplet on your phone and use the gammu command line client to connect to the phone (i.e. gammu –getallcalendar).

From here on, my best experiences where a lot of “Function not supported by phone” but some of the features are already implemented and it should get larger through time.

I needed a secretary-proof tape backup solution for a file server of mine. The only program I found was bru (with the xbru graphical front end) offered by TolisGroup. If you think about using it: don’t.

The TCL/TK GUI is crappy. The ‘automatic’ backup scheduler ‘integrated’ into this tool didn’t work so I used the bru command line tool in conjunction with cron. Which made backups but those were not usable with the graphical interface (while being made with their tools).

Today I discovered why the automatic backup scheduler didn’t work: it just enters itself as cron job which is executed every 5 minutes. The tcl script then checks the day and time vs. its stored values, i.e. “Sat = Sat and 14:30 = 14:30″. Here our setup just blows up. Locales where set to German (for the aforementioned secretary) and thus the first check turned to “Sam = Sat” (Saturday is Samstag in German). Who would ever think about using the weekly day numbers for comparisons?

The time is just checked against the stored one. There’s no check if the job has already started. If your cron job isn’t executed exactly all 5 minutes in sync with your definition it will never be executed.

I spend around 200 euro for this ’solution’.

I feel disgusted. Don’t make the same mistake.

Until recently I was using a postfix/amavisd-new/spamassassin based anti-spam solution on our mailserver. That resulted in constant high swap-in/outs, and as we are using XEN based virtual machines this lead to an unbearable cpu utilisation (100% io wait).

I changed the setup to a qpsmtpd/spamassassin configuration recently. This cut memory usage in half (500MByte to 250MByte), reduced swapping to around 0 pageins/second and decreased the load average from around 1.5-2 to 0-0.5. Overall not a bad change.

For more detailed information, including the adapted scripts for usage with postfixadmin follow the next link:

More…