DB2 Express-C 9.7: very, very cool

I’m using an old Oracle 8i database on a Linux host. I’m running it for the occasional web development. I think I first installed it on SuSE 7.3 or 8.0 system. I think Linux 2.4 was still young at the time and Glibc was probably 2.1 or 2.2. I’ve been moving the DB installation through the various Linux version until today where it still runs just fine on Gentoo Linux with a 2.6.27 kernel and Glibc 2.9. The only problem is, that I need to restart the DB periodically because the initial connection time becomes worse over time.

Anyway since the DB is so old, I’ve finally decided to move the contents to a new database. I could have used MySQL (I’m already using it for other applications), since I don’t really have any particular requirements for the database. However in the end I went for IBM DB2 Express-C 9.7. The database is completely free (of course without support), but is limited to 2 CPU cores and 2Gb of memory. The size of the database(s) is unlimited as compared to the Oracle XE version, which is limited to 4Gb (not that that wouldn’t be enough for me) and only uses 1Gb of memory and one core. I’ve read some good stuff on the Web (here and here) about DB2 and since I’m doing SAP installation with DB2 (and also with MaxDB and Oracle) as a job, I though it would be a good learning vehicle, it I would use DB2 myself for the little development I’m occasional doing. The last thing, that convinced me to use DB2 is the Migration Toolkit, which immensely simplified the move of the data from the source to the target database.

Download the platform archive and the language pack. The installation then is completely painless. Simply execute db2setup. I suggest to use the “Custom” installation option. This directly allows you to select to install the development libraries required for Perl, Python, PHP or Ruby and to install the text-search option. It took a bit to get the (D)atabase (A)dministration (S)erver running. This forum entry was the solution: the DNS entry for the host was missing. I fixed it by adding the hostname as an alias after localhost in the /etc/hosts file.

Next I played with the MTK to transfer the Oracle data to DB2. Biggest obstacle here was, that there was no indication, that the final script for migrating the Oracle data to DB2 didn’t run. The MTK explicitly uses /usr/bin/ksh to invoke the script. The was no indication in the the console window, that the shell couldn’t be started. However, on my Gentoo system, although the pdksh was installed, it didn’t allow the invocation through /usr/bin/ksh. Once I created the a symbolic link, the data was succesfully copied from Oracle to DB2. You have to examine the log files, to see if all database objects from the source database could be created in DB2.

In my particular case I had 3 issues with the conversion. The DB2 script, which creates the database objects, would try to recreate a particular view, which contained a couple definitions like “TO_CHAR(STATUS_SEIT_DATUM, 'yyyy-mm-dd' AS datum“. I had to edit the DB2 script and change these fields to “VARCHAR_FORMAT(STATUS_SEIT_DATUM, 'yyyy-mm-dd' AS datum“. I think, this is an issue, where the MTK could do better. Another issue concerned CLOB columns. The conversion process would recreate an Oracle CLOB column as “CLOB(2G) NOT LOGGED“, which is basically correct, but if you want to move the data later with the db2move command, you are limited to CLOB(1G), because the db2move command has a limitation in this respect. Another issue, I had to consider was due to the move from an ISO-8859-1 character set to Unicode. A column, which is defined as varchar(72) might get wider as 72 character, because typical German umlaut characters (öäüÖÄÜ) take up 2 charcaters. The last issue I experienced was with foreign key constrains. The conversion process drops any foreign key constraints and recreates them after successfull data loading. However my particular data had degenerated over time, so that the recreation of the foreign contraints failed in the final step. Otherwise is was all pretty smooth sailing. BTW, there is an IBM RedBook “Oracle to DB2 Conversion Guide for Linux, UNIX, and Windows“, which details the Oracle-DB2 migration process.

Next step was to install the Perl DBD::DB2 database driver by executing

DB2_HOME=/opt/db2/V9.7 perl Makefile.PL
LD_RUN_PATH=/opt/db2/V9.7/lib64 make install

The LD_RUN_PATH compiles the load path of the DB2 dynamic libraries directly into the Perl module, so that you won’t have to fiddle with the LD_LIBRARY_PATH variable. To successfully use the DBD::DB2 Perl DBI driver in connection with the Apache::DBI module and mod_perl I needed to set two environment variables in /etc/apache2/modules.d/apache2-mod_perl-startup.pl:

$ENV{DB2DIR} = "/opt/db2/V9.7";
$ENV{DB2INSTANCE} = "db2inst1";

There is only one final issue. On my Gentoo system at home running the 2.6.30 kernel, I always need to issue the db2start command twice. After the first invocation I’m seeing an error message in the db2diag.log file.

2009-08-02-20.47.49.327780+120 I4733E357           LEVEL: Severe
PID     : 31906                TID  : 140015648098048PROC : db2star2
INSTANCE: db2inst1             NODE : 000
FUNCTION: DB2 UDB, base sys utilities, DB2StartMain, probe:580
MESSAGE : ECF=0x900003DE=-1879047202=ECF_TRCAPI_MUTEX_LOCK_ERROR
          Trace facility mutex unexpected lock error

The second invocation of db2start then succeeds.

Otherwise you can consider me an DB2 fan. You will definitely notice, that DB2 is a very mature product.

Pulseaudio, Alsa and Kernel 2.6.30

I’m experiencing some hiccups with kernel 2.6.30, Pulseaudio and Alsa (driver from the kernel). In the company I’m using an older PC, which has a onboard Intel ICH5 audio system and an additional CMI el cheapo audio PCI card. This setup basically came from a time, when the Linux Flash plugin was OSS only. So, the onboard soundcard was basically used for the OSS applications and the CMI card for ALSA applications.

# lspci 
...
00:1f.5 Multimedia audio controller: Intel Corporation 82801EB/ER (ICH5/ICH5R) AC'97 Audio Controller (rev 02)
...
03:0b.0 Multimedia audio controller: C-Media Electronics Inc CM8738 (rev 10)
...

Now, since I switched to kernel version 2.6.30 I’m seeing the following in my /var/log/messages with a distorted sound, when the problem shows (irrespective of the used sound HW):

pulseaudio[4525]: alsa-util.c: snd_pcm_delay() returned a value that is exceptionally large: 452148 bytes (2563 ms).
pulseaudio[4525]: alsa-util.c: Most likely this is a bug in the ALSA driver '(null)'.
                               Please report this issue to the ALSA developers.
pulseaudio[4525]: alsa-util.c: snd_pcm_dump():
pulseaudio[4525]: alsa-util.c: Hardware PCM card 0 'C-Media CMI8738' device 0 subdevice 0
pulseaudio[4525]: alsa-util.c: Its setup is:
pulseaudio[4525]: alsa-util.c:   stream       : PLAYBACK
pulseaudio[4525]: alsa-util.c:   access       : MMAP_INTERLEAVED
pulseaudio[4525]: alsa-util.c:   format       : S16_LE
pulseaudio[4525]: alsa-util.c:   subformat    : STD
pulseaudio[4525]: alsa-util.c:   channels     : 2
pulseaudio[4525]: alsa-util.c:   rate         : 44100
pulseaudio[4525]: alsa-util.c:   exact rate   : 44100 (44100/1)
pulseaudio[4525]: alsa-util.c:   msbits       : 16
pulseaudio[4525]: alsa-util.c:   buffer_size  : 16384
pulseaudio[4525]: alsa-util.c:   period_size  : 8192
pulseaudio[4525]: alsa-util.c:   period_time  : 185759
pulseaudio[4525]: alsa-util.c:   tstamp_mode  : ENABLE
pulseaudio[4525]: alsa-util.c:   period_step  : 1
pulseaudio[4525]: alsa-util.c:   avail_min    : 15503
pulseaudio[4525]: alsa-util.c:   period_event : 0
pulseaudio[4525]: alsa-util.c:   start_threshold  : -1
pulseaudio[4525]: alsa-util.c:   stop_threshold   : 1073741824
pulseaudio[4525]: alsa-util.c:   silence_threshold: 0
pulseaudio[4525]: alsa-util.c:   silence_size : 0
pulseaudio[4525]: alsa-util.c:   boundary     : 1073741824
pulseaudio[4525]: alsa-util.c:   appl_ptr     : 16614535
pulseaudio[4525]: alsa-util.c:   hw_ptr       : 16501498
pulseaudio[4525]: ratelimit.c: 11481 events suppressed
pulseaudio[4525]: alsa-sink.c: Increasing wakeup watermark to 50,00 ms
pulseaudio[4525]: alsa-sink.c: Increasing wakeup watermark to 60,00 ms
pulseaudio[4525]: ratelimit.c: 105 events suppressed
pulseaudio[4525]: alsa-sink.c: Increasing wakeup watermark to 70,00 ms
pulseaudio[4525]: alsa-sink.c: Increasing wakeup watermark to 80,00 ms
pulseaudio[4525]: alsa-sink.c: Increasing wakeup watermark to 90,00 ms
pulseaudio[4525]: alsa-sink.c: Increasing wakeup watermark to 100,00 ms
pulseaudio[4525]: ratelimit.c: 11588 events suppressed
pulseaudio[4525]: alsa-sink.c: Increasing wakeup watermark to 110,00 ms
pulseaudio[4525]: alsa-sink.c: Increasing wakeup watermark to 120,00 ms
pulseaudio[4525]: ratelimit.c: 11278 events suppressed
pulseaudio[4525]: alsa-sink.c: Increasing wakeup watermark to 130,00 ms
pulseaudio[4525]: alsa-sink.c: Increasing wakeup watermark to 140,00 ms
pulseaudio[4525]: alsa-sink.c: Increasing wakeup watermark to 150,00 ms
pulseaudio[4525]: ratelimit.c: 314 events suppressed

This problem apparently only shows with the Pulseaudio daemon and on 32bit Linux. I did an extensive editing session with Audacity, which works directly with the ALSA devices and I didn’t experience any problems here and at home, where I’m running 64bit 2.6.30, the problem also doesn’t show.

Anyway, as a fix I downloaded the latest ALSA driver 1.0.20 from the ALSA website and installed the compiled drivers over the kernel modules. With this, sound works just fine again.

Still disk latency problems with Kernel 2.6.30

I’m still having massive disk latency issues with Linux. They started after I build myself a Core 2 Duo system and switched to 64bit Linux. I wrote about it here, here and here. This workaround appeared to fix the problem first, but after some time and filling up the disk space the latency problems were back on my XFS file systems.

This whole time I’m mostly succeeding in working around the problem by either delaying the job until the time when I’m not recording some music or video or using the much smaller root-partition as scratch space DVD creation. The programs, which pretty reliable trigger the problem are avidemux2 and dviauthor. I’m using avidemux2 to remove any advertisement from a recorded MPEG2 stream. Once I write the output file beside the original file on the same file system the problem is usually triggered. When I’m writing to /tmp on the much smaller root-partition I get by without triggering the latency problem. The same is true, when dviauthor prepares the DVD file structure. When the DVD structure is prepared on the same file system as the originating file, the problem is triggered and not, when written to /tmp.

During the development cycle of kernel 2.6.30 there was much discussion on the net about fixes of latency problems, however primarily in connection to EXT3 and EXT4 file systems. Still I was hoping, that some fix in 2.6.30 would have improved the latency problem for me and my XFS file systems as well.

It appears however, that it is not the case. avidemux2 and dviauthor still trigger the problem. People on the Gentoo forums seem to agree, that the situation might have improved, but isn’t fixed yet. I need to convince myself to backup the data on my multimedia partition and change the file system to EXT4 to see, if it might change the behaviour. This is really a problem, which decreases my trust in Linux.

Problems with libstdc++-v3 on Gentoo

I didn’t do any digital photo work for some time with Bibble Pro, however recently I wanted to do something and was greeted with a “Segmentation Fault”. Bibble is a 32bit application, that I’m running on 64bit Gentoo Linux. This is what I got:

$ LD_LIBRARY_PATH=/usr/lib/bibblelabs/bibblepro/libs bibblepro
bibblepro: /usr/lib32/libstdc++.so.5: no version information available (required by bibblepro)
bibblepro: /usr/lib32/libstdc++.so.5: no version information available (required by bibblepro)
AppPath: /usr/bin
SysHome: /usr/lib/bibblelabs/bibblepro/
Userhome set to /root/.bibble
Log
Late init
... messages deleted ...
Speicherzugriffsfehler (Segmentations Fault)

Apparently it has something to with the libstdc++.so.5 (libstdc++.so.5.0.7). Checking the compilation date of the binary package showed, that it was recompiled around the middle of February 2009. With that recompilation something must have gone wrong (newer compiler or whatever). Once I replaced the libstdc++.so.5.0.7 with a version compiled on a native 32bit Gentoo system, Bibble was working just fine again.

BTW, I noticed that there is now a public preview version of Bibble 5. This version is in the making for quite some time now. Bibble 4 is really getting long in the tooth. I hope, they can release something sometime soon.

A4 Paper CD Case in SVG

If you search on delicious for “paper cd case” apart from other you’ll find these two pages (1, 2), which present Web forms. The forms allow you to enter some text and after submitting offers you to download a PDF file, which, when printed, allows you to fold a CD case from paper. Another alternative is this page.

This all works very well. I received some pretty “Wow”s after showing those CD cases to friends. However for the times, when I backup my growing MP3 collection to DVD I wanted a bit more influence, on what can be printed on the case. Therefore I duplicated the folding lines in a SVG file. Once loaded into Inkscape, you have all Inkscape tools available to style the CD case appropriately.

X.Org 1.5.x, evdev and HAL

The move to X.Org server 1.5.x was not quite the smooth ride I expected it to be. It took me the better part of a Saturday afternoon to get everything in working order again. Some time ago (actually already 2 month have gone by) I used the opportunity to do this, while fiddling with Ubuntu 8.10 on an externally connected USB drive. I compiled the X-server in a chroot environment on my standard os partition.

The first good thing I noticed after rebooting into my standard Gentoo environment was, that all of my 9 mouse buttons were correctly working. The bad news was, that each time I pressed the Cursor Up key, that the KDE screen snapshot utility would open. I tried multiple changes in the Input section of the /etc/X11/xorg.conf file, all to no avail. The first break through came, when I completely removed all sections from the xorg.conf file having something to do with the keyboard or the mouse. After that it was only additional configuration for the HAL subsystem, to report the correct keyboard type. For this I created the file /etc/hal/fdi/policy/10-x11-input.fdi with this content:

<?xml version="1.0" encoding="UTF-8"?>
<deviceinfo version="0.2">
  <device>
    <match key="info.capabilities" contains="input.keys">
      <merge key="input.xkb.model" type="string">evdev</merge>
      <merge key="input.xkb.layout" type="string">de</merge>
      <merge key="input.xkb.rules" type="string">evdev</merge>
    </match>
  </device>
</deviceinfo>

I think, the best way to diagnose any problems is to look at the /var/log/Xorg.0.log file. For my particular keyboard/mouse the output looks like this:

(II) config/hal: Adding input device HOLTEK Wireless Keyboard/Mouse(2.4G)
(**) HOLTEK Wireless Keyboard/Mouse(2.4G): always reports core events
(**) HOLTEK Wireless Keyboard/Mouse(2.4G): Device: "/dev/input/event2"
(II) HOLTEK Wireless Keyboard/Mouse(2.4G): Found keys
(II) HOLTEK Wireless Keyboard/Mouse(2.4G): Configuring as keyboard
(II) XINPUT: Adding extended input device "HOLTEK Wireless Keyboard/Mouse(2.4G)" (type: KEYBOARD)
(**) Option "xkb_rules" "evdev"
(**) HOLTEK Wireless Keyboard/Mouse(2.4G): xkb_rules: "evdev"
(**) Option "xkb_model" "evdev"
(**) HOLTEK Wireless Keyboard/Mouse(2.4G): xkb_model: "evdev"
(**) Option "xkb_layout" "de"
(**) HOLTEK Wireless Keyboard/Mouse(2.4G): xkb_layout: "de"

The xkb_* options are the important places here. Before I managed to set xkb_layout to “de” through HAL, the log file would report “us” here, which in turn had the affect of opening the snapshot dialogue when pressing Cursor Up. Another problem was the xkb_rules option. Some blog article I found through googling suggested to set xkb_rules to xorg. This does not work however, since xorg is a symbolic link in the /usr/share/X11/xkb/rules directory and links to base, which is not the correct rules set for the evdev-driver.

These three articles (part1, part2 and part3) from the Cybso blog were very helpful. In particular if you are using lineak. You should look at part 3 and install the lineakd keyboard file for the evdev-driver. These two links were helpful as well (1, 2

The Gentoo folks have created an update guide in the meantime, which wasn’t available, when I started out.