A knotty problem – an Arabic catalog that refused to work in Koha 17.11

Watching out for custom developed themes during Koha upgradation will save you a lot of grief.

Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth.

– Sherlock Holmes [by Arthor Conan Doyle]

Early last week my good friend Vimal Kumar Vazaphally asked me if he could connect me to a friend of his who was facing some trouble after migrating to Koha 17.11 from 3.18. I agreed to take a look, since the problems that Vimal sends my way from time to time are usually interesting rather than mundane. Soon I was talking to Hussain (‎حسين رضوي‎) who manages the systems at the Public Library of Imam Hussain (AS) Holy Shrine, at Bab Alqibla, Karbala, Iraq.

The problem

Hussain informed that things used to be fine in the older 3.18 system, but after migrating to 17.11, they could no longer use the cataloging editor properly as the value builder for 008 MARC21 tag i.e. Fixed-Length Data Elements-General Information was not being displayed properly anymore, and the data being pushed back was the default value.

The chase begins

After looking at his system over Teamviewer and not making much headway, I asked Hussain to send over a copy of his database and went to bed.

By next day he had sent over his 1.2 GB database. I had it hoisted up on a test 17.11.01 system we have around, the same one which we had used the week before to migrate Bangabasi College from their earlier Koha 3.14.06. Since it was a multi-tenanted setup I could see the staged Bangabasi catalog working OK in it. Once the zebra reindexing has been done, our sleuthing started, we were able to reproduce the error.

The steps to the Truth

The first thing to check was for MARC framework validation – Ok! that was clean. Next up – detailed manual check of the framework in use. But everything looked shipshape. So what we had here was a curious beast. Since we were on a multi-tenant system and one was working just fine, the problem therefore had to be coming up from somewhere in Hussain’s database. But from what exactly was the question. When I mentioned about it on #koha (the IRC channel of Koha ILS), @wizzyrea (Liz Rea from CatalystIT) said that the issue sounded familiar and asked me to check whether it was bug no. 19473. It was not. However, 19473 looked interested so I added myself to the bug for tracking its status in future, and as I did bugzilla prompted – “The next bug in your list is bug 19965”

Bug no. 19965 turned out to be Hussain’s bug which he had reported on the Koha bug tracking database on 12th Jan 2018. This was from before we had connected. As I read through the comments on the bug by @cait – Katrin Fischer our indomitable QA lead, and incidentally visiting India at the moment for LTC2018 in Goa, I had my light bulb moment! 😉 I ran the 008 value builder and checked the JavaScript console. Firefox gave some hint that there was something wrong, but it was Google Chrome that showed exactly what was wrong.

Apparently, we had a 404 (file not found) error for marc21_field_088.xml. Now this was the file that Koha uses to load the list of values possible in the 008 field. This is what drives the drop-downs in the 008 value builder. I knew that we certainly had the file on the system, which is why things had worked for Bangabasi. The error message showed that somehow Koha was looking for the marc21_field_088.xml at a different location – intranet-tmpl/imam/en/data/marc21_field_008.xml rather than where it should intranet-tmpl/prog/en/data/marc21_field_008.xml.

The story was clear – this Koha installation had a custom staff client theme / template installed with the name of imam. So now we checked the template system preference under “Staff client” of Administration -> System preferences. But no the system preference showed only prog as the solitary value. Knowing that rhe staff client system preferences values are principally driven via a YAML file inside Koha called staff_client.pref, I decided to go for the source, in other words – the database itself.

A simple query :

SELECT * FROM `systempreferences` WHERE variable = "template"

showed what I had suspected, the actual value was set to “imam“. Another line of SQL later

UPDATE `systempreferences` SET value = "prog" WHERE variable = "template"

I was ready to test it. And Voila! there was it was… all working again! 😀

An explanation and an update

Now, you may be wondering why did Koha show the system preference template as only “prog” while its actual value was set to “imam”… is that not a bug? Well, not really! The value in the database is supposed to be one from the possible values in the YAML code for that system preference. So when you actually add a different value into the database and do not update the value in the YAML script, this is the expected fallback behavior.

You may also wonder so what happened to the bug no 19965 that Hussain had filed in the Koha bugzilla. Since, I had been able to verify that this was not a Koha bug, rather a user infused data error, I closed the bug report with the status – “RESOLVED-INVALID” along with this comment.

For Hussain, the first level solution was to update his database and point the installation to use the “prog” staff client theme / template rather than “imam”. At the secondary level, he had to port his old 3.18 theme – “imam” over to 17.11, which is quite likely to be a non-trivial exercise, should he / the library authority choose to do that.