Customizing Koha’s MARC21 frameworks? Know the rules or get help!

Either you know what you are doing or take time to learn or invest in quality support. Fail on all three counts and you are quite literally asking for an operational nightmare.

Recently a young colleague Sri Ashkar K. from Thiruvananthapuram, Kerala (India) ran into a problem. He works as a librarian with Mathrubhumi, a major media house from Kerala. Specifically he needed to have a LMS solution to efficiently manage their collection of entertainment (mostly movies) related CDs and DVDs. For him the LMS was hosted Koha. However when he tried to issue an item i.e. a movie CD, he was stumped by this error every time:

Software error
No branchcode argument passed to koha::Calender->new at /var/koha_all/mathrubhumi/lib/C4/Circulation.pm line 3558

Being on a hosted Koha platform, he approached his service provider for support. He shared with them all the relevant screenshots leading to error detailed above.

The provider’s tech support could not identify the issue and instead informed him that they could perform checkouts (issue) without any errors. As Ashkar persisted, the service provider’s support desk asked him to provide remote desktop sharing using Teamviewer so that they could see “his problem” in action. Installing Teamviewer needed clearance from his IT department which required time and thus Ashkar’s checkout problem continued to linger. Finally about 10 days back he posted about it on the official Facebook page of Koha Library System Project, asking for suggestions to resolve it.

The first flag was raised by fellow Koha dev Mark Tompsett when he asked:

“/var/koha_all/mathrubhumi/lib/C4/Circulation.pm” — That is not a standard installation path. How did you install this? And what version?

Ashkar replied that since the software was hosted, he did not know the installation details. This got my attention! If he was on hosted Koha, why was he turning to the community for support? What was his service provider doing in the first place? I decided to find out more. That’s when I discovered the details of his situation. Desperate for help, he provided me with superlibrarian access to his hosted Koha’s staff client interface. I logged in and found that the problem was very real. In fact, I found out a few rather *disturbing* things.

The hosted Mathrubhumi Koha instance wasn’t running on the stable version (which is 16.05.05 at the time of writing) of Koha ILS. In fact, it was running on an unstable development version (at the time of this writing it was using Koha 16.0600023). Development versions are not GA releases and are *never* meant for production use, they are meant for use by testers and developers. And secondly, I could not do a MARC21 export for his bibliographic data.

That set alarm bells ringing in my head and so with Ashkar’s approval, I created a backup of his Koha database and installed the backup on L2C2’s test server running the latest stable 16.05.x version.

The first clear indication of what was wrong came soon after running sudo koha-rebuild-zebra -v -f mathrubhumi successfully without any error. A wildcard search from both OPAC as well as the staff client failed to return a single result, even though the Zebra indexer and output logs showed no error. However, it was possible to access a record by directly accessing it by biblionumber.

Running the “MARC Bibliographic framework test” to check the MARC structure provided the answer. Sure enough there were two major errors as shown below:

homebranch NOT mapped the items.homebranch field MUST :

  • be mapped to a MARC subfield,
  • the corresponding subfield MUST have “Authorized value” set to “branches”
holdingbranch NOT mapped the items.holdingbranch field MUST :

  • be mapped to a MARC subfield,
  • the corresponding subfield MUST have “Authorized value” set to “branches”

The question now was to identify *which* MARC21 framework since he had three (03) of them.

marcframe1
Ashkar’s MARC21 frameworks

Checking the “MOVIES” framework, it was found that both 952 $a (homebranch) and $b (holdingbranch) were set to ignore in the Managed in tab dropdown. This explained the error displayed by “MARC Bibliographic framework test”. To know more about the the 952 MARC21 field in Koha, please read Holdings data fields (9xx) from the Koha community wiki.

The Fix

It was a simple matter of setting both 952 $a and $b to “items(10)” for the option Managed in tab. This took care of the “MARC Bibliographic framework test” error.

However, that was only the first part of the solution. Except two, none of his other 23 bibliographic records had their homebranch and holdingbranch defined. It was time for a batch item modification from the Tools page (Home > Tools). This has been covered in details in an earlier blog post – “Koha’s MARC modification templates comes to the rescue“, so if the topic sounds unfamiliar, it is suggested that you read that post first.

In order to find out all the barcodes that needed to be used to update the records, the following SQL report was used:

SELECT items.barcode 
  FROM items 
  LEFT JOIN biblioitems ON (items.biblioitemnumber=biblioitems.biblioitemnumber) 
  LEFT JOIN biblio ON (biblioitems.biblionumber=biblio.biblionumber);

With the list of barcodes in hand, it was time for the final steps:

  1. Load up barcodes for the records to be bulk modified
    marcframe4
  2. Select the two fields that we wanted to update – homebranch and holdingbranch
    marcframe2
  3. Select the actual branch option for both and click on Save
    marcframe3
And we were done! 🙂

The Explanation

Understanding the error is quite simple if you know how circulation works inside Koha. A checkout operation needs to know a few basic things – (a) who owns the item; (b) where is the item presently located; (c) what to set as the issue and due dates and (d) who is taking it. Since the items attached to bibliographic records created using the MOVIES MARC21 framework did not have their homebranch and holdingbranch defined, at the time of checkout, as Koha tried to set the issue date and calculate the due date, using the date functions from the Koha::Calender object, it failed to do so. That’s what gave Ashkar his error and prevented him from checking out an item.

This still left one question unanswered – why did Ashkar’s hosting provider keep insisting that everything was working OK at their end and wanted him to provide them with Teamviewer access instead. My best guess is they were checking out the system using only the MARC21 frameworks which *they* had shipped i.e. default and fast add (FA) frameworks. Since records generated using these two frameworks (quite correctly) had 952 $a and $b set, none of these triggered Ashkar’s error during checkout. They certainly did not need Teamviewer access, the error in Ashkar’s framework should have been easily detected and quickly fixed. In fact, it took less than 3 mins to take care of it. But they failed, which is why it is important to either invest in your own skill development (read RTFM) OR invest in quality support.

“If you pay peanuts, you get monkeys” – James Goldsmith

Moral of the story: If you work with service providers whose front line tech-support is staffed with inexperienced people, be prepared for the long haul and self support yourself. Caveat Emptor!

Planning to bulk import your patrons? Make sure you do not have in-line line breaks in your data.

In-line line breaks in a CSV file can really send your Koha patron import script into a tailspin. Here is what you need to watch out for and the couple of other gotchas which will make you upgrade your Koha instance if the version you are using is less than 3.22.7.

Last week a friend working at a local college approached me for a spot of help. He was trying to import his patrons into Koha but was failing miserably. After he nearly got his head snapped off (Me: Do I look like I’m in the fortune telling profession???) he agreed to send over his data – an MS-Excel sheet for me to take a look at.

I pulled up a 3.22.6 instance I had laying around and tried to import his data. Quite expectedly, there were errors galore and the pretty much the same ones he was complaining about.

blog_patron_1

Hang on! categorycode, branchcode and surname fields were NOT missing in *any* single record. So what was going on here??? The most interesting to note here is that patron importer script said :

272 not imported because they are not in the expected format

272 records parsed

Now this was really something as the total number of student records in that patron uploader CSV file were only 144. So where does the number 272 come from?

The answer to this was easy to find. My friend’s data had several records in a rather bad shape – they had embedded line-breaks within the cell. I’ve highlighted the first few of the badly formatted cells with yellow in the screenshot below.

blog_patron_3A

So, I copied the first 28 records over to a new file, ran a hackish utility script to clean out the line breaks and saved these 28 records as a new file and proceeded to upload it. This time of course “the fat lady sang”[1] i.e. the records got imported nice and we were done!. 😀

blog_patron_3NOTE:  Of course while doing that we encountered a few Koha bugs as well – Bug id 15840 and Bug id 16426. The work-around mentioned in comment #16 of the latter bug, by Koha QA Manager Katrin Fischer holds good, in case you get stuck here and can’t immediately upgrade. Otherwise to avoid these to bugs, your real option is to upgrade your Koha instance, something that I’m going to recommend to my friend (aside from him fixing his data).

Reference: [1] Wikipedia “It ain’t over till the fat lady sings”

 

The Koha manual is your friend :-)

The Koha User Manual is an amazingly in-depth document that keeps getting better with each iteration, thanks to Koha Library System Project’s indefatigable Documentation Manager Nicole C. Engard. This post exemplifies a situation where it can come in handy.

Yesterday a librarian colleague reached out with a query. He is on Koha 3.20.x series and since the last 3 – 4 days, he had been noticing that the “due date” for items his staff was issuing out was coming out rather odd! For instance all the due dates were set to “18th June 2016“. He clarified that he did not have a hard due date set, so what is going on???

This morning he sent over a screen shot of the checkout operation. And boy! the date now read 25th-June-2016. This was an immediate clue! On checking out his calendar – 18th and 25th turned out to be the only two days that are marked as “library is open” in June 2016 (for issues made in these last two weeks i.e.  before the holidays begin) during their summer recess.

That holiday setting working together with the system preference “useDaysMode” set to “the calendar to push the due date to the next open day” for circulations was the reason why Koha was pushing the due date beyond the usual designated due dates for each patron category.  Basically, it was just Koha doing its job (as it was asked to do) and doing it well.

If you are on 3.20, remember to read up http://manual.koha-community.org/3.20/en/calholidays.html, the relevant system preference are all explained very nicely there by Nicole C. Engard, Koha’s indefatigable Documentation Manager.