Koha and the “magic” of XSLT : displaying new MARC fields on the OPAC.

A short tutorial on using XSLT to make MARC fields in your data visible on the OPAC.

Earlier today, Suresh Kumar Tejomurtula, a member of the venerable LIS-Forum mailing list being run out of IISc Bangalore, posted a question on that list:

Under language codes of 008/35-37 and also under 041$a I added language code. for eg: tel for Telugu language. But I do not see any difference to the view of the record, except that the marc tags contains that values. Adding these fields data will enable library team to understand the language of the material. My question is, How will the users of the OPAC, who do not know about marc language codes will understand the item language [from just the 3-letter code sequence].

Short answer : Using XSLT

Much of Koha’s superpowers on the OPAC (as also on the staff client) side comes from its judicious use of XSLT. When we search for documents in Koha, the result that is returned from the database by way of the various perl modules that perform much of Koha’s internal plumbing, comes in as an XML (eXtensible Markup Language) document. More precisely it is returned as a MARCXML record. Readers of this blog who are familiar with the MarcEdit software may have often encountered a MARCXML record. Those who are not so familiar may well like to read up a bit from here before proceeding with this post.

So what is XSLT?

Wikipedia gives the definition of XSLT as –

“XSLT (Extensible Stylesheet Language Transformations) is a language for transforming XML documents into other XML documents, or other formats such as HTML for web pages, plain text or XSL Formatting Objects, which may subsequently be converted to other formats, such as PDF, PostScript and PNG.”

Where to start?

At the OPAC level, the XSLT magic is primarily driven by two files i.e. (a) MARC21slim2OPACResults.xsl and (b) MARC21slim2OPACDetail.xsl. Koha’s default settings of how we see the search results on the OPAC and the document specific details in the Normal view, are defined in these two files.

N.B. Directly editing these two files is strictly not advised unless you are a XSLT guru.

Lucky for us that Koha’s system preferences provide options to override the defaults by creating new XSLT files and telling Koha to use the new ones instead. The screenshot below shows the default setting.

2017-05-04_03-05-15

The two files are available inside a folder named xslt under the locale of the selected theme you are using e.g. /usr/share/koha/opac/htdocs/opac-tmpl/bootstrap/en/xslt/ on a package-based installation. By default most of us in India would be using the English locale i.e. “en”. The default Koha theme is “bootstrap”. Thus if you are using a custom theme and/or different locale look for the files under that.

Step #1 : Copying the XSLT files

We will be fetching the 3-letter language code from MARC 008/35-37 of each marc record. Since our Koha instance here is named as “demo”, we’ll make a copy of the file MARC21slim2OPACResults.xsl as MARC21slim2OPACResults-demo.xsl and the MARC21slim2OPACDetail.xsl as MARC21slim2OPACDetail-demo.xsl.

N.B. You can give any name to the copies you make, but it is suggested that the less adventurous you are with the names, easier it will be for you to trace your steps back if you make a mistake while editing them. And trust me you will make mistakes, at least initially.

Step #2 : Editing MARC21slim2OPACResults-demo.xsl

The MARC21slim2OPACResults.xsl is what drives the display of results of a search on the Koha OPAC. And it does not show the value of 008/35-37 i.e. MARC language code, while displaying the search result. To add this facility, we need to do the following to our copy i.e. MARC21slim2OPACResults-demo.xsl. Around line no. 70 or there about, there will be this line <xsl:variable name="controlField008-30-31" select="substring($controlField008,31,2)"/>. Add the following line after it – <xsl:variable name="controlField008_35-37" select="substring($controlField008,36,3)"/>.

Explanation: we’re defining a new variable named controlField008_35-37 and in it we are storing the 3-digit value found at 008/35-37 from another variable – controlField008.

The second step in editing this file is to add the code to check if (a) 008/35-37 actually exists (e.g. if you have a blank 008 field, 008/35-37 won’t exist) and (b) it is set as something other than “und(i.e. undefined). The following code does that first and proceeds to match the value found in 008/35-37 against 23 different languages. These languages are English and the 22 official languages of India as per the Eighth Schedule of the Indian Constitution. You can add or remove any language from this list as you require. However, when you do, please make sure that the codes you setup in this list here are coming from the Code Sequence document of the MARC Code list for languages as published by the US Library of Congress.

This means that you must also be using these same exact codes in your MARC records in Koha.

2017-05-04_05-40-55

Where we place this code will depend on where we want this language to listed in the results. In our case, we decided to add it between Material type and Format. And thus in our case, the code was added at line no 637. If you too want language to be displayed between the Material Type and Format, then look out for this line <xsl:if test="string-length(normalize-space($physicalDescription))"> and add this block before.

Saved the file and moved on to the next section.

Step #3 : Editing MARC21slim2OPACDetail-demo.xsl

The process is similar to what we just did above. Open the file MARC21slim2OPACDetail-demo.xsl for editing and go to line no. 51 (or there about) and look for this line <xsl:variable name="controlField008" select="marc:controlfield[@tag=008]"/>. Add the following line immediately after that: <xsl:variable name="controlField008_35-37" select="substring($controlField008,36,3)"/>.

2017-05-04_06-02-17

Since we wanted the “Language of document” to come after Material type, we added the code immediate after the node <xsl:if test="$DisplayOPACiconsXSLT!='0'"> is closed but before <!--Series: Alternate Graphic Representation (MARC 880) -->; in our case, this worked out to be around line no. 195. Finally, we saved and closed the file.

Getting Koha to use the new XSLT files

The Koha system preference OPACXSLTResultsDisplay was changed from its original setting (i.e. “default”) and set to the path of our new file MARC21slim2OPACResults-demo.xsl i.e. /usr/share/koha/opac/htdocs/opac-tmpl/bootstrap/en/xslt/MARC21slim2OPACResults-demo.xsl. Likewise the other system preference OPACXSLTDetailsDisplay was changed to /usr/share/koha/opac/htdocs/opac-tmpl/bootstrap/en/xslt/MARC21slim2OPACDetail-demo.xsl. It was time to test our XSLT modifications.

And it works!

Since pictures are said to be worth a 1000 words, we will let before and after screen grabs do the talking here. Also if anyone wants to see directly how it actually looks after applying the changed XSLT, visit the URL http://demo-opac.l2c2academy.co.in/cgi-bin/koha/opac-detail.pl?biblionumber=18

Before : Using the default MARC21slim2OPACResults.xsl

chrome_2017-05-04_07-05-24

After : Using the new MARC21slim2OPACResults-demo.xsl

chrome_2017-05-04_07-17-09

Before : Using the default MARC21slim2OPACDetail.xsl

chrome_2017-05-04_07-48-28

After : Using the new MARC21slim2OPACDetail-demo.xsl

chrome_2017-05-04_07-49-09

Koha quick tip : Duplicating a bibliographic record for a multi-volume book

Yesterday a young colleague called up with a query. She wanted to know how she could create separate bib records for a multiple volume book (e.g. “The Complete Works of Swami Vivekananda” which comes in 8 volumes) by editing the following volume specific info:

  1. ISBN;
  2. page count and physical description;
  3. price;
  4. date of copyright / publication;

while keeping other common information same across the records.

I told her that do this, she would first need to catalog at least *one* volume (preferably the first volume) of the book and then simply duplicating the record and changing the fields she needed to change in each case. This feature is well-documented in the Koha Manual under “Duplicating Records” inside the chapter on Cataloging. Continue reading “Koha quick tip : Duplicating a bibliographic record for a multi-volume book”

When “sex” prevents a Koha ILS installation

This is a true story from a recently concluded Koha workshop. There were roughly ~30+ participants and about 15 – 18 computers for their “hands-on” sessions. The instructor-led “how to install Koha 16.11 on Ubuntu 14.04.5” was going along smoothly. However, suddenly all hell broke loose! Everyone, including the instructor were stumped right at nearly the last leg of ‘sudo apt-get install koha-common‘. The install would not complete due to three missing files. First the repos were changed and re-tried. Well… no go! That’s when I noticed that the error was 403 forbidden. Woah! I tried to manually download the files off the repos using a browser and all it said was an “Access denied!” error coming from the Squid proxy.

A closer look at the files gave the game away. The university’s content filtering rules on the Squid proxy was the cause. Apparently the files: (a) libmoosex-markasmethods-perl (b) libmoosex-nonmoose-perl and (c) libwww-youtube-download-perl were being blocked from being fetched. The first two were triggering a false positive on the pornographic filter (the “sex” in libmoosex) and the third was hitting the keyword filter “youtube”, since Youtube was blocked on campus!!!

With no easy way to work around the proxy for connectivity, I manually downloaded the three files using a standby unfiltered link on a single computer. Next I transferred the files on to a pen drive and manually copied them over into each system’s /var/cache/apt/archives and thus fooling apt-get into thinking the files were already downloaded by it. And that is how we saved the day! 😉

Lesson

If you are planning a Koha installfest at an educational institution, please ensure that their content filtering proxy rules are tweaked before actually going ahead with the workshop.

Installing SMS::Send::IN::eSMS send driver for Koha ILS

An easy peasy HOWTO along with a few under-the-hood details explained.

Step #1 : Installing the SMS::Send::IN::eSMS module from CPAN

Login as a sudo user and start the CPAN shell using the command sudo perl -MCPAN -e shell. Install the driver by typing in install SMS::Send::IN::eSMS. If everything works OK, you should see output on screen similar to this:

cpan[1]> install SMS::Send:IN::eSMS
Reading '/root/.cpan/Metadata'
  Database was generated on Thu, 09 Feb 2017 09:41:03 GMT
Running install for module 'SMS::Send::IN::eSMS'
Checksum for /root/.cpan/sources/authors/id/I/IN/INDRADG/SMS-Send-IN-eSMS-0.01.tar.gz ok
Scanning cache /root/.cpan/build for sizes
............................................................................DONE
Configuring I/IN/INDRADG/SMS-Send-IN-eSMS-0.01.tar.gz with Makefile.PL
Checking if your kit is complete...
Looks good
Generating a Unix-style Makefile
Writing Makefile for SMS::Send::IN::eSMS
Writing MYMETA.yml and MYMETA.json
  INDRADG/SMS-Send-IN-eSMS-0.01.tar.gz
  /usr/bin/perl Makefile.PL INSTALLDIRS=site -- OK
Running make for I/IN/INDRADG/SMS-Send-IN-eSMS-0.01.tar.gz
cp lib/SMS/Send/IN/eSMS.pm blib/lib/SMS/Send/IN/eSMS.pm
Manifying blib/man3/SMS::Send::IN::eSMS.3pm
  INDRADG/SMS-Send-IN-eSMS-0.01.tar.gz
  /usr/bin/make -- OK
Running make test
PERL_DL_NONLAZY=1 PERL_USE_UNSAFE_INC=1 /usr/bin/perl "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
t/checkloginpass.t .. ok
t/checkmsgdest.t .... ok
t/compile.t ......... ok
t/liveaccttest.t .... skipped: No login information available, skipping all tests.
t/pod-coverage.t .... ok
t/pod.t ............. ok
All tests successful.
Files=6, Tests=16,  1 wallclock secs ( 0.09 usr  0.00 sys +  0.99 cusr  0.07 csys =  1.15 CPU)
Result: PASS
  INDRADG/SMS-Send-IN-eSMS-0.01.tar.gz
  /usr/bin/make test -- OK
Running make install
Installing /usr/local/share/perl/5.20.2/SMS/Send/IN/eSMS.pm
Installing /usr/local/man/man3/SMS::Send::IN::eSMS.3pm
Appending installation info to /usr/local/lib/x86_64-linux-gnu/perl/5.20.2/perllocal.pod
  INDRADG/SMS-Send-IN-eSMS-0.01.tar.gz
  /usr/bin/make install  -- OK

cpan[2]>

With the SMS::Send::IN::eSMS module installed, we’ll now set up the Koha side of things.

Step #2 : Making a one-line change to Koha’s SMS.pm

Assumption : That you are using a .deb package based installation. If you are using a git or a tarball installation you probably do not need this guide. 🙂

The code in file /usr/share/koha/lib/C4/SMS.pm is what Koha uses to communicate with the SMS provider (in this case eSMS Kerala from KSITM) via the send driver. This code is capable of accepting two parameters i.e. (a) username and (b) password of the user with the SMS service provider via whom the user wants to send out the messages. However due to Telecom Regulatory Authority of India (TRAI) rules, in India we need to use three parameters, the third one being a mandatory, 6-character, nationally unique senderid allotted to every bulk transactional SMS sender in India. As of version 16.11 Koha can not handle this AS-IS.

Therefore, to make SMS work with Koha in India, we need to make a one-line change to the code. Look for the following code around line no. 80 of the file /usr/share/koha/lib/C4/SMS.pm:

# Create a sender
$sender = SMS::Send->new( $driver,
                          _login    => C4::Context->preference('SMSSendUsername'),
                          _password => C4::Context->preference('SMSSendPassword'),
                    );

Add the following line : _senderid => C4::Context->preference('SMSSendSenderID'),, so that the code looks like this:

# Create a sender
$sender = SMS::Send->new( $driver,
                          _login    => C4::Context->preference('SMSSendUsername'),
                          _password => C4::Context->preference('SMSSendPassword'),
                          _senderid => C4::Context->preference('SMSSendSenderID'),
                    );

Save and close the file. Now login into the staff client and add the four necessary system preferences – (a) SMSSendDriver, (b) SMSSendUsername, (c) SMSSendPassword and (d) SMSSendSenderID. Now, if you are on Koha 3.22 or higher version (i.e. 16.05 or 16.11), you will find the first three under Patron preferences and you only need to create the last one i.e. SMSSendSenderID to complete the setup.

eSMS_02

eSMS_03

However, if you are still using Koha 3.18.x or 3.20.x series then you will need to create three Local use preferences i.e. SMSSendUsername, SMSSendPassword and SMSSendSenderID. Either way, set these four parameters to your actual settings and you are ready to go start using the installed driver.

N.B.This behaviour will change from Koha 17.05 onward (estimated release date May 2017) when the bug number 13029 – “Allow to pass additional parameters to SMS::Send drivers” becomes generally available as part of the stable releases.

Troubleshooting #1 – Missing ‘make’

‘Make’ is required to configure and install the driver. So, if you see an error that reads as

Running make for I/IN/INDRADG/SMS-Send-IN-eSMS-0.01.tar.gz
Can't exec "/usr/bin/make": No such file or directory at /usr/local/share/perl/5                                                                                                             .20.2/CPAN/Distribution.pm line 2197.
  INDRADG/SMS-Send-IN-eSMS-0.01.tar.gz
  /usr/bin/make -- NOT OK
  No such file or directory
Failed during this command:
 INDRADG/SMS-Send-IN-eSMS-0.01.tar.gz         : make NO

Exit from cpan by using the command exit and from the command line install make by entering the command sudo apt-get install make. Once make is installed, run sudo perl -MCPAN -e shell and then proceed with driver installation using the command install SMS::Send::IN::eSMS.

Troubleshooting #2 – CPAN index is out of date

After typing in the command ‘install SMS::Send::IN::eSMS‘ if you find yourself facing the error Warning: Cannot install SMS::Send::IN::eSMS, don’t know what it is., DO NOT panic! It happened to us on one of our test servers too 🙂

It simply means your CPAN index metadata is out-of-date, and you need to refresh the index by running the command ‘reload index‘. After the updated index is fetched, re-run the command install SMS::Send::IN::eSMS. It will work this time.

Adding autocomplete support to MARC21 260 / 264 (imprint) fields in Koha

Adding auto-complete feature to Koha’s MARC21 260 / 264 field (Imprint – place, name of publisher, distributor etc) as a cataloging aid.

As per LC’s AACR2 (as well as RDA) instructions, the imprint information as captured in MARC21 field 260 (AACR2) and 264 (RDA) should be *transcribed* and not *recorded* using the principle – “Take What You See and Accept What You Get” [1]. As a result, the 260$a (place) and 260$b (publisher, distributor etc) are usually not handled as fields whose values are controlled using authorized values.

Up until 2001, the 260 field was a NR field. It was made repeatable to accomodate frequent publisher name changes.

Since the 260$a and 260$b are not usually guided by authorized values, it is noted (at least in the Indian sub continent context) that catalogers often make typographical errors while transcribing the data, e.g. “Pearson Education” may be inadvertantly added as “Peerson Education” or as even as “Pearshon education”, while “Kolkata” may have been entered both as “Kolkata” as well as “Kolkatta” or even as “Kolkhata”. Without authority control of the field, this cataloging quality check is often overlooked. Errors like this often end up affecting the result of advanced searches or custom SQL reports.

Luckily for us, Koha ILS uses jquery extensively while (via jquery-ui) provides for nice autocomplete widgets, like the ones we see in action when we type in part of the borrower’s name in patron search (checkout) or in the authoritiy headings search, where entering 3 characters triggers the AJAX based lookup with the option to select one of the offered list *OR* to type in our own.

AJAX stands for “Asynchronous Javascript and XML”. In simple terms it encompasses a set of web development technique that allows us to fetch and load data from a remote server into our currently open page, without requiring us to refresh / reload the page. [2]

Recently a client requested that we offer them a way to look up publisher names and place names (for field 260) already entered into their Koha instance, without having to type it all in every time. For example in their database they already had the following publisher names entered – “Pearson Education, Prentice Hall India, PacktPub, Press Trust of India” etc. Now they if they encountered an item that was from these publishers they should be able to pull up a list just by entering “P” into the 260$b field and then be able to select the one applicable. And if they encountered a publisher name say “Penguin Books”, they should be able to type it in as well.

Koha 16.11 ships with 3 (three) different ysearch.pl scripts that show us how to achieve this. You can find out which ones these are by using the command `locate ysearch.pl`. NOTE: You may be required to run `sudo updatedb` once before locate finds the files. For our requirement we modeled our script which we’ll call 260search.pl on /usr/share/koha/intranet/cgi-bin/cataloguing/ysearch.pl. You can grab a copy of 260search.pl from L2C2’s github repo here [3]. The script returns a JSON based result set if results matching your input is found.

Remember that **every** script that Koha executes, needs it executable bit set, and so does this one. Therefore, do *not* forget to set the executable bit for the script with `sudo chmod a+x /usr/share/koha/intranet/cgi-bin/cataloguing/260search.pl` before you proceed to the next step.

Step #2 : Enabling the fields

With the script in place, we now need to turn to IntranetUserJS system preference and enter the following jquery snippet to enable autocomplete in 260$a and 260$b :

$(document).ready(function(){
  $( '[id^="tag_260_subfield_a"]' ).autocomplete({
    source: function(request, response) {
      $.ajax({
        url: "/cgi-bin/koha/cataloguing/260search.pl",
        dataType: "json",
        data: {
          term: request.term,
          table: "biblioitems",
          field: "place"
        },
        success: function(data) {
          response( $.map( data, function( item ) {
            return {
              label: item.fieldvalue,
              value: item.fieldvalue
            }
          }));
        }
      });
    },
    minLength: 1,
  });
  $( '[id^="tag_260_subfield_b"]' ).autocomplete({
    source: function(request, response) {
      $.ajax({
        url: "/cgi-bin/koha/cataloguing/260search.pl",
        dataType: "json",
        data: {
          term: request.term,
          table: "biblioitems",
          field: "publishercode"
        },
        success: function(data) {
          response( $.map( data, function( item ) {
            return {
              label: item.fieldvalue,
              value: item.fieldvalue
            }
          }));
        }
      });
    },
    minLength: 1,
  });
});

A video of autocomplete in action

Conclusion

By tweaking the 260search.pl script or even by completely re-writing it to use the various search functions shipped by Koha inside its /usr/share/koha/lib directory on a .deb package based installation, you can do so much more than possible with this simple hack. Happy hacking! 🙂 [4]

References:

[1] http://www.loc.gov/catworkshop/RDA%20training%20materials/LC%20RDA%20Training/Module1IntroManifestItemsSept12.doc

[2] https://developer.mozilla.org/en-US/docs/AJAX/Getting_Started

[3] https://gist.github.com/l2c2technologies/7d0449dcb80c90880381ef4571003d1d

[4] http://catb.org/jargon/html/H/hack.html

JQuery tips for Koha : Adding easy to use indicator picklists

Adding picklists for selecting indicators for MARC tags used in Koha’s cataloging worksheets.

During data audits of users’ MARC21 data, quite frequently we find that most, if not all, records are often without any form of use of indicators. Trained library professionals often give a sheepish grin when asked why they didn’t add them while cataloging the documents. 😉 But trained librarians are not the only ones who work with Library systems like Koha. There are many people who find themselves working in a library without a formal training or sufficient theoretical background on MARC21. Generally speaking reasons for not adding the indicators range from:

  • Lack of practise – thus unsure of the correct indicator to use.
  • Lack of awareness – i.e. untrained people with a very basic knowledge of cataloging
  • Lack of user-friendly mechanism to input indicators
  • And lastly – sheer laziness

Now, about the last one we can’t do anything about, however the rest of the reasons might use a bit of leg-up! So here goes the newest tutorial on how to add easy-to-use picklists to help us correctly populate the indicators.

According to the Design Principles of MARC21, indicators form a part of the family of content designators [1]. As defined, an indicator is :

A data element associated with a data field that supplies additional information about the field. An indicator may be any ASCII lowercase alphabetic, numeric, or blank.

For this tutorial we will focus on MARC21 bibliographic data fields 100 and 110 i.e. Main Entry Personal Name and Main Entry Corporate Name respectively. We will not touch the Koha template files at all, rather as per the global best practice for Koha ILS, we will utilize only JQuery (JavaScript) and HTML via the Koha system preference IntranetUserJS.

Step #1 – Finding out the DOM nodes

We will start by going to Home > Cataloging > Add MARC record in Koha and select the framework we want to work on. In this case we chose to work with the “Default framework” that is shipped with Koha. We used Google Chrome’s Developer Tools Inspect option [2] to find out what is the id of the selector (DOM node) we need for Main Entry Personal Name.

Since we need space to setup the picklist we chose to use the free space available on the div that displays information about the field that follows immediate after it. As you can see in the image below that div has an id identifying it, which is very good for us, since it makes selecting the DOM node absolutely painless.

blog_01

It should noted that when Koha renders the cataloging interface, it suffixes the HTML element IDs with a random number (one for each new tag). In this case, the id was div_indicator_tag_100_838390 where “838390” is the random suffix number. We needed to latch on to the first part i.e. div_indicator_tag_100.

Step #2 – Let the JQuery magic work

We have to add the select dropdown picklists right after the text on the div_indicator_tag_XXX DIVs. The value we will use for the indicators will come from here and here respectively.

$(document).ready(function(){
if ( $("#cat_addbiblio") ) {	// only while adding biblios
  $('div[id^="div_indicator_tag_100"]').append(' <label for="tag_100_indicators">Apply Ind1, Ind2</label> <select id="tag_100_indicators"><option>-Select-</option><option value="1">1 - Surname</option><option value="0">0 - Forename</option><option value="3">3 - Family name</option></select>');
  $('div[id^="div_indicator_tag_110"]').append(' <label for="tag_110_indicators">Apply Ind1, Ind2</label> <select id="tag_110_indicators"><option>-Select-</option><option value="2">2 - Name in direct order</option><option value="0">0 - Inverted name</option><option value="1">1 - Jurisdiction name</option></select>'); };   // end if
});
});

blog_02

While that added the picklists, we still have to add the actual logic that will allow the indicators to be populated on selecting from the list. Again we will turn to JQuery for the following snippet:

$(document).ready(function(){
  $('#tag_100_indicators').click(function(){
    var what_clicked_100 = $('#tag_100_indicators').val();
    if ( !isNaN(what_clicked_100) ) {
      $('input[name^="tag_100_indicator1"]').val(what_clicked_100);
      $('input[name^="tag_100_indicator2"]').val("#");
    } else {
      $('input[name^="tag_100_indicator1"]').val("");
      $('input[name^="tag_100_indicator2"]').val("");
    }
  });
  $('#tag_110_indicators').click(function(){
    var what_clicked_110 = $('#tag_110_indicators').val();
    if ( !isNaN(what_clicked_110) ) {
      $('input[name^="tag_110_indicator1"]').val(what_clicked_110);
      $('input[name^="tag_110_indicator2"]').val("#");
    } else {
      $('input[name^="tag_110_indicator1"]').val("");
      $('input[name^="tag_110_indicator2"]').val("");
    }
  });
});

The code above is listening to see when we click and select a value from the picklists i.e. when we trigger a click JavaScript event. Next it checks if we had selected a real value OR whether we had just “clicked” on the placeholder “-Select-” option that no value. And lastly based on what we had selected it sets the ind1 and ind2 values according.

blog_04

Conclusion

In this manner we can add easy-to-use picklists for indicators. Since it is now only a matter of selecting from the available values, it also reduces significantly the scope for typographical errors during data entry into the indicator boxes. Before we leave for today, do note that the second code listing may be better handled as a JavaScript function to which the references are passed to by a handler hook. Doing so would make for a cleaner and leaner implementation of this concept especially if you are planning to set it up for all the non control MARC21 fields you use. Also, you may wish to implement the selected dropdown value check using something other than IsNan [3].

References

[1] “MARC 21 Specifications for Record Structure, Character Sets, and Exchange Media – RECORD STRUCTURE (2000)” https://www.loc.gov/marc/specifications/specrecstruc.html

[2] “Chrome DevTools” – https://developers.google.com/web/tools/chrome-devtools/

[3] isNaN() – https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/isNaN

Koha Quick tip: Removing the dot from salutation like “Ms.” after a bulk patron import

A quick-n-dirty way to remove the trailing dot from salutation field after a bulk patron import into Koha.

Earlier today the deputy librarian at our client partner Gouri Devi Institute of Medical Sciences and Hospital (GIMSH) called us wanting to figure out how to fix a small, but crucial problem. The employee list from GIMSH’s HR department was used to create the patron bulk import CSV [1] file. All 143 records got imported without a glitch and each individual records looked perfectly OK when viewing them. However, when she tried to edit these patron records, she noticed that the salutation (e.g. Mr, Mrs, Ms etc.) would not show up in the salutation drop-down during edit! 🙁

missing salutation
Missing salutation even though salutation was entered into the database

Explanation

About 141 out of the total 143 records had a period.” after the salutation and two didn’t. These two records which did not have the period at the end of salutation were displaying the stored value correctly in the drop-down, rather than showing as being blank. As it happens, Koha uses the BorrowersTitles system preference to store the options to display. The default options are Mr|Mrs|Miss|Ms. As you can see, none of these have a period at their end. So, while the salutation field value was stored in GIMSH’s borrowers table, it was not being displayed during the edit as the value didn’t match with the values stored in BorrowersTitles syspref.

The “Fix”

***WARNING*** The following step requires you to directly execute a SQL on the Koha database. This is usually not recommended, but some times, it is required to quickly fix a problem.

Using the TRIM() string function of MySQL in tandem with the specifier trailing [2] we can take out the “.(period) at the end of the salutation from all the records in the borrowers table in the following manner:

UPDATE `borrowers` SET `title` = TRIM(TRAILING '.' FROM `title`);
salutation drop down is working now
salutation drop down is showing the stored value correctly now

References

[1] See the entry Comma separated values from Wikipedia.

[2] String Functions in MySQL 5.5 – http://dev.mysql.com/doc/refman/5.5/en/string-functions.html#function_trim

[3] Sample patron data file – patron_import.csv NB. Those of you who wish to study how the HR data was prepared for import into Koha, can look here for a sample CSV file with dummy data (do not worry about privacy as the data is fake).

Adding randomized barcodes to Koha’s holding (item) records (in Bengali)

Last Saturday, while conducting the online training for Parama Sarkhel di (Librarian, Ramakrishna Sarada Mission Vivekananda Vidyabhavan) and her team members on how to add randomized barcodes to their holding (items) records i.e. 952, we recorded their remote Teamviewer session. The voice over is in Bengali, though one can easily follow the screen to understand what is happening here. Their OPAC is at http://rksmvvlibrary.in/

Released under CC 4.0 – Attribution – ShareAlike by L2C2 Technologies

Changing languages in the drop-down for Google Transliterate API

Choose the list of languages to transliterate on the Koha OPAC

This tends to be frequent question that we often end up fielding from colleagues and customers. But before we get into this, please remember that this feature is probably at the end of it shelf-life. There are two reasons for this – the Transliterate API that this feature depends on was deprecated many, many years back, and secondly how badly this API was put together – it fetches the necessary JavaScript library from Google’s servers over HTTP! So, if you are using HTTPS on your Koha OPAC, you can kiss this feature good bye right now! Being deprecated there is almost zero chance that Big G will do anything to change that.

Life sucks right?? 😉 But as long as it is still there, here’s “one for the road”! 😀 So, read on!

Turning on Google Transliteration

googleindic

It all starts with setting GoogleIndicTransliteration system preference to “Show” from the default of “Don’t show”.

By default that provides us with this short language drop-down:
googlist

For users from India, a country with 22 official language at the last count, that default list is woefully too little, too less. The good news is that Google’s Transliteration API supports several dozens of languages. Which ones? Well you need to dig around LanguageCode and SupportedDestinationLanguages enumerations in the documentation.

Show me the money!

The code to support the function is loaded via googleindictransliteration.js file that resides under /usr/share/koha/opac/htdocs/opac-tmpl/bootstrap/js/ on a Debian package based installation.

The screen grab below shows exactly that changes we needed to make in order to show on Hindi and Bengali as the list of languages, *and* with Bengali as default language option on the select drop-down.

googchanger

So how did I know that I needed to use “bn” for Bengali? Well, that LanguageCode enum list I’ve linked to above. Simple really!

One more fact about this tip that will bug a few

Change(s) to the LanguagesCode enum like this within the googleindictransliteration.js file would cause *all* instances on a multi-tenanted Koha installation to display the same list of languages. So, if you need different sets of languages lists in different OPACs, you are probably out of luck.

Customizing Koha’s MARC21 frameworks? Know the rules or get help!

Either you know what you are doing or take time to learn or invest in quality support. Fail on all three counts and you are quite literally asking for an operational nightmare.

Recently a young colleague Sri Ashkar K. from Thiruvananthapuram, Kerala (India) ran into a problem. He works as a librarian with Mathrubhumi, a major media house from Kerala. Specifically he needed to have a LMS solution to efficiently manage their collection of entertainment (mostly movies) related CDs and DVDs. For him the LMS was hosted Koha. However when he tried to issue an item i.e. a movie CD, he was stumped by this error every time:

Software error
No branchcode argument passed to koha::Calender->new at /var/koha_all/mathrubhumi/lib/C4/Circulation.pm line 3558

Being on a hosted Koha platform, he approached his service provider for support. He shared with them all the relevant screenshots leading to error detailed above.

The provider’s tech support could not identify the issue and instead informed him that they could perform checkouts (issue) without any errors. As Ashkar persisted, the service provider’s support desk asked him to provide remote desktop sharing using Teamviewer so that they could see “his problem” in action. Installing Teamviewer needed clearance from his IT department which required time and thus Ashkar’s checkout problem continued to linger. Finally about 10 days back he posted about it on the official Facebook page of Koha Library System Project, asking for suggestions to resolve it.

The first flag was raised by fellow Koha dev Mark Tompsett when he asked:

“/var/koha_all/mathrubhumi/lib/C4/Circulation.pm” — That is not a standard installation path. How did you install this? And what version?

Ashkar replied that since the software was hosted, he did not know the installation details. This got my attention! If he was on hosted Koha, why was he turning to the community for support? What was his service provider doing in the first place? I decided to find out more. That’s when I discovered the details of his situation. Desperate for help, he provided me with superlibrarian access to his hosted Koha’s staff client interface. I logged in and found that the problem was very real. In fact, I found out a few rather *disturbing* things.

The hosted Mathrubhumi Koha instance wasn’t running on the stable version (which is 16.05.05 at the time of writing) of Koha ILS. In fact, it was running on an unstable development version (at the time of this writing it was using Koha 16.0600023). Development versions are not GA releases and are *never* meant for production use, they are meant for use by testers and developers. And secondly, I could not do a MARC21 export for his bibliographic data.

That set alarm bells ringing in my head and so with Ashkar’s approval, I created a backup of his Koha database and installed the backup on L2C2’s test server running the latest stable 16.05.x version.

The first clear indication of what was wrong came soon after running sudo koha-rebuild-zebra -v -f mathrubhumi successfully without any error. A wildcard search from both OPAC as well as the staff client failed to return a single result, even though the Zebra indexer and output logs showed no error. However, it was possible to access a record by directly accessing it by biblionumber.

Running the “MARC Bibliographic framework test” to check the MARC structure provided the answer. Sure enough there were two major errors as shown below:

homebranch NOT mapped the items.homebranch field MUST :

  • be mapped to a MARC subfield,
  • the corresponding subfield MUST have “Authorized value” set to “branches”
holdingbranch NOT mapped the items.holdingbranch field MUST :

  • be mapped to a MARC subfield,
  • the corresponding subfield MUST have “Authorized value” set to “branches”

The question now was to identify *which* MARC21 framework since he had three (03) of them.

marcframe1
Ashkar’s MARC21 frameworks

Checking the “MOVIES” framework, it was found that both 952 $a (homebranch) and $b (holdingbranch) were set to ignore in the Managed in tab dropdown. This explained the error displayed by “MARC Bibliographic framework test”. To know more about the the 952 MARC21 field in Koha, please read Holdings data fields (9xx) from the Koha community wiki.

The Fix

It was a simple matter of setting both 952 $a and $b to “items(10)” for the option Managed in tab. This took care of the “MARC Bibliographic framework test” error.

However, that was only the first part of the solution. Except two, none of his other 23 bibliographic records had their homebranch and holdingbranch defined. It was time for a batch item modification from the Tools page (Home > Tools). This has been covered in details in an earlier blog post – “Koha’s MARC modification templates comes to the rescue“, so if the topic sounds unfamiliar, it is suggested that you read that post first.

In order to find out all the barcodes that needed to be used to update the records, the following SQL report was used:

SELECT items.barcode 
  FROM items 
  LEFT JOIN biblioitems ON (items.biblioitemnumber=biblioitems.biblioitemnumber) 
  LEFT JOIN biblio ON (biblioitems.biblionumber=biblio.biblionumber);

With the list of barcodes in hand, it was time for the final steps:

  1. Load up barcodes for the records to be bulk modified
    marcframe4
  2. Select the two fields that we wanted to update – homebranch and holdingbranch
    marcframe2
  3. Select the actual branch option for both and click on Save
    marcframe3
And we were done! 🙂

The Explanation

Understanding the error is quite simple if you know how circulation works inside Koha. A checkout operation needs to know a few basic things – (a) who owns the item; (b) where is the item presently located; (c) what to set as the issue and due dates and (d) who is taking it. Since the items attached to bibliographic records created using the MOVIES MARC21 framework did not have their homebranch and holdingbranch defined, at the time of checkout, as Koha tried to set the issue date and calculate the due date, using the date functions from the Koha::Calender object, it failed to do so. That’s what gave Ashkar his error and prevented him from checking out an item.

This still left one question unanswered – why did Ashkar’s hosting provider keep insisting that everything was working OK at their end and wanted him to provide them with Teamviewer access instead. My best guess is they were checking out the system using only the MARC21 frameworks which *they* had shipped i.e. default and fast add (FA) frameworks. Since records generated using these two frameworks (quite correctly) had 952 $a and $b set, none of these triggered Ashkar’s error during checkout. They certainly did not need Teamviewer access, the error in Ashkar’s framework should have been easily detected and quickly fixed. In fact, it took less than 3 mins to take care of it. But they failed, which is why it is important to either invest in your own skill development (read RTFM) OR invest in quality support.

“If you pay peanuts, you get monkeys” – James Goldsmith

Moral of the story: If you work with service providers whose front line tech-support is staffed with inexperienced people, be prepared for the long haul and self support yourself. Caveat Emptor!