The 5-Minute Series: Adding Lightbox support to your Koha OPAC

Ever wanted to add a Lightbox enabled custom gallery page to your Koha ILS installation? Here’s how you can do it, with a bit of background and a brief touch on caveats and afterthoughts of implementing it.

Recently one of our client partners, Ms. Parama Sarkhel, Librarian, Ramakrishna Sarada Mission Vivekananda Vidyabhavan, requested us to add an image gallery with photos of the library and library events to her OPAC. We promised her that we’ll to look into it.

gallery_01

While we had setup image galleries for others, this time we decided to do something different. As any seasoned Koha user will know, you can add custom pages to your Koha OPAC. Adding a gallery typically means adding a new custom page with a bunch of photos and the nessecary CSS styles along with (usually) some Javascript to handle it. In fact, if you are not too bothered about supporting older browsers you can set up a gallery using *only* CSS3 alone.

NOTE 1: If you are a new user of Koha, you may wish read up Appendix H. Using Koha as a Content Management System (CMS) from the Koha 16.05 Manual (en). Be advised that the instructions for Koha as a CMS are slightly out of sync, so you need to change where it says “pages.tmpl” to “pages.tt” to be able to actually use these instructions.

NOTE 2: Prior to release of Koha v 3.4.3 [1] on July 25, 2011, Koha used the HTML::Template perl module for handling its template requirements. Via Bug id #5917 [2], Koha 3.4.3 switched over to Template::Toolkit templating system [3]. The reference to “pages.tmpl” is a relic from Koha’s HTML::Template past.

Introducing Lightbox2

To an average Web UI designer / programmer, Lightbox usually needs no introduction. However since the target audience of this blog includes people who do not neccesarily program, either for passion or a living, let us introduce the main “protagonist” of our post here i.e. Lightbox! Lightbox2 is a successor to the the original lightbox [4] script by Lokesh Dhakar. As described by it’s author Lokesh :

“Lightbox is small javascript library used to overlay images on top of the current page. It’s a snap to setup and works on all modern browsers.”

Users who are new to the term lightbox (you have probably used it online, without knowing that it is called a ‘lightbox’) can have a look at [4] below in the reference sub-section.

Making Lightbox2 work with Koha

Before we get out hands dirty and share with you what we did, here are a few things that we must keep in mind.

First the things that one must know

1. As on date the Koha OPAC uses an older version of Jquery javascript library i.e. version 1.7.2 (released [5] on March 21, 2012) AND a rather old version of the Twitter Bootstrap framework i.e. 2.3.1. So whatever lightbox script / library we choose to use, it *must* work correctly with these two software’s version as shipped with Koha.

2. Luckily Lightbox2 works with jquery 1.7.x or higher, which puts us in the clear about our choice of lightbox library.

3. When Lokesh’s Lightbox was released about 8 years ago, the world of web development was vastly different from today. Different browser versions (mainly IE) implementing “quirky” mode vs. “strict” modes differently, support of HTML5 / CSS3 often varied significantly from W3C standards, desktop PCs rather than smartphones and tablets ruled the world. In fact, we were also witnessing the closing moments of the second browser wars [6]. By selecting the up-to-date version of the original lightbox script, we ensure that we are using a mature javascript library with the largest possible cross-browser, cross-device compatibility *including* backward compatibility.

Now that we have all these history lessons out of the way, let us proceed with the actual steps to get this show on the road.

Step #1 – Downloading Lightbox2

Following the instructions on the Lightbox2 site, we proceeded to download the latest version of the library https://github.com/lokesh/lightbox2/archive/master.zip on our server and proceeded to unzip it.

Step #2 – Installing Lightbox2 inside our OPAC’s DocumentRoot

Since we are using Debian 8.4 on our server with Koha being installed via the .deb package route, our OPAC’s DocumentRoot is located by default at /usr/share/koha/opac/htdocs. As our server is multi-tenanted [7], we are going to create a new folder lightbox2 under this location so that every Koha instance running off this server can take advantage of the Lightbox installation.

sudo mkdir /usr/share/koha/opac/htdocs/lightbox2

Next we shall copy over the folders under the /dist folder from inside our unzipped copy of Lightbox2.

sudo cp -R dist/* /usr/share/koha/opac/htdocs/lightbox2/.

Three folders should be copied over css,images and js. The important files that we will need for our work are:

/usr/share/koha/opac/htdocs/lightbox2/css/lightbox.css
/usr/share/koha/opac/htdocs/lightbox2/js/lightbox.js
/usr/share/koha/opac/htdocs/lightbox2/images/close.png
/usr/share/koha/opac/htdocs/lightbox2/images/loading.gif
/usr/share/koha/opac/htdocs/lightbox2/images/next.png
/usr/share/koha/opac/htdocs/lightbox2/images/prev.png
Step #3 – Getting Koha to work with Lightbox

Now that we have the Lightbox2 library installed on our server, it is time to tell Koha how to use it. And the way we are going to do that using the very important OPAC system preference OPACUserJS[8] and some jQuery magic that will help us load up the library into our OPAC’s Document Object Model (DOM)[9]. This is how we will do it:

$(document).ready(function () {
  $("head").append("");
  $.getScript('/lightbox2/js/lightbox.js').done(function( script, textStatus ) {
    console.log( textStatus );
  })
});

The first line $("head").append(""); inserts the necessary stylesheet (lightbox.css) into the <head> section of the OPAC. The second line uses $.getScript[10] to dynamically load lightbox.js into the page using an AJAX based HTTP GET request when the OPAC page’s DOM has been fully loaded. The last line is used to log the success or failure of the attempt to load the file lightbox.js.

Step #4 – Using Lightbox

Adding the data-lightbox attribute a image link activates the lightbox capability for the image of our choice. The value of the data-lightbox attribute should be unique on the page on which the image is, unless we want to create a set of images on that page which should all be served by Lightbox . Basically, using the same value for a set of image links creates a manual carousal for that page. Since we are creating an image gallery we will use the same value for the entire set of images placed on and linked to from our gallery page. In fact, in our case, we wanted to show *two* galleries on the same page – one showing every day activities, another showing a recent event organised by the library. So, we ended up using *two* data-lightbox values – one for each of the sets.

This is where we define our custom page and the custom CSS styles to set things up and ready for Lightbox to take over.

A snippet of our custom page’s markup as follows, the data-title attribute adds the captions to displayed by Lightbox. As you can see, all the four example images share the same data-lightbox attribute i.e. “gallery-1” marking them as a part of a single set or gallery of images.

<div class="span3 l2c2galimg">
  <a href="gallery/rksmvv_gallery_02.jpg" data-lightbox="gallery-1" data-title="Reading Room - Journal Section">
    <img src="gallery/rksmvv_gallery_02.jpg" alt="Reading Room - Journal Section" />
  </a>
  <div class="desc">Reading Room - Journal Section</div>
</div>

<div class="span3 l2c2galimg">
  <a href="gallery/rksmvv_gallery_03.jpg" data-lightbox="gallery-1" data-title="Circulation Counter">
    <img src="gallery/rksmvv_gallery_03.jpg" alt="Circulation Counter" />
  </a>
  <div class="desc">Circulation Counter</div>
</div>

<div class="span3 l2c2galimg">
  <a href="gallery/rksmvv_gallery_04.jpg" data-lightbox="gallery-1" data-title="E-Resource Centre">
    <img src="gallery/rksmvv_gallery_04.jpg" alt="E-Resource Centre" />
  </a>
  <div class="desc">E-Resource Centre</div>
</div>

<div class="span3 l2c2galimg">
  <a href="gallery/rksmvv_gallery_05.jpg" data-lightbox="gallery-1" data-title="Photocopy Service">
    <img src="gallery/rksmvv_gallery_05.jpg" alt="Photocopy Service" />
  </a>
  <div class="desc">Photocopy Service</div>
</div>

In the following snippet note that the data-lightbox attribute is different, thus marking these entries as a separate set or gallery.

<div class="span3 offset3 l2c2galimg"><a href="gallery/rksmvv_gallery_01.jpg" data-lightbox="gallery-2" data-title="Inauguration of Research and Resource Centre - #1"> <img src="gallery/rksmvv_gallery_01.jpg" alt="Reading Room - Journal Section" />
</a>
<div class="desc">Inauguration - #1</div>
</div>
<div class="span3 l2c2galimg"><a href="gallery/rksmvv_gallery_10.jpg" data-lightbox="gallery-2" data-title="Inauguration of Research and Resource Centre - #2"> <img src="gallery/rksmvv_gallery_10.jpg" alt="Circulation Counter" />
</a>
<div class="desc">Inauguration - #2</div>
</div>

The custom CSS classes l2c2galimg and desc as seen in the above snippets are simply used to style the images and the captioning <div> and are not related to lightbox’s CSS declarations and so you are free to define your own styling.

Show me the money!

See the Lightbox enabled galleries in action here https://rksmvvlibrary.in/pages.pl?p=gallery. Simply click on any of the images and watch Lightbox take it over. As these are galleries you can scroll through the images by touching / taping / clicking the left or right side of any image in the set or even the left/right arrow keys on your keyboard to go back or forward. Clicking or touching the “XCLOSE widget or for that matter anywhere on the screen outside the lightbox popup will take you back to the gallery page.

Caveats and afterthoughts

1. You do not neccesarily need to put your list of images into separate divs of spanX class as we have done here. Lightbox will work equally well with unordered lists <ul> classed with Bootstrap 2.3.1’s thumbnail styles.

2. This set of instruction will work with all the previous versions of Koha that uses Bootstrap 2.3.1 and jQuery 1.7.2 for the OPAC. Basically that means all versions of Koha which have shipped with the bootstrap theme i.e. Koha 3.14 and above.

3. Lazy loading[11] the lightbox.js library using getScript AJAX call comes with *one* penalty though. The script is loaded timestamped and hence not cached, meaning it gets loaded each time this gallery page is opened by a visitor. You may be able to workaround the lack of caching by using the jQuery.ajax call and setting the datatype to script and indicating an expiry value for lightbox.js. We haven’t tried it, so YMMV!

4. Of course, if you deem this to an important enhancement (we don’t) then you can of course file a new enhancement bug and submit a patch to Koha community.

References:

[1] Koha 3.4.3 is now available

[2] Bug 5917 – Switch Koha to use Template::Toolkit

[3] http://template-toolkit.org/

[4] Lightbox (JavaScript) – From Wikipedia, the free encyclopedia

[5] jQuery 1.7.2 Released

[6] Second browser war – From Wikipedia, the free encyclopedia

[7] Multitenancy – From Wikipedia, the free encyclopedia

[8] 1.11.2.50. OPACUserJS – Koha 16.05 (En)

[9] Document Object Model – From Wikipedia, the free encyclopedia

[10] jQuery.getScript()

[11] Lazy loading – From Wikipedia, the free encyclopedia

Koha on an OpenVZ VPS? Better make sure you have the right time!

If your Koha ILS is on a OpenVZ VPS located in another country, then you need to specifically tell MySQL about your timezone… i.e. if you value your sanity!

The heart of everything to do with computers lies in the most intangible of things – time! We all want to know at what time was a record entered, when did our patron borrow / return a book, when is fine to be calculated, when are our IndexData Zebra re-indexing is supposed to run, when are our email / sms notifications supposed to go out etc. Mostly Koha keeps track of all these (and much more) using datetime data in the database tables. Most of these fields are set by default to CURRENT_TIMESTAMP. As a result when we transact using Koha or update our records, the MySQL server automatically fills in the exact datetime values using the system time. This “automatic” business can be an aid or a hindrance – as you will get to see in a moment.

The backstory a.k.a how we ended up with this

We were using an OpenVZ based VPS located in the US east coast for a client who is located in India. Since OpenVZ VPSes have their time controlled by the host system, there is not much we can do (unlike we can with KVM or Xen based virtualization). Luckily the host itself was setup with ntpdate, so at least the time was accurate. We took the next best route, setting up the timezone using ‘Asia/Kolkata’, which is the TZ definition for India on Linux.

$ sudo dpkg-reconfigure tzdata

It exited successfully showing the following output:

Current default time zone: 'Asia/Kolkata'
Local time is now:      Wed Sep  7 04:00:01 IST 2016.
Universal Time is now:  Tue Sep  6 22:30:01 UTC 2016.

The problem

Restarting the VPS, we proceeded with our task and things looked OK. Around 4:39 AM while importing a .mrc file we saw something that clearly showed we had missed something. The lcoal time was September 07, 2016 4:41 AM and yet Koha told us that we had uploaded the .mrc file yesterday – September 06, 2016 at 7:11 PM! Woah!

wrong-time

We quickly referred to the super awesome Koha database schema website and looked up the “import_batches” table. Sure enough, the upload_timestamp was defined as timestamp with the default set as CURRRENT_TIMESTAMP. Thus, while our timezone correction had worked with the rest of system *and* Koha, MySQL was using something else as its time.

Switching to a terminal we fired up the command-line mysql client and took a closer look.

mysql> SELECT @@system_time_zone;
+--------------------+
| @@system_time_zone |
+--------------------+
| EDT                |
+--------------------+
1 row in set (0.00 sec)

Hmmmm… Eastern Daylight time is -4:00 hours behind GMT, just as we are +5:30 hours ahead. A second query further confirmed this.

mysql> select now();
+---------------------+
| now()               |
+---------------------+
| 2016-09-06 19:29:17 |
+---------------------+
1 row in set (0.00 sec)

The solution

A quick read of http://dev.mysql.com/doc/refman/5.5/en/time-zone-support.html and a cross-check inside our /etc/mysql/my.cnf – the mysql configuration file, confirmed that we needed to setup our default timezone and move our MySQL server from the system timezone defaults coming in from the host system’s EDT timezone.

“The value can be given as a named time zone, such as ‘Europe/Helsinki’, ‘US/Eastern’, or ‘MET’. Named time zones can be used only if the time zone information tables in the mysql database have been created and populated.” – MySQL documentation

Going back to our mysql client, we decided to check the time_zone and time_zone_name tables inside the mysql system database. Both turned out to be blank.

mysql> use mysql;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed

mysql> select * from time_zone;
Empty set (0.00 sec)

mysql> select * from time_zone_name;
Empty set (0.00 sec)

Since we must have the time_zone_name table populated with our zoneinfo data (which resides under /usr/share/zoneinfo/) we turned to the nifty utility mysql_tzinfo_to_sql.

$ mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -u root -p mysql

It went through and we checked the time_zone_name table again. This time time_zone_name returned 1808 rows. With our timezone reference setup, we turned to /etc/mysql/my.cnf and added the following line under the [mysqld] section in the file:

default-time-zone='Asia/Kolkata'

Restarting the MySQL server with sudo service mysql restart we re-checked select now() and SELECT @@system_time_zone. The results were:

mysql> use mysql;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> select now();
+---------------------+
| now()               |
+---------------------+
| 2016-09-07 05:44:00 |
+---------------------+
1 row in set (0.00 sec)

mysql> SELECT @@system_time_zone;
+--------------------+
| @@system_time_zone |
+--------------------+
| IST                |
+--------------------+
1 row in set (0.00 sec)

Success!

We reloaded the Manage staged MARC records page in Koha and sure enough it now showed the correct date and time of the upload.
right-time

The 5-Minute Series: JavaScript or CSS – what is better at hiding the ‘No cover image available’ markup?

How to efficiently remove the ‘No cover image available’ placeholder using CSS rather than Jquery, when AmazonCoverImages or GoogleJackets can’t retrieve a cover image for your displayed title.

We really like our OPACs to display the cover images of the books and journals which we have often so painstakingly cataloged. Out of the box, Koha allows us to fetch and display book covers from several sources, both local and from data providers over the Internet. The commonest of these being (a) AmazonCoverImages and (b) GoogleJackets. While these two settings (either or both) will work for a large number of books, for users in India, there are also a lot that these two sources do not offer. Especially books in Indian languages or simply books printed without an ISBN (there are a lot of these in India).

When no image is found Koha displays the text “No cover image available” as a placeholder. A lot people would rather not see this. The Koha JQuery Library on the Koha Community wiki offers the following jQuery based approach:

$(document).ready(function(){
    $('.no-image').remove();
});

You simply place this snippet into the OPACUserJS system preference and hey Presto! these pesky “No cover image available” displays are history. Well, yes and no! Yes it works, and “no”, this may not prevent that text to be displayed, at least once. Why? well how about a slow PC and an even slower internet connection? Either one of the two or a combination of both will usually ensure that you get to take a look “No cover image available” before they are “removed” from the displayed page.

The simpler thing IMHO, is to take the Cascading Style Sheet (CSS) route, which as simple as placing the following one-liner into your OPACUserCSS system preference:

.no-image { display: none; }

no-image-removed

Not only will you *never* get to see the “No cover image available” markup anymore, no matter how slow your PC / Internet connection is, this is also more efficient, since rather than use jquery to first select and then .remove() a DOM element with a particular style class attribute, we simply re-define that they shouldn’t be displayed at all. The browser DOM does all the optimization in this approach.

Ramakrishna Sarada Mission Vivekananda Vidyabhavan Library partners with L2C2 Technologies to take their catalogue online

We are happy to extend a warm welcome to our newest client-partner Ramakrishna Sarada Mission Vivekananda Vidyabhavan (RKSMVV). The college has been a long-time Koha user, having started in 2011. We are happy to note that their entire physical collection of books have been catalogued into Koha and soon we’ll be updating the OPAC with their up-to-date MARC21 data.

The OPAC is available at https://rksmvvlibrary.in/ and is running currently on Koha 3.22.10 which was released on August 25, 2016.

L2C2 Technologies acknowledges with gratitude the co-operation extended by the library team at RKSMVV. We would particularly like to thank Devatmaprana Mataji the library in-charge, Smt. Parama Sarkhel, Librarian and Smt. Mousumi Adak, Guest Librarian for their trust and unstinting support.

About RKSMVV

Ramakrishna Sarada Mission Vivekananda Vidyabhavan is a partly residential degree college for women affiliated with the West Bengal State University, India. Colloquially more commonly known as “Sarada Mission College” or simply as “Sarada College”, it was the first educational institution started by Ramakrishna Sarada Mission. It came into existence in 1961 as an effort to carry out Swami Vivekananda’s ideals of education among women. Housed initially in a one-roomed makeshift structure, with thirty one students and a handful of young, idealistic and enthusiastic teachers and monastic members, the college today stands among the premier women’s institutions of the state. (Source: Wikipedia)

Koha spine label is not printing the “/” in your call numbers? Here is why.

If you have defined DDC as your classification source and have a “/” in your Koha item call number, it is not going to be displayed when generating spine labels. If you are in hurry or you are aware of the segmentation mark, you can jump straight to the section The Answer.

The “Problem”

Earlier in the day a fellow user Dyuti Samanta came up with a question :

“Sir, I’m trying print spine labels from Koha. However I see that Koha does not print the the front slash (“/”) in my itemcallnumber, even though the same is recorded in my MARC record and is otherwise displayed by Koha elsewhere. For example, the “CHA / L” in “025.4 CHA / L” is being printed as “CHAL”. So where is the problem, how can I fix it?”

The Background

Dyuti’s question made me smile. And instead of immediately telling him about the “why” I pointed him to a comment left by Anamika Das on Vimal Kumar Vazaphally‘s blog post – “Spine label creation” saying “You are not alone with that question! ;-)”.

A call number typically consists of Dewey class number + book number i.e. Cutter number (or some other means of alphabetic arrangement). The frontslash “/” is deemed as a segmentation mark (ala prime mark of in C-I-P records) in the universe of Dewey Decimal Classification[1]. Up until DDC 22 published in 2003 [2], the slash or the prime mark was used to mark the start of every standard subdivision (notation from Table 1) as well as the end of the Abridged number. However, this rule changed from DDC 22 onwards (September 1, 2005 to be exact) and remains extant for the current edition i.e. DDC 23 published in 2011. The new rule has been that only *one* single segmentation mark may be used and that too only for marking the end of the abridged number [3].

A prior and post example straight from Library of Congress

Before DDC 22 – 551.21/09797/84

DDC 22 onward – 551.210979/84

Further, if you follow LC and OCLC norms, while Dewey class number in MARC21 field 082 can definitely have (since Sep 1, 2005) a *single* segmentation mark, the call number should never have one. With this background story in place let’s look at Koha to understand what is happening here.

The Answer

The particular Koha code that has taken out the the slash from both Dyuti and Anamika’s call numbers resides in C4::Labels::Label Perl module which is located at /usr/share/koha/lib/C4/Labels/Label.pm. Even more specifically, it is the _split_ddcn subroutine in Label.pm that is taking out the “/“. As we have already noted, under LC rules, call numbers (unlike Dewey class numbers in 082) can’t have segmentation marks. Thus it takes out any “/” embedded in your call number while processing the spine label. Very specifically, it is this line in the _split_ddcn subroutine: s/\///g; # in theory we should be able to simply remove all segmentation markers and arrive at the correct call number that does it. And just why does _split_ddcn get invoked? Well, it is because of something you did during cataloging, remember that you had recorded DDC as the classification schema? It is that definition in your MARC record that calls in this sub 😀

Below you can see the _split_ddcn subroutine as on date of this post.

sub _split_ddcn {
    my ($ddcn) = @_;
    $_ = $ddcn;
    s/\///g;   # in theory we should be able to simply remove all segmentation markers and arrive at the correct call number...
    my (@parts) = m/
        ^([-a-zA-Z]*\s?(?:$possible_decimal)?) # R220.3  CD-ROM 787.87 # will require extra splitting
        \s+
        (.+)                               # H2793Z H32 c.2 EAS # everything else (except bracketing spaces)
        \s*
        /x;
    unless (scalar @parts)  {
        warn sprintf('regexp failed to match string: %s', $_);
        push @parts, $_;     # if no match, just push the whole string.
    }

    if ($parts[0] =~ /^([-a-zA-Z]+)\s?($possible_decimal)$/) {
          shift @parts;         # pull off the mathching first element, like example 1
        unshift @parts, $1, $2; # replace it with the two pieces
    }

    push @parts, split /\s+/, pop @parts;   # split the last piece into an arbitrary number of pieces at spaces
    $debug and print STDERR "split_ddcn array: ", join(" | ", @parts), "\n";
    return @parts;
}

Note: The _split_ddcn was first submitted to the Koha codebase as part of C4::Labels::Label module by Chris Nighswonger on Jul 20, 2009, by which time the LC’s single segmentation mark rule was already long in place.

So now what?

There are a few options available to you at this point.

(a) If you know what you are doing, you can modify the _split_ddcn sub routine so that it does not discard the “/” and handles the call number as you want it to. (Non trivial and not recommended)

dontsplit

(b) Go to “Manage Layouts” and editing your specific layout by un-checking the option “Split call number“. If you do this then your call number will be printed AS-IS as a single line of text. This means, if the call number is longer that the size of your labels, as they will be at several point in time, you have a *problem*

(c) Keep an eye out to this bug report filed by Katrin Fisher from earlier this year, where she has said:

Currently the call number splitting seems to be mostly implemented for DDC and LC classifications. Those are both not very common in Germany and possibly other countries. A lot of our libraries use their own custom classification schemes so the call number splitting is something that should be individually configurable.

The bad new is that so far no one has responded to this bug, simply because to Koha developers servicing clients using LC / DDC, this is not a priority. So either you can wait with the hope that someone soon will attend to this bug OR you write this functionality yourself OR you sponsor a developer to write it for you.

(d) Take the item call number listing out of Koha as a CSV file and use a 3rd party tool, e.g, gLabels to generate your spine labels.

References:

[1] https://www.loc.gov/aba/dewey/segmentation.html

[2] Dewey_Decimal_Classification – Administration_and_publication

[3] “Sweet segment solution” from 025.431: The Dewey blog

Koha OPAC over SSL breaks GoogleIndicTransliteration

GoogleIndicTransliteration is a nifty Koha feature allowing easy typing and searching in several Indian language to Indian users. However, a bug prevents it from working if the OPAC is run over SSL (i.e. https). This post provides a clear description and a fix for the problem.

Many Indian Koha users use the GoogleIndicTransliteration option to offer their users the facility to search in Indian languages on the OPAC. This nifty feature allows users to phonetically type in their search queries in Indian languages in order to search catalogs that are (a) multi-lingual or (b) in a Indian language other than English.

mixed_03

However, if you are security minded (and you *should* be if your OPAC is on the Net and allows your users to log in) and you decide to serve your site over to SSL (i.e. https), then guess what? The GoogleIndicTransliteration feature stops immediately with the browser console showing MIXED CONTENT error. Every single Koha version from 3.18.0 (when this feature made its way back into Koha after a long hiatus) up to the latest 16.05.2 (released on August 1, 2016) are affected by this problem.

mixed_02

I do not have time, just show me how to fix this

If you are in a hurry, jump over to the section “Your options until the patch is officially released” at the end of this post. Remember to read the caveat and the assumption, you have been warned! 😉

Why is HTTPS so important?

Let’s take a moment to understand why HTTPS is so important. Let’s assume that your Koha server is on your institutional LAN / intranet or hosted online, either on the cloud or on your own server connected to the Internet via a leased line.

Without HTTPS, every time you login into Koha (staff and/or OPAC) and perform *any* ILS transactions (e.g. patron contact information change, holds, fines, circulation etc) all of that information is available in PLAIN TEXT to everyone on your network.

If you are only connected to your institutional network, then that is the direct extent up to which anyone can see what you are doing. If your server is accessible over the Internet, then basically the whole wide world can see what you are doing. For instance, when you login over HTTP, it is actually the equivalent of writing down your username and password on a postcard and mailing it across the globe. Anyone who handles it during transit, or wants to, can simply read it. That’s why the world is moving away from the plain vanilla HTTP.

postcard

In simple terms, HTTPS on the other hand creates an end-to-end encrypted “tunnel” between your server and the browser that is requesting access (e.g. to the OPAC). Think of it as a secure, sealed box with the contents inside and only you, the user, have the “key” to unlock it. The actual process is depicted in the graphics below:

Image source : https://www.identrustssl.com/

Briefly HTTPS has 3 main benefits:

(a) Authentication
(b) Data integrity
(c) Secrecy

None of these are provided by HTTP, thus if your Koha server is online, the SSL (HTTPS) is simply a must these day!

The Basics Explained

GoogleIndicTransliteration feature utilizes a Google API designed for phonetic input of several Indian languages by transliterating text written in English on the fly to its Indian language equivalent. For example, if you type “Rabindranath” and it is set to transliterate to Bengali, the software will automatically convert to “রবীন্দ্রনাথ” or say “Premchand” to “प्रेमचाँद” if set for Hindi.

As with every Google API (and there are many), the Transliteration API too needs to be loaded by a minified Javascript API loader program, known simply as the “Google API Loader“.

How it works

Once GoogleIndicTransliteration system preference is set to “Show” from the Koha staff client, the code inside the file opac-bottom.inc loads up the API loader code available at www.google.com/jsapi, which in turns provides the framework so that the actual transliteration code available in the file googleindictransliteration.js can work its magic and provide the users with the transliteration feature.

GoogleIndicTransliteration system preference
The GoogleIndicTransliteration system preference is set to “Show” on the OPAC.
Why does HTTPS break it but not HTTP?

Short answer: Mixed context!

Long answer: HTTPS is important to protect both your site and your users from attacks online. As of now, Koha code in opac-bottom.inc calls the jsapi code over HTTP, instead of letting the browser handle it correctly based on the security context (i.e. whether the page is being served over HTTP or HTTPS). So when OPAC is on HTTP, jsapi is fetched over HTTP, things are on the same page. However, when the OPAC is served over HTTPS and jsapi continues to be fetched over HTTP, all modern browsers will flag it as a security violation known as “MIXED CONTENT” and halts the loading of jsapi, as seen in the screenshot below:

mixed_04
Error shown in Chrome’s browser console

As a result, googleindictransliteration.js has nothing to work with. End result, the GoogleIndicTransliteration feature does not work anymore! Bingo! We’ve found ourselves with a Koha bug!

Present status of bug

There is a patch submitted to Koha Bugzilla against Bug 17103 – Google API Loader jsapi called over http, waiting for sign-off and QA. Once it clears Koha’s project governance processes, it is expected to get pushed to the master and then be released with a stable version of Koha. Once that happenes we won’t have this issue anymore.
NOTE: Expect this fix to get backported across the current supported older releases.

Your options until the patch is officially released

(a) Do without GoogleIndicTransliteration feature until the fix is officially released by the Koha project if you are using HTTPS

OR

(b) Edit your “opac-tmpl/bootstrap/en/includes/opac-bottom.inc” file. Find the following section:

[% IF ( GoogleIndicTransliteration ) %]
    <script type="text/javascript" src="http://www.google.com/jsapi"></script>	
    <script type="text/javascript" src="[% interface %]/[% theme %]/js/googleindictransliteration.js"></script>
[% END %]

Replace the protocol notifier “http:” from jsapi URI with “https:“and save the file. It should look like this after the change:

[% IF ( GoogleIndicTransliteration ) %]
    <script type="text/javascript" src="https://www.google.com/jsapi"></script>	
    <script type="text/javascript" src="[% interface %]/[% theme %]/js/googleindictransliteration.js"></script>
[% END %]

CAVEAT: If you are doing this edit, it is assumed that you know what you are doing. If you make any mistake and break something during this, its all on you.

ASSUMPTION: This edit assumes you are on Koha 3.18.x and later and is using a .deb package based installation on Debian or Ubuntu.

New feature preview – Quick create patrons

Koha 16.11 which is set to be released sometime in Nov 2016 will include a new and useful feature [1] – to be able to quickly add a new patron. This request had been a long pending one. Afterall the request for this enhancement was posted on the Koha bugzilla almost 7 years back.

Those who are testing the Koha unstable release will be able to see it in action already, for the rest of you here are a couple of screenshots to whet your appetite. 🙂

Select the "Quick add new patron" button
Fig 1 :Select the “Quick add new patron” button
quickcreate_patron_02
Fig 2: The quick add form (with option to open up the full form)

Reference: [1] Bug 3534 – Patron quick add form

A custom subject-wise report of titles with author name, no. of copies, subject name in serialized listing

A custom SQL report for Koha that generates subject wise title lists with author name, no. of copies, subject name and biblionumber, written in response to a reader query over email.

Last week Mr. Gautam Mukhopadhyay, Librarian, Chandrapur College in Burdwan, West Bengal wrote in with a request:

Respected Sir,

I’m writing this seeking a solution for the problem relating to a report generation from Koha. I want to get a list of titles under a particular broader subject field-tag (650). Quite a number of times I’ve checked from SQL Report. But all were in vain as those were not the same what I actually want to get. Following is the specimen of the opted report:
Sr. No.   Title     Author   Copy No.    Subject
  1         ……….     X          3             Bengali
Under the subject Bengali or English or whatsoever, I want to get the titles those are belong to that particular subject. However, it won’t be a problem if there are different reports for different subjects. It’s Ok. But the SQL Query should be a general query structure that can be applicable for all such reports on the titles belong to a broader subject like Bengali, History, Geography etc.
Sir, please let me know the query structure, if possible.
Regards,
GM

 Here is a possible solution for his request, which pretty much does what Mr. Mukhopadhyay had specified in his request. In this example we’ll use a sample MARC21 file which can be downloaded from here to try out this example. This dataset has a 14 unique bibliographic records with a total of 42 item (holdings) record, belonging to 03 specific broader subjects i.e. English, Economics and Political Science. As per Mr. Mukhopadhyay’s use-case, the MARC field 650 holds the broader subject classification. However, to match real world scenarios the 650 fields in some of the cases have other subject headings defined including narrower terms. Also additionally we are going to add an additional column to our report – the biblionumber, so that if required we can cross check a title in the report generated against the biblionumber in the database.

CAVEAT EMPTOR: If you are going to try out this example, we suggest that you define a new Koha library and import this MARC file into it. Mixing this sample data with your existing records is strongly advised against.

Step #1 – Create a new Koha instance and set it up
We are going to use the koha-create Debian command to create a new Koha instance and we shall call our instance as demo.
sudo koha-create --create-db demo

You may calls your instance by whatever name you like. If you are not aware of the koha-create command, please read up “Commands provided by the Debian packages“. Next we will do a default setup and proceed to define a Library that we’ll call “L2C2 Technologies Demo Library” identified by the code “MAIN”, using these instructions here.

NB.: To use the marc file used in this example you must set the library code for your demo branch as “MAIN”, the name (of the library branch) can be whatever you want it to be.

Step #2 – Define a new Authorize value category
Since our example marc file has biblios with only (a) English (b) Economics and (c) Political science, we will define a new authorized value category which we’ll call as SUBLOOKUP, under Home › Administration › Authorized values. Once setup our new authorized value SUBLOOKUP will look like this:
gmreport_02
This authorized value list will provide the subject selection list for our custom SQL report. So if you have more subjects you will need to add them here so that they look like this. The “%” in the Authorized value is *critical*, and if you want to be really strict about it, you can drop the preceding “%” and retain only the one at the end. However should you do that, your first 650 field *must* always be the broader subject heading that you wish to filter your report on.
Step #3 – Define our custom SQL report
We will go to Home › Reports › Guided reports wizard › Create from SQL and create a new SQL report. In this case, we’ll name the report as “List title with number of copies filtered by subject”, add a note that says – “A report written at the request of Mr. Gautam Mukhopadhyay, Chandrapur College, BWN”. The SQL will be as given below. The report once saved, will allow us to run it.
SELECT 
 (@row:=@row+1) AS `S/N`, 
 gmData.Title, 
 gmData.Author, 
 gmData.Copies, 
 REPLACE (@TargetSubject:=<<Select the subject|SUBLOOKUP>>, '%', '') AS Subject, 
 gmData.biblioid AS `Biblionumber` 
FROM 
 (SELECT
 biblio.title AS Title, 
 biblio.biblionumber as biblioid, 
 ExtractValue(biblioitems.marcxml,'//datafield[@tag="245"]/subfield[@code>="c"]') AS Author, 
 count(items.itemnumber) AS Copies, 
 ExtractValue(biblioitems.marcxml,'//datafield[@tag="650"]/subfield[@code>="a"]') AS Subject 
 FROM 
 items 
 LEFT JOIN 
 biblioitems on (items.biblioitemnumber=biblioitems.biblioitemnumber) 
 LEFT JOIN 
 biblio on (biblioitems.biblionumber=biblio.biblionumber) 
 GROUP BY 
 biblio.biblionumber 
 ORDER BY 
 biblio.biblionumber) as gmData, 
 (SELECT @row := 0) r 
 WHERE Subject LIKE <<Re-select the subject tag|SUBLOOKUP>>
Let us take a moment to understand what this piece of SQL syntax really means.
(@row:=@row+1) AS `S/N`,

and

(SELECT @row := 0) r

The use of the @row variable and the counter (@row:=@row+1) gives us our “serial number” column in the report listing.  We can also see the authorized value list “SUBLOOKUP” that we had defined earlier referenced here in the SQL.

NOTE: As you may note, we are asking the user to select the subject *twice*, (first time: ‘Select the subject’ and second time: ‘Re-select the subject tag’). While theoretically we should not be required to do so, thank to the use of the runtime variable @TargetSubject, in reality we ran into a type casting error (see below), thus we used this less than pretty way of asking the user to select the subject twice, to get our job done.

gmreport_03

Step #4 – Running the report

After the report is saved, it is now time to run it, using the “Run report” option. What we’ll see now will be like this:

gmreport_04

We need to select the *same* subject from both the drop-down lists and click on “Run the report” button. Selecting “Economics” we shall in our case get the following report:

gmreport_05

Step #5 – Prettifying the custom report user interface

Having the user to select the subject twice is cumbersome as well prone to human error, so we decided it is time for some jquery magic to streamline this and leave the users with one only a single drop-down to choose from. For this we’ll turn turn to the IntranetUserJS system preference and add the following jquery snippet:

 $("label[for='sql_params_Reselectthesubjecttag']").hide()
 $('#sql_params_Reselectthesubjecttag').hide();
 $("#sql_params_Selectthesubject").change(function() {
   var subval = $('#sql_params_Selectthesubject').val();
   $("#sql_params_Reselectthesubjecttag").val(subval);
 });

If this is the first time you are hearing about the IntranetUserJS system preference, you should definitely read up this. Those of you who are indeed familiar with IntranetUserJS, all we are doing here is to (1) hide the second subject selection dropdown and its label and then (2) we are defining that whenever the user chooses a value from the *first* drop-down, the second (and now hidden) drop-down should also have the same value selected automatically. After saving the IntranetUserJS update, on running the report we shall see this:

gmreport_06

And bingo! We are done!
Extraa Innings: To see the actual report in action
  1. Go to the URL https://demo-staff.l2c2academy.co.in/
  2. Use User name / Password : demo / demo
  3. Go to the section Home › Reports › Guided reports wizard › Saved reports
  4. Select “Run” from the “Actions” dropdown at the right.gmreport_07
  5. Play with the subject selection options to see the different outcome.

 

 

MarcEdit QuickTip #3 – Getting your 952 (items / holdings data) field in place for import into Koha

Shows how to de-duplicate a .mrc file, by merging duplicate bibliographic records spread all over the file and then gather up the holdings record into repeatable 952 field that Koha expects its for item records.

Last night Pawan Sharma, a fellow user on “Koha Users” reached out for some help with importing his items into Koha. Like many other, he too had moved his catalog data from Microsoft Excel spreadsheet to MarcEdit utilizing MarcEdit’s “Delimited Text Translator” feature which at the end of the process had given him a .mrc file. This he proceeded to upload into Koha by using the More > Tools > Catalog > Stage MARC records for import option.

There were no surprises here, *except* that for every single books with multiple copies Koha imported each of the copies as a separate biblio record, instead of a single entry for the biblio with multiple item records attached to it via Marc21 952 repeatable field that Koha uses for managing holdings data. Simply put his data needed to be de-duplicated with the holding data merged back before import, typically using the ISBN number of the records (MARC field 020).

NOTE: If you wish to read more about Koha’s holding records schema see “Holdings data fields (9xx)” from the Koha Community wiki.

For someone who has not done this before, MarcEdit’s de-duplication and then merging it can seem like a daunting task. This post will hopefully demystify the process.

The discussions on Koha Users were based on a lot of assumptions, especially with no idea about Pawan’s data. So, I offered to take a look at it. He first sent me a .mrc file that had 12806 records, which I immediately converted into MarcEdit’s MarcBreaker mnemonic, human readable format.

marcedit_01

And proceeded to take a “Field count” report (see under “Tools” menu of MarcEditor) to check exactly how many records had ISBN (MARC21 field 020) out of the total number of records.

marcedit_01A

The result as can be seen above – NOT A SINGLE ONE of the 12806 biblio records had an ISBN number! Well, this file can be de-duplicated and merged, but *not* using MarcEdit. Only being told about this Pawan mentioned that he had other .mrc files that had ISBN and so he sent a second .mrc (LG-32016-32979.mrc) file over. Turns out of the total of 965 biblio records in this second file, 828 records had ISBN numbers defined.

marcedit_02A

The next task was to extract the records that *had* ISBN numbers. The remaining 137 can not be dealt with in this process and will have to be dealt with separately. For now, we closed the file LG-32016-32979.mrk file with 965 records and went back to the MarcEdit main window in order to use the “Delete Selected Records” option available under Tools > Select MARC Records

marcedit_02B

The next few steps are simple, if not immediately apparent to a new user of MarcEdit. We’ll use the numbered markers on the screenshot to explain it in steps. First, we selected the LG-32016-32979.mrk file with the 965 records in step #1; next we typed in 020 (since we want to match for ISBN) in the Display Field option (by default it shows 245$a); third step was to click on “Import File” button. After the file is imported (takes just a second or two depending on your file size) this the top-left data grid which was blank so far, will show up data similar to this. Finally in step #4, we will click the “Does Not Match” link. Records that do not have an ISBN number will be selected just like the big red arrow here shows.

marcedit_02CThe last step is to click on “Delete Selected”, this will open a File Save dialog with the title “Remaining Records”. In the case, we provided the name LG-32016-32979_ISBN.mrk and saved it and exited from this deletion utility.

This file LG-32016-32979_ISBN.mrk now has the 828 records with ISBN numbers and each of which has a holding records. This is what we will work with for the deduplication process.

marcedit_02D

Using the Tools > Record Deduplication option of MarcEditor, we will now remove the duplicate records into a separate file and save it with the name LG-32016-32979_ISBN_DEDUP.mrk. We will use ISBN as the field to use to identify duplicates. A popup showed us that 828 records processes, so we are done with deduplication. We will also need to save our original work file LG-32016-32979_ISBN.mrk. This file now contains biblio records with unique ISBN number. A quick check with the Fields Count tool showed us there were now 523 records (down from 828 records originally, the rest 305 records are the duplicates that are now saved in LG-32016-32979_ISBN_DEDUP.mrk).

marcedit_02E

 Now for the next step MARC Merge, which was the last step in this process. We have to go back to the main MarcEdit window and use the menu option Tools > Merge Records. The order of files we specify here is highly *important*. The “Source File” in this case was LG-32016-32979_ISBN.mrk (the file with the 523 records with unique ISBN numbers), the “Merge File” is LG-32016-32979_ISBN_DEDUP.mrk (the file where we had removed the duplicates to in the previous step) and finally, “Save File” is simply the name of the new merged file we are going to create (Hint: this is the final file that we will push to Koha). We named the final file as LG-32016-32979_ISBN_MERGED.mrk. The Record Identifier is of course 020 (i.e. ISBN number) and we move on the next screen.

marcedit_02F

This is next step is basically *everything* we have been working for in this post so far, we select the field to merge in from “Merge File” into the “Source File” and click next.

marcedit_02GIn this case everything went well and we were presented with the following screen that said “Merge Completed” and gave us the full path and filename to our merged file LG-32016-32979_ISBN_MERGED.mrk.

marcedit_02H

Of course we opened up the LG-32016-32979_ISBN_MERGED.mrk file in MarcEditor. The first thing was to check the Field Count report, and this is what we saw 523 biblio records with a total of 828 holding records, which sounds right! Below is example of the merged holdings.

marcedit_02I

Of course there is still the task of exporting the MarcBreaker (.mrk) back to .mrc so that Koha can ingest it for its MARC21 staging workflow, but everyone knows that 🙂

NOTE: For reference to this tutorial I’m attaching the zip file containing all the LG-32016-32979 files used in this example.

Planning to bulk import your patrons? Make sure you do not have in-line line breaks in your data.

In-line line breaks in a CSV file can really send your Koha patron import script into a tailspin. Here is what you need to watch out for and the couple of other gotchas which will make you upgrade your Koha instance if the version you are using is less than 3.22.7.

Last week a friend working at a local college approached me for a spot of help. He was trying to import his patrons into Koha but was failing miserably. After he nearly got his head snapped off (Me: Do I look like I’m in the fortune telling profession???) he agreed to send over his data – an MS-Excel sheet for me to take a look at.

I pulled up a 3.22.6 instance I had laying around and tried to import his data. Quite expectedly, there were errors galore and the pretty much the same ones he was complaining about.

blog_patron_1

Hang on! categorycode, branchcode and surname fields were NOT missing in *any* single record. So what was going on here??? The most interesting to note here is that patron importer script said :

272 not imported because they are not in the expected format

272 records parsed

Now this was really something as the total number of student records in that patron uploader CSV file were only 144. So where does the number 272 come from?

The answer to this was easy to find. My friend’s data had several records in a rather bad shape – they had embedded line-breaks within the cell. I’ve highlighted the first few of the badly formatted cells with yellow in the screenshot below.

blog_patron_3A

So, I copied the first 28 records over to a new file, ran a hackish utility script to clean out the line breaks and saved these 28 records as a new file and proceeded to upload it. This time of course “the fat lady sang”[1] i.e. the records got imported nice and we were done!. 😀

blog_patron_3NOTE:  Of course while doing that we encountered a few Koha bugs as well – Bug id 15840 and Bug id 16426. The work-around mentioned in comment #16 of the latter bug, by Koha QA Manager Katrin Fischer holds good, in case you get stuck here and can’t immediately upgrade. Otherwise to avoid these to bugs, your real option is to upgrade your Koha instance, something that I’m going to recommend to my friend (aside from him fixing his data).

Reference: [1] Wikipedia “It ain’t over till the fat lady sings”