Show me the money…. er.. the barcode?

Fixing the issue of checked out items barcodes’ not showing up for patrons logged into the Koha OPAC.

The Problem

Earlier this morning, Pranab Roy who manages the Library at the Karnavati University‘s UnitedWorld School of Business‘s Kolkata campus popped up on WhatsApp with a question – “Sir, why is Koha not showing the barcode / accession number of a borrowed document when an user logs in into their own account via the OPAC?”

This is what he meant. And this is actually the expected behaviour, with Koha doing exactly what it was asked to do. I could understand his confusion since the OPAC search’s details view showed the barcode quite nicely, then why not for the users themselves?

A bit of backstory

Karnavati University Libraries had shifted to L2C2 Technologies‘ cloud platform from a pre-existing Koha instance maintained by a 3rd party. As such the system had quite a few issues. While we had fixed a larger number of these during the initial on-boarding stage, some of these are getting ironed out only now as the librarians hit these “bumps on the road”. Pranab’s problem was one such.

The Solution

The option to display the barcodes for a logged-in OPAC user’s checkouts (issues) is driven by the SHOW_BARCODE patron attribute. In Koha, patron attributes or more correctly ExtendedPatronAttributes are library-defined custom fields that can be applied to patron records e.g. voter / aadhaar card number, registration number etc.

SHOW_BARCODE is a boolean variable that is defined as either Yes or No. and it is loaded into Koha usually during the web-installer phase of Koha’s installation from the optional SQL dump file: /usr/share/koha/intranet/cgi-bin/installer/data/mysql/en/optional/patron_atributes.sql in Debian package based installations. The tiny file contains a single SQL INSERT statement:

INSERT INTO `borrower_attribute_types` (`code`, `description`, `repeatable`, `unique_id`, `opac_display`, `staff_searchable`, `authorised_value_category`) VALUES (‘SHOW_BCODE’, ‘Show barcode on the summary screen items listings’, 0, 0, 1, 0, ‘YES_NO’);

In the case of Karnavati University this optional patron attribute was not imported during the *original* installation done by the 3rd party support provider at the time. And without “SHOW_BARCODE” being set, Koha had no way of displaying the barcodes of checked out books to patrons logged in via the OPAC.

In the end, the following two lines followed by enabling the ExtendedPatronAttributes system preference cleared off the issue for Karnavati:
$ cd /usr/share/koha/intranet/cgi-bin/installer/data/mysql/en/optional
$ mysql -uroot -p koha_karnavati < patron_atributes.sql.

A memcached restart later (to be safe rather than sorry) the OPAC started showing the barcodes to logged-in patrons.

Using an office suite to generate the sql for Koha’s calendar

An in-house technique we use for fast-track, accurate loading of the long list of Indian holidays into Koha.

LEGAL DISCLAIMAR: This tutorial is strictly meant for educational purpose. We are in no way responsible if you attempt to follow these steps and end up messing up your production Koha installation somehow.

Setting up yearly calendars is an important task in Koha. Especially in a country like India where we also use other calendars including lunar calendars. Since the religious festivals / holidays usually use these other calendars, the holidays often fall on different days in different years in the Gregorian calendar that Koha uses.

As support providers, we are often asked to setup the calendar for our client-partners. While Koha provides a rather useful user-interface to setup calendars under the Tools menu, we have often found that not only setting up the long list of Indian holidays a tedious affair, it is also quite prone to operator error. So for the Unique holidays we usually use a LibreOffice (a Free Software Office Suite) Calc spreadsheet to quickly generate the batch of SQL statements that can be directed imported into the special_holidays table of Koha using the SQL backend.

We ask our client-partners to send us a spreadsheet that lists the holidays and the days they fall on according to the Gregorian calendar (see the columns marked in yellow above). And then we use a couple of simple formulae to generate the necessary SQL. This approach has two advantages : (a) it’s very fast and (b) completely free of errors from our end.

In this present case, it took us less than 2 minute to upload all the 25 holidays for 2020 into this particular Koha instance.

Caveat The only risk here is that we are going to access the database directly from the command-line and that, unless you know what you are doing, can end up badly, *if* you mess things up.

Getting LE certbot-auto to work on an aging Debian 7.x

Stuck with Debian 7 in 2020 and need certbot-auto to work? Here’s how we did it.

Yes, it is 2020 and it is very late in the day to be still using Debian 7.x. But you just may have a piece of critical infrastructure that is still running on that Debian 7 box and moving it may not be an immediate possibility. Your infrastructure component also happens to use LetEncrypt certs for SSL. Your certificate has just expired and you ran certbot to issue a new certificate. And BAMMMM! you hit this!

Replacing certbot-auto…
Creating virtual environment…
Installing Python packages…
/opt/eff.org/certbot/venv/bin/python: No module named pip.__main__; ‘pip’ is a package and cannot be directly executed
Traceback (most recent call last):
File “/tmp/tmp.BLzjDMi7yW/pipstrap.py”, line 177, in
sys.exit(main())
File “/tmp/tmp.BLzjDMi7yW/pipstrap.py”, line 149, in main
pip_version = StrictVersion(check_output([python, ‘-m’, ‘pip’, ‘–version’])
File “/usr/lib/python2.7/subprocess.py”, line 544, in check_output
raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command ‘[‘/opt/eff.org/certbot/venv/bin/python’, ‘-m’, ‘pip’, ‘–version’]’ returned non-zero exit status 1

Well, we did. The problem stems from the fact that with certbot-auto version 0.32 it stopped working with EOLed Linux distributions. This has hit distros like Debian 7.x that EOLed towards the end of 2018 which also dropped official certbot support. Debian 7.x (wheezy) uses an ancient version of pip that cannot be run as a module (python -m pip). And hence the mess.

This is how we got over it for the moment (it is Jan 2020 at the time of writing)

1. rm -rf /opt/eff.org

2. Download old 0.31 version of certbot-auto so that we can get around the version issueswget https://raw.githubusercontent.com/certbot/certbot/75499277be6699fd5a9b884837546391950a3ec9/certbot-auto

3. chmod +x ./certbot-auto

4. Run certbot-auto with the necessary switch ./certbot-auto --no-self-upgrade

And this was the result

Bootstrapping dependencies for Debian-based OSes… (you can skip this with –no-bootstrap)
Hit http://archive.debian.org wheezy Release.gpg
Hit http://archive.debian.org wheezy Release
Hit http://archive.debian.org wheezy/contrib Translation-en
Hit http://archive.debian.org wheezy/main Translation-en
Hit http://archive.debian.org wheezy/non-free Translation-en
Hit http://archive.debian.org wheezy/main amd64 Packages
Hit http://archive.debian.org wheezy/non-free amd64 Packages
Hit http://archive.debian.org wheezy/contrib amd64 Packages
Hit http://archive.debian.org wheezy/main i386 Packages
Hit http://archive.debian.org wheezy/non-free i386 Packages
Hit http://archive.debian.org wheezy/contrib i386 Packages
Reading package lists… Done
Reading package lists… Done
Building dependency tree
Reading state information… Done
gcc is already the newest version.
python is already the newest version.
python-dev is already the newest version.
python-virtualenv is already the newest version.
openssl is already the newest version.
libffi-dev is already the newest version.
libaugeas0 is already the newest version.
libssl-dev is already the newest version.
ca-certificates is already the newest version.
augeas-lenses is already the newest version.
The following packages were automatically installed and are no longer required:
libapache2-mod-fcgid libcarp-assert-more-perl libcarp-assert-perl libcgi-compile-perl libcgi-emulate-psgi-perl libdevel-stacktrace-ashtml-perl
libfcgi-procmanager-perl libfile-pushd-perl libfilesys-notify-simple-perl libfreeradius-client2 libhash-multivalue-perl libhtml-lint-perl libhttp-body-perl
libmodule-refresh-perl libplack-perl libtest-longstring-perl libtest-requires-perl libtest-sharedfork-perl libtest-tcp-perl rt4-apache2 rt4-clients rt4-db-sqlite
Use ‘apt-get autoremove’ to remove them.
0 upgraded, 0 newly installed, 0 to remove and 155 not upgraded.
Creating virtual environment…
Installing Python packages…
Installation succeeded.
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator apache, Installer apache

Which names would you like to activate HTTPS for?
– – – – – – – – – – – – – – – – – – – – – – – – – – – – –

Shared in the hope that it may help someone else in a similar position.

Automating Koha : An Use-Case.

As Koha automates libraries around the world, automating Koha itself offers major benefits for people maintaining these Koha systems.

The background

In the context of the Indian sub-continent, labour being generally cheap, manually doing things is often the norm. Thus even as libraries adopt software like Koha to automate their operations, the management of Koha itself remains largely a manually driven process. While it is no doubt cheaper in the short term to do so, on the longer term, this involves invisible costs. For example, human errors creeping into the Koha configurations; a lack of situational awareness about the running system and thus no pre-emptive maintenance interventions that ultimate lead to higher down-times and disruption of services.

So, as our client-partners look at Koha to automate their libraries, we seek to largely automate the management and maintenance of Koha itself. The open-source nature of Koha lends well towards this. This blogpost aims to do two things : (a) share with our readers an idea of things that are possible with the most minimal of coding and (b) show case how this has benefited both us and our client-partners.

The automation use-case

Some time back we realised that it would be useful if we could let our client-partner users know in real-time exactly how many days their hosted service subscription was still valid for. The information was useful to them to plan their renewals in time. To do this we had initially opted for a small JQuery snippet placed inside IntranetUserJS system preference that took a hard-coded date which was the end date of their active subscription.

Problems encountered


At first, this was seen as a good thing by our client partners. Yet as the subscription ended and was renewed, we were hit by a problem. When they renewed it, we had to manually re-edit the JQuery code and enter the new end date. There were two problems here – (a) sometimes we simply forgot to update it after the payment was made and the system would show that the subscription had expired even though it had been renewed; (b) as anyone dabbling with JQuery knows, its easy to introduce typographical / syntax errors while editing JQuery, if you are careless while doing it.

The solution

The first confused our users and left them with a feeling of dissatisfation. The second was more our headache. So, we wondered can Koha itself help us *automate* this hosted account renewal process? Turns out the answer was “yes” as we looked at *two* basic Koha features – (a) local use system preferences aka system preferences that are generally defined by users of the system rather than by the Koha developers and (b) SQL reports web service API which allows calling on SQL reports via an definitive URL with their results being returned back in JSON format.

From hard-coding to flexibility

We started by defining a local use system preference named SubscriptionEndDate (yes, we are very imaginative ๐Ÿ˜‰ ) which will be used to store the end date in the ISO 8601 format i.e. YYYY-MM-DD HH:MM:SS.

Next, we defined a simple SQL report that extracted the end date value from the systempreferences table in Koha.

The last step was to modify our existing JQuery snippet so that it no longer looked for a hard coded end date value within the snippet itself. Instead it would first call the SQL report web service to get the end date from the SubscriptionEndDate local use syspref as JSON data. Once the data was there, the rest of snippet remained same in terms of how it calculated and displayed the number of days remaining.

Introducing automation into the picture

With the above in place, it was now just a matter of updating the SubscriptionEndDate value when the client-partner renewed by paying up. As the payment got captured in our CRM system, it called an API that automatically triggered an update operation on SubscriptionEndDate in Koha setting it with the new end date. With less than 20 lines of code we had managed to automate the subscription date renewal process. We had managed to remove the twin issues of inconsistent manual updates and the possiblity of introducing typographical errors and breaking our IntranetUserJS configuration.

Revisiting Kerala eSMS service for updated versions of Koha ILS

If you are a Koha user from Kerala using the eSMS send driver written by us and found out that your SMS alerts have stopped after upgrading to 17.05 or later, then you should definitely read this.

For the impatient: Koha users from Kerala using the SMS::Send::IN::eSMS send driver, often find that after upgrading from 16.11 or earlier versions, their SMS alerts from Koha had stopped working. Usually this affects users who had used this previous blog post as the installation guide. This post addresses that and shows how to get SMS working again for eSMS service on 18.11 or later versions.

For the *really* impatient: Jump directly to the section Handling SMS::Send::IN:eSMS on supported versions of Koha

Background

About 3 years back, in early February 2017, we had published the only open-source SMS::Send driver implementation for Kerala Government’s eSMS transactional bulk SMS service for use with Koha ILS. The development was sponsored courtesy the State Librarian, Kerala State Central Library, Thiruvananthapuram, Kerala.

That time the latest stable version of Koha was 16.11. Up until then Koha’s C4/SMS.pm which acts as a wrapper against SMS::Send posed a small problem for Indian users thanks to a the requirement of senderid by Telecom Regulatory Authority of India (TRAI). Koha provided only two fields – the login and password fields. We in India needed three. So, we had to hack C4/SMS.pm as documented here.

The big change

At the end of May 2017, Koha 17.05 was released and with it came Bug id #13029. In the Release Notes, bug 13029 was defined as “Allow to pass additional parameters to SMS::Send drivers”.

This was a game-changer. We no longer needed to hack C4/SMS.pm. Handling extra parameters like SenderIDs, API keys or for that matter any arbitrary parameter(s) specific to a particular bulk transactional message provider could now be handled using a simple YAML file. The path to this YAML file is defined in koha-conf.xml and therefore making it instance-specific and multi-tenant friendly. <sms_send_config>/etc/koha/sites/<your_instance_name>/sms_send/</sms_send_config>

Handling SMS::Send::IN:eSMS on supported versions of Koha

At the time of writing, the supported versions of Koha are 19.11, 19.05 and 18.11. If your version is lower than 18.11, you should really upgrade. There are just three things to keep in mind; (a) there is no change in the SMS::Send::IN::eSMS code, it works out of the box provided you took care of “c” below; (b) no more hand editing C4/SMS.pm to handle the senderid parameter; and (c) you now need to create a YAML file eSMS.yaml at /etc/koha/sites/<your_instance_name>/sms_send/IN/eSMS.yaml with just the following text:

senderid: <put_your_senderid_here>

Well, that’s it! It just works!

P.S. The lawyer says we must add this – L2C2 Technologies or everyone associated with it, disclaims any and all responsibilities in the event of someone facing loss, damages either financial, operational or any problem whatsoever, due to or in course of following this blog post or any other on this blog. The information presented here is on AS-IS basis for personal educational purpose alone. This post is licensed under CC 4.0 BY-SA.

Zara hatke, zara bachke… yeh hai DataTables Meri Jaan!

A short tutorial on identifying and fixing DataTables errors arising from missing data in Koha ILS

The Problem

Recently we fielded support call from Parama (Sarkhel)-di, Librarian at Ramakrishna Sarada Mission Vivekananda Vidyabhavan. Her complaint – for a particular faculty member she was not able to see the member’s checkouts, instead it showed “Loading” and then nothing happened, even though the system was showing there were 9 items checked out to her.

RKSMVV being cloud hosted, we simply punched in the specific member’s cardnumber and then clicked on “Show checkouts” button. And voila! the error was right in front of us. Experience told us that it looked like a typical DataTables error.

What is DataTables

DataTables is a JQuery plugin for displaying information in HTML tables and adding interactions to them. It provides searching, sorting and pagination without any configuration. If you wish to learn more about it, given that Koha makes good use of the plugin, please visit DataTables examples index for a quick start.

Debugging the error

Since we expected the error to a DataTables error, our first step was to check our browser’s JavaScript console. And sure enough there was an error that said that it was triggered when a NULL value was passed to the escapeHtml() function at line number 285 inside Koha’s checkouts.js JavaScript library.

We still needed to know *what* exactly was passed to the escapeHtml() function. For that we clicked on the link to the right which pointed to line number 285 inside the checkouts.js file. As the debugger’s sources tab opened the file around line number 284, it became immediately clear the exact error. One of the checked-out items did not have a barcode assigned.

Now it was just the matter of finding out *which* of the 9 items checked out to the member did not have a barcode. First, we ran a SQL query on the instance’s issues table with the member’s borrowernumber to retrieve the itemnumbers of the checked out items, and then using this list of itemnumber we queried the items table to find out which of the 9 items had a missing barcode. The result was self explanatory.

By cross-referencing the biblionumber attached to the itemnumber we opened the offending item holding record in the edit mode from the staff client and for the time put in “FIX_BARCODE” as the temporary placeholder barcode. Immediately the member’s account showed up the table of checked out documents correctly. The member was requested to temporarily return the book so that the barcode may be fixed.

But why did it happen?

The book was lent out to the member several years back from, what is now a very, very, ancient version of Koha. At that time DataTables plugin was not a norm. About an year back the version was moved to latest version of Koha and the database updated. The error was triggered now, because this was the first time this specific member had come back to the client to borrow a book. Had she tried to issue or return a book before, the error would have been caught much before.

Since we fixed the error, we also checked the entire database of any such other cases. And sure enough there were 3 more books issued to 2 other faculty members like ages back which too did not have any barcode assigned. Parama-di noted the numbers down so that these books could be recalled back and their barcode updated.

Pro-Tip to avoid such errors

For people moving very old versions of Koha to newest versions, please run SQL queries to ensure that your all your items table’s items have homebranch, holdingbranch, itype and barcode are correctly assigned rather than having NULL or whitespace before you move the updated database into production mode.

Extracting patron images from Koha DB

This is part of the series of blog posts on how to extract your data from a Koha system where you unable to access the DB directly, but where you have “superlibrarian” access. In this part, we talk about extracting the patron images stored inside the patronimage table in your Koha database.

Prequisites

  • You should be comfortable using the GNU/Linux command line
  • Familiarity with SQL syntax
  • The Koha DB schema reference for your version of Koha
  • A Linux system with cURL and LibreOffice installed
  • Superlibrarian userid and password of your Koha instance

Modus operandi

  1. Use the Koha Reports module to download the list of the borrowernumbers of patrons having their image in the db
  2. Remove the header line and save it as a text file, with one borrowernumber per line
  3. Use cURL to get the CGISESSID cookie using the authentication SVC API call
  4. Execute the BASH script, passing the authenticated session id and the file holding the list of borrowernumbers as command line parameters
  5. Use the Koha Reports module to generate the IDLINK.txt batch patron image uploader file

The background

Patron images in Koha are stored in the patronimage table of the database. The table has following schema:

Column Type Size Parents Comments
borrowernumber int 10 borrowers the borrowernumber of the patron this image is attached to (borrowers.borrowernumber)
mimetype varchar 15   the format of the image (png, jpg, etc)
imagefile mediumblob 16777215   the image

A blob is a “binary large object” which means it is not a text field. The contents won’t make any sense if we used the report module to SELECT imagefile FROM patronimage. This brings cURL into the picture.

The patron image is served on the OPAC / staff client using the patronimage.pl Perl script using an URL like this: http://<your_instance>/cgi-bin/koha/members/patronimage.pl?borrowernumber=<borrowernumber>.

But before we can run cURL on this URL, we must do one more thing. We need our BASH script to login using our superlibrarian userid and password using Koha authentication web service API call.

We need to do this for two reasons: (a) login into our Koha instance with superlibrarian access and (b) get hold of the CGISESSID cookie.

Putting it all together

  1. A simple SQL report as given below with fetch us the borrowernumber of patrons with their image stored in the DB.

    SELECT borrowernumber FROM patronimage

    We will download the result as Open Document spreadsheet.

    Next we copy and save the list of patron borrowernumbers into a text file (let’s call it list-of-pix.txt), with one borrowernumber per line.

  2. Lets put together our small BASH script. We shall call it get_pix.sh.
    #!/bin/bash
    
    mkdir patronimages
    
    while read LINE
        do  curl -b CGISESSID=$1  -o ./patronimages/$LINE.png http://<your_instance>/cgi-bin/koha/members/patronimage.pl?borrowernumber=$LINE
    done < $2

    NOTE: <your_instance> need to be changed with **your** actual server URL.
    Make the script executable by running chmod a+x get_pix.sh.

  3. From the command-line run the following command:
    $ curl -i http://<your_instance>/cgi-bin/koha/svc/authentication -d 'userid=<yourid>&password=<yourpass>'

    If we had put in the correct information, we shall see something like this:

    HTTP/1.1 200 OK
    Date: Thu, 20 Dec 2018 09:53:02 GMT
    Server: Apache/2.4.18 (Ubuntu)
    Set-Cookie: CGISESSID=4a732ae75d8991a994d7ad0df584f84c; path=/; HttpOnly
    Vary: Accept-Encoding
    Transfer-Encoding: chunked
    Content-Type: text/xml; charset=ISO-8859-1
    
    <?xml version='1.0' standalone='yes'?>
    <response>
      <status>ok</status>
    </response>

    This gives us the two things we need for the next step: (a) the CGISESSID and (b) the status as OK. OK signifies that we were successfully logged in.

  4. Now we run the get_pix.sh script
    ./get_pix.sh f03a894f1885e891e30f6d40e6e9838c ./list-of-pix.txt

    And we shall see the patron images getting downloaded.

  5. Once the patron images have all been downloaded to patronimages folder created by the get_pix.sh script, its time to generate the IDLINK.txt file, which we will place in the aforementioned patronimages folder. Again, we shall turn to Koha's Reports module with this small SQL script:
    SELECT CONCAT(borrowers.cardnumber,", ", patronimage.borrowernumber, ".png") AS 'IDLINK'
      FROM patronimage 
      LEFT JOIN borrowers ON patronimage.borrowernumber=borrowers.borrowernumber

    This will generate our listing for the IDLINK.txt. If you are not sure about what IDLINK.txt does, please read up from here.

  6. Download the report as Open Document Spreadsheet, remove the header line and copy the rest into a text file and save it as IDLINK.txt in the patronimages folder.
  7. Zip up the patronimages folder as mentioned here and we have the patron images ready for a batch upload.

RTFM Series : Memcached and “DBI Connection failed: Access denied for user [..]”

Stumped by the Koha v 18.05’s refusal to access the database after a koha-remove followed by a koha-create using the same instance name? Then read on!

This post applies strictly to Koha 18.05 which was released on May 24, 2018. The new version with its new features has created a lot of excitement among the user. However the version has some major changes and unless you **CLOSELY** read and understand the release notes you will asking for trouble.

In this post we are going to talk about Memcached which is turned on by default from 18.05. If you overlook this fact, you may see yourself wasting hours trying to troubleshoot a problem, which may inexplicably (not quite even though it looks that way) go away after re-starting your system.

Last weekend my young friend Jayanta Nayek spent nearly a day trying to understand why he was getting the error – “DBI Connection failed: Access denied for user” whenever he tried to access the web-installer part of the staff client on his new 18.05 instance. Since he was on a test system, he had followed the old rinse-and-repeat routine. So when his installation did not work out for the first time, he ran sudo koha-remove library and then re-ran sudo koha-create --create-db library to start afresh.

When he re-created the instance i.e. “library”, he was stumped with the following error :

Software error:

DBIx::Class::Storage::DBI::catch {...} (): DBI Connection failed: Access denied for user 'koha_library'@'localhost' (using password: YES) at /usr/share/perl5/DBIx/Class/Storage/DBI.pm line 1490. at /usr/share/koha/lib/Koha/Database.pm line 103

Of course, when he tried to access the koha_library database from the mysql command-line client using the user and pass from his /etc/koha/sites/library/koha-conf.xml, it worked perfectly. But if he came back to the browser and tried to access the web-installer, the error would return.

So what was happening here? In one word – memcached! Memcached is an open source, distributed memory object caching system that alleviates database load to speed up dynamic Web applications. The system caches data and objects in memory to minimize the frequency with which an external database or API (application program interface) must be accessed. (Source: What is Memcached?).

In simple terms what this means for Koha is that memcached caches the frequent database queries fired off by Koha. And if an SQL result set has not changed since it was last queried *and* is already stored into memcached, it offers the data from in-memory hashes rather than using a more time-consuming database lookup process. Memcached (along with plack is intended to make Koha work faster under heavier loads).

When Jayanta would run sudo koha-remove library and followed it up with sudo koha-create --create-db library the Memcached server was not restarted, and kept holding on to the *original* database access hashes. Whereas following koha-create command to re-create the instance, the database authentication credentials (user & pass values from koha-conf.xml) were changed. That is why when he tried to directly access via the mysql command line client, it worked as the CLI client did not know about memcached. But when he tried to access the web-installer it failed as the connection query hash offered up from memcached had the old and not longer existing credentials.

So, if any reader of this blog should find themselves facing a problem like this, simple run the command sudo service memcached restart and once memcached has restarted, access the web-installer. It will work this time. Since memcached is an in-memory storage, the restart clears up the hashes and when the web-installer tries to access the database, it leads to what is called a “cache-miss” and thus the queries get run against the actual DB using the access credentials stored in koha-conf.xml.

And for goodness sake READ THE RELEASE NOTES ๐Ÿ˜‰

Generate a single sheet custom MARC21 framework in 2 minutes

For intermediate Koha ILS users who wish to quickly generate a single tab MARC framework.

Last Thursday, Ashish Kumar Barik, librarian at our new client-partner Midnapore City College filed a support ticket asking for a custom single sheet MARC21 framework or what is more commonly referred to by LIS professionals as a “worksheet“. He wrote that he wanted the following tags 000, 003, 005, 008, 020, 040, 041, 044, 082, 100, 245, 250, 260, 300, 440, 490, 500, 504, 650, 700, 942 and that the sub-tags/fields should be set as in the default marc framework shipped with Koha. We promised him his new framework. Being new to this side of Koha, he of course had missed out two key fields without which his system would be rendered practically useless i.e. the two local use tags952 and 999. Koha uses 952 to handle holdings (item) information and 999 is purely an internal tag used to track the bibliographic records.

Now anyone who has ever setup a new MARC framework knows that it can be a laborious and time consuming task. Further, there are chances of introducing inadvertent human errors that may lead to error or bad data when used as a part of the framework. As a result, at L2C2 Technologies we have developed several well defined strategies to manage custom marc frameworks for our clients. In today’s blog, we are going to share the simplest of the techniques we use in cases like this. The outcome of this exercise is a 100% error free marc framework generated in less than 2 minutes.

LEGAL DISCLAIMER: The next steps involve directly accessing and making changes in the the Koha database. So use these instructions at your own risk, if you face any data loss, corruption or system errors we are not responsible.

The Steps

  1. We used a regex capable editor like Notepad++ to quote the fields mentioned by Ashish, so that 000, 003, 005, 008, 020, 040, 041, 044, 082, 100, 245, 250, 260, 300, 440, 490, 500, 504, 650, 700, 942 became ‘000’, ‘003’, ‘005’, ‘008’, ‘020’, ‘040’, ‘041’, ‘044’, ‘082’, ‘100’, ‘245’, ‘250’, ‘260’, ‘300’, ‘440’, ‘490’, ‘500’, ‘504’, ‘650’, ‘700’, ‘942’. And while we did that, we also added the following fields missing in his list i.e. ‘952’, ‘999’.
  2. Next we defined a new framework MCC1 (MCC Framework) by visiting Home -> Administration -> MARC bibliographic framework -> New Framework
  3. Next we copied the default framework into MCC1 as its base, since that is what Ashish had wanted. At this point, the MCC1 framework is exactly same as the default framework of Koha.
  4. Next we fired up the MySQL console and logged in with the user id and passwd from MCC’s koha-conf.xml, and chose Ashish’s database in this case koha_mcc for the next steps.
  5. Fired the following SQL query :
    UPDATE
       `marc_subfield_structure` 
    SET
       tab=0
    WHERE 
       `frameworkcode`='MCC1' 
    AND 
       `tagfield` IN ('000', '003', '005', '008', '020', '040', '041', '044', '082', '100', '245', '250', '260', '300', '440', '490', '500', '504', '650', '700', '942')
    AND
       `tab`!=0;

    MySQL client told us 152 rows were affected.

    EXPLANATION: This moved all 1XX to 9XX (except 952 and 999) marc fields into Tab 0. The images below help illustrate the condition after this step:

  6. The next step was to set the rest of the fields outside the list supplied by Ashish *plus* 952 and 999 to be ‘ignored’ by Koha when using the MCC1 framework. And thus the following SQL query:
    UPDATE 
        `marc_subfield_structure` 
    SET 
        `tab`='-1' 
    WHERE
        `frameworkcode`='MCC1'
    AND 
        `tagfield` NOT IN ('000', '003', '005', '008', '020', '040', '041', '044', '082', '100', '245', '250', '260', '300', '440', '490', '500', '504', '650', '700', '942', '952', '999') 
    AND
        `tab`!=0;

    This time MySQL reported that 3416 rows were updated.

  7. Our last step at the MySQL command line was the following query that removed the unwanted 0XX fields from Tab 0 :
    UPDATE 
        `marc_subfield_structure` 
    SET 
        `tab`='-1' 
    WHERE
        `frameworkcode`='MCC1'
    AND 
        `tagfield` NOT IN ('000', '003', '005', '008', '020', '040', '041', '044', '082', '100', '245', '250', '260', '300', '440', '490', '500', '504', '650', '700', '942', '952', '999');

    MySQL reported 341 rows were affected.

  8. Coming back to MCC’s Koha staff client, we did the most important thing i.e. running MARC Bibliographic framework test. The test came out clean without any error.
  9. That’s it! MCC’s custom MARC framework is ready for use. Click on the image below and then zoom in to see the details up close.

Displaying unique title and volume count on the Koha staff client – Part 2

In this followup to our previous post on upgrade-friendly way of displaying unique title and copiesโ€™ count on the staff client, we explore a few variant use-cases reported by our readers.

Last December when we posted about the nifty little trick of displaying unique title and volume count on the Koha staff client, we didn’t realize that so many people would find something to be so useful. We received quite a few comments from all over the world. In this post, we are going to try and look at some of these use-cases that people thought could use something like this. So, if you are reading this and haven’t read the original post, it may be helpful to read Part 1 first.

The questions we received were:

(a) what if we have multiple branches?
(b) How to show the stats for the user’s logged in branch only?
(c) How do we show the overall totals if we have multiple branches?

Recently, the library at Our Lady Queen of the Mission School, Salt Lake had started to catalog both their junior library as well as the senior library. So, they presented the perfect opportunity to showcase these 3 use-cases.

1. What if we have multiple branches?

This was a question that came from Hussein Al-Nasri from Egypt. Hussein had tried the code snippet, but since it was hard-coded to handle only one single branch (the first on in the list of the JSON data returned), it wasn’t showing him what he wanted – the data for each of his branches.

The solution
$(document).ready(function() {
  if ( $('#main_intranet-main').length ) {
  $.getJSON("https://qmsl-staff.l2c2.co.in/cgi-bin/koha/svc/report?id=1&annotated=1.", function(data) {
    if ( data.length ) {
      $('#news1').prepend('<div class="newsitem" id="mystats"><table class="table table-striped" style="width: 100%; background: none;"><thead><th colspan="3" style="text-align: center; font-weight: bold; padding: 8px; line-height: 1.42857143; vertical-align: middle; text-transform: uppercase;">Library Statistics</thead><tbody><tr id="mystatstb"><td><strong>Branch</strong></td><td><strong>Unique titles</strong></td><td><strong>Total Copies</strong></td></tr></tbody></table></div>');
      for ( var key in data ) {
        $('<tr id=\"tr'+ key + '"><td class="text-center">' + data[key].homebranch + '</td><td class="text-center">' + data[key].bibs + '</td><td class="text-center">' + data[key].items + '</td></tr>').insertAfter( $( '#mystatstb' ) );
      }
    }
  });
  }
});

2. How to show stats only for the logged in branch?

This came from Freddy Enrique Pelayo from Peru, South America. He left a comment on the blog post asking:

if the system were to have more than 2 branches, will the screen show information according to the branch the user logged in?

The solution
$(document).ready(function() {
  if ( $('#main_intranet-main').length ) {
  $.getJSON("https://qmsl-staff.l2c2.co.in/cgi-bin/koha/svc/report?id=1&annotated=1.", function(data) {
    if ( data.length ) {
      $('#news1').prepend('<div class="newsitem" id="mystats"><table class="table table-striped" style="width: 100%; background: none;"><thead><th colspan="3" style="text-align: center; font-weight: bold; padding: 8px; line-height: 1.42857143; vertical-align: middle; text-transform: uppercase;">Library Statistics</thead><tbody><tr id="mystatstb"><td><strong>Branch</strong></td><td><strong>Unique titles</strong></td><td><strong>Total Copies</strong></td></tr></tbody></table></div>');
      for ( var key in data ) {
        if ( $('#logged-in-branch-code').html() == data[key].homebranch ) {
          $('<tr id=\"tr'+ key + '"><td class="text-center">' + data[key].homebranch + '</td><td class="text-center">' + data[key].bibs + '</td><td class="text-center">' + data[key].items + '</td></tr>').insertAfter( $( '#mystatstb' ) );
        }
      }
    }
  });
  }
});

Explanation

The key line here is $('#logged-in-branch-code').html() == data[key].homebranch. Whenever an user logs into the Koha staff client, the hidden <span> element with the id logged-in-branch-code holds the code for the user’s logged in branch. In the above snippet, we simply introduce a check to see if the code matches the homebranch code in the JSON array. If it does, we show the value for that branch and not for the other branches.

3. How to show the total for all branches?

This scenario was pointed to by Dr. Apurba Jyoti Mazumdar, Assistant Librarian, Assam Univerity, Silchar, India.

The solution

$(document).ready(function() {
    if ( $('#main_intranet-main').length ) {
    $.getJSON("https://qmsl-staff.l2c2.co.in/cgi-bin/koha/svc/report?id=1&annotated=1.", function(data) {
        if ( data.length ) {
            $('#news1').prepend('<div class="newsitem" id="mystats"><table class="table table-striped" style="width: 100%; background: none;"><thead><th colspan="3" style="text-align: center; font-weight: bold; padding: 8px; line-height: 1.42857143; vertical-align: middle; text-transform: uppercase;">Library Statistics</thead><tbody><tr id="mystatstb"><td><strong>Branch</strong></td><td><strong>Unique titles</strong></td><td><strong>Total Copies</strong></td></tr></tbody></table></div>');
            var totalbibs = 0;
            var totalitems = 0;
            for ( var key in data ) {
                $('<tr id=\"tr'+ key + '"><td class="text-center">' + data[key].homebranch + '</td><td class="text-center">' + data[key].bibs + '</td><td class="text-center">' + data[key].items + '</td></tr>').insertAfter( $( '#mystatstb' ) );
                 totalbibs = totalbibs + parseInt(data[key].bibs);
                 totalitems = totalitems +  parseInt(data[key].items); 
            }
            $( '<tr><td><strong>TOTAL</strong></td><td class="text-center"><strong>' + totalbibs + '</strong></td><td  class="text-center"><strong>' + totalitems + '</strong></td></tr>' ).insertAfter(  $('#mystats tr:last')  ); 
        }
    });
    }
});

Conclusion

As you may have noticed, in this version, we added an extra check which ensured that we only display this grid of data *only if* any data is returned. We hope that by answering a few of your questions, this post is of some use to some of you.