koha-testing-docker (a.k.a. KTD)

This project provides a dockered solution for running a Koha ILS development environment inside Docker containers.

It is built using the package install with the needed tweaks (including koha-gitify) in order to create such environment.

The docker-compose.yml file is self explanatory.

Requirements

Software

This project is self contained and all you need is:

• A text editor to tweak configuration files
• Docker (install instructions)
• Docker Compose v2 (install instructions)

Note: Windows and macOS users use Docker Desktop which already ships Docker Compose v2.

Hardware

• At least 2.6 GiB of free RAM (not counting web browser)
• If you want to try Elastic, count at least 2 GiB more of free RAM.

Setup

It is not a bad idea to organize your projects on a directory. For the purpose of simplifying the instructions we will pick ~/git as the place in which to put all the repository clones:

mkdir -p ~/git
export PROJECTS_DIR=~/git

• Clone the koha-testing-docker project:

cd $PROJECTS_DIR
git clone https://gitlab.com/koha-community/koha-testing-docker.git koha-testing-docker

• Clone the koha project (skip and adjust the paths if you already have it):

cd $PROJECTS_DIR
# be patient, it's a >3GiB download
git clone https://git.koha-community.org/Koha-community/Koha.git koha

• Set some mandatory environment variables:

echo "export PROJECTS_DIR=$PROJECTS_DIR" >> ~/.bashrc
echo 'export SYNC_REPO=$PROJECTS_DIR/koha' >> ~/.bashrc
echo 'export KTD_HOME=$PROJECTS_DIR/koha-testing-docker' >> ~/.bashrc
echo 'export PATH=$PATH:$KTD_HOME/bin' >> ~/.bashrc
echo 'export LOCAL_USER_ID=$(id -u)' >> ~/.bashrc

Note: you will need to log out and log back in (or start a new terminal window) for this to take effect.

• Generate your personal .env file:

cd $PROJECTS_DIR/koha-testing-docker
cp env/defaults.env .env

Basic usage

In order to launch KTD, you can use the ktd wrapper command. It is a wrapper around the docker compose command so it accepts its parameters:

• Starting:

ktd up

• Get into the Koha container shell (instance user)

ktd --shell

• Get into the Koha container shell (root user)

ktd --root --shell

• Watching the koha container logs

ktd --logs

• Updating the used images:

ktd pull

• Shutting it down

ktd down

• Adding services to our stack

Several option switches are provided for more fine-grained control:

ktd --es7 up
ktd --selenium --os7 --plugin --sso up
...

Note: the pull command would also work if you add several option switches. So running:

ktd --es7 pull

will also download/update the Elasticsearch 7.x image to be used.

For a complete list of the option switches, run the command with the --help option:

ktd --help

Getting to the web interface

The IP address of the web server in your docker group will be variable. Once you are in with SSH, issuing a

ip a

should display the IP address of the webserver. At this point the web interface of Koha can be accessed by going to http://:8080 for the OPAC http://:8081 for the Staff interface.

Available commands and aliases

The Koha container ships with some aliases to improve productivity. They are divided in two, depending on the user for which the alias is defined.

Aliases for the instance user require that you start a shell with that user in order to be used. This is done like this:

ktd --shell

or, from the root user using kshell:

ktd --root --shell
kshell

root user

• koha-intra-err: tail the intranet error log
• koha-opac-err: tail the OPAC error log
• koha-plack-log: tail the Plack access log
• koha-plack-err: tail de Plack error log
• kshell: get into the instance user, on the kohaclone dir
• koha-user: get the db/admin username from koha-conf.xml
• koha-pass: get the db/admin password from koha-conf.xml
• dbic: recreate the schema files using a fresh DB. Accepts the --force parameter
• flush_memcached: Flush all key/value stored on memcached
• restart_all: restarts memcached, apache and plack
• reset_all: Drop and recreate the koha database [*]
• reset_all_marc21: Same as reset_all, but forcing MARC21
• reset_all_unimarc: Same as reset_all, but forcing UNIMARC
• start_plack_debug: Start Plack in debug mode, trying to connect to a remote debugger if set.
• updatedatabase: Run the updatedatabase.pl script in the right context (instance user)

Note: it is recommended to run start_plack_debug on a separate terminal because it doesn't free the prompt until the process is stopped.

[*] reset_all actually:

• Drops the instance's database, and creates an empty one.
• Calls the misc4dev/do_all_you_can_do.pl script.
• Populates the DB with the sample data, using the configured MARC flavour.
• Create a superlibrarian user.
• Updates the debian files in the VM (overwrites the ones shipped by the koha-common package).
• Updates the plack configuration file for the instance.
• Calls restart_all

kohadev user

• qa: Run the QA scripts on the current branch. For example: qa -c 2 -v 2
• prove_debug: Run the prove command with all parameters needed for starting a remote debugging session.

Running the right branch

By default the KTD that will start up is configured to work for the master branch of Koha. If you want to run an image to test code against another koha branch you should use the KOHA_IMAGE environment variable before starting the image as above.

KOHA_IMAGE=21.05 ktd up

Please note that you can only use branches defined here. If you want to work on a local feature branch in Koha, make sure that SYNC_REPO points to the correct directory on your machine, and that you are in the correct branch there. Please also note that the Koha sources are installed to /kohadevbox/koha (via koha-gitify) and not /usr/share/koha!

Advanced usage

Docker parameters

With some exceptions (when using --shell or --logs) the ktd script is mostly a wrapper for the docker compose tool. So all trailing options after the shipped option switches will be passed to the underlying docker compose command.

For example, if you want to run KTD in daemon mode, so it doesn't take over the terminal or die if you close it, you can run it like this:

ktd <options> up -d

where <options> are the valid ktd option switches. If your usage requires more options you should check docker compose --help or refer to the Docker compose documentation.

Keycloak / SSO

The --sso option switch will make ktd run a Keycloak server for testing OIDC, SAML, etc.

Keycloak will run on port 8082 and you will be required to set an alias on /etc/hosts like this:

127.0.1.1	sso

It will then be accessible on the following URL: http://sso:8082/auth.

If you need more information on how to set it up, refer to the Keycloak manual.

Notes

• When you configure keycloak as an identity provider in Koha

  1. The code you choose will be part of the URI you need to enter in Valid redirect URIs when you configure Koha as client in Keycloak. So if the code you choose is kc, the valid redirect URIs you need to enter are http://kohadev.mydnsname.org:8080/api/v1/public/oauth/login/kc/opac for opac, and http://kohadev-intra.myDNSname.org:8081/api/v1/public/oauth/login/kc/staff for staff interface
  2. Choose OIDC protocol, click on Add default OIDC configuration and in the well_known_url parameter put the following: http://sso:8082/auth/realms/master/.well-known/openid-configuration

• When you configure Koha as a client in Keycloak you should enable the Exclude Session State From Authentication Response in the Advanced settings, because Koha does not support yet the session_state parameter

Aliases

This project includes some handy aliases for easy startup, opening a shell inside the Koha container and stopping everything:

Alias	Function
ku	Start the whole thing, using MariaDB 10.1 with Debian 9
kul	Light mode (no ES, no nothing)
ku-es5	As above, plus ES5
ku-es6	As above, replacing ES5 with ES6
ku-es7	As above, replacing ES6 with ES7
ku-mdb	Start the whole thing, using latest MariaDB with Debian 9
ku-md9	Start the whole thing, using MariaDB matched to Debian 9
ku-md10	Start the whole thing, using MariaDB matched to Debian 10
ku-my8	Start the whole thing, using latest MySQL with Debian 9
kp	Start the whole thing, with mysql persistence
kup	Start the env, plugin development set 1 2
kk	Start the whole thing, with kibana
kpk	Start the whole thing, with mysql persistence and kibana
kd	Stop the whole thing
kshell	Opens a shell inside the Koha container

In order to use this aliases you need to edit your ~/.bashrc ( or ~/.profile if using Git for Windows ) file adding:

echo 'source ${KTD_HOME}/files/bash_aliases' >> ~/.bashrc

Note: If you are using Docker Compose V2 use this command instead:

echo 'source ${KTD_HOME}/files/bash_aliases_v2' >> ~/.bashrc

Manually

docker-compose -p koha up

Alternatively, you can have it run all the tests and exit, like this:

export RUN_TESTS_AND_EXIT="yes"
# Optionally you can add COVERAGE=1 so the tests generate coverage data
# Optionally you can add CPAN=1 to pull the latest versions of perl dependancies directly from cpan
docker-compose -p koha up --abort-on-container-exit

Update images

docker-compose -f docker-compose.yml pull

Database persistence

If you need to keep the DB between your different uses of the containers, you can run

docker-compose -f docker-compose.yml -f docker-compose.persistent.yml -p koha up

Alias: kp

Kibana

If you would like to use Kibana for testing/interacting with ES directly you can include an extra compose file

docker-compose -f docker-compose.yml -f docker-compose.kibana.yml -p koha up

Alias: kk

It is possible to combine this with persistence

docker-compose -f docker-compose.yml -f docker-compose.persistent.yml -f docker-compose.kibana.yml -p koha up

Alias: kpk

Having Elasticsearch run

In order for Elasticsearch to run, changes to the host OS need to be made. Please read the official docs

TL;DR

Increase vm.max_map_count kernel setting to at least 262144:

• On Linux:

# Increase vm.max_map_count
sudo sysctl -w vm.max_map_count=262144
# Make it permanent
echo "vm.max_map_count=262144" | sudo tee -a /etc/sysctl.conf

• On MacOS:

screen ~/Library/Containers/com.docker.docker/Data/vms/0/tty
# login with root and no password
sysctl -w vm.max_map_count=262144

If the screen command doesn't work try: find ~/Library/Containers/com.docker.docker/Data/ -name 'tty'

Running commands inside KTD from the host

Docker compose V2: docker exec -ti koha-koha-1 /bin/bash -c "source /root/.bashrc && restart_all"

Docker compose V1: docker exec -ti koha_koha_1 /bin/bash -c "source /root/.bashrc && restart_all"

Problems?

If you see the following error on 'ku' after initial setup, try a reboot

ERROR: Couldn't connect to Docker daemon at http+docker://localhost - is it running?

If starting fails with "database not empty", try running ktd down or kd

It's likely that last start of KTD failed and needs cleanup. Or that it was shutdown without ktd down or kd that are necessary for a clean shutdown.

---

1. You need to export the PLUGIN_REPO variable, with the full path to the plugin dir. It will fail to load if you don't export the variable first. ↩︎

2. Once started, you need to edit the kohadev koha-conf commenting the pluginsdir default and uncommenting the kohadev lines and then load the plugin using kshell ./misc/devel/install_plugins.pl ↩︎