koha.git
8 months agoBug 29002: Add bookable state to items
Martin Renvoize [Wed, 10 Nov 2021 11:42:17 +0000 (11:42 +0000)]
Bug 29002: Add bookable state to items

This patch adds a bookable boolean to enable/disable the ability to book
an item ahead of time

Test plan
1) Navigate to the 'Items' tab of a biblio
2) Note the new 'Bookable' option and select at least one item to allow
   bookings to take place
3) Note that without any items selected as 'bookable' one does not have
   the 'Place booking' option or the 'Bookings' tab on the biblio
   details page.
4) Note that when at least one item is bookable, the place booking modal
   now only displays items that are marked as bookable in the item
   selection
5) Sign off

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Janet McGowan <janet.mcgowan@ptfs-europe.com>
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Laurence Rault <laurence.rault@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 29002: Use vis-timeline in bookings view
Martin Renvoize [Mon, 15 May 2023 15:56:24 +0000 (16:56 +0100)]
Bug 29002: Use vis-timeline in bookings view

This patch uses vis-timeline to display a timeline of bookings on the
booking details display tab.

Test plan
1) Follow the steps from the previous patch test plans
2) Note the new visual timeline view of bookings on the bookings detail
   page.
3) Confirm that clicking the red cross next to a booking triggers the
   cancellation modal
4) Sign off

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Janet McGowan <janet.mcgowan@ptfs-europe.com>
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Laurence Rault <laurence.rault@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 29002: Add vis-timeline
Martin Renvoize [Mon, 15 May 2023 15:55:44 +0000 (16:55 +0100)]
Bug 29002: Add vis-timeline

This is a preparitory patch to add the vis-timeline javascript library.
We will use if to display bookings in a timeline view on the bookings
page.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Janet McGowan <janet.mcgowan@ptfs-europe.com>
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Laurence Rault <laurence.rault@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 29002: Add modal based booking cancellation
Martin Renvoize [Wed, 8 Dec 2021 14:59:17 +0000 (14:59 +0000)]
Bug 29002: Add modal based booking cancellation

This patch adds a modal based booking cancellation flow to the biblio
bookings display tab page.

Test plan
1) Follow the test plans for the previous patches
2) Note a new column now appears in the bookings table with a 'Cancel'
   action button against each booking
3) Cancel one of the bookings using the new cancel button
4) A modal dialog should appear allowing you to continue with the
   cancelation or cancel
5) Confirm the cancelled booking is removed from the bookings table
6) Sign off

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Janet McGowan <janet.mcgowan@ptfs-europe.com>
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Laurence Rault <laurence.rault@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 29002: Fix 'Place booking' for cat pages
Martin Renvoize [Thu, 18 May 2023 13:52:58 +0000 (14:52 +0100)]
Bug 29002: Fix 'Place booking' for cat pages

We added the 'Place booking' option to the cat-toolbar, but it require
includes for select2 and calendar which weren't present everywhere..
This patch corrects that.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Janet McGowan <janet.mcgowan@ptfs-europe.com>
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Laurence Rault <laurence.rault@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 29002: Add bookings view
Martin Renvoize [Mon, 15 May 2023 15:30:26 +0000 (16:30 +0100)]
Bug 29002: Add bookings view

This patch adds a new view to the biblio details page to display any
bookings that may have been made.

Test plan
1) Run through the steps from the previous patch
2) Note the appearance of a new 'Bookings' option in the left menu
3) Click the bookings option to load the new bookings page for the
   biblio
4) See that the booking from the previous patch appears on the page
5) Sign off

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Janet McGowan <janet.mcgowan@ptfs-europe.com>
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Laurence Rault <laurence.rault@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 29002: Add ability to book material from the staff client
Martin Renvoize [Wed, 29 Sep 2021 14:17:38 +0000 (15:17 +0100)]
Bug 29002: Add ability to book material from the staff client

This patch introduces a new modal to the biblio details page to allow
booking of materials.

Test plan
1) Navigate to the details page of a biblio
2) Note the new 'Place booking' button in the toolbar
3) Click the new button and note the new modal dialogue
4) Enter part of a patron name or cardnumber and then select from the
   presented results
5) Optionally pick an item from the select list
6) Select a start date and end date from the calender
7) Submit
8) Attempt to book the same item to another user, note that the dates
   previously selected are now greyed out.
9) Experiment with different items and all items options to confirm the
   available slots in the datepicker update as expected.
10) Sign off

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Janet McGowan <janet.mcgowan@ptfs-europe.com>
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Laurence Rault <laurence.rault@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 29002: Add bookings objects and API classes
Martin Renvoize [Thu, 16 Sep 2021 08:17:30 +0000 (09:17 +0100)]
Bug 29002: Add bookings objects and API classes

This patch adds new Koha::Object based classes for bookings logic and
adds API controllers to expose the new bookings data via the REST API's.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Janet McGowan <janet.mcgowan@ptfs-europe.com>
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Laurence Rault <laurence.rault@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 29002: DBIC Schema Build
Martin Renvoize [Wed, 15 Sep 2021 15:30:15 +0000 (16:30 +0100)]
Bug 29002: DBIC Schema Build

Rebuild of DBIx::Class schema files for the new table and relations
added.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Janet McGowan <janet.mcgowan@ptfs-europe.com>
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Laurence Rault <laurence.rault@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 29002: Add bookings table
Martin Renvoize [Wed, 15 Sep 2021 15:29:43 +0000 (16:29 +0100)]
Bug 29002: Add bookings table

This patch adds a new bookings table to store booking details.

Test plan
* Confirm that kohastructure and the atomicupdate match such that an
  install or upgrade result in the same table being present on the
  system.

Sponsored-by: PTFS-Europe
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Janet McGowan <janet.mcgowan@ptfs-europe.com>
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Laurence Rault <laurence.rault@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 8367: (QA follow-up) Fix broken tests in Reserves.t
Tomas Cohen Arazi [Thu, 2 Nov 2023 13:39:24 +0000 (10:39 -0300)]
Bug 8367: (QA follow-up) Fix broken tests in Reserves.t

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 8367: (QA follow-up) Fix broken tests
Tomas Cohen Arazi [Thu, 2 Nov 2023 13:22:03 +0000 (10:22 -0300)]
Bug 8367: (QA follow-up) Fix broken tests

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 18203: DBRev 23.06.00.060
Tomas Cohen Arazi [Wed, 1 Nov 2023 20:56:34 +0000 (17:56 -0300)]
Bug 18203: DBRev 23.06.00.060

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 18203: DBIC schema
Tomas Cohen Arazi [Wed, 1 Nov 2023 20:52:04 +0000 (17:52 -0300)]
Bug 18203: DBIC schema

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 18203: Rename column
Tomas Cohen Arazi [Wed, 1 Nov 2023 20:49:03 +0000 (17:49 -0300)]
Bug 18203: Rename column

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 18203: Add per borrower category restrictions on placing ILL requests
Pedro Amorim [Wed, 12 Jul 2023 09:19:15 +0000 (09:19 +0000)]
Bug 18203: Add per borrower category restrictions on placing ILL requests

    Test Plan:
    - Enable ILLModule sys pref and install any backend, or run
    bash <(curl -s https://raw.githubusercontent.com/ammopt/koha-ill-dev/master/start-ill-dev.sh)
    - Verify you can place new ILL requests in OPAC and Intranet
    - Apply patch
    - In borrower categories, verify there's a new column for 'can place ILL in opac' and is set to 'yes' by default
    - Edit your borrower's patron category and set 'can place ILL in opac' to 'No'
    - Verify you can no longer place new ILL requests in OPAC
    - Verify you also cannot place new ILL requests through URL:
    :8080/cgi-bin/koha/opac-illrequests.pl?method=create&backend=FreeForm

Signed-off-by: Sam Lau <samalau@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 18203: DB specific and atomicupdate
Pedro Amorim [Wed, 12 Jul 2023 09:15:03 +0000 (09:15 +0000)]
Bug 18203: DB specific and atomicupdate

Signed-off-by: Sam Lau <samalau@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 8367: DBRev 23.06.00.059
Tomas Cohen Arazi [Wed, 1 Nov 2023 20:39:12 +0000 (17:39 -0300)]
Bug 8367: DBRev 23.06.00.059

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 8367: (follow-up) Fix for circ rules editor
Aleisha Amohia [Wed, 18 Oct 2023 01:38:54 +0000 (01:38 +0000)]
Bug 8367: (follow-up) Fix for circ rules editor

Make sure holds pickup period field is empty when editing a 'default' rule.

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 8367: (QA follow-up) Fix QA script
Katrin Fischer [Mon, 9 Oct 2023 21:51:08 +0000 (21:51 +0000)]
Bug 8367: (QA follow-up) Fix QA script

* Add +x to atomic database update file
* Perltidy
* Add spans to rewritten tab label for translatability

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 8367: Add holds_pickup_period fallback circulation rule
Aleisha Amohia [Sun, 30 Apr 2023 22:20:23 +0000 (22:20 +0000)]
Bug 8367: Add holds_pickup_period fallback circulation rule

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 8367: (QA follow-up) Improve display when value not set for rule
Nick Clemens [Mon, 6 Mar 2023 20:03:14 +0000 (20:03 +0000)]
Bug 8367: (QA follow-up) Improve display when value not set for rule

When a rule is not set for a speficic line, the value in the syspref
will be used. This should be made clear in the interface.

To test:
1 - Add a new rule for a specific item type
2 - Do not fill the 'Hold pickup period'
3 - Confirm that the rule defaults to the system preference
4 - Hover on 'Default' in the circ rules under 'Hold pickup period'
5 - Confirm the text is readable

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 8367: (QA follow-up) Update message on Hold waiting pickup report
Nick Clemens [Mon, 6 Mar 2023 15:50:13 +0000 (15:50 +0000)]
Bug 8367: (QA follow-up) Update message on Hold waiting pickup report

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
https://bugs.koha-community.org/show_bug.cgi?id=8367
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 8367: (follow-up) possible to delete circ rule
Aleisha Amohia [Fri, 13 Jan 2023 01:39:50 +0000 (14:39 +1300)]
Bug 8367: (follow-up) possible to delete circ rule

Signed-off-by: Emmi Takkinen <emmi.takkinen@koha-suomi.fi>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 8367: (follow-up) Remove unnecessary code and atomic update
Aleisha Amohia [Tue, 1 Nov 2022 19:48:19 +0000 (08:48 +1300)]
Bug 8367: (follow-up) Remove unnecessary code and atomic update

Signed-off-by: Emmi Takkinen <emmi.takkinen@koha-suomi.fi>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 8367: (follow-up) Fix tests
Aleisha Amohia [Mon, 14 Mar 2022 03:34:54 +0000 (16:34 +1300)]
Bug 8367: (follow-up) Fix tests

Signed-off-by: Emmi Takkinen <emmi.takkinen@koha-suomi.fi>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 8367: (follow-up) Remove pickup delay message from OPAC
Aleisha Amohia [Mon, 14 Mar 2022 00:59:37 +0000 (13:59 +1300)]
Bug 8367: (follow-up) Remove pickup delay message from OPAC

It adds unnecessary complexity and information.

Signed-off-by: Emmi Takkinen <emmi.takkinen@koha-suomi.fi>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 8367: (follow-up) Fix pickup delay text on OPAC and other bits
Aleisha Amohia [Wed, 2 Mar 2022 21:03:28 +0000 (10:03 +1300)]
Bug 8367: (follow-up) Fix pickup delay text on OPAC and other bits

Signed-off-by: Emmi Takkinen <emmi.takkinen@koha-suomi.fi>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 8367: Make database update idempotent (across different MariaDB versions)
David Cook [Tue, 21 Sep 2021 05:21:42 +0000 (05:21 +0000)]
Bug 8367: Make database update idempotent (across different MariaDB versions)

The previous query seemed to work on MariaDB 10.1.48 (used by koha-testing-docker),
but it didn't work on MariaDB 10.3.29 (default in Debian 10 as of this writing),
but this patched query works on both indempotently.

Signed-off-by: Emmi Takkinen <emmi.takkinen@koha-suomi.fi>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 8367: Make database update idempotent
David Cook [Tue, 7 Sep 2021 07:09:43 +0000 (07:09 +0000)]
Bug 8367: Make database update idempotent

Signed-off-by: Emmi Takkinen <emmi.takkinen@koha-suomi.fi>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 8367: Add holds_pickup_period circulation rule
Aleisha Amohia [Mon, 20 Apr 2020 16:13:35 +0000 (16:13 +0000)]
Bug 8367: Add holds_pickup_period circulation rule

So that pickup delay can have a different value per patron category,
   item type or branch.

To test:

1) Update database, restart services
2) Set ReservesMaxPickUpDelay syspref (if not already set)
3) Edit your circulation rules and set a value under 'Holds pickup
period (day) that is DIFFERENT from ReservesMaxPickUpDelay. Set a few
different numbers for different branches as well.
4) Place a hold on a biblio from the staff client.
5) Check in an item from that biblio and confirm the hold as waiting
6) Confirm the expiration date is calculated using the 'Holds pickup
period' value instead of the ReservesMaxPickUpDelay syspref
7) Revert the waiting status and delete the hold
8) Re-place the hold on the biblio on the OPAC. Notice that when you
change the pick up location, the number of days in the pickup message
below the dropdown changes based on the circ rules.
9) Create a holiday with a date that will overlap with the 'Holds pickup
period'
10) Check in an item from that biblio and confirm the hold as waiting
11) Confirm the expiration date is calculated using the 'Holds pickup
period' value AND considers the special holiday
12) Confirm tests pass t/db_dependent/Holds/WaitingReserves.t
13) Test Talking Tech:
13a) Enable TalkingTechItivaPhoneNotification
13b) Go to Tools -> Notices & slips. Add content to the HOLD phone
(itiva) notice.
13c) In your terminal, run perl
/path/to/koha/misc/cronjobs/thirdparty/TalkingTech_itiva_outbound.pl -o
~/itiva.tmp -w 0 --type=RESERVE

Sponsored-by: Catalyst IT
Signed-off-by: Emmi Takkinen <emmi.takkinen@koha-suomi.fi>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34098: Improve translation of some strings in the patron import template
Owen Leonard [Thu, 22 Jun 2023 15:42:52 +0000 (15:42 +0000)]
Bug 34098: Improve translation of some strings in the patron import template

This patch updates a few sections of the patron import template in order
to import the way strings are identified for translation.

To test, apply the patch and install another translation if necessary.

- Go to Tools -> Import patrons
- Under "Notes," you should see a link, "Download a Starter CSV file
  with all the columns."
- Test the process of importing a batch of patron records.
  - When the import completes, confirm that the status messages are
    correct:

    X imported records
    X overwritten records
    X not imported because they are already in borrowers table and
      overwrite is disabled
    X not imported because they are not in the expected format
    X records parsed

TESTING TRANSLATABILITY

- Update a translation, e.g. fr-FR:

> cd misc/translator
> perl translate update fr-FR

- Open the corresponding .po file for the updated strings, e.g.
  misc/translator/po/fr-FR-messages.po
- Locate strings pulled import_borrowers.tt for translation and confirm
  that they are readable and have context information under 'msgctxt'

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 35091: Improve translation of usage statistics country list
Owen Leonard [Wed, 18 Oct 2023 14:18:40 +0000 (14:18 +0000)]
Bug 35091: Improve translation of usage statistics country list

This patch reworks country-list.inc so that each country name can be
wrapped separately in the t() translation function and be processed
individually during the translation process.

To test, apply the patch and update a translation, e.g. fr-FR:
   - perl misc/translator/translate update fr-FR

- Edit the updated po file for the language you chose, e.g.
  misc/translator/po/fr-FR-messages.po, and locate strings from
  country-strings.inc. Each country names should show up as a separate
  string for translation.
- Update the file to fill in translated strings for one or more
  countries. For example:

  koha-tmpl/intranet-tmpl/prog/en/includes/country-list.inc:2
  msgid "Afghanistan"
  msgstr ""

- Install the updated po file:
   - perl misc/translator/translate install fr-FR
- Enable the translation if necessary under Administration -> System
  preferences -> language.
- Go to Administration -> Share usage statistics and check the "Your
  country" dropdown. The list should look correct.
- Switch to your updated translation and confirm that the country names
  you translated show up as expected.

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 26824: Use confirmation modal when removing titles from a list in the OPAC
Owen Leonard [Fri, 20 Oct 2023 18:18:38 +0000 (18:18 +0000)]
Bug 26824: Use confirmation modal when removing titles from a list in the OPAC

This patch changes the process of removing a title from a list so that
it uses a confirmation modal dialog instead of a JavaScript alert.

To test, apply the patch and view the contents of a list in the OPAC.

 - Click the "Remove from this list" link under one of the titles.
 - You should see a modal confirmation message, "Are you sure you want
   to remove this item from the list?" It should show the title of the
   item on the list.
 - Test both the "Yes" and "No" choices.

 - Check the box next to one title on the list, and click the
   "Remove from list" link at the top of the table.
 - You should see a modal confirmation message, "Are you sure you want
   to remove this item from the list?" It should show the title of the
   item on the list.
 - Test boh the "Yes" and "No" choices.

 - Check the box next to multiple titles on the list and click the
   "Remove from list" link at the top of the table.
 - You should see a modal confirmation message, "Are you sure you want
   to remove these items from the list?" It should show the titles of
   all the records you selected.
 - Test both the "Yes" and "No" choices.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 33575: (QA follow-up) Fix date sorting on Date column
Katrin Fischer [Tue, 9 May 2023 22:11:00 +0000 (22:11 +0000)]
Bug 33575: (QA follow-up) Fix date sorting on Date column

Implements correct date sorting for different date formats on the Date column.

See:
https://wiki.koha-community.org/wiki/DataTables_HowTo#Sorting_dates_regardless_of_date_format_preference
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 33575: (follow-up) don't allow priority and pickup location to be hidden
Lucas Gass [Fri, 21 Apr 2023 14:10:51 +0000 (14:10 +0000)]
Bug 33575: (follow-up) don't allow priority and pickup location to be hidden

Signed-off-by: Sally <sally.healey@cheshiresharedservices.gov.uk>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 33575: Add dataTable to hold table on /reserve/request.tt
Lucas Gass [Thu, 20 Apr 2023 15:47:01 +0000 (15:47 +0000)]
Bug 33575: Add dataTable to hold table on /reserve/request.tt

To test:
1. Have a record with several holds on it.
2. Go to the holds page for that record ( /cgi-bin/koha/reserve/request.pl?biblionumber=144 )
3. Cannot sort by columns
4. Apply patch, restart_all
5. Now the table should be a dataTable that will allow you sort.
6. Try hiding columns by clicking on the 'Columns' icon above the table, make sure columns are correctly being hidden.
7. Try to hide columns by going to Adminstration > Table settings ( Circulation > Holds > patron_holds_table ), make sure columns are correctly being hidden.
8. Now login with a user who has the 'place_holds' permission but does NOT have the 'modify_holds_priority' permssion.
9. The table will not have the change priority columns. Make sure all columns can still be hidden/shown correctly for that use.

Signed-off-by: Sally <sally.healey@cheshiresharedservices.gov.uk>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34834: Add translation context for "Order"
Caroline Cyr La Rose [Wed, 20 Sep 2023 14:24:02 +0000 (10:24 -0400)]
Bug 34834: Add translation context for "Order"

This patch adds context for translation to the term "Order". It is
sometimes used as a verb (e.g. to order) and sometimes as a noun (e.g.
an order).

To test:
1. Go to the various pages and make sure Order appears correctly in
   English

   -  orderreceive: when receiving multiple orders, the table heading
   -  parcel: when receiving an order, there is a "View" menu with
      "Order", "MARC", and "Card"
   -  z3950_search: when adding an order from an external source, if
      you hover the mouse on the Order option, there is a thing that
      opens (a tooltip?) with the word "Order"

2. Apply patch

3. Redo step 1 in English, make sure the pages still work

4. Update the translation files

   gulp po:update --lang fr-CA

5. View the messages.po file and make sure the term is translated for
   both verb and noun context (you can put anything, just something
   different to tell them apart)

6. View the pages in the other language, the terms should be

   -  orderreceive: noun
   -  parcel: noun
   -  z3950_search: verb

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 35079: Replace --force-extract by --generate-pot={always,auto,never}
Julian Maurice [Tue, 17 Oct 2023 15:13:55 +0000 (17:13 +0200)]
Bug 35079: Replace --force-extract by --generate-pot={always,auto,never}

This restores the original behaviour of always building the POT file
in order to not break existing workflows
Option --force-extract is deleted in favor of a new option
--generate-pot that can have 3 values:

* always: always build the POT file. This is the default value
* auto: build the POT file only if it does not exist
* never: never build the POT file

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 35079: Rebuild POT files only if necessary or asked explicitely
Julian Maurice [Tue, 17 Oct 2023 14:33:37 +0000 (16:33 +0200)]
Bug 35079: Rebuild POT files only if necessary or asked explicitely

By default, gulp tasks po:create and po:update won't rebuild the POT
files if they already exist.
Both tasks gained a new option --force-extract to rebuild them
unconditionally

This makes it possible to create/update PO files for multiple languages
sequentially without rebuilding the POT file for each language.
For instance:

    gulp po:update --lang fr-FR
    # do something with the PO files
    gulp po:update --lang en-GB
    # ...

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34456: (QA follow-up) perltidy code
Kyle M Hall [Mon, 30 Oct 2023 17:31:47 +0000 (13:31 -0400)]
Bug 34456: (QA follow-up) perltidy code

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34456: (QA follow-up) Change download link language
Kyle M Hall [Mon, 30 Oct 2023 17:29:52 +0000 (13:29 -0400)]
Bug 34456: (QA follow-up) Change download link language

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34456: Add the ability to download a template rendered report as a file
Kyle M Hall [Wed, 2 Aug 2023 11:56:46 +0000 (07:56 -0400)]
Bug 34456: Add the ability to download a template rendered report as a file

Bug 34136 adds the ability to render a report using a notice template rather than displaying data in a table. It would be even more useful to be able to download the contents as a file where the notice subject can be used as the filename so the filename can be generated dynamically ( such as adding the current date as part of the filename ).

Test Plan:
1) Follow the test plan for Bug 34136
2) Run your report
3) Note under the Download menu the new item "Download as" with the
   subject line for your template
4) Click that link, note the file contains the contents of your report!

Signed-off-by: Sam Lau <samalau@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 35171: (QA follow-up): tidy up POD
Victor Grousset/tuxayo [Mon, 30 Oct 2023 19:58:21 +0000 (20:58 +0100)]
Bug 35171: (QA follow-up): tidy up POD

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 35171: Add send_empty option to runreport
Nick Clemens [Thu, 26 Oct 2023 18:03:40 +0000 (18:03 +0000)]
Bug 35171: Add send_empty option to runreport

This patch adds a new 'send_empty' option to runreport.pl

To test:
1 - Create a report in Koha that will not return any results:
    SELECT barcode FROM items WHERE 1=2
2 - perl misc/cronjobs/runreport.pl 1
3 - Output is: NO OUTPUT: 0 results from execute_query
4 - perl misc/cronjobs/runreport.pl 1 --send_empty
5 - Output is: no results were returned for the report
6 - perl misc/cronjobs/runreport.pl 1 --send_empty --email
7 - It will die on an email error unless you have SMTP configured - this si good, it means we sent an email
8 - Bonus points: Test on a system that can correctly send emails, confirm it works :-)

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34008: Harmonize attribute names
Tomas Cohen Arazi [Mon, 30 Oct 2023 16:14:16 +0000 (13:14 -0300)]
Bug 34008: Harmonize attribute names

This patch harmonizes the attribute names with what is used for `items`
and `checkouts` in terms of terminology.

It also adapts the tests so they are less random failure-prone (they had
a fixed value for the item type, which might make things explode if the
chosen value already exists on the DB.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34008: (QA follow-up) 'item_type_id' should be used
Tomas Cohen Arazi [Mon, 30 Oct 2023 14:08:15 +0000 (11:08 -0300)]
Bug 34008: (QA follow-up) 'item_type_id' should be used

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34008: Add REST endpoint for list of itemtypes
Andreas Jonsson [Sat, 16 Sep 2023 10:13:15 +0000 (10:13 +0000)]
Bug 34008: Add REST endpoint for list of itemtypes

Test plan:
* Enable the system preference RESTBasicAuth
* curl -s --request GET http://kohadev-intra.mydnsname.org:8081/api/v1/itemtypes
  should give 401 Unauthorized
* curl -s -u koha:koha --request GET http://kohadev-intra.mydnsname.org:8081/api/v1/itemtypes
  should produce JSON-list of itemtypes
* curl -s -u koha:koha --header "x-koha-embed: translated_descriptions" --request GET http://kohadev-intra.mydnsname.org:8081/api/v1/itemtypes
  should include the field translated_descriptions containing the translated descriptions, if any

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] perltidy -b t/db_dependent/api/v1/itemtypes.t # Resolve bad score of 44
[EDIT] chmod 755 t/db_dependent/api/v1/itemtypes.t
[EDIT] perltidy -b Koha/REST/V1/ItemTypes.pm
Lesson: Please run qa tools yourself and adjust accordingly?
Edit (tcohen): I restored the item_type_translated_description.yaml file
as the entire API was broken because of the lack of it.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 33664: DBRev 23.06.00.058
Tomas Cohen Arazi [Wed, 1 Nov 2023 20:08:48 +0000 (17:08 -0300)]
Bug 33664: DBRev 23.06.00.058

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 33664: (follow-up) Move check for 'received' to include Modify link
Katrin Fischer [Mon, 30 Oct 2023 15:34:19 +0000 (15:34 +0000)]
Bug 33664: (follow-up) Move check for 'received' to include Modify link

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 33664: Add ability to cancel order lines in closed baskets
Katrin Fischer [Fri, 28 Jul 2023 15:39:38 +0000 (15:39 +0000)]
Bug 33664: Add ability to cancel order lines in closed baskets

At the moment it's only possible to cancel order lines while the
basket is still open or from the pending orders page during processing
a shipment. The latter requires you to add a shipment and fake an
invoice. To make things easier we want to optionally allow cancelling
order lines in closed baskets from the basket summary page.

Before applying the patch:

1) Set up data
* Create a new basket
* Create a few order lines, at least 3
* Close the basket
* Receive shipment and receive one order line

2) Verify current behaviour
* On basket summary page: you cannot cancel while the basket is closed
* On the pending orders page you can cancel

Apply patch, run database update, restart_all

3) Verify new behaviour
* Verify that nothing has changed on pending orders and basket summary
* Switch new system preference CancelOrdersInClosedBaskets to "Allow"
* Verify that pending orders hasn't changed a bit
* Verify that you now can cancel your order lines in the closed basket
  Exception: the received order line should not be cancellable

Test anything else that you think might make sense ;)

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 35074: Add support for category_code to writeoff_debts
Martin Renvoize [Tue, 17 Oct 2023 11:39:44 +0000 (12:39 +0100)]
Bug 35074: Add support for category_code to writeoff_debts

This patch adds support for passing category_code as a limiter for the
writeoff_debts script.

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 35074: Use patron relationship alias
Martin Renvoize [Fri, 27 Oct 2023 13:02:27 +0000 (14:02 +0100)]
Bug 35074: Use patron relationship alias

This patch updates the existing Koha::Account::Line->patron accessor to
use the new aliased 'patron' relation in the schema class

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 35074: Add patron relationship alias
Martin Renvoize [Fri, 27 Oct 2023 13:01:35 +0000 (14:01 +0100)]
Bug 35074: Add patron relationship alias

This patch adds the 'patron' relationship alias to the Acountline
DBIx::Class schema.

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 32730: Add tests for Koha::Patron->get_lists_with_patron
Tomas Cohen Arazi [Wed, 1 Nov 2023 19:52:08 +0000 (16:52 -0300)]
Bug 32730: Add tests for Koha::Patron->get_lists_with_patron

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 32730: Add FIXME in POD to highlight return value
Tomas Cohen Arazi [Wed, 1 Nov 2023 19:15:07 +0000 (16:15 -0300)]
Bug 32730: Add FIXME in POD to highlight return value

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 32730: Add Patron Lists tab to patron details and circulation pages
Michael Hafen [Fri, 27 Jan 2023 18:14:32 +0000 (11:14 -0700)]
Bug 32730: Add Patron Lists tab to patron details and circulation pages

Test Plan:
1. Apply patch
2. create a patron list if there aren't any
3. search for a patron
4. observe the "Patron lists" tab showing the list that the patron is not in
5. try adding the patron to the list and removing them from the list to
   be sure the feature has full operation
6. click "Check Out" on the side bar menu to navigate to the circulation
   page for this patron
7. observe the "Patron lists" tab, and verify it operates as it did on the
   patron details page

Bug 32730: (follow-up) Minor corrections
   by Owen Leonard

This patch corrects an instance of an incorrect capital letter
("Patron Lists" -> "Patron lists") and makes minor tweaks to
indentation.

Signed-off-by: Stina Hallin <stina.hallin@ub.lu.se>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Edit: (tcohen) new files should be run through perltidy ALWAYS. Did it
and squashed it here.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 26170: DBIC booleans
Tomas Cohen Arazi [Wed, 1 Nov 2023 18:53:51 +0000 (15:53 -0300)]
Bug 26170: DBIC booleans

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 26170: API behavior change tests
Tomas Cohen Arazi [Wed, 1 Nov 2023 18:45:36 +0000 (15:45 -0300)]
Bug 26170: API behavior change tests

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 26170: DBRev 23.06.00.057
Tomas Cohen Arazi [Wed, 1 Nov 2023 18:35:33 +0000 (15:35 -0300)]
Bug 26170: DBRev 23.06.00.057

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 26170: DBIC schema
Tomas Cohen Arazi [Wed, 1 Nov 2023 18:26:31 +0000 (15:26 -0300)]
Bug 26170: DBIC schema

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 26170: Add protected status for patrons
Magnus Enger [Thu, 15 Jun 2023 12:21:19 +0000 (12:21 +0000)]
Bug 26170: Add protected status for patrons

This set of patches makes it possible to protect patrons from being accidetally
deleted or merged with other patrons, from the UI and from (well behaved) cron
jobs. The following subroutines are affected:
- Koha::Patron::delete
- Koha::Patron::merge_with
- Koha::Patron::safe_to_delete
- C4::Members::GetBorrowersToExpunge

Please note:
- This does not intend to protect patrons from being edited, only from being
  deleted

To test:

* Tests
- Run the affected tests:
  prove t/db_dependent/Members.t
  prove t/db_dependent/Koha/Patrons.t

* Editing protected status and manual deletion
- Add a new user, note the presence of the "Protected" field under "Library
  management", but leave it at the default "No", for now.
- Note that "Protected" is displayed in the "Library use" section of the patron
  details.
- Note that More > Delete is avaiable as an action when the patron is saved
- Edit the user and set "Protected" to "Yes"
- Note that More > Delete is now disabled, with a note that the patron is protected

* Batch patron deletion
- Go to Tools > Batch patron deletion and anonymization
- Check the box for "Verify you want to delete patrons"
- Choose the category of your protected patron for "whose patron category is"
  and click "Next" to run the actual deletion
- Check that your protected patron was not deleted

* Merging patrons
- Make sure you have two patrons with similar names or the same category, so
  you can find them with one search. One should be protected, one not.
- Search for the patrons, tick their boxes and click on "Merge selected patrons"
- Select one of the patrons as the "patron to keep".
. Click on "Merge patrons"
- "No valid patrons to merge were found" should be shown
- Repeat this with the other patron as the "patron to keep"
(A future enhancement could be to not allow a protected patron to be selected for
merging in the first place.)

* misc/cronjobs/delete_patrons.pl
- Make sure you have a protected patron, in a category with at least one more
  patron.
- Run something like this (at least in ktd):
  $ perl misc/cronjobs/delete_patrons.pl --category_code <code> -v --confirm
  (Replace <code> with the actual categorycode.)
- Make sure the borrowernumber of the protected patron is not mentioned in the
  output of the script.
- Check the protected patron was not deleted
- Check the non-protected patrons were deleted

* REST API (with ktd)
- Make sure you still have a protected patron, and note their borrowernumber
- Enable RESTBasicAuth and restart all the things
- Run these two commands from the command line on the host:
  $ curl -u koha:koha --request GET "http://localhost:8081/api/v1/patrons/54"
  $ curl -u koha:koha --request DELETE "http://localhost:8081/api/v1/patrons/54"
  (Replace 54 with the actual borrowernumber of your protected patron.)
- The first curl command should give you the patron details. The second should
  give this output:
  {"error":"Protected patrons cannot be deleted","error_code":"is_protected"}

There could be more functions/scripts where patrons are deleted that I have not
thought about. Please report them on the bug if you find any!

Update 2023-10-19: Fix "More > Delete" on patron, so link can not be clicked.
Update 2023-10-19: Rebase

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 26170: Database update
Magnus Enger [Thu, 15 Jun 2023 12:13:45 +0000 (12:13 +0000)]
Bug 26170: Database update

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 33353: Add compatibility with Search::Elasticsearch 8.0
Julian Maurice [Tue, 28 Mar 2023 15:02:19 +0000 (17:02 +0200)]
Bug 33353: Add compatibility with Search::Elasticsearch 8.0

For some reasons, with Search::Elasticsearch 8.0 JSON::true is
interpreted as 1 and Elasticsearch will always respond that it found "at
least 1 record". Something like this:

{
    total => {
        value => 1,
        relation => 'gte'
    }
}

Replacing JSON::true by \1 works with every version of
Search::Elasticsearch I tested (6.80, 7.717, 8.0)

Also worth noting:
Search::Elasticsearch will stop being supported by Elastic
https://github.com/elastic/elasticsearch-perl/issues/220

We might need to find an alternative for future versions of
Elasticsearch

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Philip Orr <philip.orr@lmscloud.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 33353: Add compatibility with Elasticsearch 8.x
Julian Maurice [Tue, 28 Mar 2023 14:41:08 +0000 (16:41 +0200)]
Bug 33353: Add compatibility with Elasticsearch 8.x

Elasticsearch 8 removed the support of types. This patch adapts the
requests accordingly.
With this patch, Koha will still be compatible with Elasticsearch 7.x
but will no longer work with Elasticsearch 6.x
Since Elasticsearch 6.x is no longer maintained, this should not be a
problem.

https://www.elastic.co/guide/en/elasticsearch/reference/7.17/removal-of-types.html

Tested with Search::Elasticsearch 6.80 and 7.717

Test plan:
1. Install Elasticsearch 8
   (use docker image: docker.io/koha/elasticsearch-icu:8.x)
2. Change the elasticsearch server location in $KOHA_CONF
3. Run misc/search_tools/rebuild_elasticsearch.pl -r -b -v
   It should fail
4. Apply the patch
5. Run misc/search_tools/rebuild_elasticsearch.pl -r -b -v
   It should end with "Total XXX records indexed"
6. Try to search some biblios and verify that it works the same as
   before

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Philip Orr <philip.orr@lmscloud.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 35180: DBIC schema
Tomas Cohen Arazi [Wed, 1 Nov 2023 18:26:00 +0000 (15:26 -0300)]
Bug 35180: DBIC schema

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34520: DBIC schema
Tomas Cohen Arazi [Wed, 1 Nov 2023 18:24:43 +0000 (15:24 -0300)]
Bug 34520: DBIC schema

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: Fix stray type definition
Tomas Cohen Arazi [Wed, 1 Nov 2023 18:00:13 +0000 (15:00 -0300)]
Bug 34587: Fix stray type definition

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: Fix license issues
Tomas Cohen Arazi [Wed, 1 Nov 2023 12:26:03 +0000 (09:26 -0300)]
Bug 34587: Fix license issues

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: DBRev 23.06.00.056
Tomas Cohen Arazi [Tue, 31 Oct 2023 19:37:35 +0000 (16:37 -0300)]
Bug 34587: DBRev 23.06.00.056

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: Atomicupdate tweaks
Tomas Cohen Arazi [Tue, 31 Oct 2023 19:11:49 +0000 (16:11 -0300)]
Bug 34587: Atomicupdate tweaks

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: Updated cypress tests
Matt Blenkinsop [Fri, 27 Oct 2023 16:24:16 +0000 (16:24 +0000)]
Bug 34587: Updated cypress tests

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
(cherry picked from commit 05cf3d9f6a7ddf163d2e144436a6ff8e5854d7e4)
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: (QA follow-up) Tidy
Nick Clemens [Fri, 27 Oct 2023 15:36:06 +0000 (15:36 +0000)]
Bug 34587: (QA follow-up) Tidy

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: Add IR_A1 fields to COUNTER report
Pedro Amorim [Fri, 27 Oct 2023 11:54:42 +0000 (11:54 +0000)]
Bug 34587: Add IR_A1 fields to COUNTER report

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: Fix ->validation->output in API endpoints
Matt Blenkinsop [Thu, 26 Oct 2023 09:09:10 +0000 (09:09 +0000)]
Bug 34587: Fix ->validation->output in API endpoints

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: (QA follow-up): squash
Pedro Amorim [Wed, 25 Oct 2023 13:58:22 +0000 (13:58 +0000)]
Bug 34587: (QA follow-up): squash

Tidy
$c->validation in API controllers
File permissions
.inc file filters

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: Move YOP to mus/yus instead of erm_usage_titles
Pedro Amorim [Tue, 24 Oct 2023 15:06:59 +0000 (15:06 +0000)]
Bug 34587: Move YOP to mus/yus instead of erm_usage_titles

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: Add tests for the custom report API endpoints
Matt Blenkinsop [Mon, 23 Oct 2023 13:58:05 +0000 (13:58 +0000)]
Bug 34587: Add tests for the custom report API endpoints

prove t/db_dependent/api/v1/erm_custom_reports.t

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: Add access_type and yop to the reports viewer
Matt Blenkinsop [Fri, 20 Oct 2023 15:45:26 +0000 (15:45 +0000)]
Bug 34587: Add access_type and yop to the reports viewer

Some report types include access_type and yop headers for each usage statistics

The hierarchy is as follows: yop - access_type - metric_type
I.e. for each yop there are report lines for access_type and then for each access_type there are also report lines for metric types

This patch adds helper functions to populate the report rows based on this hierarchy and the report data retrieved from the database

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: SushiCounter tests
Pedro Amorim [Tue, 29 Aug 2023 15:11:54 +0000 (15:11 +0000)]
Bug 34587: SushiCounter tests

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: Improve title uniqueness rules
Pedro Amorim [Mon, 23 Oct 2023 11:02:36 +0000 (11:02 +0000)]
Bug 34587: Improve title uniqueness rules

Improve robustness of unique matching here, to make sure we always match for same title if the case,

Some report rows may correspond to the same title as the previous row but have an empty match field, or alternatively come with a filled match field (e.g. DOI or Print_ISSN in TR_J4. Because of this we only verify a uniqueness match field if both current row and previous have it non-empty, otherwise we keep checking the remaining uniqueness match fields.

Example of this use-case, COUNTER report:
title    | publisher | platform  | Proprietary_ID | Print_ISSN | DOI     | YOP  | usages
examplet | examplep  | examplepl | 1              | 123        |        | 2020 | usages
examplet | examplep  | examplepl | 1              | 123        |someDOI | 2021 | usages

The above 2 rows is the same title, same publisher, same proprietary_id,
same Print_ISSN, etc. It just so happens that one was returned by SUSHI
with DOI and the other wasnt.
These 2 rows correspond to different usage statistics by YOP, not 2
different usage_titles

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: Add proprietary_id and platform to erm_usage_titles
Pedro Amorim [Tue, 24 Oct 2023 10:42:13 +0000 (10:42 +0000)]
Bug 34587: Add proprietary_id and platform to erm_usage_titles

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: Update database columns length
Pedro Amorim [Mon, 23 Oct 2023 12:37:32 +0000 (12:37 +0000)]
Bug 34587: Update database columns length

Some COUNTER files have long data in these fields and were causing new
database entries to fail

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: Improve SUSHI COUNTER error handling
Pedro Amorim [Mon, 23 Oct 2023 12:36:15 +0000 (12:36 +0000)]
Bug 34587: Improve SUSHI COUNTER error handling

This is doing a few things, checking for sushi errors:
1) Abort background job and provide error message if SUSHI returned
   {Severity} in response, this means SUSHI error, docs here:
https://app.swaggerhub.com/apis/COUNTER/counter-sushi_5_0_api/5.0.2#/SUSHI_error_model

2) Abort background job and provide error message if SUSHI provided
   proper response, but contains Exceptions in Report_Header

3) Abort background job and provide error message if SUSHI returned 0
   report items (e.g. everything is fine but there are no items for the
   harvest period provided

4) Abort background job and provide error message if COUNTER file
   created is larger than allowed by the database.

5) Provide error message if manual upload file size exceeds the max
   allowed by the database

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: Fix total added objects in job report
Pedro Amorim [Mon, 23 Oct 2023 10:59:57 +0000 (10:59 +0000)]
Bug 34587: Fix total added objects in job report

Remove obsolete total_records code, this was a leftover from when this code was in UsageDataProvider.pm
Add a new job_callback 'added_usage_objects' to handle total objects added (titles, platforms, databases, items)
Update job report include file

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: Fix 'Import logs' sorting by import date
Pedro Amorim [Mon, 23 Oct 2023 09:59:24 +0000 (09:59 +0000)]
Bug 34587: Fix 'Import logs' sorting by import date

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: Add report type and provider name to failed background job report
Matt Blenkinsop [Wed, 20 Sep 2023 16:51:01 +0000 (16:51 +0000)]
Bug 34587: Add report type and provider name to failed background job report

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: Fix dry run and end date in the cronjob
Matt Blenkinsop [Wed, 20 Sep 2023 14:57:14 +0000 (14:57 +0000)]
Bug 34587: Fix dry run and end date in the cronjob

The cronjob always logs a dry run even on a live harvest and the end date parameter was not working correctly. This patch addresses both issues

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: Add try catch blocks to object creation
Matt Blenkinsop [Wed, 20 Sep 2023 13:13:12 +0000 (13:13 +0000)]
Bug 34587: Add try catch blocks to object creation

Currently a harvest fails completely if a single title fails to harvest. Rather than failing the entire job, this patch introduces try catch blocks to stop this and allow the job to continue. Failed imports are recorded and displayed in the background job UI on job completion

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: Remove counter files from API embedding to improve performance
Matt Blenkinsop [Tue, 19 Sep 2023 12:07:55 +0000 (12:07 +0000)]
Bug 34587: Remove counter files from API embedding to improve performance

Rather than fetching the counter files and embedding the counter logs, we now add a foreign key to the data provider in the counter logs table and fetch them directly.

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: Update field lengths to 255 characters
Matt Blenkinsop [Mon, 18 Sep 2023 16:00:28 +0000 (16:00 +0000)]
Bug 34587: Update field lengths to 255 characters

(cherry picked from commit 9f384df2ad5f4c037b052e8914d7e4b3e5f26670)
(cherry picked from commit 02c9a57af9f27cf528a2eb924876037890cf5504)

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: Add Create from registry button
Matt Blenkinsop [Tue, 12 Sep 2023 14:53:33 +0000 (14:53 +0000)]
Bug 34587: Add Create from registry button

Currently there is no option to go back to using the registry API to create a provider once "Create manually" has been clicked

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: Fix url validation
Matt Blenkinsop [Fri, 8 Sep 2023 14:33:50 +0000 (14:33 +0000)]
Bug 34587: Fix url validation

We need to check whether the url ends in a trailing "/" and also whether it ends in "/reports" for a SUSHI harvest. Two new functions are added to manage this and prevent failed harvests.

report_type is also now lowercase in the url and the dates are truncated to YYYY-MM format

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: New sushi counter class
Pedro Amorim [Tue, 29 Aug 2023 12:52:00 +0000 (12:52 +0000)]
Bug 34587: New sushi counter class

Moves a lot of the logic for creating a counter file and parsing a SUSHI response to a new class to simplify logic

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: Move everything into a ERM/EUsage subfolder
Pedro Amorim [Tue, 29 Aug 2023 11:39:40 +0000 (11:39 +0000)]
Bug 34587: Move everything into a ERM/EUsage subfolder

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: Add borrowernumber to counter logs display
Matt Blenkinsop [Thu, 24 Aug 2023 14:40:47 +0000 (14:40 +0000)]
Bug 34587: Add borrowernumber to counter logs display

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: Add integration with the counter registry API
Matt Blenkinsop [Thu, 24 Aug 2023 14:09:57 +0000 (14:09 +0000)]
Bug 34587: Add integration with the counter registry API

The counter registry offers an API that provides SUSHI information for each provider. This will be incredibly useful for creating new providers as we can use the correct information from the registry for harvesting urls etc. This reduces the risk of user input errors when creating providers and gives greater reliability to the data required to successfully harvest from the SUSHI API

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: Improve error handling for connection testing
Matt Blenkinsop [Wed, 23 Aug 2023 11:05:25 +0000 (11:05 +0000)]
Bug 34587: Improve error handling for connection testing

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: Remove q_header parameter from API specs
Matt Blenkinsop [Tue, 22 Aug 2023 13:23:25 +0000 (13:23 +0000)]
Bug 34587: Remove q_header parameter from API specs

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
8 months agoBug 34587: File upload - Background job
Pedro Amorim [Mon, 21 Aug 2023 14:50:45 +0000 (14:50 +0000)]
Bug 34587: File upload - Background job

ErmSushiHarvester background job will now be initiated by either
enqueue_counter_file_processing_job or by enqueue_sushi_harvest_jobs
from Koha/ERM/UsageDataProvider.pm, the former if triggered by a manual
file upload, the latter if by the 'run now' button or by the cron script.

This commit also includes some rewording/refactoring, namely:
- COUNTER file validation now happens in the API, before enqueuing the job.
- Removal of no longer used POST /erm/counter_files endpoint
- Koha/ERM/UsageDataProvider.pm:
-- run method is now enqueue_sushi_harvest_jobs
-- new enqueue_counter_file_processing_job method
-- harvest method is now harvest_sushi
-- new set_background_job_callbacks method to set the background job callbacks
- REST/V1/ERM/UsageDataProviders
-- run method is now process_SUSHI_response
-- new process_COUNTER_file endpoint

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>