Bug 30650: Split the permission

We actually want one permission to manage the administration page, and
one for the circulation module.

Signed-off-by: Koha Team University Lyon 3 <koha@univ-lyon3.fr>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30650: Prevent pickup to be created on holiday

We could improve this more and add disable the holiday from the date
picker widget, but it's out of the scope here.

Sponsored-by: Association KohaLa - https://koha-fr.org/
Signed-off-by: Koha Team University Lyon 3 <koha@univ-lyon3.fr>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30650: Add filter_by_scheduled_today

A bit of cleaning

Sponsored-by: Association KohaLa - https://koha-fr.org/
Signed-off-by: Koha Team University Lyon 3 <koha@univ-lyon3.fr>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30650: Add link from the circ patron page

Sponsored-by: Association KohaLa - https://koha-fr.org/
Signed-off-by: Koha Team University Lyon 3 <koha@univ-lyon3.fr>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30650: Add link from the mainpage

Sponsored-by: Association KohaLa - https://koha-fr.org/
Signed-off-by: Koha Team University Lyon 3 <koha@univ-lyon3.fr>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30650: Allow to restrict curbside pickup for waiting holds only

Sponsored-by: Association KohaLa - https://koha-fr.org/
Signed-off-by: Koha Team University Lyon 3 <koha@univ-lyon3.fr>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30650: DB - Add policy.enable_waiting_holds_only

Sponsored-by: Association KohaLa - https://koha-fr.org/
Signed-off-by: Koha Team University Lyon 3 <koha@univ-lyon3.fr>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30650: Notify the patron when a new curbside pickup is created

Sponsored-by: Association KohaLa - https://koha-fr.org/
Signed-off-by: Koha Team University Lyon 3 <koha@univ-lyon3.fr>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30650: Add link from the member toolbar

Sponsored-by: Association KohaLa - https://koha-fr.org/
Signed-off-by: Koha Team University Lyon 3 <koha@univ-lyon3.fr>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30650: OPAC view

Sponsored-by: Association KohaLa - https://koha-fr.org/
Signed-off-by: Koha Team University Lyon 3 <koha@univ-lyon3.fr>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30650: Add dayjs to opac

Sponsored-by: Association KohaLa - https://koha-fr.org/
Signed-off-by: Koha Team University Lyon 3 <koha@univ-lyon3.fr>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30650: Add dayjs and isSameOfAfter plugin

Sponsored-by: Association KohaLa - https://koha-fr.org/
Signed-off-by: Koha Team University Lyon 3 <koha@univ-lyon3.fr>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30650: Add circulation page view

This is the main commit message.

A plugin already exists to manage curbside pickups. This new
enhancehemnt is suggesting an implementation that is ready to be integrated
into Koha core in order to provide the feature out-of-the-box.

What has been done in this patch set:
- Deal with installations using the existing plugin (upgrade the DB schema and migrate their data)
- Add a new syspref (CurbsidePickup) and two new permissions:
  * parameters.manage_curbside_pickups
  * circulate.manage_curbside_pickups
- Add an administration page to setup the configuration: admin/curbside_pickup.pl
- Add a circulation page to manage the existing pickups, and create new one
- Add a new OPAC view "your curbside pickups" to let patron manage their pickups, and create new ones
- Add link from the "member" toolbar

Improvements compared to the plugin:
- Ability to create several pickup windows per day
- Better display of the pickup times (not in a dropdown list)
- Ability to disable pickups for patrons without waiting holds
- Display pickups on the patron circulation page
- Display pickups of the library on the homepage
- Prevent pickup to be created on a holiday
- Better error handling (exceptions)
- Unit tests

More improvements are already planned, see related bug reports.

Test plan:
After you setup the feature correctly from the administration view, you
will be able to use the schedule curbside pickups from the staff
interface, and from the OPAC interface if you selected "patron-scheduled
pickup"
A. Staff interface
1. Go to Circulation > Curbside pickups
=> If the logged-in user has the circulate.manage_curbside_pickups
permission you will be able to create and manage curbside pickups
2. Go to a patron detail page and click the "Schedule pickup" button in
the toolbar
3. If the patron has waiting holds and you selected "Enable for waiting holds only",
of if you didn't select the option, you will be able to select a pickup
date and slots to create a pickup.
4. Confirm that you cannot create more pickups per slot than what you
defined in the curbside pickup configuration for this library
5. Confirm that you cannot create a pickup if the feature is disabled
for the library
6. Notice that you can mark the pickup as "stage and ready", then
"patron is outside" and finally "delivered today". You can also rollback
the change
7. Notice that once the pickup has been marked as delivered, the
item has been checked out and that a new notice has been generated (if
the patron has "Hold_Filled" in their messaging preferences
8. Confirm that the information about current pickups is displayed on
the circulation page of the patron

B. OPAC interface
1. Create a new curbside pickup from the OPAC
2. Confirm that the same limitations as from the staff interface are in
effect (waiting holds, number of patron per slots, etc.)
3. Confirm that you can cancel a pickup and alert staff of you arrival
4. Confirm that you cannot cancel a pickup that has been delivered
already

Sponsored-by: Association KohaLa - https://koha-fr.org/
Signed-off-by: Koha Team University Lyon 3 <koha@univ-lyon3.fr>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30650: Add some useful modules and tests

Sponsored-by: Association KohaLa - https://koha-fr.org/
Signed-off-by: Koha Team University Lyon 3 <koha@univ-lyon3.fr>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30650: Be more flexible with opening slots

Sponsored-by: Association KohaLa - https://koha-fr.org/
Signed-off-by: Koha Team University Lyon 3 <koha@univ-lyon3.fr>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30650: Adjust atomic update

Take into account existing installs using the plugins

Sponsored-by: Association KohaLa - https://koha-fr.org/
Signed-off-by: Koha Team University Lyon 3 <koha@univ-lyon3.fr>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30650: Admin page

This patch adds a new page in the administration area to manage curbside
pickups.

Test plan:
1. Enable the new CurbsidePickup syspref
2. Go to Administration > Curbside pickups
=> If the logged-in user has the manage_curbside_pickups permission you
should see the administration view
3. Enable the feature for some libraries and configure it.
 a. You must define a "pickup interval" and a "maximum patrons per interval"
values.
 b. If "Patron-scheduled pickup" is enabled, patrons will be able to
schedule curbside pickups from the OPAC
 c. If "Enable for waiting holds only" is enabled, only patron with waiting
holds will be allowed to schedule curbside pickups
 d. In the bottom part of the form you can define the slots available for
curbside pickups.
 e. Save

Sponsored-by: Association KohaLa - https://koha-fr.org/
Signed-off-by: Koha Team University Lyon 3 <koha@univ-lyon3.fr>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30650: Add link from the admin home page

Sponsored-by: Association KohaLa - https://koha-fr.org/
Signed-off-by: Koha Team University Lyon 3 <koha@univ-lyon3.fr>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30650: Add new permission manage_curbside_pickups

Sponsored-by: Association KohaLa - https://koha-fr.org/
Signed-off-by: Koha Team University Lyon 3 <koha@univ-lyon3.fr>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30650: Add the CurbsidePickup syspref

Sponsored-by: Association KohaLa - https://koha-fr.org/
Signed-off-by: Koha Team University Lyon 3 <koha@univ-lyon3.fr>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30650: Koha classes

Sponsored-by: Association KohaLa - https://koha-fr.org/
Signed-off-by: Koha Team University Lyon 3 <koha@univ-lyon3.fr>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30650: DB changes

Sponsored-by: Association KohaLa - https://koha-fr.org/
Signed-off-by: Koha Team University Lyon 3 <koha@univ-lyon3.fr>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31244: Fix 500 if logout and not logged in

Can't use an undefined value as a HASH reference at /kohadevbox/koha/C4/Auth.pm line 985
        C4::Auth::checkauth('CGI=HASH(0x5603b7dc4300)', 0, 'HASH(0x5603b2633238)', 'intranet', undef, 'intranet-main.tt') called at /kohadevbox/koha/C4/Auth.pm line 186
        C4::Auth::get_template_and_user('HASH(0x5603b7b83d08)') called at /kohadevbox/koha/mainpage.pl line 40

Test plan:
Open a private window
Hit /cgi-bin/koha/mainpage.pl?logout.x=1

Signed-off-by: Sally <sally.healey@cheshiresharedservices.gov.uk>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31246: <span> displayed in 'Additional fields' section

This patch avoids escaping HMTL data in the additional fields section.

To test:
1. Open the additional fields section
   http://kohadev-intra.mydnsname.org:8081/cgi-bin/koha/admin/additional-fields.pl
=> FAIL: there are some nasty <span> </span> texts in there
2. Apply this patch
3. Reload
=> SUCCESS: Things display nicely again!
4. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Emmanuel Bétemps <e.betemps@gmail.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31120: Items will renew for zero ( 0 ) days if renewalperiod is blank/empty value

If the rule renewalperiod is the blank empty string instead of being null/undefined or non-existant, Koha will interpret the renewal period as being zero days instead of falling back to the issuelength rule.

It makes sense to me that a literal 0 here should make it renew for zero days even though that is nonsensical.

Test Plan:
1) Delete all your rules
2) Create an all/all/all rules with an empty string for renewal base
   period
3) Note that renewing an item does nothing
4) Apply this patch
5) Restart all the things!
6) Renew again
7) Note the renewal uses the issuelength rule as intended

Signed-off-by: Sally <sally.healey@cheshiresharedservices.gov.uk>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31120: Add unit tests

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 22276: (QA follow-up) Add missing USE

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 22276: Add client storage of user-selected DataTables configuration

This patch enables use of DataTables' built-in session storage of table
configurations. DataTables by default uses the browser's localstorage
feature to store the state of each DataTable, including which columns
have been hidden by the user.

Our table settings system has always overridden this saved state. This
patch adds a check for a saved configuration state and if present loads
the saved state instead of the default state.

The system is enabled in this patch on the libraries administration
page. To test, apply the patch and open go to Administration ->
Libraries.

- Use the "Columns" button to change the visibility setting of multiple
  columns.
- Navigate away from the page and then back to test that the visibility
  settings have been kept.
- Go to Administration -> Table settings -> Administration and change
  the visibility settings for the "libraries" table in a way that it
  differs from the visibility settings you set manually.
- Return to the libraries administration page and confirm that your
  manual settings are still being loaded instead of the defaults from
  table settings.

Signed-off-by: Solène Desvaux <solene.desvaux@biblibre.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31211: (QA follow-up) Add class to new list element for error

To allow additional styling the class template_toolkit_error was
added to the list element.

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31211: Check slips and notices for valid Template Toolkit and report errors

It would be useful for Koha to run a basic TT validation test and report back errors.
That way if somebody misses an END tag or something of that nature, it will be easy to find and fix.

Test Plan:
1) Apply this patch
2) Edit a notice, change the content to simply "[% END %]"
   or some other invalid Template Toolkit
3) Choose "Save and continue editing"
4) Note the new "Template Toolkit error" field that displays the error
   message

Signed-off-by: Sally <sally.healey@cheshiresharedservices.gov.uk>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31229: (bug 30063 follow-up) fix column visibility

Fix bad conflict resolution with bug 29648.

Test plan:
Confirm that the "Columns" button is displayed on the patron result view
and that it works as expected.

Signed-off-by: Sally <sally.healey@cheshiresharedservices.gov.uk>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 21982: Circulation statistics wizard does not count deleted items

By default statistic assistant filter is set on the
existing items and doesn't mind about deleted items.
The patch corrects this. Items belong to the same library.

Test plan:
1)Make sure to have at least 1 or more item(s) in deleteditems database table
2)Home > Reports > Circulation statistics
3)Fill up 'Issuing library', 'Home library' and 'Holding library'
4)Submit
5)Notice that only statistics for active items are shown
6)Apply patch and repeat from 2) to 5)
7)Now active items and deleted items are shown

Signed-off-by: Sally <sally.healey@cheshiresharedservices.gov.uk>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31191: Specify FacetLabelTruncationLength is only for Zebra

This patch adds a note in the description of FacetLabelTruncationLength
that it only works with Zebra.

To test:
0) Apply patch
1) Go to Administration > Global system preference > Searching
preferences
2) Search for FacetLabelTruncationLength
3) Read the description, make sure it indicates that is requires Zebra

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 26549: Show value of global system preferences on subscription form

This patch modifies the subscription entry form so that the values of
two system preferences are displayed as a hint to the user:

The "Number of issues to display to staff" field will show the value
of the StaffSerialIssueDisplayCount preference, and the "Number of
issues to display to the public" field will show
OPACSerialIssueDisplayCount. If the user has permission the system
preference names will link to the system preferences page.

To test, apply the patch and log into the staff interface as a user with
permission to edit system preferences.

- Go to Serials -> New subscription.
- The "Number of issues to display to staff" field should have a hint
  under it: "Default: XX (StaffSerialIssueDisplayCount system
   preference)"
- The "Number of issues to display to the public" field should have a
  hint under it: "Default: XX (OPACSerialIssueDisplayCount system
  preference)"
- Log out and log back in as a user who does not have permission to edit
  system preferences. Return to the form and confirm that the hints do
  not link to system preferences.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30969: Cross site scripting (XSS) attack in OPAC authority search ( opac-authorities-home.pl )

There appears to be a cross site scripting attack vulnerability in opac-authorities-home.pl, but may be accessible from any page using C4::Output::pagination_bar.

https://MYKOHA.LOCAL/cgi-bin/koha/opac-authorities-home.pl?and_or=and%27%22()%26%25%3Csad%3E%3CScRiPt%20%3Ealert(document.domain)%3C/ScRiPt%3E&authtypecode=CORPO_NAME&excluding=1&marclist=all&op=do_search&operator=contains&orderby=HeadingAsc&type=opac&value=1

Test Plan:
1) Use the URL above to show the XSS vulnerability exists
2) Apply this patch
3) Restart all the things!
4) Reload the page, no XSS vulnerability!

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 29697: tracklinks should not explode on invalid marc record

Caught by t/db_dependent/www/regressions.t

    #   Failed test '404 for itemnumber containing different URI'
    #   at t/db_dependent/www/regressions.t line 124.
    #          got: '500'
    #     expected: '404'

    #   Failed test '302 for itemnumber with matching URI'
    #   at t/db_dependent/www/regressions.t line 126.
    #          got: '500'
    #     expected: '302'

    #   Failed test '404 for itemnumber containing different URI'
    #   at t/db_dependent/www/regressions.t line 139.
    #          got: '500'
    #     expected: '404'

    #   Failed test '302 for itemnumber with matching URI'
    #   at t/db_dependent/www/regressions.t line 141.
    #          got: '500'
    #     expected: '302'
    # Looks like you failed 4 tests of 36.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 29697: Fix t/db_dependent/Search.t

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 29697: Fix t/Search.t

 #   Failed test 'No tests run for subtest "searchResults PassItemMarcToXSLT test"'
 #   at t/Search.t line 208.
 Undefined subroutine &C4::Biblio::EmbedItemsInMarcBiblio called at t/Search.t line 194.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 29697: Reintroduce wrongly removed import

 #   Failed test 'No tests run for subtest "CancelHold"'
 #   at t/db_dependent/ILSDI_Services.t line 806.
 Undefined subroutine &C4::ILSDI::Services::CanReserveBeCanceledFromOpac called at /kohadevbox/koha/C4/ILSDI/Services.pm line 941.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 29697: Fix t/db_dependent/Exporter/Record.t

Not sure about the warn, we shouldn't need it as we are raising an
exception. But better (for now) than introducing regressions.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 29697: Replace some missing occurrences

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30878: Avoid resetting waiting reserves tab

When canceling holds waiting over X days in "Holds awaiting pickup"
view, form submission resets the selected tab to "Holds waiting". This
tab reset may cause confusion for the user and lead to unwanted
actions. Instead, the user should be returned to the same tab that they
began with.

To test:
1. Apply patch
2. Navigate to /cgi-bin/koha/circ/waitingreserves.pl
3. Select "Holds waiting over X days" tab
4. Refresh page
5. Observe "Holds waiting over X days" is still selected
6. Have some holds waiting over X days
7. Select one of them and press "Cancel selected (1)" and confirm
8. Observe "Holds waiting over X days" is still selected after form
   submission

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 29697: (follow-up) Indexing a deleted record is not expected to crash

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 29697: Don't crash ES reindex if a record is wrong

We don't want rebuild_elasticsearch.pl to crash if a record is wrong
(also fix ktd setup).
Note that now the script will output an error for the problematic
record, which is certainly the expected behaviour as we are doing it
already for authority records.

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 29697: Deal with the degraded view in detail.pl

In detail.pl we must provide a degraded view with an error message about
invalid MARC::Record.
We are then forced to reproduce the GetMarcBiblio behaviour and call
StripNonXmlChars on the MARC::XML

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Rebased-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 29697: Fix tests

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 29697: Use flag embed_items

Includes:
    Bug 29697: (follow-up) Use flag embed_items

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 29697: Remove GetHiddenItemnumbers

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
JD amended patch:
-    my @items        = $biblio->items->filter_by_visible_in_opac({ patron => $logged_in_user })->unblessed;
-    foreach my $item (@items) {
+    my $items        = $biblio->items->filter_by_visible_in_opac({ patron => $logged_in_user })->unblessed;
+    foreach my $item (@$items) {

-    for my $itm (@items) {
+    for my $itm (@$items) {

-    $dat->{ITEM_RESULTS}   = \@items;
+    $dat->{ITEM_RESULTS}   = $items;

-        @items_to_show = Koha::Items->search( { itemnumbers => [ map { $_->{itemnumber} } @all_items ] } )
-                                    ->filter_by_visible_in_opac( { patron => $patron } );
+        @items_to_show = Koha::Items->search( { itemnumber => [ map { $_->{itemnumber} } @all_items ] } )
+                                    ->filter_by_visible_in_opac( { patron => $patron } )->as_list;

-            my @items_to_show = $items->filter_by_visible_in_opac({ opac => 1, patron => $patron });
+            my @items_to_show = $items->filter_by_visible_in_opac({ opac => 1, patron => $patron })->as_list;

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 29697: Replace GetMarcBiblio occurrences with $biblio->metadata->record

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
JD Amended patch:
-# FIXME Special case here

-    print "Biblio not found\n,";
+    print "Biblio not found\n";

- my $biblio = Koha::Biblio->find($hostbiblionumber);
+ my $biblio = Koha::Biblios->find($hostbiblionumber);

Rebased-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 29697: Special case - opac not needed

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31157: DBRev 22.06.00.021

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31157: (QA follow-up) Update descriptions and wording of configuration options

Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31157: Allow selection for overdue from address in the UI

This patch adds a new system preference to allow systems librarians the
option to pick wich address to use for overdues notices.

We default to 'cron' to allow existing uses of '--frombranch' on the
command line to continue to function but now allow end users to override
this option via the new OverdueNoticeFrom preference.

Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 15187: (QA follow-up) Fix test permissions

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 15187: (QA follow-up) Add unit test

This patch adds a unit test for the newly introduced filter.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 15187: Index 880 in Zebra the same as Elasticsearch

This patch modifies the MARC21 export to Zebra, so that 880 fields
are rewritten as their linked fields, in the same way that we
already do with Elasticsearch, so that the alternate graphic
representation of fields are indexed accordingly. (ie 880 $6245-01
Chinese titles will be indexed into the title index using the 245 rules)

Test plan:
0. Apply patch
1. Turn on ICU indexing
1b. vi /etc/koha/zebradb/etc/default.idx
1c. Replace charmap word-phrase-utf.chr with icuchain words-icu.xml
1d. Replace charmap word-phrase-utf.chr with icuchain phrases-icu.xml
1e. Restart Zebra server
1f. Re-index Zebra
2. Add record with a 880 $6 245-01 $a 教牧書信 field.
3. Search for this record using a title index with the Chinese title
4. Note that the record is correctly retrieved

(Note: This test probably works better using author or series as they
 present as links on the detail page which makes the fix more obviously
 useful.)

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 26689: Prepend letter codes.

This patch prepends the credit_type_code and debit_type_code with
CREDIT_ and DEBIT_ respectively when doing a lookup on the notice letter
code.

Test plan
1) As previous patches, however instead of just naming your notice to
   match the credit_type_code or debit_type_code use
   CREDIT_credit_type_code and DEBIT_debit_type_code respectively.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 26689: Check for specific template then fallback - printinvoice

This patch adds a check for a more specific ACCOUNT_DEBIT template
(named to match the debit_type_code of the credit line) prior to
falling back to the ACCOUNT_DEBIT template.

Test plan
1/ On a patrons account page use the print option on a series of debit
lines with differing credit types (Overdue, Payout)
2/ Note that the same template 'ACCOUNT_DEBIT' is used for all types
3/ Apply the patch
4/ Run step 1 again and note all still print using ACCOUNT_DEBIT
5/ Add a new notice template under the 'circulation' module with a code
that matches one of your account debit types (OVERDUE, PAYOUT)
6/ Run step 1 again and note that where you have added a specific notice
for that debit type it has been used.

Signed-off-by: Sally <sally.healey@cheshiresharedservices.gov.uk>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 26689: Check for specific template then fallback - printfeercpt

This patch adds a check for a more specific ACCOUNT_CREDIT template
(named to match the credit_type_code of the credit line) prior to
falling back to the ACCOUNT_CREDIT template.

Test plan
1/ On a patrons account page use the print option on a series of credit
lines with differing credit types (Payment, Lost Item Return, Writeoff)
2/ Note that the same template 'ACCOUNT_CREDIT' is used for all types
3/ Apply the patch
4/ Run step 1 again and note all still print using ACCOUNT_CREDIT
5/ Add a new notice template under the 'circulation' module with a code
that matches one of your account credit types (PAYMENT, WRITEOFF,
LOST_FOUND)
6/ Run step 1 again and note that where you have added a specific notice
for that credit type it has been used.

Signed-off-by: Sally <sally.healey@cheshiresharedservices.gov.uk>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30678: (follow-up) Change call in opac scripts

Test plan:
See first patch. Add orgcode OCoLC.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30678: (follow-up) Add oclc_number routine and friends

As requested by QA, we should match the orgcode.
Adding a few small routines in a Util module.

Test plan:
Run t/Koha/Util/MARC.t

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30678: Adjust pref description

Test plan:
Look up pref OPACSearchForTitleIn.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30678: Add OCLC_NO as option to OPACSearchForTitleIn

We need to pass OCLC_NO as key to C4::Output::parametrized_url.
We get it from MARC21 035$a and strip the organizational code.

Test plan:
Add e.g. 62385712 in 035$a in a record.
Add e.g. following link to pref OPACSearchForTitleIn:
    <a  href="https://worldcat.org/search?q={OCLC_NO}" target="_blank">WorldCat (via OCLC number)</a>
Open that record on detail page. Click More searches. Follow that link.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30858: (follow-up) Add enumchron to CSV export and fix CSS class

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30858: Add serial enumeration/chronology to item search

To test:
1. Apply patch
2. Go to  Catalog > Item search and do a search for items that you know contain a items.enumchron
3. The result set should now show the items.enumchron
4. Make sure you can properly "Export results to.."

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31121: Fix date formatting in cashup summary modal

Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31204: Clearly indicate edit buttons are for record modification

To test:
1. Apply patch
2. Do a search and select some records
3. On the toolbar (#selection_ops) open the Edit dropdown.
4. The labels should read 'Batch edit records' and 'Batch delete records'.

Note: To see both you must have batchmod and batchdelete permission.

Signed-off-by: Sally <sally.healey@cheshiresharedservices.gov.uk>
Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30838: (QA follow-up) Add unit tests

Two tests added, one in t/db_dependent/Circulation.t to catch the
initial setting of to_address at enqueue time and a second in
t/db_dependent/Letters.t to catch the correcting at send time.

Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30838: (QA follow-up) Add missing semicolon

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30838: Set to_address to smsalertnumber at send

It may be helpful to know exactly what number was used for the sms alert
that was sent. As such, we should ensure it's set at the time of
sending.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30838: Don't fallback through phone numbers for sms

This patch removes the fallback handling for smsalartnumber as the
to_address in notices. We ignore the to_address field in the message
queue at send time for sms anyway and use smsalertnumber exclusively so
having this field populated is just confusing to the end user

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31086: DBRev 22.06.00.020

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31086: DBIC update

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31086: (QA follow-up) Use plain SQL in db_rev

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31086: Notify of problematic holds during upgrade

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31086: (QA follow-up) Improve atomicupdate to be more resilient

We now populate the branchcode field with the first available branch on
the system if we find it to be NULL before we set the NOT NULL

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31086: (QA follow-up) Add unit tests

This patch adds the unit tests for the change to Koha::Hold::store.

We test for and catch the exception introduced for both the create and
update cases.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31086: (QA follow-up) Fix failing tests

This test was missed first time around, branchcode needed adding to all
the calls to AddReserve.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31086: (QA follow-up) Remove DEFAULT from column

There's no need to have a default here, and in fact it doesn't make
sense.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31086: Update existing unit tests

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31086: Prevent placing hold with no pickup location

To test:
1 - Apply patch
2 - Repeat test on previous patch
3 - You should get a warning popup when attempting to save hold
4 - Reload page
5 - Inspect the HTML and manualy set #pickup value of selected option to ""
6 - Attempt to save hold
7 - You get the popup
8 - Place a hold normally (with location) and verify it works

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31086: Throw an exception when a hold is stored with no branchcode

To test:
1 - Apply first 2 patches
2 - On staff client, find a biblio, place hold, select a patron
3 - Before creating hold, right click on 'Pickup location'
4 - Inspect the element
5 - In the console find the top <li> for pickup, and right click
6 - Select 'Delete node'
7 - Save hold
8 - You get an exception/error

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31086: DB update

This update sets branchcode as NOT NULL
default set to 0 as borrowernumber is

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31138: Don't ignore DataTables errors and raise them to the end-user

DataTables is not raising error to the end user, it's only displaying a generic message in the console.
This leads to lot of questions on IRC or the mailing list ("the table is empty").
We can do better and display what went wrong.

We could also imagine add a link to the wiki for the more common problems (won't be implemented here however).

Test plan:
Revert "Bug 31104: Put each link on a separate header" and search for a
patron using a very long string
You should get an alert with the reponse of the server.
In my case:
"""
Something went wrong when loading the table.
414: Request-URI Too Long
"""

You can also make the REST API returns a 500 (bad specs or simply with a
die statement).

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 23838: (QA follow-up) Move include files

As talked with Martin, this patches were originally developed before we
added the modals/ and str/ dirs, but we need to align it with current
way of doing it. This patch does that.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 23838: Add 'view' link to item detail page

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 23838: (QA follow-up) Rename patron-renewal => checkout-renewal

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 23838: Use $patron_to_html formater

This patch updates the renewals modal to correctly format the staff
member who renewed the checkout.

Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 23838: (follow-up) Add missing template filter

Quieting an issue highlighted by the QA tool.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 23838: Use $datetime formater

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 23838: Add renewals modal

This patch adds the display of the renewals modal when appropriate. A
"View" link is displayed next to renewals count where appropriate.
Clicking the link opens the modal that displays the logged renewals.

Sponsored-by: Loughborough University
Signed-off-by: Myka Kennedy Stephens <mkstephens@lancasterseminary.edu>
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Ben Veasey <B.T.Veasey@lboro.ac.uk>
Rescued-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
I rescued this patchset by squashing previous work and updating it to
utilise the new renewals API routes introduced in bug 30275.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31128: Add effective_not_for_loan_status into items response

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31128: Unit tests

This patch adds unit tests for the new effective_not_for_loan_status
which I add to the items api responses in this patchset.

Test plan
1) Run the unit test without applying the next commit, it should fail
2) RUn the test again after applying the next commit, it should pass

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31213: Auto search simple term quoted

This patch does a second search if the first had no results and was a simple single term search

We simply quote the term, rebuild the query, and rerun the search

This targets both Zebra and ES, but is more relevant for ES

To test:
0 - Have Koha running using ES
1 - Add titles your system "Ivy + Bean" "The 6:20 Man"
2 - Search for these titles without quotes - NO results under ES
3 - Search with quotes - you find them
4 - Apply patch, restart all
5 - Search again without quotes
6 - Success!
7 - Confirm search still works under Zebra

Signed-off-by: Sally <sally.healey@cheshiresharedservices.gov.uk>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30458: Include librarian (manager_id) in accountline when using "Payout amount" button

When using the "Payout amount" button, accountlines.manager_id is not
populated. This is not the case with the "Issue payout" buttons.

Test Plan:
1) Create a manual credit
2) Issue a payout using the "Payout amount" button
3) Inspect the payout accountline, note it has no manager_id
4) Apply this patch, restart all the things!
5) Repeat steps 1 and 2
6) Note the new payout accountline has a manager_id!

Signed-off-by: Sally <sally.healey@cheshiresharedservices.gov.uk>
JK: Fix commit title

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 30471: Correct display of default rules

To test:
1 - Browse to 'Administration->Circulation and fines rules'
2 - Make sure you are editing 'Standard rules for all libraries'
3 - Set 'Lost item fee refund on return policy' to 'Refund lost item charge and charge new overdue fine'
4 - Swtich to edit rules for a specific library, click on the dropdown for 'Lost item fee refund on return policy' and note default says 'Use default (Refund lost item charge and restore overdue fine)'
5 - Switch default/all libraries rule to 'Refund lost item charge and restore overdue fine'
6 - On specific branch note the default option is now 'Use default (Refund lost item charge and charge new overdue fine)'
7 - Apply patch and repeat, note the default descriptions now display correctly

Signed-off-by: Sally <sally.healey@cheshiresharedservices.gov.uk>
Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 29001: (QA follow-up) Remove code duplication

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 29001: Fix framework edition when subfields are reordered

There is a flaw when subfields are ordered, the inputs are not retrieved
correctly.
We should not rely on the order but use an id instead.
Test plan:
Create, edit subfields
Modify values from the different subfields
Confirm that values are correctly saved

Signed-off-by: Thibault Kero <thibault.keromnes@univ-paris8.fr>
Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 31202: Don't remove optional SIP fields with a value of "0"

If the value of a SIP field is "0", that evaluates to false, so any calls to maybe_add with a value of "0" will not get added to the SIP response message.

Test Plan:
1) Apply this patch
2) prove t/db_dependent/SIP/Message.t

Signed-off-by: Michal Urban <michalurban177@gmail.com>
JK: Adjust commit title

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Bug 25449: (QA follow-up) Fix basic_workflow.t

Now that we have made item.itype a required field, we need to ensure we
set it for the corresponding selenium test.. this wasn't entirely easy
as the html select is hidden by select2. I had to find the select2 span,
trigger a click event to open it and then find and click an option.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>