Bug 25375: Fix 'available' facet in elasticsearch

Add a new boolean ES field named 'available', which is true if at least
one item is available, which means the item is not on loan, not
"notforloan", not withdrawn, not lost and not damaged

A full indexation is required

Test plan:
1. Apply patch and run updatedatabase.pl
2. Run `misc/search_tools/rebuild_elasticsearch.pl -d -b`
3. Make sure you have some biblios whose items are all unavailable, some
   biblios whose items are all available, and some biblios with at least
   one item available and at least one item unavailable
4. Use the 'available' filter on both opac and intranet and make sure it
   works as expected.

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit d79361c51e9349a5bbbe3212fde40d3ad4d6b946)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 5a48731b8ccad8868296b00845a1dd3bf937b609)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31562: [21.11.x] Treat flags as other borrower fields

Rather than generate a custom hash for these fields, we should treat them as other borrower data fields

To test:
 1 - Edit a patron, note the 'Lost card' and 'Gone no address' fields
 2 - Edit syspref BorrowerunwantedField
 3 - Set gonenoaddress and lost as unwanted
 4 - Edit patron, the fields remain
 5 - Apply patch
 6 - Edit a patron, fields are hidden
 7 - Unhide one of the fields
 8 - Edit a patron and confirm it shows and saves correctly
 9 - Unhide the other field
10 - Confirm it can be edited and saved

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Bug 31562: Remove gonenoaddress and lost from OPAC borrower field preferences

These are not shown, so cannot be unwanted or mandatory

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Bug 31562: Respect mandatory fields pref

To test:
1 - Set gonenoaddress and lost in BorrowerMandatoryField
2 - Edit a patron, confirm fields are required

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 3d6607567cd5ed274245428ce66e157863febb9c)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31367: Add subtotals if budget has children

On bug 29287 we added child funds if a fund had no parent id

It turns out funds can have a parent, and still have children (sub-sub-funds)

This patch simply checks for children, rather than checking for lack of parent.

Totals are already calculated correctly in GetBudgetHierarchy

To test:
1 - Add a fund "Fund" to a budget - assign 1000 dollars
2 - Add a sub-fund "SubFund" - assign 100 dollars
3 - Add a sub-sub-fund "SubSubFund" - assign 10 dollars
4 - Add to a basket $5 order from SubSubFund
5 - Add to a basket $50 order from SubFund
6 - Add to a basket $500 order form Fun
7 - View acqui home - note 'SubFund' does not include total from 'SubSubFund'
8 - Apply patch
9 - Confirm all sub-sub-funds totaled in sub-funds and that fund total correctly adds sub and sub-sub funds

Signed-off-by: Andrew Fuerste-Henry <andrewfh@dubcolib.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 6523db73214c57d6a6b8ea2d4c3731445ccddf1f)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 14dd1ab9e4f0bdcf12269afd3a72569482aedf97)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 27550: Add unit test for CloneBudgetHierarchy

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit ca338bae1e751228a1107237574b8f335c12b7bb)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit c7074be1c93ef8a1fc81895d448d377740414cd7)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 27550: "Duplicate budget" does not keep users associated with that budget

When a budget is duplicated, Owner is copied but not Users

Test plan :
Home > Administration > Budgets
1)Create a budget to allow (active or inactive)
2)Give it some funds by filling up the form, especially Owner and
User(s)
3)Submit then go back to the Budgets dashboard
4)Duplicate the Budget and click on its name
5)Edit it and notice that User(s) is empty
6)Apply the patch
7)Repeat from 1) to 5)

Signed-off-by: Samu Heiskanen <samu.heiskanen@hypernova.fi>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit ed97960cd7f8105eb5f913770089218cf7dd559d)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 2d5529915b55acf0563e85e5ba68a6bbff79d5f7)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31564: Pass start label in PDF link

To test:
1 - Tools->Lbel creator
2 - Make a new batch
3 - Add an item
4 - Click 'export' on the single item
5 - Enter a starting label position greater than 1
6 - Click Export
7 - Click Download as PDF
8 - Note label in 1st position
9 - Apply patch
10 - Repeat and note label position now honored

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 0c3f50d1575099f254884b0dcb2c97d41851a85b)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 03a127afbd61ccf46eba01318bc710c9a55f9d17)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 28167: Remove uninitialized variable warnings from circ/set-library.pl

With UseCashRegisters syspref disabled going to the page
/cgi-bin/koha/circ/set-library.pl in staff interface and setting a
library gives the following errors in plack-intranet-error.log:

Use of uninitialized value $register_id in string ne at /kohadevbox/koha/circ/set-library.pl line 79.
Use of uninitialized value $referer in pattern match (m//) at /kohadevbox/koha/circ/set-library.pl line 114.

In the if clause $userenv_register_id appears to be typoed, it should have
been $register_id as $userenv_register_id is always defined. As for the
$referer variable, it is undef if there is no referer so let's just initialize
it to an empty string for the regex so it doesn't give the warning.

To test:
 1) Go directly to /cgi-bin/koha/circ/set-library.pl by typing it in
    the URL bar and set a library
 2) Make sure plack-intranet-error.log doesn't contain the above mentioned errors
    after applying this patch

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 8e25fa1dc5267f6fb6148412f0774c6880a2e7c3)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 513b1160bcfef9ff6b482b96c0607f3e61ad961d)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 30890: Fixed ILL breadcrumb

Added correct breadrumb depth when managing requests.

1. Viewing a request on a ILL request page
2. Making sure the breadcrumb has three depths
   and the manage requests has the correct
   request id

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 49577829d969fba34bf6655da4e0356f1f0ec5d2)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit d84dcce94665f6a892899abd182aeeae0f0b2da9)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31598: Fix random failure on Jenkins for Upload.t

At some point the warn is made conditional for permanent
files. So it failed when TestBuilder picked a zero value
for permanent. Trivial fix.

Test plan:
Run t/db_dependent/Upload.t a few times

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 9a92ee31945af6f535e8be81b3410d13affd8b5f)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit d7bf6f7084cfe00c7f1f4cf17aa525f6c2893f11)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31497: (follow-up) Expand ID suffix, fix password2, remove debug

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit f0642e4a1fca066ca48cfade8e8d50c4e7611e63)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit bd32ab28d6e5a74034a3541c08337b4f1d5bf400)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31497: Prevent ID clash on quick add fields

The quick add clones the original fields, including ids. This can cause some clash when
other JS is running on the page.

This patch updates the ids of the fields before adding to the form, which prevents bad
copying/clearing of fields

To test:
* Go to patrons module
* Click on Quick add new patron (I chose Patron as category)
* On sample database these fields are marked as mandatory:
  * Surname
  * Cardnumber
  * Library
  * Category
* Fill in Surname, leave cardnumber empty
* Save - mandatory message is shown
* Fill in cardnumber - save
* The patron is saved
* BUT: cardnumber is empty!
APPLY PATCH
* Repeat plan above
* Cardnumber is correctly saved

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 0f599e2a2a8c9f6caed4fe4ae6060b106d2a1004)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit c8793b1cd3eacece7ccadc9fc6a0c8248a73bba2)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 30359: Unit tests

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit b1e3398e84674f443d1b3c348058e45291f45e8e)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 79bc7a8de5ef9dd1c139ae9770d22e93acbc95c4)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31154: (QA follow-up) Fix UI form builder

The change in batchMod.pl is not needed when we fix the builder.
But we could at least add a comment there!

Note that passing the biblionumber of the first item does not
make sense since we are modifying items from several biblio
records normally. We most probably do not want this MARC
record's itemcallnumber (via the syspref) in all our items.

Test plan:
Test batchMod with and without the Populate fields checkbox.
Fill syspref 'itemcallnumber' with e.g. 084a.
Check Populate, and try again.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit eb292431401a7ed0782427412eb1208fa7716e4c)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit e5c11f993369cbcd2507a604fece7254263f8ac0)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31342: Add execution locking to process_message_queue.pl

This patch makes the script use an execution lock as provided by
Koha::Script. Previous attempt got too complex and contentious, and
given we have a simple way to achieve the same thing, I decided to
submit an alternative approach.

To test:
1. Apply this patch
2. Add the following content to misc/cronjobs/process_message_queue.pl
   on line 83:

  sleep 100;

3. Save the file
4. Have two terminals open
5. On the first one, run:
   $ kshell
  k$ misc/cronjobs/process_message_queue.pl
6. On the second one, run the same
=> SUCCESS: The second one dies telling it had to skip the run
7. Undo your changes:
   $ git checkout misc/cronjobs/process_message_queue.pl
8. Sign off :-D

Sponsored-by: ByWater Solutions
Signed-off-by: Liz Rea <liz@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit a80a96b93381a12740b8548b5d92a55bbec0b085)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 9c6d9f51060f75018fc2111af40553659f52899b)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31531: Remove unnecessary use directives on opac-memberentry.pl

This patch just removes two redundant use directives in opac-memberentry.pl

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 6c893f7372b6a080435bd493730530c7f275cd06)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit af0057ae323470580798db914385ee606f4a1952)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31299: Fix duplicate output in search_for_data_inconsistencies.pl

In search_for_data_inconsistencies.pl when there are several inconsistencies on same framework, the second output of items contains the first one.

Looks like it is since Bug 21466

Test plan using koha-testing-docker :
1) Create 2 inconsistencies on the same item via SQL :
   update items set itemlost = 9 where itemnumber=900;
   update items set notforloan = 8 where itemnumber=900;
2) Without patch
3) Run ./misc/maintenance/search_for_data_inconsistencies.pl
=> You see duplicate output :
== Wrong values linked to authorised values ==
* The Framework *BKS* is using the authorised value's category *NOT_LOAN*, but the following items.notforloan do not have a value defined ({itemnumber => value }):
 {900 => 8}
* The Framework *BKS* is using the authorised value's category *LOST*, but the following items.itemlost do not have a value defined ({itemnumber => value }):
 {900 => 8} {900 => 9}

4) Apply patch
5) Run ./misc/maintenance/search_for_data_inconsistencies.pl
=> Fixed :D
== Wrong values linked to authorised values ==
* The Framework *BKS* is using the authorised value's category *LOST*, but the following items.itemlost do not have a value defined ({itemnumber => value }):
 {900 => 9}
* The Framework *BKS* is using the authorised value's category *NOT_LOAN*, but the following items.notforloan do not have a value defined ({itemnumber => value }):
 {900 => 8}

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 3d0015fe6932f2fed3aba79b1fc99ecb6d1d3504)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit e731d4cff4f6653c6c3e48d1bef8940bff228e2a)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31558: Fix image drag and drop in TinyMCE

Bug 26949 upgraded JS lib TinyMCE to 5.9.2

This version has a known issue on images drag and drop :
https://stackoverflow.com/questions/64782955/tinymce-inline-drag-and-drop-image-upload-not-working

Test plan :
1) Go to Tools > HTML customizations
2) Create a new entry (not using text editor)
3) Drag and drop an image
=> Without patch you see an error message from TinyMCE 'Dropped file type is not supported'
=> With patch : It works !

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit d0f09eaa6412eb83ebe7e17faa550628fa7e6cab)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 278cd8b827f13a68f3375f828483d40d5eaa7b31)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 28553: (QA follow-up) default to auto renew email when cron switched used

This patch defaults to using "email" rather than "sms" notices
when AutoRenewalNotices is set to use the --send-notices cron switch

Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 88f63e01d29917e9323c57be97aaa91b9e00fd92)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 681730ee1af9ad167f9526903eee6b6681d7d9ff)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 28553: Patrons can be set to receive auto_renew notices as SMS, but Koha does not generate them

Following bug 18532, one can set a patron's messaging prefs to deliver an auto-renew notice via SMS.
However, the auto_renewals cron only knows how to generate email notices. We should not allow the selection of non-functional transport types.

Test Plan:
 1 - Set pref AutoRenewalNotices - 'according to patron messaging preferences'
 2 - Set circ rule with:
    Loan period: 5 days
    No renewal before: 5
    AutoRenewal: Yes
    Renewals allowed: 5
 3 - Set SMS Send driver to Email
 4 - Find a patron and set them to receive auto renewal notices via SMS and no other transport
 5 - Provide a proivder and sms number for the patron
 6 - Checkout an item to the patron, set due date to yesterday
 7 - Run auto renewals:
    perl misc/cronjobs/automatic_renewals.pl -v --send-notices -c
 8 - Item renewed, no message sent
 9 - Check in item, checkout again due yesterday
10 - Apply patch
11 - Browse to Tools->notices and slips
12 - Edit AUTO_RENEWALS and AUTO_RENEWALS_DIGEST to copy email template to sms and add a title
13 - perl misc/cronjobs/automatic_renewals.pl -v --send-notices -c
14 - Item renewed, confirm patron has a notice in their account
15 - Check the digest option, check in item, checkout again as due, repeat cronjob and note digest notic
16 - sign off

https://bugs.koha-community.org/show_bug.cgi?id=30355

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit fef29151e0dd6287f8e4e49effc9d9666bf87582)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit e672ea689221f8c6dd7e139acd3ffbd92c7c569f)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31239: (QA follow-up) Fixing ternary formatting

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 4481817701f0789f407f8bdfdcaeeb49c5239e5e)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit fd2bc6b8978a4b35749cd7ef20d033acc4412b73)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31239: Fix incorrect AV linked to a mapped biblio column

If an AV is linked to a MARC field mapped with a biblio column, the
search_for_data_inconsistencies.pl script might explode with

The method Koha::Biblioitem->title is not covered by tests!

Trace begun at /kohadevbox/koha/Koha/Object.pm line 875
Koha::Object::AUTOLOAD('Koha::Biblioitem=HASH(0x556b67fa7168)') called at misc/maintenance/search_for_data_inconsistencies.pl line 246

Test plan:
For a given framework, pick a biblio using it
Link 245$a with an authorised value category
Run the script
=> Notice that with this script applied you will see the warning
=> Without this patch you got the error

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 54711942f1b34fc9d94a61f42734525a03db893a)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 2e363d0d0c7d20e9b386473a7e77e2478ba4c015)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31154: Avoid unecessary loop

We don't need to loop over the subfield and enter this block if there is
no record.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit b582f7d495fceda68a7972f7eab0d4e5d16bdcdf)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 23d0af2eab453deabf76b805a16ef1e97c795f7e)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31154: Add test

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit edf8640cff4c929196d1768152a705fb350317a0)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 8735f0732ccd5904d78c0beeef3b2692579e377f)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31154: (QA follow-up) Fix UI form builder

The change in batchMod.pl is not needed when we fix the builder.
But we could at least add a comment there!

Note that passing the biblionumber of the first item does not
make sense since we are modifying items from several biblio
records normally. We most probably do not want this MARC
record's itemcallnumber (via the syspref) in all our items.

Test plan:
Test batchMod with and without the Populate fields checkbox.
Fill syspref 'itemcallnumber' with e.g. 084a.
Check Populate, and try again.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit eb292431401a7ed0782427412eb1208fa7716e4c)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit e5c11f993369cbcd2507a604fece7254263f8ac0)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31154: Pass biblionumber to Koha::UI::Form::Builder::Item->new from batchMod.pl to prevent error 500

When one tries to batch mod items and has "Use default values"
checkbox checked on and they have value in syspref "itemcallnumber",
Koha dies on error 500. In logs it reads:

Can't call method "field" on an undefined value at
.../Koha/Koha/UI/Form/Builder/Item.pm line 164.

This happens because in batchMod.pl line 269 we don't pass
biblionumber as parameter to Koha::UI::Form::Builder::Item->new
and thus Koha fails to fetch biblios marc record.

To test:
1. Confirm you have/set value to syspref "itemcallnumber" e.g. 084a
2. Find some item(s) to modify
3. Provide barcodes of those item(s) and check checkbox "Use default values"
4. Hit "Continue"
=> error 500 is raised
5. Apply patch.
6. Repeat steps 3. and 4.
=> no error is raised

Sponsored-by: Koha-Suomi Oy
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 355ab980cc306a61e5e9e62608c5d9d1bf8e3b37)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit babf0f118acb88c1073f15c5f2f5815646b21752)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31513: (QA follow-up) Keep price formatting on non-editable text fields

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit c4e8f0af70344f263f64eccccf8817ddc18f8585)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 6cc62528cd5ba04665268f6ee0afc586c1572933)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31513: Remove price formatting form data- attributes

We cannot use data formatted with a decimal comma for
calculations. So we need to make sure that the data- attributes
passed to the modals are in 'database format' with decimal .

This will fix the NaN errors on the forms, but might also
negatively effect formatting in some spots.
Formatting should be fixed separately as this is for sure the
right first step, but we need to put in more work for displays.

To test:
* Set CurrencyFormat system preference to FR
* Go to any patron account > accounting tab
* Create a credit
* On Transactions tab: Payout amount at bottom = Verify NaN in input
* On Transactions tab: Issue payout on the credit line = Verify NaN in input
* Create a manual debit and pay it
* On Transactions tab: Issue refund = Verify NaN in input
* Apply patch
* Repeat test plan
* Now all input fields should be set with the correct amounts

https://bugs.koha-community.org/show_bug.cgi?id=31486

Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 9d02c71166258425f58dc020b9e7a9b95f895bb9)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit b23eda9959dcf60fe1799cc8632cb975103617eb)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 29409: Check for data that violates a constraint to avoid update problems

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit e4455a589372e3205f9a7456478ad4585c90bedb)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 6ea331417ea4a114ab9b5aec2a7885118942be21)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 29409: Check for existence of constraint before deletion

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 012e5bff1c79d8a2cc580a43c832ace5e51bec79)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit b2f44c61411fe051097570fe7d98591e628b0f01)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31516: Add error message for viewing enrollments for a non-existent club

To test:
 1) Go to non-existent club's enrollment page, e.g.
    http://localhost:8081/cgi-bin/koha/clubs/club-enrollments.pl?id=1010101001000
 2) Notice the not fully filled header text and empty table
 3) Apply patch and revisit the page to see the proper error message given

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit c757abd675c538a55c93bf3e337e16c0911d06e7)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 0de714bbadb050c3a1eecec7670ec38bf8a69cf3)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31486: Redirect to proper page after message deletion

To test:
1. Go to a patron account and create some messages
2. Try deleting some from both the patron details page and the patron checkout page.
3. Notice that is you are on the patron checkout page and delete a message you are then redirected to the patron details page.
4. Apply patch
5. Try deleting messages from both pages and confirm you are redirected back to that same page and the message is properly deleted.

Signed-off-by: Andrew Fuerste-Henry <andrewfh@dubcolib.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit a93d6fa8530aa2b92d7b443e1361e4428c1c4f1b)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 0c91e9499929f8a5ff96af64bed6971f4a6db57c)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 29658: Fix crash on cancelling cancelled order

Found this crash in our 20.11 logs:
Cannot insert order: Mandatory parameter biblionumber is missing at /usr/share/koha/acqui/cancelorder.pl line 60.
 at /usr/share/perl/5.28/Carp.pm line 289
        Carp::croak('Cannot insert order: Mandatory parameter biblionumber is missing') called at /usr/share/koha/Koha/Acquisition/Order.pm line 79
        Koha::Acquisition::Order::store('Koha::Acquisition::Order=HASH(0x55f3760e2860)') called at /usr/share/koha/Koha/Acquisition/Order.pm line 189
        Koha::Acquisition::Order::cancel('Koha::Acquisition::Order=HASH(0x55f3760e2860)', 'HASH(0x55f375a17ec0)') called at /usr/share/koha/acqui/cancelorder.pl line 60

Not sure how to reproduce this one as it happened. But might be related to repeated clicking, backspacing etc.

Test plan:
Create a new basket and order.
Open this same basket in two browser tabs.
Cancel the order line (delete catalog record) in tab 1.
Go to second tab, try again.
Without this patch, it will crash. With this patch, an error message.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 2a3ba5f8253c6bafef17caf03645d70a90a96dea)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 22e6fd7cd86087bd6a938955ed4d2df678090432)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31250: (QA follow-up) The future will be easier

As requested by Jonathan, we need more flexibility ;)
Here it comes.

Test plan:
Run t/CookieManager.t

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit cffd36bcb47439d80401e52091f0d2808bc52cb6)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit d865ce1665c403e8fe1ee887eac7a64e42a617b1)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31250: Add catalogue_editor_ cookie to koha-conf deny list

Test plan:
Add this change to your koha-conf.xml.
Flush, restart.
Test if the cookie is kept now in the interface.

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit cf8e560e37c24b0cc7fd3e1aeb286de5e370ea37)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit bf7a65e13ec2fcf7f5afa4a33ed296d96915f14b)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31250: Deny clearing cookies with numeric suffix

This change allows us to add catalogue_editor_ to the deny list
in koha-conf.xml and keep cookies like catalogue_editor_123.

Test plan:
Run t/CookieManager.t

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
To test:
1 - Sign in to staff client
2 - Search for records and edit one
3 - Switch to advanced editor
4 - View cookies (F12/developer panel/storage tab)
5 - Note cookie like 'catalogue_editor_##' with value 'advanced'
6 - Log out
7 - note cookie value deleted
8 - Log in and search/edit a record
9 - Basic editor loads
10 - Apply patch
11 - Add line to koha-conf as described in second patch
12 - Restart all
13 - Switch to advanced editor
14 - Cookie value updated
15 - Logout
16 - Cookie value remains
17 - Log in and search/edit
18 - Confirm advanced editor loads

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit cb036faf2c534a14708b517cea40d98fc6e344c2)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 58eb3713ec64c35010c663827f943de7b4b9f0da)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31542: Home page links correct font-family

In koha-tmpl/intranet-tmpl/prog/css/mainpage.css the rule of
font-family for big links is not complete :
  font-family: verdana, arial;
It is missing sans-serif in case computer has not those fonts.
On my computer I dont have them so I see a serif font.

We can remove this rule, inherited rule is serif fonts.

Test plan :
1) Use a computer without Arial nor Verdana fonts (basically a Linux)
2) Look at home page big links : Circulation, Patrons ...
   => Without patch they have serif font
   => With patch they have sans-serif font

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit ca6c132bace1b8e3ef770fb376e0101c8bfd7cd8)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit af028799dcf2e77bc48a20ba6620134d83479abb)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 30483: DBRev 21.11.13.001

Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 30483: DBIC

Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 30483: Make issues.borrowernumber and itemnumber NOT NULL

Note: This change does NOT apply to old_issues, where constraints
may result in nullifying these columns.

Test plan:
Run dbrev.
Try checkout, checkin.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 1f44891fd835080133fa1a73634ced0e68b96892)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit ce7a85ad645cce000d79844c931f10c5498eb816)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31401: (QA follow-up) Remove Koha.ArePluginsEnabled, it is no longer needed

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit c5810c360544de96bf3fb2000c135836b6e2ee10)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 0c6eef3816c60f0645c8c3da4ea6e166dfc1b029)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31401: Update administration sidebar to match entries on administration start page

This makes sure that all entries on the administration start page
have a matching entry in the administration sidebar that shows
on the left side if you are on any of the administration sub pages.

Changes made:
* Rename 'Classification sources' to 'Classification configuration'
* Make Plugins entry show and appear in correct spot
  This relied on the variable plugins_enabled that wasn't available
  in all the different templates. I therefore moved it to Auth.pm
  and cleaned up the code for the admin start page.
* Move 'MARC overlay rules' and rename to 'Record overlay rules'

To test:
* Make sure plugins are enabled and visible on admin start page
* Compare admin start page and sidebar
  * Sequence should be the same
  * All entries should appear on both pages
  * Naming should be the same

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit aa90b60ea2979279a5e134eb9cbc7cb3d6075417)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit cb5049a7bc2306863abcffbbbcdd3d4c95b30c63)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 26626: (follow-up) Be consistent with icon and accesskeys

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit cc66a51e0730bc73c0e8ecfe970b0c52d63d3d62)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 20995f10fcf8a5802cdb084d6026b7b3278c8f44)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 26626: (follow-up) Update icon selection

This patch changes the "Ignore" button to use an X icon and the "Cancel"
button to use a "Trash" icon.

Signed-off-by: Barbara Johnson <barbara.johnson@bedfordtx.gov>
Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit e62b8bde068dcac607f155d8267c2bc41cd4f023)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit e939769702e7ee9814873020829daf8ee513aba0)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 26626: Add 'Ignore' option to confirmed holds modal

This patch adds 'Ignore' to the 'Hold already waiting' modal

To test:
To recreate:
 1 - Place a hold for pickup at library A
 2 - Checkin the item at Library A
 3 - Note the modal has 'Ignore' option
 4 - Click it, nothing happens
 5 - Check in the item again, same modal
 6 - Click 'Confirm'
 7 - Check in the item again
 8 - Now 'Ignore' is replaced by 'Cancel'
 9 - Click 'Confirm' to dismiss modal
10 - Apply patch
11 - Check the item in again
12 - Note 'Ignore' button before 'Cancel' button with '-' icon
13 - Go to biblio holds page
14 - Revert waiting status
15 - Check item in again
16 - Note ignore now has '-' (minus) icon

Signed-off-by: Barbara Johnson <barbara.johnson@bedfordtx.gov>
Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 5e8da261957e98c5a56e37a910adcb943c8cf67f)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 9997adba1e2dab155483a686bf330c43bf77a0b0)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31023: Quote authtypecode in FindDuplicateAuthorities

To test:
0 - Have Koha using Elasticsearch
1 - Set  QueryRegexEscapeOptions to 'unescape escaped'
2 - Attempt to add a new 'GENRE/FORM' authority record
3 - On save you get a 500 error
Unable to understand your search query, please rephrase and try again.
 at /kohadevbox/koha/Koha/SearchEngine/Elasticsearch/Search.pm line 102
4 - Apply patch, restart all
5 - Attempt to add a new 'GENRE/FORM' authority record
6 - Success!

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit dee864c1edff70f58dfaffcffa102e0663400425)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 970cf815145b2f8e3d4a9e24a885373c25ba351f)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31023: Unit tests

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit df184560a5de428ed6eaaff437cb5dd450f45726)
(cherry picked from commit 4b9106d995ef0f686a941cf914906eae45e5328e)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 28290: (follow-up) Unit test

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit e36e6c55a7854b7f6e5f7c9d90f57f9b5e5c0f0a)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 9eb3d8332a16b89281f436f97c650fc0de953540)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 28290: Don't send subfields to 'as_string' if none to send

This fixes an error where we send a blank string to
Marc::Field->as_string

The subroutine fails as the regex is marked as incomplete as there is
nothing between the brackets

Also fixes a missing hidden input for 'item_action' when no items are
staged during import

 1 - Define a new matching rule with no subfields
    Match-threshold: 1000
    Record type: bibliographic record
    Search-index: Other-control-number
    Score: 1000
    Tag: 035
    Subfields:
    Offset:
    Length:
 2 - Attempt to stage a record with an 035 using this rule (i will
     attach example to bug)
 3 - Staging fails
 4 - Stage again with no matching rule
 5 - Manage the staged record, try to apply the matching rule
 6 - It fails, 500 error
 7 - Apply patch
 8 - Restart all the things
 9 - Apply rule again, it works!
10 - Stage the record again using the rule for matching
11 - It stages and matches!

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit eded6edacc5e3bf8dd0be21ed05842c3b78aadce)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 0c8e4780d78c0c90e2fb5d6c08868104a91e1f2c)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 29608: Made so doesn't require full permission

Made so the numbering sequence does not need to require full
permission.

1. Edit number patterns a confirm you don't need full permission

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 36d88b135d072f9335db46093c0e440e0cdfb2c3)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 41346b523e75cbd6f8c6818effd4c0b65fb9597d)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31364: Check length of the multiple syspref array instead whether it is null

The multiple systempreferences are always defined as an empty array at
the minimum, so in order to determine whether we need to remove all
the selections from the systempreference we should check whether the
array is empty and not whether the array doesn't exist.

To test:
 1) Set at least 1 value for OPACHoldsIfAvailableAtPickupExceptions
 2) Remove all the values for OPACHoldsIfAvailableAtPickupExceptions
 and notice you get the message "Nothing to save"
 3) Apply patch and repeat, removing all the values should work now.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit ce1efe7de0d71362f930a8fc5415fda797531a0b)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 191aad389e84fa60b4ef168b9fc97878cb6fcdfa)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31488: Update Max checkout text

This changes text from "you have checked out too many items" to
"You have reached the maximum limit of items checked out".

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 7a90be2ff221df5e7ab575e89063f6e8dcc99c29)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 6ccddb4be15f664f31538d0f0de0e059df03ea3f)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 30231: (bug 5927 follow-up) See (rejected) forms of series entry visible in search result (OPAC and intra)

When using zebra with IncludeSeeFromInSearches on, with traced series, the rejected
forms (4XX) of series entries are visible in result lists, both in OPAC and intra
interfaces.

Test plan:
=========
1. Have a MARC 21 installation with zebra, and with IncludeSeeFromInSearches on.
   Have an authority records for series (130, or 100/110/111 with
   title) with rejected forms (4XX fields).  So the auth record should
   contain at least:

   130 _0 $a Series title
   430 _0 $a Some other title under which the series in known

   Use (each) authority record in at least two biblio records (in appropriate
   fields, i.e.  800/810/811/830 or even deprecated 440, if it makes use
   of UNIF_TITLE).
   Don't forget to properly input also 490 (for 8XX). So the biblio record
   shold contain at least:

   245 nn $a Work title.
   490 1_ $a Series title
   830 _0 $a Series entry $9 <koha_id>

   Reindex.
2. Perform a simple biblio search (in OPAC and in intra) with the title
   from auth record.
3. You shold get a list of results.  See the Series information--you
   shold see not only the accepted entry from 1XX in auth but also the
   rejected forms from 4XX.
4. Apply the patch.
5. Repeat 3. You shold see only the entry form of series now.

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 3f90b7b43d0ed927ddac92af5be226d1ede6429f)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 8717ea25ed1e4f8bb70eb3558dd8279a1b9ea547)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31356: Use hold expiration date instead of adding days to the waiting date

The Talking Tech outbound script currently just adds the
ReservesMaxPickUpDelay to the waiting date to calculate the
"expiration date", but it should use the actual hold expiration
date which may differ from this naive calculation based on
various system preference values.

Test Plan:
1) Create a holiday for tomorrow
2) Set ReservesMaxPickUpDelay to 5
3) Set "Days mode" to "Same week day"
4) Enable ExcludeHolidaysFromMaxPickUpDelay
5) Enable TalkingTechItivaPhoneNotification
6) Create a hold and fill the hold so it is waiting
7) Enable "Hold filled" phone notices for that patron
8) Create a 'phone' version of the HOLD notice
9) Run ./misc/cronjobs/thirdparty/TalkingTech_itiva_outbound.pl --type RESERVE -w 0
10) Note the output has the expiration date ( 15th colume ) 5 days from now which is *not* the hold's expiration date
11) Apply this patch
12) Repeat the command from step 9
13) Note the expiration date column now matches the holds actual expiration date!

Signed-off-by: Kyle Hall <kyle@bywatersolutions.com>
Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit f6a08737c9ad9c85e98887e3a55cf31029c5a8c8)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit b9e27c399602f7a343ede597c87d4c3d8ea710d8)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 30262: Trim whitespace off tracklinks.pl URLs

This patch trims whitespace off tracklinks.pl URLs so that
the URL is in the same format as the URLs returned by
C4::Biblio::GetMarcUrls, which is used by tracklinks.pl internally.

Test plan:
0. Apply patch and koha-plack --restart kohadev
1. Go to http://localhost:8081/cgi-bin/koha/cataloguing/addbiblio.pl?biblionumber=29
2. Add a 856 with $u of 'https://koha-community.org ' (note the space at the end0
3. Save record
4. Change 'TrackClicks' syspref to 'Track'
5. Go to http://localhost:8080/cgi-bin/koha/opac-detail.pl?biblionumber=29
6. Click on the online resource (eg
http://localhost:8080/cgi-bin/koha/tracklinks.pl?uri=https%3A%2F%2Fkoha-community.org%20&biblionumber=29
)
7. Note you are redirected to https://koha-community.org

Signed-off-by: Magnus Enger <magnus@libriotech.no>
Added the link with a space at the end. Verified that without the
patch, clickin on "Click here to access online" gives a 404. After
applying the patch the redirect works as expected.

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 5fe49c7db1113026d242ad081c6b100fc8bae76e)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 088139a89eb8a2676fd9e29f8544dee777ef883c)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31379: Change results per page text for default

Since Bug 14715, one can change number of results per page.
The actual text in combobox for default value is : "Library default:
20"

This text is hard to translate correctly.
For example french translation use "site par défaut" wrongly meaning
"default library".

I propose to simply use "20 (defaut)".

Test plan :
1) Go to system preferences and set both 'numSearchResultsDropdown' and
   'OPACnumSearchResultsDropdown' to 'Show'
2) Do a catalogue search in the intranet using a search term that will
   bring lots of results
3) Look at restults per page combobox
4) Same for OPAC

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 29ad53a682f339a6d9fea028b1951898c6dff662)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 6cefb36995b260e23840bd3891f5a8f976e5ff82)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31863: Account for changelanguage element when adjusting editor height

To test:
1. Turn on 'EnableAdvancedCatalogingEditor'
2. Go to a record with enough MARC data that it should take up the entire width of your screen,
3. The CodeMirror editor will only be 300px.
4. Resize your screen, nothing happens to the editor height
5. Apply patch
6. Look at the same record in your editor, the editor should go to the bottom of the page now. Resizing your screen should resize the editor.
7. Have some additional language packs installed, and change the language system pref so the '#changelanguage' option will appear.
8. Make sure the editor height is still being properly set.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 9bee75e107178e4ca907cc3a8c7399f3a128fbe6)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 5abc149682d346555b0393afb17221a4138aba86)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31421: Add tests

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit f2519aec6fa2a3af0957faa60b7764471fa1305f)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit a7b67c08ac74565ef26f4277ecc6121744f16d2b)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31421: Add a limited option to template plugin

This adds a specific 'limited' routine to the Categories plugin, and uses
it for the new patron dropdowns

To test:
1 - Limit a patron category to some branches
2 - Sign in not at one of those
3 - Confirm the new patron dropdowns don't include the category

Bug 31421: (follow-up) Add POD

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 1fc56e0e47d07d2eadcdb62c6a5740aa9d51c6b2)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit e207c09ee303b7b957e3f260531ca1fb22666536)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31421: Do not apply library limits to Categories plugin

This patch simply changes from using 'search_with_library_limits'
to 'search'

The limits I think are intended to prevent users from setting a patron to the wrong category,
they are not meant to hide patrons. Or rather, they don't do that, patrons are returned in search
but cannot be processed. The dropdowns as well won't contain the category, so you can see these
patrons, but not filter by the category

Bug 31422 and bug 31032 point out further issues with limits.

For now, let's fix searching

To test:
1 - Edit Patron category 'Patron/PT'
2 - Try to do a patron search that returns patrons in this category
3 - See 'Processing' and error in JS console: Uncaught TypeError: categories_map[data] is undefined
4 - Apply patch
5 - Restart all and repeate
6 - It works now!

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 29754226ea3b6bdaf896e5b8d19f44b1ed551e11)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit fbde769aec127ab899a52197d9c0b176c54b87c4)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 29603: Compiled CSS

Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 29603: Fix responsive behavior of facets menu in OPAC search results

The OPAC was originally featured a search results facets box that would
collapse at narrower browser widths and show a "Refine your search" link
under the breadcrumbs menu. At some point this broke, with the "Refine
your search" link appearing at the bottom instead.

This patch revises the page to restore the previous behavior.

To test, apply the patch and rebuild the OPAC CSS
(https://wiki.koha-community.org/wiki/Working_with_SCSS_in_the_OPAC_and_staff_client).

- Perform catalog search in the OPAC which will return multiple results.
- View the page at various browser widths. At narrower widths the facets
  menu should appear as a "collapsed" box with a "Refine your search"
  link.
- Clicking the link should toggle the facets menu visibility.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
(cherry picked from commit 915815ddae7a69e549e0114eddd2be6a44f1486b)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31219: Prevent JS injection in patron extended attributes

We are sanitizing other attributes but "extended patron attributes".

Test plan:
Make a patron attribute editable at the OPAC
Edit an existing patron, or register a new one
Use a script tag in the new value ("<script>alert("booh!")</script>" for
instance)
With this patch the value is remove if containing an HTML tag that is
not br b i em big small strong (see C4::Scrubber)

Signed-off-by: Mark Hofstetter <koha@trust-box.at>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit cf773c9f1c21cd67fbb0475770b121d64bc5960f)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Update release notes for 21.11.13 release

Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Increment version for 21.11.13 release

Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Fix translations for Koha 21.11.13

Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Translation updates for Koha 21.11.13

Bug 31033: Explicitly define max child processes for SIP server

By default, the SIP server appears to only use 1 child process for
responding to SIP connections.

This change makes this explicit in the configuration, which should
make it so that people who need more than 1 simultaneous SIP connection
can know to just increase the value for the "max_servers" parameter
in the SIPconfig.xml file.

Test plan:
1. Add "max_servers='1'" to your SIP configuration file
2. koha-sip --restart kohadev
3. Open 3 terminals
4. Run "telnet localhost 6001" on 2 terminals
5. On the 3rd terminal, run the following:
ss -l -n -t
ps -efww | grep "sip"
6. Note that there are 2 processes called
kohadev-koha-sip: perl /kohadevbox/koha/C4/SIP/SIPServer.pm /etc/koha/sites/kohadev/SIPconfig.xml
One of these processes is the parent of the other
7. The Recv-Q in the "ss" output should show 1
(This means that 1 of your telnet connections is in the server's TCP backlog)
8. Celebrate as the configuration works as expected

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 4a6567274d944814c432ab7b3697c545e1459800)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit e63f552a888e5d0f1746e23e077fb67969f09f72)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31387: Fix othernames required label

1. Mark othernames as required via PatronSelfRegistrationBorrowerMandatoryField
2. Go to the self-reg form, the field is required but the span that makes it look required stays hidden. (<div class="required_label required">Required</div>)
3. Mark initials as required via PatronSelfRegistrationBorrowerMandatoryField.
4. Now the label_required field appears for othernames in the self ref form
5. Apply patch
6. Try again, the label should now properly show when othernames is marked as required

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 37002a81c37e759bf80e663d3e75ce7c0700feef)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit f2bb0864fab63869abd43d4faec3c2deb6efd7db)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31348: Make koha-plack stop gracefully

0. Apply patch
1. cp /usr/sbin/koha-plack /usr/sbin/koha-plack.bak
2. cp debian/scripts/koha-plack /usr/sbin/koha-plack
3. Now at roughly the same time do the following two actions:
    3a. Go to http://localhost:8081/cgi-bin/koha/about.pl
    3b. koha-plack --stop kohadev
4. Note that the HTTP request completes
5. Click on "Home"
6. Note that the error is "Service Unavailable" rather than "Proxy Error",
which would've occurred with an interrupted HTTP request/response

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 5155e43b04a4283bb13eedd39cdcb9e12693630d)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 6150bd7d812aafd8e6d4f32fe71e9a408c327448)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31390: Remove noisy warns in C4::Templates

This patch removes two noisy warnings from C4::Templates

0) Apply patch and koha-plack --restart kohadev
1) Go to http://localhost:8081/
2) Note no warnings in /var/log/koha/kohadev/plack-intranet-error.log
3) Go to http://localhost:8080/
3) Note no warnings in /var/log/koha/kohadev/plack-opac-error.log

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 368b1462e58f02e9636bb9ce5a5ba70a54b5d353)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 08dc25d75769070a5d67a8871b27c5d29256052b)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31673: (QA follow-up) Check if FK exists before attempt to remove

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 1c83bd06387a42554f89a21820297970d1377fb4)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 4c8ffab5636a46ace0091913c70bf01df6b1dd12)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31673: (bug 31086 follow-up) Drop and constraint to allow updating column

It seems that we cannot edit the column while it has a constraint.
If we remove it, update the oclumn, then re-add it, the update succeeds

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 0bd87ce0642e873c98b5ca8fe1a0f92aeef7511a)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 43cd28d4e8191e525a580a3c857f84420402f814)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31425: Minor correction to patron categories admin title

This patch corrects the patron categories administration template so
that an extra angle bracket isn't included in the page title on the main
view.

To test, apply the patch and go to Administration -> Patron categories.
The page title should read:

Patron categories › Administration › Koha

...with no extra "›" at the beginning.

Check that other views (new, edit, delete) are unaffected.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 9c11c588c11bf50888796a6d1fe175a2ddc853bd)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 830e85468853c78906651f689cfbd619350955ee)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31352: Change 'Borrower name' to 'Patron name'

Test plan:
1. Go to Tools > Patron card creator > New > Card batch
2. Enter a borrowernumber and click 'Add patron(s)'
3. Observe a table with the leftmost column name 'Borrower name' loads
4. Apply patch
5. Refresh page
6. Observe the leftmost column name is 'Patron name'

Sponsored-by: Catalyst IT, New Zealand
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit d128466dac1711e28f30b556f000f3ae0173bb4b)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 6b2566ece3b57869cbef1a7b429c1d6820dd536d)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31076: Handle year limits with equal sign or colon

To test:
1 - Enable ES and index
2 - Set expandedSearchOption to 'show'
3 - Advanced search on staff and opac using Year field on staff and Publication date range on OPAC for:
    2005
    2005-
    -2005
    <2005
    >2005
    2005-2010
4 - Confirm ranges don't work on staff side, but do on opac
5 - Apply patch
6 - Repeat searches - they work on both staff and opac
7 - prove t/db_dependent/Koha/SearchEngine/Elasticsearch/QueryBuilder.t

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit a11633379f8d9f6a933da7fa887b2e7966e3765a)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit a72ecc3b27379a48bb7f5e45d11e923909edea20)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 22605: (RM follow-up) Fix test suite

See https://jenkins.koha-community.org/view/master/job/Koha_Master_D10/560/

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
(cherry picked from commit 2ed5de93d9658c151505afd470d0f58ec0803679)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 22605: (QA follow-up) Make Braille search option stick

To test:
- Add to records, one with 007 fb and another with 007 tc
- Search for Format: Braille from the advanced search
- Verify both records are found
- Return to advanced search
- Verify that Braille is still selected
- Rerun search - same results show

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
(cherry picked from commit 558ec5b87370727e42b7000ea5058754e50fad11)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 22605: (follow-up) Add selected to audience attributes

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
(cherry picked from commit f6adf4318dc8a53a827c4679ab21c1801088fd83)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 22605: Preselect subtype limits and don't allow multiple

Dropdowns moved to an include, checked and preseleced

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
(cherry picked from commit 926bbd058899e52b57056f17db48b2f1109455a0)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 22605: (follow-up) Fix operators

- Don't add an operator on first loop
- Only add plus option on last loop
- Fix indentation of first search box
- Remove spaces from operators in query_cgi and add to query and query_desc

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
(cherry picked from commit 5bd71b5b7dfd9f1eccfd2c12a78321406018ffeb)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 22605: Add ability to edit a search on staff interface

This patch adds the ability to parse the current search and populate the advanced search page on the staff side accordingly.
When searching you will now have an 'Edit this search' link underneath your current search terms

A few notes:
1 - Previous selected items appear with a checkbox, unchecking 'disables' them and removes from the search
2 - Facets and unrecognized limits will appear at the end of the form under 'Other limits' - there could be future enhancement to parse there better, but I believe this is sufficient for now
3 - Previously selected branch or group selections will populate the dropdowns, only one is allowed

To test:
1 - Perform a variety of search on the staff interface
2 - Confirm the 'edit search' button appear below your search
3 - Confirm that options are retained and correctly parsed
4 - Test with library groups - you should be able to combine groups after initial search, or remove individual libraries when editing
5 - Test that selecting a goup disables the individual libraries options

QA note: Missing filters on deletion of keys form hash are false positives

https://bugs.koha-community.org/show_bug.cgi?id=26605

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
(cherry picked from commit 90d658ae56fcd2ad0da8ed7ab2f37e48b038ef9a)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 28327: (follow-up) Comma as default CSV delimiter

Looks like most of existing code wants comma as default value.

Also impact installer/data/mysql/mandatory/sysprefs.sql.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 7b645b0208182520d91ff370b6a4e5b79549d2e0)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 4cf509b2480935286ce93fc36b7235c72ab203d2)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 28327: Unify CSV delimiter special behavior for tabulation

System preference 'CSVdelimiter' has a special case for tabulation.
Preference value contains string 'tabulation' but string '\t' must be used in CSV file.

This is OK in many places, for exemple Bug 17590.

This patch adds C4::Context->csv_delimiter to add a uniq metod dealing
with this behavior.
Also create Koha::Template::Plugin::Koha->CSVDelimiter for calls from
Toolkit Templates.

Test plan :
1) Set system preference 'CSVdelimiter' = 'tabs'.
2) Create CSV export in impacted pages
3) Check columns are separated by tabulation character and not string 'tabulation'
4) Check with another delimiter

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 381b79593e2a86c4d856242a1f6e955d95ae1f24)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 9d8ddcc1b7db39e3ae683cc70121d6a724a73c89)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 28327: Add unit tests

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit a2e9b440742bb3e207d1906d3c478b936eb277f2)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 41b645bad4b3ee3633cfa63db00f42b5db3d5983)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31282: Fix broken characters in patron_emailer.pl verbose mode

To test:
1. Have a report containing:
   SELECT tomascohen@theke.io;
2. Have a members notice containing ¡ and other non-ASCII characters.
3. Run (changing the report number and notice code accordingly):
   $ kshell
  k$ perl misc/cronjobs/patron_emailer.pl --report 4 \
           --notice BIRTHDAY --from tomascohen@theke.io --module members
=> FAIL: non-ASCII characters are broken
4. Apply this patch
5. Repeat 3
=> SUCCESS: Things print correctly!
6. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 9afbe872b7e63ce1d88cf6896df3b1046529212b)
(cherry picked from commit 418da38327e9456592b4a7fdde8375918b3ca9fb)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 30779: fix t/db_dependent/ImportBatch.t tests count

Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 29325: Fix commit_file.pl

This patch fixes the broken commit_file.pl script.

It doesn't deal with commiting the import from the UI.

To test:
1. Pick a file for staging:
   $ kshell
  k$ misc/stage_file.pl --file TestDataImportKoha.mrc
=> SUCCESS: All good
2. Commit!
  k$ misc/commit_file.pl --batch-number 1
=> FAIL: You see
DBIx::Class::Storage::DBI::_exec_txn_begin(): DBI Exception: DBD::mysql::db begin_work failed: Already in a transaction at /kohadevbox/koha/C4/Biblio.pm line 303
3. Apply this patch
4. Repeat 2
=> SUCCESS: Commit succeeds
5. Sign off :-D

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Bug 29325: (QA follow-up) Remove unexisting parameters of BatchRevertRecords

There is no interval and callback as in BatchCommitRecords.

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Bug 29325: Call progress callback one last time to confirm comppletion

Previously after finishing the loop we were still in a transaction that never completed - we should report progress when done
one final time to commit the last records

To test:
1 - Stage a file with > 100 records
2 - Commit file
3 - Confirm batch is imported and no records left as staged

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Bug 29325: Fix import from staff client

same test as before, but via the staff client

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Bug 29325: Handle the transaction in BatchCommitRecords

Requiring the callback to commit was breaking reversion, and likely elsewhere

Let's simplify and say that the routine iteself will handle the txn and commit

TO test:
1 - Stage a file
2 - Import a file
3 - Revert a file
4 - Test staff client and command line

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Bug 29325: (QA follow-up) Tidy up

tools/manage-marc-import.pl: sub commit_batch does no longer need $schema
tools/manage-marc-import.pl: sub revert_batch: calling BatchRevertRecords which has no interval and callback
misc/commit_file.pl: sub print_progress_and_commit does no longer commit, renamed
misc/commit_file.pl: sub print_progress does no longer have a schema parameter
misc/commit_file.pl: sub revert_batch reported deleted items as added

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Revert Bug 29012:
Some rules are not saved when left blank while editing a 'rule' line in smart-rules.pl

Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31473: Fix fragile test about bad OpacHiddenItems conf

There's been a behavior change in recent MariaDB that made some cases
deal with truncated data in DOUBLE context instead of DECIMAL.

Probably this:
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-27380

This made the affected test fail consistently on those versions.

This trick for checking OpacHiddenItems is correct (as introduced on bug
28125 doesn't really make use of the warning, just prints it in
about.pl) so one option was to just check for a warning.

I decided to keep the test, but add the optional DECIMAL|DOUBLE check on
the qr. This way other eventual changes will make it fail and serve as a
warning in case something more relevant changes.

To test:
1. Apply this patch
2. Run:
   $ kshell
  k$ t/db_dependent/Koha/Items.t
=> SUCCESS: Tests pass!
3. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 3a5fb22ad88790673d6d8fbc483f6e219f25f33d)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit aebaf2495b4abc2a4d7c32db531f78feaa8df98e)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 30779: Remove _update_import_record_marc and update tests

This patch removes the now unused: _update_import_record_marc

Additionally, as items are already present in import biblio we no longer need to embed
them, so that parameter is removed and the option removed from the sub and pod and everywhere
it was used

In all cases, we were embedding, so we don't need a way to get without

Tests updated

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit b02b66acf5ddac1af00215fbcc1d842777b67c3b)
(cherry picked from commit 54ddc31c71292a1c46dc3994ccd89b22896e7d04)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 30779: Do not update import marc after importing items

We are stripping the marc item tags at import - we save them when not importing items, but
strip when importing items

I think we can save on writes by leaving them in the record. This also allows comparison to what was staged
versus items created

To test:
 1 - Stage a marc record with items, but do not look for items
 2 - Confirm the item tags remain in staged marc
 3 - Import the record
 4 - Confirm items are nto in imported marc record
 5 - Stage the record again, but look for items
 6 - Confirm the item tags are stipped from imported record
 7 - Import and confirm imported record has no item tags
 8 - Apply patch and repeat 1-5
 9 - Confirm item tags remain in record
10 - Import and confirm item tags not in imported marc

Signed-off-by: Andrew <andrewfh@dubcolib.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit b1ec071866de33ef0eb37cc74b62c8ed77d6afad)
(cherry picked from commit c2169d1dfea0dd94e4fe2d5f392ed4d98b70d4e6)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31217: OPAC coce.js is not hiding the coce-coverimg div for single-pixel images

This is because opac-detail.tt has coce-coverimg as an ID, but OPAC
coce.js targets coce-coverimg as a class.

opac-detail.tt should be changed to have coce-coverimg as a class.

Test plan:
1. Set your system preferences:
- CoceHost = https://coce.mykoha.co.nz
- CoceProviders = Select all
- OpacCoce = Enable

2. Add the ISBN of '9780262336499' to a biblio

3. Open a "Incognito" window in your browser (recommended to use Google
Chrome to test this)

4. View the biblio detail page in the OPAC. Observe a 'Coce image from
Amazon' box loads with a spinner

5. Apply patch and restart services

6. Clear your browser cache

7. Reload the same biblio detail page in the OPAC. Observe no image is
displayed.

8. View another OPAC biblio detail page that does have a successfully
loading coce image and confirm the source of that image is listed: For
example 'Coce image from Amazon.com' is displayed

Sponsored-by: Toi Ohomai Institute of Technology, New Zealand
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit cbe50d798a42b65d4439d30df6bbf2ab6623bbcd)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit b80ec4fde307a369fe1fdb77de1dc204d9f48838)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31222: Reduce query size for batchMod

The object search is currently:
Koha::Items->search({ barcode => \@contentlist }

Which generate code like:
barcode = 1 OR barcode = 2 OR barcode = 3 ....

This can get quite large

We can reduce the query size by using -in:
Koha::Items->search({ barcode => { -in => \@contentlist } }

Which generates code like:
barcode in ( 1, 2, 3 )

To test:
1 - Apply patch
2 - Perform batch modifications
3 - Confirm nothing has changed

Signed-off-by: Rachael Laritz <rachael.laritz@inlibro.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 7343e404b7b486c52a150613f308e452f4f5da05)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit dd5fcd9de3b6c9201c20e3b279f25649d1e7fac0)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 29782: Fix handling records without title or content

Test plan:
Save a new record with only a title in the non-default tab (english).
Check if you see it on the list. Edit.
Save a new record without title but only content in non-default lang.
Check if you see it.
Do the same 2 edits on the default tab only. Verify too.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 89e65657b106570b717dede6476a6d6f65511b67)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit b9e26b55a14ba42a4d62364cc7ec1aac1fe4d461)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31228: Intranet coce.js is not hiding the coce-coverimg div for single-pixel image

This is because the intranet detail.tt has coce-coverimg as an ID.
Intranet results.tt has coce-coverimg as a class. Intranet coce.js
targets coce-coverimg as a class.

Intranet detail.tt should be changed to have coce-coverimg as a class.
This is inline with the change made to the OPAC in bug 31217

Test plan:
1. Set your system preferences:
- CoceHost = https://coce.mykoha.co.nz
- CoceProviders = Select all
- IntranetCoce = Enable

2. Add the ISBN of '9780262336499' to a biblio

3. Open a "Incognito" window in your browser (recommended to use Google
Chrome to test this)

4. View the biblio detail page in the intranet. Observe a 'Coce image
from Amazon' box loads with a spinner

5. Apply patch and restart services

6. Clear your browser cache

7. Reload the same biblio detail page in the intranet. Observe no image
is displayed.

8. Do an intranet search to get the biblio returned in search results
page. Confirm no cover image box is displayed

9. View the intranet search result and detail pages for a different
biblio that you know Coce should be displaying the cover image of and
confirm that it is correctly displayed with the source of that image
listed: For example 'Coce image from Amazon.com' is displayed

Sponsored-by: Catalyst IT, New Zealand
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit daef044b6f876ea2555c4520e6c91319d2760d25)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 1b1bfdc2a8365f2dae6a66cb7f660b4422f2aeae)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 19540: Tell the patron why a hold cannot be placed on items

This patch adds an explanation when a hold cannot be placed, in some situation:
tooManyReserves
tooManyHoldsForThisRecord
tooManyReservesToday
itemAlreadyOnHold

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 01985605434b7b78cb8f234f8ea66e0a16b3f8c5)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit e1caed6c63d9a831e332ccee32123a591ae2b7e1)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 26632: Remove two more occurrences

Signed-off-by: Andrew Fuerste-Henry <andrewfh@dubcolib.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 089f5a4cf5c0ec2ae68aef9d5f3aaef47e19ad5c)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 396fc44e2c4b156bb7ab01fc74580c7ea20f393c)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 26632: (Bug 22532 follow-up) Remove random variables

AddBiblioToBatch and AddAuthToBatch were btoh passing a random number for the
parameter $update_counts

This simply removes that value as should have been done on 22532

To test:
1 - Stage a marc file
2 - Confirm it works befor and after patch

Signed-off-by: Andrew Fuerste-Henry <andrewfh@dubcolib.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 66c5ab528cf9ed8afa23dd6f14ddf890ef1e9039)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 3a8709f0bd06d2ced0724d31a63cb9eef979cb3a)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31294: Show "Required" label on mandatory fields

This patch adds "Required" label on mandatory fields for Article Requests

0. Apply the patch
1. koha-plack --restart kohadev
2. Change syspref 'ArticleRequests' to 'Enable'
3. Change syspref 'ArticleRequestsMandatoryFields' to '[Select all]'
4. Go to http://localhost:8080/cgi-bin/koha/opac-search.pl?idx=&q=test&weight_search=1
5. Click "Request article" for "Gairm"
6. Note that "Title", "Author", "Volume", "Issue", "Date", "Pages", and
   "Chapters" all say required
7. Change syspref 'ArticleRequestsMandatoryFields' to 'Title' and
   'Author'
8. Go to http://localhost:8080/cgi-bin/koha/opac-request-article.pl?biblionumber=29
9. Note that only "Title" and "Author" say required

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 5c1db538e6c25da27484e39ae619e09779aac1c2)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 369087c7de556078a31a4371d24b4f8a77510006)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>

Bug 31302: Fix spelling in sample_notices.yml

Test plan:
1. Apply patch
2. Confirm 'URL(s)', rather than urls(s), is in the line:
You can download the scanned materials via the following URL(s)

Sponsored-by: Catalyst IT, New Zealand
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit b3d8941c34887b34c69edac8f33b7de179d96267)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 528bb4af5390cbaa145bf7ef8c2a27e0e59e7587)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>