Commit graph

2630 commits

Author SHA1 Message Date
d02e91f269 Bug 29859: Use iterator instead of as_list
On bug 29844 we decided to remove wantarray from Koha::Objects->search.
Reviewing the difference occurrences I found some unnecessary uses of ->as_list,
where iterators should be used instead.

This patch only removes the obvious places, not the tricky ones.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-02-09 15:36:23 -10:00
e53667105d Bug 29844: Fix ->search occurrences
and some more...

There are lot of inconsistencies in our ->search calls. We could
simplify some of them, but not in this patch. Here we want to prevent
regressions as much as possible and so don't add unecessary changes.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-02-09 15:36:23 -10:00
c200bf6bc9 Bug 28962: Add del-unv-selfreg to crontab
We also add a test by inserting a simulated borrower modification
record in the same table which should not be deleted.

NOTE: This patch fixes the del-unv-selfreg parameter in the
cleanup db script. It did not even do what it promised :)

Test plan:
Verify if the crontab change is correct.
Run t/db_dependent/Members.t

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-01-21 21:34:51 -10:00
Hayley Pelham
50d2f00d45 Bug 29054: Add verbose check for warn in advance_notices.pl
To test:
1) Set EnhancedMessagingPreferences to Don't allow
2) In the koha-shell, run misc/cronjobs/advance_notices.pl -c
3) Note that you see the warning "The "EnhancedMessagingPreferences"
syspref is off... etc."
4) Apply the patch and restart services
5) In the koha-shell, run misc/cronjobs/advance_notices.pl -c and note
the warning no longer shows
6) Still in the shell, run misc/cronjobs/advance_notices.pl -c -v and
note the warning does show

Sponsored-by: Catalyst IT
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-01-21 21:34:51 -10:00
9b9417ca13 Bug 29704: (RM follow-up) Fix typo sned send
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-01-18 21:15:04 -10:00
d1a701573e Bug 29704: Add a triggered option to holds_reminder.pl
This patch adds an option to only trigger notices matching the number of
days waiting specified

You will need to define HOLD_REMINDER notices for the specific branch of the
patron and ensure the patron has hold reminder notices in their messaging preferences

TO test:
1 - Place a hold for a patron and check in to confirm
2 - Set the waiting date back a few days:
    update reserves set waitingdate = DATE_SUB(CURDATE(), INTERVAL 5 DAY);
3 - Run the cron and see that patron would be notified if running for 4 days weaiting
    perl misc/cronjobs/holds/holds_reminder.pl -v --days 4
4 - Apply patch
5 - perl misc/cronjobs/holds/holds_reminder.pl -v --days 4 --triggered
6 - Note patron would not be notified
7 - perl misc/cronjobs/holds/holds_reminder.pl -v --days 5 --triggered
8 - Note patron is notified when days waiting matches exactly

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-01-18 21:15:04 -10:00
8a8e105695 Bug 20076: (RM follow-up) Fix system preference boolean logic
A Yes/No system preference must use 1 for Yes and 0 for No.
So "Send" for 1/Yes and "Don't send" for 0/No.
We add too much problems with double-negation boolean system preferences (such as dontmerge).

Previous patch changed default value to 1 in atomicupdate, do the same
in installer/data/mysql/mandatory/sysprefs.sql

Also to be consistant, sets options = NULL instead of '' in atomicupdate

Also removed useless added empty line in /misc/cronjobs/overdue_notices.pl

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-01-13 16:37:33 -10:00
ThibaudGLT
43cb3b299c Bug 20076: Add syspref to remove overdues notification by email to staff if user has no email address
I took the same test plan as victor but I added the system preference to manage the case more easily, especially for users who do not have access to the koha server.

Test plan
1. Check the size of the message queue
     With the following SQL query (using an SQL report if you want)
     SELECT COUNT(*) FROM message_queue;
2. Run misc/cronjobs/overdue_notices.pl
3. Check the size of the message queue
     To ensure that no other overdues will create noise in this test plan.
     Or you can take them into account.
4. Choose a patron with no email address
5. Create an overdue (checkout an item and unfold "Checkout settings"
   and set a date in the past which is compatible with what you find in
   staff:/cgi-bin/koha/tools/overduerules.pl
6. Run misc/cronjobs/overdue_notices.pl
7. Check that you have two new messages in the queue
8. Inspect these two messages
   SELECT * FROM message_queue ORDER BY time_queued DESC LIMIT 2 \G
   1. One has the type "print" and the borrowernumber matching the patron.
   2. The other has
        subject: Overdue Notices
        borrowernumber: NULL
        message_transport_type: email
        and contains "These messages were not sent directly to the patrons."
        This is the one we don't want anymore.
        Because it's now obsolete due to the first message.
9. Apply this patch
10. Run updatedabatase.pl
11. Change syspref 'EmailOverduesNoEmail' to "Don't send"
12. Delete data from message_queue (if you have access) for a cleaner view
13. Run again misc/cronjobs/overdue_notices.pl
14. Check that only the print message is now generated and not the
      "Overdue Notices" one.

https://bugs.koha-community.org/show_bug.cgi?id=20076

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-01-13 16:37:32 -10:00
21bc236e57 Bug 29788: Make Koha::Item->safe_to_delete use Koha::Result::Boolean
This patch reuse the (awesome) Koha::Result::Boolean module to retrieve
the return of Koha::Item->safe_to_delete.

Test plan:
Try to delete an item that has previously been checked out and confirm
that you are still blocked.
Try using the cronjobs, the item and biblio detail pages, as well as the
batch delete item tool.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-01-11 12:38:35 -10:00
e4f4a7ea98 Bug 29794: (follow-up) Add license
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-01-09 21:04:17 -10:00
db5c5c0bc7 Bug 29794: Add missing include in delete_items.pl
This patch adds a missing include so the script is no longer broken.

To test:
1. Choose an item that is checked out and copy its barcode
2. Run:
   $ kshell
  k$ perl misc/cronjobs/delete_items.pl --verbose \
          --where "barcode='39999000010831'"
=> FAIL: It explodes with:
Can't locate object method "find" via package "Koha::Items"...
3. Apply this patch
4. Repeat 2
=> SUCCESS: You get:
Where statement:  where barcode='39999000010831'
Item '549' not deleted: book_on_loan
5. Sign off :-D

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-01-09 21:04:17 -10:00
d7390be41b Bug 28617: (QA follow-up) Restore FindBin in two places
Resolving:
    Name "FindBin::Bin" used only once: possible typo

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2021-12-07 12:16:28 -10:00
238fabc4ab Bug 28617: Remove kohalib.pl and rely on PERL5LIB
The purpose of this script was to load the relevant Koha lib for the
different scripts (installation, cronjob, CLI, etc.)
However it is not used consistently and we prefer to rely on PERL5LIB.

From bug 28617 comment 6 from Galen:
"""
Time marches on, and one of the motivations for having kohalib.pl - making
it possible to install Koha without setting a single environment variable -
has been obviated by the vast improvements in the ease of installing Koha.

Consequently, I think kohalib.pl can go away.
"""

Test plan:
confirm that the changes make sense and that kohalib.pl can be removed
safely.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2021-12-07 12:16:28 -10:00
1e52c2a2fc 21.11.00 - Fix translation issues
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-11-26 12:26:48 +01:00
Koha translators
bb1bcc411e Translation updates for Koha 21.11.00
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-11-26 12:26:48 +01:00
6f8088f4e5 Add release notes for Koha 21.11.00
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-11-26 12:26:22 +01:00
Joonas Kylmälä
92dce6465e Bug 29381: Prevent sending of automatic renewal messages on every cron run
Currently the auto-renewal digest messages are sent on every cron run
even if there was nothing to renew or no renewal errors.

This regression was introduced in the commit "Bug 18532: Add
individual issues to digest notice and hide auto_renewals messaging
preference when not needed".

To test:
 1) set syspref AutoRenewalNotices to be according to patron
    preferences
 2) Enable renewal digest messages on a patron's messaging preferences
 3) Checkout a book for patron, during the checkout use the Checkout
 settings menu to check the box "Automatic renewal"
 4) Run
    $ perl misc/cronjobs/automatic_renewals.pl --send-notices --confirm --digest-per-branch
    $ perl misc/cronjobs/automatic_renewals.pl --send-notices --confirm --digest-per-branch
 5) Notice you have now two renewal messages for the patron
 6) Apply patch
 7) repeat step 4) and notice you don't get anymore these unnecessary
    renewal messages

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-11-24 09:43:45 +01:00
Magnus Enger
88e9cca359 Bug 28994: Fix logical errors with amount vs amoutoutstanding
The writeoff debts script contained an error whereby we could writeoff
more than the current outstanding debt of a charge.  This patch corrects
that mistake by passing amountoutstanding in place of amount in the
writeoff and offset creation lines.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-11-15 12:38:39 +01:00
404e6d901f Bug 28474: Pass process_message_queue.pl params to before_send_messages plugin hooks
It would be useful to send most of the process_message_queue.pl script parameters to any plugin before_send_messages hooks. For example, the Twilio Voice plugin uses before_send_messages to send phone messages, but if the script is called with "-t email", it doesn't know and will make phone calls! If that info were passed in, the plugin could be made aware of it and only make calls if no "-t" or a "-t phone" were set in the parameters.

Test Plan:
1) Apply this patch
2) Install Kitchen Sink plugin v2.2.0 or later
   https://github.com/bywatersolutions/dev-koha-plugin-kitchen-sink/releases/download/v2.2.0/dev-koha-plugin-kitchen-sink-2.2.0.kpz
3) Run misc/cronjobs/process_message_queue.pl with any or all non-email
   related paramters
4) Note the plugin output shows the parameters were passed to the plugin

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-11-03 15:40:52 +01:00
Andreas Jonsson
8605082a78 Bug 26402: Add --framework parameter to commit_file.pl
This patch adds the option to pass a frameworkcode to the commit_file.pl
script.

To test:
1. Stage a MARC file using stage_file.pl.  For instance using the devbox:

   cd ~/kohaclone
   sudo koha-shell -c 'perl misc/stage_file.pl --file t/db_dependent/www/data/marc21record.mrc' kohadev

2. Note the assigned batch number and commit the batch using --framework parameter to commit_file.pl.  For example using the devbox:

   sudo koha-shell -c 'perl misc/commit_file.pl --batch-number 1 --framework ACQ' kohadev

3. Verify that the framework code was correctly assigned:

   sudo koha-mysql kohadev -e 'SELECT frameworkcode FROM biblio ORDER BY timestamp DESC LIMIT 1;'
=> SUCCESS: It picked it

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-11-03 15:40:52 +01:00
Joonas Kylmälä
9ead068920 Bug 29380: Correct table name in joins to prevent errors
The commit "Bug 29290: Rename relationships borrower =>
patron" (d46492ac23) renamed the relation for the borrowers table from
'borrower' to 'patron' but the joins were not updated accordingly so a
few scripts got broken.

To test:
 1) Notice that
    $ perl misc/cronjobs/automatic_renewals.pl -c -s -v
    returns 255 error code on exit
 2) Apply patch
 3) Notice the automatic_renewals.pl works now and exit code is 0
 4) Make sure /cgi-bin/koha/tools/batch_extend_due_dates.pl works.
 5) Try to grep for 'borrower' in Koha source code and see if there
    are any other join done using the Koha::Checkout or
    Koha::Old::Checkout objects.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Bug 29380: (follow-up) Fix renewal feature in staff interface

The commit "Bug 29290: Rename relationships borrower =>
patron" (d46492ac23) renamed the relation for the borrowers table from
'borrower' to 'patron' but the usage in the renew.pl script was not
updated.

To test:
 1) Try to renew a book in intranet through the renewal tab
 2) Notice it gives error
 3) Apply patch
 4) Notice the error is now gone

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-11-03 15:40:52 +01:00
54fa48b139 Bug 29395: use EXPORT_OK from Koha::Patron::Debarments
Missed from bug 17600.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-11-03 15:40:52 +01:00
f0858a202b Bug 29223: Use patron's branchcode for digest if not sending digest per branch
To test:
 1 - Set system preference  AutoRenewalNotices to 'according to patron messaging preferences'
 2 - Set circ rule for auto renewal - 100 renewals allowed - no renewal before 100
    This shoud make a checkout eligible for many auto renewals for testing
 3 - Checkout an item to a patron, ensure item is from a different branch than patron
 4 - Ensure the patron has an email, and set an email for patron's branch and items branch
 5 - Set the patron messaging preferences for auto-renewal to 'email' and 'wants digest'
 6 - perl misc/cronjobs/automatic_renewals.pl --send-notices --confirm
 7 - Job dies:
    Can't call method "from_email_address" on an undefined value at /usr/share/koha/bin/cronjobs/automatic_renewals.pl line 286.
 8 - Apply patch
 9 - Repeat
10 - NO errors
11 - Check patron notices tab, should be a notice enqueued, check DB to see from address is patron's branch
12 - perl misc/cronjobs/automatic_renewals.pl --send-notices --confirm --digest-per-branch
13 - No errors
14 - Check patron notices tab, should be a notice enqueued, check DB to see from address is item's branch

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-11-02 14:57:21 +01:00
27f3d0af63 Bug 14957: (QA follow-up) Clarify 'context' param
This patch renames the (passed through) 'context' param for
'overlay_context'. I propose doing so, because in Koha-land 'context'
has a special meaning, related to C4::Context and it reads ambigous.

The patch itself is pretty trivial.

Tests should pass:
1. Run:
   $ kshell
  k$ prove t/db_dependent/Biblio/MarcOverlayRules.t
=> SUCCESS: Tests pass
2. Apply this patch
3. Repeat 1
=> SUCCESS: Tests still pass!
4. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Bug 14957: (follow-up) Clarify 'context' param

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-26 16:46:02 +02:00
David Gustafsson
139e6c30d6 Bug 14957: Merge rules system for merging of MARC records
Add a rule based system for merging MARC records to for example
prevent field data from being overwritten.

To test:
    1. Apply this patch.
    2. Log in to staff client.
    3. Enable new syspref MARCMergeRules.
    4. Click the new link "MARC merge rules" in the "Catalog"
       section of the Koha administration page.
    5. Create a new rule:
       Module: source, Filter: *, Tag: 245, Preset: Protect.
    6. Clicking "Edit" should allow you to edit corresponding rule.
    7. Clicking "Delete" should remove corresponding rule after confirmation.
    8. Selecting one or more rules followed by clicking "Delete
       selected" should remove all selected rules after confirmation.
    9. Try creating a rule with tag set to "**", the other options does
       not matter. Verify that saving this rule produces an error
       message complaining about invalid tag regular expression.
    10. Try creating a rule with tag set to "008" (or other control
        field) and set Appended: Append and Removed: Skip, the other
        options does not matter. Verify that saving this rule produces
        an error message complaining about invalid combination of actions
        for control field.
    11. With the 245 rule in step 5 in place, edit a bibliographic record,
        change 245a for example (which should be Title for MARC21) and save.
    12. Verify that the changes has not been saved.
    13. Create a new rule:
        Module: source, Filter: intranet, Tag: 245, Preset: Overwrite.
    14. Repeat step 12, and verify that the changes has now been saved.
    15. Run tests in t/db_dependent/Biblio/MarcMergeRules.t and very
        that all tests pass.

Sponsored-by: Halland County Library
Sponsored-by: Catalyst IT
Sponsored-by: Gothenburg University Library
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Christian Stelzenmüller <christian.stelzenmueller@bsz-bw.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-26 16:46:02 +02:00
Petro Vashchuk
1ecf5ee106 Bug 29207: Getopt::Long should not ignore cases
To avoid Getopt::Long treating upper case and lower case options,
this patch restores removed ":config no_ignore_case".

Steps to reproduce:
1. Start koha-z3950-responder deamon for any instance.
2. Check z3950.log log file, there should be
"[fatal] Failed to listen on ...-koha-z3950" in it.
3. Alternatively you can "curl localhost:2100/biblios?version=1.1 -v"
to ensure that it doesn't work "Failed to connect to localhost
port 2100: Connection refused".
4. Apply the patch.
5. Stop and start daemon again.
6. Check the logs again, there should be no new error there.
7. Same way do the curl to ensure that this time it listens on that port.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-25 14:08:06 +02:00
Caroline Cyr La Rose
a0725b12a8 Bug 29216: Correct --where documentation in update_patrons_category.pl
This patch removes two misleading examples of the --where parameter
documentation in update_patrons_category.pl

To test:

1. Apply patch
2. In your favourite text editor, open misc/cronjobs/update_patrons_category.pl
3. Look for the documentation for the --where parameter (around line 133 or so)
4. Make sure the examples and their explanations make sense
5. Sign off :)

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-21 10:21:50 +02:00
a37bee6647 Bug 28153: Add 'Hold reminder' messaging preference
This adds a new preference for patrons to choose how they wish to receive
'Hold reminder' notices.

The notice is always digested per branch

To test:
 1 - Apply patches
 2 - Update database
 3 - Ensure EnhancedMessagingPreferences and EnhancedMessagingPreferencesOPAC are enabled
 4 - View a patron and note new messaging preference
 5 - Confirm same on the opac for a patron account
 6 - All transports should be disabled by default
 7 - Place a hold for the patron and check it in to confirm
 8 - Run hold reminder script
    perl misc/cronjobs/holds_reminder.pl -v -c
 9 - No message should be queued for patron
10 - Enable the message in a transport for which 'HOLD_REMINDER' notice has content for the patron
11 - Run the script
12 - Patron should have a message queued
13 - Ensure a different transport has content for the notice
14 - Run the script forcing a transport
   perl misc/cronjobs/holds_reminder.pl -v -c -mtt=print
15 - The patron should have a message queued in the forced transport

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-18 11:28:41 +02:00
424aca3d56 Bug 28445: Use the task queue for the batch delete and update items tool
Here we go!

Disclaimer: this patch is huge and does many things, but splitting it in
several chunks would be time consuming and painful to rebase. However it
adds many tests and isolate/refactor code to make it way more reusable.

This patchset will make the "batch item modification" and "batch item
deletion" features use the task queue (reminder: Since bug 28158, and so
21.05.00, we do no longer use the old "background job" functionality and
the user does not get any info about the progress of the job).

More than that, more of the code to build an item form and a list of
items is now isolated in module (.pm) and include files (.inc)

We are reusing the changes made by bug 27526 that simplifies the way we
edit/create items (no more unecessary serialization Koha > MARC > MARCXML
> XML > HTML)

New module:
* Koha::BackgroundJob::BatchDeleteItem
    Subclass for process item deletion in batch
* Koha::BackgroundJob::BatchUpdateItem
    Subclass for process item modification in batch
* Koha::Item::Attributes
    We needed an object to represent item's attributes that are not
    mapped with a koha field (aka "more subfields xml")
    This module will help us to create the marcxml from a hashref and the
    reverse.
* Koha::UI::Form::Builder::Item
    The code that was used to build the add/edit item form is
    centralised in this module. In conjunction with the
    subfields_for_item BLOCK (from html_helpers.inc) it will be really
    easy to reuse this code in other places where the item form is used
    (acquisition and serials modules)
* Koha::UI::Table::Builder::Items
    Same as previously for the table. We are now using this table from 3
    different places (batch item mod, batch item del, backgroung job
    detail view) and the code is only in one place.
    To use with items_table_batchmod BLOCK (still from html_helpers.inc)

This patch is fixing some bugs about repeatable subfields and regex. A UI
change will reflect the limitation: if you want to apply a regex on a
subfield you cannot add several subfields for the same subfield code.

Test plan:

Prepare the ground:
- Make sure you are always using a bibliographic/item record using the framework
you are modifying!
- Add some subfields for items that are not mapped with a koha field
(note that you can use 'é' for more fun, don't try more funny
characters)
- Make some subfields (mapped and not mapped with a kohafield)
repeatable
- Add default values to some of your subfields

There are 4 main screens to test:
1. Add/edit item form
The behaviour should be the same before and after this patch.
See test plan from bug 27526.
Those 2 prefs must be tested:
    * SubfieldsToAllowForRestrictedEditing
    * SubfieldsToUseWhenPrefill

2. Batch modification
a. Fill some values, play with repeatable and regex.
Note that the behaviour in master was buggy, only the first value was modified by the regex:
    * With subfield = "a | b"
    1 value added with "new"
    => "new | b"

    * With subfield = "a | b"
    2 new fields "new1","new2"
    => "new2 | b"

Important note: For repeatable subfields, a regex will apply on the subfields in
the "concatenated form". To apply the regex on all the different subfields of a given
subfield code you must use the "g" modifier.
This could be improved later, but keep in mind that it's not a regression or behaviour
change.

b. Play with the "Populate fields with default values from default framework" checkbox

c. Use this tool to modify items and play with the different sysprefs that
interfer with it:
    * NewItemsDefaultLocation
    * SubfieldsToAllowForRestrictedBatchmod
    * MaxItemsToDisplayForBatchMod
    * MaxItemsToProcessForBatchMod

3. Batch deletion
a. Batch delete some items
b. Check items out and try to delete them
c. Use the "Delete records if no items remain" checkbox to delete
bibliographic records without remaining items.
d. Play with the following sysprefs and confirm that it works as
expected:
    * MaxItemsToDisplayForBatchDel
e. Stress the tool: Go to the confirmation screen with items that can be
deleted, don't request the job to be processed right away, but check the
item out before.

4. Background job detail view
You must have seen it already if you are curious and tested the above.
When a new modification or deletion batch is requested, the confirmation
screen will tell you that the job has enqueued. A link to the progress
of the job can be followed.
On this screen you will be able to see the result of the job once it's
fully processed.

QA notes:
* There are some FIXME's that are not blocker in my opinion. Feel free to
discuss them if you have suggestions.
* Do we still need MaxItemsToProcessForBatchMod?
* Prior to this patchset we had a "Return to the cataloging module" link
if we went from the cataloguing module and that the biblio was deleted.
We cannot longer know if the biblio will be deleted but we could display
a "Go to the cataloging module" link on the "job has been enqueued"
screen regardless from where we were coming from.

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-18 11:28:40 +02:00
e8ba35b2fa Bug 29197: Add missing import in commit_file.pl
This patch adds a new method import to the commit_file.pl script.

To test:
1. Have MARC data staged
2. Run:
   $ kshell
  k$ perl misc/commit_file.pl --list-batches
=> FAIL: You get
Undefined subroutine &main::GetAllImportBatches called at misc/commit_file.pl line 68.
3. Apply this patch
4. Repeat 2
=> SUCCESS: You get the list of batches
5. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-12 10:14:38 +02:00
e975ed4e3e Bug 18984: Remove *-NORMARC.po files
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-07 15:37:34 +02:00
08c03af2ae Bug 18984: Remove NORMARC support
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-07 15:36:40 +02:00
7babcc0718 Bug 18631: Use - instead of _ for options
To match existing options

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-07 11:18:29 +02:00
9515875626 Bug 18631: Add option to specify logs to preserve or delete in cleanup_database.pl
This adds two new parameters:
--log_module
--preserve_log

These can be repeated to include or exclude specific modules from the cleanup

To test:
0 - Apply patch
1 - Enable cataloging log and borrowers log
2 - Make some changes to borrowers and records
3 - run cleanup databse with --logs 0 --preserve_log=MEMBERS --preserve_log=CATALOGUING
4 - nothing is removed
5 - run cleanup databse with --logs 0 --log_module=MEMBERS
6 - the borrower logs are removed, the record changes remain
7 - run cleanup database with --logs 0
8 - record changes are removed (all other logs too)

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Rebecca Coert <rcoert@arlingtonva.us>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-07 11:07:05 +02:00
2c0f722a3a Bug 25429: (QA follow-up) Add extra variable for days
I think it is a bit more clear to use another variable for the
number of days from the preferences than overwriting the flag
variable.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Tested by adding an authval for LOST.
Filling prefs ClaimReturnedLostValue, CleanUpDatabaseReturnClaims.
Claiming a return, resolving it. Setting date back via SQL.
Running the script with -v --return-claims, toggling --confirm.
The claim is counted and deleted with confirm.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-06 14:51:09 +02:00
5e4c7ce212 Bug 25429: Require command line switch to enable, control DAYS via syspref
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-06 14:51:09 +02:00
f28e817ea5 Bug 25429: Use filter_by_last_update
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-06 14:51:09 +02:00
2a8dc21a6a Bug 25429: Script should be able to run with only return claims
Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Rebecca Coert <rcoert@arlingtonva.us>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-06 14:51:09 +02:00
e18cfe0731 Bug 25429: Cleanup Database - remove resolved claims returned from db after X days
Add option to cleanup_database script to removed 'resolved' return claims from the database after a specified number of days.

Test Plan:
1) Apply this patch
2) Set the new syspref CleanUpDatabaseReturnClaims to a number of days
3) Run cleanup_database.pl
4) Note resolved claims older than the specified number of days are removed from the database

Bug 25429: Implement system preference, remove command line switch

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Rebecca Coert <rcoert@arlingtonva.us>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-06 14:51:09 +02:00
c7eb047126 Bug 28352: Only check authorised values mapped to DB fields
The errors reported seem to be caused by authorised values mapped to MARC fields
but not mapped to a koha field.

We should additionally make sure to check the Default framework

Also, adding comment to indicate we only check records with items, because we do

TO test:
1 - In a framework that is not the default map a MARC field to an authorised value, but not a koha field
2 - In SQL, force the kohafield to NULL for the mapping you just make
    UPDATE marc_subfield_structure SET kohafield = NULL WHERE frameworkcode='BKS' and authorised_value='HINGS_AS'
3 - perl misc/maintenance/search_for_data_inconsistencies.pl
4 - get the following errors:
Use of uninitialized value $tmp_kohafield in pattern match (m//) at /kohadevbox/koha/misc/maintenance/search_for_data_inconsistencies.pl line 151.
Use of uninitialized value $tmp_kohafield in substitution (s///) at /kohadevbox/koha/misc/maintenance/search_for_data_inconsistencies.pl line 154.
Can't call method "get_column" on an undefined value at /kohadevbox/koha/misc/maintenance/search_for_data_inconsistencies.pl line 157.
5 - Apply patch
6 - Repeat
7 - No more errors

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-01 16:28:55 +02:00
Agustin Moyano
954d2606a8 Bug 23678: Allow cancel holds in bulk
This patch allows staff patrons to cancel multiple holds in bulk.

To test:
1. Apply this patch
2. restart_all
3. In cataloge go to a book and place many holds
CHECK => Holds table shows a column of checkboxes
4. Play with checkboxes (have some fun ;-P)
CHECK => When you manually check all checkboxes, the checkbox in the
header also gets checked.
      => When you uncheck one of the checkboxes, the one in the header also gets unchecked.
      => If no checkbox is checked and you check the one in the header,
all checkboxes get checked.
      => If there are some checkboxes that are checked and others are
not, when you click on the checkbox in the header all checkboxes get
unchecked.
      => If all checkboxes are checked, when you uncheck the one in the
header, all checkboxes get unchecked.
      => Every time you play with checkboxes, the number in the button
"Cancel selected" changes.
5. Check some of the checkboxes and click on cancel selected.
SUCCESS => A background job gets fired to cancel all selected holds.
        => A message should appear with a link to the job.
6. Wait a few seconds and click on the link
SUCCESS => A message appears with the report of the execution of the
background job.
7. Grab a patron and search to hold
8. Select multiple biblios and click on "place hold for <patron>"
CHECK => After holds are confirmed, multiple holds table are shown.. one for
   each record. Checkboxes work exactly the same as before, but scoped
for each individual table. Checkboxes from one table will not affect
checkboxes from other tables.
9. Repeat steps 4 to 6.
10. Check In some of the items so the get in Waiting state.
11. Update expirationdate os some of those holds and set it to
    ReservesMaxPickUpDelay + 1 days earlier
NOTE => ReservesMaxPickUpDelay = 7 days by default, so sql syntax to update would be
     => update reserves set expirationdate = date_sub(expirationdate, interval 8 day) where reserve_id in (...)
12. Repeat steps 4 to 6 but in waitingreserves.pl, in both tabs.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Bug 23678: (QA follow-up) Add missing template filter

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Bug 23678: (QA follow-up) Add missing filters

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Bug 23678: (QA follow-up) Use correct indentation

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

JD amended patch: also Koha/BackgroundJob/BatchCancelHold.pm

JD Amended patch: Full rebase and adjustements made on top of bug 26080.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-01 16:02:58 +02:00
d210b8b1ca Bug 26080: Use the task queue for batch delete authorities
Same as the first patch, for authorities

Test plan:
Delete authority records using the batch record deletion tool
Confirm that the job is now delegated to the task queue and that
everything else is working as before

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-01 15:35:36 +02:00
6e4182c4fc Bug 26080: Use the task queue for batch delete biblios
This patch takes advantage of the task queue to delegate the batch
delete biblios tool.

Test plan:
Delete bibliographic records using the batch record deletion tool
Confirm that the job is now delegated to the task queue and that
everything else is working as before

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-01 15:35:36 +02:00
a0ebdab046 Bug 29078: Resolve division by zero in touch_all scripts
Illegal division by zero at misc/maintenance/touch_all_biblios.pl line 102.

Trivial fix.
Funny that the percentage is not even shown without verbose flag ;)

Test plan:
Run misc/maintenance/touch_all_biblios.pl -where 'biblionumber<0'
You should now see:
Good: 0, Bad: 0 (of 0) in 0 seconds

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-28 15:12:45 +02:00
c08fd3e987 Bug 29076: Fetch the zebraqueue entries when not confirm
This patch remove a check for confirm when fetching the entries
for possible deletion from the zebraqueue

Without this we die when we try to access the entries for counting

To test:
1 - perl  misc/cronjobs/cleanup_database.pl --sessions --zebraqueue 10 --list-invites --z3950 --logs 180 --mail 375 -v
2 - Dies
3 - Apply patch
4 - Repeat 1
5 - Success!

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-28 15:12:44 +02:00
d587341d34 Bug 17600: Remove wrong C4::Context imports
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-20 14:44:27 +02:00
765225d99f Bug 28456: Add WHERE option to membership_expiry cronjob
Some libraries have a large amount of accounts and want to be able to limit how many email reminders they send.
Adding a where statement will allow for flexibility

To test:
1 - Set some borrowers to expire soon
2 - Set MembershipExpiryDaysNotice to 10
3 - perl misc/cronjobs/membership_expiry.pl -c -v -n --before 100 --after 100
4 - Note the patrons that yo uadjusted show up
5 - Limit by various patron fields, e.g.
    perl misc/cronjobs/membership_expiry.pl -c -v -n --before 100 --after 100 --where="lastseen IS NULL"
    perl misc/cronjobs/membership_expiry.pl -c -v -n --before 100 --after 100 --where="lastseen IS NOT NULL"
    perl misc/cronjobs/membership_expiry.pl -c -v -n --before 100 --after 100 --where="surname LIKE '%a%'"
    perl misc/cronjobs/membership_expiry.pl -c -v -n --before 100 --after 100 --where="surname NOT LIKE '%a%'"
6 - Confirm expected results
7 - perl misc/cronjobs/membership_expiry.pl
8 - Confirm the help message makes sense

Signed-off-by: Azucena Aguayo <azucena.aguayo@gmail.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] Replace two inclue by include.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-30 17:02:07 +02:00
47c589bea1 Bug 22435: Update for bug 27049
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-04 14:06:43 +02:00
7436094e19 Bug 17600: (follow-up) Fully qualify routine in cancel_expired_holds.pl
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-07-30 15:53:18 +02:00
e9ae9a95a4 Bug 17600: (follow-up) Fix missing C4::AuthoritiesMARC import
Oops

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-07-30 08:06:07 +02:00