Koha/misc
Jonathan Druart 424aca3d56 Bug 28445: Use the task queue for the batch delete and update items tool
Here we go!

Disclaimer: this patch is huge and does many things, but splitting it in
several chunks would be time consuming and painful to rebase. However it
adds many tests and isolate/refactor code to make it way more reusable.

This patchset will make the "batch item modification" and "batch item
deletion" features use the task queue (reminder: Since bug 28158, and so
21.05.00, we do no longer use the old "background job" functionality and
the user does not get any info about the progress of the job).

More than that, more of the code to build an item form and a list of
items is now isolated in module (.pm) and include files (.inc)

We are reusing the changes made by bug 27526 that simplifies the way we
edit/create items (no more unecessary serialization Koha > MARC > MARCXML
> XML > HTML)

New module:
* Koha::BackgroundJob::BatchDeleteItem
    Subclass for process item deletion in batch
* Koha::BackgroundJob::BatchUpdateItem
    Subclass for process item modification in batch
* Koha::Item::Attributes
    We needed an object to represent item's attributes that are not
    mapped with a koha field (aka "more subfields xml")
    This module will help us to create the marcxml from a hashref and the
    reverse.
* Koha::UI::Form::Builder::Item
    The code that was used to build the add/edit item form is
    centralised in this module. In conjunction with the
    subfields_for_item BLOCK (from html_helpers.inc) it will be really
    easy to reuse this code in other places where the item form is used
    (acquisition and serials modules)
* Koha::UI::Table::Builder::Items
    Same as previously for the table. We are now using this table from 3
    different places (batch item mod, batch item del, backgroung job
    detail view) and the code is only in one place.
    To use with items_table_batchmod BLOCK (still from html_helpers.inc)

This patch is fixing some bugs about repeatable subfields and regex. A UI
change will reflect the limitation: if you want to apply a regex on a
subfield you cannot add several subfields for the same subfield code.

Test plan:

Prepare the ground:
- Make sure you are always using a bibliographic/item record using the framework
you are modifying!
- Add some subfields for items that are not mapped with a koha field
(note that you can use 'é' for more fun, don't try more funny
characters)
- Make some subfields (mapped and not mapped with a kohafield)
repeatable
- Add default values to some of your subfields

There are 4 main screens to test:
1. Add/edit item form
The behaviour should be the same before and after this patch.
See test plan from bug 27526.
Those 2 prefs must be tested:
    * SubfieldsToAllowForRestrictedEditing
    * SubfieldsToUseWhenPrefill

2. Batch modification
a. Fill some values, play with repeatable and regex.
Note that the behaviour in master was buggy, only the first value was modified by the regex:
    * With subfield = "a | b"
    1 value added with "new"
    => "new | b"

    * With subfield = "a | b"
    2 new fields "new1","new2"
    => "new2 | b"

Important note: For repeatable subfields, a regex will apply on the subfields in
the "concatenated form". To apply the regex on all the different subfields of a given
subfield code you must use the "g" modifier.
This could be improved later, but keep in mind that it's not a regression or behaviour
change.

b. Play with the "Populate fields with default values from default framework" checkbox

c. Use this tool to modify items and play with the different sysprefs that
interfer with it:
    * NewItemsDefaultLocation
    * SubfieldsToAllowForRestrictedBatchmod
    * MaxItemsToDisplayForBatchMod
    * MaxItemsToProcessForBatchMod

3. Batch deletion
a. Batch delete some items
b. Check items out and try to delete them
c. Use the "Delete records if no items remain" checkbox to delete
bibliographic records without remaining items.
d. Play with the following sysprefs and confirm that it works as
expected:
    * MaxItemsToDisplayForBatchDel
e. Stress the tool: Go to the confirmation screen with items that can be
deleted, don't request the job to be processed right away, but check the
item out before.

4. Background job detail view
You must have seen it already if you are curious and tested the above.
When a new modification or deletion batch is requested, the confirmation
screen will tell you that the job has enqueued. A link to the progress
of the job can be followed.
On this screen you will be able to see the result of the job once it's
fully processed.

QA notes:
* There are some FIXME's that are not blocker in my opinion. Feel free to
discuss them if you have suggestions.
* Do we still need MaxItemsToProcessForBatchMod?
* Prior to this patchset we had a "Return to the cataloging module" link
if we went from the cataloguing module and that the biblio was deleted.
We cannot longer know if the biblio will be deleted but we could display
a "Go to the cataloging module" link on the "job has been enqueued"
screen regardless from where we were coming from.

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-18 11:28:40 +02:00
..
admin Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
bin Bug 17600: Fix POSIX imports 2021-07-29 14:37:26 +02:00
cronjobs Bug 18631: Use - instead of _ for options 2021-10-07 11:18:29 +02:00
devel Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
interface_customization Bug 23148: Replace Bridge icons with transparent PNG files 2020-07-20 16:16:37 +02:00
load_testing Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
maintenance Bug 28352: Only check authorised values mapped to DB fields 2021-10-01 16:28:55 +02:00
migration_tools Bug 17600: (follow-up) Fix missing C4::AuthoritiesMARC import 2021-07-30 08:06:07 +02:00
release_notes Add release notes for Koha 21.05.00 2021-05-28 14:45:29 +02:00
search_tools Bug 18984: Remove NORMARC support 2021-10-07 15:36:40 +02:00
translator Bug 18984: Remove *-NORMARC.po files 2021-10-07 15:37:34 +02:00
add_date_fields_to_marc_records.pl Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
background_jobs_worker.pl Bug 28445: Use the task queue for the batch delete and update items tool 2021-10-18 11:28:40 +02:00
batchCompareMARCvsFrameworks.pl Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
batchdeletebiblios.pl Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
batchDeleteUnusedSubfields.pl Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
batchImportMARCWithBiblionumbers.pl Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
batchRebuildBiblioTables.pl Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
batchRebuildItemsTables.pl Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
batchRepairMissingBiblionumbers.pl Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
check_sysprefs.pl Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
commit_file.pl Bug 29197: Add missing import in commit_file.pl 2021-10-12 10:14:38 +02:00
export_borrowers.pl Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
export_records.pl Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
exportauth.pl Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
import_patrons.pl Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
koha-install-log Bug 28519: Put CGI::Session::Serialize::yamlxs in lib directory 2021-06-17 10:07:36 +02:00
kohalib.pl Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
link_bibs_to_authorities.pl Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
load_yaml.pl Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
mod_zebraqueue.pl Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
perlmodule_ls.pl
perlmodule_rm.pl Bug 21395: Make perlcritic happy 2020-06-29 12:37:02 +02:00
recreateIssueStatistics.pl Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
sax_parser_print.pl Bug 17626 (QA followup) 2016-11-22 11:29:07 +00:00
sax_parser_test.pl Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
sip_cli_emulator.pl Bug 28320: Add DB connection check to the SIP SC status message 2021-05-17 12:34:33 +02:00
stage_file.pl Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
z3950_responder.pl Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00