Bug 29288: Add current_checkouts and old_checkouts methods to Koha::Biblio

This patch adds helper methods for accessing current and past checkouts
for a given Koha::Biblio object.

To test:
1. Apply the unit tests
2. Run:
   $ kshell
  k$ prove t/db_dependent/Koha/Biblio.t
=> FAIL: Methods are not implemented
3. Apply this patch
4. Repeat 2
=> SUCCESS: Tests pass!
5. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 29288: Unit tests

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 29108: Add q parameters to items routes

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 29107: Rename item_type => item_type_id

This simple change renames the attribute in the item object API
representation.

To test:
1. Run:
   $ kshell
  k$ prove t/db_dependent/api/v1/items.t
=> SUCCESS: Tests pass
2. Run:
   $ git grep 'api/v1/items'
=> SUCCESS: The items routes are not (yet) used in Koha
3. Run:
   $ cd koha-tmpl
   $ git grep --name-only '"embed"'
=> SUCCESS: Only two templates are using embed.
4. Check:
   - register.tt uses it to embed the manager, good
   - parcel.tt uses it to embed items. Alert.
=> SUCCESS: parcel.tt only uses a count on items. No item_type involved.
5. Sign off

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 27358: Add GET /public/biblios/:biblio_id/items

This patch introduces a route to fetch items belonging to a biblio. It
is expected to return the 'public' representation of the Koha::Item
objects.

It is also enforcing the visibility rules, by using
Koha::Items->filter_by_visible_in_opac.

To test:
1. Apply this patches
2. Run:
   $ kshell
  k$ prove t/db_dependent/api/v1/biblios.t
=> SUCCESS: Test pass and they cover all the cases!
3. Try your favourite REST tool against the new route.
4. Sign off :-D

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 27358: Unit tests for public items retrieval

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 28948: Fix random failure

This patch makes the query for randomly generated libraries
deterministic, thus getting rid of the random tests failures.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 28948: Don't require catalogue permission for public route, don't allow smtp server embed

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 28948: Remove query params, 'q' param covers everything needed

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 28948: Remove FIXME

This patch reproduces what we did for `to_api_mapping`: make it always
present on Koha::Object classes. This has the side-effect of... making
things more secure!

Before this patch, if undefined, all attributes were returned.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 28948: (QA follow-up) Convert to allow-list

This patch converts the code to use an allow-list as aposed to a
deny-list.  This is more 'fail safe' than requireing maintanence of a
deny-list.

We also switch to using db fields names for the list as aposed to api
mapped names. This way, the list can be re-used for non-api related
sanitising if required.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 28948: Add GET /public/libraries routes

This patch introduces a route to fetch a list of libraries or a single
library as expected on the /public namespace. It is expected to return
the 'public' representation of the Koha::Library objects.

To test:
1. Apply this patches
2. Run:
   $ kshell
  k$ prove t/db_dependent/api/v1/libraries.t
=> SUCCESS: Test pass and they cover all the cases!
3. Try your favourite REST tool against the new route.
4. Sign off :-D

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 28948: Teach objects.search about public requests

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 28948: Make is_public stashed on public routes

This patch makes the API authentication code stash the 'is_public' value
when public routes are hit.

This will be particularly useful to have $c->objects->search generically
pass this info down to the ->to_api method.

To test:
1. Apply this patch
2. Run:
   $ kshell
  k$ prove t/db_dependent/api/v1/auth_authenticate_api_request.t
=> SUCCESS: Tests pass! When a public route is reached, the controller
has the 'is_public' value stashed
3. Sign off :-D

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 28948: Add a generic way to handle API privileged access attributes deny-list

This patch introduces a way for Koha::Object(s)->to_api to filter out
attributes that require privileged access. It is done in a way that the
'public' parameter is recursively passed to nested objects in recursive
to_api() calls.

This way, Koha::Object-based classes can determine how they will render
depending on this parameter. For example, for implementing a
route for fetching an library looks like:

GET /libraries

The controller will look like:

my $library = Koha::Libraries->find( $c->validation->param('library_id') );
return $c->render(
    status  => 200,
    openapi => $library->to_api
);

Implementing an unprivileged (public) route would look like:

GET /public/libraries/:library_id

The controller will look like:

my $library = Koha::Libraries->find( $c->validation->param('library_id') );
return $c->render(
    status  => 200,
    openapi => $library->to_api({ public => 1  })
);

To test:
1. Apply this patch
2. Run:
   $ kshell
  k$ prove t/db_dependent/Koha/Object*.t
=> SUCCESS: Tests pass (i.e. current behaviour is kept, new behaviour
        passes the tests)
3. Sign off :-D

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 17314: Implement /suggestions routes

This patch introduces routes to handle purchase suggestions, from the
staff POV.

Tests are added as well.

To test:
1. Apply this patches
2. Run:
   $ kshell
  k$ prove t/db_dependent/api/v1/suggestions.t
=> SUCCESS: Tests pass! And they are meaningful!
3. Play with your favourite REST tool (Postman?)
4. Sign off :-D

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 17314: OpenAPI spec

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 17314: Unit tests

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 17314: Make TestBuilder set good defaults for Suggestions

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 28859: Correctly handle 'Checked out by' visibility

We must have the column in the table and let DT deal with the
visibility.

This patch hides the "Checked out by" column by default if
RecordStaffUserOnCheckout is off, but the DT column settings is aware of
its existence and the end user can still display the column.

IMO that's the most optimal situation considering both maintenance and ergonomic.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 28859: Fix missing column setting

This commit adds the missing checked_out_by column to
columns_settings.yaml

Test plan:

- Follow the test plans in the bug description and note that the
symptoms are no longer seen

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 29229: (follow-up) Update method for linking flatpickr fields

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 29229: Use Flatpickr in suggestion search sidebar filter

This patch corrects the date field configurations for the suggestions
sidebar filter. These fields are now linked Flatpickr widgets instead of
jQueryUI datepickers.

To test, apply the patch and go to Suggestions.

- In the sidebar, click "Suggestion information" to expand the form.
- Test these linked date fields:
  - "Suggested date from" and "to"
  - "Managed date from" and "to"
  - "Accepted by from" and "to"
- Each should trigger Flatpickr calendars which are linked, i.e. you
  can't select a "to" date which is before a "from" date.
- Test that the dates submit correctly to filter as expected.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 29261: (bug 15812 follow-up 2) Fix translation issue in include file

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 29301: Display error with serials search flatpickr when searching Mana

This patch changes the way certain serials search fields are hidden when
a standalone search of Mana is being performed (from the "Search on
Mana" link). Instead of targeting specific labels and inputs to hide we
can hide list items by class.

Unrelated: This patch updates the page heading so that it is different
depending on whether you're searching Koha subscriptions or Mana
subscriptions.

To test, apply the patch and enable Mana.

- Go to Serials -> Advanced search (in the search header).
- The search form should include all fields, including call number,
  vendor, library, location, and "Expires before."
- Click "Search on Mana" in the sidebar.
- This page should have the heading "Mana subscriptions search," and
  there should only be three fields visible: ISSN, Title, and Publisher.
- When you perform a Mana search which returns results the page should
  have the heading "Mana subscriptions (X found).

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 29299: (follow-up) Add markup comments

This patch adds comments to the template to highlight the markup
structure.

This patch should have no effect on the page's appearance or
functionality.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 29299: Reindent serials search template

This patch updates the serials search template to give it consistent
indentation.

The patch also moves one section of markup: The <tfoot> section is moved
to after <tbody> in order to conform to current standards.

To test, apply the patch and go to Serials -> Search subscriptions (in
the header).

- Serials search should work correctly and the page should look correct.
- With Mana enabled, click on the "Search on Mana" link in the sidebar.
- Searches of Mana should work correctly too.
- If you view the diff while ignoring whitespace the only changes should
  be the move of <tfoot> and places  where line breaks were introduced.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 28613: Add missing parameters to objects.search-based routes

The objects.search helper provides convenient ways to paginate and a 'q' filter parameter to build complex queries on this Koha::Objects-based routes.

Some routes were created prior to this feature getting introduced. We need to adjust the parameters for all existing objects.search-based routes.

This patch adds that.

The only GET (as in list) routes that would still be missing the
parameters are those about ILL, that are not objects.search based, and
probably need to be rewritten.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 29300: Add 22.05 release team to teams.yaml

Add the 22.05 release team.

Test plan
1/ Check against
   https://wiki.koha-community.org/wiki/Release_Teams

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 29218: Rename .hidden class for DT visibility

.hidden get the CSS that interferes with DT behaviour

.hidden {
    display: none !important;
}

Test this patch with bug 28859.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 28349: (bug 26234 follow-up) Correct wrong array merge

Bug 26234 allowed to remove the different classes we use in aoColumnDefs
for sorting columns of table using DT.

However there is a terrible mistake, the merge of existing aoColumnDefs
array with the default one is totally wrong:
  $.extend(true, new_parameters, default_column_defs);
When we actually wanted to do:
  $.extend(true, new_parameters["aoColumnDefs"], default_column_defs);

But it's still wrong, extend is doing a deep copy and the array will be
replaced by the other one, whereas we want to append.

We want to merge default_column_defs with the existing aocolumnDefs,
this patch is doing it explicitely.

This bug only exists when there is an existing aocolumnDefs.

See commit d3f3a55e0b1544ebcd3166d141e126a4324baab1
It happens on:
 * cataloguing/z3950_search.tt
 * reports/guided_reports_start.tt
 * serials/subscription-detail.tt
 * opac-course-reserves.tt
 * opac-detail.tt

However this bug has been hidden as we have in master related bug from
bug 27945, which removes title-string.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 29272: (QA follow-up) Category setup overrides syspref

This patch removes an early short-circuit we had, which wasn't correct
regarding the OpacPasswordChange syspref. If a patron category is
allowed to change password, it overrides the syspref.

To test:
1. Tests still pass.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 29272: Make public password changing honour category constraints

This patch makes the public API routes validate
$user->category->effective_change_password before allowing the change.

To test:
1. Apply the regression tests patch
2. Run:
   $ kshell
  k$ prove t/db_dependent/api/v1/patrons_password.t
=> FAIL: Tests fail, it allows the first change instead of returning
         403.
3. Apply this patch
4. Repeat 2
=> SUCCESS: Tests pass!
5. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 29272: Regression tests

This patch introduces regression tests for the described bug.

To test:
1. Apply this patch
2. Run:
   $ kshell
  k$ prove t/db_dependent/api/v1/patrons_password.t
=> FAIL: Tests fail, excepted failures are considered success

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 26374: update from bug 19974 is not idempotent

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 29332: Set default library limits in search_for_display

Koha::AdditionalContents->search_for_display needs to filter to just
items with a 'null' branchcode if no library_id is passed.

Test plan
1/ Add some additional content blocks that should display only for
specific branches.
2/ Navigate to the OPAC and note that blocks display for all libraries
prior to login
3/ Login to the OPAC and note that now you only see you users library
block
4/ Apply the patch
5/ Repeat steps 2 and 3 and note that now you see only the 'All
libraries' blocks display prior to login and that 'All libraries' +
'Your users library' blocks appear after login.

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 29332: Unit tests

This patch adds unit tests to ensure we do not leak branch specific
addtional contents blocks outside of logged in sessions (i.e. where a
users branch is not yet set).

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 28445: Add diag for failing test

Temporary patch to help trying to fix the following random error:
     #   Failed test 'Item types should be sorted by description and an empty entries should be shown'
     #   at t/db_dependent/Koha/UI/Form/Builder/Item.t line 121.
     #     Structures begin differing at:
     #          $got->[12] = 'Be9T9FH5aO'
     #     $expected->[12] = 'REF'
     # Looks like you failed 1 test of 2.

 #   Failed test 'itemtypes'
 #   at t/db_dependent/Koha/UI/Form/Builder/Item.t line 130.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 28445: Fix LostItem call and display errors

Never a try without a catch, ever!

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 28445: Remove modified_itemitemnumbers

Wrong substitution

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 28445: Correctly count modified items

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 28445: Add tests for subfields_to_prefill

I first wrote test for another thing but the code was actually correct.
So submitting the tests.

Note that the marc for more_subfields does not need to be passed, we can
build it when needed.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 27526: Fix for prefill preventing duplication 2

Prefill feature should not be in the priority
when user choose "Duplicate" for the item

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 27526: Fix SubfieldsToUseWhenPrefill

my $a = "z";
my @x = split ( ' ', $a ) || ("");

@x will be [1]

Which is not at all what we are expecting here!
Be more verbose and don't introduce bug.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: DBIC changes - Remove MarcOverlayRulesModule.pm

Where is that coming from??

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: Fix perlcritic error on admin/marc-overlay-rules.pl

Subroutine "new" called using indirect syntax at line 30, column 13.  See page 349 of PBP.  (Severity: 4)

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 27526: Correct NULL vs empty string when editing

When an item is edit we must keep the NULL values in DB if the input
have been left empty.
It also fixes the "barcode cannot be unique" error when an item does not
have a barcode.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 29318: Tidy the code

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 29318: Remove permission check from overdrive search page

This removes the 'edit_borrowers' permission from OverDrive search

To test:
1 - Enable OverDrive via koha system preferences
2 - Sign in to opac with a user with no permissions
3 - Perform a search that will return OD results, 'love' works
4 - Click thje "Found XX results in the library's OverDrive collection"
5 - Note you are logged out of Koha
6 - Apply patch
7 - Repeat
8 - Success

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 29286: Typo: Librarien will need the manage_auth_values subpermission.

This patch fixes a typo in the Cataloging system preferences text.

"Librarien" -> "Librarian"

To test, apply the patch and go to Administration -> System preferences
-> Cataloging.

 - Check the CreateAVFromCataloguing system preference.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 29158: (bug 22435 follow-up) Remove ref to account_offset_types.sql in intaller

Caused by
  commit 3d6a6e79f79bfbc17807072faf0ff1d4e6aafb82
  Bug 22435: (follow-up) Drop account_offset_types table
that removes the file but the reference to it from the installer
process.

Test plan:
Go through the install process and notice that the following warning
does not appear in the log:
[2021/10/25 09:47:58] [WARN] Something went wrong loading file /kohadevbox/koha/installer/data/mysql/mandatory/account_offset_types.sql (Couldn't read '/kohadevbox/koha/installer/data/mysql/mandatory/account_offset_types.sql' : No such file or directory at /usr/share/perl5/DBIx/RunSQL.pm line 180.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: DBRev 21.06.00.038

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: DBIC schema changes

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: Add tests for subfield order

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: Fix field order issue

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: Fix record preview

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: Exclude leader from saved/updated record comparison in tests

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: Fix regression, rules must be refetched on add/edit/deletion

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: Improve readbility of tests

Not saying that they were not readable before, but I had to do some
changes to compare the before/after records and it was easier that way.
Feel free to obsolete if you disagree.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: Clean controller

Too many imports and unecessary complicated init of $rules

There are certainly other things to improve, like the "op" handling,
standardize names of "op", etc.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: (QA follow-up) Fix for jQuery undefined

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: (QA follow-up) Fix script name

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: (QA follow-up) Fix template indentation

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: (QA follow-up) Update for bug 27846

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: (QA follow-up) Update for bug 17858

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: (QA follow-up) Update for bug 26703

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Bug 14957: (QA follow-up) Update for bug 20054

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Bug 14957: (QA follow-up) Update for bug 20053

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Bug 14957: (QA follow-up) Update for bug 23271

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: (QA follow-up) Add missing filters

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: (QA follow-up) Comment 'bulkmarcimport' temporarily

It appears this patchset doesn't fully support bulkmarcimport (see bug
25539 for details).

At this time, to get this bug moving I think we should comment out the
ability to define rules that affect bulkmarcimport and then we can fix
and re-enable that feature at a later date.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: (QA follow-up) Rename and move admin link

This patch renames 'MARC overlay rules' to 'Record overlay rules' and
moves it up the list to just below 'Record matching rules' to make it
more contextually ordered.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: (QA follow-up) Fix translation issues

This patch fixes all the translation issues I could spot in the
marc-overlay-rules template.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: (QA follow-up) Clarify 'context' param

This patch renames the (passed through) 'context' param for
'overlay_context'. I propose doing so, because in Koha-land 'context'
has a special meaning, related to C4::Context and it reads ambigous.

The patch itself is pretty trivial.

Tests should pass:
1. Run:
   $ kshell
  k$ prove t/db_dependent/Biblio/MarcOverlayRules.t
=> SUCCESS: Tests pass
2. Apply this patch
3. Repeat 1
=> SUCCESS: Tests still pass!
4. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Bug 14957: (follow-up) Clarify 'context' param

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: (follow-up) marc_merge_rules_modules removed

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: DBIC changes

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: Feature hidden in menues if no other 'cagaloguing' permissions

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: (QA follow-up) Rename 'merge' => 'overlay'

This patch acknowledges the fact that 'merging' has a different meaning
in Koha than the behavior this great patchset introduces. The more
idiomatic way of describing the behavior is to talk about 'overlay rules'.

This patch also:
- Fixes kohastructure.sql missing table encoding a collation
- Fixes the atomic update completely missing a table
- Moves the syspref entry to 'Importing', probably a rebase issue

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Bug 14957: (QA follow-up) Fix syntax error in atomicupdate

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Bug 14957: Fix imports in tests

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Bug 14957: Fix imports in marc-overlay-rules.pl

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Bug 14957: Fix syspref's values

It didn't switch off when set to "don't use"

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
JD amended patch: Fix license statement

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: (QA follow-up) Add tooltips

This patch adds an info-tooltip to the field rules in the table header
to expand on their meaning to reduce the need to read the documentation
each time you come to use the configuration tool

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: (QA follow-up) Button styling

This patch updates the action buttons to include btn-default and
switched from btn-sm to btn-xs to match other similar UI's found in
datatable based admin pages.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: fix context for batchmod and batchimport

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: Updated schema files

Signed-off-by: Christian Stelzenmüller <christian.stelzenmueller@bsz-bw.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 14957: Merge rules system for merging of MARC records

Add a rule based system for merging MARC records to for example
prevent field data from being overwritten.

To test:
    1. Apply this patch.
    2. Log in to staff client.
    3. Enable new syspref MARCMergeRules.
    4. Click the new link "MARC merge rules" in the "Catalog"
       section of the Koha administration page.
    5. Create a new rule:
       Module: source, Filter: *, Tag: 245, Preset: Protect.
    6. Clicking "Edit" should allow you to edit corresponding rule.
    7. Clicking "Delete" should remove corresponding rule after confirmation.
    8. Selecting one or more rules followed by clicking "Delete
       selected" should remove all selected rules after confirmation.
    9. Try creating a rule with tag set to "**", the other options does
       not matter. Verify that saving this rule produces an error
       message complaining about invalid tag regular expression.
    10. Try creating a rule with tag set to "008" (or other control
        field) and set Appended: Append and Removed: Skip, the other
        options does not matter. Verify that saving this rule produces
        an error message complaining about invalid combination of actions
        for control field.
    11. With the 245 rule in step 5 in place, edit a bibliographic record,
        change 245a for example (which should be Title for MARC21) and save.
    12. Verify that the changes has not been saved.
    13. Create a new rule:
        Module: source, Filter: intranet, Tag: 245, Preset: Overwrite.
    14. Repeat step 12, and verify that the changes has now been saved.
    15. Run tests in t/db_dependent/Biblio/MarcMergeRules.t and very
        that all tests pass.

Sponsored-by: Halland County Library
Sponsored-by: Catalyst IT
Sponsored-by: Gothenburg University Library
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Christian Stelzenmüller <christian.stelzenmueller@bsz-bw.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 11175: DBRev 21.06.00.037

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 11175: Add tests

Add coverage for the message set by get_marc_components when the search
fails.
Also cleans a bit the weird return comparisons.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 11175: (QA follow-up) Test message contents for analytics error

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 11175: (QA follow-up) Restore bug 29284

This patch restores the functional fixes introduced in bug 29284 and
also prevents the 'Show analytics' link from displaying when no
component parts are found and inline display is enabled.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Amended: Fixed error with $err vs $error(s) :)

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 11175: (QA follow-up) Do not fetch twice

We were fetching components in the first call of XSLTParse4Display
(opac-detail around L220). And again (opac-detail around L660).
Same for catalogue/detail.pl.
Moving the XSLT block in both scripts and removing the code from
XSLT.pm which again makes the tests obsolete.

Not hiding the link when there are components. Might be helpful too.

Also fixing biblionumber in the XSLTParse4Display call for parts.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 11175: (QA follow-up) Remove caching _components

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 11175: Uniformize ComponentParts in template

More robust as it takes empty array into account

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 11175: Add syspref MaxComponentRecords to remove hardcoded limit

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 11175: (QA follow-up) Restore link to more results

This patch restores the link to 'more results' should you exceed the
hard coded limit of 300 component parts.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 11175: (QA follow-up) Be more specific with title fields

We were using the MARC::Record title accessor, but that just stringifies
all the subfields of field 245. The indexer, and the XSLT, specifically
use 245a so we need to do the same.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 11175: (QA follow-up) Display components tab if no holdings

This patch updates the default details tab selection to components for
the case when there are no holdings attached to the record and there is
no HTML5 content to display in preference.

Bug 11175: (QA follow-up) OPAC - Display components tab if no holdings

As for the staff client, this selects the default tab to be 'Components'
for the case where no holdings are found.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 11175: (QA follow-up) Account for difference in simple_search return

Thanks to a comment from Andrew, I discovered that the return from
simple_search_compat actually differs depending on if you are using
ElasticSearch or Zebra. The 'results' arrayref contains MARC::Record
objects for ES and raw marc strings if you are using Zebra.  This is
actually already dealt with in the misnamed new_record_from_zebra
function inside C4::Search.. so this patch simply replaces our
MARC::Record instantiation code with a call to that function.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Andrew Nugged <nugged@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 11175: (follow-up) Use clean_search_term

This patch uses the now public clean_search_term from bug 28316 to
sanitize the title string and prevent crashes in elastic search when
title contain reserved characters.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Andrew Nugged <nugged@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 11175: Display components in own tab

This patch updates the display so that rather than displaying the
components using the main XSLT and then using CSS to move them into a
box on the right side for display, which causes issues with RTL records;
We instead add them in a new tab beneath the rest of the record details
alongside holdings and other details.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Andrew Nugged <nugged@gmail.com>
Bug 11175: (QA follow-up) Fixes for bug 12561

Bug 12561 changed the prototype for XSLTParse4Display so this patcha
accounts for the additional calls to that method introduced in this
patchset.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Andrew Nugged <nugged@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 11175: (QA follow-up) Fix queries

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Andrew Nugged <nugged@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 11175: (QA follow-up) Only get biblio once

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Andrew Nugged <nugged@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 11175: (QA follow-up) Move get_component_part_query

This patch removes Koha::Util::Search in preference to embedding the
search query builder in Koha::Biblio as get_analytics_query.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Andrew Nugged <nugged@gmail.com>
Bug 11175: (QA follow-up) Rename back to get_marc_components

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Andrew Nugged <nugged@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 11175: (QA follow-up) Take account of bug 15851

We can simplify the code introduced by bug 15851 by moving the
'show_analytics_link' variable assignment into C4::XSLT and thus making
the code more DRY.

Taking the code in bug 15851 as inspiration this patch also adds proper
handling for UseControlNumber vs EasyAnalytics style 773 linking and
ensures we only return analytic component parts and no other records
containing 773's.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Andrew Nugged <nugged@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 11175: (QA follow-up) Move preference to 'Display'

The 'ShowComponentRecords' preference was displaying under the 'Export'
section in the cataloging system preferences tab. As a display related
feature, I felt it was more appropriate to put it into the 'Display'
section.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Andrew Nugged <nugged@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 11175: (follow-up) Rename routine get_marc_components

For consistency with other marc data accessor methods rename the new
'components' method to 'get_marc_components'.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Andrew Nugged <nugged@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>