Commit graph

8146 commits

Author SHA1 Message Date
1ad0e172cd Bug 3142: Use filter_by_for_holds where holdability is calculated
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-08 16:57:29 +02:00
67ead5e7ae Bug 29134: (follow-up) Specify which borrowernumber
When performing a search that includes permissions we need to specify we are comparing
the borrowers tabnle borrowernumber

To test:
1 - Browse to Admin->Funds
2 - Edit a fund
3 - Click '+ Add users'
4 - Attempt a search, no results
5 - Check plack-error.log
    C4::Utils::DataTables::Members::search(): DBI Exception: DBD::mysql::st execute failed: Column 'borrowernumber' in IN/ALL/ANY subquery is ambiguous at /kohadevbox/koha/svc/members/search line 80
6 - Apply patch
7 - Repeat 1-3
8 - search for 'a'
9 - Result!

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-08 16:48:22 +02:00
fb068b31be Bug 29062: (QA follow-up) Add comment
Wrapped it in a TODO ;)

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-07 15:49:01 +02:00
0138d09109 Bug 29062: (follow-up) Adjust SendCirculationAlert
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-07 15:49:01 +02:00
814789f8cb Bug 29062: Use primary key issued_id to fetch old_issues for letters
The code currently uses itemnumber to fetch old_issues for notices.
This doesn't seem to be used in any current notices except the CHECKINSLIP:
SELECTY letter.code,content FROM letter WHERE content LIKE 'old\\_%'\G

For issues we use itemnumber, however, issues has a constraint to limit issues for an itemnumber to 1
Old issues has no such constraint, we try to rectify this in the old code by adding 'ORDER BY returndate DESC LIMIT 1"

As the code is not used by default and buggy I think we can make a change to using 'issue_id' as the key and
announcing the change - it prevents leaky data

To test:
1. Check something out to patron A. Check it in.
2. Check something out to patron B. Check it in.
3. Check something out to patron C. Check it in and print the check-in slip. (Leave the checkin paghe open)
4. You will see the checkin repeat itself 3 times, one for each line in old_issues.
5. Apply patch and restart_all
6. Click the 'Print checkin slip' button again
7. You see a single checkin
8. Checkout a different item to patron A. Check it in and print the check-in slip
9. See the correct checkins

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-07 15:49:01 +02:00
08c03af2ae Bug 18984: Remove NORMARC support
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-07 15:36:40 +02:00
5be090b556 Bug 28730: Add option to format AH field (due date) in SIP checkout response
This patch adds a new option to sip accounts:
format_due_date

If set to 1 the AH field in checkouts will follow the dateFormat system preference and format as due dates (ignoring time portion if 23:59:59 or 11:59:59)

To test:
 1 - checkout an item via SIP:
perl misc/sip_cli_emulator.pl -a localhost -p 6001 -su term1 -sp term1 -l CPL -t CR --item 3999900000001 --patron enda -m checkout
 2 - Note the AH field is like "YYYYMMDD    HHMMSS"
 3 - Apply patch
 4 - repeat SIP checkout - nothing has changed
 5 - edit SIPConfig.xml and add format_due_date="1" to 'term1' account:
    <login id="term1"  password="term1" delimiter="|" error-detect="enabled" institution="CPL" encoding="ascii"     checked_in_ok="1" format_due_date="1"/>
 6 - restart all
 7 - repeat the SIP checkout
 8 - AH field now matches dateFormat system preference
 9 - Change your dateFormat preference
10 - repeat the SIP checkout
11 - AH matches new format

Note: If you cannot renew and don't get an AH just check the item back in:
perl misc/sip_cli_emulator.pl -a localhost -p 6001 -su term1 -sp term1 -l CPL -t CR --item 3999900000001 --patron enda -m checkin

Signed-off-by: Winfred Thompkins <Winfred.thompkins@cedarhilltx.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-07 11:18:29 +02:00
9e249435f9 Bug 28931: Use EXPORT_OK from Koha::DateUtils
It has been missed on bug 17600.

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Adjusted commit title.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-07 11:01:05 +02:00
4f533776c6 Bug 26352: Switch from using call() to call_recursive()
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-06 14:56:09 +02:00
aea8ce7fcc Bug 26352: Add plugin hooks to transform patron barcodes
Some of our partners have unusual barcode requirements that have
required us to transform scanned barcodes using javascript. This is not
the most reliable method. It would make more sense to have Koha
transform the barcodes on the backend using a plugin. We should add
hooks to transform and generate new item and patron barcodes.

Test Plan:
1) Apply this patch
2) Download and install the Barcode Transformer plugin
   https://github.com/bywatersolutions/koha-plugin-barcode-transformer/releases/download/v1.0.1/koha-plugin-barcode-transformer-v1.0.1.kpz
3) Go to the plugin configuration page, set the configuration to the example configuration from the same page
4) In the patron search and anywhere else you can scan a patron
   cardnumber, type in some cardnumbers but prefix them with A and postfix
   them with A or B, e.g. A12345A, A12345B
5) Note the letters are removed by Koha!
6) Try a cardnumber like X123456Y
7) Note Koha converts it to Z13456Z

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-06 14:56:09 +02:00
b2b1483f4d Bug 26351: Add plugin hooks to transform item barcodes
Some of our partners have unusual barcode requirements that have
required us to transform scanned barcodes using javascript. This is not
the most reliable method. It would make more sense to have Koha
transform the barcodes on the backend using a plugin. We should add
hooks to transform and generate new item and patron barcodes.

Test Plan:
1) Apply this patch
2) Download and install the Barcode Transformer plugin
   https://github.com/bywatersolutions/koha-plugin-barcode-transformer/releases/
3) Go to the plugin configuration page, set the configuration to the example configuration from the same page
4) In the item barcode field on the checkin and checkout pages,
   and anywhere else you can scan an item barcode, type in some
   valid barcodes, but prefix them with X and postfix them with
   Y, e.g. X123456Y
5) Note the letters are removed by Koha!

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Bug 26351: (QA follow-up) Fix QA script issue

* Fixes issue with barcode generate stub so perlcritic is happy
* Removes extra semicolon from return call in configure method

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Bug 26351: Add unit tests

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Bug 26351: (QA follow-up) Remove unused method barcode_transform

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Bug 26351: (QA follow-up) Rename barcode_transform to item_barcode_transform

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Bug 26351: (QA follow-up) Barcodes inputted into Koha should always pass though barcodedecode

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Bug 26351: (QA follow-up) Catch one last case of itemBarcodeInputFilter

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Bug 26351: (QA follow-up) Fix Checkouts.t

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Bug 26351: Use call_recursive() as a replacement for call()

The method `call()` is not sufficient for barcode transformations. It's
possible that more than one barcode transformation plugin will be
installed. The `call_recursive()` method takes the output of the first
plugin and uses it as the input for the next plugin and so on. This allowes
each plugin to see the current version of the barcode and modify it if
necessary.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Bug 26351: Fix t/db_dependent/Koha/Plugins/Circulation_hooks.t

Bug 26351: Revert improper change to unit test, fix number of tests

Bug 26351: Remove uneeded use Koha::Plugins statements

Left over from previous changes

Bug 26351: Add missing barcodedecode import

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-06 14:56:09 +02:00
478ffeae60 Bug 27032: (follow-up) Pass rather than fetch variables
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Rebased-by: Joonas Kylmälä <joonas.kylmala@helsinki.fi>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-06 11:44:22 +02:00
Joonas Kylmälä
c862fd1382 Bug 27032: Move auto renewal code out of CanBookBeRenewed
This moves the checks related to whether auto renewal can be done on
the checkout to its own function _CanBookBeAutoRenewed(). This makes
it more clear which parts of the code are related to auto renewal
checking and which are not.

To test:
  1) Make sure prove t/db_dependent/Circulation.t still passes

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Rebased-by: Joonas Kylmälä <joonas.kylmala@helsinki.fi>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-06 11:44:22 +02:00
Joonas Kylmälä
b246015cad Bug 27032: Remove unused variables
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Rebased-by: Joonas Kylmälä <joonas.kylmala@helsinki.fi>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-06 11:44:22 +02:00
6560049c65 Bug 27526: Fix cn_source display
For an unknown reason C4::Biblio::GetAuthorisedValueDesc (that we are
calling from Koha::Item->columns_to_str) does not deal with class
sources.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-06 11:44:22 +02:00
96826fd2d0 Bug 27526: Remove ModItemFromMarc from additem
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-06 11:44:22 +02:00
Michael Hafen
c3b28f78bc Bug 28303: Fix plugins system with multiple pluginsdir settings
C4/Templates::badtemplatecheck mucks with the config('pluginsdir') array ref.
This makes sure it operates on a copy of the array.

To test:
   1) $ prove t/db_dependent/Templates.t

Signed-off-by: David Nind <david@davidnind.com>
JK: Fix commit message styling and add test plan

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-05 10:58:13 +02:00
Aleisha Amohia
17a1b0cec5 Bug 28734: Parse display in $biblio->get_marc_notes
1. Go to Koha Administration, MARC bibliographic frameworks
2. Edit a framework, find tag 590, edit the $z subfield to use an
authorised value.
3. Do a catalogue search and edit a record using this framework. Edit
590$z and select an authorised value. Save the record.
4. Add the record to your cart.
5. View your cart. Click More Details.
6. Scroll down to the Notes section. Notice the authorised value code is
displayed instead of the description.
7. Apply the patch, restart services.
8. Refresh your cart. Click More Details again if you need to.
9. Scroll down to the Notes section. The description of the authorised
value should now be displayed.

Sponsored-by: Catalyst IT

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Bug 28734: (follow-up) Use transformMARCXML4XSLT and tests

Confirm the following tests pass:
- t/db_dependent/Koha/Biblio.t
- t/db_dependent/XSLT.t

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Bug 28734: (QA follow-up) Fix test imports and readability

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Bug 28734: (QA follow-up) Update documentation about function usage

We are now using the function from Koha::Biblio.

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Bug 28734: Remove unadvertised changes

1. Using $frameworkcode instead of the default '' is an unadvertised change.
It would make sense to use $frameworkcode, but actually we must remove this parameter and always use the default, as we decided to make the default authoritative.
I would prefer to not introduce this change, just in case..

2. Restore good import (use plurals)

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-05 10:57:10 +02:00
cbf405828a Bug 29134: Use a subquery to increase performance of patron attributes search
This patch generates a subquery and checks if a borrowrnumber is in the results to add patrons
to search results

To test:
 1 - Generate a bunch of patrons:
    SELECT surname, firstname, branchcode, categorycode FROM ( SELECT surname FROM borrowers ORDER BY rand() ) a,( SELECT firstname FROM borrowers ORDER BY rand() ) b,( SELECT branchcode FROM borrowers ORDER BY rand() ) c,( SELECT categorycode FROM borrowers ORDER BY rand() ) d LIMIT 50000
 2 - Add a patron attribute to the system and make it searchable - I used code 'TEST'
 3 - Add a value for this attribute to many patrons:
   INSERT INTO borrower_attributes (borrowernumber,code,attribute) SELECT borrowernumber, 'TEST','alphabet' FROM borrowers LIMIT 10000;
 4 - In staff client got 'Patrons'
 5 - Open the browser console (F12) and view the netwrok tab
 6 - Perform a patron search for 'a'
 7 - Note the time it takes for 'search' to complete in console
 8 - Apply patch, restart_all
 9 - Repeat search
10 - Note it is much faster
11 - prove -v t/db_dependent/Utils/Datatables_Members.t

NOTE: I tested with 500k patrons and 100k attributes - search returned in ~2 seconds with patch
and did not return before I got impatient without patch

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-05 09:17:20 +02:00
58c765c492 Revert "Bug 28510: Remove unnecessary conditional"
This reverts commit d284735d05.

The following test was failing randomly:
 #   Failed test 'take from lowest cost branch (don't use cost matrix) holding branch'
 #   at t/db_dependent/HoldsQueue.t line 1494.
 #          got: 'LHKtxLk'
 #     expected: 'JL9C_OR'
 # Wrong pick-up/hold for first target (pick_branch, hold_branch, reserves, hold_fill_targets, tmp_holdsqueue)

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-01 16:28:55 +02:00
bc67449a46 Bug 28826: Add FacetOrder system preference
This bug adds a system preference to control ordering of facets and
adds the control to both Zebra and Elasticsearch

To test:
1 - Have a koha that can use both Zebra and ES
2 - Set 'displayFacetCount' to true
3 - Search in ES and Zebra
4 - Note facets in Zebra sorted alphabetically, ES by usage
5 - Apply patch, updatedatabase
6 - Search in ES and Zebra, facets are alphabetically sorted in both
7 - Find new syspref FacetOrder and set to 'by usage'
8 - Search in both engines, facets sorted by usage

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-29 11:34:21 +02:00
d284735d05 Bug 28510: Remove unnecessary conditional
It makes sense to use items by branch to get the list of branches, as it
already tells use which branches have available items. We could use
branches to pull from instead, but all we would accomplish is added
extra ununsed loop iterations. We already know that any additional
branches in the branches to pull loop have no items to fill holds.
If they did, they would be in the items_by_branch hash.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-28 15:12:45 +02:00
ba23348db6 Bug 28510: Remove marking of closed branches as 'disable_transfer'
We no longer need to act as if closed branches were marked as
disable_transfer. This allows us to clean up a nice bit of code.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Bug 28510: Remove unused variable

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-28 15:12:45 +02:00
b7ae33b47b Bug 28510: Skip processing holds queue items from closed libraries when HoldsQueueSkipClosed is enabled
Right now we skip closed branch's items as we iterate over all items looking for ones to fill a hold. If HoldsQueueSkipClosed is enabled, no items held be a closed library can be targeted, so it would be more efficient if we never selected the items from those branches to begin with. This is how the holds queue works if we are not using the transport cost matrix, so we should make it work the same way if we *are* using the matrix.

Test Plan:
1) Apply this patch
2) prove prove t/db_dependent/HoldsQueue.t
3) All tests should continue to pass

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-28 15:12:45 +02:00
33165bd22e Bug 28803: (follow-up) Error details improvement
This patch updates the exceptions thrown by Koha::Email to include the
parameter that failed email validation and then updates the failure code
to include this parameter and finally display this field in the template.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-28 10:22:36 +02:00
9cfbdee6a8 Bug 28803: Add invalid email handling in _send_message_by_email
This patch adds a try/catch block around the call to Koha::Email->create
to catch and handle invalid emails being passed in the parameters.

The message is marked as 'failed' with an error_code of 'INVALID_EMAIL'.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-28 10:22:36 +02:00
Andrew Isherwood
f9f457ab14 Bug 24190: (follow-up) Rename AcqLog
As requested in comment #49, renamed uses of AcqLog to AcquisitionLog

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

JD amended patch: replace one missing occurrence in Budgets.t

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-21 20:22:57 +02:00
Andrew Isherwood
114f4ea5a3 Bug 24190: (follow-up) Modify order of logging
Basket creation involves two steps in Koha, creating the basket then
modifying its header. We were logging these two steps in the wrong
order. This commit fixes that.

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-21 20:22:57 +02:00
Andrew Isherwood
50109e06f9 Bug 24190: (follow-up) Respond to QA feedback
This commit makes changes in response to Jonathan's feedback in comment

- Moved from using zero padded strings to store log data to a JSON
object
- Stopped storing formatted dates in logged data

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-21 20:22:57 +02:00
Andrew Isherwood
bf90fb810e Bug 24190: Add acquisition logging
Signed-off-by: Maura Stephens <maura.stephens@gmit.ie>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-21 20:22:57 +02:00
87de9fbb8a Bug 28847: Remove dead code and make regex consistent
This updates the regex code to match throughout the routine and removes
code that would no longer be reached.

Note that the code to update library name in the query descruiption was
broken before this patch, so removal does not change behaviour

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-20 14:04:48 +02:00
065ec4e726 Bug 28847: Cleanup of branch limitsand fix OPAC_SEARCH_LIMIT
The opac had 'branch_group_limit' parameters which can be simplified to more
closely match intranet code.

Adjust C4::Auth for chaneg above to ensure dropdowns correctly populate

Expand JS to prevent selection of single and multibranch limits

To test:
1 - Enable OpacAddMastheadLibraryPulldown system preference
2 - Ensure branches and groups show as before patch
3 - Ensure single and multibranch limits from masthead apply as expected
4 - Test advanced search page, ensure you cannot select both single and multibranch limit
5 - Follow test plan on 28845 - ensure multibranch limit still correctly pre-selected

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-20 14:04:48 +02:00
777c519c5a Bug 28847: Move SearchLimitLibrary code to QueryBuilder Modules
This patch removes the code from the search scripts into QueryBuilder
modules.

To test:
1 - Have a library group defined as a search group for both staff and opac
2 - Search on staff client and opac with that group limit and a single branch limit
3 - Note your results/counts
4 - Note the visuals of the search description
5 - Apply patch
6 - Repeat searches
7 - All should work as before

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-20 14:04:48 +02:00
91a557bd08 Bug 28464: Remove useless check that gives the incorrect error message
Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-20 14:04:48 +02:00
Andreas Roussos
5de0705275 Bug 28472: handle items with NULL shelving location
The UpdateItemLocationOnCheckin System Preference can be set to update the
location of ALL items during check in, regardless of their shelving location.
However, this does not currently work 100% as it is excluding items with no
shelving location (i.e. value of NULL in the corresponding database field).

This patch, based on the comment made by Nick Clemens, fixes that.

Test plan (based on the original Bug Description by Andrew Fuerste-Henry):

1) Have a shelving location CART
2) In UpdateItemLocationOnCheckin, enter "_ALL_: CART" (without the quotes)
3) Check in an item that has a shelving location, confirm it changes to CART
4) Check in an item with a NULL shelving location, confirm it doesn't go to CART
5) Apply this patch
6) Repeat step 4): this time the item should move to the CART shelving location

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-20 12:07:02 +02:00
ea88cfb3ab Bug 28676: Cache and retrieve match_count when searching a cached heading
We use match_count to determine if a new authority record should be created, however,
we were not adding this count to the cache, so if a record returned too many matches on first
lookup, we would create a new record on the second lookup

To test:
1 - Set Linker Module to 'Default'
2 - Enable  AutoCreateAuthorities  and  BiblioAddsAuthorities and  CatalogModuleRelink and LinkerRelink
3 - Add two copies of a single authority via Z39
4 - Add two headings for that authority to a bib record (e.g. a 610 and 710)
5 - Save the record and note a new authority is generated
6 - Repeat and see another is generated
7 - Apply patch
8 - Restart all the things
9 - Save the record again, no new authority created

Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-20 12:06:56 +02:00
80b8c6e225 Bug 25078: Fix indentation and style for additional descriptions
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-31 10:27:52 +02:00
Julian Maurice
4605304f62 Bug 28306: Allow to query database with minimal memory footprint
The goal is to be able to build a database handler (dbh) and to execute
queries without loading unnecessary stuff. This will be useful to reduce
memory usage of daemons that need to check the database
periodically

The patch provides a new method Koha::Database::dbh which returns a
database handler without loading the DBIx::Class schema. This method is
also used by DBIx::Class, so whether you use DBI or DBIx::Class, the
same method is used to initialize the connection.

The patch also moves some code in order to avoid loading C4::Context:
- C4::Context::timezone moves to Koha::Config
- C4::Context::db_scheme2dbi moves to Koha::Database

To measure memory usage I used the following commands:

* before the patch:
perl -MKoha::Database \
    -E 'Koha::Database->schema->storage->dbh->do("select 1");' \
    -E '$|=1; say $$; sleep 2' \
    | while read pid; do ps -p $pid -o rss=; done

* after the patch:
perl -MKoha::Database \
    -E 'Koha::Database->dbh->do("select 1");' \
    -E '$|=1; say $$; sleep 2' \
    | while read pid; do ps -p $pid -o rss=; done

It will give you the RSS (Resident Set Size) of the perl process in kB

What I get:
* before the patch: between 96.9MB and 97.2MB
* after the patch: between 17.8MB and 18.2MB

Note that if a timezone is configured (either from $KOHA_CONF or
TZ environment variable), Koha will load DateTime::Timezone to check if
it's valid, and it increases RSS to 36MB

Another interesting metric is the number of modules loaded:
* before the patch:
perl -MKoha::Database \
    -E 'Koha::Database->schema->storage->dbh;' \
    -E 'say scalar keys %INC'

Result: 567

* after the patch:
perl -MKoha::Database \
    -E 'Koha::Database->dbh;' \
    -E 'say scalar keys %INC'

Result: 51

Test plan:
1. Apply the patch & restart starman
2. Make sure Koha is still ok (ie. can access the database, does not
have encoding issues, ...)
3. Run the tests in t/Context.t, t/Koha/Config.t,
t/db_dependent/Koha/Database.t, t/timezones.t

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-31 10:27:52 +02:00
70d2deb103 Bug 28774: Don't store blank values for rental discount
This patch adds 'can_be_blank => 0' for the rentaldiscount rule to prevent
storing blank values in the database

Additionally, if there is no charge we do not need to check for a discount
and can simply return

To test:
1 - Set rental discount to "" to a rule in circulation rules
2 - Checkout an item that will follow this rule
3 - Check the intranet log:
    [WARN] Argument "" isn't numeric in subtraction (-) at /kohadevbox/koha/C4/Circulation.pm line 3385.
4 - Apply patch and restart all
5 - Update database
6 - Set the rule to "" again
7 - Check the DB, no rule is stored
    SELECT * FROM circulation_rules WHERE rule_name = 'rentaldiscount';
8 - Checkout the item again
9 - No warns in log

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-31 10:27:52 +02:00
fc20c964b0 Bug 28373: Add PassItemMarcToXSLT system preference
Default stylesheets do not reference item fields for XSLT display, however, we
spend time translating the values in the item fields.

This patch adds a system preference, PassItemMarcToXSLT. and unless enabled we remove
item fields before processing

To test:
1 - Perform some search on the staff client and opac
2 - Use the console (F12) to view the time spent on the network tab
3 - Note performance
4 - Apply patch, updatedatabase, restart_all
5 - Repeat searches
6 - Note that display has not changed
7 - Note performance, results should display slightly faster

Signed-off-by: Emmi Takkinen <emmi.takkinen@koha-suomi.fi>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-31 08:49:04 +02:00
6d85e5c80b Bug 26302: (QA follow-up) Prefix prefs with OPAC
git grep -l "ResultsMaxItems" | xargs sed -i -e "/ResultsMaxItems/ s/ResultsMaxItems/OPACResultsMaxItems/g"

And line:
koha-tmpl/opac-tmpl/bootstrap/en/xslt/MARC21slim2OPACResults.xsl:    <xsl:variable name="OPACResultsMaxItems" select="number(marc:sysprefs/marc:syspref[@name='resultsMaxItems']+0)"/>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-30 17:04:31 +02:00
7b9f0106a3 Bug 26302: Make syspref's names start with uppercase
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-30 17:04:31 +02:00
513d72135e Bug 26302: Add resultbranch and other status in C4/XSLT
Groundwork for changes in the OPAC results xslt.

NOTE: Adds both new prefs too.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-30 17:04:30 +02:00
deece7b76b Bug 28830: Add cni index for 003
This patch adds the cni/Control-number-identifier index to enable
searches to use the 003 field.

Test plan
1/ Apply patch
2/ Re-index using updated configurations
3/ Confirm cni:number searches yield the expected results
4/ Signoff

Split-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Pasi Kallinen <pasi.kallinen@koha-suomi.fi>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-30 17:02:07 +02:00
9f78462025 Bug 25619: Adjust POD and move date check before logging
Potentially we could have logged a change when no date was passed.

This patch moves the test before logging and updates POD

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-27 15:28:23 +02:00
f008b3ebe7 Bug 25619: Adjust POD
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-27 15:28:23 +02:00
d303e49f09 Bug 25619: Add ability to adjust expiration date for waiting holds
There are times when an item that is waiting for pickup needs to have the expiration date extended. This would give staff the ability to modify one by one, as needed, the reserves.expirationdate for a given item awaiting pickup.

Test Plan:
1) Place a hold, trap an item for it such that is is waiting
2) Attempt to update the expiration date
3) Note the new date is not saved
4) Apply this patch, restart all the things!
5) Attempt to update the expiration date
6) The new date should be saved!

Signed-off-by: Abbey Holt <aholt@dubuque.lib.ia.us>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-27 15:28:23 +02:00
46a1342226 Bug 22690: Remove uneeded return and add no_triggers
* C4/Items.pm
  - Koha::Biblios not used

* Koha/Item.pm
  - Koha::Item->orders must return an empty set if no order attached
  - no_triggers should be passed to other update calls

* Item.t
  - No need to build a fund
  - Add new test to test Koha::Item->orders when no order attached

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-27 15:28:23 +02:00
Ere Maijala
2615ee1010 Bug 22690: Refactor merging of records to improve performance (Elasticsearch)
This patch allows merging of records with many items without the web server timing out.

Test plan:

Without the patch:

- Create 2 records (one with e.g. 1000 items).
- Do a cataloguing search that displays both records, select them and click "Merge selected".
- Choose the record with many items as the one to be eliminated.
- Start the merging.
- After a while the web server should give you a timeout error (the merging process may still continue)

With the patch:
- Do the same as above
- This time verify that the records are merged without timeout
- Create a new biblio with an item
- Add with the item:
  * acquisition order
  * hold (reserve)
- Merge the biblio to another one
- Verify that the item and its related data was moved
- Verify that tests pass:
  prove -v t/db_dependent/Koha/Biblio.t
  prove -v t/db_dependent/Koha/Item.t
  prove -v t/db_dependent/Koha/SearchEngine/Indexer.t

Signed-off-by: Michal Denar <black23@gmail.com>
Rebased-by: Joonas Kylmälä <joonas.kylmala@helsinki.fi>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-27 11:02:40 +02:00
421e3972c6 Bug 28776: Warns from GetItemsInfo when biblio marked as serial
GetItemsInfo sorts the items using serial fields if the biblio is
defined as a serial

The fields may not be defined. We can add a fallback to blank in this case

To test:
1 - Mark a record with items as serial in the marc (942$s)
2 - At least one of the items should have no enumchron defined
3 - Load the detail page
4 - Warns in logs:
    [WARN] Use of uninitialized value in string comparison (cmp) at /kohadevbox/koha/C4/Items.pm line 811.
5 - Apply patch and restart all
6 - Reload the details page
7 - No more warns

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-18 14:53:59 +02:00