Commit graph

8128 commits

Author SHA1 Message Date
cbf405828a Bug 29134: Use a subquery to increase performance of patron attributes search
This patch generates a subquery and checks if a borrowrnumber is in the results to add patrons
to search results

To test:
 1 - Generate a bunch of patrons:
    SELECT surname, firstname, branchcode, categorycode FROM ( SELECT surname FROM borrowers ORDER BY rand() ) a,( SELECT firstname FROM borrowers ORDER BY rand() ) b,( SELECT branchcode FROM borrowers ORDER BY rand() ) c,( SELECT categorycode FROM borrowers ORDER BY rand() ) d LIMIT 50000
 2 - Add a patron attribute to the system and make it searchable - I used code 'TEST'
 3 - Add a value for this attribute to many patrons:
   INSERT INTO borrower_attributes (borrowernumber,code,attribute) SELECT borrowernumber, 'TEST','alphabet' FROM borrowers LIMIT 10000;
 4 - In staff client got 'Patrons'
 5 - Open the browser console (F12) and view the netwrok tab
 6 - Perform a patron search for 'a'
 7 - Note the time it takes for 'search' to complete in console
 8 - Apply patch, restart_all
 9 - Repeat search
10 - Note it is much faster
11 - prove -v t/db_dependent/Utils/Datatables_Members.t

NOTE: I tested with 500k patrons and 100k attributes - search returned in ~2 seconds with patch
and did not return before I got impatient without patch

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-05 09:17:20 +02:00
58c765c492 Revert "Bug 28510: Remove unnecessary conditional"
This reverts commit d284735d05.

The following test was failing randomly:
 #   Failed test 'take from lowest cost branch (don't use cost matrix) holding branch'
 #   at t/db_dependent/HoldsQueue.t line 1494.
 #          got: 'LHKtxLk'
 #     expected: 'JL9C_OR'
 # Wrong pick-up/hold for first target (pick_branch, hold_branch, reserves, hold_fill_targets, tmp_holdsqueue)

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-01 16:28:55 +02:00
bc67449a46 Bug 28826: Add FacetOrder system preference
This bug adds a system preference to control ordering of facets and
adds the control to both Zebra and Elasticsearch

To test:
1 - Have a koha that can use both Zebra and ES
2 - Set 'displayFacetCount' to true
3 - Search in ES and Zebra
4 - Note facets in Zebra sorted alphabetically, ES by usage
5 - Apply patch, updatedatabase
6 - Search in ES and Zebra, facets are alphabetically sorted in both
7 - Find new syspref FacetOrder and set to 'by usage'
8 - Search in both engines, facets sorted by usage

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-29 11:34:21 +02:00
d284735d05 Bug 28510: Remove unnecessary conditional
It makes sense to use items by branch to get the list of branches, as it
already tells use which branches have available items. We could use
branches to pull from instead, but all we would accomplish is added
extra ununsed loop iterations. We already know that any additional
branches in the branches to pull loop have no items to fill holds.
If they did, they would be in the items_by_branch hash.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-28 15:12:45 +02:00
ba23348db6 Bug 28510: Remove marking of closed branches as 'disable_transfer'
We no longer need to act as if closed branches were marked as
disable_transfer. This allows us to clean up a nice bit of code.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Bug 28510: Remove unused variable

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-28 15:12:45 +02:00
b7ae33b47b Bug 28510: Skip processing holds queue items from closed libraries when HoldsQueueSkipClosed is enabled
Right now we skip closed branch's items as we iterate over all items looking for ones to fill a hold. If HoldsQueueSkipClosed is enabled, no items held be a closed library can be targeted, so it would be more efficient if we never selected the items from those branches to begin with. This is how the holds queue works if we are not using the transport cost matrix, so we should make it work the same way if we *are* using the matrix.

Test Plan:
1) Apply this patch
2) prove prove t/db_dependent/HoldsQueue.t
3) All tests should continue to pass

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-28 15:12:45 +02:00
33165bd22e Bug 28803: (follow-up) Error details improvement
This patch updates the exceptions thrown by Koha::Email to include the
parameter that failed email validation and then updates the failure code
to include this parameter and finally display this field in the template.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-28 10:22:36 +02:00
9cfbdee6a8 Bug 28803: Add invalid email handling in _send_message_by_email
This patch adds a try/catch block around the call to Koha::Email->create
to catch and handle invalid emails being passed in the parameters.

The message is marked as 'failed' with an error_code of 'INVALID_EMAIL'.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-28 10:22:36 +02:00
Andrew Isherwood
f9f457ab14 Bug 24190: (follow-up) Rename AcqLog
As requested in comment #49, renamed uses of AcqLog to AcquisitionLog

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

JD amended patch: replace one missing occurrence in Budgets.t

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-21 20:22:57 +02:00
Andrew Isherwood
114f4ea5a3 Bug 24190: (follow-up) Modify order of logging
Basket creation involves two steps in Koha, creating the basket then
modifying its header. We were logging these two steps in the wrong
order. This commit fixes that.

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-21 20:22:57 +02:00
Andrew Isherwood
50109e06f9 Bug 24190: (follow-up) Respond to QA feedback
This commit makes changes in response to Jonathan's feedback in comment

- Moved from using zero padded strings to store log data to a JSON
object
- Stopped storing formatted dates in logged data

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-21 20:22:57 +02:00
Andrew Isherwood
bf90fb810e Bug 24190: Add acquisition logging
Signed-off-by: Maura Stephens <maura.stephens@gmit.ie>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-21 20:22:57 +02:00
87de9fbb8a Bug 28847: Remove dead code and make regex consistent
This updates the regex code to match throughout the routine and removes
code that would no longer be reached.

Note that the code to update library name in the query descruiption was
broken before this patch, so removal does not change behaviour

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-20 14:04:48 +02:00
065ec4e726 Bug 28847: Cleanup of branch limitsand fix OPAC_SEARCH_LIMIT
The opac had 'branch_group_limit' parameters which can be simplified to more
closely match intranet code.

Adjust C4::Auth for chaneg above to ensure dropdowns correctly populate

Expand JS to prevent selection of single and multibranch limits

To test:
1 - Enable OpacAddMastheadLibraryPulldown system preference
2 - Ensure branches and groups show as before patch
3 - Ensure single and multibranch limits from masthead apply as expected
4 - Test advanced search page, ensure you cannot select both single and multibranch limit
5 - Follow test plan on 28845 - ensure multibranch limit still correctly pre-selected

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-20 14:04:48 +02:00
777c519c5a Bug 28847: Move SearchLimitLibrary code to QueryBuilder Modules
This patch removes the code from the search scripts into QueryBuilder
modules.

To test:
1 - Have a library group defined as a search group for both staff and opac
2 - Search on staff client and opac with that group limit and a single branch limit
3 - Note your results/counts
4 - Note the visuals of the search description
5 - Apply patch
6 - Repeat searches
7 - All should work as before

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-20 14:04:48 +02:00
91a557bd08 Bug 28464: Remove useless check that gives the incorrect error message
Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-20 14:04:48 +02:00
Andreas Roussos
5de0705275 Bug 28472: handle items with NULL shelving location
The UpdateItemLocationOnCheckin System Preference can be set to update the
location of ALL items during check in, regardless of their shelving location.
However, this does not currently work 100% as it is excluding items with no
shelving location (i.e. value of NULL in the corresponding database field).

This patch, based on the comment made by Nick Clemens, fixes that.

Test plan (based on the original Bug Description by Andrew Fuerste-Henry):

1) Have a shelving location CART
2) In UpdateItemLocationOnCheckin, enter "_ALL_: CART" (without the quotes)
3) Check in an item that has a shelving location, confirm it changes to CART
4) Check in an item with a NULL shelving location, confirm it doesn't go to CART
5) Apply this patch
6) Repeat step 4): this time the item should move to the CART shelving location

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-20 12:07:02 +02:00
ea88cfb3ab Bug 28676: Cache and retrieve match_count when searching a cached heading
We use match_count to determine if a new authority record should be created, however,
we were not adding this count to the cache, so if a record returned too many matches on first
lookup, we would create a new record on the second lookup

To test:
1 - Set Linker Module to 'Default'
2 - Enable  AutoCreateAuthorities  and  BiblioAddsAuthorities and  CatalogModuleRelink and LinkerRelink
3 - Add two copies of a single authority via Z39
4 - Add two headings for that authority to a bib record (e.g. a 610 and 710)
5 - Save the record and note a new authority is generated
6 - Repeat and see another is generated
7 - Apply patch
8 - Restart all the things
9 - Save the record again, no new authority created

Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-20 12:06:56 +02:00
80b8c6e225 Bug 25078: Fix indentation and style for additional descriptions
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-31 10:27:52 +02:00
Julian Maurice
4605304f62 Bug 28306: Allow to query database with minimal memory footprint
The goal is to be able to build a database handler (dbh) and to execute
queries without loading unnecessary stuff. This will be useful to reduce
memory usage of daemons that need to check the database
periodically

The patch provides a new method Koha::Database::dbh which returns a
database handler without loading the DBIx::Class schema. This method is
also used by DBIx::Class, so whether you use DBI or DBIx::Class, the
same method is used to initialize the connection.

The patch also moves some code in order to avoid loading C4::Context:
- C4::Context::timezone moves to Koha::Config
- C4::Context::db_scheme2dbi moves to Koha::Database

To measure memory usage I used the following commands:

* before the patch:
perl -MKoha::Database \
    -E 'Koha::Database->schema->storage->dbh->do("select 1");' \
    -E '$|=1; say $$; sleep 2' \
    | while read pid; do ps -p $pid -o rss=; done

* after the patch:
perl -MKoha::Database \
    -E 'Koha::Database->dbh->do("select 1");' \
    -E '$|=1; say $$; sleep 2' \
    | while read pid; do ps -p $pid -o rss=; done

It will give you the RSS (Resident Set Size) of the perl process in kB

What I get:
* before the patch: between 96.9MB and 97.2MB
* after the patch: between 17.8MB and 18.2MB

Note that if a timezone is configured (either from $KOHA_CONF or
TZ environment variable), Koha will load DateTime::Timezone to check if
it's valid, and it increases RSS to 36MB

Another interesting metric is the number of modules loaded:
* before the patch:
perl -MKoha::Database \
    -E 'Koha::Database->schema->storage->dbh;' \
    -E 'say scalar keys %INC'

Result: 567

* after the patch:
perl -MKoha::Database \
    -E 'Koha::Database->dbh;' \
    -E 'say scalar keys %INC'

Result: 51

Test plan:
1. Apply the patch & restart starman
2. Make sure Koha is still ok (ie. can access the database, does not
have encoding issues, ...)
3. Run the tests in t/Context.t, t/Koha/Config.t,
t/db_dependent/Koha/Database.t, t/timezones.t

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-31 10:27:52 +02:00
70d2deb103 Bug 28774: Don't store blank values for rental discount
This patch adds 'can_be_blank => 0' for the rentaldiscount rule to prevent
storing blank values in the database

Additionally, if there is no charge we do not need to check for a discount
and can simply return

To test:
1 - Set rental discount to "" to a rule in circulation rules
2 - Checkout an item that will follow this rule
3 - Check the intranet log:
    [WARN] Argument "" isn't numeric in subtraction (-) at /kohadevbox/koha/C4/Circulation.pm line 3385.
4 - Apply patch and restart all
5 - Update database
6 - Set the rule to "" again
7 - Check the DB, no rule is stored
    SELECT * FROM circulation_rules WHERE rule_name = 'rentaldiscount';
8 - Checkout the item again
9 - No warns in log

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-31 10:27:52 +02:00
fc20c964b0 Bug 28373: Add PassItemMarcToXSLT system preference
Default stylesheets do not reference item fields for XSLT display, however, we
spend time translating the values in the item fields.

This patch adds a system preference, PassItemMarcToXSLT. and unless enabled we remove
item fields before processing

To test:
1 - Perform some search on the staff client and opac
2 - Use the console (F12) to view the time spent on the network tab
3 - Note performance
4 - Apply patch, updatedatabase, restart_all
5 - Repeat searches
6 - Note that display has not changed
7 - Note performance, results should display slightly faster

Signed-off-by: Emmi Takkinen <emmi.takkinen@koha-suomi.fi>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-31 08:49:04 +02:00
6d85e5c80b Bug 26302: (QA follow-up) Prefix prefs with OPAC
git grep -l "ResultsMaxItems" | xargs sed -i -e "/ResultsMaxItems/ s/ResultsMaxItems/OPACResultsMaxItems/g"

And line:
koha-tmpl/opac-tmpl/bootstrap/en/xslt/MARC21slim2OPACResults.xsl:    <xsl:variable name="OPACResultsMaxItems" select="number(marc:sysprefs/marc:syspref[@name='resultsMaxItems']+0)"/>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-30 17:04:31 +02:00
7b9f0106a3 Bug 26302: Make syspref's names start with uppercase
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-30 17:04:31 +02:00
513d72135e Bug 26302: Add resultbranch and other status in C4/XSLT
Groundwork for changes in the OPAC results xslt.

NOTE: Adds both new prefs too.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-30 17:04:30 +02:00
deece7b76b Bug 28830: Add cni index for 003
This patch adds the cni/Control-number-identifier index to enable
searches to use the 003 field.

Test plan
1/ Apply patch
2/ Re-index using updated configurations
3/ Confirm cni:number searches yield the expected results
4/ Signoff

Split-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Pasi Kallinen <pasi.kallinen@koha-suomi.fi>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-30 17:02:07 +02:00
9f78462025 Bug 25619: Adjust POD and move date check before logging
Potentially we could have logged a change when no date was passed.

This patch moves the test before logging and updates POD

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-27 15:28:23 +02:00
f008b3ebe7 Bug 25619: Adjust POD
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-27 15:28:23 +02:00
d303e49f09 Bug 25619: Add ability to adjust expiration date for waiting holds
There are times when an item that is waiting for pickup needs to have the expiration date extended. This would give staff the ability to modify one by one, as needed, the reserves.expirationdate for a given item awaiting pickup.

Test Plan:
1) Place a hold, trap an item for it such that is is waiting
2) Attempt to update the expiration date
3) Note the new date is not saved
4) Apply this patch, restart all the things!
5) Attempt to update the expiration date
6) The new date should be saved!

Signed-off-by: Abbey Holt <aholt@dubuque.lib.ia.us>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-27 15:28:23 +02:00
46a1342226 Bug 22690: Remove uneeded return and add no_triggers
* C4/Items.pm
  - Koha::Biblios not used

* Koha/Item.pm
  - Koha::Item->orders must return an empty set if no order attached
  - no_triggers should be passed to other update calls

* Item.t
  - No need to build a fund
  - Add new test to test Koha::Item->orders when no order attached

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-27 15:28:23 +02:00
Ere Maijala
2615ee1010 Bug 22690: Refactor merging of records to improve performance (Elasticsearch)
This patch allows merging of records with many items without the web server timing out.

Test plan:

Without the patch:

- Create 2 records (one with e.g. 1000 items).
- Do a cataloguing search that displays both records, select them and click "Merge selected".
- Choose the record with many items as the one to be eliminated.
- Start the merging.
- After a while the web server should give you a timeout error (the merging process may still continue)

With the patch:
- Do the same as above
- This time verify that the records are merged without timeout
- Create a new biblio with an item
- Add with the item:
  * acquisition order
  * hold (reserve)
- Merge the biblio to another one
- Verify that the item and its related data was moved
- Verify that tests pass:
  prove -v t/db_dependent/Koha/Biblio.t
  prove -v t/db_dependent/Koha/Item.t
  prove -v t/db_dependent/Koha/SearchEngine/Indexer.t

Signed-off-by: Michal Denar <black23@gmail.com>
Rebased-by: Joonas Kylmälä <joonas.kylmala@helsinki.fi>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-27 11:02:40 +02:00
421e3972c6 Bug 28776: Warns from GetItemsInfo when biblio marked as serial
GetItemsInfo sorts the items using serial fields if the biblio is
defined as a serial

The fields may not be defined. We can add a fallback to blank in this case

To test:
1 - Mark a record with items as serial in the marc (942$s)
2 - At least one of the items should have no enumchron defined
3 - Load the detail page
4 - Warns in logs:
    [WARN] Use of uninitialized value in string comparison (cmp) at /kohadevbox/koha/C4/Items.pm line 811.
5 - Apply patch and restart all
6 - Reload the details page
7 - No more warns

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-18 14:53:59 +02:00
fd829dbdcb Bug 25078: Keep atomic updates in "atomicupdate" dir
The atomicupdate directory will contain all the atomic update files (old
and new version).
That will ease job for the RM and RMaints, no file (except skeletons)
must be in this directory before push.

This patch also fixes an inconsistency we had: the atomic update was run
before the other db revs on the UI but after when the CLI script was
used.

Now we make sure that the CLI does not deal with the atomic update files
when called from the installer (UI)

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-16 11:55:56 +02:00
Julian Maurice
25dda58f04 Bug 25078: Fix encoding issue with db_revs output
From `perldoc -f open`:

The scalars for in-memory files are treated as octet strings: unless the
file is being opened with truncation the scalar may not contain any code
points over 0xFF.

So $out need to be decoded first in order to be used in other Perl
strings

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-16 11:55:56 +02:00
Julian Maurice
005f20507d Bug 25078: Close open filehandle
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-16 11:55:56 +02:00
b86a54ad73 Bug 25078: (follow-up) Update 'NewVersion' for output_version changes
This patch updates the NewVersion compatability method to add arrayref
description handling to split it into description + report.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-16 11:55:56 +02:00
Julian Maurice
ef9aab0e59 Bug 25078: Separate update "report" from its description
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-16 11:55:56 +02:00
312c5f16d8 Bug 25078: Re-introduce NewVersion
This is ugly, we re-add the code we removed in the previous patch.

We need to continue supporting "old" versions.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-16 11:55:56 +02:00
8c07ba4c80 Bug 25078: Put db revs into different files to handle them better
This patch suggests to stop using updatedatabase.pl to add new DB revs.
Each DB rev will be in a separate pl files (installer/data/mysql/db_revs).

The switch should ideally be done from 21.06.00.000.

Each DBrev is executed in a try block and a transaction. If something
went wrong, the whole DB rev is rolled back.

Why do /var/log/koha/kohadev/updatedatabase_*.log (not -error) contain
    Status: 500
    Content-type: text/html

    <h1>Software error:</h1>
etc.

Test plan:
- git checkout c4b4db21d2 (master on 2021-07-08)
- Set the version syspref to 21.0500000:
    > update systempreferences set value="21.0500000" where variable="version";
- Apply "Bug 25078: [DO NOT PUSH] DB revs for testing" (restart_all)
- Read the different DBrevs created as examples
- Make sure the different use cases are covered
- execute the updatedatabase script (CLI)
- Set the version syspref to 21.0500000
- Update the DB from the UI
- Set the version syspref to 21.0500000
- execute the updatedatabase script with the --force parameter (for
testing purpose)

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-16 11:55:56 +02:00
89ade834d6 Bug 24387: Rename "News" with "Additional contents"
One big patch for one big move.

The "News" feature (opac_news) has been hijacked to handle some system
preferences (bug 26050). The goal was to take profit of the UI (editor)
and the ability to translate the value.

Disclaimer: This patch is NOT offering the best implementation but, as
we still don't have bug 24975, it cannot be done now. And no, we don't
want to wait for it to move forward here. This patch is going into the
right direction anyway.

This enhancement is going to rename the "News" with a more genertic
"Additional contents". We have two different "categories" of content:
"news" and "html customizations".

What does it bring?
- A split on the UI for disambigate the two types of content (news and
syspref/html customizations)
- A simplification of the edit form: all languages will be translatable
on the same view (like the "notice templates")
- Ground will be prepared for different types of content (if needed later)
- Staff news can be translated

How was the "News" area working before this patch?
The opac_news DB table contained a (very inconsistent) 'lang' column.
The different values were:
- '' => news to display at the OPAC and staff interfaces
- 'koha' => news for staff only
- 'slip' => news for slip notices
- $lang => news for OPAC only, translated in $lang ('en', 'es-ES', etc.)
- "$location_$lang" => A syspref moved to this "news" area. The syspref
is $location, and is translated in $lang. Eg. OpacLoginInstructions_en,
OpacLoginInstructions_fr-FR, opacheader_es-ES

This patch is improving the DB structure with the following changes:
- renaming 'opac_news' with 'additional_contents'
- new 'category' column
 => 'news' or 'html_customizations'
- new 'location' column
 => For 'news': 'staff_and_opac', 'staff_only', 'slip'
 => For 'html_customizations': the old syspref name (eg. 'OpacLoginInstructions').
- new 'code' column (see later for more info)
- the 'lang' column will only contain the language code ('en', 'es-ES',
etc.). BUT a 'default' entry will ALWAYS exist for fallback behaviour.

We are getting closer to the 'notice template' table structure because
we want to match its UI. The 'code' column will bring us the ability to
group the different 'additional_contents' rows. The code for a given
news will be the same, but the (lang, title, content) will differ.
Examples:
  News 1 will have, for each of the translated versions
  (category, code,    location, branchcode)
  ('news',   'News1', $location, $branchcode||undef)
  And the 3 following columns will differ:
  (title,              content,               lang)
  ('title for news 1', 'content for news 1', 'default')
  ('titulo para 1',    'contenido para 1',   'es-ES')

Note that the "category" is not strictely necessary, but it seems better
to have the ability to split the different content by category/type
easily.

Additional changes:
- Syspref 'NewsToolEditor' is renamed 'AdditionalContentsEditor'
- Koha::NewItem => Koha::AdditionalContent
- Koha::News => Koha::AdditionalContents
- Script and template renamed from koha-news to additional-contents
- Foreign keys have been renamed
- Subpermission edit_news has been renamed edit_additional_contents
- The UI can now be accessed via a "News" or "HTML customizations" link
from the tools module. The related contents will then be displayed (both
categories are now split)

Changes not done here:
- Primary key 'idnew' could be renamed 'id'

Limitations of the upgrade:
News cannot be grouped by a unique code for existing translations.
=> A given news will be now displayed several times on the translated
interface
Any ideas to improve the upgrade behaviour?
We will have to add a warning in the release notes to tell libraries to
review their news.

Test plan:
0. Don't apply the patches
1. Translate the interfaces in some languages
. Create some news for staff and OPAC
. Create some content for different entry of HTML customizations
Note that you are forced to define a 'default'.
Also note that you are only forced to fill the title (not the content).
This is certainly problematic (see FIXME in the code) as sometime only
the content is displayed.
. Play with the interface (edit, delete, filter)
. Go to the different places the news are displayed, and confirm they
are displayed correctly (staff home, opac home, opac rss)
. Create 1+ news for 'slip', check an item out and 'print slip' (from
the circulation page). You must see the news.
. Go to the different places you are expecting the HTML customizations
to be present and confirm that you see them.
. Switch the lang of the interface and confirm that you now see the
content in the translated version
. Generate the templates in another language, don't translate the
content
. Use this language for the interface and confirm that the 'default'
version is displauyed.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-16 11:55:55 +02:00
977774f095 Bug 12561: Change prototype of XSLTParse4Display
Use a hash for better readability and reusability.

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-16 09:57:05 +02:00
56c2856e26 Bug 12561: Remove non-XSLT views
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-16 09:57:04 +02:00
24103412bc Bug 12561: Remove HighlightOwnItemsOnOPAC and HighlightOwnItemsOnOPACWhich
This is the only remaining non-XSLT only feature. We should remove the
non-XSLT views that have been deprecated for a while now, and remove
this feature.

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-16 09:57:04 +02:00
Janusz Kaczmarek
e9428f662f Bug 26852: subfield $e missing in X11 definition of MARC21 headings
This patch adds $e to 111 and 611, but removes it from 100 and 110 as it's
used for the relator term there and should not be copied. Same for 111$j.

Test plan:

1. Have a biblio record with, for example, 711 field with a $e subfield.
2. Have two auth records: one with 111 field reflecting the 711 field and second similar but without $e subfield
3. Have LinkerModule set to Default
4. Run link_bibs_to_authorities.pl
5. 711 should remain unlinked since there were two record found by the Default Linker and no one was chosen.

6. Apply the patch.
7. Run link_bibs_to_authorities.pl
8. 711 should be linked now.

Repeat testing in similar fashion for authorities:

9. 100/110 $e
10. 111 $j

NB zebra index definition is OK:
kohaidx:index_match_heading tag="111" subfields="acdefghjklnpqstvxyz" subdivisions="vxyz">
in etc/zebradb/marc_defs/marc21/authorities/authority-koha-indexdefs.xml

Signed-off-by: Sara Brown <sbrown@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-11 13:27:52 +02:00
229aaf6d2b Bug 28813: Rename delivery_note to failure_code
We now use the codes from the half implimented error_code in place of
the plain text that was being added to the delivery_note field. As part
of that we rename the field to failure_code to clarify it's intended
content.

Test plan
1/ Confirm t/db_dependent/Letters.t passes

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-11 12:53:54 +02:00
0f4bad73de Bug 28630: Add track_login_daily to ILSDI::AuthenticatePatron
This patch adds a call to track_login_daily for the AuthenticatePatron
method in ILSDI.

Many libraries use this ILSDI endpoint for electron book borrowing and
need to maintain the lastseen based on this type of lending.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-04 14:06:43 +02:00
b5a09e4654 Bug 22435: ->apply() should always use 'APPLY' for offset_type
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-04 14:06:43 +02:00
660b3936b6 Bug 15067: (QA follow-up) Ensure update is idempotent
These tables really shoul;d have some unique keys, we need a test so we can add
those

This ended up being a bigger work than expected, RM feel free to reject or move
to another bug and let the risk of duplicated languages

This is an old one, let's get it in :-)

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-04 14:06:43 +02:00
83e899f8b1 Bug 15067: Follow up to fix sorting
It looks like C4::Languages:getLanguages relies on implicit database
order for languages. They appear to be sorted by subtag so this patch
adds that sorting to the database query to ensure the dropdown is
sorted.

Signed-off-by: Aleisha <aleishaamohia@hotmail.com>

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-04 14:06:43 +02:00
946b9dc0b1 Bug 28754: Only adjust holds on specific biblio and don't go past end of array
Our query for lowest priority holds only needs to adjust holds on the biblio we are looking at
so I add biblionumber

Additionally we can simply find the end of the array and use that rather than 99998
so I set new_rank to scalar @priority

Lastly, we don't need to fetch the lowest priority holds if we are ignoring lowest priority
so I move it into the conditional

To test:
 1 - Add holds with lowest priorty to 2 records in the catalog
 2 - Add a hold on a third record
 3 - Note errors in log like:
    [2021/07/23 17:47:17] [WARN] splice() offset past end of array at /kohadevbox/koha/C4/Reserves.pm line 1649
 4 - Apply patch and restart all the things
 5 - Add a new hold on third record - no warns
 6 - Make one of the holds on third record have lowestPriority (click rightmost arrow with line at bottom)
 7 - Note no warns
 8 - Adjust other holds on record and note no warns

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Joonas Kylmälä <joonas.kylmala@helsinki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-04 09:14:47 +02:00