Same as Load, but for Dump.
Test plan:
Edit ES mappings, replace withdrawn's label with "withdrawn ✔️❤️ ★"
Export the mappings
perl misc/search_tools/export_elasticsearch_mappings.pl > admin/searchengine/elasticsearch/mappings.yaml
Reset mappings from the UI
=> Notice that the label is correct
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Joonas Kylmälä <joonas.kylmala@helsinki.fi>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
From tht YAML pod:
"""
This module has been released to CPAN as YAML::Old, and soon YAML.pm will be changed to just be a frontend interface module for all the various Perl YAML implementation modules, including YAML::Old.
If you want robust and fast YAML processing using the normal Dump/Load API, please consider switching to YAML::XS. It is by far the best Perl module for YAML at this time. It requires that you have a C compiler, since it is written in C.
"""
See also
https://gitlab.com/koha-community/qa-test-tools/-/merge_requests/35
Test plan:
Try some place where YAML::XS is not used and confirm that it works
correctly
QA note: This patch removes some uses of YAML that were not useful
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Joonas Kylmälä <joonas.kylmala@helsinki.fi>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch allows multiple --category_code <categorycode> in delete_patrons.pl
The t/db_dependent/Members.t test has 2 added testcases with multiple category_code.
To test:
1) Run the test: prove -l . -v t/db_dependent/Members.t
2) This requires records in the database with at least two category codes. For example PT and ST.
3) Run the script: delete_patrons.pl --category_code PT
4) Confirm the number of patrons that would have been deleted.
5) Run the script: delete_patrons.pl --category_code PT --category-code ST
6) Confirm the number of patrons that would have been deleted.
7) Sign Off
Sponsored-by: Lunds Universitetsbibliotek
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This feature is not used as far as we know and it's not known to work.
It's preferable to remove it.
Test plan:
Make sure the OpacGroupResults pref code is removed, as well as the
PazPar2 files and code.
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
The check-url.pl script has been deprecated since Koha 3.8 - this patch removes it.
If any users are still referencing that script in their cronjobs they will need to
update to the new script upon upgrade.
To test:
1 - verify the script is gone
2 - verify check-url-quick.pl works
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
If the script misc/import_patrons.pl is run without "--confirm" it reports
that it is "running in dry-run mode" but it is not!
Test Plan:
1) Import a CSV of patrons using import_patrons.pm *without* the --confirm parameter
2) Note that it does actually import the patrons
3) Apply this patch
4) Note that this time the patrons are not actually imported
5) Import again with --confirm
6) Note that the patrons were actually imported this time!
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Test Plan:
1. Go to Administration
2. Go to System Preferences
3. Find 'delimiter' (note the name)
4. Apply patch
5. Repeat step 1 and 2
6. Find 'CSVDelimiter' (note the name)
Signed-off-by: Sally <sally.healey@cheshiresharedservices.gov.uk>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
*** WARNING: empty section in previous paragraph at line 28 in file misc/cronjobs/advance_notices.pl
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch removes the double ups in the perldoc of the above file,
deleting the less clear double ups. The missing "-c" in OPTIONS
appears to have been fixed in an earlier patch (it is there
already).
Task Plan:
1. Run perldoc misc/cronjobs/advance_notices.pl in the terminal. Note
the double ups of NAME, SYNOPSIS, and DESCRIPTION
2. Apply the patch
3. Run the command again. The double ups should be gone, with the more
clear version of each of NAME, SYNOPSIS, and DESCRIPTION remaining.
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Test Plan:
1. Go to Administration
2. Go to System Preferences
3. Find 'opaclanguages' (note the lack of capitalisation)
4. Apply patch
5. Repeat step 1 and 2
6. Find 'OPACLanguages' (note the use of capitalisation)
Signed-off-by: Barbara Johnson <barbara.johnson@bedfordtx.gov>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Bug 23290 explicitly disabled entity expansion during XML parsing for security reasons.
However, many XSLTs define the following entity:
<!ENTITY nbsp " " >
They don't use the entity  , but its presence could lead to confusion.
Signed-off-by: Eden Bacani <eden.bacani@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Update script to use Koha::Logger to capture method failures.
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This script simply iterates through installed plugins that impliment a
cronjob_nightly method and runs said method.
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This script ensure that no sysprefs have been changed directly in the database and/or
that the cache has not become corrupted. We have occasionally seen this happen on production sites
To test:
1 - In the staff interface go to Administration
2 - Search for system preference 'IntranetUserJS'
3 - Add content to the syspref:
console.log('Hi!');
4 - On the command line launch mysql
sudo koha-mysql kohadev
5 - Alter the syspref directly
UPDATE systempreferences SET value = "console.log('Bye!');" WHERE variable = 'IntranetUserJS';
6 - run the script
perl misc/maintenance/check_syspref_cache.pl
7 - You are warned about the altered system preference
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch removes code that prevents fines from being update on holidays.
The fines are already being calculated, and that takes the calendar into account if
needed, so fines won't be assessed on holidays if they shouldn't.
To test:
1 - set finesCalendar to 'ignore'
2 - Checkout an item due yesterday
3 - Ensure circ rules have a fine amount set
4 - Make today a holiday
5 - run fines.pl
6 - No fine assessed
7 - Apply patch
8 - run fines.pl
9 - Fine assessed!
10 - set finesCalendar to 'use'
11 - checkout an item, due yesterday
12 - run fines.pl
13 - no fine assessed
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
One last instance of ModBiblioMarc in bulkmarcimport still passed the
frameworkcode.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Tested with :
For a framework (not the default) :
Creation of a biblio record
Edition of this biblio record
Creation of an item of this record
Creation of an item of this record
./misc/batchRepairMissingBiblionumbers.pl OK
prove t/db_dependent/Biblio/ModBiblioMarc.t OK
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
To test:
1 - Check out an over due to a patron
2 - Set overdue triggers to a trigger less than number of days overdue
3 - Make sure the patron has no email
4 - Set content of ODUE notice to:
[%# Comment will not print %]
5 - perl misc/cronjobs/overdue_notices.pl
6 - Check the message queue
7 - There is a notice sent to the library:
"These messages were not sent directly to the patrons."
With no messages
8 - Apply patch
9 - Repeat
10 - No new message generated
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
To prevent
DBD::mysql::db begin_work failed: Already in a transaction at /usr/share/perl5/DBIx/Class/Storage/DBI.pm line 1588, <GEN45> line 1.
Test plan:
1 - Export more than 100 records
2 - Use bulkmarcimport (with -commit=10) to import them
perl misc/migration_tools/bulkmarcimport.pl -b -file testbmi.xml -v -m=MARCXML --commit=10
3 - Modify a record to make the import fail (for instance having a too long
lccn)
4 - Use bulkmarcimport (with -commit=10) to import them
perl misc/migration_tools/bulkmarcimport.pl -b -file testbmi.xml -v -m=MARCXML --commit=10
5 - Notice that the import stops but that the imported record are imported (apart from the last batch of 10)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
MySQL 5.7 does not allow queries using 0000-00-00
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
The SQL code was duplicated, I combine them here
frombranch needs to take input
switch owning to frombranch in second script
initialize hash as empty list, not a reference
add a newline after printed output if not mailing notices
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
As discussed, this patch updates the --owning parameter to more clearly
represent it's function.
--owning boolean => --frombranch=item-homebranch|item-issuebranch
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch fixes a perlcritic issue and also replaces tabs with four
spaces where appropriate.
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
The provided patch adds the following functionality:
* Add --owning to both advance_notice.pl and overdue_notice.pl
* Add --library to advance_notice.pl like the way overdue_notice.pl already works
When specifying "--owning" both scripts will use items.homebranch instead of
issues.branchcode to determine sending library.
For advance_notice.pl this affects non-digest DUE and PREDUE, but not digest notices.
To test:
1. Have a patron who wants advance notices as email with 2 days in advance (not digest)
2. Have the first overdue date set 3 days past due date
3. Have issue for that patron where date_due is 2 days away where the item homebranch differs from the issuing branch
4. Have issue for that patron where date_due is 3 days old
5. Run advance_notices.pl without --owning
6. Run overdue_notices.pl without --owning
7. Confirm that two messages were created for that patron with the sender being the issuing branch
8. Delete messages or create two more issues according to (3) and (4)
9. Run advance_notices.pl with --owning
10. Run overdue_notices.pl with --owning
11. Confirm that the two messages created has the item homebranch as sender
Extra feature for advance_notices.pl is that it adds "--library" the same way overdue_notices.pl has.
Adding variants of that flag in steps (5) and (9) above can confirm this option as well.
Sponsored-by: Gothenburg University Library
Signed-off-by: Björn Nylén <bjorn.nylen@ub.lu.se>
Signed-off-by: Timothy Alexis Vass <timothy_alexis.vass@ub.lu.se>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
misc/cronjobs/cleanup_database.pl provides some database cleanup.
This patch adds cleanup for messages table.
Test plan :
- Count messages : select count(*),date(message_date) from messages group by date(message_date);
- Run cleanup : misc/cronjobs/cleanup_database.pl -v --confirm --messages 365
- Recount messages
Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
-library <branchname> only deal with overdues from this library (repeatable : several libraries can be given)
Should be <branchcode>
Help text should also be proper sentences.
To test:
0) Run misc/cronjobs/overdue_notices.pl -h
1) Check that the text is now:
-library <branchcode> Only deal with overdues from this library.
2) Check that the rest of the help text are proper sentences,
with leading capital letter and punctuation.
3) Sign off.
Sponsored-by: Lunds Universitetsbibliotek
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
When multithreaded indexing is used, the commit size for children are spread
out resulting in them being of type float. When records are processed and the
commit counter decreased it may then never reach *exactly* 0. This means records
are never commited. This patch makes sure the counter is an integer to avoid the
problem.
To test you must find a set of circumstances that causes the issue. For me:
1. Run: ./rebuild_elasticsearch -v -b -p 2 -c 400
2. Note that only one process is logging "Committing xxx records..."
3. Kill processes.
4. Apply patch.
5. Repeat 1
6. Note that both processes are logging "Committing xxx records..."
Sponsored-by: Lund University Library
Signed-off-by: Joonas Kylmälä <joonas.kylmala@helsinki.fi>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This script checks that an authority is not being used before deleting it.
We do not need to check biblios for the authority id as we have already verified it is unused.
TO test:
1 - Reset db to koha test database
2 - perl misc/migration_tools/remove_unused_authorities.pl -t
3 - Note a number of unsued authorities
4 - perl misc/migration_tools/remove_unused_authorities.pl -c
5 - Note authorities are removed
6 - Reset db
7 - Apply patch
8 - perl misc/migration_tools/remove_unused_authorities.pl -t
9 - Note results are the same
10 - perl misc/migration_tools/remove_unused_authorities.pl -c
11 - Note results are the same
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
The merging of Bug 23051 has added a paramter that can be passed to
C4::AddRenewal. This bug does also, so the offset of this bug's new
paramter was wrong. This patch fixes it.
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch adds support for unseen renewals.
Here we retrofit knowledge of unseen renewals and add the display of unseen
renewal counts and warnings, in addition to adding the ability to
specify a renewal as being "unseen".
The functionality added here is goverened by the UnseenRenewals syspref.
Signed-off-by: Sally Healey <sally.Healey@cheshirewestandchester.gov.uk>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
misc/translator/translate was doing three different things:
- extract translatable strings
- create or update PO files
- install translated templates
This patch separates responsibilities by moving the string extraction
code into several 'xgettext-like' scripts and adds gulp tasks to
automate string extraction and PO files update
This has several benefits:
- gulp runs tasks in parallel, so it's a lot faster (updating all PO
files is at least 10 times faster with my 4-cores CPU)
- there is no need for $KOHA_CONF to be defined
LangInstaller.pm relied on $KOHA_CONF to get the different paths
needed. I'm not sure why, since string extraction and PO update should
work on source files, not installed files
- string extraction code can be more easily tested
This patch also brings a couple of fixes and improvements:
- TT string extraction (strings wrapped in [% t(...) %]) was done with
Template::Parser and PPI, which was extremely slow, and had some
problems (see bug 24797).
This is now done with Locale::XGettext::TT2 (new dependency) which is
a lot faster, and fixes bug 24797
- Fix header in 4 PO files
For backward compatibility, 'create' and 'update' commands of
misc/translator/translate can still be used and will execute the
corresponding gulp task
Test plan:
1. Run `yarn install` and install Locale::XGettext::TT2
2. Run `gulp po:update`
3. Verify the contents of updated PO files
4. Run `cd misc/translator && ./translate install <lang>`
5. Verify that all (templates, sysprefs, xslt, installer files) is
correctly translated
6. Run `gulp po:create --lang <lang>` and verify that it created all PO
files for that language
7. Run `prove t/misc/translator`
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Need to install yarn & gulp, no errors
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch moves the CatalogModuleRelink lookup outside the loop to
improve performance (especially for large databases), and includes a
tester friendly test plan.
Test plan:
0. Using koha-testing-docker
1. koha-shell -c "./misc/link_bibs_to_authorities.pl -l --test" kohadev
2. Note 436 bibs checked, 1315 headings linked, 676 headings unlinked
3. koha-shell -c "./misc/link_bibs_to_authorities.pl -l --test -g 700" kohadev
4. Note 436 bibs checked, 248 headings linked, 111 headings unlinked
5. Note output appear to all be names (rather than subjects or other authorities)
6. Note also that the 2nd run with -g 700 is almost 2x faster than without
Signed-off-by: Andreas Roussos <a.roussos@dataly.gr>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
When running link_bibs_to_authorities.pl, it could be useful to have
the ability to specify which MARC field the script should operate on.
For example, when you have a catalog where most links are in place,
and you don't want the script to process every MARC field but rather
limit it to a specific MARC field (e.g. 700) because you've observed
that there are many missing links for that particular MARC field.
This patch provides that enhancement.
Test plan:
1) Run the script as follows (preferably in a test DB):
time link_bibs_to_authorities.pl -v -l --test
... and notice how long it takes to complete.
2) Apply the patch.
3) Run the script again, this time providing the MARC field to work on:
time link_bibs_to_authorities.pl -v -l --test -g=700
... and, again, notice how long it takes to complete.
The 2nd run should run faster than the 1st one.
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
While the ES index is incremental and provides results as it commits, we currently index from the oldest records to the newest.
This patch provides the option to go the other direction
To test:
1 - Have ES setup and running for Koha
2 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b
3 - Note the biblios index from number 1 the end
4 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a
5 - Notice the same
6 - Apply patch
7 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b
8 - Still in ascending order
9 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b --desc
10 - Now records index in descending order
11 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a
12 - Still ascending
13 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a --desc
14 - Now descending
Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
JD amended patch: fix typo "inde" vs "index" and add commit body
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch adds the --reason option to cancel_expired_holds which allows
the library to optionally set a reason for cancellation when running the
cronjob. This will prompt the HOLD_CANCELLED notice to be sent to the
patron.
To test:
1/ Ensure the unit tests continue to pass after the patch
(t/db_dependent/Reserves/CancelExpiredReserves.t)
Also:
1 - Add an expired hold for a patron:
INSERT INTO RESERVES (borrowernumber, biblionumber, expirationdate, found,branchcode,itemnumber) VALUES (5,5,'2020-01-01','W','CPL',983);
2 - Set ExpireReservesMaxPickUpDelay to Allow
3 - Run the cronjob:
perl misc/cronjobs/hold/cancel_expired_holds.pl --reason EXPIRED
4 - Visit the patron's notices tab
5 - Confirm they have been sent a cancellation notice
Signed-off-by: Lisette Scheer <lisettes@latahlibrary.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
