Commit graph

2958 commits

Author SHA1 Message Date
33e463c108
Bug 36068: (QA follow-up) Add OPTIONS section in POD
Test plan:
Run script with --help.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-19 18:06:31 +02:00
e50582293b
Bug 36068: Add maint script acq_cancel_obsolete_orders.pl
Introducing $orders->filter_by_obsolete and $orders->cancel.

Test plan:
Run t/db_dependent/Koha/Acquisition/Orders.t
Create basket with few orders. Remove some biblio records.
Run acq_cancel_obsolete_orders.pl -c

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-19 18:06:30 +02:00
c6641ef24f
Bug 34621: Tidy import_patrons.pl
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-12 14:02:43 +02:00
b89d744e86
Bug 34621: implement Patron import option to 'Renew existing patrons' 'from the current membership expiry date'
Test Plan:
1) Test importing a patron with the "from the current membership expiry date" option,
   note it does not work
2) Apply this patch
3) Restart all the things!
4) Re-test, note the expiration was renewed from the patron's current
   expiration date!

Signed-off-by: David Nind <david@davidnind.com>
Rebased-by: Victor Grousset/tuxayo <victor@tuxayo.net>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-12 14:02:41 +02:00
Magnus Enger
4848def5e2
Bug 36517: Fix output from install_plugins.pl
When misc/devel/install_plugins.pl does not find any plugins, it prints
the list of pluginsdir, but with a literal \n separating the dirs, and
no newline at the end.

To test:
- Edit koha-conf.xml and add a second entry for <pluginsdir>, so there
  are two entries. The second one could just be a copy of the original.
- Run "perl misc/devel/install_plugins.pl"
- Note the output looks something like this:

No plugins found
pluginsdir contains:
/var/lib/koha/kohadev/plugins\n/var/lib/koha/kohadev/pluginsroot@kohadevbox:koha(master)$

- Apply the patch and run the script again. Output should be:

No plugins found
pluginsdir contains:
/var/lib/koha/kohadev/plugins
/var/lib/koha/kohadev/plugins
root@kohadevbox:koha(master)$

- Sign off

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-12 11:26:01 +02:00
0026ee3e97
Bug 36516: Fix useless warning from translation script
I lost too much time trying to know what the translation script outputs
  sh: 1: Syntax error: end of file unexpected

To recreate:
touch 'koha-tmpl/intranet-tmpl/prog/en/modules/<' (yes, don't ask!)
koha-translate -i de-DE --dev kohadev
or
koha-translate -u de-DE --dev kohadev

And you get the shell error without any ideas what's happening!

With this patch applied you will not get any errors because the quote
will actually copy the file (what is expected).

But without the quote the cp command will fail and the explanation will be
displayed
sh: 1: Syntax error: end of file unexpected
Cannot copy /kohadevbox/koha/koha-tmpl/intranet-tmpl/prog/en/modules/< to /kohadevbox/koha/koha-tmpl/intranet-tmpl/prog/de-DE/modules/< at /kohadevbox/koha/misc/translator/tmpl_process3.pl line 385.

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-05 11:51:19 +02:00
3feca570d8
Bug 36031: Remove obsoleted script
Since it is not up to date, we should remove it.
The folder structure changed and this would need digging into SELinux to update that side as well.
In other words, remove this until someone finds time do it. But dont let it here in its current outdated state.

No test plan here.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-05 11:51:16 +02:00
ade75ea561
Bug 23296: Set patron branchcode when preparing non-digest version
To test:
1. APPLY PATCH
2. Set up circ rules to allow for auto renewals.
3. Set AutoRenewalNotices to 'according to message prefs'
4. Make a branch specific version of the the AUTO_RENEWAL notice.
5. Find a patron belonging to that branch and set the message prefs to allow for a AUTO_RENEWAL. But no digest option!
6. Check some things out and make sure they are eligble for auto renewal.
7. Run the cron.
8. Make sure the patron gets the branch specific notice.
9. Delete the branch specific notice.
10. DO 6-7 again.
11. Make sure they now get the all library notice.

Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] Tidied one line.
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-05 11:51:12 +02:00
1ff71cf200
Bug 36323: Move koha_perl_deps.pl to misc/devel
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-03 17:05:51 +02:00
fd70ee6a94
Bug 31286: Embed see-from headings into bibliographic records export
In misc/export_records.pl add an option to add see-from headings (from authorities 4xx) into bibliographic records export.
Like it is done during record indexing.

Test plan :
1) Choose a biblio record having a field (for example 650) linked to an authority with a see-form.
2) Export this record without see-from headings :
   misc/export_records.pl --starting_biblionumber=X --ending_biblionumber=X --filename /tmp/record_without.xml --format xml
3) Export this record with see-from headings :
   misc/export_records.pl --starting_biblionumber=X --ending_biblionumber=X --filename /tmp/record_with.xml --format xml --embed_see_from_headings
4) Compare /tmp/record_without.xml and /tmp/record_with.xml
=> you should see two 650, one with main heading and one with see-from heading

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-25 09:53:24 +01:00
Emily Lamancusa
7552b73503
Bug 35653: (QA follow-up) Perltidy
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-25 09:53:21 +01:00
3198621787
Bug 35653: Log import_patrons.pl info to action logs
To test:
1. APPLY PATCH
2. Create a CSV file with a few lines that contain at least firstname, surname, cardnumber
3. Turn on "CronjobLog".
4. Run the import patron script. I did this: perl /kohadevbox/koha/misc/import_patrons.pl --file 'test1.csv' --matchpoint cardnumber -c
5. Check the action logs for cronjobs.
6. You should see info. about the import patron script.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-25 09:53:21 +01:00
d8e04545b8
Bug 35536: Refine verbose handling in some Koha::Plugins calls
Three routines in Plugins got the verbose parameter on 35507.
We can refine this a bit further.
The idea here is report when you are installing plugins but not
report when just calling plugins (flooding logs).

[1] GetPlugins: Most callers do not expect (or check) results for
    failing plugins. This patch makes GetPlugins only return
    errors when passing the *errors* flag (in 2 cases).
    [a] The misc/devel script prints warnings now using verbose,
        so does not need the errors flag anymore.
    [b] plugins/plugins-home is the only case left. Tiny adjustment
        to keep current behavior. Fixed colspan in template.
        Does not need verbose in favor of 'errors' (passed to
        template).
    [c] For most calls we do not want verbose. New default is 0.

[2] InstallPlugins
    [a] Disabled verbose in plugin-upload. Not really needed.
        Added a FIXME; we need to improve individual install.
    [b] misc/devel: No warnings anymore when calling InstallPlugins
        after GetPlugins.

[3] get_enabled_plugins
    [a] Plugins->call does not need verbose.
    [b] Plugins->feature_enabled does not need it too.

Test plan:

[1] See previous plan. With TestMR data but without patch, run
misc script and go to plugins-home. Do you see load errors on
commandline or form?

[2] Run plugins/plugins-upload (uploading just some file is good
enough); verify that you do not see TestMR lines in logfile.

[3] Run t/db_dependent/Koha/Plugins/Plugins.t for the additional
test on verbose and errors flag.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-22 12:26:40 +01:00
30816a7fc5
Bug 30349: Add missing double dash (--delete) to POD
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-22 10:35:24 +01:00
David Schmidt
06e7553dc9
Bug 30349: More double dashes
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-22 10:35:24 +01:00
David Schmidt
c84c614146
Bug 30349: Use double dash for multi character option names
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-22 10:35:23 +01:00
David Schmidt
ccf18acb97
Bug 30349: Tidy bulkmarcimport.pl to fix indentations
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-22 10:35:23 +01:00
a1a7193110
Bug 34611: Confirm run if pseudonymized transactions found and add verbose feedback
Signed-off-by: AFHDubCoLib <andrewfh@dubcolib.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-19 10:25:06 +01:00
59368f5c9d
Bug 34611: Add a script to pseudonymize statistics from before a given date
This script takes a date parameter in SQL format and pseudonymizes all statistics
found before this date.

Only values that can be found will be added, i.e. no deleted patron or item info
will be present.

Additionally - the values stored will be the current values from patrons and items, so
some info will be approximate, much as it is when joining from the statistics table for reporting.

To test:
 1 - Perform some issues/returns/renewals/on-site checkouts
 2 - Make sure Pseudonymization is disabled
 3 - perl misc/maintenance/pseudonymize_statistics.pl
 4 - Script ends and reports that preference is not active
 5 - Enable the pref, and choose some borrower and item fields
    NOTE: See bug 28911 if you need a bcrypt key for your koha-conf.xml
 6 - perl misc/maintenance/pseudonymize_statistics.pl
 7 - sudo koha-mysql kohadev
 8 - SELECT * FROM pseudonymized_transactions
 9 - Confirm data is correctly stored
10 - DELETE FROM pseudonymized_transactions;
11 - UPDATE statistics SET datetime = '2023-01-01 00:00:00';
12 - perl misc/maintenance/pseudonymize_statistics.pl -b "2022-12-31 23:59:59";
13 - SELECT * FROM pseudonymized_transactions;
14 - Confirm no entries were made
15 - Select different options in Pseudonmyization prefs, including borrower attributes
     This wil require defining an attribute that can be kept for pseudonymization
16 - Confirm options are correctly pseudonymized

Signed-off-by: AFHDubCoLib <andrewfh@dubcolib.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-19 10:25:05 +01:00
David Gustafsson
6aa7118c13
Bug 25539: Strip items when adding new biblio to preserve previous behaviour of the script
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-18 18:03:40 +01:00
David Gustafsson
e1479daf11
Bug 25539: Remove AddBiblio option "defer_marc_save"
Items are no longer embedded in the MARCXML and because of this the MARC data
does not need to be saved once more after changing record items data.
The "defer_marc_save" is no longer needed since bulkmarcimport.pl was the only
place this option was utilized in order to resave MARC data after possibly changing
items data. There is also a bug bulkmarcimport.pl where the record data is re-saved
without stripping items if duplicate items are found and the dedup barcodes
option is enabled that is resolved by this change.

This change enables MARC Overlay rules to be enabled for bulkmarkcimport.pl as using
the defer_marc option would previously effectively bypass any defined rules.

To test
1) Apply patch 29440 (which this depends on), but no not yet apply this
   patch
2) Remove comments around "bulkmarcimport: _("bulkmarcimport.pl"),"
   in koha-tmpl/intranet-tmpl/prog/en/modules/admin/marc-overlay-rules.tt
   line 463
3) Enable MARC overlay rules and add a rule (with source => "*" or
   'bulkmarcimport') to protect some field.
4) Import a biblio using the bulkmarcimport script. Overlay rules will
   only be applied on updates so a match condition matching the
   record in Koha when bulkmarcimport.pl is run the next time for the same
   record needs to be supplied (for example -match "control-number,001")
5) Edit the imported biblio and change the value of the protected field
6) Run bulkmarcimport with the same parameters as in 4) and verify the
   the field that should have been protected has been overwritten
7) Revert changes for line 463 in marc-overlay-rules.tt (or will result in
   conflict applying the patch), apply the patch and repeat steps 4-5,
   verify that the field now was protected from being overwritten.

Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-18 18:03:40 +01:00
David Gustafsson
70de25b439
Bug 29440: Fix AddBiblio/ModBiblio options args
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-18 11:03:47 +01:00
David Gustafsson
188ea40db1
Bug 29440: Replace removed syspref BiblioAddsAuthorities with AutoLinkBiblios
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-18 11:03:46 +01:00
David Gustafsson
6b92516214
Bug 29440: Only log items has been inserted if actually have been so, and remove extra out of place logging of record insertion
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-18 11:03:46 +01:00
David Gustafsson
fd3efab876
Bug 29440: Rename $modify_biblio_marc_options to $mod_bilbio_options
This variable actually contains ModBibio options and
ModifyBiblioMarc accepts a subset of these

Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-18 11:03:45 +01:00
David Gustafsson
33d274bbb9
Bug 29440: Remove pointless call to _strip_item_fields
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-18 11:03:45 +01:00
David Gustafsson
2d50eb242d
Bug 29440: replace usage of removed GetMarcBiblio
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-18 11:03:44 +01:00
David Gustafsson
08908b5765
Bug 29440: defer biblio autolinking until records are indexed
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-18 11:03:44 +01:00
David Gustafsson
2dddbb4d14
Bug 29440: Fix QA-tool issues
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-18 11:03:43 +01:00
David Gustafsson
aa86825c33
Bug 29440: Make authorities import behavior consistent with biblios
The current behavior for when importing authorities, when match
option is enabled, is to insert as a new authority fetching from db
fails. This is both a bit confusing and not consistent with the biblio
case where an error is generated. Also clean up error and log messages
so both messages/errors and when messages/errors are issued are more
consistent for the biblio and authority case.

Signed-off-by: Frank Hansen <frank.hansen@ub.lu.se>
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-18 11:03:43 +01:00
David Gustafsson
261baeff85
Bug 29440: Refactor and clean up bulkmarcimport.pl
To test:
1) Import biblios using ./bulkmarcimport.pl -v -b -file biblios.mrc -insert -update -c=MARC21 -match "control-number,001" -n=1 -l "/tmp/import.log"
2) Verify in /tmp/import.log that one biblio has been inserted
3) Import again without the "-n" option, verify that one biblio has been
   updated and the rest inserted
4) In the staff interface search the catalog for some string appearing in
   the imported biblios to verify records have been indexed
5) View/edit some of the biblios the staff interface to very have been
   properly imported
6) Import authorities using ./bulkmarcimport.pl -v -m=MARCXML -a -file authorities.xml -insert -update -c=MARC21 -keepids="024a" -match "heading-main,100a" -l "/tmp/import.log" -yaml="/tmp/ids.yml"
7) Go to the authorities page and find the imported authority by for example search for a
   400 or 100 field value. Verify that the 001 field of the incoming record has been moved to 024a.
   Also check that the value of 024a appear as a key in /tmp/ids.yml.
8) Edit the authorities.xml file and change the value of field 400a
9) Import authorities again using the same options
10) Verify that the 400a field of the authority in Koha has not been
   updated (the import was skippped since revision did not change)
11) Edit authorities.xml, replace the value of field 005 in the file with field 005
    of the authority in Koha incremented by one
12) Import again with the same options
13) Verify that the 400a field of the authority in Koha now have been
    updated

Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-18 11:03:42 +01:00
e29e058851
Bug 33898: Implement reaping for database polling
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-15 09:38:37 +01:00
bc42006019
Bug 33898: Alternative approach with receive frame timeout
See bug 33898 comment6.

Test plan:
Based on comment12: Stage MARC import and manage. Look at the same
time to the results of ps aux|grep Z. Verify that the lines with
[background_jobs] <defunct> disappear within 10 seconds.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-15 09:38:36 +01:00
3b8a3c7b86
Bug 31427: Remove a few refs to CanBookBeRenewed
Actually, the module is not even needed anymore here.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-07 16:36:44 +01:00
c86e6ee82d
Bug 35819: (QA follow-up) Prevent warning on uninitialized retries count
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-07 16:36:39 +01:00
7c1d4716be
Bug 35819: Adjust es_indexer_daemon
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] Add forgotten module
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-07 16:36:39 +01:00
30e69ebc56
Bug 35819: Improve logging
Log (warn) if the job will be processed later, but add a debug however.

Have a specific log for bad status

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-07 16:36:38 +01:00
8394092568
Bug 35819: nack and not requeue if frame is invalid
If a frame cannot be correctly processed (most probably because the body
is not valid JSON) then we are not acking or nacking the frame and the
worker is stuck.

In this specific case we should nack without requeuing the frame.

NOTE that requeue must be 'true' or 'false', not 1 or 0, or the default
'true' will be used.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-07 16:36:38 +01:00
0feb10df7f
Bug 35819: Add simple delay
Here I add 500 ms. In my testing with the 1s sleep from the test
plan, I might see one or two 'not found' lines. Obviously things
depend on the time needed before the txn commits. But it will
reduce a flood of these messages.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-07 16:36:37 +01:00
1a51c2e973
Bug 35819: Notify NACK and requeue when Job ID not found
This patch makes the worker reject the incoming frame for putting the
message back in the queue, in the event the job id doesn't exist yet.
Which is the case when some actions are being triggered inside a
transaction which hasn't been commited to the DB yet.

To test you will need 3 KTD shells
(a) mysql:
   $ ktd --shell
  k$ sudo koha-mysql kohadev

(b) logs:
   $ ktd --shell
   # for restarting the worker and looking at the logs
  k$ sudo koha-worker --restart kohadev  ; tail -f /var/log/koha/kohadev/worker-*.log
(c) running the test:
   $ ktd --shell

1. Have (a), (b) and (c) terminals ready
2. On (c), run:
   $ perl -MKoha::Database -MKoha::BackgroundJob::BatchUpdateBiblioHoldsQueue -e 'Koha::Database->schema->txn_do( sub { Koha::BackgroundJob::BatchUpdateBiblioHoldsQueue->new->enqueue({ biblio_ids => [ 1 ] }); sleep 1;  } );'
=> FAIL:
   * (b) shows (once) an error about a job not existing
3. On (a) run:
   > SELECT * FROM background_jobs;
=> FAIL: Notice the job ID mentioned on 2 stands as 'new'.
4. Apply this patch
5. Ctrl+c on (b), and re-run to launch the worker with the patch applied
6. Repeat 2
=> SUCCESS (partial): The error about the job not existing is displayed
many times
7. Repeat 3
=> SUCCESS: The job ID mentioned on 6 stands as 'finished'.
8. Sign off :-D

Discussion:

* The `requeue` header I added is correct, but it is the default
  behavior anyway. I prefered to make it explicit, though.

* To avoid that bunch of retries, we should requeue with some delay. I
  didn't manage to make it work (yet) but there's a 'delay' plugin for
  rabbit [1]. We already install the 'stomp' plugin in
  koha-common.postinst. But this plugin requires downloading it. Which
  would require further investigation.

* As Nick and Marcel pointed, we need to revisit the whole architecture,
  the need of a MQ (DB polling wouldn't have this problem), etc. But
  that's for another place.

[1] https://hevodata.com/learn/rabbitmq-delayed-message/#:~:text=To%20delay%20a%20message%2C%20the,to%20queues%20or%20other%20exchanges.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-07 16:36:36 +01:00
Julian Maurice
972efe8b5c
Bug 36009: Document background_jobs_worker.pl --queue elastic_index
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-07 15:02:49 +01:00
0fe82b601b
Bug 36084: Add CSRF token support to svc/authentication
GET svc/authentication will return a CSRF token in a response header

POST svc/authentication requires a CSRF token which can be sourced
from the response header of GET svc/authentication or some other
place like the meta element on a HTML page

Note: misc/migration_tools/koha-svc.pl is a simple script which
can be used to practically evaluate svc/authentication and svc/bib

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2024-03-01 11:00:51 +01:00
632146366a
Bug 34478: op =~ ^cud- in pl/pm - Manual revert changes to recreateIssueStatistics.pl
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2024-03-01 10:57:15 +01:00
a1338de383
Bug 34478: op =~ ^cud- in pl/pm - Manual revert changes to sip_cli_emulator.pl
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2024-03-01 10:57:14 +01:00
18e808240f
Bug 34478: op =~ ^cud- in pl/pm
This is the result of
  bash op_must_start_with_cud-perl.sh

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2024-03-01 10:56:11 +01:00
d7a633048e
Bug 35479: (QA follow-up): Tidy
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-01-31 14:44:41 +01:00
ea30aaff62
Bug 35479: (follow-up) Match formatting for other cronjob logs
To test:
1 - Install the Kitchen Sink plugin
2 - Restart all
3 - Enable 'CronjobLog' system preference
4 - perl misc/cronjobs/plugins_nightly.pl
5 - Note you see on the command line 'Remember to clean the kitchen' - this indicates the plugin cron ran
6 - Tools->log viewer, select 'cronjob' and view
7 - Note you only see 'plugins_nightly.pl' Run and End lines
8 - Apply patches
9 - perl misc/cronjobs/plugins_nightly.pl
10 - View logs agian
11 - Note you now see Run and End lines for 'Koha::Plugin::Com::ByWaterSolutions::KitchenSink'
12 - Confirm they look like the other lines
13 - Edit KitchenSink.pm and add 'die "Kittens";' to the cronjob nightly
14 - perl misc/cronjobs/plugins_nightly.pl
15 - View logs, confirm there is a FAILED error message for the KitchenSink cron

Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-01-31 14:44:41 +01:00
c4b777b056
Bug 35479: Log start, end, and any failures for each plugin's nightly cronjob
When running the plugins_nightly.pl cronjob, we should record the plugins that have a nightly method, logging the start and end of each plugins routine

Test Plan:
1) Enable CronjobLog
2) Install a plugin with a nightly cronjob ( e.g.
   https://github.com/bywatersolutions/koha-plugin-book-list-printer )
3) Run plugins_nightly.pl
4) Note new entries in the cronjob viewer for the start and end of the
   plugin's nightly cronjob run
5) Edit the plugin, add a line like "die 'this is a test';" to the
   plugin's nightly cronjob
6) Run plugins_nightly.pl
7) View the action logs, not the log for the error you added!

Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-01-31 14:44:40 +01:00
Caroline Cyr La Rose
633533049a
Bug 35596: Error in writeoff_debts documentation
This patch corrects an error in the documentation of the
writeoff_debts.pl script. The --type parameter accepts debit type codes,
not credit type codes.

To test:
-  Run ./misc/cronjobs/writeoff_debts.pl
-  Make sure the sentence for --type makes sense, grammar and spelling
   are correct

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-01-29 12:24:54 +01:00
Katrin Fischer
898d96adb9
Bug 35373: Remove false FIXME comment from gather_print_notices.pl
Removes a FIXME line mentioning bug 8000 that implemented
SendAllEmailsTo. But the note was added to the "From" line
and the "To" address is taken from a command line parameter.

To test:
* Review change carefully

Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-01-26 15:14:07 +01:00