Commit graph

553 commits

Author SHA1 Message Date
ed23bc1364
Bug 35086: (follow-up) Use 5000 as example in conf file
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-01-16 12:06:02 +01:00
9951e230e4
Bug 35086: Add chunk_size option to elasticsearch configuration
Whne performing batch operations we can send a large numebr of records for reindexing at once.
Currently this can create requetss that are too large for Elasticsearch to process. We need
to break these requests into chunks/

This patch adds a chunk_size configuration to the elasticsearch stanza in koha-conf.xml

If blank we default to 5000.

To test:
0 - Have Koha using Elasticsearch
1 - Create and download a report of all barcodes:
    SELECT barcode FROM items
2 - Batch modify these items
3 - Note a single ESindexing job is created
4 - Create and download a report of all authority ids:
    SELECT auth_header.authid FROM auth_header
5 - Setup a marc modification template, and batch modify all the authorities
6 - Again note a single ES backgorund job is created
7 - Apply patch
8 - Repeat the modifications above - you still get a single job
9 - Edit koha-conf.xml and add <chunk_size>250</chunk_size> to elasticsearch stanza
10 - Repeat modifications - you now get several background ES jobs
11 - prove -v t/db_dependent/Koha/SearchEngine/Elasticsearch/Indexer.t

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-01-16 12:06:00 +01:00
7375d82c40
Bug 35455: Remove = in ICU for indexing/searching
This change more closely aligns ICU and CHR so that ICU also
removes the = character. This fixes issues in ICU when searching
with a : which gets transformed into a =. Without this change,
the Analytics features won't work for titles with a colon in them.

Test plan:
0. Apply the patch and import bibs from Bugzilla (using Staged MARC tools)
1. cp ./etc/zebradb/etc/phrases-icu.xml /etc/koha/zebradb/etc/phrases-icu.xml
2. cp ./etc/zebradb/etc/words-icu.xml /etc/koha/zebradb/etc/words-icu.xml
3. vi /etc/koha/zebradb/etc/default.idx
Change "charmap word-phrase-utf.chr" to "icuchain words-icu.xml" for "index w"
and "icuchain phrases-icu.xml" for "index p"
4. koha-zebra --stop kohadev
5. pkill zebrasrv
6. koha-zebra --start kohadev
7. koha-rebuild-zebra -a -b -f -v kohadev

8. Search for "Awesome title" and open the detail page
9. Note that the "Analytics: Show analytics" line shows up
10. Click that link
11. Note that it opens the "Cool article" record and it displays
"In: Awesome title: awesome subtitle"
12. Click that link
13. Note that it opens the "Awesome title" record

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2023-12-20 15:13:46 +01:00
e172b279c3
Bug 34868: (QA follow-up) Fix XML
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-10 10:59:39 -03:00
1feeead338
Bug 34868: (QA follow-up) Rename new option, add comments
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-10 10:59:38 -03:00
5788a0f262
Bug 34868: Add ability for SIP2 to distinguish missing item from other lost types
The SIP circulation status specifies that a 12 means an item is lost, and 13 means an item is missing. In Koha, missing items are simply a type of lost item so we never send a 13. This is an important distinction for some SIP based inventory tools. It would be good to be able to specify when lost status means "missing" at the SIP login level.

Test Plan:
1) Apply this patch
2) prove t/db_dependent/SIP/Transaction.t

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-10 10:59:38 -03:00
7e87b91ce3
Bug 25814: Add comment to example sip config file
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-08 11:41:23 -03:00
Matthias Meusburger
c125aa7ecd
Bug 25814: SIP: Add a message on successful checkin.
Currently, Koha does not return a message on successful SIP checkin.

This patchs adds the show_checkin_message option to SIPconfig.xml, disabled by
default. When enabled, the following message is displayed on SIP checkin:

"Item checked-in: {homebranch|permanent_location} - {location}"

The UseLocationAsAQInSIP system preference is used to determine whether the
homebranch or the permanent location will be used.

Test plan:

 - Perform a successful checkin using SIP
 - Check that the message is in the checkin response (AF field)
 - prove t/db_dependent/SIP/Transaction.t

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Edit (tcohen): tidied the whole subtest.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-08 11:41:23 -03:00
ea22082649
Bug 25672: Convert to restricting to listed repositories only
I think instead of a plain on/off switch we should use it in combination
with the plugin_repo's and set it to restrict to only those repos' (i.e.
disable uploads entirely if no repo's are listed, or just allow those
repo's when there are).

This patch achieves that, but only if plugins are installed via the
plugin browser method. We disable all direct upload avenues, so install
is blocked for other cases.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 11:02:43 -03:00
bfd633c461
Bug 25672: Enable enable_plugin_browser_upload by default
This patch enables enable_plugin_browser_upload by default,
since the current behaviour for Koha is to enable browser upload
when enable_plugins is 1.

Signed-off-by: Nicolas Legrand <nicolas.legrand@bulac.fr>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 11:02:41 -03:00
e2e61ec6ca
Bug 25672: Use enable_plugin_browser_upload flag to control plugin upload
This patch adds a enable_plugin_browser_upload flag to koha-conf.xml, which
controls whether or not Koha intranet users can upload Koha plugins via
their browser. Like "enable_plugins", it defaults to 0 for new installs.

This is useful when you want to provide Koha intranet users with plugins
that are pre-installed by administrators (by CLI) or restricting them
to plugins from a Github repo. See the following for more information:
Bug 23975 - Add ability to search and install plugins from GitHub
Bug 23191 - Administrators should be able to install plugins from the command line

To test:
1) Apply the full patchset
2) Confirm <enable_plugins>1</enable_plugins> is present in koha-conf.xml
3) Add <plugins_restricted>1</plugins_restricted> to koha-conf.xml
4) Ensure that the <plugin_repos> block is not commented and contains at
   least one trusted organisation in koha-conf.xml
   If needed get it from: debian/templates/koha-conf-site.xml.in
5) Run restart_all (in koha-testing-docker)
6) Go to /cgi-bin/koha/plugins/plugins-home.pl and note that you don't see
   an option to upload plugins
7) You should however see a search option and upon search you should have
   results returned from the chosen trusted organisations listed in the
   <plugin_repos> block mentioned above.
8) Clicking install on one of the results should work as expected and install
   the plugin.
9) Go directly to /cgi-bin/koha/plugins/plugins-upload.pl and note that it says
   "Plugin upload is restricted to only those plugins listed by your server
   administrator" and gives instructions on how to enable unrestricted browser
   upload.

Signed-off-by: Nicolas Legrand <nicolas.legrand@bulac.fr>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Rebased-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 11:02:41 -03:00
cf90b426d1
Bug 30708: Add apache RewriteRule
Sponsored-by: BULAC - http://www.bulac.fr/
Signed-off-by: BULAC - http://www.bulac.fr/
Signed-off-by: Heather Hernandez <heather_hernandez@nps.gov>
Signed-off-by: Laurence Rault <laurence.rault@biblibre.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-18 15:41:42 -03:00
7f2d38d85e
Bug 34153: Add ability to allow items with additional materials notes to be checked out via SIP
At this time, any item with an additional materials date is blocked from checkout via SIP with a screen message to take the item to a circulation desk for checkout.
Some libraries wish to allow patrons to check out items via SIP even if the item has additional materials.

Test Plan:
1) Create an item with an additional materials note
2) Attempt to check it out via SIP
3) Note the failure and message
4) Enable the new SIP account option "allow_additional_materials_checkout"
5) Restart the SIP server
6) Attempt the checkout again
7) Note the checkout success and new AF field message!
8) prove t/db_dependent/SIP/Message.t

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Rebased-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-04 16:40:18 -04:00
6a0955946e
Bug 30843: Add mfa_range configuration option for TOTP
This change adds a mfa_range configuration option for TOTP
to koha-conf.xml, and overrides the "verify" method from
Auth::GoogleAuth in order to provide a new default for "range"

Test plan:
0. Apply the patch
1. koha-plack --restart kohadev
2. Go to
http://localhost:8081/cgi-bin/koha/admin/preferences.pl?op=search&searchfield=TwoFactorAuthentication
3. Change the syspref to "Enable"
4. Go to
http://localhost:8081/cgi-bin/koha/members/moremember.pl?borrowernumber=51
5. Click "More" and "Manage two-factor authentication"
6. Register using an app
7. In an Incognito window, go to
http://localhost:8081/cgi-bin/koha/mainpage.pl
8. Sign in with the "koha" user
9. Note down a code from your Authenticator app
10. Wait until after 60 seconds and try it
11. Note it says "Invalid two-factor code"
12. Try a new code from the app
13. Note that it works

14. Add <mfa_range>10</mfa_range> to /etc/koha/sites/kohadev/koha-conf.xml
15. Clear memcached and koha-plack --restart kohadev
16. Sign in with the "koha" user
17. Note down a code from your Authenticator app
18. Wait 4 minutes and then try it
19. Note that it works

20. Disable your two-factor authentication and click to re-enable it
21. Use a code older than 60 seconds when registering for the two
factor authentication
22. Note that the code works

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-09-25 10:53:51 -03:00
120b991529
Bug 33926: Add ability to specify fields allowed in a response
At this time, we can specify fields to hide in SIP response at the login level. From a security perspective, it would be useful to also be able to specify which fields are allowed in a response.

Test Plan:
1) Apply this patch
2) prove t/db_dependent/SIP/Message.t

Signed-off-by: Sam Lau <samalau@gmail.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-09-25 10:53:50 -03:00
cc175067b5
Bug 32911: (follow-up) Remove partner_code
from debian and /etc koha-conf.xml files

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-09-01 11:43:42 -03:00
Sam Lau
c2b066d4be
Bug 34101: Limit items types that can be checked out via SIP2
This bug adds the ability to define a list of item types that are blocked from being issued at that SIP account

To test:
1) Apply this patch
2) Visit Administration->Item types and select edit on the music item type
3) Make the rental charge 0 and save changes (this allows for the item to be checked out via SIP)
4) In the terminal, vim /etc/koha/sites/kohadev/SIPconfig.xml
5) Edit the term1 account and add the following *inside* the login section:
     blocked_item_types="BK|MU"
     You should have something similar to this: <login id ="term1" ........... checked_in_ok="1" blocked_item_types="BK|MU" />
6) Restart SIP (sudo koha-sip --restart <instancename>)
7) Run a checkout query for an item with the item type book. Here is an example you could use:
    perl misc/sip_cli_emulator.pl -a localhost -p 6001 -su term1 -sp term1 -l CPL --patron 23529001000463 --item 39999000011418-m checkout
8) Notice the checkout failed and you are given the screen msg "Item type cannot be checked out at this checkout location"
9) Run a checkout query for an item with the item type music. Here is an example you could use:
    perl misc/sip_cli_emulator.pl -a localhost -p 6001 -su term1 -sp term1 -l CPL --patron 23529001000463 --item 39999000008715 -m checkout
10) Notice the checkout failed and you are given the screen msg "Item type cannot be checked out at this checkout location"
11) vim /etc/koha/sites/kohadev/SIPconfig.xml and delete the BK from the blocked_item
12) Delete the BK from blocked_item_types. It should now look like :
    blocked_item_types="MU"
13) Restart SIP (sudo koha-sip --restart <instancename>)
14) Run a checkout query for the item with the item type book
    perl misc/sip_cli_emulator.pl -a localhost -p 6001 -su term1 -sp term1 -l CPL --patron 23529001000463 --item 39999000011418    -m checkout
15) Checkout succesful
16) Run a checkout query for the item with the item type music
    perl misc/sip_cli_emulator.pl -a localhost -p 6001 -su term1 -sp term1 -l CPL --patron 23529001000463 --item 39999000008715     -m checkout
17) Still fails (because it is blocked)
18) prove t/db_dependent/SIP/Message.t
19) Congratulate yourself for making it through the long test and sign-off :)

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-07-05 12:17:23 -03:00
12a986e668
Bug 24517: Add missing 6th position to date-entered-on-file index
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Adding the biblio-zebra-indexdefs.xsl on same patch (as should
be generated with xsltproc).
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-06-23 09:46:40 -03:00
fbd4d3a9fe
Bug 33580: Bring back ability to mark item as seen via SIP2 item information request
Prior to Koha 22.05, the SIP2 item information message had a side affect of updating the datelastseen field for items. This bug has been fixed, but was being utilized by inventory tools that used SIP2. We should bring back this affect and formalize it as an optional SIP2 config account setting.

Test Plan:
1) Apply this patch set
2) prove t/db_dependent/SIP/Message.t

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-05-12 11:22:47 -03:00
Katrin Fischer
c39d4ea0b2
Bug 32937: Make Zebra ignore the copyright symbol in searches
It's common to catalog the year with the copyright symbol attached
to it. This creates some issues with search that we can fix by
adding the © to the list of characters Zebra ignores in search.

To test:
* Search for any existing publication year with ©, example: ©1951
* Verify your record is not found
* Add the copyright symbol to a record, verify it's now found with ©,
  but not without
* Apply patch
* On ktd:
  * sudo cp -i /kohadevbox/koha/etc/zebradb/etc/word-phrase-utf.chr  /etc/koha/zebradb/etc/word-phrase-utf.chr
  * sudo koha-zebra --restart kohadev
  * sudo koha-mysql kohadev
  * DELETE FROM biblio WHERE biblionumber = 369;
  * The reindex woudl fail for me with this broken record present.
  * sudo koha-rebuild-zebra -f kohadev
* Repeat searches, the records should be found when searching
  with and without ©.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-05-12 11:22:36 -03:00
0c548e8590
Bug 29185: Facilitate searching for cnum
Instead of typing the case sensitive Control-number each time.
4 strikes instead of 15 on your keyboard. Wow! Gain of 73%.

Test plan:
Copy ccl.properties to /etc/koha/zebradb, restart Zebra and
search for cnum=SOME_ID in opac or intranet.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-05-10 15:26:28 -03:00
154f12d67b
Bug 33277: Adjust source file for zebra mappings
The previous patches adjusted the mappings directly - moving this
change to the correct build file

Not needed for sign off, but QA can test that nothing changes when rebuilding the files:
xsltproc etc/zebradb/xsl/koha-indexdefs-to-zebra.xsl etc/zebradb/marc_defs/marc21/authorities/authority-koha-indexdefs.xml > etc/zebradb/marc_defs/marc21/authorities/authority-zebra-indexdefs.xsl

Signed-off-by: Frank Hansen <frank.hansen@ub.lu.se>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-05-10 15:26:25 -03:00
7911eb6666
Bug 33277: Search thesaurus for 'notspecified'
Signed-off-by: Frank Hansen <frank.hansen@ub.lu.se>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-05-10 15:26:24 -03:00
d314d53724
Bug 33360: Update koha-conf.xml
Test plan:
We already tested it. Just look at changes in this patch.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-04-14 11:35:33 -03:00
Galen Charlton
a29964778a
Bug 32716: larger proxy_buffer_size in NGINX example config
This patch updates the example NGINX config to increase the
proxy_buffer_size to 16k. The default value of 4k (on some platforms)
has empirically been shown to be a bit too small for the Link
headers emitted by the REST API when pagination is requested.

To test
-------
[1] Set up a Koha system with NGINX as a reverse proxy in
    front of it (either in front of Apache or in front of
    of Starman).
[2] Perform a patron search that returns at least two pages
    of results and navigate to the second page.
[3] Note that the navigation can fail with a 502 HTTP error
    and an "upstream sent too big header while reading response
    header from upstream" error in the NGINX log.

    The problem is most likely when the pagesize of the server
    running NGINX is 4096 bytes.
[4] Update the NGINX configuration per this patch and restart
    NGINX.
[5] This time, repeating step 2 should work.

Signed-off-by: Galen Charlton <gmc@equinoxOLI.org>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-04-14 11:35:18 -03:00
2799cac081
Bug 21548: ILLLIBS patron category / partner_code seems extraneous
This patch updates the default partner category used by the partner_code config to be in line with sample data in sample_patrons.yml

Preparation:
Apply patch
Enable ILLModule sys pref
Install an ILL backend (e.g. FreeForm)
Add this change to your koha-conf.xml
Flush, restart.
Search for patron of category inter-library loan and assign a primary e-mail address to it

Test plan:
Create an ILL request and click 'place request with partners'
Verify that the 'select partner libraries' has the correct patron of IL category

Run tests and ensure they pass:
prove t/db_dependent/Illrequest/Config.t
prove t/Koha/Config.t

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-03-31 12:14:51 +02:00
877f8ed898
Bug 32684: Implement SIP patron status field "too many items lost"
The SIP patron status and information responses always return false foe "too many items lost". It would be reasonable to check the count of lost items still checked out to the patron and compare that to a threshold set in the sip config file. Though not all libraries operate in this way, it seems like a good and reasonable implementation as long is it is properly documented.

This patch adds the ability to set the SIP "too many items lost" flag
for a patron based on the number of lost checkouts the patron has where
the lost flag on those items is greater than the given flag value.

For example, one could specify that the flag be set if the patron has
more than 2 items checked out where itemlost is greater than 3.

By default the feature is disabled to retain the existing functionality.
If enabled, the default itemlost minimum flag value is 1 unless
specified.

Test Plan:
1) Apply this patch
2) prove t/db_dependent/SIP/Message.t

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-03-14 08:49:29 -03:00
311d68815f Bug 32558: Add ability for background_jobs_worker.pl to process multiple jobs simultaneously up to a limit
Right now background_jobs_worker.pl only processes jobs in serial. It would make sense to handle jobs in parallel up to a user definable limit.

Test Plan:
1) Apply this patch
2) Stop background_jobs_worker.pl
3) Generate some background jobs by editing records, placing holds, etc
4) Watch processes in a new terminal: watch -n 0.1 'ps aux | grep background_jobs_worker.pl'
5) Run background_jobs_worker.pl with parameter -m 3 or some other
   number of max processes
6) Note the multiple forked processes in the ps output

Test notes - also tested the following on KTD:
1. Stop background_jobs_worker.pl
2. Edit /etc/koha/sites/kohadev/koha-conf.xml - set max_processes to 10
3. Generate some background jobs
4. Watch processes in a new terminal: watch -n 0.1 'ps aux | grep background_jobs_worker.pl'
5. Restart all
6. Confirm multiple forked processes in the ps output

Both methods work as expected and generate multiple forked processes
based on the value set for max processes.

Signed-off-by: emlam <emily.lamancusa@montgomerycountymd.gov>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-03-06 13:57:53 -03:00
4c51596a4b
Bug 32992: Move background_jobs_worker to misc/workers
On bug 32594 we are adding a new worker, dedicated to Elastic indexing.
We should have a common place for workers, and we agreed on misc/workers

To test:
1 - Apply patch
2 - reset_all in koha testing docker
3 - ps aux | grep background
4 - Confirm the workers are running, and running in the new directory
5 - Perform a batch item modification
6 - Ensure the job is processed by the worker

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-02-24 17:52:19 -03:00
09e0f8e30d
Bug 32612: (QA follow-up) Remove superfluous line
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-02-03 10:30:03 -03:00
3b1f2c57ea
Bug 32612: Add worker-error to log4perl conf
To test:
1 - Apply patch
2 - vim /etc/koha/sites/kohadev/log4perl.conf, Add lines below:
log4perl.logger.worker = WARN, WORKER
log4perl.appender.WORKER=Log::Log4perl::Appender::Screen
log4perl.appender.WORKER.stderr=1
log4perl.appender.WORKER.mode=append
log4perl.appender.WORKER.layout=PatternLayout
log4perl.appender.WORKER.layout.ConversionPattern=[%d] [%p] %m %l%n
log4perl.appender.WORKER.utf8=1
3 - Restart all
4 - Edit misc/background_jobs_worker.pl
-        my $job = Koha::BackgroundJobs->find($args->{job_id});
+        my $job;# = Koha::BackgroundJobs->find($args->{job_id});
5 - In another terminal: tail -f /var/log/koha/kohadev/koha-worker-error.log
6 - Force enqueue a job (that won't be found because of #4
  perl -e 'use Koha::BackgroundJob::BatchUpdateItem; my $bg = Koha::BackgroundJob::BatchUpdateItem->new(); $bg->enqueue({ record_ids=>['888888']});'
7 - Note error in log like:
  [2023/01/11 19:26:10] [WARN] No job found for id=2983 main:: /kohadevbox/koha/misc/background_jobs_worker.pl (111)

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-02-03 10:30:02 -03:00
a4999a2895 Bug 32741: Fix {biblios|authorities}/bib1.att duplicate codes
This patch fixes a duplicate attribute code for Author-in-order in the
biblios definition.
The picked code matches what was already in ccl.properties.

Also Chronological-term for authorities gets fixed.

To test:
1. Apply the regression tests
2. Run:
  k$ prove xt/verify_bib1.att.t
=> FAIL: Some failiures
3. Apply this patch
4. Repeat 1
=> SUCCESS: Tests now pass!
5. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-02-02 11:59:26 -03:00
Matthias Meusburger
60e6844bf0
Bug 25812: Fines can be displayed on SIP checkin/checkout
Test plan:

 - Enable show_outstanding_amount in SIPconfig.xml

 - Check that the total outstanding amout for the patron is displayed on SIP
   checkout (if it exists), for example:
   Patron has fines - You owe $10.00.

 - Check that the outstanding amout for a given item is displayed on SIP
   checkin (if it exists), for example:
   "You owe $10.00 for this item."

 - Check that it is not displayed when show_outstanding_amount is disabled.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-01-31 10:21:52 -03:00
e88f31d8a2
Bug 32416: Add a warning to the file
Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-01-27 16:08:23 -03:00
e6efa82099
Bug 32416: Update attribute number for arp search to 9015
Bug 20078 updated the attribute for arp to 2014 to avoid conflict with 9013 not-on-loan-count

Bug 28830 then added Control-number-identifier as 2014, breaking arp again

This patch updates the number to 9015

To test:
1 - Apply first patch
2 - Attempt searching by arp, no results (add a unique 526$d to a record to ease searching)
3 - Apply this patch
4 - Copy bib1.att and ccl.properties to the correct locations
    cp etc/zebradb/biblios/etc/bib1.att /etc/koha/zebradb/biblios/etc/bib1.att
    cp etc/zebradb/ccl.properties /etc/koha/zebradb/ccl.properties
5 - Restart zebra
6 - Rebuild indexes
7 - Search again, success!

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-01-27 16:08:19 -03:00
7a1b35c9d0
Bug 32030: Add Apache RewriteRule
This is going to be awesome!

Signed-off-by: Jonathan Field <jonathan.field@ptfs-europe.com>

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-11-08 09:43:48 -03:00
2bc0433b97
Bug 25716: (QA follow-up) Make sure a comment doesn't break tests
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-10-24 18:52:35 -03:00
57ea65e725
Bug 15048: Index all possible searched subfields for index-term-genre
Currently we only index a - but we can setup the system such that avxyz are searched

To test:
 1 - define both a 655$a *and* 655$x value in a bib, save, reindex
 2 - Set system preferences:
      TraceSubjectSubdivisions: Include
      TraceCompleteSubfields: Force
 3 - View the record edited above in the opac
 4 - Click on the subject heading
 5 - No results found
 6 - Copy zebra files:
  sudo cp ./etc/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml \
  /etc/koha/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml
  sudo cp etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl \
  /etc/koha/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl
 7 - restart all and reindex
 8 - Click on the subject heading in OPAC
 9 - Sucess!
10 - Repeat with other fields (vyz)
11 - Repeat under ES, reindexing and resetting mappings

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-10-24 14:39:38 -03:00
2c673944ba
Bug 31532: Add preprocessor to Zebra indexing pipeline to index 880 as linked field
This patch adds a preprocessor XSLT to the Zebra indexing pipeline,
so that 880 fields get indexed as the fields they're linked to. For example,
a "880 $6 245" field would be indexed as a "245" field.

However, because the preprocessor only occurs in the indexing part of the pipeline,
it does not affect the retrieval of MARCXML from Zebra. That MARCXML will be
the same MARCXML that was sent to Zebra from Koha.

Test plan:

0. Revert bug 15187, and apply patch for 31532
1. cp ./etc/zebradb/biblios/etc/dom-config.xml /etc/koha/zebradb/biblios/etc/dom-config.xml
2a. cp etc/zebradb/marc_defs/marc21/biblios/preprocess_marcxml.xsl /etc/koha/zebradb/marc_defs/marc21/biblios/.
2b. cp etc/zebradb/marc_defs/normarc/biblios/preprocess_marcxml.xsl /etc/koha/zebradb/marc_defs/normarc/biblios/.
2c. cp etc/zebradb/marc_defs/unimarc/biblios/preprocess_marcxml.xsl /etc/koha/zebradb/marc_defs/unimarc/biblios/.
3. koha-rebuild-zebra -b -f -v kohadev
4. Note that in search results the 880$6245$a data appears before the 245$a data
5. Note that you can do a title index search on the 880$6245$a data

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-10-24 14:07:46 -03:00
8d5cbcadd5
Bug 25716: (QA follow-up) Move additional options to etc/z3950/config.xml
This followup moves the configuration to the z3950 etc file, either the
default and or the custom file is used as per the existing script code.

In addition, the options and be set using an environment variable named Z3950_ADDITIONAL_OPTS.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-10-24 13:11:27 -03:00
005d67063f
Bug 25716: Add ability to specify additional options in koha-conf.xml for z3950_responder.pl when using koha-z3950-responder
The z39.50 responder has a number of command line options that are not
accessible if using the debian scripts to control it. We should be able
to set those options in the koha conf file to be passed to the script
itself.

Test Plan:
1) Apply this patch
2) Copy your kohaclone's koha-z3950-responder to /usr/sbin/koha-z3950-responder if necessary
3) Add "<z3950_responder_options>--add-item-status k</z3950_responder_options>" inside your <config> block in your koha-conf.xml file
4) Use koha-z3950-responder to start/restart the z39.50 responder, note the item status is now in subfield k!

Signed-off-by: Liz Rea <wizzyrea@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-10-24 13:11:26 -03:00
ef81785d76
Bug 31469: log4perl.conf: Add %n to conversionpattern for Plack
The PLACKOPAC, PLACKAPI and PLACKINTRANET appenders still need %n
(i.e. a newline). Note that this patch does not add %l since it
is a bit confusing because it adds a lot of Plack internal noise like:
    [2022/09/29 08:51:34] [WARN] Test mainpage CGI::Compile::ROOT::usr_share_koha_mainpage_2epl::__ANON__ /usr/share/koha/mainpage.pl (49)

The patch is a result of:
    git grep -l "log4perl.appender.PLACK" | xargs sed -i -e"/ConversionPattern/ s/%m$/%m%n/"

Test plan:
First run: sed -i -e"/ConversionPattern/ s/%m$/%m%n/" /etc/koha/sites/[YOUR_CLONE]/log4perl.conf
Edit that file, change PLACKOPAC to debug level like:
    log4perl.logger.plack-opac = DEBUG, PLACKOPAC
Restart.
Hit an OPAC page twice.
Check plack-opac logfile and verify that it contains a newline between last two messages like:
    [2022/09/29 08:04:30] [DEBUG] kohaversion : 22.0600054
    [2022/09/29 08:04:42] [DEBUG] kohaversion : 22.0600054

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-10-12 14:58:40 -03:00
fdb5275407
Bug 31614: Adds a koha-conf.xml config to set Zebra connection timeout
This patch adds <zebra_connection_timeout>30</zebra_connection_timeout>
to the koha-conf.xml file.

Sometimes, a Zebra search might take longer than 30 seconds. If it does,
Koha will say that 0 records have been found. While slow searching
is not desirable, it's more desirable to get the result set regardless.

Test plan:
0. Apply patch
1. Add <zebra_connection_timeout>.1</zebra_connection_timeout> to
your relevant koha-conf.xml file (e.g. /etc/koha/sites/kohadev/koha-conf.xml)
2. echo 'flush_all' | nc -q 1 memcached 11211
3. koha-plack --restart kohadev
4. Go to http://localhost:8081/cgi-bin/koha/catalogue/search.pl?q=test
5. Note that no results are returned
6. Change zebra_connection_timeout to 30
7. echo 'flush_all' | nc -q 1 memcached 11211
8. koha-plack --restart kohadev
9a. Go to http://localhost:8081/cgi-bin/koha/catalogue/search.pl?q=test
9b. Note that 3 results are returned
10. Remove zebra_connection_timeout from koha-conf.xml
11. echo 'flush_all' | nc -q 1 memcached 11211
12. koha-plack --restart kohadev
13a. Go to http://localhost:8081/cgi-bin/koha/catalogue/search.pl?q=test
13b. Note that 3 results are returned
14. Celebrate

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-10-12 09:10:07 -03:00
14cfdfd727 Bug 31236: Add ability to send custom item fields via SIP using Template Toolkit
Koha has been able to send arbitrary item fields via the "item_field" parameter in the config. We have partners that need the ability to created custom item fields from templates, as the item_fields feature cannot accomplish what they need. We need to add a templated custom field feature for items, similar to what we have for patrons.

Test Plan:
1) Apply this patch
2) Choose a SIP login to use, edit that account and add the following
   *inside* the login section:
   <custom_item_field field="IN" template="[% item.itemnumber %]" />
3) Restart SIP
4) Run an item information query
5) Note the itemnumber is sent in the IN field!

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Edit: fixed tests count
2022-09-12 17:44:36 -03:00
cffd36bcb4
Bug 31250: (QA follow-up) The future will be easier
As requested by Jonathan, we need more flexibility ;)
Here it comes.

Test plan:
Run t/CookieManager.t

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-09-12 15:34:11 -03:00
cf8e560e37
Bug 31250: Add catalogue_editor_ cookie to koha-conf deny list
Test plan:
Add this change to your koha-conf.xml.
Flush, restart.
Test if the cookie is kept now in the interface.

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-09-12 15:34:11 -03:00
4a6567274d
Bug 31033: Explicitly define max child processes for SIP server
By default, the SIP server appears to only use 1 child process for
responding to SIP connections.

This change makes this explicit in the configuration, which should
make it so that people who need more than 1 simultaneous SIP connection
can know to just increase the value for the "max_servers" parameter
in the SIPconfig.xml file.

Test plan:
1. Add "max_servers='1'" to your SIP configuration file
2. koha-sip --restart kohadev
3. Open 3 terminals
4. Run "telnet localhost 6001" on 2 terminals
5. On the 3rd terminal, run the following:
ss -l -n -t
ps -efww | grep "sip"
6. Note that there are 2 processes called
kohadev-koha-sip: perl /kohadevbox/koha/C4/SIP/SIPServer.pm /etc/koha/sites/kohadev/SIPconfig.xml
One of these processes is the parent of the other
7. The Recv-Q in the "ss" output should show 1
(This means that 1 of your telnet connections is in the server's TCP backlog)
8. Celebrate as the configuration works as expected

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-08-25 08:53:36 -03:00
5e29433bdc
Bug 31276: Report results are limited to 999,999 no matter the actual number of results
In Koha, any report that uses C4::Reports::Guided will be limited to 999,999 rows. This is causing problems for larger libraries where some reports may have over a million results.

Test Plan:
1) Create a report "SELECT * FROM borrowers" and run it, note the number
   of results
2) Apply this patch
3) Add the line `<report_results_limit>3</report_results_limit>`
   within the <config> block of your koha-conf.xml
4) Restart all the things!
5) Run the report, download the results as a CSV
6) Note your CSV only has 4 lines, the header and 3 patrons

Signed-off-by: Rachael Laritz <rachael.laritz@inlibro.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-08-16 09:14:18 -03:00
00f489de6d
Bug 31296: Add ability to disable demagnetizing items via SIP2 based on itemtypes
Some libraries have certain item types that can only do in house checkouts via SIP self check machines. In these cases, the items should not be demagnetized since the items cannot leave the library.

Test Plan:
1) Apply this patch
2) prove t/db_dependent/SIP/Message.t

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-08-08 13:13:26 -03:00
fbf1c32a6f
Bug 30879: Handle index/sorting for UNIMARC
Same as before, but test with UNIMARC setup

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-08-02 09:33:13 -03:00