Commit graph

2607 commits

Author SHA1 Message Date
Andrew Isherwood
8595e80b78 Bug 24083: Add support for unseen_renewals
This patch adds support for unseen renewals.

Here we retrofit knowledge of unseen renewals and add the display of unseen
renewal counts and warnings, in addition to adding the ability to
specify a renewal as being "unseen".

The functionality added here is goverened by the UnseenRenewals syspref.

Signed-off-by: Sally Healey <sally.Healey@cheshirewestandchester.gov.uk>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-11-11 16:09:58 +01:00
f5017e4ed6 Bug 25067: Fix perlcritic errors
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-11-06 17:06:44 +01:00
Julian Maurice
3cfc2ec7bd Bug 25067: Move PO file manipulation code into gulp tasks
misc/translator/translate was doing three different things:
- extract translatable strings
- create or update PO files
- install translated templates

This patch separates responsibilities by moving the string extraction
code into several 'xgettext-like' scripts and adds gulp tasks to
automate string extraction and PO files update

This has several benefits:

- gulp runs tasks in parallel, so it's a lot faster (updating all PO
  files is at least 10 times faster with my 4-cores CPU)

- there is no need for $KOHA_CONF to be defined
  LangInstaller.pm relied on $KOHA_CONF to get the different paths
  needed. I'm not sure why, since string extraction and PO update should
  work on source files, not installed files

- string extraction code can be more easily tested

This patch also brings a couple of fixes and improvements:

- TT string extraction (strings wrapped in [% t(...) %]) was done with
  Template::Parser and PPI, which was extremely slow, and had some
  problems (see bug 24797).
  This is now done with Locale::XGettext::TT2 (new dependency) which is
  a lot faster, and fixes bug 24797

- Fix header in 4 PO files

For backward compatibility, 'create' and 'update' commands of
misc/translator/translate can still be used and will execute the
corresponding gulp task

Test plan:
1. Run `yarn install` and install Locale::XGettext::TT2
2. Run `gulp po:update`
3. Verify the contents of updated PO files
4. Run `cd misc/translator && ./translate install <lang>`
5. Verify that all (templates, sysprefs, xslt, installer files) is
   correctly translated
6. Run `gulp po:create --lang <lang>` and verify that it created all PO
   files for that language
7. Run `prove t/misc/translator`

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Need to install yarn & gulp, no errors

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-11-06 09:46:11 +01:00
b0ce0d6463 Bug 14708: Enforce restriction in delete_patrons.pl
Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-11-06 09:40:25 +01:00
4dd798302a Bug 26641: Optimize CatalogModuleRelink lookup
This patch moves the CatalogModuleRelink lookup outside the loop to
improve performance (especially for large databases), and includes a
tester friendly test plan.

Test plan:
0. Using koha-testing-docker
1. koha-shell -c "./misc/link_bibs_to_authorities.pl -l --test" kohadev
2. Note 436 bibs checked, 1315 headings linked, 676 headings unlinked
3. koha-shell -c "./misc/link_bibs_to_authorities.pl -l --test -g 700" kohadev
4. Note 436 bibs checked, 248 headings linked, 111 headings unlinked
5. Note output appear to all be names (rather than subjects or other authorities)
6. Note also that the 2nd run with -g 700 is almost 2x faster than without

Signed-off-by: Andreas Roussos <a.roussos@dataly.gr>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-11-04 12:59:34 +01:00
Andreas Roussos
ab2e3b0e89 Bug 26641: pass the MARC field as an argument to link_bibs_to_authorities.pl
When running link_bibs_to_authorities.pl, it could be useful to have
the ability to specify which MARC field the script should operate on.

For example, when you have a catalog where most links are in place,
and you don't want the script to process every MARC field but rather
limit it to a specific MARC field (e.g. 700) because you've observed
that there are many missing links for that particular MARC field.

This patch provides that enhancement.

Test plan:
1) Run the script as follows (preferably in a test DB):

   time link_bibs_to_authorities.pl -v -l --test

   ... and notice how long it takes to complete.

2) Apply the patch.

3) Run the script again, this time providing the MARC field to work on:

   time link_bibs_to_authorities.pl -v -l --test -g=700

   ... and, again, notice how long it takes to complete.

   The 2nd run should run faster than the 1st one.

Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-11-04 12:59:34 +01:00
e42b4088ed Bug 26180: Add descending option to rebuild_elasticsearch.pl
While the ES index is incremental and provides results as it commits, we currently index from the oldest records to the newest.

This patch provides the option to go the other direction

To test:
 1 - Have ES setup and running for Koha
 2 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b
 3 - Note the biblios index from number 1 the end
 4 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a
 5 - Notice the same
 6 - Apply patch
 7 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b
 8 - Still in ascending order
 9 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b --desc
10 - Now records index in descending order
11 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a
12 - Still ascending
13 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a --desc
14 - Now descending

Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

JD amended patch: fix typo "inde" vs "index" and add commit body

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-11-04 12:59:33 +01:00
9a11366382 Bug 12656: Allow --reason to be passed to cancel_expired_holds
This patch adds the --reason option to cancel_expired_holds which allows
the library to optionally set a reason for cancellation when running the
cronjob. This will prompt the HOLD_CANCELLED notice to be sent to the
patron.

To test:
1/ Ensure the unit tests continue to pass after the patch
   (t/db_dependent/Reserves/CancelExpiredReserves.t)

Also:
1 - Add an expired hold for a patron:
    INSERT INTO RESERVES (borrowernumber, biblionumber, expirationdate, found,branchcode,itemnumber) VALUES (5,5,'2020-01-01','W','CPL',983);
2 - Set ExpireReservesMaxPickUpDelay to Allow
3 - Run the cronjob:
    perl misc/cronjobs/hold/cancel_expired_holds.pl --reason EXPIRED
4 - Visit the patron's notices tab
5 - Confirm they have been sent a cancellation notice

Signed-off-by: Lisette Scheer <lisettes@latahlibrary.org>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-11-04 12:59:33 +01:00
ebbee18822 Bug 25333: Change message transport type for Talking Tech from "phone" to "itiva"
From its inception, phone notices via Talking Tech have not behaved like
other notices. Instead of reading notices generated by Koha, the Talking
Tech scripts largely generate their own notices.

We would like to pave the way to having "generic" phone notices that can
be processed by plugins to support arbitrary telephony vendors ( we will
be targeting Twilio initially ).

To that end, it seems sensible to begin by changing the messaage
transport type for Talking Tech from 'phone' to 'itiva' to
highlight its specificity and difference from standard message
transports.

Test Plan:
1) Apply this patch
2) Run updatedatabase.pl
3) Restart all the things!
4) Test Talking Tech outbound script
5) Note no changes in functionality

Signed-off-by: Christopher Zorn <Christofer.Zorn@ajaxlibrary.ca>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

JD amended patch: remove uneeded indentation change in sample_notices_message_transports.sql

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-11-04 12:59:32 +01:00
9441537e54 Bug 26832: Make Elasticsearch mappings export use UTF-8
Script misc/search_tools/export_elasticsearch_mappings.pl allows to export current search engine configuration into a YAML file.
This export should use UTF-8 encoding, like other exports.

Test plan :
1) Go to Administration > Search engine configuration (Elasticsearch)
2) Edit a field label to use a diacrtic, for example local-number => Numéro
3) Save
4) Edit file etc/koha-conf.xml to enable 'elasticsearch_index_mappings'
5) Export mappings to file via misc/search_tools/export_elasticsearch_mappings.pl -t $MARCFLAVOUR
6) Reset memcached and plack
7) Back to Administration > Search engine configuration (Elasticsearch)
8) Click on 'Reset Mappings' and accept
9) Look at field 'local-number'
=> Without patch diacritic 'é' is broken
10) You may try with an emoji B-)

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-11-04 12:59:32 +01:00
Mark Hofstetter
28d20b92be Bug 26601: Add utf8 encoding to text output of overdue_notices.pl
This patch sets the UTF8 encoding for output of overdue_notices.pl when
outputting to txt.

To test:
0/ Check out an item with due date yesterday (use specify due date)
1/ Create ODUE text with utf8 encoded characters, chinese characters, umlauts etc
   <url>/cgi-bin/koha/tools/letter.pl?op=add_form&module=circulation&code=ODUE
2/ Run
   perl overdue_notices.pl -text ./
3/ Verify encoding in the generated text file is incorrect
4/ Apply patch and repeat 2/
3/ Confirm text now inludes properly encoded characters

Sponsored-by: Styrian State Library
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
With German umlauts on my system the editor still showed the
characters correctly (probably guessing the right encoding).
Using Chinese made the problem visible.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-10-26 00:14:41 +01:00
0eae966d25 Bug 25549: Remove plugin methods for broken plugins
To test:
 1 - Enable plugins in the koha-conf
 2 - Install the kitchen sink plugin
 3 - Your staff client should be orange now :-)
 4 - edit the plugin module
     /var/lib/koha/kohadev/plugins/Koha/Plugin/Com/ByWaterSolutions/KitchenSink.pm
 5 - Add a line to break compilation, like:
     this won't compile
 6 - Restart all
 7 - Your koha is now broken
 8 - kshell
 9 - perl misc/devel/install_plugins.pl
10 - Restart all
11 - Koha remains broken
12 - Apply patch
13 - kshell
14 - perl misc/devel/install_plugins.pl
15 - Koha now works!
16 - Koha is not orange because the plugin methods are removed

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-10-22 10:04:30 +02:00
Julian Maurice
96cc447045 Bug 25898: Prohibit indirect object notation
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-10-15 12:56:30 +02:00
5b5083bc7b Bug 22417: Process the jobs even if the message broker is not reachable
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-10-05 15:26:36 +02:00
ee4b07f18b Bug 22417: Send message to /queue
This cannot be:
  /queue/$namespace/$job_type

I got: '/koha_kohadev/batch_biblio_record_modification' is not a valid queue destination

So keeping the dash in $namespace-$job_type

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: David Cook <dcook@prosentient.com.au>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-10-05 15:23:08 +02:00
bd385809cd Bug 22417: Restore the concept of namespace
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: David Cook <dcook@prosentient.com.au>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-10-05 15:22:44 +02:00
42b3529424 Bug 22417: Fix the batch authority tool
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: David Cook <dcook@prosentient.com.au>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Bug 22417: Add a note about the existence of the DB row

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: David Cook <dcook@prosentient.com.au>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Bug 22417: (follow-up) Fix the batch authority tool

Signed-off-by: David Cook <dcook@prosentient.com.au>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-10-05 15:22:44 +02:00
9251623b8d Bug 22417: Switch to STOMP
apt install rabbitmq-server
service rabbitmq-server start
rabbitmq-plugins enable rabbitmq_stomp
apt install libnet-stomp-perl
cp debian/scripts/koha-functions.sh /usr/share/koha/bin/koha-functions.sh
cp debian/scripts/koha-worker /usr/bin/
koha-worker --start kohadev
tail -f /var/log/rabbitmq/*

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: David Cook <dcook@prosentient.com.au>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Bug 22417: Remove useless use statement

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-10-05 15:22:44 +02:00
1a6c56f949 Bug 22417: Add debian script koha-worker
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: David Cook <dcook@prosentient.com.au>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-10-05 15:22:44 +02:00
c4ebc20aa9 Bug 22343: Fix runreport encoding issues
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-10-02 15:23:52 +02:00
2577f1f683 Bug 22343: Adapt runreport.pl
This script gets adapted to use the new Koha::Email syntax and SMTP
transport.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-10-02 10:54:40 +02:00
59ef597f6e Bug 22343: Adapt controller scripts
This patch adapts controller scripts that used sendmail. Also the syntax
for Koha::Email has changed and this patch adapts it.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-10-02 10:54:40 +02:00
aa8ee99fb7 Bug 25624: Add --where option to update_patrons_category.pl
The script did not allow to find empty fields or use wildcards

With this new option we will now have the ability to filter patrons by
some of their attributes.

Test plan:
1 - Run the script with no parameters and verify the help explains the parameters
2 - Try the script with one or more --where parameters, like:
  --where "firstname='koha'"
3 - Test null values
  --where "firstname IS NULL"
4 - Test like values with wildcards
  --where "firstname LIKE '%a%'
5 - Test like with the word null to find fields containing the word rather than being unset

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-09-28 16:09:10 +02:00
ff08e99965 Bug 25758: Return on_reserve over too_soon when not calling from automatic_renewals cron
Bug 19014 altered CanBookBeRenewed to return (auto_)too_soon over on_reserve

For cron purposes this is the correct behaviour.

For display purposes we wish to see on_reserve over too_soon

This patchset adds a switch to 'CanBookBeRenewed' to alter the priority of these statuses

To test:
 1 - set NoRenewalBeforePrecision to date only
 2 - set a circ rule to auto-renewal=yes, no renewal before=0, checkout period to 7 days
 3 - check item out
 4 - confirm item shows Scheduled For Automatic Renewal in issues table
 5 - place a hold on the item for another patron
 6 - reload issues table for patron 1, confirm checkout still shows "scheduled for automatic renewal" rather than "on hold"
 7 - change No Renewal Before value to 7
 8 - reload issues table for patron 1, confirm checkout now shows "on hold"
 9 - Apply patch
10 - restart_all
11 - Reload the issues table - confirm 'on_hold' still shows
12 - Change No Renewal Before to 0
13 - Refresh issues table, still shows 'On hold'
14 - perl misc/cronjobs/automatic_renewals.pl -v
15 - Result shows 'auto_too_soon'
16 - prove -v t/db_dependent/Circulation.t

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-09-28 11:08:03 +02:00
9622561636 Bug 26231: Remove incorrect use of AddAuthority() when 001 present
This patch removes the use of AddAuthority with a non-empty
authid argument, since that triggers an update rather than an insert.

In practice, the update also fails, but the error isn't raised,
as the database connection doesn't have RaiseError set.

Test plan:
1) Do not apply patch
2) Try to bulkmarcimport an authority file with a 001
3) Observe that the script reports success but no authority is added
4) Apply the path
5) Try to bulkmarcimport an authority file with a 001
6) Observe that the script reports success and the authority is added

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-09-28 10:09:52 +02:00
Andreas Roussos
642c29dc23 Bug 26451: fix typo in bulkmarcimport.pl
bulkmarcimport.pl displays a small typo when a MARC modification
template is being used: "modofication".

This patch fixes that.

Test plan: apply the patch, and confirm the script no longer
displays the typo.

Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-09-18 11:49:29 +02:00
8e397330f6 Bug 26420: Use translated notices for overdue_notices.pl
This script does not pass the patron's preferred lang for notices.

Test plan:
Enable TranslateNotices
Create a patron with a preferred lang for notices != default
Translate ODUE for this language
Setup the overdue notices for this patron's category
Check an item out for this patron (select the correct due date to
trigger the notice)
Run the script overdue_notices.pl
Check the message_queue table and confirm that the notice enqueued is
translated

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-09-18 10:38:05 +02:00
15174e7bd3 Bug 26407: fix query in 'title exists' in search_for_data_inconsistencies.pl
Bug 23871 added check for 'title exists' to `search_for_data_inconsistencies.pl`.
There is a typo in DBIx query to create or : hashref instead of arrayref.

Test plan :
1) Change title of biblio record XXX in SQL :
   UPDATE biblio SET title = '' WHERE biblionumber=XXX
2) Change title of biblio record YYY in SQL :
   UPDATE biblio SET title = NULL WHERE biblionumber=YYY
3) Run misc/maintenance/search_for_data_inconsistencies.pl
4) Check you see both records

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-09-18 10:37:56 +02:00
f1f9c6dc74 Bug 26384: Fix executable flags
.pm must not have -x
.t must have -x
.pl must have -x

Test plan:
Apply only the first patch, run the tests and confirm that the failures
make sense
Apply this patch and confirm that the test now returns green

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-09-11 09:56:56 +02:00
Agustin Moyano
52deed335e Bug 23816: (follow-up) Fix many things
This patch:
 * reverts changes on misc/admin/set_password.pl
 * makes category param mandatory for AuthUtils::is_valid_password and AuthUtils::generate_password
 * changes onboarding.pl to set category param in AuthUtils::is_valid_password
 * Completes t/db_dependent/AuthUtils.t and drops t/AuthUtils.t
 * Removes offending <input type="number"/> and replaces it by <input type="text" inputmode="numeric" pattern="[0-9]*"/>

https://bugs.koha-community.org/show_bug.cgi?id=23826

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-09-09 15:39:52 +02:00
Agustin Moyano
5848da810e Bug 23816: Add minimum password length and require strong password overrides by category
This patch adds the capability to override minPasswordLenth and RequireStrongPassword settings by category

To test:
1. koha-shell kohadev
2. koha-mysql kohadev

3. drop database koha_kohadev;
4. create database koha_kohadev;

5. go to admin page and start webinstaller. There continue the steps until onboarding.
6. reach step 3 of onboarding and create a new administrator patron
CHECH => Password control woks as normal (Minimum length 3 and strong required)

7. finish Koha installation and enter admin with your new administrator
8. set minPasswordLength to 3 and RequireStrongPassword to “Don’t require”
9. Create a new category (CAT2 from now on.. CAT1 is the category you made in onboarding process) and set minimum password length to 8 and require strong password
10. Create two new patrons, one with CAT1(patron1) and one with CAT2 (patron2)
CHECK => In both cases, try different combinations of length and strength. For patron1 the only requirement is to have 3 letters, but for patron2 the minimum length will be 8 and will require strong password.
CHECK => Try changing patron category before saving. Password requirements will change with category change.

11. Edit CAT1 and set minimum password length to 5
12. Go to patron1 details page, and change password.
CHECH => Now password minimum length is 5, but still it doesn’t require strong password

13. Edit CAT1, leave blank minimum password length and set require strong password to yes.
14. Go to patron1 details page, and change password.
CHECH => Password minimum length is back to 3, but now strong password is required

15. Set minimum password length in CAT2 to 12.
16. Go to patron2 details page, and click to fill a random generated password
CHECK => generated password should be 12 characters length

17. Set PatronSelfRegistration to Allow in admin settings
18. Go to OPAC and fill self registration from.
CHECK => Play with patron category. For each change in category, password requirements are modified.
CHECK => Set CAT1 as patron category, set ‘aA1’ as password (or another valid password for CAT1) and before hitting submit button, change to CAT2. Form should enter invalid state, and CAT2 password requirements should be displayed as error in password input.

19. Create a patron for CAT1 and another for CAT2, leaving password blank
CHECK => For CAT1’s patron, generated password length is 8 (minimum length for generated passwords), but for CAT2’s patron should be 12

20. In admin set PatronSelfRegistrationVerifyByEmail to require
21. Fill self registration form again with CAT2 as category
CHECK => Password requirements works as previous case.
22. Leave password blank and click submit

23. select * from message_queue;
24. Copy the link in the message and paste it in OPAC
CHECH => Generated password is 12 characters long. (Copy user id for next steps)

25. In admin set OpacResetPassword to Allow
26. Go back to OPAC, reload and click on “Forgot password?” link
27. Paste user id and click submit
28. Repeat steps 23 and 24
CHECK => Info message says “Your password must contain at least 12 characters, including UPPERCASE, lowercase and numbers.”
CHECK => enter an invalid password and you’ll get the same message in warning.

29. Login OPAC with the last user and your newly created password
30. Go to “Change your password” option
CHECK => Info message says “Your password must contain at least 12 characters, including UPPERCASE, lowercase and numbers.”
CHECK => enter an invalid password and you’ll get the same message in below “New password” input.

31. prove t/db_dependent/AuthUtils.t t/db_dependent/Koha/Patron/Category.t

32. Sign off

Sponsored-by: Northeast Kansas Library - NEKLS

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-09-09 15:39:52 +02:00
e8a08cdfa7 Bug 23682: Fix use Koha::Plugins::Handler statements
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-09-03 15:00:48 +02:00
85405a2143 Bug 23682: Dedup plugin calls my moving to a single call in process_invoice()
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-09-03 15:00:48 +02:00
d8696702ce Bug 23682: Add ability to manually import EDI invoices as an alternative to automatic importing on download
Some library would like to delay the importing of invoices until
a time of their choosing. The invoices should be imported into
the database as they do now, but the invoice processing should
be skipped. Instead, any invoice file with a status of 'new'
should have an 'Import' button to process the invoice.

Test Plan:
1) Apply this patch
2) Run updatedatabase.pl
3) Enable the new syspref EdifactInvoiceImport
4) Run the edi cronjob to import a new invoice file
5) View EDI messages table at /acqui/edifactmsgs.pl
6) Note the invoice files is not processes, and retains the status of 'new'
7) Use the 'import' button to process the invoice
8) Note the invoice is now marked 'received' and the 'import' button is gone
9) Verify the invoice was actually processes

Signed-off-by: Debi Stears <DDStears@washoecounty.us>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-09-03 15:00:48 +02:00
72deefe8cf Bug 24197: (QA follow-up) Shorten syspref name to AddressForFailedEmailNotices
If you define the address we use it, if not we fallback,
it's not really a redirect, and that just makes the name longer.

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-09-03 14:18:23 +02:00
b48cdd227f Bug 24197: Added new local use syspref which sets the email address that failed overdue notices are set to.
Test plan:
1. Set no email addresses in patron record then checkout
items to the patron making the due date in the past

2. Visit patron's home library page: Administration > Libraries
Set a branch email and in global system preferences observe there is no syspref
named RedirectAddressForFailedOverdueNotices

3. Manually run overdue_notices.pl

4. Check the message_queue database table and observe there is a
print overdue notice and a email notice with to_address of the
branch email address

5. Apply patch

6. Run database update:
cd installer/data/mysql
sudo koha-shell <instance_name>
./updatedb.pl

7. Confirm there is a new system preference named:
RedirectAddressForFailedOverdueNotices

Give it a different email address to that in the branch email.

7. Repeat steps 1,3,4 and observe that the failed overdue notices have been
sent to the email defined in RedirectAddressForFailedOverdueNotices

Sponsored-By: Catalyst IT
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-09-03 14:18:23 +02:00
Katrin Fischer
ed066caf58 Bug 26015: Terminology: Fix tons more staff clients to staff interfaces
This takes care of more occurences of staff client and changes it to
staff interface, including in code comments.

To test:
- I think in this case careful code review is what we look for.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-08-31 16:10:25 +02:00
4d9ab04a93 Bug 26175: Remove warn if undefined barcode in misc/export_records.pl
When using misc/export_records.pl --deleted_barcodes if barcode is NULL you get warn :
Use of uninitialized value in say at /home/koha/src/misc/export_records.pl

Also exporting barcode if empty string makes no sense.

Test plan :
1) Delete an item with barcode = NULL
2) Run : misc/export_records.pl --date=`date +%d/%m/%Y` --deleted_barcodes --filename=/tmp/deleted_barcodes
3) Check you se no warn "Use of uninitialized value in say ..."
4) Delete an item with barcode = ''
5) Run 2)
6) Check there is no empty line in /tmp/deleted_barcodes

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-08-26 15:21:41 +02:00
66382458dc Bug 25958: (QA follow-up) Implement filter in database query instead of in loop
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-08-25 12:01:43 +02:00
48edb29471 Bug 25958: Allow LongOverdue cron to exclude specified lost values
When lost items are not marked as returned, they are still subject to
the long overdue cron, so an item that has already been marked Lost may
automatically roll to Long Overdue. In some cases, a library may not
want that lost value to change. This becomes especially important with
the introduction of Claims Returned, which uses a variety of lost.

Test Plan:
1) Set up a long overdue item that would be moved to lost by longoverdue.pl
2) Run the cronjob with the new --skip-lost-value option
3) Note the item is not altered
4) Include that value in the new system preference DefaultLongOverdueSkipLostStatuses
5) Run the cronjob *without* the new option
6) Note the item is not altered
7) Run the cronjob again with the new command line option, but set it to
a different value so the item will be affected
8) Note the item is altered as it would have been before this patch was
applied

Signed-off-by: Lisette Scheer <lisetteslatah@gmail.com>

Signed-off-by: Rebecca Coert <rcoert@arlingtonva.us>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-08-25 12:01:43 +02:00
Julian Maurice
df0ef02de5 Bug 24591: Add --help option to misc/devel/get-prepared-letter.pl
Also fix code style with perltidy

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-08-07 16:54:40 +02:00
Julian Maurice
f74ab4e9de Bug 24591: Add developer script to preview a letter
The script is very simple, it just calls GetPreparedLetter with
arguments given on command line and print the resulting letter content

Usage example:

misc/devel/get-prepared-letter.pl --module circulation \
    --letter_code ODUE --tables '{"borrowers":1,"branches":"CPL"}' \
    --repeat '{"item":[{"biblio":1,"items":1}]}' \
    --loops '{"overdues":[1]}'

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Works for the example and other cases.
Correct option is 'letter-code', with dash, not underscore.
An usage message would be nice.
No errors

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

JD amended patch: tidy the new file and rename it matching the other
scripts' names in this directory

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-08-07 16:54:40 +02:00
Katrin Fischer
e7661a37e6 Bug 25853: Fix file permissions for update_patrons_category script
The permissions for this were -rw-r--r-- and need to be -rwxr-xr-x.

To test:
- Verify the file permissions before and after applying the patch.

Signed-off-by: Didier Gautheron <didier.gautheron@biblibre.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-08-07 16:54:40 +02:00
c7ae33f60c Bug 25683: Fix grouping of results
DBIX doesn't seem to work as I expected - if trying to group by we can only
get the columns we grouped by, otherwise the queries are much more complicated
and come out wrong

This patch removes the 'amountoutstanding' from the query. Note that we do return patron
object, however, to access all the columns we must discard_changes (or refetch)

Also fixes a typo 'verbosse'

To test:
1 - charge a $1 fine to a patron
2 - pay off the fine
3 - perl misc/cronjobs/update_patrons_category.pl -f J -t J -fu=5 -v
4 - Note the patron is returned twice
5 - export DBIC_TRACE=1
6 - repeat 3 - view the SQL query and see how odd it is
7 - Apply this patch
8 - repeat 3 - simpler query
9 - patron returned only once

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-08-06 11:37:07 +02:00
e14c1702b4 Bug 23696: take deleted biblios into account when building oai sets
Test plan:
0) do not apply the patch
1) set up OAI and create at least one set
2) run build_oai_sets.pl
3) remember/write down number of records added
4) delete a biblio, the is included in set
5) run buid_oai_sets.pl again
6) the set is 1 record smaller
7) apply the patch
8) run build_oai_sets.pl
9) the number of record should be the same as in 3)

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-07-24 14:09:30 +02:00
005219ea6f Bug 25955: Fix getting index_name
To test:
1 - Have a site running ES
2 - perl misc/maintenance/compare_es_to_db.pl
3 - Lots of output, all bad
4 - Apply patch
5 - Repeate
6 - Whew, looks correct

Signed-off-by: Didier Gautheron <didier.gautheron@biblibre.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-07-22 10:17:53 +02:00
0c0666c531 Bug 25511: Add POD for --force
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-07-20 17:45:31 +02:00
34666e2690 Bug 25511: Add --force option to update_dbix_class_files.pl
This adds a --force option to the aforementioned script allowing those
in the know to overwrite the schema when changes above the found are
detected.

Signed-off-by: Ere Maijala <ere.maijala@helsinki.fi>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-07-20 17:45:31 +02:00
Julian Maurice
73ff914d5d Bug 25922: Fix typo 'arial-label' -> 'aria-label'
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-07-20 17:45:31 +02:00
280f1bdeb3 Bug 25922: Add aria-label to the allowlist for translation
This patch adds aria-label to the allowlist for translations.

To test:
- Apply patches from here and bug 25244
- kshell
- cd misc/translator
- perl translate update de-DE
- Check for the aria-label in the diff
- Verify the strings appear in the po files now
- Translate them
- perl translate install de-DE
- Verify the translated strings appear in the template

Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-07-20 17:45:31 +02:00
1625519b9e Bug 23148: Replace Bridge icons with transparent PNG files
This patch adds SVG versions of the Bridge item type icon set to
misc/interface_customization/itemtypeimg/bridge. It adds some
new icons, and updates some existing icons. All GIF versions of the
icons are replaced with transparent PNG files.

Icon changes:

- English text has been removed from all icons
- New icon for "Periodical," to differentiate from "Journal"
- New icon, "Controller" for video games
- New icon for "Laserdisk" to differentiate from "DVD"
- New icon for "Kit" to differentiate from "Archive"
- New icon, "Reference books."
- Updated "badges" on "Juvenile book" and "Reserve book" to try to
  express the information without English-specific letters.

Before applying the patch, update some item types or collection codes
with Bridge icons. Apply the patch and run the database update process.

 - Open the advanced search page in the staff client and confirm that
   your item type or collection code icons were updated correctly.
 - Go to Administration -> Item types and edit an item type.
   - Check the "bridge" tab and confirm that the icon options look
     correct.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-07-20 16:16:37 +02:00
b715de657d Bug 24153: (QA follow-up) Removing one redundant initialization line
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-07-20 16:08:22 +02:00
60038291e1 Bug 24153: Add the confirm flag to the crontab calls
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-07-20 16:08:22 +02:00
3d24d4de44 Bug 24153: Make sure $count will not be reused for another purpose
The $count variable should be defined per block and set to 0 to avoid
confusion.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-07-20 16:08:22 +02:00
1a1e6d2163 Bug 24153: Improve verbose messages in dry-run mode
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-07-20 16:08:22 +02:00
e66fe17f6f Bug 24153: Remove warning for message queue
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-07-20 16:08:22 +02:00
dbc0575eee Bug 24153: Add a confirm flag to cleanup_database
How handy this will be, right?

This patch will change the behavior of this script as the --confirm flag
will now be required for effective changes.
Without --confirm and with --verbose you will see what could have done
the script in non dry-run mode.

Test plan:
Use different options of the script without the --confirm flag and have
a look at the output. It now tells you what would have been deleted if
the flag was passed.
Now use the --confirm flag and confirm that the changes are now
effective.

Sponsored-by: Association KohaLa - https://koha-fr.org/

Signed-off-by: Signed-off-by: Sonia Bouis <sonia.bouis@univ-lyon3.fr>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-07-20 16:08:22 +02:00
c136ad1d28 Bug 24152: Allow purge with 0 day
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-07-20 16:08:22 +02:00
c85d6f257f Bug 24152: (QA follow-up) Add tests for alternative from and to pars
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-07-20 16:08:22 +02:00
f59475a360 Bug 24152: Add the ability to purge pseudonymized tables
anonymized_* tables have been added by bug 24151, this patch adds the
usual way to purge data from them.

The cleanup_database.pl script has been adjusted to take new parameters
that will help to delete pseudonymized data.

Test plan:
Call the scrip with the new parameter to remove pseudonymized data
* --pseudo-transactions DAYS will remove entries from pseudonymized_transactions older
than DAYS day
* --pseudo-transactions can be used without the parameter DAYS but with
-- pseudo-transactions-from and/or --pseudo-transactions-to instead, to provide a
range of date

You can use the patch from bug 24153 to make the tests easier, data will
not be deleted if the new --confirm flag is not passed.

Sponsored-by: Association KohaLa - https://koha-fr.org/

Signed-off-by: Signed-off-by: Sonia Bouis <sonia.bouis@univ-lyon3.fr>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-07-20 16:08:22 +02:00
98fde332d8 Bug 24152: Add a warning about the delete of statistics's table entries
Signed-off-by: Signed-off-by: Sonia Bouis <sonia.bouis@univ-lyon3.fr>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-07-20 16:08:22 +02:00
2e35954c9c Bug 24152: Add method Koha::Objects->filter_by_last_update
Signed-off-by: Signed-off-by: Sonia Bouis <sonia.bouis@univ-lyon3.fr>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-07-20 16:08:22 +02:00
Julian Maurice
d3f0fc135b Bug 21395: Fix misc/admin/koha-preferences
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-06-29 15:43:41 +02:00
Julian Maurice
133a3ef168 Bug 21395: Fix creation of PO file
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-06-29 15:43:41 +02:00
e968af4377 Bug 21395: (QA follow-up) Remove some introduced issues
This patch removes some new error cases introduced during rebase

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-06-29 12:37:02 +02:00
Julian Maurice
1682341b58 Bug 21395: Fix QA errors
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-06-29 12:37:02 +02:00
Julian Maurice
b168f4a2e9 Bug 21395: Make perlcritic happy
This patch adds a .perlcriticrc (copied from qa-test-tools) and fixes
almost all perlcrictic violations according to this .perlcriticrc
The remaining violations are silenced out by appending a '## no critic'
to the offending lines. They can still be seen by using the --force
option of perlcritic
This patch also modify t/00-testcritic.t to check all Perl files using
the new .perlcriticrc.
I'm not sure if this test script is still useful as it is now equivalent
to `perlcritic --quiet .` and it looks like it is much slower
(approximatively 5 times slower on my machine)

Test plan:
1. Run `perlcritic --quiet .` from the root directory. It should output
   nothing
2. Run `perlcritic --quiet --force .`. It should output 7 errors (6
   StringyEval, 1 BarewordFileHandles)
3. Run `TEST_QA=1 prove t/00-testcritic.t`
4. Read the patch. Check that all changes make sense and do not
   introduce undesired behaviour

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-06-29 12:37:02 +02:00
692fb7e134 Bug 24159: (QA follow-up) Make terminology consistent
This patch changes the new circulation rule that's introduced from
useDaysMode to daysmode to improve consistency with other rule names.

We also update the accessors and code using them to reflect the new
term.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-06-25 10:51:59 +02:00
71e235751f Bug 24159: Set days_mode according to circ rules in 3 other places
There are 3 other occurrences where the new circ rule can be used:
 * C4::Circulation::checkHighHolds
 * Koha::Hold->set_waiting
 * misc/cronjobs/thirdparty/TalkingTech_itiva_outbound.pl

Test plan:
* checkHighHolds
Enable decreaseLoanHighHolds and fill decreaseLoanHighHoldsDuration
Setup things to hit a "high demand" alert with a shortened due date
Check an item out
=> The due date must be recalculated depending on the circ rule useDaysMode.

* set_waiting
Set ExcludeHolidaysFromMaxPickUpDelay to "1" (note that there is currently
a bug in the description of the syspref, see bug 22381 comment 19)
Mark a hold waiting
The expiration date should have been set depending on the value of the
circ rule.

* TalkingTech cronjob
Cannot test this

Signed-off-by: Simon Perry <simon.perry@itcarlow.ie>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-06-25 10:51:59 +02:00
b48869f1b8 Bug 25097: Add option to message_queue to allow for only specific sending notices
The message_queue.pl currently will send all the messages queued up.  It would be nice to have some control over which messages are to be sent.

Example: We have libraries who only wanted to send 'hold notices' but not overdue, advance notice, etc during the covid-19 quarantine.  And some that were the opposite - turn off hold notices, but send other generated notices.

Test Plan:
1) Apply this patch
2) Enqueue some messages with different letter codes
3) Run process_message_queue.pl with the new -c parameter using one of the letter codes enqueued
4) Note that only the messages with the selected code were processed!

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Alex Arnaud <alex.arnaud@biblibre.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-06-24 15:15:41 +02:00
6beda5cf1d Bug 21591: Check for record level item type issues too
To test:
1 - sudo koha-mysql kohadev
2 - UPDATE biblioitems SET itemtype = NULL where biblionumber = 1
3 - UPDATE items SET itype = NULL where biblionumber = 1
4 - perl misc/maintenance/search_for_data_inconsistencies.pl
5 - Notice warnings
6 - Apply patch
7 - Undefined itemtype on bibliolevel is now warned
7 - Test also with itype=""

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-06-15 11:53:36 +02:00
Bernardo Gonzalez Kriegel
7eb471fcdb Bug 25346: Supress warning message when install dir exists
To see it enable verbose mode

To test:
1) Install lang with install dir
   (cd misc/translator; ./tranlste install de-DE)
   check message: "de-DE installer dir /...mysql/de-DE already exists."
2) Apply patch
3) Repeat 1, now message is gone
4) Repeat 1 in verbose mode (./translate install de-DE -v)
   check the message now appears at the bottom.

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Alex Arnaud <alex.arnaud@biblibre.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-06-15 11:53:36 +02:00
David Roberts
051b0b8d2f Bug 22470: Missing the table name on misc/migration_tools/switch_marc21_series_info.pl
This patch adds the table name to the SQL update statements.

To test:

1) Apply the patch
2) Run the script. Check that there are no errors, and that the script
behaves as expected.

Signed-off-by: Andreas Roussos <a.roussos@dataly.gr>
Created a test record with values in field 440. Applied the patch, ran the
script with the -c -f flags and observed that the values were moved to field
490. Also, the relevant Koha to MARC mappings were changed accordingly.

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-06-15 11:53:36 +02:00
Julian Maurice
2b05647b63 Bug 25642: Fix description of new feature in update_dbix_class_files.pl
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-06-15 10:30:01 +02:00
77c945d380 Bug 25642: Add technical notes
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-06-15 10:30:01 +02:00
d148412c1d
Update release notes for 20.05.00 release
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-05-31 22:01:25 +01:00
aef33d840c
Translation fixes
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-05-31 21:34:23 +01:00
Koha translators
50d619d156
Translation updates for Koha 20.05.00
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-05-31 21:16:45 +01:00
40c562fa52
Translation Fixes
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-05-22 11:23:58 +01:00
Koha translators
9364ddad0e
Translation updates for Koha 20.05.00
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-05-22 11:23:22 +01:00
Julian Maurice
c97921bb68
Bug 25501: Supress warnings on installing translation
To test:
1) Verify the error installing translations for any language
2) Apply the patch
3) Repeat 1), check warning is gone

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-05-19 15:19:56 +01:00
e97ca8ab15
Bug 25157: Silent patron deletion cmd line script
Without -v the script will no longer display messages unless the error
is major.

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-05-11 09:57:07 +01:00
Katrin Fischer
8cfe35b044
Bug 25082: Make 041 descriptions display, if authorised value list is used
When 041$a or another subfield is linked to an authorised value list,
Koha will replace the codes with the descriptions from the authorised
value in the MARCXML before passing it to the XSLT.

The XSLT was made to translate codes into descriptions - as there is
no code now, this fails and 'unknown language' is shown.

The patch changes the handling so that when the XSLT is handled something
that doesn't match a known code, it displays the subfield without
processing.

To test:
- Create some records with 041.
  Examples: http://www.loc.gov/marc/bibliographic/bd041.html
- Verify this displays nicely in staff and OPAC
- Apply patch
- Verify display is unchanged
- Remove the patch
- Change one of your MARC frameworks (use another framework than
  for your first test records)
  and link the 041 subfields to the LANG authorised value list
- Go to administration and create some entries for LANG with language
  codes and descriptions
- Catalog a 041 using the normal editor (not advanced)
- You can now use a pull down to select the values you want
- Verify the display is now 'Unknown language' for this record
- Apply patch
- Verify all records display the language descriptions now, using
  an authorised value or not
- Run misc/maintenance/generate_MARC21Languages.pl

Verify the output at the end reads like:
<xsl:otherwise>
    <!-- when 041\$a is mapped to an authorised value list, this will show the description -->
    <xsl:value-of select="\$code" />
</xsl:otherwise>

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-05-11 09:46:02 +01:00
83c65d975f
Bug 25305: Translation process - Open all files specifying the utf8 encoding
To test:
1) Create translation files for a new language
( cd misc/translator; ./translate create xx-YY)
A new language means one that isn't already in Koha, xx-YY=>something you
invent.
2) Verify double encoding
egrep "Aix-Marseille|Jean Prunier|periodika|Bokm" misc/translator/po/xx-YY-*
check strange strings
3) Apply the patch
4) Create po files again
( cd misc/translator; rm -f po/xx-YY*; ./translate create xx-YY)
5) Verify no more double encoding
egrep "Aix-Marseille|Jean Prunier|periodika|Bokm" misc/translator/po/xx-YY-*
check normal string

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Didier Gautheron <didier.gautheron@biblibre.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-05-04 08:50:07 +01:00
11bf5d7afa
Bug 23137: Move cache flushing to the method
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-04-29 17:02:15 +01:00
9fa6bb6401
Bug 23137: Add reset option to rebuild_elasticsearch.pl
Setup:
1 - Be using Elasticsearch
2 - Reload mappings from the db
    Admin->Search configuration
    Reset Mappings
3 - Reindex ES and confirm searching is working

To test:
1 - Apply patch
2 - Alter your mappings file for elastic (just change a description for a field)
3 - perl misc/search_tools/rebuild_elasticsearch.pl -r -v
    Verbose not necessary, but good for letting you know things are progressing
4 - Confirm the mapping change shows in the interface
5 - Confirm reindex worked and searching is working

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-04-29 17:01:55 +01:00
4249b20a39
Bug 22828: Elasticsearch - display errors encountered during indexing on the command line
To test:
 1 - Use the Koha sample data, or insert a blank 245$b into a record (easiest way is using advanced cataloging editor
 2 - Reindex elasticsearch
 3 - Check the ES count on the about page
 4 - Check the count in the DB (SELECT count(*) FROM biblio)
 5 - They don't match!
 6 - perl misc/search_tools/rebuild_elastic_search.pl -v -v
 7 - No errors indicated
 8 - Apply patch
 9 - perl misc/search_tools/rebuild_elastic_search.pl -v
10 - You should be notified of an error
11 - perl misc/search_tools/rebuild_elastic_search.pl -v -v
12 - You should be notified of the specific biblio with an error and a (somewhat) readable reason
13 - perl misc/search_tools/rebuild_elastic_search.pl
14 - No output

Signed-off-by: Ere Maijala <ere.maijala@helsinki.fi>
Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr>
Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-04-29 16:58:00 +01:00
Bernardo Gonzalez Kriegel
8b5719a0e7
Bug 24871: Add new *-installer*.po translation files
This patch simply adds new po files for each supported
language.

The are not created on update, so the need to add this files.

To test:
1) Update your preferred language
2) Check missing *installer*po files
3) Create new translation files:
   cd /misc/translation
   ./translate create xx-YY

   check *installer*po files
4) Apply patch
5) Repeat 1, verify installer files are updated
   see last modification time

Pre-filled translations for some languages
(https://translate.koha-community.org/projects/marc21/)

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-04-27 11:09:49 +01:00
Bernardo Gonzalez Kriegel
2dac989f1d
Bug 25257: Preserve prolog on translated XML files
This patch fix the missing xml prolog in translated
files, XML or TT.

Is fixed teaching C4::TTParse not to ignore <?..?> constructs,
then teaching xgettext.pl to ignore those strings. Net result is
that they are copied in the translated file.

To test:
1) Update & install your preferred language,
(cd misc/translator/; perl translate update xx-YY; perl translate install xx-YY )

2) Compare the first lines (head -2) of:
koha-tmpl/opac-tmpl/bootstrap/en/xslt/compact.xsl
koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-opensearch.tt
koha-tmpl/intranet-tmpl/prog/en/xslt/plainMARC.xsl
and
koha-tmpl/opac-tmpl/bootstrap/xx-YY/xslt/compact.xsl
koha-tmpl/opac-tmpl/bootstrap/xx-YY/modules/opac-opensearch.tt
koha-tmpl/intranet-tmpl/prog/xx-YY/xslt/plainMARC.xsl

Check the missing prolog

3) Install this patch, repeat 1 and 2, now the prolog is present
on translated files.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-04-27 11:04:34 +01:00
82090e2673
Bug 24183: (RM follow-up) Remove re-introduced UseKohaPlugins
We removed UseKohaPlugins in bug 20415 but incidentally added a
reference to it bug 24183. This patch corrects that error.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-04-22 12:24:27 +01:00
4de2185614
Bug 20116: Flush the caches
When languages are added or removed when need to flush the caches to
keep the interface consistent

Signed-off-by: Liz Rea <wizzyrea@gmail.com>
Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-04-17 09:23:21 +01:00
Kyle M Hall
52339bcef0
Bug 20101: Cronjob automatic_item_modification_by_age.pl does not log run in action logs
C4::Log::cronlogaction() was never implemented in automatic_item_modification_by_age.pl.

Test Plan:
1) Apply this patch
2) Enable CronjobLog
3) Run automatic_item_modification_by_age.pl succesfully
4) Note the run is noted in the action logs

Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-04-17 09:17:52 +01:00
e5e26645ee
Bug 23571: (QA follow-up) Add missing newline in warning
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-04-14 16:59:15 +01:00
0d4fed8899
Bug 23571: Prevent concurrent execution of fines.pl
This patch introduces locking in fines.pl. It does so by leveraging on
bug 25109, which introduces a generic locking implementation on
Koha::Script.

The introduced changes:
1. Try to get the lock
2.a. If success, normal execution happens
2.b. If rejected, cronlogaction is called with a meaningful message and
  a normal exit happens, so we don't flood the logs in vain. --verbose
  will make the script print the same message on STDERR, as already is
  the case with this script

To test:
1. Apply this patch
2. In two separate consoles run:
   $ kshell
  k$ perl misc/cronjobs/fines.pl --verbose
SUCCESS => The first one runs normally, the second one exists really fast
  and prints a message about the lock.
3. Sign off :-D

Sponsored-by: Orex Digital
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-04-14 16:59:01 +01:00
ec6c143f1b
Bug 23871: data inconsistencies - check title exists
In biblio records, a field must be linked to biblio.title.
200$a in UNIMARC, 100$a MARC21 (and other).

If this field is undefined, some pages like checkouts table can fail.

Test plan :
1) Remove biblio title in SQL :
   UPDATE biblio SET title='' WHERE biblionumber=XXX
2) Run misc/maintenance/search_for_data_inconsistencies.pl
3) You see the record as an inconsistency
4) Reset a title
   UPDATE biblio SET title='50 shades of Grey' WHERE biblionumber=XXX
5) Run misc/maintenance/search_for_data_inconsistencies.pl
6) Record is no longer an inconsistency

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-04-14 16:55:13 +01:00
74c63d27c7
Bug 21865: allow test arg alone
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-04-14 16:54:24 +01:00
e788ac4e58
Bug 21865: add confirm to remove_unused_authorities.pl script
Like in most scripts in misc, add confirm argument
to ensure script is not run without knowing what it does.

Test plan:
1) Run misc/migration_tools/remove_unused_authorities.pl -h
2) You see help line for confirm
3) Run misc/migration_tools/remove_unused_authorities.pl
4) You see help and script does nothing
5) Run misc/migration_tools/remove_unused_authorities.pl -c
6) Script runs like wanted

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-04-14 16:54:19 +01:00
fc4cffe1f0
Bug 21865: ES compatiblity of remove_unused_authorities.pl
The script misc/migration_tools/remove_unused_authorities.pl directly checks if Zebra search is OK.

This patch changes so that this test is only if Zebra is the search engine.
It also adds a test on the search off any authority number indexed (index 'an').
With Zebra its : an,alwaysmatches=''
With ES its : an:*
This test ensure that biblios records are indexed and that not all autorities will be deleted.

Test plan:
1) On a catalog create a new authority
2) Use Zebra in systempreference SearchEngine
3) Stop Zebra server
4) Run misc/migration_tools/remove_unused_authorities.pl -c
5) The script does nothing and says :
   Zebra server seems not to be available. This script needs Zebra runs.
6) Restart Zebra server
7) Delete biblio index base
8) Run misc/migration_tools/remove_unused_authorities.pl -c
9) The script does nothing and says :
   Searching authority number in biblio records seems not to be available : an,alwaysmatches=''
10) Use ElasticSearch in systempreference SearchEngine
11) Delete biblio index base
12) Run misc/migration_tools/remove_unused_authorities.pl -c
13) The script does nothing and says :
    Searching authority number in biblio records seems not to be available : an:*

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-04-14 16:54:10 +01:00
bf60a9c88c
Bug 21865: improve remove_unused_authorities.pl script
remove_unused_authorities.pl script can be improved.

This patch changes changes verbosity so than test mode can be used
to know the autorities that are used and those that can be deleted.
It also writes a line in output if limited authority type(s).

This patch also removes the unused vars $thresholdmin and $thresholdmax.

It also changes the query to use SQL with parameters for authority types.

Test plan :
1) On a catalog create a new authority
2) Be sure catalog is well indexed
3) Run misc/migration_tools/remove_unused_authorities.pl -t
4) You will see the line :
   *** Testing only, authorities will not be deleted. ***
5) You will see lines of :
   authid=x type=y : used X time(s)
6) You will see the line for the authority created in 1) :
   authid=x type=y : can be deleted
7) You will see at the end :
   x authorities parsed
   y can be deleted because unused
   z unchanged because used
8) Run misc/migration_tools/remove_unused_authorities.pl
9) You don't see the line :
   *** Testing only, authorities will not be deleted. ***
10) You will see lines of :
   authid=x type=y : used X time(s)
11) You will see the line for the authority created in 1) :
   authid=x type=y : deleted
12) You will see at the end :
   x authorities parsed
   y deleted because unused
   z unchanged because used
13) Run misc/migration_tools/remove_unused_authorities.pl --auth NP --auth CO
14) You see the line :
    Restricted to authority type(s) : NP,CO.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-04-14 16:53:58 +01:00
d6fc05891b
Bug 22784: Add a cronjob to archive suggestions given age or status criteria
In order to deal automatically with suggestions that need to be
archived, this cronjob will help to add the "archived" flag following
their age or status.

Test plan:
- Execute the script with the --help parameter and read the documentation.
- Does it make sense?
- Use it without the --confirm flag and try several combination

For instance you will want to archive suggestions that have been
rejected for more than a week:
  perl misc/cronjobs/archive_purchase_suggestions.pl
  --age-date-field=rejecteddate --age=weeks:1

or even suggestion that have been suggested for more than a year and
have a ORDERED status:
  perl misc/cronjobs/archive_purchase_suggestions.pl
  --age-date-field=suggesteddate --age=years:1 --status=ORDERED

- Use it with the --confirm flag and make sure the suggestions have been
archived.

Sponsored-by: BULAC - http://www.bulac.fr/
Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-04-14 16:22:02 +01:00
3dd79180f3
Bug 25040: monkeypatch Schema::Loader for recent MariaDB
Recent versions of MariaDB changed the output of 'DESCRIBE' for
timestamp columns with defaults from `CURRENT_TIMESTAMP` to
`current_timestamp()`.  As such the code inside
DBIx::Class::Schema::Loader which catches such cases and outputs
`\"current_timestamp"` as a sensible cross platform default is missed
and this leads of inconsistent class files and bugs with out default
lookup code in Koha::Objects.

This patch serves as a backport of the code I have submitted upstream
such that out developers can continue to use update_dbix_class_files.pl
to build their schema classes from the database and regardless of their
db server version get a consistently correct output.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-04-14 08:26:10 +01:00
Aleisha Amohia
f406aa2ae3
Bug 24266: (alternate patch) Only use defined borrowernumbers in reconcile_balances.pl cron
I believe the error is triggered when borrowernumbers are left empty in
the accountlines table. Not sure why this would happen, but it appears
to be what causes the problem.

Do not apply the first patch if testing this patch.

To test:
1) sudo koha-mysql INSTANCENAME
2) Create a test borrower, add any payment etc to create an accountline,
    then delete this borrower
3) ensure the AccountAutoReconcile syspref is disabled
4) Go to another borrower's accounting tab
5) Create a manual credit or debit. Confirm this shows in the 'Make a
payment' tab as an amount that COULD be applied, but isn't automatically
applied
6) in your terminal, run the reconcile_balances.pl script
7) Confirm the error does not show in the logs and the balance for
the borrower is correctly reconciled.

Sponsored-by: Horowhenua District Council
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-04-14 08:21:31 +01:00
94ec55d767
Bug 24840: (follow-up) Catch some further cases of DateTime->now
This patch corrects a few additional cases where DateTime->now is called
directly instead of via Koha::DateUtils.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-04-08 11:54:36 +01:00
6ac037f817
Bug 24840: Replace DateTime->now with dt_from_string
We should use Koha::DateUtils instead of Date::Time directly

This patch simplay replaces calls to now() with a call to dt_from_string()
which does effectively the same thing.

Probably reading the code and verifying changes is sufficient but...

To test:
1 - confirm the files all compile
2 - confirm all tests pass
3 - confirm Koha still works

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-04-08 11:54:23 +01:00
8a56f176c0
Bug 24183: Add before_send_messages hook
This patch adds a simple hook to the process_message_queue cronjob
to allow plugins to pre-process messages in the messaging queue before
they are sent.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-04-06 10:43:25 +01:00
6464ada6c6
Bug 23473: Allow overwrite of passwords during import
To test:
 1 - Have some patrons in your system
 2 - Export some of their info via reports
    SELECT cardnumber, userid, surname, firstname, password, branchcode, categorycode
 3 - Edit the file from above, changing all the password lines
 4 - Import the file with overwrite
 5 - Confirm passwords have not changed (run the report again and confirm the hashes are the same)
 6 - Apply patch
 7 - Restart all the things
 8 - Check the new box on import screen to overwrite passwrods
 9 - Import file again
10 - Confirm passwords have changed
11 - Signin using new password to verify the hash is the password as supplied
12 - Repeat via commandline import supplying --overwrite_passwords option
13 - Verify works as expected
14 - Prove -v t/db_dependent/Koha/Patrons/Import.t

Sponsored-by: ByWater Solutions
Signed-off-by: Ron Marion <ron.marion@goddard.edu>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:21:59 +00:00
5982159f35
Bug 24883: (RM follow-up) Use Koha::Script
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:12:31 +00:00
Bernardo Gonzalez Kriegel
52b1314154
Bug 24883: Add POD entry and remove unused variable
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:11:32 +00:00
1439abcfe2
Bug 24883: Move new sub from load_yaml.pl
Use the sub from C4::Installer to avoid dup of code.

Note:
We are going to modify the script and so will do more stuffs.
We may want to rename it, maybe installer_utilities.pl,
misc/installer/yaml_utility.pl, any suggestions?

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:11:23 +00:00
Bernardo Gonzalez Kriegel
f52c9fde3f
Bug 24883: Command line utility to load yaml files
This patch adds a command line utility to load
yaml files.

To test:
1) Apply the patch
2) Try loading a file, e.g. sample notices
 a) mysql -e "delete from letter"
 b) misc/load_yaml.pl -f installer/data/mysql/en/mandatory/sample_notices.yml
3) Try loading a file with a syntax error,
   edit previous file adding ':' at the end of a field without quotes
   then load again

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:11:05 +00:00
93393036e5
Bug 23204: (RM follow-up) Use Koha::Script
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:08:54 +00:00
c7dbc27420
Bug 23204: Adjust copyright and license
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 11:55:52 +00:00
Alex Arnaud
e160af97a6
Bug 23204: Add exec permission on the script
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 11:55:14 +00:00
Alex Arnaud
85f3a3a302
Bug 23204: Move code in a unit tested sub
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 11:54:51 +00:00
Alex Arnaud
e1f8522391
Bug 23204: Add script for exporting Elasticsearch mappings
Test plan:

  - launch perl misc/search_tools/export_elasticsearch_mappings.pl >
    /path/to/my_mappings.yaml
  - set koha-conf.elasticsearch_index_mappings to
    /path/to/my_mappings.yaml,
  - go to admin -> Search engine configuration,
  - click on "Reset mappins",
  - check that your search fields and mappings are as expected

Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 11:54:35 +00:00
21c9b685bf
Bug 20415: Remove UseKohaPlugins system preference
Owen Leonard 2018-03-16 10:47:47 UTC :
<<
I don't think the system preference adds any security. There are already multiple permissions required for working with plugins:

- Configure plugins
- Manage plugins ( install / uninstall )
- Use report plugins
- Use tool plugins

And even with those permissions your server must be configured to allow the use of plugins.
>>

Test plan :
1) Install kitchen sink plugin https://github.com/bywatersolutions/koha-plugin-kitchen-sink
2) Run misc/devel/install_plugins.pl
3) Set config enable_plugins=1
4) Check all parts of the plugin are working
5) Set config enable_plugins=0
6) Check all parts of the plugin are disabled

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-26 11:42:02 +00:00
e3c6c4beed
Bug 24476: Rename autorenewal to autorenew_checkouts
Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-24 11:23:54 +00:00
47f9ec9446
Bug 24476: Allow patrons to opt out of auto-renewal
To test:
1 - Verify on staff side that patron can be edited to opt in our out of auto renewal
2 - Check out some items to a patron opted in to auto renewal
3 - Ensure the items are checked out and set to autorenew
4 - Login on the opac at the patron
5 - Verify items cannot be renewed as scheduled for auto-renewal
6 - On staff side, opt patron out of auto renewal
7 - Verify on opac items are no longer marked for auto renewal
8 - Run the auto renewal cron job, items are not renewed
9 - Set 'no renewal before' to a setting that would prevent renewal
10 - Verify that opting patron in or out of auto renewal changes only the reason items cannot be renewed
11 - Set 'no renewal before' to a setting that would allow for renewal
12 - Verify that opting patron in/out changes their ability to renew
13 - Verify that when opted out cron does not renew
14 - Verify that when opted in the item is auto renewed
15 - Reset the due date, opt out, verify manual renewal succeeds

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-24 11:23:47 +00:00
ce6229cd75
Bug 22589: Remove C4::Overdues::BorType
Test plan:
0) Apply patch
1) try to run affected scripts and ensure the amounts are corectly
calculated for different patron categories:
    installer/data/mysql/fix_unclosed_nonaccruing_fines_bug17135.pl
    misc/cronjobs/fines.pl
    misc/cronjobs/staticfines.pl

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-24 10:57:55 +00:00
Matthias Meusburger
15dd83268b
Bug 15214: bulkmarcimport should log if an authority has been updated
When importing with bulkmarcimport, authorities may or may not be updated based
on which authority is newer (005 are compared). This patch allows to keep track
in the result yaml file if an authority has been updated or not:

Signed-off-by: Jon Knight <J.P.Knight@lboro.ac.uk>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-24 10:54:46 +00:00
Rogan Hamby
5e2fe4418d
Bug 18414: Allow to use other select criteria with delete_patrons.pl
Adds the ability to specify a file with the --file flag that
should be a list of borrowernumbers for deletion.
If used without other flags it will delete the list of borrowers,
if used with other flags it will treat the other criteria
as filters for the list.

To Test:
1) Specify a list, e.g. borrowers.txt, of borrowernumbers in the database and put them in a text file with one per line
2) Invoke the script like ./delete_patrons.pl --file borrowers.txt to delete those patrons
3) Populate the list with a mix of borrowers in a category and those not, say categoryfoo
4) Invoke the scirpt as ./delete_patrons.pl --category_code categoryfoo --file borrowers.txt
5) Only the borrowers in the file that have the specified category will be deleted

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-24 10:42:18 +00:00
Liz Rea
60c31d70bb
Bug 24324: Resolve error with delete_records_via_leader.pl
To test:

Have a record with the 000 position 5 set to "deleted" (record status)
Use koha-shell to run the job with the -i flag: perl misc/cronjobs/delete_records_via_leader.pl -c -i -v
* items and the record should be deleted
Mark another record with the 000 position 5 set to deleted (record status)
Use koha-shell to run the job without the -i flag: perl misc/cronjobs/delete_records_via_leader.pl -c -v
* if the record had items, it will not be deleted
* if the record did not have items, it will be deleted.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-24 08:03:30 +00:00
a3687711b0
Bug 23463: Remove DelItemCheck and ItemSafeToDelete
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-23 09:26:31 +00:00
bbb504c86f
Bug 23463: Replace ModItem with Koha::Item->store
Starting to replace the ModItem calls with Koha::Item->store

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-23 09:26:30 +00:00
87acc552d5
Bug 23411: Do not fall back to 'email' for SMS messages
To test:
 1 - Have a patron wiht an sms number, but no email
 2 - Enable sms by setting SMSDriver syspref to Email
 3 - Set Notice triggers to send an overdue via sms and email
 4 - Ensure the selected notice is defined only for 'email'
 5 - Checkout an overdue to the patron above
    You can set a specify a due date in the past
 6 - perl misc/cronjobs/overdue_notices.pl -t
 7 - Check the patron notices tab, they have a print notice and an SMS notice pending
 8 - The SMS notice used the 'email' template
 9 - Apply patch
10 - sudo koha-mysql kohadev
11 - DELETE * FROM message_queue WHERE borrowernumber={borrower as above};
12 - Run the cron again
13 - Patron should have a print notice queued, no sms
14 - Define an SMS notice
15 - delete the message queue
16 - Run again
17 - The patron should have a print and an SMS

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-20 15:19:27 +00:00
29fe1f4fb4
Bug 24726: Replace one occurrence in print statement
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-19 09:34:53 +00:00
a696d4ecc3
Bug 24726: Update overdue_notices to use inbound_email_address
This patch updates the overdue_notices functionality to use the
inbound_email_address method to obtain the correct email address
for a branch.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-19 09:34:34 +00:00
Andreas Roussos
1d26fb3a9d
Bug 24827: Standardise on 'UTF-8' as the encoding name
This patch standardises the encoding name used in direct calls
to new_from_xml() to 'UTF-8' instead of 'utf8' or 'utf-8'.

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-16 14:45:39 +00:00
07b0c5bc0e
Bug 17845: Adjust few other occurrences
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-10 15:18:28 +00:00
Bernardo Gonzalez Kriegel
31774a2ee8
Bug 24262: (follow-up) admit null values on labels
This patch adds handling of null values on translatable
labels.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-09 13:54:24 +00:00
Bernardo Gonzalez Kriegel
33c1ddf86b
Bug 24262: Translate installer data in YAML format
This patch adds the ability to:
1) Create new translation files from yaml installer files
2) Create installer directory for a given language

It will not create a installer directory if it already exists.

New (possible) translation files:
  xx-YY-installer.po
  xx-YY-installer-MARC21.po
  xx-YY-installer-UNIMARC.po

Needs Bug 13897 (for yaml files)

NOTE: updated version adding ability to process multiline
fields, discard small ( < 2) strings, and discard strings
with pure html, TT or punctuation.

To test:
1) Apply patches from Bug 13897
2) Apply this patch
3) Go to misc/translation
4) Create translation files for a NEW language
   $ ./translate create xx-YY

   check new file 'xx-YY-installer.po'

5) Copy ../../installer/data/mysql/en/optional/auth_val.yml
   into ../../installer/data/mysql/en/marcflavour/marc21/mandatory/
   and ../../installer/data/mysql/en/marcflavour/unimarc/mandatory/

   remove po/xx-YY*, then repeat creation

   check new files 'xx-YY-installer.po', 'xx-YY-installer-MARC21.po'
   and 'xx-YY-installer-UNIMARC.po'

   remove all new files

6) Create for xx-YY again and try update
   $ ./translate create xx-YY
   edit ../../installer/data/mysql/en/optional/auth_val.yml
   and change one char in one of the translatable values,
   also edit 'xx-YY-installer.po', translate the same
   string (in msgstr).

   Do an update
   $ ./translate update xx-YY

   check in 'xx-YY-installer.po' a fuzzy value for the
   translated value and the preservation of the translation
   Fix the translation, or add a new one.

7) Create an install dir for xx-YY
   ./translate install xx-YY

    Check new dir '../../installer/data/mysql/xx-YY/'
    Check files on it
    $ tree ../../installer/data/mysql/xx-YY/
    and compare with ../../installer/data/mysql/en/

    All installation files must be present

8) Try a new Koha install using this language in the
   usual way.
   Check in authorised_values table for the translated string.

9) Try create an install dir for an existing language
   (eg. es-ES, de-DE or fr-FR ), eg.
   ./translate install de-DE

   Install dir is detected and not changed, a note is printed.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-09 13:50:41 +00:00
52e95fff34
Bug 24735: Remove QueryParser-related code
At the last development meeting we have voted to remove the
QueryParser-related code
https://wiki.koha-community.org/wiki/Development_IRC_meeting_19_February_2020

Hea tells us that it has not been adopted, and the code/bug tracker that
it is not really usable as it. As nobody is willing to work on it, we
decided to remove it instead.

Test plan:
 % prove t/db_dependent/Search.t
must return green

See commits from bug 9239 and confirm that the code is removed in this
patch.

Also play with the search on the UI and confirm that you do not see
obvious regressions

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-02 11:13:03 +00:00
Bernardo Gonzalez Kriegel
95e4e0a153
Bug 24664: Add missing *-messages-js.po
If you try to update some language translation files
translate script will complain of missing *-messajes-js.po
file.

This patch adds those missing files

To test:
1) Go to misc translation
2) Update some language, eg- fr-FR
Check error
/usr/bin/msgmerge: error while opening "/.../misc/translator/po/es-ES-messages-js.po" for reading: No such file or directory
3) Apply this patch
4) Repeat 2), no error message

Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-02 10:51:25 +00:00
Julian Maurice
cab8dd8cf1
Bug 24734: Fix paths in LangInstaller.pm for JS files
Use intrahtdocs and opachtdocs in order to get the correct paths for all
kind of installations

Test plan:
1. With a dev install, run ./translator create/update/install xx-XX and
   verify that there is no error message
2. With a standard install, run ./translator create/update/install xx-XX
   and verify that there is no error message

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Tested for dev/single/standard installs, no errors,
just a (normal) warn from po2json.

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-02 09:49:56 +00:00
7d8b96803f
Bug 24545: Fix license statements
Bug 9978 should have fixed them all, but some were missing.
We want all the license statements part of Koha to be identical, and
using the GPLv3 statement.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-02-24 13:31:26 +00:00
Bernardo Gonzalez Kriegel
e875e39806
Bug 24211: Compress/uncompress translation files
This patch adds the ability to compress/uncompress
translations files.

On update/install the files are uncompressed first

The only gain is to use less space.

To test:
1) Apply the patch
2) Go to misc/translator
3) Try it
   $ ./translate compress fr-FR (check po/*.gz)
   $ ./translate uncompress fr-FR ( check normal files)
4) Try again with verbose mode
   $ ./translate compress fr-FR -v (list compressed files)
   $ ./translate uncompress fr-FR -v (list uncompressed files)
5) Try update compressed files
   $ ./translate compress fr-FR
   $ ./translate update fr-FR (result is uncompressed)
6) Try install compressed files
   $ ./translate compress fr-FR
   $ ./translate install fr-FR
7) Try compress all
   $ ./translate compress  (add '-v' for verbose output)
8) Try uncompress all
   $ ./translate uncompress  (add '-v' for verbose output)

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-02-24 13:19:31 +00:00
cd2a99e456
Bug 24114: (QA follow-up) Fix counts for lock, anonymize and delete
Since these operations impact on the resultset, the counts should be
saved before.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-02-24 13:17:39 +00:00
7e1db56dd8
Bug 24114: Remove warn statements from Koha::Patrons
The warn must be done in the cronjob.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Followed this test plan (with two follow-ups applied):
[1] Prefs: UnsubscribeReflectionDelay=1, PatronAnonymizeDelay=2, PatronRemovalDelay=3, FailedLoginAttempts was undef
[2] Pick borrower and set expiry to NOW-2, and lock him (login_attempts=-1) Could be achieved too by settings FailedLoginAttempts and trying wrong passwords. Run cleanup job:
    Locked 0 patrons
    Anonymized 1 patrons
    Deleted 0 patrons
[3] Pick borrower, set expiry to NOW-3. Run cleanup job:
    Locked 0 patrons
    Anonymized 0 patrons
    Deleted 1 patrons

Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-02-24 13:17:39 +00:00
18afaf438b
Bug 24527: misc/cronjobs/update_totalissues.pl problem with multiple items
misc/cronjobs/update_totalissues.pl when issuing statistics table, no-incremental,  has a problem for biblio with multiple items.
The first SQL query gets the biblios linked to items with issues in statistics :

SELECT
  biblio.biblionumber, COUNT(statistics.itemnumber)
FROM
  biblio
LEFT JOIN
  items
ON
  (biblio.biblionumber=items.biblionumber)
LEFT JOIN
  statistics
ON
  (items.itemnumber=statistics.itemnumber)
WHERE
  statistics.type = 'issue'
$limit
GROUP BY
  biblio.biblionumber

The second SQL query is :
SELECT
  biblio.biblionumber, 0
FROM
  biblio
LEFT JOIN
  items
ON
  (biblio.biblionumber=items.biblionumber)
LEFT JOIN
  statistics
ON
  (items.itemnumber=statistics.itemnumber)
WHERE
  statistics.itemnumber IS NULL
GROUP BY
  biblio.biblionumber

The problem is that this second query will set to 0 where ANY item has no entry in statistics table.
So when running it sets 0 to the biblio that had a value from first
query.

I think the best fix is to use "WHERE statistics.type = 'issue'" inside que JOIN :
LEFT JOIN statistics ON (items.itemnumber=statistics.itemnumber AND statistics.type = 'issue')

Test plan :
1) Begin with an empty database
2) Create a biblio 1 with no items
3) Create a biblio 2 with 3 items
4) Create a biblio 3 with 2 items
5) Checkout and checkin all items of biblio 2
6) Checkout and checkin the firt item of biblio 3
7) run misc/cronjobs/update_totalissues.pl --use-stats -v
8) Check biblio 1 has biblioitems.totalissues = 0
9) Check biblio 2 has biblioitems.totalissues = 3
10) Without patch the biblio 3 has biblioitems.totalissues = 0
11) With patch the biblio 3 has biblioitems.totalissues = 1
12) Check misc/cronjobs/update_totalissues.pl --incremental is OK

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-02-24 13:14:24 +00:00
aa83cb59ce
Bug 21633: Remove finesMode 'test'
Signed-off-by: Magnus Enger <magnus@libriotech.no>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-02-19 11:32:11 +00:00
71d0775cc7
Bug 24651: Add maxdays option to fines cronjob
This patch adds an optional maxdays parameter to the fines.pl cronjob.
The new parameter allows system administrators to set a maximum number
of days for the job to process overdues.  This can improve performance.

Signed-off-by: Sally Healey <sally.healey@cheshiresharedservices.gov.uk>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-02-17 11:26:22 +00:00
82716a0172
Bug 23084: Replace grep {^$var$} with grep {$_ eq $var}
We certainly faced 3 similar bugs due to this syntax: bug 23006, bug
22941 and bug 17526.

To prevent other issues related to this syntax this patch suggests to
replace them all in one go.

Test plan:
Confirm that the 2 syntaxes are similar
Eyeball the patch and confirm that there is no typo!

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-02-17 10:44:45 +00:00
Julian Maurice
9d6ec5c64b
Bug 21156: Add plural translation capabilities to JS files
It adds Javascript equivalent of Koha::I18N's exported subroutines, and
they are used the same way.

String extraction is done only on *.js files and require gettext 0.19
(available in Debian jessie, and also in wheezy-backports)

It adds Javascript library Gettext.js for handling translation and a
Perl script po2json to transform PO file into JSON.

Gettext.js and po2json both come from Locale::Simple.
There are several tools named po2json. It's simpler to integrate this
one into Koha than to check if the good one is installed on the system.
Locale::Simple is not needed.

To avoid polluting the global namespace too much, this patch also
introduce a global JS object named Koha and add some stuff in Koha.i18n

Test plan:
1. Add a translatable string in a JS file. For example, add this:
     alert(__nx("There is one item", "There are {count} items", 3,
     {count: 3}));
   to staff-global.js
2. cd misc/translator && ./translate update fr-FR
3. Open misc/translator/po/fr-FR-messages-js.po, verify that your
   string is present, and translate it
4. cd misc/translator && ./translate install fr-FR
5. (Optional) Verify that
   koha-tmpl/intranet-tmpl/prog/fr-FR/js/locale_data.js exists and
   contains your translation
6. Open your browser on the staff main page, change language and verify
   that the message is translated
7. Repeat 1-6 on OPAC side

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Works well, translation is OK and test message is displayed correctly.
Current qa-tool error is a false positive.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-02-10 10:14:46 +00:00
17400198f0
Bug 24526: Add the confirm flag to the cronjob files
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-02-10 10:13:35 +00:00
27cc2d6a33
Bug 24526: Set verbose if confirm not set
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-02-10 10:13:31 +00:00
02275b8205
Bug 24526: Replace commit with confirm
Usually commit takes a value

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-02-10 10:13:27 +00:00
2dc6ef9631
Bug 24526: Use sprintf for readability
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-02-10 10:13:22 +00:00
aa9ad782ce
Bug 24526: (follow-up) clean up verbose conditional
Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-02-10 10:13:16 +00:00
6e0a4d18d4
Bug 24526: Add verbose and commit options to automatic_renewals cronjob
To test:
 1 - Apply patch
 2 - Have some items marked for auto-renewal
 3 - Run the job with no parameters
 4 - It should print 'Test mode'
 5 - Provide -v
 6 - It should print 'Test mode' then a line for each item
 7 - Provide -v -c
 8 - It should not say test mode, but should provide a line for each item
 9 - Provide -c
10 - Shoudl run as expected with no output

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-02-10 10:13:01 +00:00
Julian Maurice
1bcc2159a5
Bug 21177: Use koha-conf.xml in misc/devel/update_dbix_class_files.pl
It is annoying to have to specify database connection parameters each
time DBIx::Class files need to be updated.

This patch adds a new option --koha-conf that takes an optional <path>
which defaults to the value of KOHA_CONF environment variable, and use
the database connection parameters found in that file.
--db_* options override values from $KOHA_CONF

Test plan:
1. Run the script with the same parameters as before the patch and see
   that it still works.
   Example:
     misc/devel/update_dbix_class_files.pl --db_name koha_dev \
     --db_user koha --db_pass koha

2. Verify that KOHA_CONF is set and execute:
     misc/devel/update_dbix_class_files.pl --koha-conf
   Verify that Koha/Schema files were updated accordingly

3. Execute:
    misc/devel/update_dbix_class_files.pl --koha-conf \
    /path/to/another/koha-conf.xml
   Verify that Koha/Schema files were updated accordingly

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-02-10 10:12:15 +00:00
c0f3170c44
Bug 19008: Replace issues with checkouts in print statement
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-02-04 13:38:48 +00:00
Magnus Enger
88315db11a
Bug 19008: More database cleanups - QA followup
This patch tries to change the POD as suggested in the QA process.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-02-04 13:38:44 +00:00
07968fc5f6
Bug 19008: More database cleanups - item transfers
misc/cronjobs/cleanup_database.pl provides some database cleanup.
Yet some tables that could need cleanup are not in this script.

This patch adds cleanup for branchtransfers table.

Test plan :
- Count : select count(*),datearrived from branchtransfers group by datearrived;
- Run cleanup : misc/cronjobs/cleanup_database.pl -v --transfers 30
- Recount

Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-02-04 13:38:38 +00:00
fc7d7eece8
Bug 19008: More database cleanups - old reserves
misc/cronjobs/cleanup_database.pl provides some database cleanup.
Yet some tables that could need cleanup are not in this script.

This patch adds cleanup for old_reserves table.

Test plan :
- Count : select count(*),date(timestamp) from old_reserves group by date(timestamp);
- Run cleanup : misc/cronjobs/cleanup_database.pl -v --old-reserves 30
- Recount

Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-02-04 13:38:33 +00:00
9b5ff17d79
Bug 19008: More database cleanups - old issues
misc/cronjobs/cleanup_database.pl provides some database cleanup.
Yet some tables that could need cleanup are not in this script.

This patch adds cleanup for old_issues table.

Test plan :
- Count : select count(*),date(timestamp) from old_issues group by date(timestamp);
- Run cleanup : misc/cronjobs/cleanup_database.pl -v --old-issues 30
- Recount

Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-02-04 13:38:27 +00:00
ca511ad957
Bug 19008: More database cleanups - deleted patrons
misc/cronjobs/cleanup_database.pl provides some database cleanup.
Yet some tables that could need cleanup are not in this script.

This patch adds cleanup for deletedborrowers table.

Test plan :
- Count : select count(*),date(updated_on) from deletedborrowers group by date(updated_on);
- Run cleanup : misc/cronjobs/cleanup_database.pl -v --deleted-patrons 30
- Recount

Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-02-04 13:38:22 +00:00
9ddb78f0d3
Bug 19008: More database cleanups - deleted catalog
misc/cronjobs/cleanup_database.pl provides some database cleanup.
Yet some tables that could need cleanup are not in this script.

This patch adds cleanup for deleted catalog tables.
Note that deletedbiblio_metadata is managed by foreign key on biblionumber.

Test plan :
- Count :
select count(*),year(timestamp) from deleteditems group by year(timestamp);
select count(*),year(timestamp) from deletedbiblio group by year(timestamp);
select count(*),year(timestamp) from deletedbiblioitems group by year(timestamp);
- Run cleanup : misc/cronjobs/cleanup_database.pl -v --deleted-catalog 30
- Recount

Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-02-04 13:38:15 +00:00
eae2dfce06
Bug 19008: More database cleanups - statistics
misc/cronjobs/cleanup_database.pl provides some database cleanup.
Yet some tables that could need cleanup are not in this script.

This patch adds cleanup for statistics table.

Test plan :
- Count statistics : select count(*),date(datetime) from statistics group by date(datetime);
- Run cleanup : misc/cronjobs/cleanup_database.pl -v --statistics 30
- Recount statistics

Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-02-04 13:38:06 +00:00
14d1804652
Bug 21466: Keep all AVs for a given framework
There was a major (and silly) issue in the previous version, only one AV
was kept as we erased the hash value at the end of each iteration:
$invalid_locations_per_framework->{$framework->frameworkcode } =
  { items => $items, av_category => $mss->authorised_value, kohafield => $kohafield };

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-02-04 13:30:33 +00:00
Liz Rea
6fa4fbbbd7
Bug 24105: Fix error when running longoverdue.pl with --itemtypes
To test:

Run Longoverdue.pl as koha-shell with both 1 and several --itemtypes specified
notice that it has an error
apply this patch
run longoverdue.pl as koha-shell with both 1 and several --itemtypes specified
notice there isn't an error anymore!

Signed-off-by: David Roberts <david.roberts@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-01-23 10:26:05 +00:00
56fcf5a34d
Bug 22831: (RM follow-up) Code golf
Rather than performing a symmetric diff and then splitting the results
in a further loop this patch changes to logic to use two asymetric diffs
to get the results more directly.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-01-23 10:22:13 +00:00
e9f7a0dd1c
Bug 22831: (follow-up) Sort results, group by db
Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-01-23 10:11:28 +00:00
d5c79c201d
Bug 22831: (follow-up) Determine where the record is missing and provide link, show counter
Signed-off-by: Ere Maijala <ere.maijala@helsinki.fi>
Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-01-23 10:11:22 +00:00
2b32aee825
Bug 22831: Elasticsearch - add a maintenance script for checking DB vs index counts
This script uses Array::Utils and adds a dependency

To test:
 1 - Have Koha with ES running
 2 - Delete some records from ES
     curl -X DELETE "es:9200/koha_kohadev_biblios/data/5"
     curl -X DELETE "es:9200/koha_kohadev_authorities/data/5"
 3 - perl misc/maintenance/compare_es_to_db.pl
 4 - Note you are notified of problems in both indexes
 5 - perl misc/search_tools/rebuild_elastic_search.pl  -a
 6 - perl misc/maintenance/compare_es_to_db.pl
 7 - Note you are only notified about problems in biblios (assuming you don't have other issues)
 8 - perl misc/search_tools/rebuild_elasticsearch.pl  -b
 9 - perl misc/maintenance/compare_es_to_db.pl
10 - Both counts match, no problems

Signed-off-by: Ere Maijala <ere.maijala@helsinki.fi>
Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-01-23 10:11:09 +00:00
991f51b272
Bug 24397: Remove out of sync script populate_db.pl
populate_db.pl from koha-misc4dev must be used instead.
This script existed before the creation of koha-misc4dev, it does no
longer make sense to have it in Koha.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-01-23 09:07:52 +00:00
4e8ecf0d3d
Bug 21466: Catch any values that should be defined in the AV but are not
Let's expand the idea of the first patch to catch the wrong values for
any others AV.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-01-13 13:57:52 +00:00
e48d396002
Bug 21466: Search for items.location inconsistencies
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-01-13 13:57:42 +00:00
263c013121
Bug 23377: (QA follow-up) Use OVERRIDE_SYSPREF
SYSPREF_OVERRIDE unfortunately is not supported ;)

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-01-10 10:56:41 +00:00
Ian Walls
f036038318
Bug 23377: use ENV syspref override instead of complete disable
In an attempt to save time, bulkmarcimport temporarily sets CataloguingLog
and AuthoritiesLog to 0.  It does this by disabling syspref caching and saving
the changes to the database (then replacing the original values at completion).

Unfortunately, this disables other key sysprefs from being cached, and results in
a 50% increase in processing time for the script.

This patch instead utilizes the ENV variable override feature of sysprefs, which
preempts the cache in C4::Context->preference().

To test:
1. Perform a bulkmarcimport with a reasonable number of biblios (~1000 will do)
2. Note the time taken to complete
3. Apply patch
4. Revert the biblio load performed
5. Perform another bulkmarcimport with the same biblios and commandline options
6. Note the time taken to complete
7. Compare times.  The time from step 6 should be about 33% less than the time from step 2
8. Check Cataloguing and Authorities Logs to verify imported records were not logged
9. Profit!

Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-01-10 10:56:32 +00:00
c121a53dd7
Bug 24123: Fix import of UTF-8 encoded MARC21 MARCXML using bulkmarcimport (elastic only)
If elastic is used as search engine, the bulkmarcimport.pl will not
handle correctly UTF-8 encoded MARCXML

Koha::SearchEngine::Search->new uses a require statement to load the correct Search module.
This is done l.257 of bulkmarcimport.pl:
  257 my $searcher = Koha::SearchEngine::Search->new

Koha::SearchEngine::Elasticsearch::Search will `use MARC::File::XML`, and so resets the arguments set before:
  216     $MARC::File::XML::_load_args{BinaryEncoding} = 'utf-8';

  220     $MARC::File::XML::_load_args{RecordFormat} = $recordformat;

An easy (but dirty) fix could be to move the declaration of my $searcher before in the script.
The tricky (but correct) fix would be to remove the long standing "ugly hack follows" comment.

This patch is the easy, and dirty, fix

Test plan:
Use the command line tool to import MARXCML records that contains unicode characters into Koha

Something like `misc/migration_tools/bulkmarcimport.pl -biblios -file record.marcxml -m=MARCXML`

Without this patch you will notice that unicode characters will not be displayed correctly

Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-01-10 08:27:47 +00:00
2aedb2dfa6
Bug 24365: (bug 24217 follow-up) Fix encoding issue for translator script
I am not sure what happened here, but adding strict for TmplTokenizer.pm
breaks a test for translation.

It has been caught by a failing test:
 % prove xt/author/translatable-templates.t
will display lines "invalid multibyte sequence"

I did a diff between the generated files and found:
< msgid "Ã rbok"

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-01-07 15:07:31 +00:00
dc5fe323d2
Bug 24217: use Modern::Perl for modules when strict is missing
We absolutely need that for modules.
We are enabling strict by using Modern::Perl.
Note that other modules from C4 use strict and warnings, instead of
Modern::Perl

Test plan: git grep 2505 **/*.pm should not return any results

And let's see later what needs to be fixed.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-01-03 10:45:46 +00:00
Bernardo Gonzalez Kriegel
6e3e8fef6a
Bug 18688: Warnings about UTF-8 charset when creating a new language
This patch removes some warnings when creating new languages,
those related with uft8 and two not related.

Tipical messages are:
>Warning: Can't determine original templates' charset, defaulting to UTF-8
>Warning: Charset Out defaulting to UTF-8
>...
>Removing empty file /../xx-YY-marc-NORMARC.po
>I UTF-8 O UTF-8 at /../kohaclone/misc/translator/tmpl_process3.pl line 365.
>Use of uninitialized value $pref_name in concatenation (.) or string at /.../misc/translator/LangInstaller.pm line 197.
>Created /../misc/translator/po/xx-YY-messages.po.

First four are 'normal' messages
The fifth one correspond to an unquoted preference string
The sixth one is a normal output message from msgint command
We will hide them unless verbose '-v' option is selected,
and fix the error.

To test:
1) Go to misc/translator
2) Create new language files
  $ perl translate create xx-YY
3) Check warnings and messages
4) Apply the patch
5) Remove created files, and create them again
   Ther must be no messages
6) Repeat 5) with verbose option '-v', check
   old messages

7) BONUS: without the patch update a language,
   for example de-DE:
   $ perl translate update de-DE
   Note (git status) a new file, po/de-DE-messages.po~
   This is a backup file from msgmerge command
   With the patch this backup is removed.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-12-23 12:02:15 +00:00
a72417f04f
Bug 24268: Break assignment and warn into separate actions
Signed-off-by: Liz Rea <wizzyrea@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-12-20 12:06:00 +00:00
Radek Šiman
f497008d7f
Bug 24235: DUEDGST not sent if PREDUDGST checkbox is off
DUEDGST notice required PREDUEDGST to be on, but both messages have to
be generated independently

Test plan:
0) Do not apply the patch and reproduce the bug
    - turn OFF sms PREDUEDGST and turn ON sms DUEDGST for a given borrower
    - Sms version of DUEDGST is never generated into the message queue.
1) Apply the patch
2) Ensure that the bug is fixed

Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-12-16 21:01:08 +00:00
0553463d04
Update release notes for 19.11.00 release
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-11-28 18:25:06 +00:00
2135352417
Translation Fixes
Correct some fatal errors in the translation files

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-11-27 15:54:31 +00:00
Koha translators
d5a386db8e
Translation updates for Koha 19.11.00
(cherry picked from commit bc938166712f17d731a1f190628c27593a082419)
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-11-27 15:54:31 +00:00
Bernardo Gonzalez Kriegel
7657163d8f
Bug 23452: Multiple select options in system preferences are not translatable
Translate script ignores multiple options in system preferences,
this patch fix that.

To test:
1) Apply the patch
2) Go to misc/translator
cd misc/translator
3) Update your preferred language
perl translate update xx-YY
4) Updated pref file must contain new strings, eg.
egrep "when cataloguing an item" po/xx-YY-pref.po
5) Edit & translate that string
6) Install the updated translation
perl translate install xx-YY
7) Check that the translated pref file has
a) multiple options present
b) the message from 5) is translated
see for example circulation.pref
en: koha-tmpl/intranet-tmpl/prog/en/modules/admin/preferences/circulation.pref
xx: koha-tmpl/intranet-tmpl/prog/xx-YY/modules/admin/preferences/circulation.pref

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-11-26 11:45:41 +00:00
Magnus Enger
1f3a9722e4
Bug 24072: Typos in advance_notices.pl causes DUEDGST not to be sent
There are two typos in advance_notics.pl that cause DUEDGST messages
not to be sent. See Bugzilla for full details.

If you think the typo is sufficiently obvious, you can just eyeball the
patch and sign off, methinks. Otherwise, testing can be done something
like this:
- Make sure you have enabled enhanced messaging preferences, and a
  patron with "Email" and "Digests only" set for "Item due" messages
- Issue an item to this patron, with due date today
- Run something like this to generate advance notices:
  $ sudo koha-shell -c "perl \
  /home/vagrant/kohaclone/misc/cronjobs/advance_notices.pl -n -c" kohadev
- See that no notices are shown (-n means messages will go to stdout,
  instead of into the message queue).
- Apply the patch and run advance_notices.pl again, as before. A DUEDGST
  message should now be displayed.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-11-25 08:44:27 +00:00
Magnus Enger
5df9c63eea
Bug 24064: DUEDGST typoed as DUEGST
Bug 20478 introduced a typo where the letter code DUEDGST was changed
to DUEGST. This patch fixes it.

To test:
- Run "grep -r DUEDGST *" on the Koha git repo. Notice that this letter
  code is used in e.g. sample notices.
- Run "grep -r DUEGST *" and notice that this code only occurs twice,
  in misc/cronjobs/advance_notices.pl.
- Aply this patch.
- Run "grep -r DUEGST *" again, and notice there are no more occurences
  of this typo.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-11-19 15:16:58 +00:00
cce358f9af
Bug 23846: Handle exception gracefully at the OPAC
I do not think we should have the same trick as the intranet, and
display a message. This should be enough.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-11-13 08:04:23 +00:00
1e1c5d17a2
Bug 23846: Add a check to the data inconsistencies script
This may be quite long for big catalogue, but I think it is a good one
to have.

Test plan:
Same as first patch, then execute search_for_data_inconsistencies.pl
Notice the error.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-11-13 08:04:20 +00:00
a6117a187d
Bug 23903: Replace OPAC icons with Font Awesome
This patch replaces CSS-based background image icons with Font Awesome
icons. This will eliminate the use of sprite.png altogether.

To test, apply the patch and regenerate the OPAC CSS
(https://wiki.koha-community.org/wiki/Working_with_SCSS_in_the_OPAC_and_staff_client).

Because JavaScript events are often triggered by class, the changes made
in this patch require testing of JavaScript-driven behavior as well as
visual confirmation of the changes.

Enable OPAC features to expose them for testing: Login, Holds, Article
requests, Tags, Cart, Search term highlighting.

 - Perform a search in the OPAC
   - In the header at the top of the search results, check the
     'Unhighlight/Highlight' link.
   - Select some search results and verify that the multi-item controls
     are enabled: Add to cart/Lists; Place hold; Tag. Check that these
     controls work correctly.
   - Verify that the following links look correct and work correctly:
     'Place hold,' 'Request article,' 'Log in to add tags,' 'Add to
     cart.'
   - Log in to the OPAC and return to the search results page. Check the
     'Add tag' and 'Save to lists' links.

 - View the lists page
   - Test the 'Edit' and 'Delete' controls.
   - View the contents of a list. Test the following:
      - Controls at the top of the table of results ('New list,'
        'Download list,' 'Send list,' 'Print list,' 'Edit list,' and
        'Delete list').
      - Multi-selection operations: 'Place hold,' 'Tag,' and 'Remove
        from list.'
      - Controls for each item on the list: 'Place hold,' 'Add tag,'
        'Save to another list,' 'Remove from this list,' and 'Add to
        cart.'

   - View the bibliographic detail page for any search result
     - In the right-hand sidebar menu, verify that these controls look
       correct and work correctly: 'Place hold,' 'Print,' 'Request
       article,' 'Save to your lists,' 'Add to your cart,'
      'Unhighlight,' 'Send to device,' 'Save record,' and 'More
      searches.'
     - Test the shelf browser by clicking 'Browse shelf' under the call
       number for any of the holdings.
       - Test the 'Next' and 'Previous' controls.
     - Check the tabs for switching between 'Normal,' 'MARC,' and 'ISBD'
       views and verify that they look correct on each of those pages.

 - Add some items to the Cart and open the Cart
   - Test the cart controls at the top: 'More details/Brief display,'
     'Send,' 'Download,' 'Empty and close,' 'Hide window,' 'Print.'
   - Test the multiple-selection controls: 'Remove,' 'Add to list,'
     'Place hold,' and 'Tag.'

 - Log in to the OPAC as a user who has items checked out which can be
   renewed. On the 'Your summary' page, on the 'Checked out' tab, test
   the 'Renew selected' and 'Renew all' links.

Signed-off-by: Liz Rea <wizzyrea@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-10-31 15:40:25 +00:00
Andrew Isherwood
780f79ae1a
Bug 15260: Modify next/prev_open_day
We require next_open_day & prev_open_day to be more flexible. We could
create a separate sub, but that's not very DRY given that they'll do
pretty much the same thing.

So next_open_day becomes next_open_days and prev_open_day becomes
prev_open_days and both functions accept an additional parameter which
determines how many days they add or subtract.

All calls of these two functions have been modified accordingly.

Sponsored-by: Cheshire West and Chester Council
Sponsored-by: Cheshire East Council
Sponsored-by: Newcastle City Council
Sponsored-by: Sefton Council
Signed-off-by: Liz Rea <wizzyrea@gmail.com>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-10-31 11:52:26 +00:00
fe5c65c5cd
Bug 22826: Allow indexing of individual authorities in Elasticsearch
To test:
1 - perl misc/search_tools/rebuild_elastic_search.pl -h
2 - Note it indicates the bn option can be passed to index individual authids
3 - perl misc/search_tools/rebuild_elastic_search.pl -a -bn 92 -v
4 - Note the error
5 - Apply patch
6 - perl misc/search_tools/rebuild_elastic_search.pl -h
7 - Note new option ai|authid for indexing individual authids
8 - Note updated text for bn|biblionumber option
9 - perl misc/search_tools/rebuild_elastic_search.pl -a -bn 92 -v
10 - No errors, but no records indexed
11 - perl misc/search_tools/rebuild_elastic_search.pl -a -ai 92 -v
12 - 1 record indexed
13 - perl misc/search_tools/rebuild_elastic_search.pl -ai 92 -bn 92 -v
14 - 1 authority record and 1 biblio record indexed

Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-10-28 12:45:28 +00:00
c1fe8d1206
Bug 17168: (follow-up) QA concerns and rebase issues
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-10-25 13:20:22 +01:00
3cc10b7e97
Bug 17168: Remove j2a.pl
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-10-25 13:20:18 +01:00
006107c429
Bug 17168: Add Koha:Script -cron, remove Dumper
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-10-25 13:20:14 +01:00
436ad74399
Bug 17168: (follow-up) Address QA concerns and update for strict mode
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-10-25 13:20:07 +01:00
424df02e23
Bug 17168: (follow-up) Clean up QA errors
Cleanup variable names
Remove Dumper

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-10-25 13:20:00 +01:00
c7226ad5ea
Bug 17168: (follow-up) Tidy, clean params/options, use date tools
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-10-25 13:19:54 +01:00
af15cbbe1b
Bug 17168: Add a command line script for updating patron category based on status
This patch adds a new script update_patrons_category.pl which allows for
updating patron categories in a batch upon meeting provided criteria.
This script additionally can replace j2a.pl.

To test:
1 - perl update_patrons_category.pl -h
2 - Ensure help text makes sense and covers all options
3 - Test converting patrons supplying only fromcat and tocat
    perl update_patrons_category.pl -f PT -t J -v --confirm
    perl update_patrons_category.pl -f J -t PT -v --confirm
4 - All patrons should have been switched to and from Juveniles
5 - Try without --confirm switch
    perl update_patrons_category.pl -f PT -t J -v
6 - Should list all patrons but not update
7 - Set the age for juvenile patrons to be outside the range provided in
categories (or set the upper age limit for juveniles to '2')
8 - Test with verbosity and with without --confirm
    perl update_patrons_category.pl -f J -a -t PT -v
    perl update_patrons_category.pl -f J -a -t PT -v --confirm
9 - Repeat above and verify linked/unlinked guarantors are removed in above scenario
10 - Test various fine and registration limits
11 - Test matching on specific fields i.e. --field surname=acosta
12 - Sign off

Sponsored by: Round Rock Public Library
(https://www.roundrocktexas.gov/departments/library/)
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-10-25 13:19:47 +01:00
d661141890
Bug 23049: Update existing code to use debit_type
* Update C4::Accounts::chargelostitem
* Update C4::Accounts::manualinvoice
* Update C4::Circulation::_FixOverduesOnReturn
* Update C4::Circulation::_FixAccountForLostAndReturned
* Update C4::Overdues::UpdateFine
* Update C4::Overdues::GetFine
* Update C4::Overdues::GetOverduesForBranch
* Update Koha::Account->pay
* Update Koha::Account->add_debit
* Update Koha::Account->non_issues_charges
* Update Koha::Account::Line->apply
* Update Koha::Account::Line->adjust
* Update controller scripts
* Update reports scripts
* Update tests

Test Plan
1) Run the test suit and ensure everything still passes
2) Test reports/cash_register_stats still works
3) Test that adding manual invoices still works
4) Test that making payments still works
5) Test that lost item fee handling still works
6) Test that invoice printing still works
7) Test that the sco still works

Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-10-24 17:24:15 +01:00
72e10519e3
Bug 23049: Add debit_type
* Add account_debit_types table
* Add ac_debit_types_branches table
* Add account_debit_types defaults
* Add Koha::Account::DebitType and Koha::Account::DebitTypes
* Prevent deletion of defaults
* Migrate MANUAL_INV values
* Remove MANUAL_INV references
* Migrate accounttype values

Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-10-24 17:23:48 +01:00
Julian Maurice
82e6052b5d
Bug 20434: Add default authority type
Signed-off-by: Alex Arnaud <alex.arnaud@biblibre.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-10-17 15:02:47 +01:00
Julian Maurice
5e0c2164d6
Bug 20434: Add missing fields
018 - [Reserved for other International Standard Numbers]
033 - Other System Persistent Record Identifier
183 - Coded Data Field: Type of Carrier
203 - Content Form and Media Type
231 - Digital File Characteristics
283 - Carrier Type
338 - Funding Information Note

Signed-off-by: Sonia BOUIS <sonia.bouis@univ-lyon3.fr>
Signed-off-by: Alex Arnaud <alex.arnaud@biblibre.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-10-17 15:02:43 +01:00
Julian Maurice
422bcea5b2
Bug 20434: Add missing authority types
Signed-off-by: Sonia BOUIS <sonia.bouis@univ-lyon3.fr>
Signed-off-by: Alex Arnaud <alex.arnaud@biblibre.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-10-17 15:02:40 +01:00
Julian Maurice
109d527ccd
Bug 20434: Add UNIMARC field 214 and its subfields
Signed-off-by: Sonia BOUIS <sonia.bouis@univ-lyon3.fr>
Signed-off-by: Alex Arnaud <alex.arnaud@biblibre.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-10-17 15:02:37 +01:00
Julian Maurice
9e76fe8ee5
Bug 20434: Update UNIMARC framework - script
Test plan:
1. Take a look at files in misc/migration_tools/ifla/data. It
   contains all data that will be inserted into Koha. Its content is
   based on the previous patches.
2. Run the script misc/migration_tools/ifla/update.pl and verify
   that it effectively added the new fields, subfields, authorised
   values and authority types.
3. Run the script again and see that it doesn't update existing fields
4. Run with --force and verify that it update existing fields (you can
   modify unimarc_ifla.yml to see changes)
5. Run with --force --po-file misc/migration_tools/ifla/language/fr.po
   and verify that the labels are now in french

There is a POT file in misc/migration_tools/ifla/language/template.pot,
use it to create PO files for other languages.

Signed-off-by: Sonia BOUIS <sonia.bouis@univ-lyon3.fr>
Signed-off-by: Alex Arnaud <alex.arnaud@biblibre.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-10-17 15:02:33 +01:00