Commit graph

37862 commits

Author SHA1 Message Date
41905c7eaa
Bug 22001: Make the DBD error regex less strict
Now the message is looking like
DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::mysql::st execute failed: Duplicate entry

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-31 17:46:21 +01:00
8e6cebb6b4
Bug 22001: Add the DBI error message
Using DBIx::Class error handler, we should not hide the error when
encapsulating it

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-31 17:46:06 +01:00
3533aacef8
Bug 22001: Set unsafe only if RaiseError is not set
Sooooo....

That was tricky, and the solution looks trivial.
However it's not.
We have unsafe set for "historical reason".
Having it on when RaiseError is on have the effect of overwritting the
DBIC error handler.

The problem is:
t/db_dependent/Circulation/MarkIssueReturned.t (and other tests) is failing with:

  # expecting: Koha::Exceptions::Object::BadValue
  # found: DBIx::Class::Exception ({UNKNOWN}: Can't locate object method "rethrow" via package "DBD::mysql::st execute failed: Incorrect datetime value: 'bad_date' for column 'returndate' at row 1 [for Statement "UPDATE `issues` SET `returndate` = ? WHERE ( `issue_id` = ? )" with ParamValues: 0='bad_date', 1=238] at /usr/share/perl5/DBIx/Class/Storage/DBI.pm line

In Koha::Object->store, the exception is not a DBIx::Class::Exception object (as we except), but
a string (on which we cannot call rethrow).

Swithing unsafe off restores the expected behavior.
To make sure the UI will not be affected, it is only turned off when
RaiseError is set.

The situation is still wrong (for UI), from the POD https://metacpan.org/pod/DBIx::Class::Storage::DBI (/unsafe)
"""
Note that your custom settings can cause Storage to malfunction, especially if you set a HandleError handler that suppresses exceptions and/or disable RaiseError.
"""

And also https://metacpan.org/release/DBIx-Class/source/lib/DBIx/Class/Storage/DBI.pm#L1531

Many thanks Tomas for the digging exploration!

We need to turn RaiseError and remove the unsafe flag, for UI as well,
but that should be done at the beginning of a dev cycle.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-31 15:17:23 +01:00
9c383aa286
Bug 23463: (follow-up 3) Fix timestamp default value
This is the only situation I found where:
 * t/db_dependent/Koha/Item.t is passing
 * t/db_dependent/Koha/Object.t is passing
 * MySQL 8 is happy (and not fail with "Column 'timestamp' cannot be null"), which is certainly what I missed in the previous follow-up

About the change to Object.t: the next store was called without the
generated timestamp, so it is needed to call discard_changes.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-31 09:52:57 +01:00
af1275ce7d
Bug 22001: Fix Object.t - handle correct RaiseError and PrintError
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-31 09:50:12 +01:00
9816e240aa
Bug 25018: Use new KOHA_TESTING envvar to detect environment testing
In some tests we want to know if we are in a testing environment.
When run the usual way, our trick works, the perl interpreter matches 'prove':
  $ENV{_} eq 'prove'

In other situations, we have the KOHA_NO_TABLE_LOCKS environment variables, for the SendCirculationAlert race conditions (see bug 15854 and bug 18364).

For unknown reasons, Jenkins runs the tests with /usr/bin/perl.

This patch suggests to rename KOHA_NO_TABLE_LOCKS and use KOHA_TESTING
instead, when prove is not used (or not correctly detected as it it the
case for Jenkins)

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-30 09:35:56 +01:00
624a24b820
Bug 24614: Fix Reports/Guided.t
DBD::mysql::st execute failed: Column 'public' cannot be null [for Statement "UPDATE `saved_sql` SET `last_modified` = ?, `notes` = ?, `public` = ?, `report_group` = ?, `report_name` = ?, `report_subgroup` = ?,
`savedsql` = ? WHERE ( `id` = ? )" with ParamValues: 0='2020-03-27T16:03:04', 1=undef, 2=undef, 3=undef, 4='Just another report', 5=undef, 6=undef, 7=25] at /usr/share/perl5/DBIx/Class/Storage/DBI.pm line 1836.
    # Looks like you planned 17 tests but ran 6.

update_sql expect all the paramters to be set, otherwise it will be
nulled.

The best way to fix it is at test level. There is only 1 occurrence in
controller/module, and it sends all the parameters. That is the correct
way to do and will make things easier when we will removed them to use
Koha::Reports directly

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 16:17:14 +00:00
ff19fd4806
Bug 18177: (follow-up) remove aqbooksellers.currency
There was an occurrence in tests

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 15:48:15 +00:00
7d8ddfb675
Bug 23204: Fix tests swapping fields (they are not ordered)
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 15:39:34 +00:00
Nazlı Çetin
2cf5ae07f1
Bug 25000: SQL report not updated
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:35:52 +00:00
e684fac28d
Bug 23204: DBRev 19.12.00.060
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:32:14 +00:00
5b4eb87483
Bug 23473: (QA follow-up) Fix number of tests
=> Looks like you planned 158 tests but ran 159.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:23:22 +00:00
68776ef2e6
Bug 23473: (follow-up) Fix QA complaints (tabs, filters, Dumper)
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:23:17 +00:00
214d78309d
Bug 23473: Don't allow staff password changes for imports
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:23:12 +00:00
9a6134710d
Bug 23473: Don't allow setting blank passwords via import
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:23:08 +00:00
8c7ae857a2
Bug 23473: (follow-up) Catch and show password exceptions
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:23:04 +00:00
481bc59de5
Bug 23473: (follow-up) check if overwrite_passwords is set
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:23:00 +00:00
4c6517d05b
Bug 23473: (follow-up) Make passwords 'secure' for tests
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:22:47 +00:00
6464ada6c6
Bug 23473: Allow overwrite of passwords during import
To test:
 1 - Have some patrons in your system
 2 - Export some of their info via reports
    SELECT cardnumber, userid, surname, firstname, password, branchcode, categorycode
 3 - Edit the file from above, changing all the password lines
 4 - Import the file with overwrite
 5 - Confirm passwords have not changed (run the report again and confirm the hashes are the same)
 6 - Apply patch
 7 - Restart all the things
 8 - Check the new box on import screen to overwrite passwrods
 9 - Import file again
10 - Confirm passwords have changed
11 - Signin using new password to verify the hash is the password as supplied
12 - Repeat via commandline import supplying --overwrite_passwords option
13 - Verify works as expected
14 - Prove -v t/db_dependent/Koha/Patrons/Import.t

Sponsored-by: ByWater Solutions
Signed-off-by: Ron Marion <ron.marion@goddard.edu>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:21:59 +00:00
a444f643ad
Bug 18177: DBRev 19.12.00.059
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:19:26 +00:00
f50fee2546
Bug 18177: Do not remove columns if at least one value exists
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:16:11 +00:00
57788803e0
Bug 18177: DBIC changes
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:16:04 +00:00
287416fcad
Bug 18177: DB changes
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:15:59 +00:00
93ed21862a
Bug 18177: remove aqbooksellers.currency
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:15:55 +00:00
6070492462
Bug 18177: remove aqbooksellers.bookselleremail
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:15:51 +00:00
a961487d65
Bug 18177: remove aqbooksellers.booksellerurl
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:15:47 +00:00
0d9ff747a2
Bug 18177: remove aqbooksellers.booksellerfax
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:15:42 +00:00
b1122cf0f3
Bug 18177: remove aqbooksellers.othersupplier
This patchset removes some columns from the aqbooksellers table that are
not used:
 * booksellerfax
 * booksellerurl
 * bookselleremail
 * othersupplier
 * currency

The first 3 ones are certainly leftover from bug 10402.

Test plan:
For each for these fields you will make sure that they cannot be edited
from the interface.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:15:32 +00:00
97913b8a64
Bug 23888: (follow-up) Wrap English string in translation function
This patch makes alert added in subscription-add.js translatable using
the new __() function.

To test, apply the patch and confirm that the alert still works
correctly when submitting the subscription edit form with an invalid
vendor id.

Test that the string can be translated (using fr-Fr for example):

1. cd misc/translator && ./translate update fr-FR
2. Translate strings in misc/tranlator/po/fr-FR-messages-js.po
3. cd misc/translator && ./translate install fr-FR

Switch to the fr-FR translation and confirm the the alert shows your
translated string.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:14:21 +00:00
c52e15689f
Bug 23888: Do not allow invalid vendor id on creating a subscription
It will avoid crash and invalid data when creating/updating a
subscription.

This could have been done with a AJAX query but seems more convenient
this way.

Test plan:
- Create or update a subscription
- In the "Vendor" input try an empty string, a valid vendor's id, and
invalid one.
=> With an empty string you get the existing alert message
=> With a valid id you do not get any messages
=> With an invalid id you are not allowed to go to page 2

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:14:09 +00:00
01551437c0
Bug 24386: Prevent double form submission on adding orders to basket from a file
To test:
 1 - Have a marc record file
 2 - Go to Acquisitions, find a vendor, then an open basket
 3 - Add to basket from a new file (if you already have one staged you can choose that instead)
 4 - Import the file, then add to basket
 5 - On the page where you select the records and set info select a record, set matching to 'Do not check' fill in the fund for the order
 6 - When ready, click save as many times as you can
 7 - When you go to the basket there are many copies of the order added
 8 - Don't despair, apply the patch
 9 - Repeat above
10 - When ready to add orders, click furiously
11 - The order is only added once!
12 - Success!

Signed-off-by: Sally <sally.healey@cheshiresharedservices.gov.uk>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:13:21 +00:00
5982159f35
Bug 24883: (RM follow-up) Use Koha::Script
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:12:31 +00:00
Bernardo Gonzalez Kriegel
52b1314154
Bug 24883: Add POD entry and remove unused variable
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:11:32 +00:00
1439abcfe2
Bug 24883: Move new sub from load_yaml.pl
Use the sub from C4::Installer to avoid dup of code.

Note:
We are going to modify the script and so will do more stuffs.
We may want to rename it, maybe installer_utilities.pl,
misc/installer/yaml_utility.pl, any suggestions?

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:11:23 +00:00
30c0f96aa0
Bug 24883: Move to a flat array with all values to an array of arrayref
For the following SQL query:
INSERT INTO cities(city_name, city_country) VALUES ('Madrid', 'Spain'), ('Buenos Aires', 'Argentina');

We move from:
[ 'Madrid', 'Spain', 'Buenos Aires', 'Argentina' ]
to:
[ [ 'Madrid', 'Spain' ], [ 'Buenos Aires', 'Argentina' ] ]

Which make more sense to split, build and construct the queries

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:11:18 +00:00
876d12375c
Bug 24883: Centralize code to load YAML installer files to a sub
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:11:13 +00:00
Bernardo Gonzalez Kriegel
f52c9fde3f
Bug 24883: Command line utility to load yaml files
This patch adds a command line utility to load
yaml files.

To test:
1) Apply the patch
2) Try loading a file, e.g. sample notices
 a) mysql -e "delete from letter"
 b) misc/load_yaml.pl -f installer/data/mysql/en/mandatory/sample_notices.yml
3) Try loading a file with a syntax error,
   edit previous file adding ':' at the end of a field without quotes
   then load again

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:11:05 +00:00
93393036e5
Bug 23204: (RM follow-up) Use Koha::Script
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 12:08:54 +00:00
c7dbc27420
Bug 23204: Adjust copyright and license
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 11:55:52 +00:00
e944f2f6fd
Bug 23204: Update the mappings.yaml file with the output generated
We need to be consistent and provide the mappings.yaml file generated by
the script.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 11:55:48 +00:00
e578d0b36b
Bug 23204: Fix ordering of fields in exported file
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 11:55:42 +00:00
0c9585333d
Bug 23204: (follow-up) Also sort by marc field
Test plan :
Launching script several times and see that marc field is alway ordered increasing

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 11:55:30 +00:00
Alex Arnaud
c338829aba
Bug 23204: Sort search fields by name
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 11:55:23 +00:00
Alex Arnaud
e160af97a6
Bug 23204: Add exec permission on the script
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 11:55:14 +00:00
Alex Arnaud
32ea7a6378
Bug 23204: Export field weights
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 11:55:04 +00:00
Alex Arnaud
85f3a3a302
Bug 23204: Move code in a unit tested sub
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 11:54:51 +00:00
Alex Arnaud
e1f8522391
Bug 23204: Add script for exporting Elasticsearch mappings
Test plan:

  - launch perl misc/search_tools/export_elasticsearch_mappings.pl >
    /path/to/my_mappings.yaml
  - set koha-conf.elasticsearch_index_mappings to
    /path/to/my_mappings.yaml,
  - go to admin -> Search engine configuration,
  - click on "Reset mappins",
  - check that your search fields and mappings are as expected

Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 11:54:35 +00:00
Ere Maijala
70109d4f94
Bug 24700: Avoid validating REST API spec multiple times
This patch changes the spec loading so that it is fully loaded and merged with any plugin routes before validation. The individual parts are separately validated only if the resulting complete spec is invalid. Note: the OpenAPI plugin validates the spec given to it, so normally no manual validation is necessary.

Test plan:
1. prove -v t/db_dependent/Koha/REST/*
2. prove -v t/db_dependent/api/v1/*

Sponsored-by: The National Library of Finland
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 11:53:44 +00:00
a44ce976f6
Bug 24982: (follow-up) Retain checkbox selections after for submit
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 11:52:32 +00:00
4c6bd0bfee
Bug 24982: (follow-up) Move checkboxes to left side of label
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-27 11:52:28 +00:00