See comment 0.
https://metacpan.org/pod/distribution/DBIx-Class/lib/DBIx/Class/Manual/Cookbook.pod#Nested-transactions-and-auto-savepoints
Otherwise:
DBIx::Class::Storage::txn_rollback(): A txn_rollback in nested transaction is ineffective! (depth 1) at t/db_dependent/Koha/Objects.t line 274
Possible side-effects? Slowness?
We need to push it to master ASAP and see how our test suite behave.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Sooooo....
That was tricky, and the solution looks trivial.
However it's not.
We have unsafe set for "historical reason".
Having it on when RaiseError is on have the effect of overwritting the
DBIC error handler.
The problem is:
t/db_dependent/Circulation/MarkIssueReturned.t (and other tests) is failing with:
# expecting: Koha::Exceptions::Object::BadValue
# found: DBIx::Class::Exception ({UNKNOWN}: Can't locate object method "rethrow" via package "DBD::mysql::st execute failed: Incorrect datetime value: 'bad_date' for column 'returndate' at row 1 [for Statement "UPDATE `issues` SET `returndate` = ? WHERE ( `issue_id` = ? )" with ParamValues: 0='bad_date', 1=238] at /usr/share/perl5/DBIx/Class/Storage/DBI.pm line
In Koha::Object->store, the exception is not a DBIx::Class::Exception object (as we except), but
a string (on which we cannot call rethrow).
Swithing unsafe off restores the expected behavior.
To make sure the UI will not be affected, it is only turned off when
RaiseError is set.
The situation is still wrong (for UI), from the POD https://metacpan.org/pod/DBIx::Class::Storage::DBI (/unsafe)
"""
Note that your custom settings can cause Storage to malfunction, especially if you set a HandleError handler that suppresses exceptions and/or disable RaiseError.
"""
And also https://metacpan.org/release/DBIx-Class/source/lib/DBIx/Class/Storage/DBI.pm#L1531
Many thanks Tomas for the digging exploration!
We need to turn RaiseError and remove the unsafe flag, for UI as well,
but that should be done at the beginning of a dev cycle.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
In some tests we want to know if we are in a testing environment.
When run the usual way, our trick works, the perl interpreter matches 'prove':
$ENV{_} eq 'prove'
In other situations, we have the KOHA_NO_TABLE_LOCKS environment variables, for the SendCirculationAlert race conditions (see bug 15854 and bug 18364).
For unknown reasons, Jenkins runs the tests with /usr/bin/perl.
This patch suggests to rename KOHA_NO_TABLE_LOCKS and use KOHA_TESTING
instead, when prove is not used (or not correctly detected as it it the
case for Jenkins)
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Bug 9978 should have fixed them all, but some were missing.
We want all the license statements part of Koha to be identical, and
using the GPLv3 statement.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
To make our sql mode list compatible with MySQL 8.0.11
NO_AUTO_CREATE_USER has been removed in MySQL 8.0.11
https://dev.mysql.com/doc/relnotes/mysql/8.0/en/news-8-0-11.html
I do not think we needed it.
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
See C4::Circulation::SendCirculationAlert
3355 my $do_not_lock = ( exists $ENV{_} && $ENV{_} =~ m|prove| ) || $ENV{KOHA_NO_TABLE_LOCKS};
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Some shells may not pass the program name in underscore, and cron also
does not like it here:
Use of uninitialized value $ENV{"_"} in pattern match (m//) at Koha/Database.pm line 79.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
This patch will turn the strict SQL modes on When tests are ran with prove.
Test plan:
- Apply the first patch
- prove t/db_dependent/Koha/Database.t
=> Will pass if you have <strict_sql_modes>1</strict_sql_modes> in your
koha-conf.xml
=> Will fail otherwise
- Apply the second patch
- prove t/db_dependent/Koha/Database.t
=> Will pass whatever the value of strict_sql_modes in your
koha-conf.xml
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
To avoid disrupting testers and new developers it will be turned off by default.
We will turn it on for Jenkins so devs will have to take care of the
regressions they introduce (!)
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
So far we have bug 17258 (omnibus to list the issue we have with the new
default SQL modes), bug 20144 (which fixed our test suite with these
modes) and bug 20229.
This last one forces the SQL modes to avoid to modify the DBMS
configuration and define the SQL modes we support.
We should let developers catch the issues and report/fix them.
Also Jenkins must alert us if there is a regression with the test suite.
I suggest to enable the problematic SQL modes on dev installs.
Test plan:
- Revert commit 0180524bb9b1464c441bb1b858d0d8df37524d72
- prove t/db_dependent/Koha/Biblios.t
=> If you have dev_install defined in your Koha config file, the test
will fail with "Field 'datecreated' doesn't have a default value"
=> If you do not have dev_install, the tests will pass
NOTE: The commit number was wrong. Using
git log -- t/db_dependent/Koha/Biblios.t
This showed bug 20176 which was 524eab6788
Also dev_install was in koha_conf.xml already, so I just had to
toggle 0 and 1.
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Most of Koha depends on the local timezone of the server, except for Koha::Database which support an ENV override with the key TZ.
We should take this a step further. We should not only accept the TZ environment variable for all of Koha, we should really be able to set the timezone in the koha conf file as well so we don't have to pass that environment variable to things like cronjobs and one-off scripts.
Test Plan:
1) Apply this patch
2) Set a timzone in your koha_conf file, that is *not* your local time zone
Available timzones are listed here: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
3) Restart apache/plack
4) Perform some actions, check the timestamps in the database and in the
html output, note they are for the set timezone and not the local
timezone.
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Sponsored-by: Hotchkiss School
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
In our installation procedure we ask the administrator to edit the
MySQL|MariaDB configuration file to specify the SQL modes we support
(see bug 17258 comment 6 and 7 for more information).
We are on the way to catch and fix all these issues and support these
stricter modes (as they highlight problem in our codebase/DB structure)
but in the meanwhile it may be good to remove this step and revert the
changes when we are ready.
TODO:
- Remove that for dev installations (to let developers catch these bugs)
- Edit the wiki page to remove this step
Test plan:
0. Do not apply this patch
1. Edit your MySQL|MariaDB config and add:
sql-mode = "ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION"
to the [mysqld] section (or edit it)
2. Restart your DBMS
3. Try to make the app explode (watch the logs)
(tips: you should get "'koha_kohadev.me.id' isn't in GROUP BY" when
editing an order)
4. Apply the patch, restart_all, restart your DBMS
5. Try to recreate the failure
=> You should no longer see the error in the logs
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Safe way of getting and flushing the $database->{schema} -cache.
This is needed by Test::DBIx::Class to overcome pre-initialization connection
caching from C4::Context and others.
See Bug 18286.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Resolve this warning:
updatedatabase.pl: Use of uninitialized value $tls in string eq at /usr/share/koha/masterclone/Koha/Database.pm line 63.
Test plan:
[1] Check if you do not see the warning anymore.
Signed-off-by: Magnus Enger <magnus@libriotech.no>
Warning disappears after applying the patch.
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
In summary, changes are:
1) If you have chosen MySQL, Makefile.PL will ask you if you want TLS (default:
"no"), and then the locations for CA cert, client cert and client key
(reasonable defaults are provided). Settings <tls>, <ca>, <cert> and <key> are
added in koha-conf.xml
2) If <tls>yes</tls> in koha-conf.xml, the installer and database connection
scripts add the TLS options in both DBI connection strings and mysql command
line
To test
1/ Apply patch
2/ Check everything still works and db connections are the same as before
3/ Either run Makefile.PL and step through the options or edit your koha-conf.xml to
enable TLS
4/ Check db connections are still working
Patch provided to me by Dimitris Kamenopoulos and I reformatted it into a git patch,
any errors are probably mine
Signed-off-by: Mirko Tietgen <mirko@abunchofthings.net>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Executing the installer process and inserting all the sample data take a
lot of clics and time.
The idea of this script is to provide a quick way to insert all the
sample data easily to get a working Koha install asap.
Test plan:
- Set your database config to a non-existent DB
- Execute perl misc/devel/populate_db.pl
You will get an error
- Create an empty DB
- Execute perl misc/devel/populate_db.pl
It will insert all the MARC21 sample data
- Execute perl misc/devel/populate_db.pl
You will get an error because the DB is not empty (systempreferences and
borrowers tables)
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
These changes were a mistake, let's revert them.
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Brendan Gallagher <bredan@bywatersolutions.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Asking on #dbix-class, ribasushi told me to set quote_names to the
connection options.
Indeed it does the fix, globally :)
Test plan:
1/ Add the following snippet to the a script (mainpage.pl is a good candidate)
use Koha::Virtualshelves;
my $s = Koha::Virtualshelves->search({}, { order_by => '1,(select case when (3*2*1=6 AND 000227=000227) then 1 else 1*(select table_name from information_schema.tables)end)=1' });
$s->next;
2/ Execute the script
=> Without the patch, you should not get any error. If you have the mysql logs
enable, you will see the query
=> With the patch applied, you will get a "unknown column" error
Signed-off-by: Mirko Tietgen <mirko@abunchofthings.net>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>
Signed-off-by: Brendan Gallagher brendan@bywatersolutions.com
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
We don't want to recreate a new connection to the DB every time we want
a new schema.
This patch creates a $database package level variable on the same way
it's done in C4::Context for $dbh.
Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This patch makes Koha::Database lazy-load the whole Koha::Schema libraries.
It doesn't seem to have negative effects, and makes scripts not using
DBIx::Class notably faster [1].
Regards
[1] If you read the DBIx::Class::Schema docs, it explains that it it uses
Module::Find to load all schema files...
http://search.cpan.org/~ribasushi/DBIx-Class-0.082810/lib/DBIx/Class/Schema.pm#load_namespaces
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Happy to sign this one. The only extra comment would be that DBIx::Class
performance issues remain after this patch, but is being handled in a
different bug.
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Since we're passing an existing DBI database handle to
DBIC, and that handle doesn't have RaiseError set to true
by default, don't let DBIC override that -- for now.
Test plan: verify that the DB-dependent test suite passes.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Right now there is no connection between the database handles used by
C4::Context::dbh and Koha::Datbase/Schema. This makes it impossible to
use transactions in unit tests to temporarily modify the database to
test subroutines that take advantage of Koha::Database. This patch fixes
that issue.
Test Plan:
1) Apply this patch
2) prove -v t/db_dependent/ILSDI_Services.t and
prove -v t/db_dependent/Items.t and
prove -v t/db_dependent/Circulation_issue.t should
all start passing
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This patch fixes an issue whereby the DBIx::Class schema objects
were not connecting to the underlying database in UTF8 mode. This
most visibility manifested as patron list pages not displaying
diacritics correctly.
To test:
[1] Create a patron list, and make sure that it contains at least
one patron whose name or patron category description contains
a non-ASCII character.
[2] View the list contents; the diacritics should appear mangled.
[3] Apply the patch.
[4] View the patron list again. This time, the diacritics should
be displayed correctly. Note that Apache will also log
"list.pl: Wide character in print ...", but this is the lesser
of two evils.
[5] Verify that prove -v t/db_dependent/Koha_Database.t passes.
[6] (extra credit) Verify that t/db_dependent/Koha_Database.t
passes when connect to a PostgreSQL database.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch restores the ability to request a DBI database handle
or a DBIx::Class schema object connected to a PostgreSQL database.
To address the concerns raised in bug 7188, only "mysql" and "Pg"
are recognized as valid DB schemes. If anything else is passed
to C4::Context::db_scheme2dbi or set as the db_scheme in the Koha
configuration file, the DBD driver to load is assumed to be "mysql".
Note that this patch drops any pretense of Oracle support.
To test:
[1] Apply patch, and verify that the database-dependent tests
pass when run against a MySQL Koha database.
[2] To test against PostgreSQL, create a Pg database and
edit koha-conf.xml to set db_scheme to Pg (and adjust
the other DB connection parameters appropriately). The
following tests should pass, at minimum:
t/Context.t
t/db_dependent/Koha_Database.t
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as described, some additional notes:
- Installed Postgres following
http://wiki.ubuntuusers.de/PostgreSQL
- Created a database user koha
- Created a database koha
- Changed the koha-conf.xml file
<db_scheme>Pg</db_scheme>
<database>koha</database>
<hostname>localhost</hostname>
<port>5432</port>
<user>koha</user>
<pass>xxxx</pass>
- Installed libdbd-pg-perl
- Ran the web installer until step 3 everything looked ok
Step 3 complains:
Password for user koha: psql: fe_sendauth: no password supplied
- Both t/Context.t and t/db_dependent/Koha_Database.t pass
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This corrects a failing test and follows a recommendation
by the maintainer of DBIx::Class. This patch also
adds a couple new directories for t/00-testcritic.t to
check.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Jonathan Druart
Marcel de Rooy
Kyle Hall
Kyle M Hall
Tomás Cohen Arazi
Olli-Antti Kivilahti
Dimitris Kamenopoulos
Robin Sheat
Jonathan Druart
Galen Charlton
Chris Cormack