anonymized_* tables have been added by bug 24151, this patch adds the
usual way to purge data from them.
The cleanup_database.pl script has been adjusted to take new parameters
that will help to delete pseudonymized data.
Test plan:
Call the scrip with the new parameter to remove pseudonymized data
* --pseudo-transactions DAYS will remove entries from pseudonymized_transactions older
than DAYS day
* --pseudo-transactions can be used without the parameter DAYS but with
-- pseudo-transactions-from and/or --pseudo-transactions-to instead, to provide a
range of date
You can use the patch from bug 24153 to make the tests easier, data will
not be deleted if the new --confirm flag is not passed.
Sponsored-by: Association KohaLa - https://koha-fr.org/
Signed-off-by: Signed-off-by: Sonia Bouis <sonia.bouis@univ-lyon3.fr>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Since our use ok Koha::Objects, there's been an implicit instantiation
happening in ->search that has somehow hidden the fact that classes need
instantiation...
This change in ->delete and ->update made it clear because of the
failures/hangs on the tests.
This patch instantiates Koha::Patron::Categories before attempting to
call ->update.
The original patches for this bug are correct and the only issue here is
that we need to document better than for some methods, instantiation is
not implicit and is required first. This is awkward and I would prefer
to force everyone to call ->new first. But I understand it would be
inconvenient.
To test:
1. Run:
$ kshell
k$ prove t/db_dependent/Template/Plugin/Categories.t
=> FAIL: Notice it hangs, Ctrl+c to abort
2. Apply this patch
3. Repeat 1
=> SUCCESS: Tests dong hang, and in fact pass
4. Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch improves the semantics of the update routine to more clearly
separate the 'fields' we're modifying from the 'options' we wish to
apply to the modification.
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
In case we already started to iterate.
This fixes a failing tests in t/db_dependent/Koha/Biblios.t
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
If we call Koha::Libraries->delete but Koha::Library->delete exists
(ie. Koha::Object->delete is overridden), then we Koha::Objects->delete
will be called and the overridden will not be executed.
This patch suggests to test if the method is overridden (using
Class::Inspector->function_exists). If so we loop on the different
objects of the set in a transaction and call the overridden ->delete method.
Existing tests widely cover this change.
t/db_dependent/Koha/Objects.t
subtest 'Return same values as DBIx::Class' => sub {
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
In OO we would usually instantiate the class and then interact with it
like in:
my $rs = Koha::Patrons->new;
$rs->empty;
Koha's practice is to call things like Koha::Patrons->search without
instantiating the Koha::Patrons class. To keep consistency, this patch
instantiates the resultset object on behalf of the caller if required.
To test:
1. Apply the tests patch
2. Run:
$ kshell
$k prove t/db_dependent/Koha/Objects.t
=> FAIL: Tests fail because it is expected to have the class
instantiated
3. Apply this patch
4. Repeat 2.
=> SUCCESS: Tests pass, instantiation happens implicitly.
5. Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch adds a new ->empty method to Koha::Objects, that can be used
to make the underlying DBIC resultset empty. This way, we can have
consistency in our method's return values without the need to build
a query that we know in advance that will be empty. No need to hit the
DB at all.
To test:
1. Apply this patches
2. Notice the tests cover what is expected
3. Run:
$ kshell
k$ prove t/db_dependent/Koha/Objects.t
=> SUCCESS: Tests pass! Yay!
4. Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Bug 9978 should have fixed them all, but some were missing.
We want all the license statements part of Koha to be identical, and
using the GPLv3 statement.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch makes the Koha::Object(s) derived classes expose information
about prefetch-able relations. This is then used by a new helper to
generate the prefetch information for the DBIC query.
To test:
1. Apply this patch
2. Run:
$ kshell
k$ prove t/db_dependent/Koha/Object* \
t/db_dependent/Koha/REST/Plugin/Objects.t \
t/Koha/REST/Plugin/Query.t
=> SUCCESS: Tests pass!
3. Sign off :-D
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
We certainly faced 3 similar bugs due to this syntax: bug 23006, bug
22941 and bug 17526.
To prevent other issues related to this syntax this patch suggests to
replace them all in one go.
Test plan:
Confirm that the 2 syntaxes are similar
Eyeball the patch and confirm that there is no typo!
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
and remove 'scalar' keyword in calls where it's not needed.
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch adds from_api_mapping to Koha::Objects, in order to be able to get the mapping from a result set.
To test:
1. apply this patch
2. prove t/db_dependent/Koha/Objects.t
3. sign off
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch makes the parameters for Koha::Object(s)->to_api a hashref
preparing the ground for new parameters. The proposed syntax is:
$object->to_api(
{
embed => {
'items' => {
'children' => {
'homebranch' => {}
}
}
}
}
);
Tests are added for the Koha::Objects implementation and tests for
Koha::Object are adjusted to the new syntax, and a test for the single
result accessor is added as well.
To test:
1. Apply this patches
2. Run:
$ kshell
k$ prove t/db_dependent/Koha/Object.t t/db_dependent/Koha/Objects.t
=> SUCCESS: Tests pass!
3. Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch makes Koha::Object(s)->to_api have an 'embeds' parameter that
using dot notation (e.g. resource.related_resource.another_one) allows
embedding objects recursively.
To test:
1. Apply this patch
2. prove t/db_dependent/Koha/Object.t
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Agustin Moyano <agustinmoyano@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch makes the 'attributes_from_api' method from the singular
class available from the result set class.
To test:
1. Apply this patch
2. Run:
$ kshell
k$ prove t/db_dependent/Koha/Objects.t
=> SUCCESS: Tests pass!
4. Sign off :-D
5. Yeah, I skipped 3 :-P
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Generic method that changes the attribute names of an object in the
presence of the to_api_mapping method. It otherwise falls back to
returning the TO_JSON output.
This is WIP submitted early for scrutiny. Tests and example usage in an
API controller coming.
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
When a method is not allowed yet (not covered by tests), developers get
an error the is not really useful: "The method XXX is not covered by
tests!"
It would be more useful to have a stack trace in order to know where
this error come from.
Test plan:
Modify some code to display this stack trace and confirm it makes sense
to have it, i.e. it will make debug easier
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
This patch creates the 'build_query_params' helper, instead of the
original function in Koha::Objects.
Unit tests are removed for Koha::Objects::_build_query_params_from_api and
written for the helper plugin.
The objects.search helper gets a call to build_query_params added. Tests for it
updated to match this behaviour change.
To test:
- Apply this patches
- Run:
$ kshell
k$ prove t/Koha/REST/Plugin/Query.t \
t/db_dependent/Koha/Objects.t \
t/db_dependent/Koha/REST/Plugin/Objects.t
=> SUCCESS: Tests pass!
- Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Lari Taskula <lari.taskula@jns.fi>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Lari Taskula <lari.taskula@jns.fi>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Lari Taskula <lari.taskula@jns.fi>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Following the discussion about the best way to make things simpler for
developing the REST api, I quote Lari's email:
"As many other endpoint will have the exact same usage, by looking at your example, I would prefer to avoid writing parameter / pagination / sorting / header handling for each list operation in our API controllers. Do you think it's possible to centralize all of this e.g. by passing $c into a customized search sub? Perhaps in Koha::Objects?
so instead we could have something like (ignore my bad choice of naming)...:
sub list_vendors {
my $c = shift->openapi->valid_input or return;
my $args = $c->validation->output;
my $vendors;
return try {
$vendors = Koha::Acquisition::Booksellers->api_list_search($c);
return $c->render(status => 200, openapi => $vendors);
} catch {
...
}
}"
We all agreed we neeed something like that. Here's a possible implementation. I take
advantage of the previously written Mojo helpers, that are fully covered by tests.
I submit this early so anyone can take a look and gather ideas to make it even better.
I'm already using it (effectively) for the /acquisitions/orders endpoint I'm writing
on bug 18731.
Thanks!
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Lari Taskula <lari.taskula@jns.fi>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch adds ->is_paged to Koha::Objects. It is inherited from the underlying resultset
from DBIC so there's no code besides adding it to the known methods in AUTOLOAD.
Tests are added for the newly exported method.
To test:
- Apply this patch
- Run:
$ sudo koha-shell kohadev
k$ cd kohaclone
k$ prove t/db_dependent/Koha/Objects.t
=> SUCCESS: Tests pass!
- Sign off :-D
Sponsored-by: Camden County
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
The following warning was raised in Letters.t:
DBIx::Class::ResultSource::_minimal_valueset_satisfying_constraint():
NULL/undef values supplied for requested unique constraint 'primary' (NULL
values in column(s): 'id'). This is almost certainly not what you wanted,
though you can set DBIC_NULLABLE_KEY_NOWARN to disable this warning.
This warning is triggered by this line in C4/Letters.pm:
Koha::SMS::Providers->find( $member->{'sms_provider_id'} );
As you already guessed, the sms_provider_id returns undef.
Resolved in sub find by testing if there are parameters and if so, they
should not be all undefined. (In most cases there will be only one
parameter; but this report is about composite keys.)
Added a trivial test case in Objects.t too.
Test plan:
Run t/db_dependent/Koha/Object.t
Run t/db_dependent/Koha/Objects.t
Run t/db_dependent/Letters.t
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Changes the $id parameter to an array. (IssuingRule has three keys.)
The build_object method in TestBuilder.pm has been adjusted to pass
multiple primary key values to find.
Also adjusted the POD section to show more clearly that we accept
the same parameters as DBIx ResultSet does.
Test plan:
Run t/db_dependent/Koha/Object.t
Run t/db_dependent/Koha/Objects.t
Run t/db_dependent/TestBuilder.t
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Reading https://perlmaven.com/how-to-return-undef-from-a-function
this sound like the more correct behaviour.
Considering:
$template->param(
stuff => Koha::Stuffs->find( $id ),
foo => 1,
);
without this patch, if the $id does not represent any rows in the DB,
stuff will be assigned to 'foo' and $foo will be undef in the template.
That can lead to very bad side-effects.
With this patch we make sure that it will never happen again.
Test plan:
prove t/db_dependent/Koha/Objects.t
should return green
Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Reading https://perlmaven.com/how-to-return-undef-from-a-function
this sound like the more correct behaviour.
Considering:
$template->param(
stuff => Koha::Stuffs->find( $id ),
foo => 1,
);
without this patch, if the $id does not represent any rows in the DB,
stuff will be assigned to 'foo' and $foo will be undef in the template.
That can lead to very bad side-effects.
With this patch we make sure that it will never happen again.
Test plan:
prove t/db_dependent/Koha/Objects.t
should return green
Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Test plan:
prove t/db_dependent/Koha/Objects.t
Should return green
Followed test plan, result as expected
Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
We should call slice in list context with the correct indexes.
Test plan:
Run t/db_dependent/Koha/Objects.t
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
DBIx::Class does not provide such method, but it can be handy in some
cases.
Test plan:
prove t/db_dependent/Koha/Objects.t
should return green
Test returned green.
Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Bug 17927 fixes data types on the current REST endpoints. If you
test those endpoints, you will notice trying to access them (for listing
or retrieving single objects yields a data type error. Notably on
booleans but also on integers.
Integers fail due to https://rt.cpan.org/Ticket/Display.html?id=119904
but it needs some global solution until there's a DBD::mysql release
backported to the supported distros. There's the option to use
http://search.cpan.org/~frew/DBIx-Class-Helpers-2.033002/lib/DBIx/Class/Helper/Row/NumifyGet.pm
to get the integer columns fixed as a workaround:
__PACKAGE__->add_columns(
borrowernumber => {
data_type => 'integer',
is_nullable => 0,
is_numeric => 1,
}
);
I didn't find bug reports related to this (maybe because we don't use
warnings everywhere) But I don't think is worth going such a heavy
overhead.
A similar situation takes place for Boolean values. They need to be
prepared for JSON output. This could have been done using DBIx filters
as pointed out by Martin:
__PACKAGE__->filter_column(
lost => {
filter_to_storage => sub { $_[1] ? 1 : 0 },
filter_from_storage =>
sub { $_[1] ? Mojo::JSON->true :
Mojo::JSON->false }
}
);
but this could have other consequences that are worth exploring on
another bug (i.e. it would mean we need to take care of every place
where this boolean data is used/set needs to handle this data types
nicely. Such would be the case if we were a Mojo-only app, but we
aren't. We use Koha::Obect(s) in the whole app. Period.
This patch adds the need to specify on the schema files, columns that
are actually boolean, because we have no way to detect them for now
(i.e. they are all tinyint, but we use tinyint for non-boolean stuff
too).
So if this patch is accepted, we would need to specify boolean columns
like this:
__PACKAGE__->add_columns(
'+lost' => {
is_boolean => 1
}
);
This patch adds a TO_JSON method for Koha::Object(s) to be used for
serializing Koha::Object-derived objects into JSON strings.
To test it (as Koha::Object(s) need to be instantiated) I provide tests
on top of the Koha::Patron(s) classes in the followup patches.
[1] Yes, we use TINYINT(1) for booleans, but from DBIC's perspective
there's no way to read the (1) in runtime.
Sponsored-by: ByWater Solutions
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Returns one and only one object that is part of this set.
Returns undef if there are no objects found.
->single is faster than ->search->next
This is optimal as it will grab the first returned result without instantiating
a cursor.
It is useful for this Bug as we only want to select the top row of found issuing
rules.
To test:
1. Run t/db_dependent/Koha/Objects.t
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This reverts commit c253af2e51.
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
In many cases it should be possible to derive the Koha::Objects class
name from the DBIx result class name in a trivial manner.
If not, the DBIx result class should have a class method called
koha_objects_class providing that non-trivial name, such as Libraries
for Branch.
Note: We are only interested in the plural form here (Koha::Objects).
Test plan:
Run t/db_dependent/Koha/Objects.t
Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Koha::Objects->search_related should return a Koha::Objects-based
object.
This search_related method should follow the same rules as the search
method, i.e. take into account what the caller want (scalar or list).
The problem here is that we do not know (in Koha::Objects) what is the
kind of objects we want to instanciate. To know it, this patch adds a
get_object_class, it will return the class of the object and the
resultset Koha::Object-based object.
The drawback of this method is that we will have to keep it up-to-date
every time we add a new Koha::Object class.
Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
In order to search on relations, we need this new method.
Test plan:
Confirm that the changes in Objects.t make sense and that the tests
pass.
Tested all 3 patches together, followed test plan, result OK
Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
To make sure the methods will be covered by tests and devs won't
overused this method, I think it may be a good idea to restrict its
usage.
Signed-off-by: Hector Castro <hector.hecaxmmx@gmail.com>
AUTOLOAD added. No Koha QA errors. Test passed successfully
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Test plan:
prove t/db_dependent/Koha/Objects.t
should return green
Signed-off-by: Hector Castro <hector.hecaxmmx@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Test plan:
prove t/db_dependent/Koha/Objects.t
should return green
Signed-off-by: Hector Castro <hector.hecaxmmx@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Agustin Moyano
Julian Maurice
Lari Taskula
