Browse Source
When batch editing, 2 reindex calls are sent to ES/Zebra.
We can easily avoid that reusing the skip_modzebra_update (renamed skip_record_index)
Additionally we should only send one request for biblio, and we should
only do it if we succeed
As the whole batch mod is in a transaction it is possible to fail in which case
Zebra queue is reset, but ES indexes have already been set
In addition to the skip param this patchset moves Zebra and Elasticsearch calls to
Indexer modules and introduces a generic Koha::SearchEngine::Indexer so that we don't
need to check the engine when calling for index
The new index_records routine takes an array so that we can reduce the calls to
the ES server.
The index_records routine for Zebra loops over ModZebra to avoid affecting current behaviour
Test plan:
General tests, under both search engines:
1 - Add a biblio and confirm it is searchable
2 - Edit the biblio and confirm changes are searchable
3 - Add an item, confirm it is searchable
4 - Delete an item, confirm it is not searchable
5 - Delete a biblio, confirm it is not searchable
6 - Add an authority and confirm it is searchable
7 - Delete an authority and confirm it is not searchable
Batch mod tests, under both search engines
1 - Have a bib with several items, none marked 'not for loan'
2 - Do a staff search that returns this biblio
3 - Items show as available
4 - Click on title to go to details page
5 - Edit->Item in a batch
6 - Set the not for loan status for all items
7 - Repeat your search
8 - Items show as not for loan
9 - Test batch deleting items
a - Test with a list of items, not deleting bibs
b - Test with a list of items, deleting bibs if no items remain where all items are only item on a biblio:
SELECT MAX(barcode) FROM items GROUP BY biblionumber HAVING COUNT(barcode) IN (1)
c - Test with a list of items, deleting bibs if no items remain where some items are the only item on a biblio:
SELECT MAX(barcode) FROM items GROUP BY biblionumber HAVING COUNT(barcode) IN (1,2)
10 - Confirm records are update/deleted as appropriate
Signed-off-by: Bob Bennhoff <bbennhoff@clicweb.org>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Bug 25265: Rename skip_modzebra_update to skip_record_index
Signed-off-by: Bob Bennhoff <bbennhoff@clicweb.org>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Bug 25265: Fix copy paste error for parameter
Signed-off-by: Bob Bennhoff <bbennhoff@clicweb.org>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Bug 25265: (follow-up) Don't index malformed records
This is analogous to 26522, we shoudl skip record that cannot be retrieved for indexing
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Bug 25265: (QA follow-up) Add shebang to Indexer.t
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Bug 25265: (QA follow-up) Rename biblionumber in ModZebra, index_records
ModZebra:
The name is very misleading: we can index authid's too here.
And yes, it should not be in C4/Biblio too ;) A first step..
Adding the same change here in Koha/SearchEngine/Zebra/Indexer.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Bug 25265: (QA follow-up) Check server type in Elasticsearch::index_records
Doing the same change as previously (renaming biblionumber), but fixing
at the same the record fetch. If (theoretically) an authority is passed
without a record, it would have fetched a biblio record.
Test plan:
You need Elasticsearch here.
Replaced this line in AddAuthority:
$indexer->index_records( $authid, "specialUpdate", "authorityserver", $record );
by
$indexer->index_records( $authid, "specialUpdate", "authorityserver", undef );
And updated an authority record. Check if you can search for the change.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
20.05.x
12 changed files with 282 additions and 84 deletions
@ -0,0 +1,59 @@ |
|||
package Koha::SearchEngine::Indexer; |
|||
|
|||
# This file is part of Koha. |
|||
# |
|||
# Copyright 2020 ByWater Solutions |
|||
# |
|||
# Koha is free software; you can redistribute it and/or modify it |
|||
# under the terms of the GNU General Public License as published by |
|||
# the Free Software Foundation; either version 3 of the License, or |
|||
# (at your option) any later version. |
|||
# |
|||
# Koha is distributed in the hope that it will be useful, but |
|||
# WITHOUT ANY WARRANTY; without even the implied warranty of |
|||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
|||
# GNU General Public License for more details. |
|||
# |
|||
# You should have received a copy of the GNU General Public License |
|||
# along with Koha; if not, see <http://www.gnu.org/licenses>. |
|||
|
|||
# This is a shim that gives you the appropriate search object for your |
|||
# system preference. |
|||
|
|||
=head1 NAME |
|||
|
|||
Koha::SearchEngine::Indexer - instantiate the search object that corresponds to |
|||
the C<SearchEngine> system preference. |
|||
|
|||
=head1 DESCRIPTION |
|||
|
|||
This allows you to be agnostic about what the search engine configuration is |
|||
and just get whatever indexer object you need. |
|||
|
|||
=head1 SYNOPSIS |
|||
|
|||
use Koha::SearchEngine::Indexer; |
|||
my $searcher = Koha::SearchEngine::Indexer->new({ index => Koha::SearchEngine::Index}); |
|||
|
|||
=head1 METHODS |
|||
|
|||
=head2 new |
|||
|
|||
Creates a new C<Search> of whatever the relevant type is. |
|||
|
|||
=cut |
|||
|
|||
use Modern::Perl; |
|||
use C4::Context; |
|||
use C4::Biblio qw//; |
|||
|
|||
sub new { |
|||
my $engine = C4::Context->preference("SearchEngine") // 'Zebra'; |
|||
my $file = "Koha/SearchEngine/${engine}/Indexer.pm"; |
|||
my $class = "Koha::SearchEngine::${engine}::Indexer"; |
|||
require $file; |
|||
shift @_; |
|||
return $class->new(@_); |
|||
} |
|||
|
|||
1; |
|||
@ -0,0 +1,65 @@ |
|||
package Koha::SearchEngine::Zebra::Indexer; |
|||
|
|||
# Copyright 2020 ByWater Solutions |
|||
# |
|||
# This file is part of Koha. |
|||
# |
|||
# Koha is free software; you can redistribute it and/or modify it |
|||
# under the terms of the GNU General Public License as published by |
|||
# the Free Software Foundation; either version 3 of the License, or |
|||
# (at your option) any later version. |
|||
# |
|||
# Koha is distributed in the hope that it will be useful, but |
|||
# WITHOUT ANY WARRANTY; without even the implied warranty of |
|||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
|||
# GNU General Public License for more details. |
|||
# |
|||
# You should have received a copy of the GNU General Public License |
|||
# along with Koha; if not, see <http://www.gnu.org/licenses>. |
|||
|
|||
use Modern::Perl; |
|||
use C4::Biblio qw(ModZebra); # FIXME This is terrible, we should move the indexation code outside of C4::Biblio |
|||
use base qw(Class::Accessor); |
|||
|
|||
=head1 NAME |
|||
|
|||
Koha::SearchEngine::Elasticsearch::Indexer - handles adding new records to the index |
|||
|
|||
=head1 SYNOPSIS |
|||
|
|||
my $indexer = Koha::SearchEngine::Zebra::Indexer->new(); |
|||
$indexer->index_records( $record_numbers, $op, $server, $records); |
|||
|
|||
|
|||
=head1 FUNCTIONS |
|||
|
|||
=head2 new |
|||
|
|||
This is a dummy function to create the object. C4::Biblio->ModZebra is doing the real work |
|||
now and needed variables are passed to index_records |
|||
|
|||
=cut |
|||
|
|||
sub new { |
|||
my $class = shift @_; |
|||
my $self = $class->SUPER::new(@_); |
|||
} |
|||
|
|||
=head2 index_records($record_numbers, $op, $server, $records) |
|||
|
|||
This is simply a wrapper to C4::Biblio::ModZebra that takes an array of records and |
|||
passes them through individually |
|||
|
|||
The final parameter $records is not used in Zebra, it exists for parity with Elasticsearch calls |
|||
|
|||
=cut |
|||
|
|||
sub index_records { |
|||
my ( $self, $record_numbers, $op, $server, $records ) = @_; |
|||
$record_numbers = [$record_numbers] if ref $record_numbers ne 'ARRAY' && defined $record_numbers; |
|||
foreach my $record_number ( @$record_numbers ){ |
|||
ModZebra( $record_number, $op, $server ); |
|||
} |
|||
} |
|||
|
|||
1; |
|||
Loading…
Reference in new issue