Remove:
- BIB_INDEX_MODE and AUTH_INDEX_MODE env var
- bib_index_mode and auth_index_mode options from scripts
- Warnings from about page, just kept one if zebra_bib_index_mode or
zebra_auth_index_mode still exist in config and are set to grs1
Test plan:
- Install Koha from src
- Install Koha from pkg
- Read the code, carefully!
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Rebased
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
To test:
1 - prove -v t/db_dependent/Search.t
2 - Koha asks "Why am I getting these crazy facets!?!"
3 - Apply patch
4 - prove -v t/db_dependent/Search.t
5 - Koha purrs (tests pass)
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
The test should remove the holdingbranch key in the expected results when
there is only one branch.
See the logic in C4::Koha.
Test plan:
[1] Run Search.t on a database with multiple branches.
[2] Run Search.t on a database with one branch. Without this patch the test
would fail: Failed test '_get_facets_info returns the correct data'.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
GetMarcSubfieldStructure: In Biblio.t we are adding a subtest that checks
the structure returned by this routine. Is it a hashref pointing to
arrayrefs of hashrefs? ;)
In Search.t this routine was mocked. The change in the returned structure
is now applied to this mock too (moving the marc tag hashes into
arrayrefs).
GetMarcFromKohaField: In Biblio.t we add a subtest for it. We are checking
if it returns multiple mappings (per kohafield) and calling it in scalar
context too.
GetMarcSubfieldStructureFromKohaField: The existing subtest in Biblio.t is
expanded to also test the call in list context.
TransformKohaToMarc: This sub has its own test script. We are adding a
subtest in TransformKohaToMarc.t for a test with multiple mappings,
and for mapping to a control field in another framework. This also tests
the additional framework parameter.
Additionally, we add a test for the new no_split option used for items.
TransformMarcToKoha: This implicitly tests its helper sub _get_inverted.
This patch adds a new test script for this routine.
TransformMarcToKohaOneField: A few tests are added to the previous new
test script.
Test plan:
[1] Run t/db_dependent/Biblio.t
[2] Run t/db_dependent/Biblio/TransformKohaToMarc.t
[3] Run t/db_dependent/Biblio/TransformMarcToKoha.t
[4] Run t/db_dependent/Search.t
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
TEST PLAN
---------
0) back up database
1) drop database; create database
2) run web installer
3) prove t/db_dependent/Search.t
-- the spammy warnings (other than zebra index warnings) may
already be fixed, but the mocked value was not realistic.
4) apply patch
5) prove t/db_dependent/Search.t
-- notice only zebra index warnings.
6) run qa test tools
Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
The test files do not need to return 1
Patch generated with:
perl -p -i -e "s/^1;\n//xsm" t/**/*.t
Test plan:
git grep '^1;$' t/**/*.t
should not return any results
NOTE: does not fix C4/SIP/t, nor xt tests.
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Note that some are caused by CGI, see bug 18632
Test plan:
Confirm there are less warnings with this patch applied
Signed-off-by: Lee Jamison <ldjamison@marywood.edu>
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
The config tweaks from Search.t break check_sysprefs.t later on.
Clearing the cache resolves that.
Test plan:
[1] Run prove t/db_dependent/Search.t t/db_dependent/check_sysprefs.t (in
one statement) before and after applying this patch.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
A prove t/db_dependent did not fail check_sysprefs.t (only skipping two
tests: 00-strict.t and Koha/IssuingRules.t; not enough patience)
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
When a ccl query is used, the buildQuery subroutine does not handle
the available limit (not an index).
This available limit is handle later in the subroutine.
This affect the author links on the detail page for instance (an=xx).
A much better solution would be to keep an 'available' zebra index up-to-date.
Test plan:
(OPAC or staff interface, it does not matter)
- Launch a search, click on a result and then on an author link to
launch another query (an:xx)
- Limit to available items without the 'facet'
=> Without this patch you won't get any results
=> With this patch applied you should get relevant result (regarding the
known bugs 16970, 13715, 13658, 5463, etc.)
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This patch makes the tests clean the 'config' namespace for cache before
it runs any tests.
To test:
- Run:
$ prove t/db_dependent/Search.t
=> FAIL: Self explanatory
- Apply the patch
- Run:
$ prove t/db_dependent/Search.t
=> SUCCESS: All green
- Sign off :-D
Signed-off-by: Hector Castro <hector.hecaxmmx@gmail.com>
All test pass successfuly
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Cannot reproduce, but I trust everyone telling me it's failing ; )
No regressions spotted
I really don't know how these tests passed before 16431, the mapping was not complete.
Test plan:
prove t/db_dependent/Search.t
should return green
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This patch makes the t/db_dependent/Search.t tests correctly set
'Zebra' for the 'SearchEngine' syspref.
To test:
- On top of bug 12478 branch, run
$ prove t/db_dependent/Search.t
=> FAIL: Tests fail
- Apply the patch
- Run:
$ prove t/db_dependent/Search.t
=> SUCCESS: Tests now pass.
Signed-off-by: Chris <chrisc@catalyst.net.nz>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This patch removes code related to stopwords usage. The following methods are removed:
C4::Search->remove_stopwords
C4::Context->stopwords
C4::Context->_new_stopwords
And the buildQuery API was changed (removed the \@removed_stopwords return value).
A follow-up is provided for database changes, to make rebasing easier.
To test:
- Apply this patch
- Do some searches in both intranet and opac interfaces
- Nothing should break
Sponsored-by: Universidad Nacional de Córdoba
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This patch makes the t/db_dependent/Search.t tests use a real DB instead of
mocking the DB handler (which is not actually used and generates unnecesary
warnings when running the tests).
It does so by grabbing the DB configuration data using C4::Context->config()
and passing it (writing %ENV) to rewrite-config.PL, all is done in zebra_config.pl
which is only used on the Search.t file.
To test:
- Run
$ prove t/db_dependent/Search.t
=> FAIL: Warnings are raised about DB being mocked and DBIC not recognising 'Mock'
as a valid DB driver.
- Apply the patch
- Run
$ prove r/db_dependent/Search.t
=> SUCCESS: Same tests results, no warning about DBIC driver.
- Sign off :-D
Disclamer: As of writing this patch, the Search.t tests pass. The patch is not dealing
with tests results, but how they use the DB.
Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Test pass before & after, a 'little' less noisy
No errors
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
No need to mock the items' columns (?)
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch removes the $facets_info calculation from the _get_facets_data_from_record
sub so it is not done for each record. It introduces a new sub, _get_facets_info
that is called from the getRecords loop, that does the job only once.
To test:
- Apply on top of the previous patches
- Run
$ prove -v t/db_dependent/Search.t
=> SUCCESS: _get_facets_info gets tested and it passes for both MARC21 and UNIMARC.
Facets rendering should remain unchaged on the UI.
- Sign off :-D
Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: Nick Clemens <nick@quecheelibrary.org>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This patch refactors the facet extraction loop into a proper function.
The loop is changed so the MARC::Record objects are created only once
instead of the old/current behaviour: once for each defined facet (in
C4::Koha::getFacets).
To test:
- Apply the patch
=> SUCCESS: verify facets functionality remains unchanged.
- Run:
$ prove -v t/db_dependent/Search.t
=> SUCCESS: tests for _get_facets_data_from_record fail, because
100$a is considered for fields with indicator 1=z (field added
by IncludeSeeFromInSearches syspref).
- Sign off :-D
Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: Nick Clemens <nick@quecheelibrary.org>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as described, passes tests and QA script.
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Changes to searchResults deserve unit tests.
Edit: minor tweaks:
- Remove warning in line 702
- Modernize
- License added
- "This file is a stub" removed
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
$ git grep authoritysep
This showed:
1) the updatedatabase.pl script, as expected,
2) lots of po files,
3) a test file that was missed.
TEST PLAN
---------
1) prove -v t/db_dependent/Search.t
2) Run koha QA test tools
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Work (both patches). Test pass. No koha-qa errors
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch adds a regression test for the condition noted
in bug 9578, where attempting a sort of a Zebra search that
fails because of an invalid query crashes.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch makes Koha <-> Zebra use MARCXML for the serialization when
using DOM, and USMARC for GRS-1.
* The following functions are modified to set the Zebra record syntax
according to the current sysprefs and configuration:
- C4::Context->Zconn
- C4::Context-_new_Zconn
* A new function 'new_record_from_zebra' is introduced, which checks the
context we are in, and creates the MARC::Record object using the right
constructor.
The following packages get touched to make use of the new function:
- C4::Search
- C4::AuthoritiesMarc
and the same happens to the UI scripts that make use of them (both in
the OPAC and STAFF interfaces).
* Calls to the unsafe ZOOM::Record->render()[1] method are removed.
Due to this last change the code for building facets was rewritten. And
for performance on the facets creation I pushed higher version
dependencies for MARC::File::XML and MARC::Record (we rely on
MARC::Field->as_string).
* Calls to MARC::Record->new_from_xml and MARC::Record->new_from_usmarc
are wrapped with eval for catching problems [2].
* As of bug 3087, UNIMARC uses the 'unimarc' record syntax. this case is
correctly handled.
* As of bug 7818 misc/migration_tools/rebuild_zebra.pl behaves like:
- bib_index_mode (defaults to 'grs1' if not specified)
- auth_index_mode (defaults to 'dom')
here we do exactly the same.
To test:
- prove t/db_dependent/Search.t should pass.
- Searching should remain functional.
- Indexing and searching for a big record should work (that's what the
unit tests do).
- Test an index scan search (on the staff interface):
Search > More options > Check "Scan indexes".
- Enable 'itemBarcodeFallbackSearch' and try to circulate any word, it
shouldn't break.
- Searching for a biblio in a new subscription shouldn't break.
- Running bulkmarcimport.pl shouldn't break.
- And so on... for the rest of the .pl files.
[1] http://search.cpan.org/~mirk/Net-Z3950-ZOOM/lib/ZOOM.pod#render()
[2] a record that cannot be parsed by MARC::Record is simply skipped (bug 10684)
Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This makes it easier to read. Also fixes the test count, and is rebased
against 11499 which is needed to correctly test the patches for this bug
(i.e. indexing mode needs to be correctly set in koha-conf.xml)
To test:
[1] Run prove -v t/db_dependent/Search.t
Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This adds direct tests of indexing and retrieving
bib records that are larger than the ISO2709 limit
of 99,999 octets.
To test: verify that prove -v t/db_dependent/Search.t passes.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch fixes an issue where chosing 'None' as the sort order
for an authority search would result in zero hits if QueryParser is
eanbled.
This patch also adds some additional test cases.
To test:
[1] Enable QueryParser.
[2] Perform an authority search in the staff interface that
uses 'Heading A-Z' as the sort order and returns hits.
[3] Run the same search, but with the sort order set to 'None'.
No hits are returned.
[4] Apply the patch.
[5] Do step 3 again. This time, hits should be returned.
[6] Verify that prove -v t/db_dependent/Search.t passes.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch duplicates the SearchAuthority tests, with QP enabled.
Searching shouldn't be affected by QP. Instructions for manually
reproducing this can be read in the first comment for the bug entry.
Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
t/db_dependent/Search.t creates its own temporary koha-conf.xml and
related files, and it fails to properly set indexing modes for
bibliographic and authority records.
This patch adds regression tests for that.
To test:
$ prove -v t/db_dependent/Search.t 2> /dev/null | grep 11499
The test should fail unless the main page is applied.
Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Under certain circumstances, namely where QueryParser is off,
QueryWeightFields is on, and QueryFuzzy and QueryAutoTruncate are
off, a search with a double-quote embedded in it can fail: for
example
web application
would return results where
web "application
does not.
This patch adds an automated regression test.
To test:
[1] Apply this patch and run prove -v t/db_dependent/Search.t;
two tests should fail.
[2] After applying the main patch that fixes the bug, the
prove -v t/db_dependent/Search.t should succeed.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch adds a regression test for Koha's ability
to gracefully handle records that Zebra will index but
which MARC::Record won't parse. In particular, MARC::File::USMARC
croaks on records that contain non-alphanumeric characters in the
tag name.
To test:
[1] After applying this test, run prove -v t/db_dependent/Search.t
The test script will die when it gets to the regression test.
[2] After applying the main patches that fix the bug, the test
script should pass.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
QueryParser was loading redundant search aliases, which in turn resulted
in certain queries involving two or more clauses joined by a Boolean
operator getting parsed in correctly.
To test:
[1] Enable QueryParser.
[2] Arrange your database so that some some bib records contain
the keyword 'history' while a smaller subset contain
both 'history' and 'earth'. (The exact words used don't
matter.)
[3] Run the following search from the OPAC global search bar:
kw:history && kw:earth
[4] Observe that the records that are returned include *all* bibs
in the database with the keyword 'history'. In other words, the
restriction that the records must also contain 'earth' is not
observed.
[5] Apply the patch and rerun the search.
[6] This time, the results that are returned should include only those
that contain both 'history' and 'earth'.
[7] Verify that prove -v t/QueryParser.t passes.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes test plans, all tests and QA script.
Also checked the db_dependent/Search.t tests.
Tried some other searches, all seems to be working correctly.
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
I reproduce the bug and I confirm this patch fixes it.
Without the patch, kw:history && kw:earth, I got:
Search biblios OK 4 1 1+0 RPN @attrset Bib-1 @attr 4=6 @attr 1=1016 history
With the path, I get:
Search biblios OK 2 1 1+0 RPN @attrset Bib-1 @and @attr 4=6 @attr 1=1016 history @attr 4=6 @attr 1=1016 earth
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch adds regression tests to verify that
Boolean searches using QueryParser function correctly.
This patch also ensures that QP is correctly initalized
when Search.t is run.
To test:
[1] Apply this patch and the following patch.
[2] Verify that prove -v t/QueryParser.t works
[3] Verify that prove -v t/db_dependent/Search.t works
[4] (optional) instead of applying both patches at the same
time, apply only the regression test patch and run the
tests listed in steps 2 and 3. The following tests
should fail:
t/db_dependent/Search.t (Wstat: 512 Tests: 198 Failed: 2)
Failed tests: 42, 71
Non-zero exit status: 2
t/QueryParser.t (Wstat: 256 Tests: 28 Failed: 1)
Failed test: 12
Non-zero exit status: 1
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch adds a regression test for doing a form-based catalog
search using an index that contains the string 'ns' in its name. In
this case, the index being tested is 'Code-institution'.
To test
-------
[1] Prove -v t/db_dependent/Search.t should have two failed tests;
one each for GRS-1 and DOM testing.
[2] After applying the main patch for this bug, the tests should
pass.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch enhances t/db_dependent/Search.t to test indexing and
searching of authority records. It supplies a sample authority file
for MARC21 (consisting of the record for William Shakespeare from the
Library of Congress) and a sample UNIMARC authority file supplied
by Henri-Damien Laurent.
It also adds tests for both MARC21 and UNIMARC authorities.
When the main patch for bug 7421 is applied, t/db_dependent/Search.t
should pass.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch adds a regression test that verifies that searching the
UNIMARC 'item' index for a value that is indexed by virtue of being
included in the fallback index definition for the 995 field works for
both GRS-1 and DOM.
The main patch will allow t/db_dependent/Search.t to pass.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch adds a few basic search and indexing tests for UNIMARC,
using sample data provied by Frédéric Demians. The new tests
by no means completely cover the possibilities, but should be
considered a starting point for future tests.
To test:
[1] Verify that prove -v t/db_dependent/Search.t passes.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch refactors t/db_dependent/Search.t so that it can be
subsequently modified to add tests for indexing and searching
UNIMARC records without duplicating the setup code.
This patch also moves the MARC21 records from t/db_dependent/data/zebraexport
to t/db_dependent/data/marc21/zebraexport so that UNIMARC sample
data can be slotted in t/db_dependent/data/unimarc/zebraexport.
To test:
[1] Verify that prove -v t/db_dependent/Search.t passes.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
There was some churn in previous patches about the desired
name of the music publisher number index, so this patch
adds tests to confirm that both the old and new names
work as CCL keywords.
To test:
[1] Verify that t/db_dependent/Search.t passes.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch expands t/db_dependent/Search.t to run
the same tests using both the GRS-1 and DOM indexing
modes. It also adds hooks in zebra_config.pl to make
it easier to stage test cases for non-MARC21 Zebra
indexing.
Note that in DOM mode one of the tests is currently a
TODO, as relevance ranking for wegihted queries differs
between GRS-1 and DOM.
To test:
[1] Verify that prove -v t/db_dependent/Search.t passes.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch updates the wthdrawn field in items and deleteditems to be
withdrawn instead. No functional changes are made.
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Comment: Save for translation files (that will be fixed on next release),
only occurrence of wthdrawn is on updatedatabase.pl
No koha-qa errors.
This touch many files, and I did not test everything,
but all seems normal. I think that any problem could
be fixed later.
Perhaps both entries in updatedatabase.pl could be joined
into one, but thats for QA.
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
