If the staff search results, if an item is both checked out and lost,
the result will appear as two item lines where one line has the lost
status and the other line has the rest of the item's data.
Test Plan:
1) Check an item out to a patron
2) Mark the item as lost *without* removing the item from the patron's
record, either by using longoverdue.pl or by editing the itemlost
field in the database directly.
3) Perform a search where that item will be in the results
4) Note the improper display of the item's data
5) Apply this patch set
6) Reload the search restults
7) Note the item now displays correctly
Signed-off-by: Nick <nick@quecheelibrary.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Nick <nick@quecheelibrary.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Any search limits including a ccode will break the search facts.
Test Plan:
1) Run an advanced search using a ccode limit
2) Try using any of the facet links of the left
3) Note they are broken
4) Apply this patch
5) Refresh the results
6) Note the facet links are no longer broken
Note: I have not been able to reproduce this issue on my own test
system, but have noted the problem on at least a dozen Koha servers.
I could not reproduce the bug either, but I saw it on the Bywater Demo (comment #1).
Applied patch and tested facets, no problems found, signing off
Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Can not reproduce the problem, but I can also not find a regression.
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This enhancement adds the ability to search on all isbn variations when
searching on the isbn index.
Test plan:
0/ Don't apply the patch
1/ Create or choose a notice with an isbn with dashes.
2/ Try to search the notice using the isbn index by it isbn without
dashes.
=> It does not work.
3/ Apply the patch, enable the new pref SearchWithISBNVariations and
disable UseQueryParser.
4/ repeat 2 and note that the record is now returned.
Note that this only works if UseQueryParser is disabled.
It looks like QueryParser does not manage more than 1 operator.
See:
QueryParser does not manage more than 1 operator?
http://lists.koha-community.org/pipermail/koha-devel/2014-December/041028.html
and
commit 036f2a50e1
Author: Galen Charlton <gmc@esilibrary.com>
Date: Mon May 5 19:31:00 2014 +0000
Bug 10500: (follow-up) disable AggressiveMatchOnISBN if
UseQueryParser is on
Signed-off-by: Morag Hills <the.invinnysible.one@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Imagine this scenario: we have one record with four items. Two of those
items are checked out, one of those items is a waiting hold, and one of
those items is available. We would expect to see this on the search
results page. Instead, we will see both non-checked out items as
unavailable due to waiting holds.
This is due to a semantic issue GetReserveStatus.
C4::Search::searchResults uses GetReserveStatus to get the reserve
status of each item, but unlike all other calls to the sub, this one
passes in not only itemnumber, but biblionumber.
When no reserve is found for the available item, the subroutine uses the
biblionumber to grab what is essentially an arbitrary reserve to use for
the status. This makes no sense and this functionality should be
entirely removed from the subroutine so regressions like this will be
prevented in the future.
Test Plan:
1) Create one record with 4 items
a) check two of the items out to patrons
b) set one of the items as a waiting hold
c) leave the fourth item as available
2) Run a search where this record will be in the results list
3) Note that the results list 2 items on loan, two unavailable
4) Apply this patch, reload the search results
5) Note that the results list 1 available, 2 on loan, 1 unavailable
Signed-off-by: John Andrews <jandrews@washoecounty.us>
Signed-off-by: Sheila Kearns <sheila.kearns@state.vt.us>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Note: This is for the staff search result list!
Works as expected.
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
The search patch should fix non-latin character searches.
Signed-off-by: Paola Rossi <paola.rossi@cineca.it>
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Dobrica Pavlinusic <dpavlin@rot13.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
To reproduce, edit, index notice with utf-8 char and search for it
Signed-off-by: Paola Rossi <paola.rossi@cineca.it>
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Dobrica Pavlinusic <dpavlin@rot13.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
See the wiki page for the explanation.
Signed-off-by: Paola Rossi <paola.rossi@cineca.it>
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Dobrica Pavlinusic <dpavlin@rot13.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
[New commit on 18 Aug 2014 : rebased, and DOM indexing only]
Issues to fix :
Most of 6XX may contain a $2 that identifies the system used for indexing. It should not be indexed.
In French libraries, $2 contains "rameau". So searching books about the music composer "Rameau" retreive thousands of records!
For some 6XX fiels, other subfields should not be indexed, for example dates of persons and family, or adresses.
In Unimarc guide, 600$t,601$t,602$t are said to exist but to be "not used". I keep them indexed.
Additionnally, subject indexing could be improved by using specific indexes for each 6XX if possible :
In ccl.properties :
- su-to, su-geo and su-ut are defined as aliases of Subject.
- a specific index is defined, but not used in record.abs : Subject-name-personal, alias su-na
We can use these indexes and create new specific indexes by using existing bib1 attributes.
We could also index $j,$x,$y,$z subdivision in specific indexes.
This patch does the following changes :
1) For all 6XX : Not indexing $2 (LSCH, Rameau...), $3 and $5
2) Suppressing the indexing of some specific subfields, depending on the field:
600 : Personal name used as a subject // see Marc21 600
not indexing c (additional elements),f (dates),p (address/affiliation)
602 : Family name used as a subject // see Marc21 600 3X
not indexing f (dates)
616 : Trademark
not indexing c,f
3) For all 6XX : index $j,$x,$y,$z in several indexes in addition to the specfific index for their 6XX field:
4) Define in ccl.properties some specific indexes :
Subject-name-conference 1=1073 => alias su-conf
Subject-name-corporate 1=1074 => alias su-corp
Subject-genre-form 1=1075 => alias su-genre and su-form
Subject-geographical 1=1076 => alias su-geo
Subject-chronological 1=1077 => alias su-chrono
Subject-title 1=1078 => alias su-ut and su-ti
Subject-topical 1=1079 => alias su-to
5) Adding new aliases in Search.pm :
su-chrono, su-form, su-genre, su-corp, su-conf, su-ti
6) Using these new indexes in for
600 : Subject and Subject-Personal-Name ; all subfields except subdivisions in Personal-name
601 : Subject, Subject-name-conference and Subject-name-corporate and Subject-name-conf ; all subfields except subdivisions in Corporate-name and Conference-name
602 : same as 600 but could be improved later
604 : Subject and Subject-title ; $a in Subject-Personal-Name ; all subfields except subdivisions in Name-and-Title
605 : Subject and Subject-title
606 : Subject and Subject-topical
607 : Subject and Subject-geographical ; all subfields except subdivisions in Name-geographic
608 : Subject and Subject-genre-form
To test :
A. In a UNIMARC-DOM indexing environment
1) Apply the patch
2) Rebuild zebra
3) Create a record A with some values in critical fields, for example:
- the string "test9828" in 600$c 600$f 600$p, 602$f, 616$c, 616$f, 606$2,600$2
- the string "subform" in 600$j
4) Create a record B with the string "subgeo" in 606$y
5) Create a record C with the string "subdate" in 606$z
6) try to search "su:test9828". You should have no results
7) try to search "su-genre:subform". You should have 1 result : record A
8) try to search "su-geo:subgeo". You should have 1 result : record B
9) try to search "su-chrono:subdate". You should have 1 result : record C
10) on existing records, try su-ut, su-to, su-na, su-form, su-corp, su-geo indexes, and see it results are relevant
Indexing of subjects could maybe be improved later
Signed-off-by: Nick Clemens <nick@quecheelibrary.org>
All seems to work as expected, I am not super-familiar with UNIMARC but I wonder if in su-corp and su-conf the subdivisions might be useful (e.g. France-Gendarmie / Staatsbibliothek-Berlin)
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
On the 23 July development meeting it was decided to formally deprecate
GRS-1 indexing mode for Zebra. This patch makes code fallback to DOM
on the remaining places. No behaviour change should be noticed, as DOM
has been the default for a while.
Regards
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes tests and QA script.
Also checked running Makefile.PL
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This patch makes _get_facet_from_result_set rely on a new syspref (FacetsMaxCount)
to set the max facets to show for each facet category. It defaults to 20 if
the syspref is absent or empty.
To test:
- Have a search with lots of facet results (with some category showing the "See more" link).
- Jump to "See more", notice it shows more than 20 facet values.
- Apply the patch, reload the page.
=> SUCCESS It only shows 20 (default hardcoded value)
- Change the FacetsMaxCount syspref to other value (e.g. 15 or 100).
- Reload
=> SUCCESS: it shows the expected amount.
- Sign off :-D
Regards
Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Test plan completed successfully
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Passes tests and QA script, works as described.
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This patch fixes the "Publication date" and "Acquisition date"
searches when using non-QueryParser and QueryParser searches.
It adds structure attributes to the template, which is consistent
with how phrase searching is currently handled.
It removes unnecessary code from Search.pm, adds some necessary
code which is consistent with existing code, and adds a lot of
explanatory comments.
_TEST PLAN_
Before applying:
0) Turn off QueryParser
1) Try a "Publication date" or "Acquisition date" search from the
staff client advanced search.
2) Note that even though the description on the result page makes
it seem like you're doing an index-specific search, you're actually
doing a keyword search. You can verify this by checking the 008
from positions 7 to 10 for "Publication date" or "Date accessioned"
on items for "Acquisition date".
3) Turn on QueryParser
4) Try doing the searches from Step 1.
5) A "Publication date" search should probably produce zero results
After applying patch:
6) Keep QueryParser on
7) Try doing the searches from Step 1.
8) Notice that you're actually getting results consistent with
your search (ie the 008/7-10 shows the date you searched for,
and there is a "Date accessioned" in items which matches your
search)
9) Turn off QueryParser
10) Note that your results are exactly the same as step 8
(N.B. this is because QueryParser is falling back to non-QP mode
instead of producing a bad query.)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This patch adds a variable to koha-conf.xml controlling the use of Zebra facets.
Usage:
- use_zebra_facets = 1 | 0
Zebra facets work only on DOM.
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This patch adds the following routines to C4::Search
- GetFacets
This is a wrapper routine, that given a ZOOM::ResultSet, extracts
the relevant facets. To do the job, it uses the internal functions:
_get_facets_from_records and _get_facets_from_zebra. The choice is done
on querying the indexing mode: grs1 will use records, and dom zebra's facets.
- _get_facets_from_records
Just refactoring the already existent main loop from getRecords into a function.
- _get_facets_from_zebra
Given a result set, loop through all defined facets in C4::Koha::getFacets
and call _get_facet_from_result_set for each, to build the facets
information. It retrieves the facets from Zebra's facets.
- _get_facet_from_result_set
Given a result set and a facet index name, retrieve the facets
for the given index, and build the result for rendering.
To test this preliminay work:
- Apply the patches, install on DOM, using MARC21, NORMARC and UNIMARC.
- Reindex some DB with lots of records.
- Check facets work.
Note: UNIMARC is the only dialect that has more than one subfield (concatenated)
for facets values, so it is better to test on uNIMARC. The approach leaves room
for re-thinking facets in MARC21/NORMARC, but it is outside of the scope of the bug
(e.g. we could have more author facets)
Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: David Cook <dcook@prosentient.com.au>
Seems to work with DOM and MARC21.
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This patch removes the $facets_info calculation from the _get_facets_data_from_record
sub so it is not done for each record. It introduces a new sub, _get_facets_info
that is called from the getRecords loop, that does the job only once.
To test:
- Apply on top of the previous patches
- Run
$ prove -v t/db_dependent/Search.t
=> SUCCESS: _get_facets_info gets tested and it passes for both MARC21 and UNIMARC.
Facets rendering should remain unchaged on the UI.
- Sign off :-D
Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: Nick Clemens <nick@quecheelibrary.org>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This patch adds a test for field 100, to skip it on facet calculation
if ind1=z.
To test:
- Have IncludeSeeFromInSearches set.
- Create a biblio record, when adding an author, create a new authority record
that contains a 400$a field (see from).
- Rebuild zebra db.
- Search for the record making sure the search returns more than one record.
=> FAIL: the facets contain the 'see from' field.
- Run
$ prove -v t/db_dependent/Search.t
=> FAIL: it fails
- Apply the patch
- Run
$ prove -v t/db_dependent/Search.t
=> SUCCESS: it passes
- Re-run the search, notice the 'see from' doesn't show anymore on the facets.
- Sign off :-D
Edit: minor stylistic change
Regards
To+
Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: Nick Clemens <nick@quecheelibrary.org>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This patch refactors the facet extraction loop into a proper function.
The loop is changed so the MARC::Record objects are created only once
instead of the old/current behaviour: once for each defined facet (in
C4::Koha::getFacets).
To test:
- Apply the patch
=> SUCCESS: verify facets functionality remains unchanged.
- Run:
$ prove -v t/db_dependent/Search.t
=> SUCCESS: tests for _get_facets_data_from_record fail, because
100$a is considered for fields with indicator 1=z (field added
by IncludeSeeFromInSearches syspref).
- Sign off :-D
Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: Nick Clemens <nick@quecheelibrary.org>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as described, passes tests and QA script.
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Some libraries would like the ability choose to display the home branch
on search results while having circulation rules based on the holding
branch. This is currently impossible because both the display of the
branch in search results, and the selection of the home or holding
branch for circulation rules are controlled by the same system
preference: HomeOrHoldingBranch. This preference is described as being
used only for circulation rules, and makes no mention of its use for
display control. The display control should be split off into a separate
preference.
Test Plan:
1) Apply this patch
2) Run updatedatabase.pl
3) Note the value of the new system preference StaffSearchResultsDisplayBranch
matches the current value of HomeOrHoldingBranch
4) Set the preference to home branch
5) Perform a staff catalog search with results having items with differing home and
holding branches.
6) Note the home branch displays
7) Set the preference to holding branch
8) Repeat step 5
9) Note the holding branch displays
Signed-off-by: Jason Robb <jrobb@sekls.org>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as described, logic is now tied to a new system preference.
Passes tests and QA script.
A use C4::Charset was added deep in the body of the code
we have already imported it at the top of the file
(the by convention normal place) As use is executed at compile time
specifying it in the code body does not serve a
useful purpose and detracts from the readability of an already
overly complex subroutine.
Remove the superfluous statement
also removed the tabs introduced to the surrounding lines
by the same commit
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Search still works, no errors.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
It's quite common to have [something] within facet data, and it produces following error:
Unmatched [ in regex; marked by <-- HERE in m/^[ <-- HERE
This problem was intoduced in Bug 12151 but is trivial to fix.
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Good catch.
To test:
- Created a bibliographic record, linked to an authority record (personal name). Did a search that returned the author as a facet.
- Added a [ symbol to the author name.
- Repeated the search
=> FAIL: "Unmatched [ in regex; marked by <-- HERE in m/^[ <-- HERE"
- Apply the patch
- Retry the search
=> SUCCESS: No error, bracket shows correctly.
Passes koha-qa.pl too.
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This patch removes the use of smartmatch operators in the search code.
Regards
To+
Edit: this revision uses 'grep' instead of Lists::MoreUtils::any
Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.
Tested search, no problems found.
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
On the cataloguing search (cataloguing/addbook.pl), if an item has a
notforloan value > 0, the item is not listed in the Location column.
It is quite confusing, the current behavior let patrons believe that
there is not item for the biblio (or less than the real count).
Test plan:
1/ Create 2 biblio records A and B
2/ Create some items for A
3/ Create 1+ item(s) for B with a notforloan status > 0
4/ Reindex both records
5/ Launch a search on the cataloguing module and verify that the
notforloan items are not listed in the 'Location' column.
6/ Apply this patch and verify the not for loan items are listed ("Not
for loan (XXX)").
Signed-off-by: Nicole C. Engard <nengard@bywatersolutions.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes tests and QA script, not for loan items now show up.
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This patch reduces three repeated code fragments into a single
internal subroutine, which is easier to read, has comments,
and should make it easier to refactor more buildQuery code
in the future.
_TEST PLAN_
Before applying
1) Run a bunch of different searches in the staff client and OPAC
in separate tabs
2) Apply the patch
3) Run the same searches again (maybe in yet more tabs) and notice
that the results are exactly the same.
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Same results, no errors.
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
The Zoom specification defines that a ScanSet should provide a way
to retrieve terms suitable for displaying and another one for using
on further searches [1].
The Net::Z3950::ZOOM implementation actually provides both [2] but we
were using the wrong one.
Using $scanset->display_term(...) instead of $scanset->term(...) fixes
the problem.
To test:
- Do a index scan search (advanced search > more options > check
'index scan')
- Notice non-latin characters are replaced by one or more '@' symbols.
- Apply the patch
- Re-do the search, everything shows as it should.
- Try to follow any of the terms (clicking on them) and notice that
it actually gives you relevant results (i.e. is not searching for
@!!!!).
[1] http://zoom.z3950.org/api/zoom-1.4.html#3.6.3
[2] http://search.cpan.org/~mirk/Net-Z3950-ZOOM/lib/ZOOM.pod#term()_/_display_term()
Sponsored-by: Universidad Nacional de Cordoba
Followed test plan. Patch behaves as expected.
Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
I reproduced the issue and I confirm this patch fixes it.
I put "Fuß" in a title, reindex the record. Launch a search on Title
checking the "scan index" checkbox. And the non-latin characters are
well displayed.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Like biblio, this feature provides an authority search history.
This history is available for connected and disconnected user.
If the user is not logged in Koha, the history is stored in an
anonymous user sessin.
The search history feature is now factorized in a new module.
This patch adds:
- 1 new db field search_history.type. It permits to distinguish the
search type (biblio or authority).
- 1 new module C4::Search::History. It deals with 2 different storages:
DB or cookie
- 2 new UT files: t/Search/History.t and t/db_dependent/Search/History.t
- 1 new behavior: the 'Search history' link (on the top-right corner of
the screen) is always displayed.
Test plan:
1/ Switch on the 'EnableOpacSearchHistory' syspref.
2/ Go on the opac and log out.
3/ Launch some biblio and authority searches.
4/ Go on your search history page.
5/ Check that all yours searches are displayed.
6/ Click on some links and check that results are consistent.
7/ Delete your biblio history searches.
8/ Delete your authority searches history searches.
9/ Launch some biblio and authority searches
10/ Delete all your history (cross on the top-right corner)
11/ Check that all your history search is empty.
12/ Launch some biblio and authority searches.
13/ Login to your account.
14/ Check that all previous searches are displayed.
15/ Launch some biblio and authority searches.
16/ Check that these previous searches are displayed under "Current
session".
17/ Play with the 4 delete links (current / previous and biblio /
authority).
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
All patches together pass QA script and tests.
Also, new tests in t/db_dependent/ pass.
Tested in all 4 OPAC themes, being logged in and anonymous.
Anonymous search history will be appended to personal search
history after logging in.
Also verified that cleanup_database still purges search history,
now also including the authority searchs.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch adds :w and :p versions to the index for »Lexile number«
(it has only :n so far) and adds indexes for 653 (Index term
uncontrolled), 655 (Index term Genre/Form), 041 (language-audio) and
041 (language-subtitle). It also adds the »curriculum«-index to
Search.pm.
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
When using QueryWeightFields to add ranking on a search without index,
the search actually uses:
- rank 1 : Title-cover,ext : exact title-cover
- rank 2 : ti,ext : exact title
- rank 3 : Title-cover,phr : phrase title-cover
- rank >7 : queries without index
This relevance sets title as phrase in priority and then any index.
This patch adds title as words list before search on any index, so
that records with all searched terms in title, even not well ordered,
are more relevant.
Test plan :
- Enable QueryWeightFields syspref
- Perform a search, with sort by relevance, with two words ofen
contained in title, but never one near the other.
For example: 'History France'
=> Records with both words in title are first. For example:
"Histoire de France" and "La France : 100 ans d'histoire"
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Relevance ranking and field weighting are hard to test,
as many MARC fields are indexed into the used indexes.
If we had an index that only indexed 245$a/200$a the
effect might be more visible.
I found no regressions by this patch, change reads
logical.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
When searching with a sort (means not by relevance) and there is an error
in Zebra connexion (server is down or query is wrong), you get the message :
Error : Can't call method "sort" on an undefined value at /home/kohaadmin/src/C4/Search.pm line 405.
This patch corrects by not performing sort if there are no no results.
Steps to reproduce the error without patch:
In OPAC go to Advanced Search
Choose "Title" in first "Search for:" end enter "ccl=( and )"
Display "More options"
Set "Sort by" to "Title (A-Z)"
Click "Search" at bootom of page
Result:
Error:
Can't call method "sort" on an undefined value at /usr/share/kohaclone/C4/Search.pm line 430.
After applying the patch, try that search again. This time,
it should report not results found with out the error message.
Alternative Test plan :
- Set OPACdefaultSortField on something else than relevance
- Perform a simple search with a wrong CCL query. For example : ccl=( and )
=> You get the messge : No results found ...
Patch behaves as expected.
Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Adds another check to prevent a bad Zebra error message.
Works as described, passes all tests and QA script.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch fixes two problems with the generation of
links to execute a Z39.50 search from the staff client
catalog and cataloguing search results page.
First, if using URI::Escape 3.30 or earlier, performing a simple search
with a double quote (e.g., "histoire algerie"), the Javascript is broken
in results page because of :
function GetZ3950Terms(){
var strQuery="&frameworkcode=";
strQuery += "&" + "title" + "=" + ""histoire%20algerie"";
Second, the encoding of non-ASCII characters in the search
term was broken.
This patch moves URI escaping from Perl to template with uri TT filter.
Test plan :
- To reproduce the issue with double quotes, the server
must be running URI::Escape 3.30 or earlier; the current
version of URI::Escape properly escapes double quote.
- In staff interface, perform a search with double quotes
that will return no result, ie "aaa xxx"
=> Without patch, javascript is broken
=> With patch, javascript is not broken
- Click on Z3950 button on results page
=> Without patch, the Title input is empty
=> With patch, the Title input contains the search terms
Additional test:
Do a search with something like äöü and then click Z3950
button on results page.
Without patch, encoding is broken in Z3950 form
With patch, encoding is correct.
Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Fixed a few tabs. Passes tests and QA script.
I can't reproduce the Javascript problem, but I can reproduce
the Z39.50 encoding problem and can detect no regression.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch makes Koha <-> Zebra use MARCXML for the serialization when
using DOM, and USMARC for GRS-1.
* The following functions are modified to set the Zebra record syntax
according to the current sysprefs and configuration:
- C4::Context->Zconn
- C4::Context-_new_Zconn
* A new function 'new_record_from_zebra' is introduced, which checks the
context we are in, and creates the MARC::Record object using the right
constructor.
The following packages get touched to make use of the new function:
- C4::Search
- C4::AuthoritiesMarc
and the same happens to the UI scripts that make use of them (both in
the OPAC and STAFF interfaces).
* Calls to the unsafe ZOOM::Record->render()[1] method are removed.
Due to this last change the code for building facets was rewritten. And
for performance on the facets creation I pushed higher version
dependencies for MARC::File::XML and MARC::Record (we rely on
MARC::Field->as_string).
* Calls to MARC::Record->new_from_xml and MARC::Record->new_from_usmarc
are wrapped with eval for catching problems [2].
* As of bug 3087, UNIMARC uses the 'unimarc' record syntax. this case is
correctly handled.
* As of bug 7818 misc/migration_tools/rebuild_zebra.pl behaves like:
- bib_index_mode (defaults to 'grs1' if not specified)
- auth_index_mode (defaults to 'dom')
here we do exactly the same.
To test:
- prove t/db_dependent/Search.t should pass.
- Searching should remain functional.
- Indexing and searching for a big record should work (that's what the
unit tests do).
- Test an index scan search (on the staff interface):
Search > More options > Check "Scan indexes".
- Enable 'itemBarcodeFallbackSearch' and try to circulate any word, it
shouldn't break.
- Searching for a biblio in a new subscription shouldn't break.
- Running bulkmarcimport.pl shouldn't break.
- And so on... for the rest of the .pl files.
[1] http://search.cpan.org/~mirk/Net-Z3950-ZOOM/lib/ZOOM.pod#render()
[2] a record that cannot be parsed by MARC::Record is simply skipped (bug 10684)
Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
If a search gives results with 6 facets, one of those facets won't be
displayed. This is due to a bug in the code that only considers great
than 6 facets in one area, and less than 6 in another.
Test Plan:
1) Perform a search that should give results for 6 different libraries
2) Note you only see 5 libraries in the facets with no option to expand
3) Apply this patch
4) Repeat step 1
5) Note you now have the option to expand the facets list
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
This patch should provide a regression test but I really don't know how
to write it.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch makes the following changes to UNIMARC biblio indexing :
A. Changes to UNIMARC conf files
1. add comments to biblio-koha-indexdefs.xml
2. make biblio-koha-indexdefs.xml more compact by grouping some
declarations
Ex : 200$f and 200$g => one declaration for 200$fg
3. suppress unneeded declarations (indexing of some 4XX fields and 6XX
fields not in unimarc format)
4. unindex some (sub)fields unneeded by most users (318, 207,230,210a,
215, 4XXd)
5. change the way 308 field is indexed (no visible changes)
6. replace Title-host with Host-item -- see bug 11119
7. index 208 in Material-Type -- see bug 11119
8. index 100 pos 8-9 and 9-12 in pubdate:y and pubdate:n
9. index 100 pos 8-9 in pubdate:s instead of 210$d
10. Index all subfields of note 334 and 327 in note index
11. Index 304 and 327 in title index as well as note index
327 can contain a list of titles included in a work
304 can contain the title of the original work in case of a
translation
12. Index 314 in author index as well as note index
314 can contain authors not mentionned in 200$f/g (the 4th, 5th etc.
author)
13. Index 328 note in Dissertation-information as well as note
14. Index 328$t in Title
B. Changes to ccl.properties :
1. add a new index Dissertation-information (1056)
2. fix EAN, pubdate and acqdate (they were not linked with bib1 attributes)
C. Changes to Search.pm
1. add Dissertation-information and suppress Title-host and UPC
D. Changes to QP config file queryparser.yaml
1. add Dissertation-information
2 fix EAN, pubdate and acqdate
Test plan :
If you cannot test in GRS1, test only in DOM, as GRS will be deprecated.
1. Apply the patch in a UNIMARC Koha running with DOM and ICU
2. copy src/etc/searchengine/queryparser.yaml into the main config
directory of QP
3. copy src/etc/zebradb/ccl.properties into the main config directory
of Zebra
4. copy src/etc/zebradb/marc_defs/unimarc/biblio/* into the main config
directory of Zebra
5. reindex biblios (rebuild_zebra.pl -r -b -x -v)
6. test note index : make some searches on 334$b or 327$b
7. test author index : make some searches on 314 field
8. test title index : make some searches on 304 and 327 field, make a
search on 328$t subfield
9. test dissertation-information index : make some searches on 328 field
10. In a record, put in the dates of 100 fields the values "1000" (1st
date) and "1001" (2d date) ; try to search a book written in year
1000, you should find the record ; idem for year 1001
11. make some searches and sort by date. It should work better as before,
especially if you have values like "c2009" or "impr. 2010" in 210
field
12. Regression test : make some searches on several indexes, like EAN,
etc. It should work as before
Test 10-12 with and without Queryparser activated.
Be careful: with Queryparser activated, the index names (title,
dissertation-information...) must be entered in lowercase only.
Of course, to test search and sort by dates, you need to have full
records, with dates in 100 field as well as 210 field.
Signed-off-by: Paola Rossi <paola.rossi@cineca.it>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Adding Number-local-acquisition in C4::Search known indexes allows to
search without using "ccl=" prefix.
Also corrects in ccl.properties : inv must be an alias of
Number-local-acquisition.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
In Koha 3.8, if a standard catalog search was performed and the user
clicked the Z39.50 search button, the search string would automatically
be placed in the isbn field for the Z39.50 search form.
Changes to the code have since broken this functionality.
Test Plan:
1) From mainpage.pl, use "Search the catalog" to search for the string
"9781570672835"
2) Click the Z39.50 Search button
3) Note the string is placed in the title field
4) Apply this patch
5) Repeat steps 1-2
6) Note the string is placed in the isbn field
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Tested old and new ISBN with and without hyphens.
Also tested some other keyword searches.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Note that the behavior will be a bit odd if you do a 'replace via
Z39.50' from a bib record whose title happens to be an ISBN, but
this scenario seems unlikely enough to ignore.
It could be useful to index the original language of a document (i.e.
"fre" for the English translation of a French novel).
This patch renames the Bib-1 use attribute 1095 from
Code-language-original to language-original and uses it to index:
- MARC21 041$h subfield
- UNIMARC 101$c subfield
It adds "language-original" in the list of index in Search.pm.
Test plan :
A. in a MARC21 GRS1 environment
1. Copy Zebra config files (zebradb/biblios/etc/bib1.att,
zebradb/ccl.properties, marc_defs/marc21/biblios/record.abs) from
your source etc/ directory to your main koha etc/ directory
2. Reindex zebra
3. Make some searches, like "language-original:fre"
B. in a MARC21 DOM environment
4. Copy Zebra config files (zebradb/biblios/etc/bib1.att, zebradb/ccl.properties,
marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl) from your source etc/
directory to your main koha etc/ directory
5. Reindex zebra
6. Make some searches, like "language-original:fre"
C. in a UNIMARC GRS1 environment
7. Copy Zebra config files (zebradb/biblios/etc/bib1.att,
zebradb/ccl.properties, marc_defs/unimarc/biblios/record.abs) from
your source etc/ directory to your main koha etc/ directory
8. Reindex zebra
9. Make some searches, like "language-original:fre"
A. in a UNIMARC DOM environment
10. Copy Zebra config files (zebradb/biblios/etc/bib1.att,
zebradb/ccl.properties, marc_defs/unimarc/biblios/biblio-zebra-indexdefs.xsl)
from your source etc/ directory to your main koha etc/ directory
11. Reindex zebra
12. Make some searches, like "language-original:fre"
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Under certain circumstance, a search term without quotation marks
returns the expected results while the same search with a
double quote embedded in it would fail.
Koha should ignore the quotation marks and return results anyway.
This appears when QueryWeightFields syspref is activated (and
QueryAutoTruncate is off), as field weighting builds a complex CCL
query using double quotes around search words. This patch simply
replaces double quotes in search words by a space.
Test plan :
- Set QueryAutoTruncate off (you may also need to set QueryFuzzy to off)
- Set QueryWeightFields off
- Perform a serch on two words where you have results, like : centre "ville
=> you get results
- Set QueryWeightFields on
- Perform same serch
=> you get the same results
Signed-off-by: Leila <koha.aixmarseille@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
If you select an index in the search dropdown and then enter in a QP
query starting with the field, Koha will prepend the index you do not
want to use at the beginning of the search, resulting in a search that
probably does not match what you were hoping for.
To test:
1) Select an index in the search dropdown in the OPAC. Author is fine.
2) Enter a search term using manually entered indexes. For example:
ti:cat in the hat
3) Note that the search fails.
4) Apply patch.
5) Repeat steps 1 and 2.
6) Note that the search succeeds.
7) Sign off.
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
MARC::Record 2.0.6+ enables the warnings pragma, and as a
consequence, started logging cases where a routine in
C4::Search was calling MARC::Field->subfield() with an undef
subfield label. This patch removes the log noise.
To test:
- Run prove -v t/db_dependent/Search.t
- There will be warnings about
"Use of uninitialized value $code_wanted in string" in MARC::Field.
- Apply the patch.
- Those warnings are gone.
Signed-off-by: Liz Rea <liz@catalyst.net.nz>
Tests now pass
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
When a record fails to decode during a search, Koha dies with an error.
Koha should ignore bad records and continue on ( and log the error ).
An example of a record that Zebra will happily ingest but which MARC::Record
doesn't like is one that contains a punctuation character in a tag label.
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
When an index does not contain a structure part, the structure "wrdl"
is automatically added and a structure is mandatory to build search
query (to convert ':' into '=').
But the code that tests that the structure is not already defined looks
in entire index string :
$index =~ /(st-|phr|ext|wrdl|nb|ns)/
It should look for a comma followed by a structure and in the case of
"nb" and "ns" look for an exact match.
The consequence is that an index containing ns or nb or phr or etc does
not work.
This patch modifies the regexp for this part and other parts looking at
structures into index.
Test plan :
- Desactivate all searching sysprefs.
- Create a new index called "ansa" number 8999 into bib1.att,
ccl.properties and records.abs
- Index a biblio with a value on this index, ie "VALUE"
- Perform a search on this index by editing URL:
http://<server>/cgi-bin/koha/catalogue/search.pl?idx=ansa&q=VALUE
=> Without patch, the search does not work. The PQF query is
"@and ansa: VALUE"
=> With patch, the search works. The PQF query is "@attr 1=8999 VALUE";
- Perform same test with an index containing a structure ie "aphra"
- Set QueryAutoTruncate syspref to automatically
=> Check * is added to operand : PQF query is
"@attr 1=8999 @attr 4=6 @attr 5=1 VALUE"
- You may check stopwords removal but this feature is obsolete
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Comment: as far as I can test, this works. Small tab error reported
by koha-qa, fixed in a followup.
This kind of patch is difficult to test without explicit instructions,
not everyone knows how to check what kind of PQF search is used.
I don't know. But I can test search results.
Test:
1) Deactivate search sysprefs
QueryAutoTruncate = only if * is added
QueryFuzzy = Don't try
QueryStemming = Don't try
QueryWeightFields = Disable
UseQueryParser = Do not try
2) Create new index 'ansa'
bib1.att : att 8999 ansa
ccl.properties : ansa 1=8999
records.abs : melm 999 ansa:w,ansa:p
1) and 2) from comment 3 on Bug
3) In the undestanding that index refers to field 999,
edited default framework, made 999a visible on editor
4) Edit sample record, add 'VALUE' to 999a, save, reindex
5) Search with /cgi-bin/koha/catalogue/search.pl?idx=ansa&q=VALUE
No results
6) Apply patch, repeat search
Got results
That's all I can test. If not enough for QA, then this
must wait until further and explicit test instructions
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
There is (for MARC21, at least), an exising indexes that this patch
fixes: Code-institution.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Before fixing UNIMARC DOM indexing, we must fix GRS-1 indexing
1) In advanced search, some Coded fields index are not working: Print,
Illustration, Content
2) Country-heading index is not working
3) Some subfields are indexed in wrong indexes :
102$a should be in Country-publication instead of Country-heading
(non defined in bib1.att)
106$a, filled only for printed works, should be in ff88-23 (form of
item) instead of itype. (ff88-23 is made for Marc21 008 pos
23, which contains the same data as 106a)
200$b should be in Material-type instead of (or in addition to) itype
and itemtype: (Material-type :"free-form string, ... that
describes the material type of the item, e.g., cassette, kit,
computer database, computer file.")
100$a pos 22-24 should not be indexed as "ln" : it is the language of
the record, not the language of the ressource
4) Index names are too long : if we index new positions of coded fields,
with existing names it breaks Zebra indexing (there must be a limit
in line lenghth in record.abs?)
5) There are a lot of warns when rebuiding zebra.
This patch make some changes in bib1.att (could be used later to improve
search) :
- fixing wording for att 51 and 1012
- adding comments for attributes based on MARC21 008 field (8800-8841)
- creating 8806 (tpubdate), 8838 (Modified-code), 8818 (ff8-18), 8840
(ff8-18-21), 8819 (ff8-19), 8821 (ff8-21), 8828 (ff8-28), 8830
(ff8-30), 8831 (ff8-31)
- creating attributes specific to UNIMARC : 9701-9707 (Video-mt,
Graphics-type, Graphics-support, Title-page-availability,
Cumulative-index-availability, script-Title, char-encoding)
- setting apart 3 blocks of attributes, so it could be easy to make
further changes :
-- common to Marc21 and UNIMARC : 8806, 8822, 8838
-- slightly different in Marc21 and UNIMARC (different meanings
according to the type of the record => don't match a single
UNIMARC field)
-- specific to UNIMARC : 9701-9707
In ccl.properties :
- creating a new index: Country-publication 1=1053
- suppressing some warns by mapping with bib1 att:
Date-time-last-modified, Name, rtype, Music-number
- defining indexes using the 3 blocks attributes defined in bib1
(common to Marc21 and UNIMARC, slightly different, specific to UNIMARC)
In record.abs :
- renaming some index for 100-105-110 fields
- correcting indexing of 102$a (country of publication)
106$a (ff88-23)
100$a pos 22-24 (language of record, no more
indexed)
105$a pos. 0-3 (illustration code)
200$b (for the moment, I keep it indexed in
itype and itemtype, but also Material-Type)
In C4/Search.pm :
- adding "Country-publication" index
In OPAC and staff interface template subtypes_unimarc.in :
- renaming indexes to take into account the changes made to Zebra
config files
To test (this cannot be done with a sandbox) :
1) Apply the patch in a UNIMARC GRS-1 Koha instance
2) Copy the following files from the etc/zebradb of your source
directory into the etc/zebradb of your main Koha directory:
-- etc/zebradb/biblios/etc/bib1.att
-- etc/zebradb/ccl.properties
-- etc/zebradb/marc_defs/unimarc/biblios/record.abs
3) Reindex your data (rebuild_zebra -x -b -r -v)
4) Try to use those Coded fields indexes in Advanced search, in OPAC
and Staff interface (available after clicking on "More options",
then on "Coded information filters"):
Audience, Print, Literary genre, Biography, Illustration, Content,
Video Types, Serials, Serial Type, Periodicity, Regularity
5) Try to search "Country-publication=FR" in simple search
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
No koha-qa errors.
Tests for GRS-1
Followed test plan
Search by coded fields works, but only on OPAC,
on staff there are few options
Search by Country-publication works after patch
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch updates the wthdrawn field in items and deleteditems to be
withdrawn instead. No functional changes are made.
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Comment: Save for translation files (that will be fixed on next release),
only occurrence of wthdrawn is on updatedatabase.pl
No koha-qa errors.
This touch many files, and I did not test everything,
but all seems normal. I think that any problem could
be fixed later.
Perhaps both entries in updatedatabase.pl could be joined
into one, but thats for QA.
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Add an option to cleanup_database.pl to purge the search_history
entries older than X days.
Test plan:
- Apply patch
- Check that your test DB has some entries a little older than 30 days
and a few ones even older than that in search_history:
SELECT * FROM search_history WHERE time < DATE_SUB( NOW(), INTERVAL 30 DAY );
If not, modify some existing entries.
- Run cleanup_database with a fixed number of days (replace XX with
something higher than 30)
/misc/cronjobs/cleanup_database.pl --searchhistory XX
- Check that entries older than XX days got deleted from search_history
- Run without the day parameter
/misc/cronjobs/cleanup_database.pl --searchhistory
- Check that entries older than 30 days got deleted from search_history
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
To test:
[1] Turn on the syspref for enabling OPAC holds.
[2] Create an item and bring it up on the OPAC search
results. Run through the following possibilities,
by changing the item, and verify that the place hold
link in OPAC search results appears only when the item is
- not lost AND
- not withdrawn AND
- not damaged (or is damged and AllowHoldsOnDamagedItems is ON) AND
- the item is not marked not-for-loan OR
the item has a negative notforloan value (e.g., it is on order)
Note that it is necessary to reindex the test bib after making
each change to the test item.
[3] Also verify that whether or not in the item is in transit does
NOT affect whether the place hold link appears.
[4] Verify that there is no regression on bug 8975 (i.e., if an
item is on order, that status should be displayed in staff client
search results).
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
