Commit graph

462 commits

Author SHA1 Message Date
Mathieu Saby
451f67c055 Bug 9940: Add a new index for the original language of a document
It could be useful to index the original language of a document (i.e.
"fre" for the English translation of a French novel).

This patch renames the Bib-1 use attribute 1095 from
Code-language-original to language-original and uses it to index:

- MARC21 041$h subfield
- UNIMARC 101$c subfield

It adds "language-original" in the list of index in Search.pm.

Test plan :
A. in a MARC21 GRS1 environment
1. Copy Zebra config files (zebradb/biblios/etc/bib1.att,
   zebradb/ccl.properties, marc_defs/marc21/biblios/record.abs) from
   your source etc/ directory to your main koha etc/ directory
2. Reindex zebra
3. Make some searches, like "language-original:fre"
B. in a MARC21 DOM environment
4. Copy Zebra config files (zebradb/biblios/etc/bib1.att, zebradb/ccl.properties,
   marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl) from your source etc/
   directory to your main koha etc/ directory
5. Reindex zebra
6. Make some searches, like "language-original:fre"
C. in a UNIMARC GRS1 environment
7. Copy Zebra config files (zebradb/biblios/etc/bib1.att,
   zebradb/ccl.properties, marc_defs/unimarc/biblios/record.abs) from
   your source etc/ directory to your main koha etc/ directory
8. Reindex zebra
9. Make some searches, like "language-original:fre"
A. in a UNIMARC DOM environment
10. Copy Zebra config files (zebradb/biblios/etc/bib1.att,
    zebradb/ccl.properties, marc_defs/unimarc/biblios/biblio-zebra-indexdefs.xsl)
    from your source etc/ directory to your main koha etc/ directory
11. Reindex zebra
12. Make some searches, like "language-original:fre"

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-12-25 15:37:14 +00:00
Fridolyn SOMERS
868855df4b Bug 7518: certain searches with quotation marks don't work
Under certain circumstance, a search term without quotation marks
returns the expected results while the same search with a
double quote embedded in it would fail.

Koha should ignore the quotation marks and return results anyway.

This appears when  QueryWeightFields syspref is activated (and
QueryAutoTruncate is off), as field weighting builds a complex CCL
query using double quotes around search words.  This patch simply
replaces double quotes in search words by a space.

Test plan :
- Set QueryAutoTruncate off (you may also need to set QueryFuzzy to off)
- Set QueryWeightFields off
- Perform a serch on two words where you have results, like : centre "ville
=> you get results
- Set QueryWeightFields on
- Perform same serch
=> you get the same results

Signed-off-by: Leila <koha.aixmarseille@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-12-17 16:36:26 +00:00
Jared Camins-Esakov
03f342aa67 Bug 10688: Don't prefix index inappropriately
If you select an index in the search dropdown and then enter in a QP
query starting with the field, Koha will prepend the index you do not
want to use at the beginning of the search, resulting in a search that
probably does not match what you were hoping for.

To test:
1) Select an index in the search dropdown in the OPAC. Author is fine.
2) Enter a search term using manually entered indexes. For example:
    ti:cat in the hat
3) Note that the search fails.
4) Apply patch.
5) Repeat steps 1 and 2.
6) Note that the search succeeds.
7) Sign off.

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-12-10 01:33:02 +00:00
Galen Charlton
477b9716f1 Merge branch 'new/bug11269' 2013-11-19 16:27:29 +00:00
2f6f6d2b0c Bug 11269: fix incorrect use of MARC::Field->subfield()
MARC::Record 2.0.6+ enables the warnings pragma, and as a
consequence, started logging cases where a routine in
C4::Search was calling MARC::Field->subfield() with an undef
subfield label.  This patch removes the log noise.

To test:
- Run prove -v t/db_dependent/Search.t
- There will be warnings about
  "Use of uninitialized value $code_wanted in string" in MARC::Field.
- Apply the patch.
- Those warnings are gone.

Signed-off-by: Liz Rea <liz@catalyst.net.nz>
Tests now pass

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-11-19 16:22:46 +00:00
Jonathan Druart
82233bae25 Bug 10684: (follow-up) adjust how undecodable records are logged
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-11-14 18:04:52 +00:00
6a78fad4e0 Bug 10684: make sure that Koha search does not dies on undecodable records
When a record fails to decode during a search, Koha dies with an error.
Koha should ignore bad records and continue on ( and log the error ).

An example of a record that Zebra will happily ingest but which MARC::Record
doesn't like is one that contains a punctuation character in a tag label.

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-11-14 18:04:50 +00:00
Bernardo Gonzalez Kriegel
c6721b2fce Bug 10271: (follow-up) correct tabs
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-28 16:14:20 +00:00
Fridolyn SOMERS
e17abe0e28 Bug 10271: fix searching using an index containing "ns" or "nb"
When an index does not contain a structure part, the structure "wrdl"
is automatically added and a structure is mandatory to build search
query (to convert ':' into '=').

But the code that tests that the structure is not already defined looks
in entire index string :

  $index =~ /(st-|phr|ext|wrdl|nb|ns)/

It should look for a comma followed by a structure and in the case of
"nb" and "ns" look for an exact match.

The consequence is that an index containing ns or nb or phr or etc does
not work.

This patch modifies the regexp for this part and other parts looking at
structures into index.

Test plan :
- Desactivate all searching sysprefs.
- Create a new index called "ansa" number 8999 into bib1.att,
  ccl.properties and records.abs
- Index a biblio with a value on this index, ie "VALUE"
- Perform a search on this index by editing URL:
  http://<server>/cgi-bin/koha/catalogue/search.pl?idx=ansa&q=VALUE
=> Without patch, the search does not work. The PQF query is
   "@and ansa: VALUE"
=> With patch, the search works. The PQF query is "@attr 1=8999 VALUE";
- Perform same test with an index containing a structure ie "aphra"
- Set QueryAutoTruncate syspref to automatically
=> Check * is added to operand : PQF query is
   "@attr 1=8999 @attr 4=6 @attr 5=1 VALUE"
- You may check stopwords removal but this feature is obsolete

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>

Comment: as far as I can test, this works. Small tab error reported
by koha-qa, fixed in a followup.

This kind of patch is difficult to test without explicit instructions,
not everyone knows how to check what kind of PQF search is used.
I don't know. But I can test search results.

Test:
1) Deactivate search sysprefs
QueryAutoTruncate = only if * is added
QueryFuzzy = Don't try
QueryStemming = Don't try
QueryWeightFields = Disable
UseQueryParser = Do not try

2) Create new index 'ansa'
bib1.att : att 8999 ansa
ccl.properties : ansa 1=8999
records.abs : melm 999 ansa:w,ansa:p

1) and 2) from comment 3 on Bug

3) In the undestanding that index refers to field 999,
edited default framework, made 999a visible on editor

4) Edit sample record, add 'VALUE' to 999a, save, reindex

5) Search with /cgi-bin/koha/catalogue/search.pl?idx=ansa&q=VALUE
No results

6) Apply patch, repeat search
Got results

That's all I can test. If not enough for QA, then this
must wait until further and explicit test instructions

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

There is (for MARC21, at least), an exising indexes that this patch
fixes: Code-institution.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-28 16:14:06 +00:00
Mathieu Saby
041e3603a1 Bug 8252: Fix indexing of UNIMARC 1xx for GRS-1
Before fixing UNIMARC DOM indexing, we must fix GRS-1 indexing

1) In advanced search, some Coded fields index are not working: Print,
   Illustration, Content
2) Country-heading index is not working
3) Some subfields are indexed in wrong indexes :

  102$a should be in Country-publication instead of Country-heading
        (non defined in bib1.att)
  106$a, filled only for printed works, should be in ff88-23 (form of
         item) instead of itype.  (ff88-23 is made for Marc21 008 pos
         23, which contains the same data as 106a)
  200$b should be in Material-type instead of (or in addition to) itype
        and itemtype: (Material-type :"free-form string, ... that
        describes the material type of the item, e.g., cassette, kit,
        computer database, computer file.")
  100$a pos 22-24 should not be indexed as "ln" : it is the language of
        the record, not the language of the ressource

4) Index names are too long : if we index new positions of coded fields,
   with existing names it breaks Zebra indexing (there must be a limit
   in line lenghth in record.abs?)
5) There are a lot of warns when rebuiding zebra.

This patch make some changes in bib1.att (could be used later to improve
search) :

- fixing wording for att 51 and 1012
- adding comments for attributes based on MARC21 008 field (8800-8841)
- creating 8806 (tpubdate), 8838 (Modified-code), 8818 (ff8-18), 8840
  (ff8-18-21), 8819 (ff8-19), 8821 (ff8-21), 8828 (ff8-28), 8830
  (ff8-30), 8831 (ff8-31)
- creating attributes specific to UNIMARC : 9701-9707 (Video-mt,
  Graphics-type, Graphics-support, Title-page-availability,
  Cumulative-index-availability, script-Title, char-encoding)
- setting apart 3 blocks of attributes, so it could be easy to make
  further changes :
-- common to Marc21 and UNIMARC : 8806, 8822, 8838
-- slightly different in Marc21 and UNIMARC (different meanings
   according to the type of the record => don't match a single
   UNIMARC field)
-- specific to UNIMARC : 9701-9707

In ccl.properties :
- creating a new index: Country-publication 1=1053
- suppressing some warns by mapping with bib1 att:
  Date-time-last-modified, Name, rtype, Music-number
- defining indexes using the 3 blocks attributes defined in bib1
  (common to Marc21 and UNIMARC, slightly different, specific to UNIMARC)

In record.abs :
- renaming some index for 100-105-110 fields
- correcting indexing of 102$a (country of publication)
                         106$a (ff88-23)
                         100$a pos 22-24 (language of record, no more
                               indexed)
                         105$a pos. 0-3 (illustration code)
                         200$b (for the moment, I keep it indexed in
                               itype and itemtype, but also Material-Type)

In C4/Search.pm :
- adding "Country-publication" index

In OPAC and staff interface template subtypes_unimarc.in :
- renaming indexes to take into account the changes made to Zebra
  config files

To test (this cannot be done with a sandbox) :
1) Apply the patch in a UNIMARC GRS-1 Koha instance
2) Copy the following files from the etc/zebradb of your source
   directory into the etc/zebradb of your main Koha directory:
-- etc/zebradb/biblios/etc/bib1.att
-- etc/zebradb/ccl.properties
-- etc/zebradb/marc_defs/unimarc/biblios/record.abs
3) Reindex your data (rebuild_zebra -x -b -r -v)
4) Try to use those Coded fields indexes in Advanced search, in OPAC
   and Staff interface (available after clicking on "More options",
   then on "Coded information filters"):
   Audience, Print, Literary genre, Biography, Illustration, Content,
   Video Types, Serials, Serial Type, Periodicity, Regularity
5) Try to search "Country-publication=FR" in simple search

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
No koha-qa errors.

Tests for GRS-1
Followed test plan
Search by coded fields works, but only on OPAC,
on staff there are few options
Search by Country-publication works after patch

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 15:06:10 +00:00
dbaefb626c Bug 10550: Fix database typo wthdrawn
This patch updates the wthdrawn field in items and deleteditems to be
withdrawn instead. No functional changes are made.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>

Comment: Save for translation files (that will be fixed on next release),
only occurrence of wthdrawn is on updatedatabase.pl
No koha-qa errors.

This touch many files, and I did not test everything,
but all seems normal. I think that any problem could
be fixed later.

Perhaps both entries in updatedatabase.pl could be joined
into one, but thats for QA.

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-09-08 01:58:41 +00:00
Mirko Tietgen
1d5a4a0a81 Bug 10361: Add option to cleanup_database.pl to purge OPAC search history
Add an option to cleanup_database.pl to purge the search_history
entries older than X days.

Test plan:

- Apply patch
- Check that your test DB has some entries a little older than 30 days
  and a few ones even older than that in search_history:

  SELECT * FROM search_history WHERE time < DATE_SUB( NOW(), INTERVAL 30 DAY );

  If not, modify some existing entries.

- Run cleanup_database with a fixed number of days (replace XX with
  something higher than 30)

  /misc/cronjobs/cleanup_database.pl --searchhistory XX

- Check that entries older than XX days got deleted from search_history
- Run without the day parameter
  /misc/cronjobs/cleanup_database.pl --searchhistory
- Check that entries older than 30 days got deleted from search_history

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-09-04 14:22:50 +00:00
Fridolyn SOMERS
397ba93f97 Bug 10741: reformat test for displaying place hold link in OPAC search results
To test:

[1] Turn on the syspref for enabling OPAC holds.
[2] Create an item and bring it up on the OPAC search
    results.  Run through the following possibilities,
    by changing the item, and verify that the place hold
    link in OPAC search results appears only when the item is

    - not lost AND
    - not withdrawn AND
    - not damaged (or is damged and AllowHoldsOnDamagedItems is ON) AND
    - the item is not marked not-for-loan OR
      the item has a negative notforloan value (e.g., it is on order)

    Note that it is necessary to reindex the test bib after making
    each change to the test item.

[3] Also verify that whether or not in the item is in transit does
    NOT affect whether the place hold link appears.
[4] Verify that there is no regression on bug 8975 (i.e., if an
    item is on order, that status should be displayed in staff client
    search results).

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-08-23 15:03:12 +00:00
Fridolyn SOMERS
239513dc7a Bug 10741: restore holdability of in-transit items on OPAC search results
In search results, one could not place a hold on an item in transit
and for loan (items.notforloan=0).  This appears when AllowOnShelfHolds
is allowed.

This patch repairs a regression introduced by the patch for bug 8975.

Test plan :
- Set AllowOnShelfHolds to on
- Create a record with a normal item : not lost, not withdrawn, not
  damaged, notforloan=0
- Index this record
- Perform a search on OPAC that returns this record (and others)
=> You see in actions "Place hold"
- Add this item in transit : /cgi-bin/koha/circ/branchtransfers.pl
- Re-perform the search on OPAC
=> You see in actions "Place hold" and item "in transit"

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-08-23 14:45:12 +00:00
Jared Camins-Esakov
8c6fe639b8 Bug 9961: Add truncation support to QP driver
The original implementation of QueryParser did not handle truncation
based on the QueryAutoTruncate system preference. This patch adds support.

To test:
1) Apply patch.
2) Turn on UseQueryParser.
3) Set QueryAutoTruncate to "automatically."
4) Search for "har". Note that it returns results with words
   like "Harry" (i.e. with right truncation).
5) Search for "har*". Note that it still returns results with right
   truncation.
6) Set QueryAutoTruncate to "only when * is added."
7) Search for "har". Note that it returns only records that have the
   exact word "har" in them (most likely there will be none unless you
   have Hebrew items).
8) Search for "har*". Note that once again it returns results for "Harry"
   (i.e. right truncated results).
9) Sign off.

This patch also reindents a hash in Koha/QueryParser/Driver/PQF.pm
because it was hard to read before.

Signed-off-by: Mirko Tietgen <mirko@abunchofthings.net>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
All tests and QA script pass.
Thx for fixing this Jared!
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-05-08 09:44:23 -04:00
Karam Qubsi
2d29440c5a Bug 10176: Solve the Invalid URLs in 'Refine Search Results' when it uses UNICODE symbols
I just add use utf8; to the Search.pm and the problem
was solved .

Test plan :
1- Add bib records with non-latin characters
2- search for some of these records
3- try to refine your search using Subject / Author

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>

Comment: Work fixing URLs in facets. Now they work correctly.
No errors.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.

I tested facets with the 22 Arabic records provided on
bug 9579 successfully. Before the patch the links are not
correct, after applying the patch the links work as
expected.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-05-06 08:22:35 -04:00
Galen Charlton
a7eb34f2c8 Revert "Bug 6554 - make Koha internally utf-8 clean"
This reverts commit d542740ab8.

Rolling back bug 6554 work until we have more comprehensive tests.

Conflicts:

	opac/opac-search.pl
2013-04-29 15:12:32 -07:00
Fridolyn SOMERS
39bf1c502a Bug 9588: weighted search query with index
When QueryWeightFields is enabled, the searching query is created with several options.
In C4::Search::_build_weighted_query, when no index is defined, the query is build with fuzzy and stemming options. When an index is defined, theses options are missing, only unconditional right truncation is used.
The consequence is that when QueryStemming is disabled, a search with index can give more results (due to right truncation) that a search without.

This patch adds stemming and fuzzy on search with index, conditioned with QueryFuzzy and QuerryStemming sysprefs.
Also changes world list search (wrld) weight to r6 in order to set fuzzy search to r8 and stemming search to r9 (like search without index).

Test plan :
- Go to searching preferences (admin/preferences.pl?tab=searching)
- Set QueryAutoTruncate to "only if * is added"
- Set QueryFuzzy and QuerryStemming to "Don't try"
- Set QueryWeightFields to "Enable"
- Go to advanced search page
- Select an indexe (ie Title) and perform a search on a short word
=> Look at zebrarv log and see that query does not contain right truncation : @attr 5=1
- Set QueryFuzzy to "Try"
- Perform same search
=> Look at zebrarv log and see that query contains fuzzy : @attr 5=103
- Set QueryFuzzy to "Don't try" and QuerryStemming to "Try"
- Perform same search
=> Look at zebrarv log and see that query contains rigth truncation on stemmed word : @attr 5=1

Signed-off-by: koha.aixmarseille <koha.aixmarseille@gmail.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
This patch makes Fuzzy and Stemming influence search results on weighted
queries when using an index. Side-effect is however that the results for a
search like index=term* (add truncation manually too) could be LOWER than the
the number of hits for index=term. Further comments on Bugzilla.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-04-21 20:54:21 -04:00
Dobrica Pavlinusic
d542740ab8 Bug 6554 - make Koha internally utf-8 clean
In current implementation (mostly commented out in this patch)
uses heuristic to guess which strings need decoding from utf-8
to binary representation and doesn't support utf-8 characters
in templates and has problems with utf-8 data from database.

With this changes, Koha perl code always uses utf-8 encoding
correctly. All incomming data from database is allready
correctly marked as utf-8, and decoding of utf8 is required
only from Zebra and XSLT transfers which don't set utf-8 flag
correctly.

For output, standard perl :encoding(utf8) handler is used
so it also removes various "wide character" warnings as side-effect.

Test scenario:
1. make sure that you have utf-8 characters in your biblio
   records, patrons, categories etc.
2. try to search records on intranet and opac which contain
   utf-8 characters
3. install language which has utf-8 characters, e.g. uk-UA
   dpavlin@koha-dev:/srv/koha/misc/translator(bug_6554) $
   PERL5LIB=/srv/koha/ perl translate install uk-UA
4. switch language to uk-UA and verify that templates
   display correctly
5. test search and Z39.50 search and verify that caracters
   are correct

Signed-off-by: Owen Leonard <oleonard@myacpl.org>

I followed the test plan, adding utf-8 characters to library names,
patron categories, titles, and authorized values. I tried the uk-UA
translation and everything looked good.

When performing Z39.50 searches for titles containing utf-8 characters I
got results which were still occasionally contaminated with dummy
characters [?] but I assume this is Z39.50's fault not the patch's.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Already signed, add mine.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-04-01 18:51:49 -04:00
Paul Poulain
66b5b75d2c Bug 7440 follow-up removing forgotten NoZebra references
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-03-19 21:17:05 -04:00
4dcee58a4d Bug 7440 - Remove NoZebra vestiges
Removed NoZebra vestiges. This comprises several code blocks that depend on the NoZebra syspref and NZ related functions/methods.

C4::Biblio->
 GetNoZebraIndexes
 _DelBiblioNoZebra
 _AddBiblioNoZebra

C4::Search->
 NZgetRecords
 NZanalyse
 NZoperatorAND
 NZoperatorOR
 NZoperatorNOT
 NZorder

C4::Installer->
 set_indexing_engine

Sponsored-by: Universidad Nacional de Córdoba
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-03-19 21:17:04 -04:00
Jared Camins-Esakov
c716c28fa6 Bug 9239 QA follow-up: better error handling
* Fix a long-standing bug in the linker that could crash the linker when
run against odd data.
* Sanitize input to SimpleSearch.
* Correctly handle CCL indexes with QP.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-03-16 21:32:34 -04:00
Jared Camins-Esakov
d88bd37f30 Bug 9239 QA follow-up: the last QA follow-up was missing a require
This patch also corrects the definition of the an= index, which was
missing exactness.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-03-16 21:32:34 -04:00
Jared Camins-Esakov
a8f23264dd Bug 9239 QA follow-up: escape CGI input
Koha was not previously escaping CGI input, which caused problems for
highlighting and is a security issue.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Thx for fixing this.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-03-16 21:32:34 -04:00
Jared Camins-Esakov
006df7aced Bug 9239 QA-follow-up: quiet warnings and fix problem with limits
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-03-16 21:32:33 -04:00
Jared Camins-Esakov
ed521b8ebb Bug 9239 QA follow-up: fix highlighting and merge error
QP searches with && broke search highlighting on the OPAC details page.
This patch corrects encoding of the query_desc parameter that is passed
to the details page.

My last attempt at rebasing also transposed the variable for index
names with the variable for operators, meaning that the dropdown in
the basic search did not work.

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Fixes some problems raised during QA successfully.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-03-16 21:32:33 -04:00
Jared Camins-Esakov
144c7f4e4e Bug 9239: Allow the use of QueryParser for all queries
With the inclusion of this patch, all searches will (try) to use
QueryParser for handling queries for both the bibliographic and authority
databases if UseQueryParser is enabled. If QueryParser is unavailable,
UseQueryParser is disabled, or the search uses CCL indexes, the old
search code will be used.

To test:
1) Apply patch.
2) Run the unit test with `prove t/QueryParser.t`
3) Enable the UseQueryParser syspref.
4) Try searches that should return results in the following places:
   * OPAC (simple search)
   * OPAC (advanced search)
   * OPAC (authorities)
   * Staff client (header search)
   * Staff client (advanced search)
   * Staff client (cataloging search)
   * Staff client (authorities)
   * Staff client (importing a batch using a match point)
   * Staff client (searching for an item for adding to a label)
   * Staff client (acquisitions)
   * Staff client (searching for a record to create a serial)
   * ANYWHERE ELSE I HAVE FORGOTTEN
5) Disable the UseQueryParser syspref. Repeat at least some of the
   searches you did above.
6) If all searches worked, sign off.

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Elliott Davis <elliott@bywatersolions.com>
Searching still works as expected for variuos places.
QueryParser syspref seemed to be enabled by default

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-03-16 21:32:32 -04:00
Jonathan Druart
fad1f44d42 Bug 9367: Code optimization: CheckReserves is too often called
This patch rewrites the GetReserveStatus routine in order to take in
parameter the itemnumber and/or the biblionumber.

In some places, the C4::Reserves::CheckReserves routine is called when
we just want to get the status of the reserve. In these cases, the
C4::Reserves::GetReserveStatus is now called.
This routine executes 1 sql query (or 2 max).

Test plan:
Check that there is no regression on the different pages where reserves
are used. The different status will be the same than before applying
this patch.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-03-16 11:49:43 -04:00
Fridolyn SOMERS
fa1ee846f7 Bug 9377 - itemtype in duplicate biblio search
In C4::Search::FindDuplicate, when biblio has no ISBN, the duplicate search adds :
$query .= " and itemtype=$result->{itemtype}".
This is wrong when itemtype is defined in items.

This patch simply removes the itemtype from dublicate search.

Test plan :
- Go to a biblio details page
- Click on "Edit as new (duplicate)"
- If ISBN is defined, remove it
- Click on save
=> a duplicate is detected
- Change biblio item type and save
=> a duplicate is detected
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-03-11 08:06:50 -04:00
Jared Camins-Esakov
c8e8328cb1 Bug 9183: Refactor ZOOM event loop
Prior to this patch, there were three identical ZOOM event loops in
C4::Search. This is wasteful, and goes against all good programming
practice. This patch refactors the ZOOM event loops into a separate
subroutine which is called by SimpleSearch, searchResults, and
GetDistinctValues call.

The new routine, _ZOOM_event_loop process the ZOOM event loop and,
once it has been fully processed, passes control to a closure provided
by the calling routine for processing the results, and destroys the
result sets.

To test (after applying patch):
1) Do a regular bibliographic search that should return results.
2) Do a search in the Cataloging module that should return results.
3) If you get results from both searches, the patch works.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2012-12-27 10:09:07 -05:00
4f986f34e2 Bug 8975 - search results should say on order in staff client
In order to allow holds on items with notforloan = -1, processing
of "unavailable" items in Search.pm was altered to exclude items
with notforloan < 0. (See Bug 2341: items marked 'on order' not
 reserveable from search results). Doing so meant that such items
were excluded from the list, in staff client search results, of
items which are unavailable.

This patch changes the logic of that processing so that items
with notforloan < 0 are considered unavailable, but can still
be placed on hold.

To test, edit a record with a single item and view that record
in search results. When the item is is on order (notforloan -1)
it should say so. The holds link should be INactive only if:

- item is withdrawn AND/OR
- item is lost AND/OR
- item is damaged (and AllowHoldsOnDamagedItems is off) AND/OR
- item is not for loan, with notforloan > 0

Signed-off-by: Nicole C. Engard <nengard@bywatersolutions.com>

All tests pass (note that a reindex is required if changing item
statuses - which is why my first tests failed).

Passed-QA-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2012-11-20 07:40:40 -05:00
Fridolyn SOMERS
24216136f4 Bug 8996: In result page items with negative notforloan are available
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Passed-QA-by: Marcel de Rooy <M.de.Rooy@rijksmuseum.nl>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2012-11-09 21:29:51 -05:00
Lyon3 Team
0e8e6e7133 Question marks in series title facet links
When series title include a question or exclamation mark, theese must be removed
to prevent search failure.

http://bugs.koha-community.org/show_bug.cgi?id=8888
Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Before the patch: Series facet links with ! or ? return no results.
After the patch the same links return valid results.

Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-10-25 18:15:37 +02:00
881ee1b1a1 Bug 7401 - Shelving Location facet
This patch enables the shelving location facet as an
alternative to the branches fact in two situations:
A) SingleBranchMode is enabled
B) There is only one branch in the branches table

Test Plan:
1) Catalog multiple items with different shelving locations.
2) Test enable by enabling SingleBranchMode
3) Test enable by deleting all but one branch

Based on initial patch by Ian Walls.

Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Tested cases 2) and 3) successfully in OPAC and staff client

Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-09-19 14:25:04 +02:00
Jared Camins-Esakov
4feb6a4736 Bug 8211: Add exploded search options
When working with hierarchical subject headings, it is sometimes helpful
to do a search for all records with a specific subject, plus
broader/narrower/related subjects. This patch adds support for these
"exploded" subject searches to Koha.

To test:
1) Make sure you have a bunch of hierarchical subjects. I created
   geographical subjects for "Arizona," "United States," and "Phoenix,"
   and linked them together using 551s, and made sure I had a half
   dozen records linking to each (but not all to all three).
2) Do a search for su-br:Arizona (or choose "Subject and broader terms"
   on the advanced search screen with "more options" displayed), and
   check that you get the records with the subject "Arizona" and the
   records with the subject "United States"
3) Do a search for su-na:Arizona (or choose "Subject and narrower terms"
   on the advanced search screen with "more options" displayed), and
   check that you get the records with the subject "Arizona" and the
   records with the subject "Phoenix"
4) Do a search for su-rl:Arizona (or choose "Subject and related terms"
   on the advanced search screen with "more options" displayed), and
   check that you get the records with the subject "Arizona," the
   records with the subject "United States," and the records with the
   subject "Phoenix"
5) Ensure that other searches still work (keyword, subject, ccl,
   whatever)
6) Sign off

Technical details:
This patch adds a shim in front of C4::Search::buildQuery in order to
preprocess the query and call the _handle_exploding_search callback.
This shim will allow us to gradually offload query parsing to a new
query parser module.

Signed-off-by: wajasu <matted-34813@mypacks.net>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-09-18 15:20:48 +02:00
Fridolyn SOMERS
4792d78c19 Bug 7690: hidelostitems not working in Normal Display
Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Tested by toggling both the hidelostitems preference and the
OpacHiddenItems preference. Both work as expected in the normal
search results display.

Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-09-17 18:39:08 +02:00
Jonathan Druart
cf97a72abf Bug 4491: FIX buildQuery if called with an empty limit
if @limit=('') buildQuery failed

Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
All searches that I tried (keyword, indexed, CCL, with limits, without,
etc.) worked fine. There are warnings about uninitialized variables in
the OPAC, but they exist on master as well and therefore should not
block these patches.

Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-09-17 18:19:49 +02:00
Paul Poulain
f22d9e1337 Bug 8743 follow-up, throw a more explanatory error message
When testing bug 8743, I discovered a missing index in my authority file.
The error message was
"CCL parsing error (10014) Unknown qualifier ZOOM"

which is not very helpfull because it does not show the query that was made.
This patch add the query itself after the zebra error
2012-09-14 15:59:50 +02:00
Jared Camins-Esakov
bc05b5d163 Bug 7417: Include see from references in bibliographic searches
This patch adds the Koha::Indexer::RecordNormalizer and
Koha::Indexer::MARC::RecordNormalizer::EmbedSeeFromHeadings packages
to enable the inclusion of alternate forms of headings in bibliographic
searches. When the new syspref IncludeSeeFromInSearches is turned on
(default is off) rebuild_zebra.pl will insert see from headings from
authority records into bibliographic records when indexing, so that a
search on an obsolete term will turn up relevant records.

To test:
1) Enable IncludeSeeFromInSearches
2) Add a heading that has an alternate form to a record (for example,
   "Cooking" has the alternate form "Cookery," if you have authority
   records from LC)
3) Index the zebraqueue (or reindex if you haven't indexed your system
   yet)
4) Confirm that if you search for "Cookery" you get the record you
   just modified

Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on master 5 August 2012
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on master 11 September 2012

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>

Also checked:
- Verified database update works correctly
- Checked system preference and its description
- Checked staff/opac detail pages with feature on/off
- Checked staff/opac search facets
- Downloaded and tested records in various formats
- Tried different searches for 'see from' entries of authorities
- Ran all unit tests

No problems found.
2012-09-13 14:19:28 +02:00
Chris Cormack
c5ca865b54 Bug 8293 : Fixing POD to match actuality.
Signed-off-by: Fridolyn SOMERS <fridolyn.somers@biblibre.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-09-05 14:37:34 +02:00
Jared Camins-Esakov
1adb0afa4c Bug 8648: searchResults uses SQL query rather than GetMarcFromKohaField
Around line 1470-something:

    my $sth =
    $dbh->prepare(
            "SELECT tagfield FROM marc_subfield_structure WHERE kohafield LIKE
            'items.itemnumber'"
            );
    $sth->execute;

This patch replaces that with a call to GetMarcFromKohaField.

To test:
1) Apply patch.
2) Do a search that returns both available and unavailable items.
   You'll know if the patch isn't working.

Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-08-29 18:05:36 +02:00
ff7417fa91 Bug 8679 [REVISED] Remove usage of Amazon API
This patch removes the AmazonReviews and AmazonSimilarItems
features from the OPAC and staff client. With on Amazon
feature remaining, cover images, the *AmazonEnabled preference
is also removed in favor of checking the *AmazonCoverImages
preference. Two other system preferences, AWSAccessKeyID and
AWSPrivateKey are removed as they were required only by the
removed features.

Handling of book cover images from Amazon is unchanged.

Signed-off-by: Nicole C. Engard <nengard@bywatersolutions.com>

Turned on amazon covers in opac and staff client and all
worked as expected. Then tested to make sure other cover image
services still worked and they do.

Signing off.

Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-08-29 16:05:29 +02:00
Jonathan Druart
c56bc65de8 Bug 8172: Followup: Removes useless empty params
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-08-29 15:38:59 +02:00
Mark Tompsett
2c3ee1b578 Bug 8458 - $stemmed_operand in C4::Search _build_stemmed_operand is not initialized. Added =q{} to initialize it to the empty string at declaration time.
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-07-18 11:46:05 +02:00
Meenakshi.R
8bd8aba577 Bug 4255 - Add item type to facet list
This is for MARC 21 only.

Made following changed:

- In getFacets in C4/Koha.pm added item type facet for 952y and 942c
- In getRecords in C4:Search.pm added code to get description of itemtype codes
- facets.inc in both staff and opac to show item types related label in the facets block

To test:

Add records such that a certain itype (say BK) is present in both 942c and 952y in two DIFFERENT records.

Run a search where both test records are present. Test to see if itype types are presented in the facets block (both OPAC and staff).

Click on the itype (say BK), both the test records should appear in the refined results. This shows that the feature works for both 942c and 952y.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Ian Walls <koha.sekjal@gmail.com>

QA Comment: fixed capitalization in template includes according to HTML4 coding
guideline ("Item types" instead of "ItemTypes")

Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-06-20 22:27:21 +02:00
Chris Cormack
509d673f10 Bug 7941 : Fix version numbers in modules
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-06-11 17:29:38 +02:00
Galen Charlton
7a0f991764 Bug 7818: use Title-cover for title phrase weighting
This is offerred as a compromise alternative to creating
a new Title-rel index to avoid having the statement of
responsiblity unduly affect field weight when using the DOM
filter and MARC21 -- the problem with creating a Title-rel index
is that it would *force* reindexing upon upgrade.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-06-09 11:44:11 +02:00
Jared Camins-Esakov
fab56d3257 Bug 7847: OPAC search dies with Plack
Due to a dependency cycle between C4::Search and C4::Items, searches
in the OPAC die spectacularly under Plack. This counter-patch extends
dpavlin's solution and replaces use with require for C4::Search in
C4::Items and for C4::Items in C4::Search.

Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-06-05 15:46:20 +02:00
Stéphane Delaune
ab35e1f5f7 Bug 8041: lost items are no longer available in search
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-05-15 16:44:50 +02:00
Magnus Enger
44ab1c531d Bug 7445 - Clicking on a tag gives "Language ... does not exist"
To test:
- Syspref QueryStemming = Try
- Install Norwegian bokmål:
  cd misc/translator/
  perl translate install nb-NO
- Go to Home › Administration › System Preferences > I18N/L10N
  and enable "Norsk bokmål(nb-NO)" for opaclanguages as well as
  setting opaclanguagesdisplay = Allow
- Make sure you have selected "Norsk bokmål" as the active language
  in the OPAC
- Find a record that has a tag (which does not contain any digits)
- Click on the tag and see that you get the error in the title of
  this bug
- Apply the patch
- Click on the tag again and the error should be gone

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Easy to test with a great test plan. Works nicely.

Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-04-06 17:49:35 +02:00