Commit graph

554 commits

Author SHA1 Message Date
Mark Tompsett
d5986c9b97 Bug 19040: Refactor GetMarcBiblio parameters
Change parameters to a hashref.

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Looks good to me.
Two calls in migration_tools/22_to_30 still in old style.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2017-08-25 10:23:42 -03:00
aabf001232 Bug 18854: Make sure offset will not be < 0 - protect from DoS
There was a bug that meant a very large offset in the search params
will cause the search script to run forever (or long enough to crash
the machine)

To test

1/ Get ready with sudo top so you can kill the thread before it causes
your machine to OOM
2/ Hit a page like yourdomain.com/cgi-bin/koha/opac-search.pl?q=1&offset=-9999999999999999999
3/ Notice the process runs for a long time
4/ Kill the process
5/ Apply the patch
6/ Hit the page again, notice the it loads (offset is set to zero)
7/ Do the same to search in the staff client

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2017-07-14 12:02:04 -03:00
8d5b4306e0 Bug 17835: Replace GetItemTypes with Koha::ItemTypes
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>

Signed-off-by: Lari Taskula <lari.taskula@jns.fi>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-04-14 10:43:51 -04:00
7df47b48ba Bug 17821 - due date in intranet search results should use TT date plugin
Intranet search results displays due date from item onloan.
This should use the TT date plugin.

Test plan :
- set syspref dateformat not on yyyy-mm-dd, for example dd/mm/yyyy
- checkout an item
- at intranet, perform a search where you see the item
=> You must see : "date due : dd/mm/yyyy"

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-04-14 10:43:04 -04:00
50f4024055 Bug 15541 - Prevent normalization during matching/import process
This patch allows you to use the "qualifier,qualifier" syntax
in the Record Matching Rules "Search Index" when using the QueryParser.
While QueryParser doesn't support this syntax, it will now fallback
correctly to non-QueryParser functionality. Without the patch, your search
will just fail silently.

This patch also adds a "skip_normalize" option to C4::Search::SimpleSearch(),
and uses the option during C4::Matcher::get_matches. This prevents
the s/:/=/g and s/=/:/g normalization. This normalization is heavy-handed,
and while it is necessary sometimes to generate a valid CCL query or
QueryParser query, C4::Matcher::get_matches() already produces a valid
CCL query, so we don't need to do this normalization.

Additionally, this normalization causes problems when you use a
Zebra register which isn't normalized: namely the "u" register.
Strings are stored "as is", so http://localhost/koha/resource/1 is
stored as is during indexing. When you search, you need to pass
the exact same thing through the query to get a match. Using
http=//localhost/koha/resource/1 in your query will yield zero results.

_TEST PLAN_

0) Apply patch

1) Create a Record Matching Rule in Koha with the following details:
    Matching rule code: TEST
    Description: Test
    Match threshold: 100
    Record type: Bibliographic
    Match point 1:
        Search index: id-other,st-urx
        Score: 100
        Tag: 024
        Subfields: a
        Normalization rule: None
2) Create a record in Koha with an indexable URI
    2a) Default framework
    2b) 024 $a http://koha-community.org/test $2 uri
    2c) 040 $c test
    2d) 245 $a This is a test record
    2e) 942 $c Books
    2f) Save (save again if cautioned about missing fields as these should autofill)
3) Do a full re-index of Zebra
4) Download your record from Koha as a .mrc file (ie isomarc, binary marc, etc)
5) Go to "Stage MARC records for import"
    5a) Upload your .marc file.
    5b) Change your "Record matching rule" to "Test"
    5c) Click Stage for import
9) It should say "1 records with at least one match in catalog per matching rule "Test".

NOTE: For completeness, you can go through this process on a clean master branch,
and note that it will say '0 records with at least one match in catalog per matching rule "TEST"'

Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-03-31 13:52:44 +00:00
8db17f9eca Bug 17764: (bug 17556 follow-up) Fix search for logged out users and lost items
If you are non logged-in and you the search result contain lost items,
you will get:
Can't call method "category" on an undefined value at
/home/liz/koha-src/koha/C4/Search.pm line 2091.

This is because bug 17556 assumed that $userenv was not defined when the
user is logged out. Actually it is, with non defined or empty string
values.

Test plan:
Do a search in the opac that would turn up a whole list of results (and
not just that one) with the lost item included.
 => Without this patch you should get an error
 => With this patch applied you should see the search results

Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
2016-12-22 20:25:05 +00:00
662f5625c2 Bug 17556: Koha::Patrons - Remove GetHideLostItemsPreference
The subroutine C4::Members::GetHideLostItemsPreference can easily be
replaced with Koha::Patron->find(42)->category->hidelostitems

Test plan:
Create 2 patron categories, 1 with "Lost items in staff client" set to
"shown" and another one to "Hidden by default"
Create 2 patrons using them
On the result search page, the detail page of a record, the item list
page and the page to place a hold, make sure the lost items are
shown/hidden as expected

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-12-09 18:53:40 +00:00
167c74bfd3 Bug 15822: Fix 500 error when searching if items.onloan=0000-00-00
The calls
  output_pref({ dt => dt_from_string( $date ) })
are wrong and should be replaced with
  output_pref({ str => $date })
For better error handling.

Here we fix the problem of items.onloan when searching

Test plan:
- Set items.onloan=0000-00-00 (UPDATE items SET onloan='0000-00-00')
This can come from old data or bad migration
- Execute a search
=> Without this patch you get
Can't locate object method "ymd" via package "dateonly" (perhaps you forgot to load "dateonly"?) at /home/vagrant/kohaclone/Koha/DateUtils.pm line 225.
=> With this patch you won't get the error

Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-12-09 16:22:10 +00:00
b19bd97556 Bug 17278: Fix "show all items" link
Then we need to remove the "available" part from the query.
They are really awkward patches...

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-11-22 17:44:43 +00:00
5cc7c17c64 Bug 17278: Use available limit when requesting zebra when ccl query is used
When a ccl query is used, the buildQuery subroutine does not handle
the available limit (not an index).
This available limit is handle later in the subroutine.
This affect the author links on the detail page for instance (an=xx).
A much better solution would be to keep an 'available' zebra index up-to-date.

Test plan:
(OPAC or staff interface, it does not matter)
- Launch a search, click on a result and then on an author link to
  launch another query (an:xx)
- Limit to available items without the 'facet'
=> Without this patch you won't get any results
=> With this patch applied you should get relevant result (regarding the
known bugs 16970, 13715, 13658, 5463, etc.)

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-11-22 17:44:36 +00:00
a16a750180 Bug 17642: Add and use get_descriptions_by_koha_field
Ok I am silly, we needed to replace to use the cache mechanism for
search_by_koha_field, not find_by_koha_field...
Let's create another subroutine

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-11-18 15:52:00 +00:00
baf78b2a33 Bug 17253: Koha::AuthorisedValues - Remove GetKohaAuthorisedValues
This patch replaces the call to C4::Koha::GetKohaAuthorisedValues with
Koha::AuthorisedValues->search_by_koha_field

Test plan:
AV descriptions should be displayed on the following pages:
- XSLT view - location and ccode
- Bibliographic detail, moredetail and OPAC pages - location, ccode, copynumber
- returns - location
- opac-basket - ccode, location
- The 3 reports: catalogue_stats.pl, issues_stats.pl and
  reserves_stats.pl - location, ccode

Signed-off-by: Claire Gravely <claire_gravely@hotmail.com>

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-10-21 15:35:34 +00:00
0b6f129ac3 Bug 17250: Do not retrieve marc subfield structure when the authorised value is not defined
GetAuthValCode did not return anything if the authorised_value column
was not defined. Our new calls to Koha::MarcSubfieldStructures->search
should behave the same

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-10-21 15:21:27 +00:00
d019f86f67 Bug 17250 - Koha::AuthorisedValues - Remove GetAuthValCode
The subroutine C4::Koha::GetAuthValCode returned the authorised value
category for a given kohafield.
This can be acchieve easily using a new Koha::AuthorisedValues->search_by_koha_field
method which will mimic search_by_marc_field.

Test plan:
Confirm that the description is correctly displayed on the following
pages:
- detail and moredetail of a bibliographic page (itemlost, damaged, materials)
- Set AcqCreateItem=ordering and receiving items.
The description for notforloan, restricted, location, ccode, etc.
field should be displayed.
- Items search form
- On the checkout list from the circulation.pl and returns.pl
pages, the description for "materials" should be displayed

Note that GetKohaAuthorisedValuesMapping is going to be removed on bug
17251.

Followed test plan, works as expected.
Signed-off-by: Marc Véron <veron@veron.ch>

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-10-21 15:21:27 +00:00
bb1e9c500c Bug 17248 - Koha::AuthorisedValues - Remove GetKohaAuthorisedValueLib
The subroutine C4::Koha::GetKohaAuthorisedValueLib just retrieves a description
(lib) for a given authorised value.

We can easily replace it using:
  Koha::AuthorisedValues->search({ category => $cat, authorised_value => $av })->lib
or
  Koha::AuthorisedValues->search({ category => $cat, authorised_value => $av })->opac_description

Test plan:
- On the detail page of a bibliographic record, the description for notforloan,
restricted and stack (?) should be correctly displayed
- View a shelf, the location (LOC) description should be displayed
- On the search result page, the location description should be displayed in the
  facets
- Set AcqCreateItem=ordering and receiving items.
The description for notforloan, restricted, location, ccode, etc. field
should be displayed.
- When creating item in the acquisition module, the dropdown list for
  field linked to AV should display the AV' descriptions
- On the transfers page, the description of the location should be
  displayed.
- On the checkout list from the circulation.pl and returns.pl pages, the
  description for "materials" should be displayed
- Fill some OPAC_SUG AV and create a suggestion, the reason dropdown
  list should display the description of OPAC_SUG

Signed-off-by: Claire Gravely <claire_gravely@hotmail.com>

Signed-off-by: Katrin Fischer  <katrin.fischer@bsz-bw.de>

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
2016-10-11 13:11:55 +00:00
974bcdf260 Bug 17193: Remove C4::Search::SearchAcquisitions
This subroutine is not used and can be removed

Test plan:
  git grep SearchAcquisitions
should not return any results.

Signed-off-by: Claire Gravely <claire_gravely@hotmail.com>

Signed-off-by: Katrin Fischer  <katrin.fischer@bsz-bw.de>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-09-09 14:07:57 +00:00
632c78c073 Bug 15758: [QA Follow-up] Corrections related to GetBranches
[1] C4/Search
A call to Koha::Libraries is added to routine pazGetRecords, but the
results of that call are not used. So removing it again.

[2] catalogue/itemsearch.pl
Although A=>B=>C=>D works, we'd better use here A=>B, C=>D.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-09-08 14:36:06 +00:00
0079ca7c72 Bug 15758: [QA Follow-up] Some corrections related to GetBranchName
[1] Acquisition.pm
The lines filling $row in GetBasketGroupAsCSV may have side-effects when
the library name is not found. This change restores former behavior. Just
theoretically more safe.
Note that it also contained a typo: $row->{deliveryplace} should have been
$row->{$place}.

[2] Auth.pm
checkauth: $branchname = Koha::Libraries->find($branchcode)->branchname;
Should normally be fine, but I rather have an empty string here than
crashing on "Can't call method branchname on undefined value".
Same for sub check_api_auth.
Note that this holds for a larger number of calls, but I am adding a check
here because it is checkauth.
Also removed a duplicate use Koha::Libraries-statement.

[3] Search.pm
Also removed a duplicate use statement for Libraries.

[4] svc/holds
Added an (explicit) use statement for Koha::Libraries.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-09-08 14:36:05 +00:00
df97814f30 Bug 15758: Koha::Libraries - Remove GetBranches
Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-09-08 14:36:03 +00:00
9b9803b69c Bug 15758: Koha::Libraries - Remove GetBranchesLoop
Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-09-08 14:36:02 +00:00
19a977dc7b Bug 15758: Koha::Libraries - Remove GetBranchName
This is the fourth and last patch set to remove C4::Branch.
The real purpose of this patch is to standardise and refactor some code
which is related to the libraries selection/display.
Its unconfessed purpose is to remove the C4::Branch package.

Before this patch set, only 6 subroutines still existed in the C4::Branch
package:
- GetBranchName
- GetBranchesLoop
- mybranch
- onlymine
- GetBranches
- GetBranch

GetBranchName basically returns the branchname for a given branchcode.
The branchname is only used for a display purpose and we don't need to
retrieve it in package or pl scripts (unless for a few exceptions).
We have a `Branches` template plugin with a `GetName` method which does
exactly this job.
To achieve this removal, we will use this template plugin and delete the
GetBranchName from pl and pm files.
The `Branches.all()` will now select the library of the logged in user
if no `selected` parameter has been passed.
This new behavior could cause regressions, for instance there are some
places where we do not want an option preselected (batch item
modification for instance), keep that in mind when testing.

GetBranchesLoop took 3 parameters: $branch and $onlymine.
The first one was used to set a "selected" flag, for a display purpose:
select an option in the libraries dropdown lists.
The second one was useless: If not passed or set to 0, the
`C4::Branch::onlymine` subroutine was called.
This onlymine flag was use to know if the logged in user was able to see
other libraries infos.
A patron can see the infos from other libraries if IndependentBranches
is not set OR if he has the superlibrarian permission.
Prior to this patch set, the "onlymine test" was done on different
places (neworderempty.pl, additem.pl, holidays.pl, etc.), including the
Branches TT plugin. In this patch set, this test is only done on one
place (C4::Context::only_my_library, code moved from
C4::Branch::onlymine).
To accomplish the same job as this subroutine, we just need to call the
`Branches.all()` method from the `Branches` TT plugin. It already
accepts a `selected` parameter to set a flag on the option to select.
To avoid the repetitive
  [% IF selected %]<option selected="selected">[% ELSE %]<option>[% END %]
pattern, a new `html_helpers` TT include file has been created, it
defines an `options_for_libraries` block, which takes a `selected`
parameter. We could imagine to use this include file for other
selects.

The 'mybranch` and `onlymine` subroutines of the C4::Branch package have
been moved to C4::Context. onlymine has been renamed with
only_my_library. There are only 4 occurrences of it, against 11 before
this patch set.
There 2 subroutines are Context-centric and it makes sense to put them
in `C4::Context` (at least it's the least worst place!)

GetBranches is the tricky part of this patch set: It retrieves all the
libraries, independently of the value of IndependentBranches.
To keep the same way as the existing calls of `Branches.all()`, I have
added a `unfiltered` parameter. If set, the `Branches.all()` will call
a usual Koha::Libraries->search method, otherwise
Koha::Libraries->search_filtered will be called. This new method will
check if the logged in user is allowed to see other libraries or only
its library.
Note that this `GetBranches` subroutine also created a `category` key:
it allowed to get the list of groups (of libraries) where this library
existed. Thanks to a previous patch set (bug 15295), this value was
not used anymore (I may have missed something!).

Note that the only use of `GetBranch` was buggy (see bug 15746).

Test plan (for the whole patch set):
The best way to test this whole patch set is to test with 2 instances: 1
with the patch set applied, 1 using master, to be sure there is no
regression.
It would be good to test the same with `IndependentBranches` and the
without `IndependentBranches`.
No difference should be found.
The tester must focus on the library dropdowns on as many forms as
possible.
You will notice changes in the order of the options: the libraries will
now be ordered by branchname (instead of branchcode in some places).
A special attention will be given to the following page:
- acqui/neworderempty.pl
- catalogue/search.pl
- members/members-home.pl (header?)
- opac/opac-topissues.pl
- tools/holidays.pl
- admin/branch_transfer_limits.pl
- admin/item_circulation_alerts.pl
- rotating_collections/transferCollection.pl
- suggestion/suggestion.pl
- tools/export.pl

Notes for QA:
- There are 2 FIXMEs in the patch set, I have kept the existing behavior,
but I am not sure it's the good one. Feel free to open a bug report and
I will fill a patch if you think it's not correct. Otherwise, remove the
FIXME lines in a follow-up patch.
- The whole patch set is huge and makes a lot of changes.
But it finally will tremendously reduce the number of lines:
716 insertions for 1910 deletions

Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-09-08 14:36:01 +00:00
4884f3a00a Bug 17107 - Add ident and Identifier-standard to known indexes
Add ident and Identifier-standard to known indexes in C4::Search::getIndexes().
Those indexes can be very useful, for example for IdRef feature.

Test plan :
- Make sure some records have a field indexed with Identifier-standard, ISBN=1234 for example
- Perform a search /cgi-bin/koha/opac-search.pl?idx=ident,phr&q=1234
=> you find the record
- Perform a search /cgi-bin/koha/opac-search.pl?q=ident:1234
=> Without patch : you get no results
=> With patch    : you find the record
Idem for 'Identifier-standard'

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-08-18 16:05:15 +00:00
Barton Chittenden
84f51549c9 Bug 6499: Add Zebra index "Other-control-number" covering MARC21 035$a, 035$z and 035 (entire tag)
1) Apply patch
2) Make sure that you have a bib that has MARC21 035$a (and possibly also 035$z) populated.

pre 3) Replace all modified zebra files and restart zebra server

3) Rebuild zebra: misc/migration_tools/rebuild_zebra.pl -x -b -z
4) Add the following to the intranetuserjs syspref:

$(document).ready(function(){
    // Add Other Control Number to advanced search
    if (window.location.href.indexOf("catalogue/search.pl") > -1) {
        $(".advsearch").append('<option value="Other-control-number">Other Control Number</option>');
    }
});

5) Do an advanced search, select "Other Control Number" from the search menu, then add the Other Control Number in 035$a for the bib specified in step 1.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Works, no koha-qa errors

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-08-09 10:13:10 +00:00
4141d9524c Bug 15263: Bug 15263: (follow-up 2) XSLT display fetches sysprefs for every result processed
Don't retrieve prefs if we won't need them

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-05-04 13:40:35 +00:00
cb36f36382 Bug 15263: (follow-up) XSLT display fetches sysprefs for every result processed
Set variables ($sysxml, $xslfilename, $lang) if they are not passed to
the subroutine. This happens from catalogue/detail.pl,
opac/opac-shelves.pl, opac/opac-tags.pl and virtualshelves/shelves.pl.

Signed-off-by: Mirko Tietgen <mirko@abunchofthings.net>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-05-04 13:40:34 +00:00
Mirko Tietgen
dd73cb4ce9 Bug 15263: XSLT display fetches sysprefs for every result processed
On search, every single result goes through some XSLT processing.
This includes fetching the relevant sysprefs every single time.
We should do it only once per search.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-05-04 13:40:31 +00:00
85e7d186ec Bug 16167: Remove Authorised value images prefs
There are 2 prefs to drive this feature: StaffAuthorisedValueImages and
AuthorisedValueImages. AuthorisedValueImages is not added by
sysprefs.sql and does not appear in updatedatabase.pl, we could easily
imagine that nobody uses it.

With XSLT enabled, the feature is only visible on a record detail page
at the OPAC, if AuthorisedValueImages is set. Otherwise you need to turn
the XSLT off. In this case you will see the images on the result list
(OPAC+Staff interfaces) and OPAC detail page, but not the Staff detail
page.

This patch suggests to remove completely this feature as it does not
work correctly.

The ability to assign an image to an authorised value is now always
displayed, but the image will only be displayed on the advanced search
if defined.

Test plan:
Confirm that the authorised value images are no longer visible at the
opac and the staff interfaces.
The prefs should have been removed too.

Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-04-29 13:59:58 +00:00
Jesse Weaver
37ea74669c Bug 12748: (QA followup) fix several tiny QA tools warnings
* admin/searchengine/elasticsearch/mappings.yaml:
    * Fix indentation so QA tools believe it's valid YAML
  * C4::AuthoritiesMarc: Remove tab character
  * C4::Search: Fix POD
  * Koha::BiblioUtils: remove "return undef"
  * Koha::Schema::Result::Search*: rename type to _type
  * Koha::SE::ES::Search:
    * Fix spelling/@_[0]
    * Add my to loop variable
    * Remove unused var $error
    * Add use Modern::Perl
    * Fix undefined variables
  * Koha::ES::Indexer: fix POD
  * Koha::SE::ES::QueryBuilder:
    * Don't define variables in conditionals
    * Fix POD
  * Koha::SE::Zebra::Search:
    * Fix debug warns/POD/barewords
    * Koha::SE::Zebra::Search: add use Modern::Perl
  * kohastructure.sql: remove old, unused elasticsearch_mappings table,
    add new tables
  * labels/label-item-search.pl: remove C4::Dates
  * opac/elasticsearch.pl: remove old file

Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
2016-04-26 20:20:13 +00:00
8d04291153 Bug 12478: Display facet terms ordered by number of occurrences
By default ES returns the facet terms ordered by most used, which makes
sense.

This patch removes resort done in the scripts (catalogue/search.pl and
opac/opac-search.pl) and moves it to the module.

For Zebra it's now done in C4::Search::getRecords, and there is no
change to expect (still alphabetically).

On the Elastic search side, we could imagine to let the library define
the order of the facets. The facet terms are now sorted by most used.

To test easily this change, turn on the displayFacetCount pref.

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
2016-04-26 20:20:12 +00:00
Robin Sheat
a792e84719 Revert "Bug 12478: Display facet terms ordered by number of occurrences"
This reverts commit cd4905c2969b067476881016d0b03271f0bcc7c8.

This commit caused an error in C4::Search::GetFacets when running in
zebra mode.

Conflicts:
	Koha/SearchEngine/Elasticsearch/Search.pm

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
2016-04-26 20:20:12 +00:00
246f2dddaf Bug 12478: Display facet terms ordered by number of occurrences
By default ES returns the facet terms ordered by most used, which makes
sense.

This patch removes resort done in the scripts (catalogue/search.pl and
opac/opac-search.pl) and moves it to the module.

For Zebra it's now done in C4::Search::getRecords, and there is no
change to expect (still alphabetically).

On the Elastic search side, we could imagine to let the library define
the order of the facets. The facet terms are now sorted by most used.

To test easily this change, turn on the displayFacetCount pref.

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
2016-04-26 20:20:11 +00:00
Robin Sheat
1c737eaaf8 Bug 12478: a replacement to the SimpleSearch interface implemented
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
2016-04-26 20:20:07 +00:00
Robin Sheat
7dbd13e66f Bug 12478 - pile of elasticsearch code
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
2016-04-26 20:20:03 +00:00
Barton Chittenden
690ab60da2 Bug 14277: add zebra indexes for lexile that respect 521 indicator 1.
Added the following indexes:

Interest-age-level | 591$a ind1=1
Interest-grade-level | 591$a ind1=2
lexile-number | 591$a ind1=8
Reading-grade-level | 591$a ind1=0

Moved 'lex' from a zebra index to a ccl alias to lexile-number.

Changed the handling of st-numeric in C4/Search.pm to allow for search ranges.

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Hector Castro <hector.hecaxmmx@gmail.com>
Works as advertised
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>

Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
2016-04-07 10:35:18 -06:00
c564f30155 Bug 16169: Change prototype for C4::Biblio::TransformMarcToKoha
subroutines should not take $dbh in parameter.
C4::Biblio::TransformMarcToKoha has it and does not use it.

Test plan:
Look at the patch and confirm that all occurrences of
TransformMarcToKoha have been modified.

Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Brendan Gallagher brendan@bywatersolutions.com
2016-04-07 00:04:21 +00:00
798d38e4c7 Bug 16011: $VERSION - Remove comments
perl -p -i -e 's/^.*set the version for version checking.*\n//' **/*.pm

+ manual adjustements

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>

Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
2016-03-24 17:20:29 +00:00
017699c345 Bug 16011: $VERSION - Remove the $VERSION init
Mainly a
  perl -p -i -e 's/^.*3.07.00.049.*\n//' **/*.pm
Then some adjustements

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>

Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
2016-03-24 17:20:28 +00:00
3830d78d46 Bug 16011: $VERSION - remove use vars $VERSION
perl -p -i -e 's/^(use vars .*)\$VERSION\s?(.*)/$1$2/' **/*.pm

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>

Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
2016-03-24 17:20:26 +00:00
Nicole C Engard
39f9b9ddb0 Bug 15694: Add aliases for date/time last modified
This patch will add indexes for Date/time-last-modified.

To test:

1. apply patch
2. reindex
3. search for dtlm:DATE and date-time-last-modified:DATE
4. confirm that you get results

Signed-off-by: Hector Castro <hector.hecaxmmx@gmail.com>
Works as advertised

Signed-off-by: Frédéric Demians <f.demians@tamil.fr>
  I confirm Hector signing-off. A simple Zebra server restart suffice to get
  working the searches on date-time-last-modified and dtlm.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
2016-03-11 21:56:50 +00:00
8627ec5f6a Bug 4941: Remove the singleBranchMode system preference
The singleBranchMode system preference does not make sense.
Either the install has only 1 library defined or several. In both case,
we can easily guess the behavior to follow.

So the idea of this patch is to replace the fetch of this syspref with a
call to count the number of libraries defined in DB.

Test plan:
1/ From a fresh Koha install, execute the DB entry to remove the pref.
2/ Define only 1 library
3/ Confirm that Koha behaves the same as before (try to change your
library, look at the facets)
4/ Create another library (or more) and reinsert the pref and set it:
  insert into systempreferences (variable, value)
    values('singleBranchMode', 1);
5/ Execute the DB entry
You should get a warning message.
6/ Repeat 3.

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>

Does what it says, but will change behaviour for any Koha install that
has 2 branches defined, One circulation, and this preference set.
If that is an acceptable change, we might need to make sure this is noted well in the
release notes.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-02-26 12:13:09 +00:00
7ab9d7d91d Bug 13022 [QA Followup] - Allow the pref to be unset for unlimited item status checks
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>

Signed-off-by: Brendan Gallagher brendan@bywatersolutions.com
2015-12-31 18:11:21 +00:00
d3184fa85b Bug 13022 - Hardcoded limit causes records with more than 20 items to show inaccurate statuses
If a record has more than 20 items, all the items over 20 will show as
available on the search results even if they are not!

This is a hard coded limit in the Search module. This number should be
configurable.

Test Plan:
1) Create a record with more than 20 items
2) Set all the items to waiting holds or in transit
3) Search for results that will include that item
4) Note some say they are available even though they are not
5) Apply this patch
6) Run updatedatabase.pl
7) Set the new system preference MaxSearchResultsItemsPerRecordStatusCheck
   to a number larger than the number of items on your record
8) Re-run the search
9) Note that the hold and transit statuses for the items are now correct

Signed-off-by: Andreas Hedström Mace <andreas.hedstrom.mace@sub.su.se>

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>

Signed-off-by: Brendan Gallagher brendan@bywatersolutions.com
2015-12-31 18:11:21 +00:00
bc31dc85f9 Bug 9819 [QA Followup] - Removed missed stopwords refs, remove Schema file
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2015-12-30 15:54:56 +00:00
e83ff6d633 Bug 9819: Make the tests pass
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2015-12-30 15:50:15 +00:00
826326064e Bug 9819 - 'stopwords'-related code removed
This patch removes code related to stopwords usage. The following methods are removed:

C4::Search->remove_stopwords
C4::Context->stopwords
C4::Context->_new_stopwords

And the buildQuery API was changed (removed the \@removed_stopwords return value).
A follow-up is provided for database changes, to make rebasing easier.

To test:
- Apply this patch
- Do some searches in both intranet and opac interfaces
- Nothing should break

Sponsored-by: Universidad Nacional de Córdoba
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2015-12-30 15:49:35 +00:00
Marc Véron
b8b0b1370f Bug 14985: Remove C4::Dates from files in folder C4/*.pm (part one)
This patch removes C4::Dates from following files in folder C4:

- C4/Members.pm
- C4/Reserves.pm
- C4/Search.pm
- C4/Utils/DataTables.pm
- C4/Utils/DataTables/Members.pm
- C4/VirtualShelves/Page.pm

To test:
 -run tests as appropriate,
- have a close look at the code changes
- try to find regressions

http://bugs.koha-community.org/show_bug.cgi?id=14985

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>

Bug 14985: (followup) Remove eval if dates come from database

This patch removes some evals from date-formatting where the dates come
from the database.

See comments #7 - #9

Additionaly, C4/VirtualShelves/Page.pm is removed from the patches (obsolete).

Bug 14985: (followup) Remove C4::Dates from C4/Overdues.pm

Ths patch removes a stray C4::Dates from C4/Overdues.pm

- To test got to a patron who has overdues
  (Home > Circulation > Checkouts > [Patron])
- Print overdues

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2015-11-13 11:24:54 -03:00
Hector Eduardo Castro Avalos
de272e5b41 Bug 14198: RDA: Indexing 264 field (Zebra)
This patch add zebra indexes to RDA 264 field.
The new Provider index is added too.
QA comments corrected.

To test:
1) Download RDA records with 264 fields from this attachment <http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=36825>. Import the file and re-index/rebuild zebra. These records contain 260 and 264 fields per record.
2) Do a search with pb:Bethany two records will appear with title The guardian. Search with pl:Minneapolis too, the two records will appear.
3) Select one record of both records and delete the 260 field keeping the 264 field and save, rebuild your zebra.
4) Search again with pb:Bethany and just one record will appear. Thats mean 264 is not indexed.
5) Apply patches.
6) Rebuild your zebra but this time all biblio records.
7) Search again with pv:Bethany or Provider:Bethany, this time will appear the two records, 264 is indexed. Note that if you search again with pb only one record appear. This is because the suggestion of LOC.
10) Search with copydate:2013 only launch records with 260 fields and pv:2013 show both fields, i.e., 260 and 264.
11) Apply QA Test Tools

Sponsored-by: Universidad de El Salvador
Signed-off-by: Nick Clemens <nick@quecheelibrary.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2015-11-02 11:41:36 -03:00
d9426ddfc5 Bug 14100: Fix Search.t tests
So, this one is VERY weird, let me try to explain what I have
understood.

Bisecting using run prove t/db_dependent/Search.t, I have found that the
following commit make the test fail:
  commit 0f63f89f66
    Bug 14100: Generic solution for language overlay - Item types
The error is
  DBI bind_columns: invalid number of arguments: got handle + 0, expected handle + between 1 and -1
  Usage: $h->bind_columns(\$var1 [, \$var2, ...]) at /usr/lib/i386-linux-gnu/perl5/5.20/DBI.pm line 2065.

Note that the interface (admin/itemtypes.pl) which calls the same
subroutine with the same parameter (style => 'array') works great.

The problem comes from the change in C4::Search::searchResults, if I
only apply the change done to this subroutine on 0f63f89f^1, I reproduce
the issue.

Looking closely at how %itemtypes is built, we could actually call
GetItemTypes with the style => 'hash' to get exactly what we want.
The following piece prove it for you:
    use Test::More;
    use C4::Koha;
    my $i = GetItemTypes;
    my $j = GetItemTypes(style => 'array');
    my %itemtypes;
    for my $itemtype ( @$j ) {
        $itemtypes{ $itemtype->{itemtype} } = $itemtype;
    }
    is_deeply( \%itemtypes, $i);

So changing the code accordingly and just forget this last hour...

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2015-10-29 08:47:33 -03:00
d58f2cfaf6 Bug 14100: Fix some missing occurrences
This patch fixes:
- reports/bor_issues_top.pl
- sort order
- adv search and search results
- opac-topissues.pl

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2015-10-27 12:34:07 -03:00
Jonathan Druart
0f63f89f66 Bug 14100: Generic solution for language overlay - Item types
Test plan:
1/ update the Schema (misc/devel/update_dbix_class_files.pl)
2/ Translate templates for some languages (es-DE, de-DE for instance)
3/ Enable them in the pref (search for 'lang') for the staff interface
4/ Go on the item type admin page (admin/itemtypes.pl)
5/ Edit one
6/ Click on the 'translate for other languages' link
7/ You are now on the interface to translate the item type's description
in the languages you want. So translate some :)
8/ Go back on the item type list view (admin/itemtypes.pl)
9/ You should see the original description (non translated)
10/ Switch the language
11/ You should see the translated description in the correct language.
If the description is non translated, the original description is
displayed.

12/ On the different page where the item type is displayed, confirm that
the translated description appears.

Think further / Todo:
1/ Update all occurrences of the item type's description (DONE)
2/ Implement for authorised values
3/ Implement for syspref value (at least textarea)
4/ Implement for branch names
5/ Centralize all the translation on a single page in the admin area
...
N/ Implement a webservice to centralize all the translations and give
the ability to sync the item types/authorised values description with
the rest of the world (push and pull).

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2015-10-27 12:34:06 -03:00