Koha/C4
Mathieu Saby b6118db2f5 Bug 11202: Improve UNIMARC biblio indexing
This patch makes the following changes to UNIMARC biblio indexing :
A. Changes to UNIMARC conf files
1. add comments to biblio-koha-indexdefs.xml
2. make biblio-koha-indexdefs.xml more compact by grouping some
   declarations
   Ex : 200$f and 200$g => one declaration for 200$fg
3. suppress unneeded declarations (indexing of some 4XX fields and 6XX
   fields not in unimarc format)
4. unindex some (sub)fields unneeded by most users (318, 207,230,210a,
   215, 4XXd)
5. change the way 308 field is indexed (no visible changes)
6. replace Title-host with Host-item -- see bug 11119
7. index 208 in Material-Type -- see bug 11119
8. index 100 pos 8-9 and 9-12 in pubdate:y and pubdate:n
9. index 100 pos 8-9 in pubdate:s instead of 210$d
10. Index all subfields of note 334 and 327 in note index
11. Index 304 and 327 in title index as well as note index
    327 can contain a list of titles included in a work
    304 can contain the title of the original work in case of a
    translation
12. Index 314 in author index as well as note index
    314 can contain authors not mentionned in 200$f/g (the 4th, 5th etc.
    author)
13. Index 328 note in Dissertation-information as well as note
14. Index 328$t in Title

B. Changes to ccl.properties :
1. add a new index Dissertation-information (1056)
2. fix EAN, pubdate and acqdate (they were not linked with bib1 attributes)

C. Changes to Search.pm
1. add Dissertation-information and suppress Title-host and UPC

D. Changes to QP config file queryparser.yaml
1. add Dissertation-information
2 fix EAN, pubdate and acqdate

Test plan :
If you cannot test in GRS1, test only in DOM, as GRS will be deprecated.

1. Apply the patch in a UNIMARC Koha running with DOM and ICU
2. copy src/etc/searchengine/queryparser.yaml into the main config
   directory of QP
3. copy src/etc/zebradb/ccl.properties into the main config directory
   of Zebra
4. copy src/etc/zebradb/marc_defs/unimarc/biblio/* into the main config
   directory of Zebra
5. reindex biblios (rebuild_zebra.pl -r -b -x -v)
6. test note index : make some searches on 334$b or 327$b
7. test author index : make some searches on 314 field
8. test title index : make some searches on 304 and 327 field, make a
   search on 328$t subfield
9. test dissertation-information index : make some searches on 328 field
10. In a record, put in the dates of 100 fields the values "1000" (1st
    date) and "1001" (2d date) ; try to search a book written in year
    1000, you should find the record ; idem for year 1001
11. make some searches and sort by date. It should work better as before,
    especially if you have values like "c2009" or "impr. 2010" in 210
    field
12. Regression test : make some searches on several indexes, like EAN,
    etc. It should work as before

Test 10-12 with and without Queryparser activated.
Be careful: with Queryparser activated, the index names (title,
dissertation-information...) must be entered in lowercase only.
Of course, to test search and sort by dates, you need to have full
records, with dates in 100 field as well as 210 field.

Signed-off-by: Paola Rossi <paola.rossi@cineca.it>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-02-19 21:01:15 +00:00
..
AuthoritiesMarc
Barcodes Bug 11539: removing 2 unused files 2014-01-14 20:55:28 +00:00
ClassSortRoutine Bug 9770: fix sorting of Dewey call numbers that contain prefixes 2013-07-15 16:12:47 +00:00
Creators Bug 10763 - [SIGNED-OFF] Update POD of C4::Creators::Lib::html_table() to use TT syntax 2013-08-20 14:31:34 +00:00
External Bug 10320: (follow-up) correct license statement 2013-09-08 07:04:45 +00:00
Form
Heading
ILSDI Bug 9823: Refactor return from GetReservesFromBiblionumber 2014-01-30 16:19:55 +00:00
Installer Bug 11757: remove dependency on POE 2014-02-15 01:38:15 +00:00
Labels Bug 11402: make Labels::_guide_box return undef if undefned data is passed 2013-12-30 16:17:35 +00:00
Linker
Members
OAI Bug 9295: Introduce operator equal/ notequal to OAI set mapping instead of hardcoded 'equal' value. 2013-10-10 23:03:30 +00:00
Output
Patroncards
Reports Bug 10761: (follow-up) use explicit return in C4::Reports::Guided::delete_report() 2013-08-21 14:45:41 +00:00
Search
Serials Bug 7688: (follow-up) update license statements 2013-10-30 02:56:32 +00:00
SIP Bug 11783: ensure CD field in SIP patron information response is populated 2014-02-19 16:55:13 +00:00
Utils Bug 11474: (follow-up) correct typos in POD 2014-01-06 05:46:32 +00:00
VirtualShelves Bug 10853: All existing routing to get a CSV should return a MARC csv 2013-10-11 02:16:33 +00:00
Accounts.pm Bug 2546: make description of standard fine types translatable 2013-12-05 18:44:33 +00:00
Acquisition.pm Bug 11777: ensure "created by" is displayed by the order receiving page 2014-02-19 17:13:26 +00:00
Auth.pm Bug 10952: (follow-up) clear seach history from session after saving it to DB 2014-01-10 16:49:01 +00:00
Auth_cas_servers.yaml.orig
Auth_with_cas.pm Bug 10927: remove disused C4::Utils module 2013-10-03 14:53:02 +00:00
Auth_with_ldap.pm Bug 10908: restore docs on principal_name LDAP option 2013-10-21 15:28:11 +00:00
AuthoritiesMarc.pm Bug 11533: fix authority searching with no sorting when QueryParser is enabled 2014-01-24 14:02:48 +00:00
BackgroundJob.pm Bug 10601: (follow-up) improvements to ->set() and ->get() 2013-09-18 17:23:44 +00:00
Barcodes.pm
Biblio.pm Bug 9823: Refactor return from GetReservesFromBiblionumber 2014-01-30 16:19:55 +00:00
Bookseller.pm Bug 7965: Silence warns in staff log 2014-01-10 15:42:09 +00:00
Boolean.pm Bug 10080 - Change system pref IndependantBranches to IndependentBranches 2013-05-22 07:58:23 -07:00
Branch.pm Bug 11051: remove unneccessary SQL queries in GetBranches 2014-01-10 16:02:39 +00:00
Breeding.pm Bug 11419: display Z39.50 search errors more completely 2013-12-27 00:25:39 +00:00
Budgets.pm Bug 10983: Remove unused private subroutines from C4::Budgets 2013-12-13 15:14:18 +00:00
Calendar.pm
Category.pm Bug 7919: FIX the "all" categories method 2013-01-02 16:50:52 -05:00
Charset.pm Bug 11730: ensure that C4::Charset loads C4::Context 2014-02-18 21:52:21 +00:00
Circulation.pm Bug 10277 - Add C4::Context->IsSuperLibrarian() 2013-12-30 15:47:23 +00:00
ClassSortRoutine.pm
ClassSource.pm Bug 10643: fix inappropriate uses of $sth->finish() in C4::ClassSource.pm 2013-08-09 15:32:22 +00:00
Context.pm Bug 11389: reenable Pg as a DB scheme that Koha can connect to 2014-01-13 20:56:14 +00:00
Contract.pm
CourseReserves.pm bug 8215: (followup) rename GetItemReservesInfo 2013-05-21 15:51:03 -07:00
Creators.pm
Csv.pm Bug 10853: All existing routing to get a CSV should return a MARC csv 2013-10-11 02:16:33 +00:00
Dates.pm
Debug.pm
Heading.pm
HoldsQueue.pm Bug 10243: (follow-up) prevent TransportCostMatrix from overriding library holds policy 2013-10-30 03:13:41 +00:00
HTML5Media.pm Bug 8377: Followup move style in a css file and do not pass template to a pm 2012-12-27 09:28:00 -05:00
Images.pm
ImportBatch.pm Bug 8015: (follow-up) trap exceptions thrown by SetUTF8Flag() 2013-10-31 22:48:59 +00:00
ImportExportFramework.pm Bug 11666: remove SQL as an option for MARC framework exports and imports 2014-02-05 19:48:27 +00:00
Input.pm
Installer.pm Bug 10523: Remove two obsolete routines from Installer.pm 2013-10-31 16:51:47 +00:00
ItemCirculationAlertPreference.pm
Items.pm Bug 11473 - add 'biblio' and 'item' to cataloguing logging info 2014-01-17 05:42:44 +00:00
ItemType.pm Bug 10513: display a warning/message when returning a chosen item type 2013-09-16 17:45:31 +00:00
Koha.pm Bug 11279: (follow-up) remove unnecessary check on number of quotes 2013-12-19 15:06:39 +00:00
Labels.pm
Languages.pm Bug 10309 - New OPAC theme based on Bootstrap 2013-10-14 23:13:05 +00:00
Letters.pm Bug 9788: (follow-up) for expirationdate in Letter.pm 2014-01-17 05:08:39 +00:00
Linker.pm
Log.pm Bug 7067 - OPAC Borrower Self Registration 2012-12-14 08:08:59 -05:00
Maintainance.pm
MarcModificationTemplates.pm Bug 8015: (follow-up) don't require userenv 2013-10-31 22:51:13 +00:00
Matcher.pm Bug 10230: no need to use SimpleSearch for matching auths 2013-05-19 08:40:13 -04:00
Members.pm Bug 10277 - Add C4::Context->IsSuperLibrarian() 2013-12-30 15:47:23 +00:00
Message.pm
NewsChannels.pm
Output.pm Bug 10016: force zero browser-side caching of SCO pages 2013-10-21 18:05:12 +00:00
Overdues.pm Bug 2720 - Overdues which debar automatically should undebar automatically when returned 2013-10-31 01:53:22 +00:00
Patroncards.pm
Print.pm
Ratings.pm
Record.pm Bug 11066: make RIS and Bibtex exports RDA compatible 2014-01-03 15:54:38 +00:00
Reports.pm
Reserves.pm Bug 9823: (follow-up) improve POD for C4::Reserves::GetReservesFromBiblionumber 2014-01-30 16:48:26 +00:00
Review.pm
Ris.pm Bug 11066: make RIS and Bibtex exports RDA compatible 2014-01-03 15:54:38 +00:00
RotatingCollections.pm Bug 11384: rename the collections_tracking.ctId column 2013-12-23 16:14:57 +00:00
Scheduler.pm
Scrubber.pm
Search.pm Bug 11202: Improve UNIMARC biblio indexing 2014-02-19 21:01:15 +00:00
Serials.pm Bug 10851: (follow-up) fix issues reported by QA script 2014-02-04 17:48:37 +00:00
Service.pm
ShelfBrowser.pm Bug 10856: (follow-up) if callnumbers are equal, order should be on itemnumber 2013-10-04 15:57:03 +00:00
SMS.pm
SocialData.pm
SQLHelper.pm Bug 11221: ensure that SQLHelper uses NULL rather than 0000-00-00 as default date value 2013-11-19 15:29:08 +00:00
Stats.pm
Suggestions.pm Bug 10277 - Add C4::Context->IsSuperLibrarian() 2013-12-30 15:47:23 +00:00
Tags.pm Bug 9136: C4::Tags not Plack-compatible 2012-12-22 15:47:48 -05:00
Templates.pm Bug 7684: multiple fixes for inventory 2013-11-01 00:10:49 +00:00
TmplToken.pm
TmplTokenType.pm
TTParser.pm
UploadedFile.pm
VirtualShelves.pm Bug 10386: improvements to VirtualShelves.t 2013-06-28 05:19:49 -07:00
XISBN.pm Bug 9728: XISBN unit test update 2013-06-18 09:46:37 -07:00
XSLT.pm Bug 10550: Fix database typo wthdrawn 2013-09-08 01:58:41 +00:00