I ran the xsltproc on both MARC21 and UNIMARC files (biblios and
authorities). With my follow-up the only changed one is this one.
I skipped NORMARC as it is supposed to be removed by now (so unused in
Norway).
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
This patch makes the generated xslt not mention index_sort_title unless
the entry is defined on the xml file. Otherwise there's a call to
<xslo:apply-templates mode="index_sort_title"/>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
This patch moves the code that generates the xsl for MARC21 biblio sorting
to it's own template that is only called when specified in the xml
To test:
1 - xsltproc etc/zebradb/xsl/koha-indexdefs-to-zebra.xsl etc/zebradb/marc_defs/marc21/authorities/authority-koha-indexdefs.xml > etc/zebradb/marc_defs/marc21/authorities/authority-zebra-indexdefs.xsl
2 - git diff
3 - Note that authority-zebra-indexdefs.xsl now has 245 Title:s info
4 - Apply patch
5 - xsltproc etc/zebradb/xsl/koha-indexdefs-to-zebra.xsl etc/zebradb/marc_defs/marc21/authorities/authority-koha-indexdefs.xml > etc/zebradb/marc_defs/marc21/authorities/authority-zebra-indexdefs.xsl
6 - git diff
7 - There are lines added about title sort, but no 245 block
8 - xsltproc etc/zebradb/xsl/koha-indexdefs-to-zebra.xsl etc/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml > etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl
9 - git diff
10 - Note lines changes to ...title_sort
11 - 245 block does not change
Signed-off-by: Hayley Pelham <hayleypelham@catalyst.net.nz>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
This adds mrl as appreviation of Multipart-resource-level to
the Zebra configuration and Search.pm so it can be used when
searching.
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Sponsored-by: Bibliotheksservice-Zentrum Baden-Wuerttemberg
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
As the field is already dfined, we don't need to add anything here.
Bib1.att can use the existing number as well
To test, enable zebra debugging in koha-conf, adding 'request' to the list:
<zebra_loglevels>none,fatal,warn,request</zebra_loglevels>
Restart all the things
Repeat matching (redo matching with no rule, then with OCN rule)
Tail the zebra-output.log and note 1=Ohter-control-number is searched and match is found
Perform a search in the staff client for: other-control-number:expialodocious
Note in logs that 1=1211 is searched
Previous test plan did not mention copying ccl.properties and bib1.att to the package install,
so highlighted that things work without these changes
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
To test:
1 - Apply patch
2 - Copy zebra files to destination:
cp /kohadevbox/koha/etc/zebradb/marc_defs/marc21/authorities/authority-koha-indexdefs.xml /etc/koha/zebradb/marc_defs/marc21/authorities/authority-koha-indexdefs.xml
cp /kohadevbox/koha/etc/zebradb/marc_defs/marc21/authorities/authority-zebra-indexdefs.xsl /etc/koha/zebradb/marc_defs/marc21/authorities/authority-zebra-indexdefs.xsl
3 - Reindex authorities
4 - Edit an authority and add 035$aExpialodocious
5 - Export the authority
6 - Create a new record matchign rule:
Matching rule code: OCN
Description: Other control number
Match threshhold: 1000
Record type: Authority record
Search-index: Other-control-number
Score: 1000
Tag: 035
Subfields: a
7 - Stage the record and use the new matchign rule
8 - Match found!
Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch adds the cni/Control-number-identifier index to enable
searches to use the 003 field.
Test plan
1/ Apply patch
2/ Re-index using updated configurations
3/ Confirm cni:number searches yield the expected results
4/ Signoff
Split-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Pasi Kallinen <pasi.kallinen@koha-suomi.fi>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch adds a 'koha' set to the pqf properties and maps some basic
koha fields to all for searchign
To test:
1 - Apply patch
2 - cp etc/z3950/pqf.properties /etc/koha/sites/kohadev/z3950/pqf.properties
3 - sudo koha-z3950-responder --restart kohadev
4 - Test a search:
curl -XGET "http://localhost:2100/biblios?version=1.1&operation=searchRetrieve&query=koha.itemtype=BK&maximumRecords=60&recordSchema=marcxml"
5 - Test other fields added:
koha.withdrawn
koha.lost
koha.classification-source
koha.materials-specified
koha.damaged
koha.restricted
koha.cn-sort
koha.notforloan
koha.ccode
koha.itemnumber
koha.homebranch
koha.holdingbranch
koha.location
koha.barcode
koha.onloan
koha.itemtype
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] Copied the changes from z3950 to zebradb too.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Test plan:
Diff etc/zebradb/pqf.properties with the etc/z3950 one.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
When using Zebra for searching, Koha performs a number of searches in order
to improve relevancy. This means that even for 'wordlist' search, we perform a phrase search.
When selecting 'Corporate-name' as an index, this expansion of the search causes errors and fails
the search
We can fix this for 'Corporate-name' searches by adding a phrase index
To test:
1 - Edit koha-conf.xml and uncomment the zebra debug line and add 'request' to the list
2 - Restart all
3 - tail -f /var/log/koha/kohadev/zebra-output.log
4 - Edit a record to add a 110 field e.g. 'House plants'
5 - Enable syspref IntranetCatalogSearchPulldown
6 - Search for 'Corporate name' and term 'House plants'
7 - No results
8 - View the log, see 'ERROR' and full search terms listed
9 - Apply patch
10 - copy the zebra files to the production instance:
cp etc/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml /etc/koha/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml
cp etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl /etc/koha/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl
11 - restart all
12 - rebuild: sudo koha-rebuild-zebra -v -f kohadev
13 - Repeat search
14 - Success!
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
For a good management of autorities linking to biblio records,
MARC21 uses index_heading and index_match_heading in authorities zebra configuration.
UNIMARC configuration must use the same.
This patch adds in UNIMARC authorities zebra configuration index_heading and index_match_heading to earch heading
in order to be maximum close to MARC21 authorities zebra configuration.
See changes made in MARC21 :
32cf2af700
It fixes some indexes names : Personal-name-see => Personal-name-see-from
Removes useless Term-geographic index, a duplicate of Name-geographic.
Sometimes parallel 7xx form whas only on $a, it must contains same subfields
has the main heading.
Test plan :
===========
1.0) Use a UNIMARC install without patch
1.1) Set sysprefs
BiblioAddsAuthorities = ON
AutoCreateAuthorities = ON
LinkerModule = First Match
1.2) Replace authorities zebra configuration files
cp $KOHA_CLONE/etc/zebradb/marc_defs/unimarc/authorities/authority-koha-indexdefs.xml $KOHA_CONF_DIR/zebradb/marc_defs/unimarc/authorities/authority-koha-indexdefs.xml
cp $KOHA_CLONE/etc/zebradb/marc_defs/unimarc/authorities/authority-zebra-indexdefs.xsl $KOHA_CONF_DIR/zebradb/marc_defs/unimarc/authorities/authority-zebra-indexdefs.xsl
1.3) Restart zebra server and indexer services
1.4) Reindex authorities
./misc/migration_tools/rebuild_zebra.pl -r -a -v
1.5) Search in Z3950 a record with complex heading (with subdivisions),
for example ISBN 2877620115 "Facteurs culturels et sociaux de la santé en Afrique de l'Oues"
1.6) Import this record and save it : authorities are created
go to staff:/cgi-bin/koha/cataloguing/addbooks.pl
1.7) Reimport the same record (when asked, say that it's not a duplicate)
1.8) The authority should have been duplicated :
different url and different $9 value
2.0) Apply this patch
2.1) Replace again the authorities zebra configuration files
2.2) Restart zebra server and indexer services
2.3) Reindex authorities
2.4) Reimport the same record
2.5) The authority should have not been duplicated. Compare with both
existing records to see which the 3rd has been matched against.
3.0) Play with authorities search to check every mode :
Search main heading ($a only)
Search main heading
Search all headings
Search entire record
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Like for MARC21, UNIMARC authorities has subdivisions form, general,
chronological and geographic.
In C4::Heading::UNIMARC, use subdivisions in _get_search_heading like in C4::Heading::MARC21.
Adds subdivisions variables into UNIMARC authorities zebra configuration.
Note that unlike MARC21 geographic is subfield $y and chronological is subfield $z.
See https://www.ifla.org/publications/unimarc-formats-and-related-documentation
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Add mappings for ︡a to a, and for t︠ to t.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch removes tokenization for phrase register/index when
using ICU.
This should cause "exact" searches to work the same way with ICU
as they do with CHR.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
According to the list of two-letter language codes found in ISO 639-1
(https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes), the correct
Zebra language option for Greek should be 'el', not 'gr'.
This patch fixes that.
Test plan:
0) Apply the patch.
1) Confirm that 'gr' has been changed to 'el' in all relevant files.
2) Create a new Koha instance using `koha-create --zebralang el testinst1`.
3) Edit /etc/koha/koha-sites.conf and set ZEBRA_LANGUAGE to 'el'. Then,
create another Koha instance with `koha-create testinst2`.
4) Confirm that both instances created in steps 2) and 3) have picked up
the Zebra CHR file from the /etc/koha/zebradb/lang_defs/el/ directory.
This is done by inspecting the value of 'profilePath' in these files:
/etc/koha/sites/testinst{1,2}/zebra-authorities-dom.cfg
/etc/koha/sites/testinst{1,2}/zebra-biblios-dom.cfg
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Removes last remaining bit of configuration for the Titles facet
configuration.
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Some Sami languages need few more characters to be mapped, or
are otherwise very hard to search for.
Test plan:
1) Catalogue a new record with title "Ǩoǯeŋa"
2) Make sure zebra indexed that record, then try to search for
it with the text "kozena"
3) Apply patch
4) Redo 2, now the record should be found.
Signed-off-by: Pasi Kallinen <pasi.kallinen@koha-suomi.fi>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
To test:
1 - Add a record with a unique publisher "Supercalifragilistic" in the
264 b field
2 - Search for the value
3 - Record not found
4 - Apply patch (may need ot copy the .xml file into koha install)
5 - Reindex all the things
6 - Search for the value
7 - Success!
Signed-off-by: Felicia Martin <felicia.martin@dncr.nh.gov>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
To test:
1 - Define a matching rule for authorities on field 001 index Local-Number
2 - In koha-conf.xml raise the zebra_loglevels
<zebra_loglevels>none,fatal,warn,request,info</zebra_loglevels>
3 - Export some authorities using the tools->export data
4 - Import those authorities
5 - Note no matches found
6 - View the zebra output log, you should see lots of error 114
7 - Apply patch
8 - Copy the indexdefs files to the installed versions
9 - Reapply matchign rules to staged files
10 - Matches should now be found
11 - Logs should not have errors
Signed-off-by: Claire Gravely <claire.gravely@bsz-bw.de>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Test plan:
0) Do not apply the patch
1) Add some word with character "ů" into metadata, for example author "Martinů, Bohuslav"
2) Try to search it with "Martinu" and you'll see you can't find it
3) Apply the patch
4) Copy file etc/zebradb/etc/word-phrase-utf.chr to your /etc/koha directory
sudo cp etc/zebradb/etc/word-phrase-utf.chr /etc/koha/zebradb/etc/
5) koha-zebra --restart kohadev
6) koha-rebuild-zebra -f kohadev
6) try to search "Martinu" again - you should be able to find your record
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
To test:
1 - Find or add a record by author Slavoj Žižek
2 - Search for 'Zizek'
3 - No results
4 - Apply patch
5 - Reindex
6 - Search again
7 - Success!
https://bugs.koha-community.org/show_bug.cgi?id=22073
Signed-off-by: Eric Phetteplace <ephetteplace@cca.edu>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
To test:
1 - Apply first patch
2 - Attempt searching by arp, no results
3 - Apply this patch
4 - Copy bib1.att and ccl.properties to the correct locations
5 - Restart zebra
6 - Rebuild indexes
7 - Search agian, success!
Signed-off-by: Margie Sheppard - Central Kansas Library System CKLS <msheppard@ckls.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
This patch adds the index definitions for zebra faceting of ccode in
koha for marc21, normarc and unimarc.
We also add lines to the templates to expose the new facet and enable
non-zebra faceting for ccode too.
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
This patch adds a numeric index 'not-onloan-count' containing the value
of 999$x. This subfield is filled by 'rebuild_zebra.pl' by making use of
(bug's 18208) 'EmbedItemsAvailability' filter.
bib1.att and indexes definitions are updated accordingly.
To test:
- Apply the patch
- Pick the right biblio-zebra-indexdefs.xsl file for your setup and
replace the one your Zebra uses [1]
- Replace your bib1.att
- Replace your ccl.properties
- Have at least one record with more than one item, checkout some
item(s) from that record(s).
- Rebuild zebra's indexes:
$ sudo koha-shell kohadev
k$ cd kohaclone
k$ misc/migration_tools/rebuild_zebra.pl -r -b -v -k
(notice the dump directory is kept, you can try the XSLT yourself
running:
$ xsltproc \
etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl \
/tmp/the_dump_dir/biblios/exported_records | less
=> SUCCESS: There are records with the not-onloan-count index, and the
value is correct!
- Check Zebra yourself:
$ yaz-client unix:/var/run/koha/kohadev/bibliosocket
Z> base biblios
Z> find @attr 1=9013 @attr 2=5 @attr 4=109 0
=> SUCCESS: The search matches the amount of records with not-onloan
items.
Z> s 1+1
=> SUCCESS: Records with 999$x having a value higher than 0 are rendered
- Sign off :-D
Note: While this work is complete on its purpose, it is part of an
attempt to create a better way of filtering by availability.
Sponsored-by: ByWater Solutions
[1] In kohadevbox this would be
/etc/koha/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl
Edit: Added the missing XSLT changes for UNIMARC and NORMARC
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
TEST PLAN
1. Make sure you have no items checked out.
2. Run sudo koha-rebuild-zebra -f -v kohadev.
3. Go to search the catalog and search.
4. Check items availability and then click on limit to currently
available items.
5. This should return no results.
6. Apply patch and reload.
7. Results should show.
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Attribute 14: " Specifies whether un-indexed fields should be ignored. A
zero value (default) throws a diagnostic when an un-indexed field is
specified. A non-zero value makes it return 0 hits."
From http://www.indexdata.com/zebra/doc/querymodel-zebra.html
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Looking at the default framework's fields that are linked to authority
records, there's a divergence with the Zebra index definitions.
This yields to authority usage count be incorrect for users searching
for authority records.
MariaDB [koha_kohadev]> SELECT tagfield,tagsubfield,authtypecode FROM
marc_subfield_structure WHERE authtypecode IS NOT NULL AND
authtypecode<>'' AND frameworkcode='' GROUP BY
tagfield,tagsubfield,authtypecode ;
+----------+-------------+--------------+
| tagfield | tagsubfield | authtypecode |
+----------+-------------+--------------+
| 100 | a | PERSO_NAME |
| 110 | a | CORPO_NAME |
| 111 | a | MEETI_NAME |
| 130 | a | UNIF_TITLE |
| 440 | a | UNIF_TITLE |
| 600 | a | PERSO_NAME |
| 610 | a | CORPO_NAME |
| 611 | a | MEETI_NAME |
| 630 | a | UNIF_TITLE |
| 648 | a | CHRON_TERM |
| 650 | a | TOPIC_TERM |
| 651 | a | GEOGR_NAME |
| 654 | a | TOPIC_TERM |
| 655 | a | GENRE/FORM |
| 656 | a | TOPIC_TERM |
| 657 | a | TOPIC_TERM |
| 658 | a | TOPIC_TERM |
| 662 | a | GEOGR_NAME |
| 690 | a | TOPIC_TERM |
| 691 | a | GEOGR_NAME |
| 696 | a | PERSO_NAME |
| 697 | a | CORPO_NAME |
| 698 | a | MEETI_NAME |
| 699 | a | UNIF_TITLE |
| 700 | a | PERSO_NAME |
| 710 | a | CORPO_NAME |
| 711 | a | MEETI_NAME |
| 730 | a | UNIF_TITLE |
| 796 | a | PERSO_NAME |
| 797 | a | CORPO_NAME |
| 798 | a | MEETI_NAME |
| 799 | a | UNIF_TITLE |
| 800 | a | PERSO_NAME |
| 810 | a | CORPO_NAME |
| 811 | a | MEETI_NAME |
| 830 | a | UNIF_TITLE |
| 896 | a | PERSO_NAME |
| 897 | a | CORPO_NAME |
| 898 | a | MEETI_NAME |
| 899 | a | UNIF_TITLE |
+----------+-------------+--------------+
This patch adds the missing ones to the authority number index as it is
done for the rest of the fields.
To test:
- Verify that
etc/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml
contains intries pointing the $9 subfield of all the fields in the
'tagfield' column above, to the Koha-Auth-Number:w index.
- Sign off :-D
Signed-off-by: Hugo Agud <hagud@orex.es>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This patch restores access to zebra facets (or zebra::snippet) with YAZ 5.8.1 or higher.
It was failing due to The <retrieval syntax="xml" name="zebra::*" /> entry in
retrieval-info-bib-dom.xml which IndexData said it wasn't even needed to
get that access.
Edit: I amended the commit message (tcohen)
Signed-off-by: Colin Campbell <colin.campbell@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
I tested on kohadevbox and found no regression or behaviour change. I
will provide a followup for the packages.
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This patch makes Zebra index the 648$9 link for chronological terms on
bibliographic records. This way an authority search on chronological terms
will show the right number in 'Used in X records' message.
To test:
- Have a record with a 648 field, linked to an authority record (i.e. with an authid on 648$9).
- Search for the record, notice it is indexed.
- Perform an authority search for the chronological term
=> FAIL: the term is linked to our record, but koha shows '0' count.
- Apply the patch
- Run:
$ cd kohaclone
$ xsltproc etc/zebra/xsl/koha-indexdefs-to-zebra.xsl \
etc/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml \
> etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl
$ git diff
=> SUCCESS: Notice the shipped etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl
is up-to-date
- Run:
$ sudo cp etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl \
/etc/koha/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl
$ sudo koha-restart-zebra kohadev
$ sudo koha-rebuild-zebra -f -b -v kohadev
- Search for the record, notice it is indexed.
- Perform an authority search for the chronological term
=> SUCCESS: the term is linked to our record, usage count is 1
- Sign off :-D
I assume NORMARC is similar on this regard. Feel free to fail it if the NORMARC part of the
patch is wrong.
Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: Hugo Agud <hagud@orex.es>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Test plan:
1) Apply patch
2) Create new instance with parameter --zebralang cs
3) Insert some record with basic latin characters and some with "czech" characters (for example: "č" - should be sorted after "c", "š" - should be sorted after "s")
4) Try to search in katalog (staff and opac) and sort by other field then relevance - title or author for instance
5) Records should be sorted correctly by Czech rules
6) Look at code and confirm it is ok
Signed-off-by: radiuscz <radek.siman@centrum.cz>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
I did not test this patch, but trust in the author and signoffer
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
to test bug...
1/ make a random user
2/ change to random user
3/ access any zebra database with random user and no authentication
4/ read zebra database
here is a transcript of the bug...
---------------------------
root@xen1:~# adduser bob
root@xen1:~# su -l bob
bob@xen1:~$ cd /var/lib/koha
bob@xen1:/var/lib/koha$ ls
topsecret
bob@xen1:/var/lib/koha$ yaz-client unix:/var/run/koha/topsecret/bibliosocket
Connecting...OK.
Sent initrequest.
Connection accepted by v3 target.
ID : 81
Name : Zebra Information Server/GFS/YAZ
Version: 4.2.30 98864b44c654645bc16b2c54f822dc2e45a93031
Options: search present delSet triggerResourceCtrl scan sort extendedServices namedResultSets
Elapsed: 0.001002
Z> base biblios;
Z> find the
Sent searchRequest.
Received SearchResponse.
Search was a success.
Number of hits: 1130, setno 2
SearchResult-1: term=the cnt=1130
records returned: 0
Elapsed: 0.005518
Z> show
Sent presentRequest (1+1).
Records: 1
[biblios]Record type: USmarc
01824cam a2200397 a 4500
001 000045782309
003 AuCNLKIN
005 20111013213222.0
008 100707s2011 maua 001 0 e
...
---------------------------
5/ apply changes to a Koha instance's config files, that you plan to test
6/ restart zebra for instance
# sudo koha-restart-zebra topsecret
7/ repeat steps 2 and 3, but receive a 'bad user/passwd ' error from zebra
bob@xen1:~$ yaz-client unix:/var/run/koha/topsecret/bibliosocket
Connecting...OK.
Sent initrequest.
Connection rejected by v3 target.
1: code=1011 (Init/AC: Bad Userid and/or Password),
NOTE: this patch currently will only fixes newly created instances, it wont fix existing instances
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Good catch Mason
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Removes commented line from bib1.att.
Adjust OCLC-number to Other-control-number in comment of ccl properties.
No need to explicitly add 035$a and $z if you index 035 completely in
record.abs as well as biblio-koha-indexdefs.xml.
Rerun koha-indexdefs-to-zebra.xsl on index defs.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
1) Apply patch
2) Make sure that you have a bib that has MARC21 035$a (and possibly also 035$z) populated.
pre 3) Replace all modified zebra files and restart zebra server
3) Rebuild zebra: misc/migration_tools/rebuild_zebra.pl -x -b -z
4) Add the following to the intranetuserjs syspref:
$(document).ready(function(){
// Add Other Control Number to advanced search
if (window.location.href.indexOf("catalogue/search.pl") > -1) {
$(".advsearch").append('<option value="Other-control-number">Other Control Number</option>');
}
});
5) Do an advanced search, select "Other Control Number" from the search menu, then add the Other Control Number in 035$a for the bib specified in step 1.
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Works, no koha-qa errors
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This patch indexes 024$a into the "phrase" index type, and the "url" index type,
if the 024$2 equals "uri".
TEST PLAN
1) Apply the patch.
1b) If you're using a gitified Koha or a git install,
you'll need to upgrade your instance or copy your zebradb files
over to /etc/koha/zebradb or your "kohadev" directory.
2) Add a 024$a with a URL like http://libris.kb.se/resource/bib/219553
to a bibliographic record
3) Re-index Zebra
4) Type "id-other,st-urx,fuzzy=http://libris.kb.se/resource/bib/219553"
into the "Search the catalog" box in the Staff Client and search
5) Note that you retrieve your record
NOTE: The fuzzy is required because Koha's query "parsing" functions change
http:// to http=// which won't correctly match the value in the "Identifier-other:u" index.
NOTE: Alternatively, you could do the following search instead:
"id-other,phr=http libris kb se resource bib 219553".
It would work as well by using the "Identifier-other:p" index.
Advanced tester version:
4) In a terminal window, find the "koha-conf.xml" file in your "etc" directory.
5) Open "koha-conf.xml" and find <listen id="biblioserver">.
Copy the URI you find there. (e.g. unix:/home/dcook/koha-dev/var/run/zebradb/bibliosocket).
6) Type "yaz-client unix:/home/dcook/koha-dev/var/run/zebradb/bibliosocket"
7) After it connects, type "base biblios" and press enter
8) Type "format xml" and press enter
9) Type "elements zebra::index" and press enter
10) Type "f id-other,st-urx=http://libris.kb.se/resource/bib/219553" and press enter
11) Note that you should have at least one result
12) Type "show 1"
13) If you scroll through the results, you should find something like the following:
<index name="Identifier-other" type="w" seq="28">@^</index>
<index name="Identifier-other" type="w" seq="1"></index>
<index name="Identifier-other" type="w" seq="29">http</index>
<index name="Identifier-other" type="w" seq="30">libris</index>
<index name="Identifier-other" type="w" seq="31">kb</index>
<index name="Identifier-other" type="w" seq="32">se</index>
<index name="Identifier-other" type="w" seq="33">resource</index>
<index name="Identifier-other" type="w" seq="34">bib</index>
<index name="Identifier-other" type="w" seq="35">219553</index>
<index name="Identifier-other" type="p" seq="28">http libris kb se resource bib 219553</index>
<index name="Identifier-other" type="u" seq="36">http://libris.kb.se/resource/bib/219553</index>
Signed-off-by: Hector Castro <hector.hecaxmmx@gmail.com>
Works as advertised the record is retrieved
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Some of the statements in the commit message do not work for me.
A search like "id-other,phr=http libris kb se resource bib 219553" does not
have results. Searching for "id-other,phr=libris.kb.se resource" does.
The steps in the advanced tester version do not work for me too.
I verified the following in yaz-client:
[1] Z> f @attr 1=9012 @attr 4=104 http://libris.kb.se/resource/bib/219553
Sent searchRequest.
Received SearchResponse.
Search was a success.
Number of hits: 1, setno 16
[2] First removed $2 and reindexed. Then searched again:
Z> f @attr 1=9012 @attr 4=104 http://libris.kb.se/resource/bib/219553
Sent searchRequest.
Received SearchResponse.
Search was a success.
Number of hits: 0, setno 1
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
changed ocurrences of 'lex' to 'lexile-number' in record.abs
Edits were made to the deprecated file record.abs *solely* to quiet
warnings in tests -- this makes sense until GRS-1 code is removed
from Koha.
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Added the following indexes:
Interest-age-level | 591$a ind1=1
Interest-grade-level | 591$a ind1=2
lexile-number | 591$a ind1=8
Reading-grade-level | 591$a ind1=0
Moved 'lex' from a zebra index to a ccl alias to lexile-number.
Changed the handling of st-numeric in C4/Search.pm to allow for search ranges.
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Hector Castro <hector.hecaxmmx@gmail.com>
Works as advertised
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
This patch will add indexes for Date/time-last-modified.
To test:
1. apply patch
2. reindex
3. search for dtlm:DATE and date-time-last-modified:DATE
4. confirm that you get results
Signed-off-by: Hector Castro <hector.hecaxmmx@gmail.com>
Works as advertised
Signed-off-by: Frédéric Demians <f.demians@tamil.fr>
I confirm Hector signing-off. A simple Zebra server restart suffice to get
working the searches on date-time-last-modified and dtlm.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
Only in MARC21 is possible to use ind2 of tag 245 to skip articles.
This patch is based on inserting a special template in
koha-indexdefs-to-zebra.xsl With this patch you must not insert index
Title:s in biblio-koha-indexdefs.xml, it is defined in
koha-indexdefs-to-zebra.xsl. It is not the best setup, but I find very
difficult to use biblio-koha-indexdefs.xml.
To test it in a english MARC21 setup:
Insert same records with titles and correct values in ind2 of 245.
If you have articles not in the skiping list of sort-string-utf.chr (The|the|a|A|an|An)
you can see that the sort by articles use also articles.
Insert the patch
Rebuilt indexes from scratch
Now all articles of titles are skipped
TO TEST WITHOUT INDEXING:
1. Go to etc/zebradb/marc_defs/marc21/biblios directory.
2. Put the sample MARCXML file in this directory.
3. Transform the file into Zebra indexes:
xsltproc biblio-zebra-indexdefs.xsl record.xml
Observe that the Title:s index contains:
01 Business and Technologies
4. Apply the patch.
5. Repeat:
xsltproc biblio-zebra-indexdefs.xsl record.xml
Observe that the Title:s index contains:
Business and Technologies
Signed-off-by: Frederic Demians <f.demians@tamil.fr>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Verified working using yaz-client (as in
http://wiki.koha-community.org/wiki/Understanding_Zebra_indexing#Examine_Zebra_index,
though note that the `elem zebra::index` seems to be unneeded).
Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
Test plan:
Without this patch, doing a zebra reindex on a fedora-based install will cause errors like this:
15:10:47-01/05 zebraidx(16108) [warn] No such record type: dom./etc/koha/zebradb/biblios/etc/dom-config.xml
With this patch, reindexing should just work.
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
I have tested this doesn't break on debian/ubuntu systems, someone with a non
debian system will need to test it on that
Signed-off-by: Bob Ewart bob-ewart@bobsown.com
It works on openSUSE Leap 42.1
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Just noting that the debian zebra files already contain much more paths here.
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This patch adds the Zebra special attribute 14 to ccl.properties and
opac-search.pl, so that we can turn on OpacSuppression and still return
results even if there are no records in Zebra for the Suppress index.
_TEST PLAN_
Before applying:
1) Make sure that you have no suppressed records indexed in Zebra
2) Turn on OpacSuppression system preference
3) Search using a keyword which should bring up records
4) Note that no records are returned in the results
5) Change UseQueryParser system preference to "Try"
6) Repeat steps 3-4
Apply the patch.
After applying:
7) Repeat step 3 (ie search using a keyword which should bring up records)
8) Confirm that records are appearing in the results!
9) Change UseQueryParser system preference to "Do not try"
10) Repeat step 3
11) Confirm that records are appearing in the results!
Signed-off-by: Hector Castro <hector.hecaxmmx@gmail.com>
Works as advertised. No more, won't need to have at least one record with the
value "1" in the field mapped with this index. All records in OPAC returned.
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Single quotes in common language (not in programming) are usually ', but
there is also the form known as ’ in HTML. See
https://fr.wikipedia.org/wiki/Apostrophe_%28typographie%29
This bug proposes to transliterate all forms into a space.
Test plan :
(I'll use the code ’ instead of the unicode character)
- Without the patch
- Create a record with title : L’avion d’argile
- Index this record
- Search for "L’avion d’argile" => You find the record
- Search for "L'avion d'argile" => You do not find the record
- Apply patch
- Search for "L’avion d’argile" => You find the record
- Search for "L'avion d'argile" => You find the record
- Search for "L avion d argile" => You find the record
Signed-off-by: Frederic Demians <f.demians@tamil.fr>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch add zebra indexes to RDA 264 field.
The new Provider index is added too.
QA comments corrected.
To test:
1) Download RDA records with 264 fields from this attachment <http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=36825>. Import the file and re-index/rebuild zebra. These records contain 260 and 264 fields per record.
2) Do a search with pb:Bethany two records will appear with title The guardian. Search with pl:Minneapolis too, the two records will appear.
3) Select one record of both records and delete the 260 field keeping the 264 field and save, rebuild your zebra.
4) Search again with pb:Bethany and just one record will appear. Thats mean 264 is not indexed.
5) Apply patches.
6) Rebuild your zebra but this time all biblio records.
7) Search again with pv:Bethany or Provider:Bethany, this time will appear the two records, 264 is indexed. Note that if you search again with pb only one record appear. This is because the suggestion of LOC.
10) Search with copydate:2013 only launch records with 260 fields and pv:2013 show both fields, i.e., 260 and 264.
11) Apply QA Test Tools
Sponsored-by: Universidad de El Salvador
Signed-off-by: Nick Clemens <nick@quecheelibrary.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Currently, Norwegian vowels are not sorted correctly, even when
you have chosen "nb" as the ZEBRA_LANGUAGE during installation.
To test:
- Make sure you have three records with titles that begin with ÆØÅ
respectively
- Do a search that turns up those three records and some others, and
sort the results by title, bot ascending and descending.
- Verify that ÆØÅ is shown in some weird order.
- Edit your sort-string-utf.chr* so it is in line with the current
patch. It should include these two lines:
lowercase {0-9}{a-z}æøå
uppercase {0-9}{A-Z}ÆØÅ
- Restart Zebra and reindex all the records, e.g.:
$ sudo koha-restart-zebra <instancename>
$ sudo koha-rebuild-zebra -f -v <instancename>
- Do the search again, make sure you order by title and check that
ÆØÅ are sorted in the order of 1. Æ 2. Ø 3. Å, and after all other
characters
* = If you are on a gitified install, you need to edit this file:
/etc/koha/zebradb/lang_defs/<your ZEBRA_LANGUAGE>/sort-string-utf.chr
NOT the file in your git clone (yeah, i wasted some time there...)
Signed-off-by: Mirko Tietgen <mirko@abunchofthings.net>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
