Commit graph

19 commits

Author SHA1 Message Date
5b4259be9c Bug 6499: [QA Follow-up] Trivial adjustments
Removes commented line from bib1.att.
Adjust OCLC-number to Other-control-number in comment of ccl properties.
No need to explicitly add 035$a and $z if you index 035 completely in
record.abs as well as biblio-koha-indexdefs.xml.
Rerun koha-indexdefs-to-zebra.xsl on index defs.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-08-09 10:13:11 +00:00
Barton Chittenden
84f51549c9 Bug 6499: Add Zebra index "Other-control-number" covering MARC21 035$a, 035$z and 035 (entire tag)
1) Apply patch
2) Make sure that you have a bib that has MARC21 035$a (and possibly also 035$z) populated.

pre 3) Replace all modified zebra files and restart zebra server

3) Rebuild zebra: misc/migration_tools/rebuild_zebra.pl -x -b -z
4) Add the following to the intranetuserjs syspref:

$(document).ready(function(){
    // Add Other Control Number to advanced search
    if (window.location.href.indexOf("catalogue/search.pl") > -1) {
        $(".advsearch").append('<option value="Other-control-number">Other Control Number</option>');
    }
});

5) Do an advanced search, select "Other Control Number" from the search menu, then add the Other Control Number in 035$a for the bib specified in step 1.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Works, no koha-qa errors

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-08-09 10:13:10 +00:00
ff6523765d Bug 15555: Index 024$a into Identifier-other:u url register when source $2 is uri
This patch indexes 024$a into the "phrase" index type, and the "url" index type,
if the 024$2 equals "uri".

TEST PLAN

1) Apply the patch.
1b) If you're using a gitified Koha or a git install,
you'll need to upgrade your instance or copy your zebradb files
over to /etc/koha/zebradb or your "kohadev" directory.
2) Add a 024$a with a URL like http://libris.kb.se/resource/bib/219553
 to a bibliographic record
3) Re-index Zebra
4) Type "id-other,st-urx,fuzzy=http://libris.kb.se/resource/bib/219553"
 into the "Search the catalog" box in the Staff Client and search
5) Note that you retrieve your record

NOTE: The fuzzy is required because Koha's query "parsing" functions change
http:// to http=// which won't correctly match the value in the "Identifier-other:u" index.
NOTE: Alternatively, you could do the following search instead:
"id-other,phr=http libris kb se resource bib 219553".
 It would work as well by using the "Identifier-other:p" index.

Advanced tester version:
4) In a terminal window, find the "koha-conf.xml" file in your "etc" directory.
5) Open "koha-conf.xml" and find <listen id="biblioserver">.
Copy the URI you find there. (e.g. unix:/home/dcook/koha-dev/var/run/zebradb/bibliosocket).
6) Type "yaz-client unix:/home/dcook/koha-dev/var/run/zebradb/bibliosocket"
7) After it connects, type "base biblios" and press enter
8) Type "format xml" and press enter
9) Type "elements zebra::index" and press enter
10) Type "f id-other,st-urx=http://libris.kb.se/resource/bib/219553" and press enter
11) Note that you should have at least one result
12) Type "show 1"
13) If you scroll through the results, you should find something like the following:

<index name="Identifier-other" type="w" seq="28">@^</index>
<index name="Identifier-other" type="w" seq="1"></index>
<index name="Identifier-other" type="w" seq="29">http</index>
<index name="Identifier-other" type="w" seq="30">libris</index>
<index name="Identifier-other" type="w" seq="31">kb</index>
<index name="Identifier-other" type="w" seq="32">se</index>
<index name="Identifier-other" type="w" seq="33">resource</index>
<index name="Identifier-other" type="w" seq="34">bib</index>
<index name="Identifier-other" type="w" seq="35">219553</index>
<index name="Identifier-other" type="p" seq="28">http libris kb se resource bib 219553</index>
<index name="Identifier-other" type="u" seq="36">http://libris.kb.se/resource/bib/219553</index>

Signed-off-by: Hector Castro <hector.hecaxmmx@gmail.com>
Works as advertised the record is retrieved

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Some of the statements in the commit message do not work for me.
A search like "id-other,phr=http libris kb se resource bib 219553" does not
have results. Searching for "id-other,phr=libris.kb.se resource" does.
The steps in the advanced tester version do not work for me too.
I verified the following in yaz-client:
[1] Z> f @attr 1=9012 @attr 4=104 http://libris.kb.se/resource/bib/219553
Sent searchRequest.
Received SearchResponse.
Search was a success.
Number of hits: 1, setno 16
[2] First removed $2 and reindexed. Then searched again:
Z> f @attr 1=9012 @attr 4=104 http://libris.kb.se/resource/bib/219553
Sent searchRequest.
Received SearchResponse.
Search was a success.
Number of hits: 0, setno 1

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-04-29 13:19:28 +00:00
Barton Chittenden
690ab60da2 Bug 14277: add zebra indexes for lexile that respect 521 indicator 1.
Added the following indexes:

Interest-age-level | 591$a ind1=1
Interest-grade-level | 591$a ind1=2
lexile-number | 591$a ind1=8
Reading-grade-level | 591$a ind1=0

Moved 'lex' from a zebra index to a ccl alias to lexile-number.

Changed the handling of st-numeric in C4/Search.pm to allow for search ranges.

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Hector Castro <hector.hecaxmmx@gmail.com>
Works as advertised
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>

Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
2016-04-07 10:35:18 -06:00
Zeno Tajoli
cc878aee77 Bug 14332: Skip articles in MARC21 using indicator n.2 of field 245
Only in MARC21 is possible to use ind2 of tag 245 to skip articles.
This patch is based on inserting a special template in
koha-indexdefs-to-zebra.xsl With this patch you must not insert index
Title:s in biblio-koha-indexdefs.xml, it is defined in
koha-indexdefs-to-zebra.xsl.  It is not the best setup, but I find very
difficult  to use  biblio-koha-indexdefs.xml.

To test it in a english MARC21 setup:

Insert same records with titles and correct values in ind2 of 245.
If you have articles not in the skiping list of sort-string-utf.chr (The|the|a|A|an|An)
you can see that the sort by articles use also articles.

Insert the patch
Rebuilt indexes from scratch

Now all articles of titles are skipped

TO TEST WITHOUT INDEXING:

1. Go to etc/zebradb/marc_defs/marc21/biblios directory.

2. Put the sample MARCXML file in this directory.

3. Transform the file into Zebra indexes:
   xsltproc biblio-zebra-indexdefs.xsl record.xml
   Observe that the Title:s index contains:
   01 Business and Technologies

4. Apply the patch.

5. Repeat:
   xsltproc biblio-zebra-indexdefs.xsl record.xml
   Observe that the Title:s index contains:
   Business and Technologies

Signed-off-by: Frederic Demians <f.demians@tamil.fr>

Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Verified working using yaz-client (as in
http://wiki.koha-community.org/wiki/Understanding_Zebra_indexing#Examine_Zebra_index,
though note that the `elem zebra::index` seems to be unneeded).

Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
2016-01-27 06:17:16 +00:00
Hector Eduardo Castro Avalos
de272e5b41 Bug 14198: RDA: Indexing 264 field (Zebra)
This patch add zebra indexes to RDA 264 field.
The new Provider index is added too.
QA comments corrected.

To test:
1) Download RDA records with 264 fields from this attachment <http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=36825>. Import the file and re-index/rebuild zebra. These records contain 260 and 264 fields per record.
2) Do a search with pb:Bethany two records will appear with title The guardian. Search with pl:Minneapolis too, the two records will appear.
3) Select one record of both records and delete the 260 field keeping the 264 field and save, rebuild your zebra.
4) Search again with pb:Bethany and just one record will appear. Thats mean 264 is not indexed.
5) Apply patches.
6) Rebuild your zebra but this time all biblio records.
7) Search again with pv:Bethany or Provider:Bethany, this time will appear the two records, 264 is indexed. Note that if you search again with pb only one record appear. This is because the suggestion of LOC.
10) Search with copydate:2013 only launch records with 260 fields and pv:2013 show both fields, i.e., 260 and 264.
11) Apply QA Test Tools

Sponsored-by: Universidad de El Salvador
Signed-off-by: Nick Clemens <nick@quecheelibrary.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2015-11-02 11:41:36 -03:00
Barton Chittenden
cab481dbb2 Bug 14617: Add fields to ISBN and ISSN indexes: 020$z, 022$y, 022$z
1) Import MARC21 bibs containing

- ISBN in 020$z
- ISSN in 022$y
- ISSN in 022$z

2) Make sure that bibs are indexed

3) Search by ISBN and ISSN above -- bibs should not show in search.

4) Apply patch, re-index

5) Search again; ISBN in 020$z and ISSN in 022$y and 022$z should return
results.

Signed-off-by: kholten@switchinc.org
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2015-10-19 10:12:04 -03:00
Katrin Fischer
e799e1cbc3 Bug 11620: Add dissertation-information index for MARC21 (502)
Bug 11202 introduced a new index 'dissertation-information' for
UNIMARC. This patch adds the index also for MARC21 installations.

http://www.loc.gov/marc/bibliographic/bd502.html

To test:
- Apply patch
- Copy files in etc/zebradb changed by this patch to your
  corresponding directory (koha-dev..)
- Make sure you have records with 502
- Reindex
- Verify you can search the field contents with
  dissertation-information= and
  diss=

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Can find by dissertation-information,
No errors

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>
2015-07-20 10:31:06 -03:00
Zeno Tajoli
c29a53ea20 Bug 12948: Use word indexing for language (MARC21)
This patch is for MARC21. To test:
1)Setup a site with
 MARC21
2)Insert 2 record, one lang A in 041 and 008 pos
 35-37 an other with lang A in 041 and lang B in 008 pos
 35-37
3)Index them
4)Search in advanced search with filter
 'languare' for lan A. You will see 2 records
5)Search in
 advanced search with filter 'languare' for lan B. You will
 see 0 records
6)Apply the patch
7)Full reindex
8)Search in advanced search
 with filter 'languare' for lan B. You will see 1 records

http://bugs.koha-community.org/show_bug.cgi?id=12948

Signed-off-by: Magnus Enger <magnus@enger.priv.no>
I have *not* actually tested this, but the changes are identical to the ones
done for NORMARC, which I have tested, so I think it is safe to sign off. If
anyone disagrees, please reset the bug to "Needs signoff".

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2015-02-20 11:51:59 -03:00
ccf7ae56f6 Bug 11232: (qa followup) Add missing fields/subfields to the item types faceta
The itype facet was missing 952$y for both MARC21 and NORMARC.
This patch adds that. And also modifies the zebra-biblios-dom.cfg file
(also the debian/ version) so facetNumRecs is set to 1000 for zebra.

It is the amount of records that are taken into account. The more record,
the more exact the facets for the result set. 1000 was chosen as it changed
the time to reindex 1000 records from 18s to 19s.

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2014-10-15 12:55:47 -03:00
e95cd1b126 Bug 11232: (followup) remove unnecesary namespace definition from all XML elements
The previous patches for facet extraction from Zebra indexes set a default
namespace on the following files:

etc/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml
etc/zebradb/marc_defs/normarc/biblios/biblio-koha-indexdefs.xml
etc/zebradb/marc_defs/unimarc/biblios/biblio-koha-indexdefs.xml

and hence the XML file index_subfields can be cleaned by removing the namespace.

To test:
- Apply this patch
- Run

$ for i in marc21 normarc unimarc
  do xsltproc etc/zebradb/xsl/koha-indexdefs-to-zebra.xsl \
              etc/zebradb/marc_defs/$i/biblios/biblio-koha-indexdefs.xml \
              > etc/zebradb/marc_defs/$i/biblios/biblio-zebra-indexdefs.xsl
  done

=> SUCCESS: no errors reported

- Run
$ git diff
=> SUCCESS: no differences on the xsl files

- Sign off :-D

Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: David Cook <dcook@prosentient.com.au>

Seems to work with DOM and MARC21.

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2014-10-15 12:55:44 -03:00
2cc293ecd6 Bug 11232: MARC21 facet definition and updated XSL file for DOM
This patch adds the facets definitions to the biblio-koha-indexdefs.xml, based
on what is hardcoded on C4::Koha::getFacets().

The biblio-zebra-indexdefs.xsl file for MARC21 is generated using the usual:

xsltproc ...koha-indexdefs-to-zebra.xsl ...marc21/biblios/biblio-koha-indexdefs.xml > \
    ...marc21/biblios/biblio-zebra-indexdefs.xsl

Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: David Cook <dcook@prosentient.com.au>

Seems to work with DOM and MARC21.

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2014-10-15 12:55:36 -03:00
Mirko Tietgen
84bdb55549 Bug 9972: Add/change some zebra indexes (MARC21)
This patch adds :w and :p versions to the index for »Lexile number«
(it has only :n so far) and adds indexes for 653 (Index term
uncontrolled), 655 (Index term Genre/Form), 041 (language-audio) and
041 (language-subtitle). It also adds the »curriculum«-index to
Search.pm.

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-04-20 16:24:08 +00:00
Galen Charlton
aaff735269 Bug 10544: (follow-up) update MARC21 DOM index definitions
This patch updates the MARC21 DOM index definitions to
index the 952$i as 'Number-local-acquisition' rather than
'stocknumber'.

To test (for a MARC21/DOM setup):

[1] Copy the MARC21 biblio-zebra-indexdefs.xsl over to the
    active Zebra configuration directory.
[2] Reindex the bib records.
[3] Verify that 'stocknumber', 'inv', and 'number-local-acquisition'
    searches work.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-02-19 20:41:37 +00:00
Mathieu Saby
451f67c055 Bug 9940: Add a new index for the original language of a document
It could be useful to index the original language of a document (i.e.
"fre" for the English translation of a French novel).

This patch renames the Bib-1 use attribute 1095 from
Code-language-original to language-original and uses it to index:

- MARC21 041$h subfield
- UNIMARC 101$c subfield

It adds "language-original" in the list of index in Search.pm.

Test plan :
A. in a MARC21 GRS1 environment
1. Copy Zebra config files (zebradb/biblios/etc/bib1.att,
   zebradb/ccl.properties, marc_defs/marc21/biblios/record.abs) from
   your source etc/ directory to your main koha etc/ directory
2. Reindex zebra
3. Make some searches, like "language-original:fre"
B. in a MARC21 DOM environment
4. Copy Zebra config files (zebradb/biblios/etc/bib1.att, zebradb/ccl.properties,
   marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl) from your source etc/
   directory to your main koha etc/ directory
5. Reindex zebra
6. Make some searches, like "language-original:fre"
C. in a UNIMARC GRS1 environment
7. Copy Zebra config files (zebradb/biblios/etc/bib1.att,
   zebradb/ccl.properties, marc_defs/unimarc/biblios/record.abs) from
   your source etc/ directory to your main koha etc/ directory
8. Reindex zebra
9. Make some searches, like "language-original:fre"
A. in a UNIMARC DOM environment
10. Copy Zebra config files (zebradb/biblios/etc/bib1.att,
    zebradb/ccl.properties, marc_defs/unimarc/biblios/biblio-zebra-indexdefs.xsl)
    from your source etc/ directory to your main koha etc/ directory
11. Reindex zebra
12. Make some searches, like "language-original:fre"

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-12-25 15:37:14 +00:00
Galen Charlton
8ea3462517 Bug 8252: (follow-up) standardize name of Identifier-publisher-for-music index
To test:

[1] When running t/db_dependent/Search.t, veify that no warnings like
    this are shown:

15:52:07-10/10 zebraidx(2006) [warn] Index 'Number-music-publisher' not found in attset(s)

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 16:05:33 +00:00
Jared Camins-Esakov
7d9b4d58e3 Bug 8665: DOM indexing fails to index some bib records
Use a user-specified field for z:id.

This patch also fixes an excess space before the index in the MARC21
biblio index definitions, which someone fixed in the generated file
but not in the source file it should have been fixed in.

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Elliott Davis <elliott@bywatersolutions.com>
2012-10-29 19:12:38 +01:00
Colin Campbell
1e8423167e Bug 8653 remove erroneous whitespace blocking indexing
The superfluous whitespace after the definition of subject
tag $9s is causing an error when carried over into dom config
files so that the authority links fail to index

Also removed the (harmless) trailing space in the equivalent
Unimarc files

A good editor and git can help in not creating excess whitespace

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-09-14 17:20:34 +02:00
Galen Charlton
64680c18b3 Bug 7818: Zebra DOM filter index definitions for MARC21 bibs
The file biblio-zebra-indexdefs.xsl, which is the stylesheet that
is used by the Zebra DOM filter to convert an incoming MARC21 bib
to its indexed form, was generated by the following two steps:

misc/maintenance/make_zebra_dom_cfg_from_record_abs \
  --input  etc/zebradb/marc_defs/marc21/biblios/record.abs \
  --output etc/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml

xsltproc etc/zebradb/xsl/koha-indexdefs-to-zebra.xsl \
  etc/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml \
  > etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl

Records indexed using this XSLTshould behave similarly to records
indexed using the GRS-1 filter and the old record.abs definition, with
the following big exception (and improvemwent): indexed phrases now
span subfield boundaries if a specific subfield wasn't specified in the
index definition.  For example, the GRS-1 filter index definition

melm 245 Title

would allow 245 $a Cats on boxes : $b cardboard fantasies

to be searched as the phrases "cats on boxes" or "cardboard fantasies",
but a title phrase seach of "cats on boxes cardboard fantasises"
wouldn't work.  The DOM filter equivalent,

<index_data_field xmlns="http://www.koha-community.org/schemas/index-defs" tag="245">
  <target_index>Title:w</target_index>
  <target_index>Title:p</target_index>
</index_data_field>

*does* allow phrase searches to span subfield boundaries.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-06-09 11:44:06 +02:00