This button lets you replace existing authorities using a Z39.50 search.
http://bugs.koha-community.org/show_bug.cgi?id=11961
Signed-off-by: Nicole C. Engard <nengard@bywatersolutions.com>
All tests pass
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
I would prefer not to hide this "stuff".
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
When using a z3950 connexion with UNIMARC authorities, you get an error :
Unsupported UNIMARC character encoding [ ] for XML output for UNIMARCAUTH; 100$a -> 20141119
I've seen thant Bug 2060 when adds authorities import adds a special behavior for UNIMARC : marc flavor must be UNIMARCAUTH instead of just UNIMARC.
This patch adds the same behavior when using z3950 connexion and import.
Test plan :
- Use a UNIMARC install
- Define a z3950 connexion for UNIMARC authorities
- Go to Authorities module
- Click on "New from Z39.50"
- Perform a search
=> Without patch : you get the error
=> With patch : you get results
- Import one result
=> You get the authoritie creation form with all datas
You may check same plan with MARC21 install
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
NOTE: depending on the target, the syntax in the configuration
might not be UNIMARC, but MARC21/USMARC instead!
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This fixes regression introduced by Bug 6536 so that multiple
words in title search will work.
Test scenario:
1. try z39.50 search with more than one word
2. verify that no results apper
3. apply patch and re-run search
4. verify that there are results
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
The last test on the first series, fails randomly on Perl 5.18:
not ok 12 - Third query makes no difference
# Failed test 'Third query makes no difference'
# at t/db_dependent/Breeding.t line 104.
# got: ''
# expected: '1'
# Looks like you failed 1 test of 12.
not ok 1 - _build_query
This change makes tests pass. Please consider if this needs to be fixed
(i.e. sort order matters) or the test needs to be rewritten.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
I agree with adding the sort. (The need for doing this in Perl 5.18 is another
topic.)
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Resolved:
[1] FAIL C4/Breeding.pm
FAIL critic ControlStructures::ProhibitMutatingListFunctions
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
No warnings anymore.
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This patch only removes surrouding spaces at comma and equals-sign while
passing the options in sru_fields to the ZOOM object.
Test plan:
If you add spaces between options in sru_fields, searching should still work.
E.g. sru_fields= sru = get , sru_version = 1.1
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Use the stylesheets listed in field add_xslt of z3950servers to transform
search results of Z3950/SRU search.
Additionally, the template has been changed to make more error messages (or
warnings) visible when displaying results. Until now, error message were
shown in the results table and when connection errors occurred, no results
were displayed at all.
Test plan:
Create some stylesheets (or see the sample patch on bug 6536).
Add these stylesheets to some Z3950/SRU servers.
Do Z3950 search and verify the transformations.
Do a search with 2 targets; make one target fail (by manipulating its server
data). Do you see the connection error and the results for the other target?
Generate a XSLT error by modifying one stylesheet. Check search results. You
should see warnings.
Signed-off-by: Giuseppe Angilella <giuseppe.angilella@ct.infn.it>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This patch makes it possible to include SRU servers in Z3950 search.
It adjusts the Z3950Search routine in Breeding module.
It also replaces SQL code with DBIx statements in Breeding.pm/Z3950Search
and the associated scripts z3950search.pl in cataloguing and acqui.
Test plan:
Verify if a normal Z3950 search still works in cataloging/acqui.
Add a SRU target. (You could just use Koha's port 9998.)
Define sru_options like sru=get.
Use that target in a Z3950 search in cataloging and acqui. (Import.)
Test sru_fields translation by comparing search results between various
settings for some of the fields. For instance, leave title empty and
after that set it to the title field of your SRU target.
Signed-off-by: Giuseppe Angilella <giuseppe.angilella@ct.infn.it>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Replaces name by servername, type by servertype for running Z3950 search.
Limit search scripts to zed (z3950) servers until sru is supported.
Test plan:
Perform a Z3950 search in Cataloguing and Acquisition.
Verify that it still works as it did.
Signed-off-by: Giuseppe Angilella <giuseppe.angilella@ct.infn.it>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This patch removes the ImportBreeding() routine, which lost its
last caller as of the patch for bug 10462.
To test:
[1] Verify that prove -v t/Breeding.t passes.
[2] Perform a Z39.50 search in the staff interface.
[3] Perform a cataloguing reservoir search in the staff
interface; verifying that cached records from the search
done in step 2 are retrieved.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
When importing records, the ISBN is normalized and stored
into database (see C4::ImportBatch::_add_biblio_fields). But when
searching with ISBN into reservoir, it is not normalized
(see C4::Breeding::BreedingSearch). So search does not match.
This patch adds the normalisation to reservoir search. Also, it
replaces call private method _isbn_cleanup by GetNormalizedISBN,
the correct public method. Also allows the reservoir search
on ISBN with hyphens.
This is intended to fix only reservoir searches.
Revised Test plan
-----------------
1) Back up DB
2) Save copy of attached example somewhere findable
2) Home -> Tools -> Stage MARC records for import
3) Click Browse and select the example MARC file
4) Click Upload file
5) Tweak as desired then click Stage for import
6) Click Manage staged records
7) Click Import this batch into the catalog
8) Home -> Cataloging
9) In the Cataloging search text box type 978-0-691-14289-0 and
click Submit
-- ISBN13 with hypens not found in reservoir
10) In the Cataloging search text box type 9780691142890 and
click Submit
-- ISBN13 without hypens not found in reservoir
11) In the Cataloging search text box type 0-691-14289-0 and
click Submit
-- ISBN10 with hypens not found in reservoir
12) In the Cataloging search text box type 0691142890 and
click Submit
-- ISBN10 without hypens found in reservoir
13) Apply patch
14) Repeat steps 9-12, this time it is always found! :)
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
When a z39.50 server isn't able to be searched successfully, the yellow
error box came up empty. This patch fixes the problem.
Test Plan:
1) Go to Administration/z39.50 servers
2) Create a fake z39.50 server with a made up address
3) Go to cataloging, search only that server
4) Note the empty yellow alert box
5) Apply this patch
6) Re-run the search, not the alert box has a message in it now
Signed-off-by: Nora Blake <nblake@masslibsystem.org>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.
Works according to test plan.
When one of the selected servers gives result no dialog
box is shown before and after applying the patch.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
- improve POD
- remove extraneous comments
- correct license statement in new files
- remove backticks in database update SQL
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch introduces a new Z39.50 interface for searching Z39.50
compliant databases for MARC authority records.
These databases aren't as common as their bibliographic equivalents,
but they're out there and very useful. I have included info at the
bottom of this messsage for sample authority databases you can try.
To test this patch:
1) Set up Z39.50 client targets for authority databases. (I've included
information at the bottom of this message for LibrariesAustralia's
test server for authorities as well as instructions on how to use
your Koha's z39.50 authority server as well. The Library of Congress
also has authority databases available (unsure if these are test or
prod), and you might have access to others through OCLC or RLIN. OCLC
provides login credentials for their test databases.
2) Go to the Authorities module
3) Click on the new "Z39.50 search button"
4) Select your authority search targets from the list.
5) Do a search for an authority you would like using either the "Raw"
input box or the more specific input boxes for names, subjects, subject
sub divisions, or titles. (I like searching Name (personal): Eric on
the LibrariesAustralia test DB.)
6) You should see a table listing the server, heading, authority type,
and two other columns (MARC and a nameless column). "Authority type"
is the type of authority it will become when imported in to Koha. In
the Eric example, "PERSO_NAME".
7) Click on "MARC" next to the results of interest to review the MARC
authority record.
8) When you're satisfied with a record, click on "Import".
9) The pop-up window will close and your original Koha window will
change to the "Adding authority Personal Name" screen (in the Eric
example).
10) All the relevant fields should be filled out for the record. Review
them and make any changes as necessary. (N.B. The 001 will be cleared
when saved, so if you have a use for the imported control number, move
it to the 010, 016, or 035 as appropriate. If you have a default value
for the 003, this will also likely be overwritten. Move it if necessary.
The 005 will also be updated when saved, so do not worry about that.)
11) When you're satisfied, click save.
12) Presto! You've imported your first authority record via Z39.50!
--
Here is the info for the LibrariesAustralia test Z39.50 authority
database:
Z39.50 server: LibrariesAustralia Authorities
Hostname: z3950-test.librariesaustralia.nla.gov.au
Port: 210
Database: AuthTraining
Userid: ANLEZ
Password: z39.50
Syntax: MARC21/USMARC
Encoding: utf8
-
The U.S.A. Library of Congress also provides Z39.50 access to its Name
and Subject Authorities (http://www.loc.gov/z3950/lcserver.html).
Name Authority:
Z39.50 server: Library of Congress Name Authority File
Hostname: lx2.loc.gov
Port: 210
Database: NAF
Syntax: MARC21/USMARC
Encoding: utf8
Subject Authority:
Z39.50 server: Library of Congress Subject Authority File
Hostname: lx2.loc.gov
Port: 210
Database: SAF
Syntax: MARC21/USMARC
Encoding: utf8
(N.B. Both of these databases also include title authorities.)
-
For testing purposes, you can also set up a Z39.50 client target,
which points at your own Koha instance's Z39.50 authority server.
To find the hostname, go to /etc/koha-conf.xml and find the value for
the <listen id="authorityserver"> element. Depending on your
configuration, this could be something like the following:
unix:/zebra/koha/var/run/zebradb/authoritysocket
(N.B. You might be using a different scheme than unix sockets...)
To find the database, scroll down to the bottom of koha-conf.xml until
you reach the <config> element. Within this, look for the value of the
element <authorityserver>. It should probably be "authorities".
To set up this Z39.50 client target in Koha...
Z39.50 server: my koha authorities
Hostname: unix:/zebra/koha/var/run/zebradb/authoritysocket
Port:
Database: authorities
Userid:
Password:
Syntax: MARC21/USMARC (or whichever flavour you need)
Encoding: utf8
Signed-off-by: Mason James <mtj@kohaaloha.com>
Bug 10096 [FOLLOW-UP] - Add a z39.50 interface for authority searching
This patch adds the "recordtype" column to the "z3950servers" table.
The value in this column (biblio or authority) then controls whether
the z3950 server shows up in a bibliographic search (through the
Acq and Cataloguing modules) or in an authority search (through
the Authorities module).
I also edited the z3950 management console to show this value
and allow users to edit it. The default value is "biblio", since
the vast majority of z3950 targets will be bibliographic. However,
there is an option to add/edit a z3950 target as a source of
authority records.
Test Plan:
1) Apply both patches
2) Run updatedatabase.pl (after setting your KOHA_CONF and PERL5
environmental variables)
3) Use the test plan from the 1st patch
N.B. Make sure that your Z39.50 client target has a Record Type
of Authority, otherwise it won't display when you're doing a
Z3950 search for authorities.
Signed-off-by: Mason James <mtj@kohaaloha.com>
Bug 10096 [FOLLOW-UP] - fix tabs/whitespace errors to pass QA
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch corrects the mixup for LC call number and control number.
Further, as suggested by Galen, it would be better to not introduce hardcoded
tags in the Z3950Search subs in Breeding.pm.
This patch resolves that by calling TransformMarcToKohaOneField.
Note that this only involves changes to _addrowdata and _isbn_show. These
subs are only used in building the displayed results table.
Additionally, for French UNIMARC installs publicationyear is used to fill
the Date column (copyrightdate is not used in those installs). The edition
statement is only used in unimarc_lecture_pub not in unimarc_complet.
Test plan:
Do some Z3950 searches and look for values in all result columns.
For MARC21 on LOC (and/or others):
Look for isbn 9780415964845 (check LCCN).
Look for author Rowling.
For UNIMARC on BNF2 (and/or others):
On BNF2 look for isbn 2070518426: result contains date and multiple isbn's.
Look for title: Guide des candidats aux emplois de commissaire de police.
Third result show edition statement (if you use 205$a with pub install).
Note that there are no results with LCCN here (just as before).
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Tested for MARC21 and UNIMARC (French lecture_pub install).
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
As Jonathan correctly noted, the new Z3950 response only showed one isbn
although more isbn numbers could be in the record and would be imported.
To resolve this display problem, I traverse them all now in the updated
routine _isbn_show. There is no change in the imported records.
Note that before this patch TransformMarcToKoha did put all isbn numbers in
one field, separated by pipes (for display only). This behavior is restored
now. The three regexes on the individual isbn numbers now seem to be
overkill, but I left them there for completeness.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Tested this on a fresh French install under UNIMARC with BNF server.
Tested it too for MARC21.
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Refactors Z3950Search.
Disable batch record counts for z3950 records.
Test plan:
Do various Z3950 searches on multiple targets from Cataloging and Acquisition.
Behavior should not have changed.
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Searching for stdid: Standard ID, srchany: RAW (any) somehow did not work
anymore.
Probably my fault :) Note that these two fields are in Cataloging Z3950 search
and not in Acquisition.
Fixing encoding problems: When adding -utf flag for CGI in acqui/z3950 and
cataloging/z3950, the decoding statements in C4/Breeding, Z3950Search should be
removed.
Test plan:
Search in Cataloging with:
Standard ID: 9782358670043 on LOC
RAW (any): musee [add an accent aigu on first e] on LOC -- Add diacritic!!!
Search in Acquisition
Somewhere, does not matter, but use a diacritic.
A note: My git version still has a hard time with utf8. Need to upgrade to version 1.7.10 to resolve this..
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Comment: Work as described. No errors
Without patch z39.50 search for example Std ID OR musee gives no results,
with patch there are.
No problems in acq search.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Good catch, passes all tests and QA script.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Housekeeping: close the results sets and connections from Z3950 searches.
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
No regression found, all tests and QA script pass.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Removing some unused variables.
Restoring timeout parameter that was only used in cataloging.
Restoring copyrightdate and editionstatement in row data for template.
Small adjustment at the end of the while loop with template vars.
Discovered while doing so, that the paging feature needs some further corrections; will propose a patch under a separate report.
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Remove some debug warnings, fix indentation
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
As a first step in realizing the goals of report 6536 (Z3950 Search improvements), this patch moves identical code in acquisition and cataloging to module level.
A followup deals with formatting.
Note that this patch should not change any behavior.
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
I did not find any regression
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Some Z39.50 server may use the MARC-8 encoding, which uses separated
diacritics. By forcing a normalization, all imported records will have
combined diacritics.
Records with separated diacritics might not show up in Zebra searches if
the search terms use accented characters.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
http://bugs.koha-community.org/show_bug.cgi?id=8610
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
checked it still works after the patch with UNIMARC and BNF server (that
provide utf-8 records)
Searching for a 10 or 13 digit numeric string that does not exist in
one's catalog will fail to affect SQL statement correctly
in C4::Breeding. Moved string substitution, which was triggering error
when search was undefined, and fixed if statements accordingly.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
svc/import_bib:
* takes POST request with parameters in url and MARC XML as DATA
* pushes MARC XML to an impoort bach queue of type 'webservice'
* returns status and imported record XML
* is a drop-in replacement for svc/new_bib
misc/cronjobs/import_webservice_batch.pl:
* a cron job for processing impoort bach queues of type 'webservice'
* batches can also be processed through the UI
misc/bin/connexion_import_daemon.pl:
* a daemon that listens for OCLC Connexion requests and is compliant
with OCLC Gateway spec
* takes request with MARC XML
* takes import batch params from a config file and forwards the lot to
svc/import_bib
* returns status
ImportBatches:
* Added new import batch type of 'webservice'
* Changed interface to AddImportBatch() - now it takes a hashref
* Replaced batch_type = 'batch' with
batch_type IN ( 'batch', 'webservice' ) in some SELECTs
Signed-off-by: MJ Ray <mjr@phonecoop.coop>
Reimplementation of Nahuel's patch from 2010-03-02
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
* also start to standardize ISBN normalization
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
* IsStringUTF8ish - determine if scalar contains a string in UTF8
* MarcToUTF8Record - convert MARC blob or MARC::Record to UTF8
* SetMarcUnicodeFlag - set appropriate MARC21 or UNIMARC field to
indicate that record is in UTF-8.
Design points of this module include:
* No dependencies on other C4 modules, making it easier to add
more test cases
* All character conversion code in one place
* Single entry point for doing a character conversion on a
MARC record
* Capture of errors and warnings produced by Text::Iconv
and MARC::Charset
* Start of support for guessing the source character set of
a MARC record.
Several functions were moved from other scripts
or modules to C4::Charset:
* C4::Koha->FixEncoding (expanded and renamed
MarcToUTF8Record)
* C4::Koha->char_decode5426
* fMARC8ToUTF8 from bulkmarcimport.pl (renamed
_marc_marc8_to_utf8)
Several batch jobs were adjusted to use MarcToUTF8Record instead of
FixEncoding.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Adding encoding for z3950 server information.
Uses Text::Iconv for conversion (ISO6937 and ISO_5428 and ISO5427)
For ISO 5426 (ANSEL or MARC-8) new char_decode5426 based on marc4j tool.
Not Tested on LOC or any USMARC z3950 source. But tested OK on BNF and SUDOC.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Revamps the import options on the tools menu to have two parts:
[1] Staging (load file into reservoir)
[2] Managing (review the list of staged batches, then
choose to commit or undo a given batch.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
* Completely removed old marc_breeding table
* Started updated Tools import function to stage records
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
== Biblio.pm cleaning (useless) ==
* some sub declaration dropped
* removed modbiblio sub
* removed moditem sub
* removed newitems. It was used only in finishrecieve. Replaced by a Koha2Marc+AddItem, that is better.
* removed MARCkoha2marcItem
* removed MARCdelsubfield declaration
* removed MARCkoha2marcBiblio
== Biblio.pm cleaning (naming conventions) ==
* MARCgettagslib renamed to GetMarcStructure
* MARCgetitems renamed to GetMarcItem
* MARCfind_frameworkcode renamed to GetFrameworkCode
* MARCmarc2koha renamed to TransformMarcToKoha
* MARChtml2marc renamed to TransformHtmlToMarc
* MARChtml2xml renamed to TranformeHtmlToXml
* zebraop renamed to ModZebra
== MARC=OFF ==
* removing MARC=OFF related scripts (in cataloguing directory)
* removed checkitems (function related to MARC=off feature, that is completly broken in head. If someone want to reintroduce it, hard work coming...)
* removed getitemsbybiblioitem (used only by MARC=OFF scripts, that is removed as well)
Uses a complete new ZEBRA Indexing.
ZEBRA is now XML and comprises of a KOHA meta record. Explanatory notes will be on koha-devel
Fixes UTF8 problems
Fixes bug with authorities
SQL database major changes.
Separate biblioograaphic and holdings records. Biblioitems table depreceated
etc. etc.
Wait for explanatory document on koha-devel
actually existed; so if there was no isbn, and the issn was blank,
the item would be assigned a random biblionumber and the breeding farm
would report that the item already exists in the catalog (even though
it didn't). This fix adds a check to determine whether the imported
record has an issn before assigning a matching biblionumber.
