Commit graph

19733 commits

Author SHA1 Message Date
Jonathan Druart
5ad6350616 Bug 10853: All existing routing to get a CSV should return a MARC csv
Test plan:
Verify that existing CSV lists list MARC CSV profiles and not SQL CSV
profiles.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-11 02:16:33 +00:00
Bernardo Gonzalez Kriegel
06c5d812bd Bug 10853 - followup to correct tabs
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-11 02:16:17 +00:00
Jonathan Druart
0bf14c2d3a Bug 10853: Add DB field export_format.type ('marc' or 'sql').
This patch:
- adds a new column 'type' to the export_format table.
- renames the field name export_format.marcfields with
  export_format.content.

Test plan:
- Check that existing profiles have the type "marc" selected by default
- Create a new profile with a type "sql"
- Save and verify the profile is correctly displayed when you select it.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Comment: Work as described. koha-qa reports Small tabs errors,
corrected in followup

Test:
1) go to Tools > CSV profiles, Create profile, current
2) Apply patch, run updatedatabase
3) Go to Tools > CSV profiles, new option present
old profile with type MARC
4) Create new profile MARC, save and show correct
5) Create new profile SQL, save and show correct

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
All tests and QA script pass with all 3 patches applied.
Works as described. Functionality for SQL profiles will be
added by another patch. For now it's possible to add/edit/delete
them.
Existing CSV profiles can still be exported correctly.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-11 02:12:41 +00:00
Galen Charlton
280b0df19d Bug 10240: DBRev 3.13.00.027
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-11 01:57:05 +00:00
Galen Charlton
57848b642b Bug 10240: (follow-up) don't display patrons as lost or gone-no-address incorrectly
Because borrowers.lost and borrowers.gonenoaddress are nullable, when
downloading patron information for an offline patron sync, convert null
values to zero.

To test:

[1] Set borrowers.lost to NULL for a patron.
[2] Do an offline sync, then disconnect network access (or
    stop the webserver) and enter the offline interface.
[3] Try checking out to the test patron.  A warning will
    (incorrectly) disply stating that the patron's card is lost.
[4] Apply the patch.
[5] Do steps 2 and 3 again.  This time, there will be lost card
    warning.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-11 01:57:05 +00:00
Jared Camins-Esakov
c1ec92f685 Bug 10240: (follow-up) correctly record fines and fix label
At some point in rebasing I managed to remove the part of the code
that saved fine payments. This patch re-adds that feature. This patch
also corrects the label on the check out tab to not mention partial
names for checkouts when offline, and partial name searches are not
supported in offline circ.

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-11 01:57:05 +00:00
Jared Camins-Esakov
b6ea9da689 Bug 10240: (follow-up) make sure datepicker icons are available offline
The next month/previous month icons for the datepicker were not available
offline. With this patch they are.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-11 01:57:05 +00:00
Jared Camins-Esakov
2c89aee3c8 Bug 10240: (follow-up) warn when patron's card is expired
This patch improves the alert messages to be slightly better English
and warns the librarian if a patron's card has expired. Like all alerts,
this is non-fatal since in the case of network failure there is no
particular reason to expect that the offline database is current.
To test this particular patch you can try checking something out to an
expired patron, otherwise test plan remains the same as above.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-11 01:57:04 +00:00
Jared Camins-Esakov
46190d5ac4 Bug 10240: (follow-up) disable logout/set library
Logging out/setting the library does not work while using offline
mode, so it makes no sense to present those options to the user.
Much better is some sort of explanatory message informing them that
those two links don't work.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-11 01:57:04 +00:00
Jared Camins-Esakov
618ddd3490 Bug 10240: (follow-up) fix display issues
Fix the following two issues:
1) After paying a fine when offline the fine amount becomes NaN.
2) For previous checkouts for a patron, the title and barcode
   fields have the wrong infomation in them (i.e. they have been swapped)

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-11 01:57:04 +00:00
Jared Camins-Esakov
cbf7e3b11b Bug 10240: (follow-up) don't delete transactions if auth fails
When uploading transactions, we were not checking that authentication
had succeeded before deleting the transactions from the local database.
That was bad. With this patch, we check. That is good.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-11 01:57:04 +00:00
Jared Camins-Esakov
9db23b8483 Bug 10240: QA follow-up
Address the following issues:
    1/ Address minor qa issues with the templates:
     FAIL    koha-tmpl/intranet-tmpl/prog/en/modules/circ/offline-mf.tt
      FAIL      forbidden patterns
                forbidden pattern: intranet-tmpl should certainly
                replaced with [% interface %] (line 24)
                [etc.]
     OK      tt_valid
     OK      valid_template

    FAIL    koha-tmpl/intranet-tmpl/prog/en/modules/circ/offline.tt
     FAIL      forbidden patterns
               forbidden pattern: intranet-tmpl should certainly
               replaced with [% interface %] (line 509)
               [etc.]
    FAIL      tt_valid
        lines 5, 5
    2/ Run perltidy on new scripts
    3/ download.pl returns data.finished = 1 if number of returned
       data < 5000 (avoids 1 ajax call)
    4/ Replace qq{} around sql queries with q{}

Also, a race condition existed that resulted in pending transactions
only getting deleted from the local database in certain circumstances
(fast connections under Chrome, mostly). This patch fixes that so that
successfully-uploaded transactions are always deleted.

This patch also addresses Jonathan's suggestion:
3/ add a message on check in (currently the input becomes empty but the
   user is not informed).

... and Magnus's suggestion about moving the Synchronize link to the
right on the homepage.

Also, this addresses the further issues Jonathan noted:

- The tab of checkouts always shows "*0* Checkouts"
- If I am not well-educated, I click on the "Check out" link on the
  offline home page, I enter a barcode, click on "Check out" and I get a
  js error (without user message): "TypeError: curpatron is undefined"
  (with chromium I get: Numeric transaction modes are deprecated in
  IDBDatabase.transaction. Use "readonly" or "readwrite").
- There is a "border-right" css rule on the h5.patron-title. It is
  display when there is no patron selected) [really minor!].
- tables are displayed even if there is no data
- The "Clear screen" link (X) points to an old script:
  circ/offline-circulation.pl
- There is a warning when clicking on the "Synchronize" link when the
  user is offline, but not for the "Pending offline circulation actions"
  link.
- Still exists:
> The "Checked in item." message text never disappear (even if I go on the
> offline home page (circ/offline.pl#offline-home)).

Finally, this patch adds a link to the Pending offline operations page
on the synchronize page for easier navigation.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-11 01:57:03 +00:00
Jared Camins-Esakov
44d0ad451f Bug 10240: Offline circulation using HTML5 and IndexedDB
This patch adds an HTML5-based offline mode to Koha's existing
circulation module, allowing librarians to check out items using a
basically familiar interface. The feature will be implemented using
the Application Cache and IndexedDB features of the HTML5 specification,
both of which are fully supported on Firefox 10+ and Chrome 23+, with
limited support going back to Firefox 4 and Chrome 11. The basic
workflow enabled by this patch is as follows:

Part 1: While connected to the Internet
1. Enable offline functionality by turning on the
   "AllowOfflineCirculation" system preference.
2. Sync the offline circulation database on the computer that will be
   used for offline circulation by following the "Offline circulation
   interface" link on the Circulation home page, choosing "Synchronize (must be online)",
   and clicking the "Download records" button. This process may take a while.
3. Bookmark /cgi-bin/koha/circ/offline.pl (the page you are currently
   on) for easy access when offline.

Part 2: While disconnected from the Internet
4. Navigate to /cgi-bin/koha/circ/offline.pl using the bookmark you
   created while online.
5. Start checking books in by scanning the barcode of an item that has
   been returned into the box in the "Check in" tab.
6. Scan the barcodes of any additional items that have been returned.
7. Start checking out books to a patron by scanning the patron's barcode
   in the box in the "Check out" tab.
8. Set a due date (the "Remember for session" box will be checked by
   default, since circulation rules are not computed during offline
   transactions and therefore a due date must be specified by the
   librarian).
9. Scan an item barcode (if you did not set a due date, it will prompt
   you) to check the item out to the patron.
10. If a patron has a fine you can see the total amount (current to when
    the offline module was synced), and record a payment. Unlike when in
    online mode, there will be no breakdown of what item(s) fines are
    for, and you will only be able to record the payment amount and not
    associate it with a particular item.

Part 3: While connected to the Internet
11. Click the "Synchronize" link and choose "Upload transactions" to
    upload the transactions recorded during the offline circulation
    session.
12. Navigate to /cgi-bin/koha/offline_circ/list.pl (there will be a
    link from the Offline circulation page) and review the
    transactions, as described in the documentation for the Firefox
    Offline circulation plugin:
    http://wiki.koha-community.org/wiki/Offline_circulation_firefox_plugin

RM note: the IndexedDB jQuery plugin bundled with this patch is
copyright 2012 by Parashuram Narasimhan and other contributors and is
licensed under the MIT license.  The home page for the plugin is
http://nparashuram.com/jquery-indexeddb/.

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>

Comment: Works very well, no koha-qa errors

Test with Firefox 24.0
1) did some checkouts pre sync
2) synchronize database (Download)
3) go offline
4) Proceed to checkin some items from patron
5) Proceed to checkout items to patrons, setting date
6) Proceed to checkout to expired patron, warning appears
7) go online
8) Upload records
9) go to review transacctions and proceed
10) verified on patrons that checkin/out are done

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-11 01:53:34 +00:00
Chris Hall
015b9c660c bug 10365: change routing slips to use date published rather than planned date
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 23:38:11 +00:00
Galen Charlton
5307d10a91 Bug 10602: DBrev 3.13.00.026
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 23:30:26 +00:00
Bernardo Gonzalez Kriegel
329e7511b3 Bug 10602: (follow-up) to remove tabs
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 23:27:48 +00:00
Jonathan Druart
e273e11b08 Bug 10602: Set default value for authority fields via the framework
This patch allows to define default values in the authorities framework.

Some code already existed but the feature did not work.

Test plan:
1/ Choose a framework, field and subfields.
2/ Define a default value.
3/ Create a new authority and check that the subfield is
automatically filled with the default value.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Work as described. koha-qa reports some tabs, fixed in followup

Test
1) Apply patch, run updatedatabase.pl
2) Edit auth framework, put default value someware, save
3) Add new auth, default value present

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.
Verified database update is done correctly.

Controlfields 0xx
- Edited an existing field (001)
- Set a default value for subfield @
- Edited subfield again, checking default was saved correctly
- Verified the default shows up correctly when creating a
  new authority using this authority type

Fields
- Edited an existing field (100)
- Set a default value for subfield e
- Edited subfield again, checking default was saved correctly
- Verified the default shows up correctly when creating a
  new authority using this authority type

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 23:25:47 +00:00
Galen Charlton
81cdaf131f Bug 9295: DBRev 3.13.00.025
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 23:06:47 +00:00
Mirko Tietgen
8369aa8431 Bug 9295: Introduce operator equal/ notequal to OAI set mapping instead of hardcoded 'equal' value.
In OAI set mappings, the value "is equal to" is hardcoded. This
enhancement changes it to a dropdown menu to choose between "is equal
to" and "not equal to".

To test:

* define a set
* define a mapping for said set with "is equal to"
* run /misc/migration_tools/build_oai_sets.pl -r -v
* confirm that you have correct entries in SQL: select * from
  oai_sets_biblios;
* change mapping to 'not equal to', save
* run /misc/migration_tools/build_oai_sets.pl -r -v
* confirm that you have correct entries in SQL: select * from
  oai_sets_biblios;

Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Amended patch: Fix bug id in updatedb.pl

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 23:03:30 +00:00
Galen Charlton
9f6999bfdb Bug 9282: (follow-up) remove log noise caused by authorities/ysearch.pl
This patch (which is incidental to the main patches for bug 9282),
improves the AJAX authority search code by fixing incorrect contruction
of the parameters to SearchAuthorities() that led to errors like this
in the Apache log when doing auto-completion searches on "main entry"
and "anywhere" in the authority finder:

   ysearch.pl: Use of uninitialized value $i in string eq ...

In the process, this patch also removes a use of the now-deprecated
Perl smartmatch operator.

To test:

[1] Verify that the main test plan for bug 9282 still works.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 22:46:18 +00:00
Fridolyn SOMERS
c1a289a191 Bug 9282: (follow-up) add authtypecode to authority auto-completion search
Signed-off by: Mathieu Saby <mathieu.saby@univ-rennes2.fr>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 22:26:45 +00:00
Fridolyn SOMERS
95c7bb4adf Bug 9282: improve auto-completion for authority record finder
This patch adjusts the auto-completion on the authority record
finder (accessed from the bib editor) so that if you do
start typing in the "Main entry ($a only)" input field, it will
return only the $a of the main heading for matching authority
records.

This fixes a problem where typing "shakes", choosing
"Shakespeare, William, 1564-1616" from the auto-completion
result list, then hitting the search button fails to bring
up results, as the dates come from the $d of the 100 field
(in MARC21).

Signed-off-by: Mathieu Saby <mathieu.saby@univ-rennes2.fr>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Works as advertised.
Tested with an authority where I added my search term in $b.
The modified authority came up in main entry, not in mainmainentry.
That was the desired result.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 22:23:09 +00:00
Galen Charlton
aa86e7eb75 bug 5202: (follow-up) tweak display of merge action link on staged batch page
This patch adds a vertical bar between the link to display the
matching authority record and the link to initiate a record
merge; prior to this patch, the two links ran together.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 21:57:29 +00:00
Galen Charlton
4a2aa0e768 bug 5202: (follow-up) adjust for the move of the merge record Javascript
This patch applies to change from bug 10650 to the new
authority record merge feature.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 21:50:08 +00:00
Jared Camins-Esakov
ec3986c1db Bug 5202: QA follow-up - correct license and POD errors
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 21:39:15 +00:00
Jared Camins-Esakov
adabb74aa1 Bug 5202: QA follow-up - improve merge reference selection
The link on the merge reference had been wrong, and framework types
were not indicated when you were merging records of two types. This
patch fixes those problems.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 21:38:45 +00:00
Jared Camins-Esakov
a512481af3 Bug 5202: (follow-up) Quiet warning
$mergereference is not always defined, so we should check it is
defined before checking its value.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 21:38:11 +00:00
Jared Camins-Esakov
cbd9591010 Bug 5202: QA follow-up 2 - fix overeager error message
When rebasing the patch for this, I accidentally moved the line
setting the mergereference when merging from breeding to *before*
I checked whether we were merging from breeding. Needless to say,
this caused problems. This patch fixes that, and:
* Makes the error translatable.
* Adds a missing authtypesloop argument for the template.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 21:37:31 +00:00
Jared Camins-Esakov
1460065d24 Bug 5202: QA follow-up: quiet warnings
Due to massively incorrect data in the default authority frameworks,
we were getting warnings about undefined values and spaces from checking
which tab subfields/fields were displayed in. This patch eliminates those
warnings.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 21:36:59 +00:00
Jared Camins-Esakov
b5bd2b7586 Bug 5202: merge authorities from the authority file and reservoir
This patch gives Koha the ability to merge authority records using the
same interface used by bibliographic records, though slightly different
methods for selecting which records to merge. The two ways to select
records are as follows:

1) Records can be selected from authority search results by clicking
   the "Merge" link for two records.
2) Authority records can be merged from the reservoir by clicking the
   merge-related links in the Manage staged MARC batch screen.

To test:
1) Apply patch.
2) Do a search for an authority record that will turn up multiple
   identical records (or at least two records that you don't mind
   merging).
3) Click the "Merge" link for the first record.
4) Click the "Merge" link for the second record.
5) Choose which fields from which record you want to appear in the
   resulting record.
6) Confirm that those are the fields that exist in the resulting record.
7) Stage an authority record (for example, an authority record you
   saved from your catalog.
8) Search for a record to merge with it using the "Search for a record
   to merge in a new window" link.
9) Merge these records, confirming that the resulting record (after
   going through the entire merging process) matches your expectations.
10) Set up a matching rule for authorities, and export an authority from
    your catalog that will match based on that rule. For MARC21, the
    following is a good choice for a rule:
    Matching rule code:  AUTHPER
    Description:         Personal name main entry
    Match threshold:     999
    Record type:         Authority record
    [Match point 1:]
    Search index:   mainmainentry
    Score:          1000
    Tag:            100
    Subfields:      a
11) Stage the record you just exported, choosing the matching rule you
    just created.
12) Merge the record using the "Merge" link, confirming that the
    resulting record (after going through the entire merging process)
    matches your expectations.

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Testing notes on last patch in series.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 21:32:27 +00:00
Mathieu Saby
4d8b1ec786 Bug 7421: support indexing UNIMARC authority records using the DOM Filter
I took as a base the patch of F. Demians, but made a lot of changes,
so I think it is more logical to create a new patch as the behavior is
not the same as previous patch.

I tried to define DOM config files as a "miror" of record.abs, so the
behavior be the same.

If it is OK, we will be able to improve indexing later, for example
suppressing warns, managing indicators or subdivisions, etc.

I made some little changes to record.abs :
- comments
- 216 was indexed in Conference-name as well as Trademark. I suppose
  that "Conference-name" is an error, so I indexed only in Trademark
- index 2 new notes : 340 / 356

The only difference between record.abs and DOM is that DOM config files
does not index complete fields, but subfields.

Ex :
melm 200 ===> <kohaidx:index_subfields tag="200" subfields="abcdfgjxyz">
I took all the subfields from the UNIMARC Authorities manual. The only
subfields not indexed are numeric subfields : $7, $8 for language of
record, and $0,2,3,5,6 for 4XX/5XX/7XX

To test :
- index a set of bib and auth records with GRS-1
- make some searches on different kind of authorities
- index the same records with DOM
- make the same searches
- You are not supposed to see differences

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
As I am not a UNIMARC user it's hard for me to test this, but
while testing other authority related patches I noticed that I couldn't
index the UNIMARC authorities of the sample base. The files are obviously
missing and reindex_zebra.pl notes this. With this patch applied,
indexing works and authorities are searchable in my installation.

Signed-off-by: Vitor Fernandes <fvernandes@keep.pt>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 21:03:15 +00:00
Galen Charlton
521e599dd8 Bug 7421: add regression tests for UNIMARC authorities DOM indexing
This patch enhances  t/db_dependent/Search.t to test indexing and
searching of authority records.  It supplies a sample authority file
for MARC21 (consisting of the record for William Shakespeare from the
Library of Congress) and a sample UNIMARC authority file supplied
by Henri-Damien Laurent.

It also adds tests for both MARC21 and UNIMARC authorities.

When the main patch for bug 7421 is applied, t/db_dependent/Search.t
should pass.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 21:02:04 +00:00
Janusz Kaczmarek
68c44093a4 Bug 10335: display translated forms of headings in UNIMARC authorities correctly
To reproduce and test:

To reproduce:
1) Create an authority record with main heading (100) in Latin script
   (e.g. Oppenheimer, Aharon -- subfields $a and $b) and parallel form
   (700) in Hebrew (אופנהיימר, אהרן -- subfields $a and $b).
    Mark it correctly in $8 with freheb (or engheb if you like);
2) Reindex and search;
3) You will see:

Oppenheimer Aharon
freheb: אופנהיימר

Whereas you would rather like to see (mind language and lack of $b above):

Oppenheimer, Aharon
Hebrew: אופנהיימר, אהרן

The patch corrects the issue and should not harm those who (improperly)
put only one triple in $8

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Work as described. No koha-qa errors.
Same result on OPAC and STAFF

Turns out that test plan is wrong,
you neet to fill tag 200ab, not 100ab, for main heading.
I filled 100a with some example data from UNIMARC auth manual.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Took me a bit to figure it out, works according to test plan.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 19:14:44 +00:00
Mathieu Saby
5298140c67 Bug 10037: fix item index in UNIMARC DOM indexing
In UNIMARC DOM indexing, "item" index was working only for subfields
of 995 field mapped with specific indexes, and also in index (ex :
$a, $b...). It was not working for the other subfields (ex : $g),
because a comment from record.abs was integrated in DOM config files.

This patch removes the comment.

To test, in a DOM UNIMARC environment :
1) In a item, write some value "Test10037" in 995$g
2) Search for this value in simple search, this way : item=Test10037
   => you should have no results
3) Apply the patch. if necessary, copy the modified
   etc/zebradb/marc_defs/unimarc/biblios/biblio-koha-indexdefs.xml and
   etc/zebradb/marc_defs/unimarc/biblios/biblio-zebra-indexdefs.xsl into
   the /etc/... directory in your main Koha directory
4) Reindex Zebra biblios
5) Do the same search as 2) => you should have one result

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Work as described. No koha-qa errors.

Test

NOTE: default UNIMARC framework don't have 995g,
so I must add it first.

1) Added test string to 995b on some record
2) Reindex and search as indicated, no results
3) cp files to destination
4) reindex
5) search and result ok !

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 18:54:12 +00:00
Galen Charlton
2173a97b29 Bug 10037: regression test for searching UNIMARC item index in DOM mode
This patch adds a regression test that verifies that searching the
UNIMARC 'item' index for a value that is indexed by virtue of being
included in the fallback index definition for the 995 field works for
both GRS-1 and DOM.

The main patch will allow t/db_dependent/Search.t to pass.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 18:50:13 +00:00
Galen Charlton
c9b09051e7 Bug 8252: (follow-up) add basic UNIMARC indexing tests
This patch adds a few basic search and indexing tests for UNIMARC,
using sample data provied by Frédéric Demians.  The new tests
by no means completely cover the possibilities, but should be
considered a starting point for future tests.

To test:

[1] Verify that prove -v t/db_dependent/Search.t passes.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 18:31:02 +00:00
Galen Charlton
e3ccbe145c Bug 8252: (follow-up) make it easier to add UNIMARC indexing tests
This patch refactors t/db_dependent/Search.t so that it can be
subsequently modified to add tests for indexing and searching
UNIMARC records without duplicating the setup code.

This patch also moves the MARC21 records from t/db_dependent/data/zebraexport
to t/db_dependent/data/marc21/zebraexport so that UNIMARC sample
data can be slotted in t/db_dependent/data/unimarc/zebraexport.

To test:

[1] Verify that prove -v t/db_dependent/Search.t passes.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 17:58:30 +00:00
Galen Charlton
104c10b248 bug 8252: (follow-up) add search tests on music number
There was some churn in previous patches about the desired
name of the music publisher number index, so this patch
adds tests to confirm that both the old and new names
work as CCL keywords.

To test:

[1] Verify that t/db_dependent/Search.t passes.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 17:28:51 +00:00
Galen Charlton
1737ff4cc3 bug 8252: (follow-up) test both GRS1 and DOM indexing
This patch expands t/db_dependent/Search.t to run
the same tests using both the GRS-1 and DOM indexing
modes.  It also adds hooks in zebra_config.pl to make
it easier to stage test cases for non-MARC21 Zebra
indexing.

Note that in DOM mode one of the tests is currently a
TODO, as relevance ranking for wegihted queries differs
between GRS-1 and DOM.

To test:

[1] Verify that prove -v t/db_dependent/Search.t passes.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 17:18:43 +00:00
Galen Charlton
8ea3462517 Bug 8252: (follow-up) standardize name of Identifier-publisher-for-music index
To test:

[1] When running t/db_dependent/Search.t, veify that no warnings like
    this are shown:

15:52:07-10/10 zebraidx(2006) [warn] Index 'Number-music-publisher' not found in attset(s)

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 16:05:33 +00:00
Galen Charlton
45d0365d12 Bug 8620: (follow-up) apply to NORMARC and MARC21 authorities
This applies the fix for the Any index to NORMARC bib
and MARC21 authority DOM Zebra indexes.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 15:56:13 +00:00
Galen Charlton
5024e519ad Bug 8252: (follow-up) tidy up long lines in bib1.att
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 15:56:13 +00:00
Mathieu Saby
475a9d19d1 Bug 8252: (follow-up) fix biblio-zebra-indexdefs.xsl
This patch fixes biblio-zebra-indexdefs.xsl files.
It was generated from biblio-koha-indexdefs.xsm with the new
koha-indexdefs-to-zebra.xsl amended by F. Démians's patch.

To test :
- Take a DOM UNIMARC Koha
- Apply all the patchs of 8252 bug, including this one
- Copy src/etc/zebradb/marc_defs/unimarc/biblios/biblio-zebra-indexdefs.xsl
  to your etc/zebradb/marc_defs/unimarc/biblios/ located in your
  installation directory
- Run rebuid_zebra -b -x -r -v
- make advanced searches on staff interface and opac, on coded fields
  indexes (Audience, Literary genre, Biography, Illustration, Content,
  Video Types, Serial Type, Periodicity, Regularity, Picture)

Signed-off-by: Frédéric Demians <f.demians@tamil.fr>

Ok for me. This patch put in sync indexes XSL definition with
authoritative XML definition. Subsequently, it won't be difficult to
amend DOM UNIMARC indexes defintion if necessary. And, as it is, I don't
see any regression, whereas I can see huge improvements. Thanks Mathieu!

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 15:21:56 +00:00
f9addcc98b Bug 8852: DOM XSL now handles subfield substring extraction
This patch modify koha-indexdefs-to-zebra.xsl in order to add the
ability to populate indexes with subfield substring.

It's now possible to understand such construction as:

<index_subfields xmlns="http://..." tag="100" subfields="a" offset="7" length="1">
  <target_index>tpubdate:s</target_index>
</index_subfields>

Signed-off-by:Mathieu Saby <mathieu.saby@univ-rennes2.fr>

I applied the patch and ran
  xsltproc koha-indexdefs-to-zebra.xsl ../marc_defs/unimarc/biblios/biblio-koha-indexdefs.xml \
     > ../marc_defs/unimarc/biblios/biblio-zebra-indexdefs.xsl
I looked at the generated file. It looks nice.
Then I copied it file in my INSTALLDIR/etc/zebra.... and reindexed my
records with rebuild_zebra.pl
I made some searches on coded position index and non coded position
indexes, everything works.

http://bugs.koha-community.org/show_bug.cgi?id=8252

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 15:19:20 +00:00
Mathieu Saby
43809d2835 Bug 8252: Followup for Date/time-last-modified and Music number
This followup restores the original wording of "Date/time-last-modified"
index, and change the name of "Music-number" index to
"Number-music-publisher"

To test :
1. In a UNIMARC Koha instance
2. Apply patchs #1, #2 and this followup
3. Copy from src/etc/zebradb directory to the etc/zebradb/ in your main
   Koha directory the following files:
-- zebradb/biblios/etc/bib1.att
-- zebradb/ccl.properties
-- zebradb/marc_defs/unimarc/biblios/record.abs
-- zebradb/marc_defs/unimarc/biblios/biblio-koha-indexdefs.xml
-- zebradb/marc_defs/unimarc/biblios/biblio-zebra-indexdefs.xsl
4. Rebuild zebra with -b -x -v -r options
5. Write a value like "test071a" in 071$a field in a record
6. Check if you can find this record with this search:
   "ccl=Number-music-publisher:test071a"

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
No koha-qa errors.

Test
Copy files
reindex full
Modify a couple of record to add 071a with test message
Reindex -v -z -b -x
Search test message as described and found modified records.

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 15:16:27 +00:00
Mathieu Saby
8034566027 Bug 8252: Fix indexing of UNIMARC 1xx for DOM
This patch makes the same changes in UNIMARC DOM configuration as patch
1 made for GRS-1.

positions of subfields are indexed that way :
In biblio-koha-indexdefs.xml :
tag="100" subfields="a" offset="17" length="1"
In biblio-zebra-indexdefs.xsl :
xslo:value-of select="substring(., 17, 1)"

I had to edit biblio-zebra-indexdefs.xsl by hand, because
etc/zebdradb/xml/koha-indexdefs-to-zebra.xsl does only support
"subtring" in handle-one-index-control-field template.

It is good for MARC21, but not for UNIMARC : in MARC21, indexing
subtrings is needed for controled field (001-009, with no subfields)
But in UNIMARC it is needed for subfields of 1XX fields.
So if DOM indexing is working with these new files, we may need to
change koha-indexdefs-to-zebra.xsl.

Test plan (not possible in a sandbox) :
1) In a Koha instance using UNIMARC and DOM indexing
2) Apply Patch 1 and Patch 2 (this one)
3) Copy the following files from the etc/zebradb directory of your
   source into the etc/zebradb directory of your main Koha directory :
-- etc/zebradb/marc_defs/unimarc/biblios/biblio-koha-indexdefs.xml
-- etc/zebradb/marc_defs/unimarc/biblios/biblio-zebra-indexdefs.xsl
-- etc/zebradb/ccl.properties
-- etc/zebradb/biblios/etc/bib1.att
4) rebuild zebra with -x -b -r -v options
5) check if coded filters in advanced search are usable in OPAC and
   Staff interface

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Works. No koha-qa errors.

Test for DOM
Apply patches
Don't forget to copy files
reindex
Search by coded fields works, also Country-publication

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 15:15:04 +00:00
Mathieu Saby
041e3603a1 Bug 8252: Fix indexing of UNIMARC 1xx for GRS-1
Before fixing UNIMARC DOM indexing, we must fix GRS-1 indexing

1) In advanced search, some Coded fields index are not working: Print,
   Illustration, Content
2) Country-heading index is not working
3) Some subfields are indexed in wrong indexes :

  102$a should be in Country-publication instead of Country-heading
        (non defined in bib1.att)
  106$a, filled only for printed works, should be in ff88-23 (form of
         item) instead of itype.  (ff88-23 is made for Marc21 008 pos
         23, which contains the same data as 106a)
  200$b should be in Material-type instead of (or in addition to) itype
        and itemtype: (Material-type :"free-form string, ... that
        describes the material type of the item, e.g., cassette, kit,
        computer database, computer file.")
  100$a pos 22-24 should not be indexed as "ln" : it is the language of
        the record, not the language of the ressource

4) Index names are too long : if we index new positions of coded fields,
   with existing names it breaks Zebra indexing (there must be a limit
   in line lenghth in record.abs?)
5) There are a lot of warns when rebuiding zebra.

This patch make some changes in bib1.att (could be used later to improve
search) :

- fixing wording for att 51 and 1012
- adding comments for attributes based on MARC21 008 field (8800-8841)
- creating 8806 (tpubdate), 8838 (Modified-code), 8818 (ff8-18), 8840
  (ff8-18-21), 8819 (ff8-19), 8821 (ff8-21), 8828 (ff8-28), 8830
  (ff8-30), 8831 (ff8-31)
- creating attributes specific to UNIMARC : 9701-9707 (Video-mt,
  Graphics-type, Graphics-support, Title-page-availability,
  Cumulative-index-availability, script-Title, char-encoding)
- setting apart 3 blocks of attributes, so it could be easy to make
  further changes :
-- common to Marc21 and UNIMARC : 8806, 8822, 8838
-- slightly different in Marc21 and UNIMARC (different meanings
   according to the type of the record => don't match a single
   UNIMARC field)
-- specific to UNIMARC : 9701-9707

In ccl.properties :
- creating a new index: Country-publication 1=1053
- suppressing some warns by mapping with bib1 att:
  Date-time-last-modified, Name, rtype, Music-number
- defining indexes using the 3 blocks attributes defined in bib1
  (common to Marc21 and UNIMARC, slightly different, specific to UNIMARC)

In record.abs :
- renaming some index for 100-105-110 fields
- correcting indexing of 102$a (country of publication)
                         106$a (ff88-23)
                         100$a pos 22-24 (language of record, no more
                               indexed)
                         105$a pos. 0-3 (illustration code)
                         200$b (for the moment, I keep it indexed in
                               itype and itemtype, but also Material-Type)

In C4/Search.pm :
- adding "Country-publication" index

In OPAC and staff interface template subtypes_unimarc.in :
- renaming indexes to take into account the changes made to Zebra
  config files

To test (this cannot be done with a sandbox) :
1) Apply the patch in a UNIMARC GRS-1 Koha instance
2) Copy the following files from the etc/zebradb of your source
   directory into the etc/zebradb of your main Koha directory:
-- etc/zebradb/biblios/etc/bib1.att
-- etc/zebradb/ccl.properties
-- etc/zebradb/marc_defs/unimarc/biblios/record.abs
3) Reindex your data (rebuild_zebra -x -b -r -v)
4) Try to use those Coded fields indexes in Advanced search, in OPAC
   and Staff interface (available after clicking on "More options",
   then on "Coded information filters"):
   Audience, Print, Literary genre, Biography, Illustration, Content,
   Video Types, Serials, Serial Type, Periodicity, Regularity
5) Try to search "Country-publication=FR" in simple search

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
No koha-qa errors.

Tests for GRS-1
Followed test plan
Search by coded fields works, but only on OPAC,
on staff there are few options
Search by Country-publication works after patch

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 15:06:10 +00:00
Galen Charlton
d88e15acfb bug 6201: (follow-up) update unit tests
This patch updates the unit tests for the BibTeX export
to add a regression test for supplying the author for
non-UNIMARC records.  It also adjusts the test to reflect
the change in quote character from "" to {}.

To test:

[1] Verify that prove -v t/db_dependent/Record.t passes

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-09 05:26:27 +00:00
Blou
41ae12b561 Bug 6201: Add fields 1xx to marc2bibtex (for MARC21 and NORMARC)
Modified Record::marc2bibtex to varlidate fields 100, 110 and 111 in
non-UNIMARC flavours.

Test plan:
1)Search any books in the OPAC with a main entry (1XX in MARC21, 700-720 in UNIMARC)
2)Export the record in the bibtex format
==>The output won't contain the main entry.
3)Apply the patch
4)Export the record
==>The record will contain the main entry

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Fixes a long standing bug.
Passes all tests and QA script.
Tested with multiple records, seems to work well.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-09 05:25:23 +00:00
Katrin Fischer
4f3b3403fb Bug 10705: follow-up - change one more instance of "shippment cost"
Term also occurs on the spent page accessible from the funds
table on the acquisition module start page.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-09 05:00:37 +00:00