Commit graph

58 commits

Author SHA1 Message Date
4ea2f11718
Bug 33412: Fix comparison when checking for overlay framework
TO test:
 1 - Cataloging -> Export catalog data
 2 - Export a record fom your catalog
 3 - Cataloging -> Stage record for import
 4 - Import the record, use Record matchign rule: KohaBiblio (999c)
 5 - Stage, view batch
 6 - Record should match the one that was exported
 7 - When replacing record use this framework, choose a different value than current framework of record
 8 - Import this batch
 9 - View and edit record - note framework did not change
10 - Apply patch
11 - Restart all
12 - Repeat 4-8
13 - View and edit record, confirm framework has changed
14 - Sign off

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-05-09 10:58:04 -03:00
ae4c09ed5e
Bug 15869: (QA follow-up) Don't pass placeholder framework '_USE_ORIG'
This param in the dropdown is meant to signify no change, distinct from
"" when means to set to default. In rebasing this condition was lost, this patch
restores undef of the variable to avoid setting framework to an invalid value

To test:
1 - Stage a record with a match (export and import a record)
2 - Attempt to import and overlay with option 'When replacing records use this framework:' 'Keep original framework'
3 - It fails, logs note 'value too long for field frameworkcode'
4 - Apply this patch
5 - Try again, it succeeds - framework is not changed

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-02-07 10:32:23 -03:00
e5f251a709
Bug 15869: Change framework on overlay
This patch allows for selection of framework to use when overlaying
records - by default it is set to keep the initial framework

To test:
 1 - Create some records using one framework
 2 - Export the records
 3 - Edit the records to add fields not in original framework
 4 - Stage records using a rule that will find matches
 5 - Import
 6 - Note records contain new fields on display, but they are lost on edit
 7 - Apply patch
 8 - Stage records again
 9 - Select a framework that contains the new fields on import
10 - Import records
11 - Note records now use selected framework and are displayed/edited
correctly

Signed-off-by: Andrew Fuerste-Henry <andrewfh@dubcolib.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-02-07 10:32:22 -03:00
9470cf8c1b
Bug 27421: Commit and revert
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-09-19 15:14:56 -03:00
1eee121bed Bug 29325: (QA follow-up) Tidy up
tools/manage-marc-import.pl: sub commit_batch does no longer need $schema
tools/manage-marc-import.pl: sub revert_batch: calling BatchRevertRecords which has no interval and callback
misc/commit_file.pl: sub print_progress_and_commit does no longer commit, renamed
misc/commit_file.pl: sub print_progress does no longer have a schema parameter
misc/commit_file.pl: sub revert_batch reported deleted items as added

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-06-24 11:36:19 -03:00
1f62cbc618 Bug 29325: Fix commit_file.pl
This patch fixes the broken commit_file.pl script.

It doesn't deal with commiting the import from the UI.

To test:
1. Pick a file for staging:
   $ kshell
  k$ misc/stage_file.pl --file TestDataImportKoha.mrc
=> SUCCESS: All good
2. Commit!
  k$ misc/commit_file.pl --batch-number 1
=> FAIL: You see
DBIx::Class::Storage::DBI::_exec_txn_begin(): DBI Exception: DBD::mysql::db begin_work failed: Already in a transaction at /kohadevbox/koha/C4/Biblio.pm line 303
3. Apply this patch
4. Repeat 2
=> SUCCESS: Commit succeeds
5. Sign off :-D

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Bug 29325: (QA follow-up) Remove unexisting parameters of BatchRevertRecords

There is no interval and callback as in BatchCommitRecords.

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Bug 29325: Call progress callback one last time to confirm comppletion

Previously after finishing the loop we were still in a transaction that never completed - we should report progress when done
one final time to commit the last records

To test:
1 - Stage a file with > 100 records
2 - Commit file
3 - Confirm batch is imported and no records left as staged

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Bug 29325: Fix import from staff client

same test as before, but via the staff client

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Bug 29325: Handle the transaction in BatchCommitRecords

Requiring the callback to commit was breaking reversion, and likely elsewhere

Let's simplify and say that the routine iteself will handle the txn and commit

TO test:
1 - Stage a file
2 - Import a file
3 - Revert a file
4 - Test staff client and command line

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-06-24 11:36:19 -03:00
9d8d9321e8
Bug 30738: Log warnings for background MARC import
This change logs warnings from the CGI background MARC import.

Test plan:
0) Apply patch
1) Export a record with an item
2) Update the 952$f subfield to "Circulation" (any value over 10 chars)
3) Import the record and try to add the item
4) The import will stay stuck with "Importing" status and 0% job
progress
5) Check the /var/log/koha/kohadev/intranet-error.log file
and notice there's an error with the following text:
"Data too long for column 'coded_location_qualifier' at row 1"

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-06-08 11:22:41 -03:00
159a1a76c3 Bug 30525: Items batch modification broken
This patch modifies instances in the code which try to get cookies
using:

my %cookies = parse CGI::Cookie($cookie);

In some cases the relevant lines can be removed because they are unused.
In others it can be replaced with:

my %cookies = CGI::Cookie->fetch();

To test, apply the patch and restart_all. Test the following pages to
confirm they load without errors:

- Circulation -> Offline circulation file upload
- Circulation -> Offline circulation -> Add to queue

- Tools -> Batch item modification
- Tools -> Stage MARC for import
- Tools -> Staged MARC management
- Tools -> Batch patron modification
- Tools -> Upload local cover image

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-04-21 13:41:36 -10:00
9d6d641d1f Bug 17600: Standardize our EXPORT_OK
On bug 17591 we discovered that there was something weird going on with
the way we export and use subroutines/modules.
This patch tries to standardize our EXPORT to use EXPORT_OK only.

That way we will need to explicitely define the subroutine we want to
use from a module.

This patch is a squashed version of:
Bug 17600: After export.pl
Bug 17600: After perlimport
Bug 17600: Manual changes
Bug 17600: Other manual changes after second perlimports run
Bug 17600: Fix tests

And a lot of other manual changes.

export.pl is a dirty script that can be found on bug 17600.

"perlimport" is:
git clone https://github.com/oalders/App-perlimports.git
cd App-perlimports/
cpanm --installdeps .
export PERL5LIB="$PERL5LIB:/kohadevbox/koha/App-perlimports/lib"
find . \( -name "*.pl" -o -name "*.pm" \) -exec perl App-perlimports/script/perlimports --inplace-edit --no-preserve-unused --filename {} \;

The ideas of this patch are to:
* use EXPORT_OK instead of EXPORT
* perltidy the EXPORT_OK list
* remove '&' before the subroutine names
* remove some uneeded use statements
* explicitely import the subroutines we need within the controllers or
modules

Note that the private subroutines (starting with _) should not be
exported (and not used from outside of the module except from tests).

EXPORT vs EXPORT_OK (from
https://www.thegeekstuff.com/2010/06/perl-exporter-examples/)
"""
Export allows to export the functions and variables of modules to user’s namespace using the standard import method. This way, we don’t need to create the objects for the modules to access it’s members.

@EXPORT and @EXPORT_OK are the two main variables used during export operation.

@EXPORT contains list of symbols (subroutines and variables) of the module to be exported into the caller namespace.

@EXPORT_OK does export of symbols on demand basis.
"""

If this patch caused a conflict with a patch you wrote prior to its
push:
* Make sure you are not reintroducing a "use" statement that has been
removed
* "$subroutine" is not exported by the C4::$MODULE module
means that you need to add the subroutine to the @EXPORT_OK list
* Bareword "$subroutine" not allowed while "strict subs"
means that you didn't imported the subroutine from the module:
  - use $MODULE qw( $subroutine list );
You can also use the fully qualified namespace: C4::$MODULE::$subroutine

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-07-16 08:58:47 +02:00
6f204fdf96 Bug 28591: Don't pass debug to get_template_and_user
There is a "debug" parameter we are passing from the controller scripts
to C4::Auth::get_template_and_user, but it's not actually used!

Test plan:
Confirm the assumption
Review the changes from this patch

Generated with:
perl -p -i -e 's#\s*debug\s*=\>\s*(0|1),?\s*##gms' **/*.pl

git checkout misc/devel/update_dbix_class_files.pl # Wrong catch
+ Manual fix in acqui/neworderempty.pl

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-06-22 12:04:32 +02:00
Agustin Moyano
8b3a108558 Bug 23019: Add profiles to stage-import-batch and magnage-import-batch pages
This patch adds the logic and the needed UI elements to be able to pre-load an import profile. It also displays which profile was used to stage an import in staged import manager.

To test:
1. Apply all patches
2. Updatedatabase
3. Go to Stage MARC records for Import tool in admin, and upload a file with MARC records.
CHECK => after uploading, there is a fieldset with the legend “Profile settings”
              => inside the fieldset there is a select labeled “Pre fill values with profile”. The only value it has is “Do not use profile”.

4. Change some settings, and set “profile 1” as profile name and click on “Add profile”
SUCCESS => The select now has the new profile selected

5. Change profile select to “Do not use profile”
SUCCESS => Default values are now displayed in the form

6. Reload the page and upload the file again
SUCCESS => the select still has the profile recently added

7. Select the profile, change some parameter in the form and set the profile name to “profile 2”, and click add profile
SUCCESS => there are two profiles now, and if you toggle between them, the parameter changes

8. Select profile 1, change one parameter and click on update profile
SUCCESS => if you toggle that profile with the other, the new parameter of the value is shown when you select profile 1

9. Select profile 2, change some parameter and click Add profile (leaving the name as profile 2)
SUCCESS => the page complains there is another profile with the same name, and asks if you want to replace it.

10. Click on accept
SUCCESS => profile 2 now has the new value in the parameter

11. Select profile 2 and change the name to profile 1
SUCCESS => the page complains there is another profile with that name, and asks if you want to replace it

12. Click on accept
SUCCESS => in profile select there is only one profile called profile 1 that has the values of profile 2

13. Select profile 1 and click remove profile
SUCCESS => there is no profile in profile select.

14. Create a profile and click on “Stage for import”
15. Go to Staged MARC management page
SUCCESS => Improt should have the name of the profile in profile column, and when you click on the file name, there should be the name of the profile in the details.

16. prove t/db_dependent/ImportBatch.t t/db_dependent/api/v1/import_batch_profiles.t
17. Sign off

Signed-off-by: Abbey Holt <aholt@dubuque.lib.ia.us>

Signed-off-by: Abbey Holt <aholt@dubuque.lib.ia.us>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-11-11 15:55:49 +01:00
Julian Maurice
96cc447045 Bug 25898: Prohibit indirect object notation
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-10-15 12:56:30 +02:00
638786e719 Bug 24663: Remove authnotrequired if set to 0
It defaults to 0 in get_template_and_user

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-09-03 10:40:35 +02:00
Katrin Fischer
4a2cc4e903
Bug 14369: Only show 'Create labels' link on staged records import when status is imported
The label creator won't be able to add a batch for items, if they are
not imported into the database (missing itemnumber). This patch hides
the link, when the status is not imported, to avoid the error situation.

To test:
- Export a record with items from your Koha installation
- Delete items and record from Koha
- Stage the record with the items - the "Create labels" link should be showing
- Try to create the labels - note error
- Apply patch
- Retry - the link should no longer show
- Import the records - the link should show now
- Create labels again - a new batch should have been created
- Verify the link on the batch # leads to the batch in the labels module

Bonus: Moves the message from the .pl file to the template to make
it translatable.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-04-21 12:28:56 +01:00
ad37e87b0a Bug 20180: [sql_modes] Remove GROUP BY clause in manage-marc-import.pl
Replaced with a distinct.

Fix for:
'koha_kohadev.me.frameworktext' isn't in GROUP BY

t/db_dependent/www/search_utf8.t

NOTE: I added a [% frameworks.count %] into the template file,
      and compared master and a branch with this applied when
      managing a staged marc import. Values were identical.

Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2018-04-09 16:15:17 -03:00
Jenny Way
cad2db6016 Bug 20017: use Modern::Perl in Tools perl scripts
Test plan:
Check that the following files don't contain use strict; use warnings;
and have use Modern::Perl instead.
background-job-progress.pl
batchMod.pl
copy-holidays.pl
exceptionHolidays.pl
holidays.pl
import_borrowers.pl
koha-news.pl
letter.pl
manage-marc-import.pl
newHolidays.pl
overduerules.pl
quotes-upload.pl
quotes.pl
quotes/quotes-upload_ajax.pl
quotes/quotes_ajax.pl
scheduler.pl
stage-marc-import.pl
upload-cover-image.pl

Also the credits have been added to newHolidays.pl

Signed-off-by: Jon Knight <J.P.Knight@lboro.ac.uk>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2018-02-05 09:46:45 -03:00
96b1f92ae2 Bug 18577 - Importing a batch using a framework not fully set up causes and endless loop
To test:
1 - Create a new framework - don't set it up
2 - Stage some records for import
3 - Manage the import, note your new framwork is in the list
4 - Apply patch
5 - Reload the page
6 - Note the framework is no longer an option

Followed test plan,  worked as intended
Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz>

Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2017-07-10 17:55:19 -03:00
3db0401366 Bug 18806: Fix batch revert
Since bug 18242: When a biblio is deleted, a new transaction is created.
tools/manage-marc-import.pl use the AutoCommit flag, which does not
allow nested transaction.
AutoCommit must not be used.

Test plan:
Import a batch, then revert it.
The biblios must have been correctly deleted, they were not before this
patch

Signed-off-by: Lee Jamison <ldjamison@marywood.edu>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2017-06-19 15:34:34 -03:00
8f436a5075 Bug 15801: Koha::BiblioFrameworks - Remove C4::Koha::getframeworks
C4::Koha::getframeworks returned a hashref of biblio frameworks.
It was mainly used to generate the dropdown list of frameworks.
The scripts modified in this patch did not necessary order the element
by description (frameworktext), the displays were not consistent from
one screen to another.
Using the same search method everywhere:
  Koha::BiblioFrameworks->search({}, { order_by => ['frameworktext'] });
We will know always get the framework in the same order.

Test plan:
Following the different pages modified by this patch, and make sure the
frameworks are displayed correctly in the dropdown list:
1/ acqui/z3950_search.pl - Create an order from an external source.
2/ admin/fieldmapping.pl - Define some mappings keyword / MARC field
3/ admin/marctagstructure.pl - On the MARC frameworks admin page, select
another framework than the default one and click on the 'Search' button
4/ catalogue/MARCdetail.pl - On the MARC defail page, change the
framework you want to use to display the record
5/ cataloguing/addbiblio.pl - Add or edit a biblio record, change its
framework. When editing, the framework of the record should be selected
by default
6/ cataloguing/addbooks.pl - Go on the cataloguing home page and click
on the "New record" button. You should see all the frameworks
7/ cataloguing/merge.pl - Select 2 biblio records to merge. On the first
step (select the merge reference), you should be allowed to select the
framework to use.
8/ tools/inventory.pl - On the inventory page, the "Item statuses" part
should be populated as before this patch
9/ tools/manage-marc-import.pl - Stage records for import. Before
importing them into the catalog, you should see the framework dropdown
list.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Works Ok.
No errors

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-10-28 12:04:59 +00:00
Aleisha
08e448ee4b Bug 9259: Ability to delete a staged file once it has been cleaned
To test:
1) Go to Tools -> Staged MARC Management and clean a file. If you have no files to clean, go to 'Stage MARC for import' and upload one to clean following the necessary steps.
2) Confirm that once the file has been cleaned, the Action column now shows a Delete button. Confirm this button only shows for cleaned files.
3) Click the Delete button.
4) Confirm that clicking Cancel exits the pop-up message and does not delete the file.
5) Confirm that clicking OK refreshes the list of staged records and the one you just deleted is no longer on it (has been deleted). You can confirm this by checking for the file in mysql (SELECT * FROM import_batches WHERE import_batch_id = X;)
6) Run prove -v t/db_dependent/ImportBatch.t (have written unit tests for CleanBatch and DeleteBatch)

Sponsored-by: Catalyst IT
Signed-off-by: Liz Rea <liz@catalyst.net.nz>
Catalyst sign off, so needs another one but YAY this is great.

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-07-08 13:43:53 +00:00
02492bd6ea Bug 15005: Replace $ENV{SCRIPT_NAME} with the hardcoded script paths
Since I don't manage to make SCRIPT_NAME works directly with RequestHeader and/or
ReverseProxyPath, the easier way it to fix all the different occurrences.

Test plan:
On the import patrons page, confirm that the "Import" button does not redirect to a 'Not found' page.
You should stay on the import patrons page.

Note that if this change works, all others should work too.
Have a look at the diff of this patch and confirm there is no typo.

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>

Tested in patron import and item batch modification

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2015-10-19 09:36:43 -03:00
Jonathan Druart
a6c9bd0eb5 Bug 9978: Replace license header with the correct license (GPLv3+)
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>

http://bugs.koha-community.org/show_bug.cgi?id=9987

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2015-04-20 09:59:38 -03:00
Jonathan Druart
e20270fec4 Bug 11944: use CGI( -utf8 ) everywhere
Signed-off-by: Paola Rossi <paola.rossi@cineca.it>
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Dobrica Pavlinusic <dpavlin@rot13.org>

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2015-01-13 13:07:21 -03:00
afd2418d73 Bug 11349: Change .tmpl -> .tt in scripts using templates
Since we switched to Template Toolkit we don't need to stick with the
sufix we used for HTML::Template::Pro.

This patch changes the occurences of '.tmpl' in favour of '.tt'.

To test:
- Apply the patch
- Install koha, and verify that every page can be accesed

Regards
To+

P.S. a followup will remove the glue code.

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2014-07-17 11:05:49 -03:00
0713f3bf03 Bug 10558 [QA Follow-up]
This patch addresses a number of issues with the main patch:

- regression on bug 2060 (i.e., displaying authority import batches
  correctly)
- regression on bug 10170 (translation of import record states)A
- use of datatables.inc
- lack of clarity as to the licensing of tools/batch_records_ajax.pl
- insufficent sanitizing of input used to generate an SQL statement

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-03-10 15:48:32 +00:00
4d32421634 Bug 10558 - Convert records table in manage-marc-import.pl to Ajax DataTable
Some libraries would like to sort by columns for the records of an
import batch. This seems like a good use of Ajax DataTables.

Test plan:
1) Apply this patch
2) Import a record batch into Koha
   a) Use some form of matching
   b) Have some records that will match and some that won't
   c) Have at least 30 records so you can test the pager
3) Verify the new table is functionally equivalent to the old static one

Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Tests fine and looks good with the exception of the corrections I put in
a follow-up.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-03-10 15:46:49 +00:00
Elliott Davis
1dba9c6409 Bug 7131: teach MARC import how to overlay items
When staging biblios with items attached you previously had only two
options, add or don't add.

This patch adds a third option to replace an item record if a match is
found on itemnumber or barcode, else it adds the item.

Test Plan:
1) Stage a file of biblios with items attached.
2) Import the batch into the catalog.
3) Run the indexer so the matcher will match
4) Modify the item data for at least one bib in the file
5) Re-stage the file with the item matching option set to "Replace
   items if matching bib was found"
6) Let the indexer run again
7) You should see updated item information after the overlay

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Henry Bankhead <hbankhead@losgatosca.gov>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-30 04:31:07 +00:00
Jonathan Druart
7d93064906 Bug 10527: remove disused routine C4::Branch::get_branch_code_from_name
This routine is not in used and does not make sense. It should not be
used later.

Signed-off-by: Srdjan <srdjan@catalyst.net.nz>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Passes koha-qa.pl, not references to get_branch_code_from_name found.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-07-12 20:20:20 +00:00
Jared Camins-Esakov
a278407a47 Bug 10230: show correct matching record
The correct matching record was not always shown on the manage staged
MARC page. This patch corrects the issue and provides the template with
the appropriate ID for the matched record regardless what type of record
it is.

To test:
1) Create a matching record for authorities. For MARC21, the following
   is a good choice:
   Matching rule code:  AUTHPER
   Description:         Personal name main entry
   Match threshold:     999
   Record type:         Authority record
   [Match point 1:]
    Search index:   mainmainentry
    Score:          1000
    Tag:            100
    Subfields:      a
2) Create a record that has the appropriate fields for being matched
   with that rule (if you don't already have one).
3) Save the authority record as MARC (Unicode/UTF-8).
4) Stage the file choosing your new matching rule.
5) Note that with these patches, you get a link to the existing
   authority and without them you could get any number of strange
   things.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script. Additional tests done:
- staged bibliographic records and matched with different
  rules and actions. Undid the import.
- staged autohrity recods and matched with the example rule.
  Undid the import.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-05-19 08:40:13 -04:00
Jared Camins-Esakov
65c0174e63 Bug 2060 follow-up: add support for UNIMARCAUTH
The staged MARC management script was not correctly informing
the decoder ring that we had UNIMARC authorities, and the decoder
ring was dutifully trying to turn the authority records into a
bibliographic box of cereal.
2012-09-19 17:16:23 +02:00
Jared Camins-Esakov
7ad5e203da Bug 2060: Update command line MARC import scripts
Expose authority import functionality to the command line import
scripts, and rename them from commit_biblios_file.pl and
stage_biblios_file.pl to commit_file.pl and stage_file.pl.

To test (note that these instructions assume you have a MARC21
installation and are using the provided sample file):
1. Find a file of authorities (a sample file with MARC21 authorities
   is attached to bug 7475) and download it to your server
2. Stage the file using the following command (replace <filename> with
   the name of the file you saved in step 1):
   > misc/stage_file.pl --file <filename> --authorities
3. Note the batch number the script assigns to your batch
4. Commit the records using the following command (replace <batchnumber>
   with the batch number you made note of in step 3):
   > misc/commit_file.pl --batch-number <batchnumber>
5. Index the authorities Zebraqueue (or wait)
6. Confirm that the new authorities appear.
7. Create a matching rule with the following settings:
   Code: AUTHTEST
   Description: Personal name main entry
   Match threshold: 999
   Record type: Authority record
   Search index: Heading-main
   Score: 1000
   Tag: 100
   Subfields: a
   Offset: 0
   Length: 0
   (note the ID of this matching rule)
8. Stage the authority file again, this time using the following
   command:
   > misc/stage_file.pl --file <filename> --authorities \
     --match <matchingrule>
7. Revert the import with the following command:
   > misc/commit_file.pl --batch-number <batchnumber> --revert
8. Index the authorities Zebraqueue (or wait)
9. Confirm that the records have been removed
10. Import an authority record with the Stage MARC/Manage staged MARC
    tools in exactly the way you would for a bibliographic record,
    but choose "Authority" instead of "Bibliographic" for the record
    type.

Signed-off-by: Elliott Davis <elliott@bywatersolutions.com>

Testing plan delivers as it should.

Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on latest master 11 September 2012
2012-09-19 17:16:18 +02:00
Jared Camins-Esakov
6e71b80ca3 Bug 7475: Teach matching rules to handle authorities
* Add the code necessary to handle authorities with matching rules and
  import batches.
* Update all the scripts that use the matcher and import batch code
  to use the new API.
* Add authority records to the matching rules interface in the staff
  client.

http://bugs.koha-community.org/show_bug.cgi?id=2060
Signed-off-by: Elliott Davis <elliott@bywatersolutions.com>

Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on latest master 11 September 2012
2012-09-19 17:15:56 +02:00
Dobrica Pavlinusic
130e3d9c10 Bug 8315 - remove use C4::* version
This patch touches a lot of code, but basically it removes version
information from use C4::* in our code.

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
All script still compiles after the patch (confirmed by pre-applypatch hook)
2012-07-13 14:17:20 +02:00
Srdjan
12ff7355bb bug_7613: OCLC Connexion gateway
svc/import_bib:
* takes POST request with parameters in url and MARC XML as DATA
* pushes MARC XML to an impoort bach queue of type 'webservice'
* returns status and imported record XML
* is a drop-in replacement for svc/new_bib

misc/cronjobs/import_webservice_batch.pl:
* a cron job for processing impoort bach queues of type 'webservice'
* batches can also be processed through the UI

misc/bin/connexion_import_daemon.pl:
* a daemon that listens for OCLC Connexion requests and is compliant
  with OCLC Gateway spec
* takes request with MARC XML
* takes import batch params from a config file and forwards the lot to
  svc/import_bib
* returns status

ImportBatches:
* Added new import batch type of 'webservice'
* Changed interface to AddImportBatch() - now it takes a hashref
* Replaced batch_type = 'batch' with
  batch_type IN ( 'batch', 'webservice' ) in some SELECTs

Signed-off-by: MJ Ray <mjr@phonecoop.coop>
2012-04-06 17:26:20 +02:00
Dobrica Pavlinusic
8d7d8af618 Bug 7790 - tools/manage-marc-import.pl breaks with plack
transfer $template to add_matcher_list for correct scoping
and score $dbh and $runinbackground with our

Signed-off-by: Matthias Meusburger <matthias.meusburger@biblibre.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-03-26 18:10:20 +02:00
Ian Walls
973371a68b Bug 6733: change branchcode call in Labels to use userenv instead of LoginBranchname
In the 5 places in the label editor, the user's current branch is determined by taking
the LoginBranchname and doing a reverse look up for the branchcode.  This is an unnecessary
query, since C4::Context->userenv has the immediate access to the branchcode; this is how
it's summoned in all other points in the Koha code.

This addresses bug 6673 because, since the move to T:T, the call in tools/manage-marc-import.pl
has used in correct syntax:  $template->param('LoginBranchname') instead of $template->{VARS}->param('LoginBranchname')

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Ian Walls <ian.walls@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
2011-09-28 20:29:39 +13:00
Chris Cormack
9573f444fc Merge remote-tracking branch 'kc/new/bug_5449' into kcmaster 2011-05-09 08:25:36 +12:00
b9cee99da4 Bug 5065 Add ability to choose framework on import
Signed-off-by: Nicole C. Engard <nengard@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
2011-03-21 20:36:35 +13:00
Julian Maurice
6e85ee6db0 Bug 5449: JSON malformed in Koha - Blocker with jQuery 1.4.x
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
2011-03-12 08:53:41 +13:00
Lars Wirzenius
49e0a8fc78 Fix FSF address in directory tools/
Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
2010-03-16 20:17:51 -04:00
0bbcfeda0e replace syslog with warns
As discussed with Chris Nighswonger on #koha, this patch
removes the calls to syslog and replaces them with warns
so that error messages generated by the labels code
are sent to the Apache error log.  This avoids splitting
this sort of logging across multiple files and is consistent
with current practice in most of the rest of Koha.

Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
2009-09-05 21:43:12 -04:00
Chris Nighswonger
02822ac805 [26/40] Refactoring auto label batch generation feature in bulk import tool and adding error trapping/handling. 2009-09-01 15:52:22 -04:00
ae5b76c89b bug 2157: add ability to 'clean' staged record batches
batches, it is now possible to 'clean' a batch by
removing all bib and item records staged in the batch.  This
has the effect of helping to reduce database space used
by old import batches as well as removing staged records
from the cataloging reservoir search.  Note that 'cleaning'
a batch affects only the copies of the records that were staged;
if the batch was committed, cleaning the batch does not
affect any bibs and items that were committed into the catalog.

Also note that once you clean a committed batch of records, it is
impossible to undo the previous commit operation.

Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
2009-08-10 20:34:55 -04:00
f202faba18 bug 2505: enable warnings in tools/manage-marc-import.pl
Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
2009-08-10 20:34:50 -04:00
Galen Charlton
7ce6822973 removed incorrect module import
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
2009-06-07 20:09:22 -05:00
Mason James
31adf4ef83 patch for feature 2952 - v3
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
2009-02-12 16:39:33 -06:00
Galen Charlton
57fb7e45b7 kohabug 2186 - improve display of empty import batches
[1] Display the "No records have been staged" message only
    if no import batches exist.
[2] When displaying a batch that has no bibs in it, display
    a "There are no records in this batch to import".  Also,
    do not display the "import record into catalogue" button
    for an empty batch.
[3] Add CSS class "problem" to both messages.

Documentation changes: minor; only required if screenshots
exist of the manage staged MARC records page that has no
import batches on it.

Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2008-05-31 21:59:03 -05:00
Galen Charlton
438ed23337 staging import - enhance record overlay behavior
Enhanced the ability of catalogers to specify how
bib and item records should be added, replaced, or
ignored during a staging import.

When an import batch of bib records is staged and commit,
the user can now explicitly specify what should occur
when an incoming bib record has, or does not have, a match
with a record already in the database.  The options are:

if match found (overlay_action):
  create_new (just add the incoming record)
  replace (replace the matched record with the incoming one)
  use_template (option not implemented)
  ignore (do nothing with the incoming bib; however, the
          items attached to it may still be processed
           based on the item action)

if no match is found (nomatch_action):
  create_new (just add the incoming record)
  ignore (do nothing with the incoming bib; in this
          case, any items attached to it will be
          ignored since there will be nothing to
          attach them to)

The following options for handling items embedded in the
bib record are now available:

  always_add (add the items to the new or replaced bib)
  add_only_if_match (add the items only if the incoming bib
                     matches an existing bib)
  add_only_if_add (add the items only if the incoming bib
                   does *not* match an existing bib)
  ignore (ignore the items entirely)

With these changes, it is now possible to support the following use cases:

[1] A library joining an existing Koha database wishes to add their
    items to existing bib records if they match, but does not want
    to overlay the bib records themselves.
[2] A library wants to load a file of records, but only handle
    the new ones, not ones that are already in the database.
[3] A library wants to load a file of records, but only
    handle the ones that match existing records (e.g., if
    the records are coming back from an authority control vendor).

Documentation changes:

* See description above; also, screenshots of the 'stage MARC records
for import' and 'manage staged MARC records' should be updated.

Test cases:

* Added test cases to exercise staging and committing import batches.

UI changes:

* The pages for staging and managing import batches now have
  controls for setting the overlay action, action if no match,
  and item action separately.
* in the manage import batch tool, user is notified when they
  change overlay action, no-match action, and item action
* HTML for manage import batch tool now uses fieldsets

Database changes (DB rev 076):

* added import_batches.item_action
* added import_batches.nomatch_action
* added 'ignore' as a valid value for import_batches.overlay_action
* added 'ignored' as a valid value for import_records.status
* added 'status' as a valid value for import_items.status

API changes:

* new accessor routines for C4::ImportBatch

    GetImportBatchNoMatchAction
    SetImportBatchNoMatchAction
    GetImportBatchItemAction
    SetImportBatchItemAction

* new internal functions for C4::ImportBatch to
  determine how a given bib and item are to be
  processed, based on overlay_action, nomatch_action,
  and item_action:

    _get_commit_action
    _get_revert_action

Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2008-04-30 21:58:46 -05:00
Galen Charlton
fc6ccb1a61 granular permissions - updated Tools
Updated all scripts appearing on the tools page
to respect a granular permission defined for
each of them.

The tools menu and home page have been changed so
that only the specific tools that a user has
access are displayed.  This is simple, but depending on
the module and circumstance, it may be better to
display functions that the user has does not have
access to, but disable the links and do some sort
of visual styling to indicate that a function exists
but requires additional privileges to access.

Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2008-04-10 15:53:30 -05:00
Ryan Higgins
46826aaee0 Mod C4::Labels::add_batch to accept an array of items.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2008-03-17 21:32:31 -05:00