Commit graph

79 commits

Author SHA1 Message Date
df97814f30 Bug 15758: Koha::Libraries - Remove GetBranches
Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-09-08 14:36:03 +00:00
0e9c3d446d Bug 15451: Do not considered a Resultset as a Result
The previous calls were wrong, but there is something bad with the DB
structure: export_format.profile should be a unique key.
This patch fixes the previous calls and add a FIXME not to forget to fix
the DB structure.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Previous test where done with all patches applied,
including this one, and all worked.
No errors

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-07-22 17:18:36 +00:00
f3d7f27107 Bug 15451: Koha::CsvProfiles - Remove the residue
This patch erase all traces of C4::Csv since it's not used anymore.
All occurrences have been replaced by previous patches to use
Koha::CsvProfiles.

Note that GetMarcFieldsForCsv was not used prior this patch set.

Test plan:
  git grep 'C4::Csv'
should not return any result.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
No more traces of the file.
This produces a koha-qa fail, due to the missing file.
No other errors

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-07-22 17:18:36 +00:00
bea95dace6 Bug 15451: Koha::CsvProfiles - Remove GetCsvProfileId
This subroutine returned the export_format_id for a given profile name.
This can be done easily with the Koha::CsvProfiles->search method.

Test plan:
Export records using the misc/export_records.pl script and the
export tool.
If you are exporting using the MARC format, the profile filled in the pref
ExportWithCsvProfile will be used (or the one passed in parameter of
misc/export_records.pl).
If you are exporting using the CSV format, you can choose a profile in
the dropdown list.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Exported using tool & cmd, marc & csv. Pref is used.
No errors

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-07-22 17:18:35 +00:00
04c7a8a845 Bug 15451: Koha::CsvProfiles - Remove GetCsvProfiles
This subroutine did the same job as GetCsvProfilesLoop, so this patch
applies the same changes as the previous patch.

Test plan:
1/ Claim some serials, sql profiles should be listed
2/ Export records using the export tool. MARC profiles should be listed.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Listed sql & marc profiles
No errors

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-07-22 17:18:35 +00:00
00960f1964 Bug 16859: Fix wrong item field name in export.pl
Field callnumber should be itemcallnumber.
For completeness, prefixing the joined fields with items table prefix.

Note: You should expect Tools/Export to export only the items within a
given itemcallnumber range, but if the biblio has one item in that
range, the biblionumber is selected for export with ALL items. The script
is designed that way (first select biblio numbers based on criteria, and
run export with those biblio numbers).

Test plan:
[1] Select a biblio N with one item with say itemcallnumber X.
[2] Goto Tools/Export. Select biblio range N,N and no itemcallnumber
    range. The biblio should be exported.
[3] Select biblio range N,N and itemcallnumber range A,B. No output.
[4] Select biblio range N,N and itemcallnumber range A,<empty>. Output.
[5] Select biblio range N,N and itemcallnumber range <empty>,X. Output.

Signed-off-by: Srdjan <srdjan@catalyst.net.nz>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-07-08 12:58:16 +00:00
a44bb07b88 Bug 16855 - Poor performance due to high overhead of SQL call in export.pl
This patch eliminates all columns retrieved in the biblioitems query
except for biblionumber.

To test:
1 - Go to tools->Export data
2 - Export using various filters and note you get expected records
3 - Apply patch
4 - Ensure exported results match results before patch

Signed-off-by: Srdjan <srdjan@catalyst.net.nz>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-07-08 12:54:27 +00:00
c133bedf54 Bug 16154: Fix some other occurrences
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
2016-04-26 23:16:44 +00:00
a159d2b067 Bug 16154: CGI->multi_param - Assign a list
This patch replaces the occurrences of
  @foo = $cgi->param('foo');
with
  @foo = $cgi->multi_param('foo');

perl -p -i -e
's/^(\s*@\w+\s*=\s*)\$(cgi|input|query)\->param\(/$1\$$2\->multi_param\(/xms'
**/*.pl

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
2016-04-26 23:16:43 +00:00
66aacace08 Bug 16154: CGI->multi_param - Declare a list
This patch replaces the occurrences of
  my @foo = $cgi->param('foo');
with
  my @foo = $cgi->multi_param('foo');

perl -p -i -e
's/^(\s*my\s*@\w+\s*=\s*)\$(cgi|input|query)\->param\(/$1\$$2\->multi_param\(/xms'
**/*.pl

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
2016-04-26 23:16:42 +00:00
24e502e4a9 Bug 15923: Take the id list file into account when exporting records
When exporting records (tools/export.pl or misc/export_records.pl), a
file of ids (authid or biblionumber) can be passed to filter the
results.
Bug 14722 has broken this behavior.

Test plan:
Export records and specify a list of records to filter the results.
Prior to this patch, the record with the id 1 was exported.

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Brendan Gallagher brendan@bywatersolutions.com
2016-03-02 00:43:11 +00:00
7283069a9d Bug 15381: Remove getauthtypes and getauthtype
These 2 subroutines are now not necessary anymore, we can use
Koha::Authority::Types to retrieve the authority types.

This patch won't be easy to test. Indeed, a lot of file are updated.
To test it I would suggest to focus on the merge authority

You should also confirm that the authtypecode and the authtypetext
values are correctly displayed when navigating in the authority module.

Signed-off-by: Frédéric Demians <f.demians@tamil.fr>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Brendan Gallagher brendan@bywatersolutions.com
2015-12-31 18:59:02 +00:00
03c89fa02e Bug 15380: Move Koha::Authority to Koha::MetadataRecord::Authority
The existing Koha::Authority does not use Koha::Objects and does not
exclusively deal with DB objects.
It makes much sense to move it to let the space free for a
Koha::Authority and Koha::Authorities modules based on Koha::Object.

Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Brendan Gallagher brendan@bywatersolutions.com
2015-12-31 18:50:41 +00:00
38f92df4e5 Bug 14722: Refactor the export tool
Why a refactoring was need for this script?
The export tool (tools/export.pl) can be called from the command line
and some parts of code were unnecessarity complicated (just look at the
code, you will understand).

Worse still, the script does not provide the same options for both
interface. For instance you cannot export records given a range of
biblionumbers, authids, callnumbers, etc. from the commandline.

What does this patch?
1/ Important: The script tools/export.pl does not work anymore if called from
the command line (should be in the release notes).
2/ The code used to generated a file (csv, iso2709 or xml) has been moved to a new
module (Koha::Exporter::Record) and tests have been provided.
3/ No change is done on the web interface
4/ Some new options have been added to the commandline script
(misc/export.pl):
    - starting_authid
    - ending_authid
    - authtype
    - starting_biblionumber
    - ending_biblionumber
    - itemtype
    - starting_callnumber
    - ending_callnumber
    - start_accession
    - end_accession
5/ There is a change in the behavior if an error occurs:
Can't call method "as_usmarc" on an undefined value at Koha/Exporter/Record.pm line 114.
record (number 5530) is invalid and therefore not exported because its reopening generates warnings above at Koha/Exporter/Record.pm line 117.

Before this patch, they were not displayed (using the command line).

What does not do this patch?
It does not provide the 'clean', 'timestamp' and 'deleted_barcodes' options to
the web interface (same as before).

What about the perfs?
With a DB with ~800 biblios (MARC21)
Before: perl tools/export.pl 14.79s user 0.83s system 71% cpu 21.905 total
After:  perl misc/export.pl  17.19s user 0.84s system 75% cpu 24.018 total

With a DB with ~6400 biblios (UNIMARC)
Before: perl tools/export.pl 26.55s user 0.76s system 76% cpu 35.498 total
After:  perl misc/export.pl  26.78s user 0.84s system 80% cpu 34.494 total

How to test this patch?
Test plan:
A. Web interface:
1/ On the current master, export some records, biblios and authorities (with
the 3 differents exports) playing with the different filters (item type,
libraries, callnumber, accession date, don't export items, remove
non-local items, don't export fields, etc.).
2/ Apply this patch, export again the same records, and compare the
generated files. They must be identical!
3/ Confirm that the export features on the checkout list
(circ/circulation.pl) works as before this patch.

B. The command line
1/ On the current master, export some records, biblios and authorities (with
the 2 differents exports) playing with the different options (date,
deleted_barcodes, clean).
2/ Apply this patch, export again the same records, and compare the
generated files. They must be identical!

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2015-10-27 17:01:28 -03:00
30667f0555 Bug 14100: (follow-up) Language overlay for item types
Display the translated description for item types in the following pages:
> admin/smart-rules.pl
> catalogue/detail.pl
> catalogue/itemsearch.pl
> catalogue/moredetail.pl
> reports/acquisitions_stats.pl
> reports/bor_issues_top.pl
> reports/cat_issues_top.pl
> reports/catalogue_out.pl
> reports/catalogue_stats.pl
> reports/issues_avg_stats.pl
> reports/issues_stats.pl
> reports/itemslost.pl
> reports/manager.pl
> reports/reserves_stats.pl
> suggestion/suggestion.pl
> tools/export.pl

> Opac:
> opac-detail.pl
> opac-MARCdetail.pl
> opac-search.pl

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2015-10-27 12:34:06 -03:00
Marc Véron
33e679b905 Bug 14997: Remove C4::Dates from tools (import / export)
This patch removes C4::Dates from:

- tools/export.pl
- tools/import_borrowers.pl

Note: For testing, both need preparation without patch, see below.

To test export:
- Without patch applied, go to
  Home > Tools > Export data > Export bibliographic records
- Define Start date / End date for Accession dates
- Export bibliographic records as 'without-patch.mrc'

- Do the same with patch, export as 'with-patch.mrc
- Compare the files, they should be the same

To test Import patrons:
- Without patch
- Go to Home > Tools > Import patrons
- Create a patron category like 'TEST' (useful for filtering...)
- Prepare a file with some patrons with category TEST to import.
  Fill date of birth, enrolment date, expiry date with values
  formatted in syspref format, in iso format and garbage
- Import
- Review the imported patrons (search for category TEST)

- With patch: Change cardnumber and names in import file
- Import
- Review again and compare with results from previous import.

Signed-off-by: Mirko Tietgen <mirko@abunchofthings.net>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2015-10-27 10:10:18 -03:00
6ad59e7ab4 Bug 14709: Do not access to C4::Context->userenv from the commandline
The script dies with
  Can't use an undefined value as a HASH reference at tools/export.pl
line 149.
if it is called from the command line.

This is introduced by bug 13040.

The C4::Context->userenv is not defined in this case.

Test plan:
Execute the script using the command line.
With the patch applies you should not get the error.

Signed-off-by: Joonas Kylmälä <j.kylmala@gmail.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>
2015-09-02 10:02:33 -03:00
dfe07fa410 Bug 14592: Export data bad encoding
Tools > Export data produces a file badly encoded, when marc/xml file
format is chosen. It works with CSV.

To test:

1. Tools > Export data
2. Choose a limited interval of biblionumber
3. Export in marc/xml/csv.
4. Check files exported at 3: csv file is ok. marc/xml files are
   corrupted, ie badly encoded.
5. Apply the patch
6. Repeat steps 3&4, and confirm that all file formats are valid.

Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2015-08-17 10:49:02 -03:00
Jonathan Druart
fcef1ea273 Bug 12404: tools/export.pl allows CSV export
Signed-off-by: Courret <scourret@gmail.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2015-04-27 10:43:23 -03:00
Jonathan Druart
ba0f84b46c Bug 9978: (followup) Replace license header with the correct license (GPLv3+)
There was another form of the v2.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2015-04-20 09:59:43 -03:00
Jonathan Druart
e2968fe081 Bug 12860: the export tools explodes if a field is not exported
To reproduce:
1/ Go on tools/export.pl
2/ Export some records and specify you don't want the fields 245 (the
whole field, do not specify a subfield).
3/ Export, BOOM
Software error:
Arguments must be MARC::Field object at /home/koha/src/tools/export.pl line 400.

Test plan:
Apply the patch and confirm the fields 245 are not exported and the
export works as expected.

Signed-off-by: Robin Sheat <robin@catalyst.net.nz>

Export fails to asplode with this patch

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2015-02-05 15:10:00 -03:00
Robin Sheat
e1eb47dede Bug 13040 - multiple branch selections in exporter
This allows the exporter (Tools -> Export) to have any combination of
branches selected, rather than it being all or only one.

Test Plan:
* Apply the patch
* Go to the exporter, see that instead of a dropdown you now have an
  elegently laid out grid of branches you can select from
* Select some branches, run the export
* Note that only records with items in the selected branches are
  returned.
* Repeat this with the item related options (as that code was refactored
  slightly) and make sure everything is sane.

Sponsored-By: South Taranaki District Libraries
Signed-off-by: Thomas <tomsStudy@gmail.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2015-01-26 16:22:07 -03:00
Jonathan Druart
e20270fec4 Bug 11944: use CGI( -utf8 ) everywhere
Signed-off-by: Paola Rossi <paola.rossi@cineca.it>
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Dobrica Pavlinusic <dpavlin@rot13.org>

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2015-01-13 13:07:21 -03:00
Jonathan Druart
9c71bf4227 Bug 13308: The output format should be 'iso2709' instead of 'marc'
In the template of the export tools, the value 'marc' is set for
the 'format' variable.
It should be 'iso2709'.

There was a mess in the tools/export.pl script.
2 variables did the same thing: $format and $output_format

This patch fixes that replacing $format with $output_format

Test plan:
Try to export biblio in iso2709 and csv using the export tools and from
the checkout list.

Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Before applying the patch, attemping an MARC export from tools/export.pl
resulted in an error. After the patch the export works correctly.
Exports from the checkouts page also work correctly.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes tests and QA script. Also tested authority export.

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2014-11-20 18:10:44 -03:00
Stéphane Delaune
59b7744e1c Bug 13082: fix to prevent adding of invalid records in marc file
Test:

1. Edit record, add 100.000 chars text to 500a

2. xml export produce the record,

3. mrc export do not produce the record, warning on log export.pl:
   Record length of 111000 is larger than the MARC spec allows (99999
   bytes). at /usr/share/perl5/MARC/File/USMARC.pm line 314.  record
   (number 139489) is invalid and therefore not exported because its
   reopening generates warnings above at...

Signed-off-by: Frederic Demians <f.demians@tamil.fr>

I confirm that exporting biblio records larger than 10000 characters in
ISO2709 produces invalid files. After applying this patch, the culprit
record (too large, but also other inconsistencies preventing record
parsing with MARC::File::USMARC) is not exported anymore. A warning is
produced in Koha Apache log file. Warnings to the user on WUI would be
better, but it isn't the case yet, so it isn't a regression.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>

I agree that a visible warning/result message in the staff interface
would be nice, but this works as described.

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2014-11-14 11:47:00 -03:00
Jonathan Druart
47e2331bd0 Bug 13190: Reintroduce the checkouts export feature
Bug 11703 breaks the checkouts export feature.

To reproduce: Fill the ExportWithCsvProfile pref and go on the
circ/circulation.pl page. The export column appears, but not the export
button.

Test plan:
Go on the checkout list (circ/circulation.pl and members/moremember.pl)
and verify the export column and the export button appears.
If you click on the button, a file should be generated.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2014-11-06 10:00:30 -03:00
Stéphane Delaune
48b7e1ecef Bug 13085: bugfix to allow call of export.pl in commandline without id_list_file parameter
Test plan
1/ run ./tools/export.pl
2/ notice an error like cannot open 0: No such file or directory at ./tools/export.pl line 209.
3/ apply patch
4/ run again
5/ no errors

Signed-off-by: Chris <chris@bigballofwax.co.nz>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as described, adding an additional check.

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2014-10-22 09:31:30 -03:00
afd2418d73 Bug 11349: Change .tmpl -> .tt in scripts using templates
Since we switched to Template Toolkit we don't need to stick with the
sufix we used for HTML::Template::Pro.

This patch changes the occurences of '.tmpl' in favour of '.tt'.

To test:
- Apply the patch
- Install koha, and verify that every page can be accesed

Regards
To+

P.S. a followup will remove the glue code.

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2014-07-17 11:05:49 -03:00
Galen Charlton
96b60b5140 Bug 12238: (follow-up) tweak descriptions of the ID list
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-05-23 15:53:21 +00:00
e3e38d6902 Bug 12238: (follow-up) PATH does not need to be absolute
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-05-23 15:50:23 +00:00
Stéphane Delaune
03f420e6f5 Bug 12238: Add option to supply list of record IDs to MARC export tool
The MARC export tool now includes the option to specify a file
containing record IDs to export.  When run on the command line,
this is specified using the --id_list_file switch.

Note that the list of IDs acts as a filter on other criteria
for selecting records to import.  For example, if you export
all bibs belong to a given library and also specify an ID file,
the bibs must both belong to the library and be in the ID file.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Work as described. No errors

Tested writing txt file with biblionumbers, loading in tools and exporting

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-05-23 15:46:34 +00:00
Stéphane Delaune
e524ab07c8 Bug 12229: when exporting MARC records (as ISO2709), refuse to output records that would be longer than the MARC limit
Test:
1. Edit a bib record, add 100.000 chars text to 500a
2. Go to Tools -> Export data (or run tools/export.pl on
   the command line)
3. Verify that exporting as XML includes the record
4. Verify that exporting as MARC does not include the record,
   and that you get the following warning:

export.pl: record (number 2025) length 102668 is larger than the MARC spec allows (99999 bytes) at...

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Work as described. No errors

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-05-23 15:32:27 +00:00
fb4025b67b Bug 10277 - Add C4::Context->IsSuperLibrarian()
The method of checking the logged in user for superlibrarian privileges
is obtuse ( $userenv && $userenv->{flags} % 2 != 1 ) to say the least.
The codebase is littered with these lines, with no explanation given. It
would be much better if we had one subroutine that returned a boolean
value to tell us if the logged in user is a superlibrarian or not.

Test Plan:
1) Apply this patch
2) Verify superlibrarian behavior remains unchanged

Signed-off-by: Joel Sasse <jsasse@plumcreeklibrary.net>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Comments on second patch.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-12-30 15:47:23 +00:00
Colin Campbell
248283fc29 Bug 10996: Allow numeric subfields to be stripped on export
Regex assumed all numeric characters were part of the tag number
This is obviously false as $9 would be a likely candidate
to be removed on export.

The code can be any printing ascii character other than space
according to LoC's website.

Also changed regexp to no longer allow a zero length tag number
which is nonsensical.

The old regex would accept shorter than 3 digit tags but these were
not stripped so I've removed that option, considering it a bug
not a feature

NB assumption that the code makes that a tag is always numeric is
incorrect but works in practice. Handling non-numeric tags is
a 'To be done'

Made code dependent on the regex succeeding. Picking up results
from a previous regex on failure can lead to weird hard to
identify bugs

In the course of testing Paola Rossi <paola.rossi@cineca.it>
spotted that the delete operation was flawed, only removing
the first occurence of the specified tag/tag subfield. Reworked
the delete loop to operate on all occurences

Signed-off-by: Paola Rossi <paola.rossi@cineca.it>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-11-21 17:53:02 +00:00
Jesse Weaver
5cf5da12ef Bug 9300 - filtering Export (MARC) data by accession date does not work
Small typo in query construction, fixed.

To test:
1) Try filtering a MARC export using "Start date"
2) Notice the lack of filtering
3) Apply patch
4) Try export again, records are filtered by dateaccessioned

Signed-off-by: Cedric Vita <cedric.vita@dracenie.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-06-07 08:41:31 -07:00
b00ec06968 Bug 10080 - Change system pref IndependantBranches to IndependentBranches
Test Plan:
1) Enable IndependantBranches
2) Apply this patch
3) Run updatedatabase.pl
4) Verify that the system preference still functions correctly

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-05-22 07:58:23 -07:00
Fridolyn SOMERS
24eac82456 Bug 9415: XML catalog export is missing root node
When exporting some biblio or authorities records with tools/export.pl (via web client or command-line) :
If choosing XML format, you get a concatenation of full XML records.

This patch uses MARC::File::XML to create a well formated file.
See http://search.cpan.org/~gmcharlt/MARC-XML-0.93/lib/MARC/File/XML.pm#close%28%29

Test plan:
- Go to Tools/Export data
- Enter numbers in from and to biblio number
  (make sure that at least two records will
   be exported).
- Select xml in file format
- Click "Export bibliographic records"
- Save file somewhere
- Look at downloaded file
=> File should look like :
   <?xml version="1.0" encoding="UTF-8"?>
   <collection
   ...

   <record>
   ...
   </record>

   <record>
   ...
   </record>

   </collection>
=> "collection" is the root node and XML declaration exists only once
- Do the same for authorities export and command-line use of tools/export.pl

Second test plan:

- From the command line, run tools/export.pl, e.g.,

  tools/export.pl  --format=xml --filename=bibs.xml

- Verify that the output is valid XML, e.g.,

  xmllint --noout bibs.xml # if the file is valid, no error messages will be displayed

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-03-19 22:07:02 -04:00
Jonathan Druart
a469663d7b Bug 9108: Followup: send the dateformat value from C4::Auth
- the dateformat value is send to all templates (from
  C4::Auth::get_template_and_user)
- remove all assignment of dateformat in all .pl files
- the DHTMLcalendar_dateformat variable is unused

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Fixed conflicts:
 - opac/sco/sco-main.pl
 - reports/acquisitions_stats.pl
 - tools/cleanborrowers.pl

All tests pass, perlcritic problems appeared in some files
before and after these patches were applied.

Checked sorting in following pages:
- acqui/addorderiso2709.tt - list of staged imports in acq
- acqui/histsearch.tt - sorting of dates in acq search result list
- acqui/invoices.tt - billing date in list of invoices in acq
- acqui/lateorders.tt - list of late orders in acq
- acqui/ordered.tt - ordered titles and estimated costs for a fund
- acqui/parcels.tt - receive shipment page
- acqui/spent.tt - received titles and actual costs for a fund
...
- serials-search.tt - subscription search result list
...
- opac/sco/sco-main.tt - due dates in list of checked out items
- reports/acquisitions-stats.tt - date searches, display of dates
- tools/cleanborrowers.tt
- tools.holidays.tt - different views of dates library is closed,
  adding dates

Checked dates display according to system preference everywhere and
searching, entering dates etc. still worked as expected.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-01-17 21:59:30 -05:00
86fa020ef8 Bug 7368: Update GetXmlBiblio documentation
Only changing some documentation about GetXmlBiblio

Signed-off-by: Mirko Tietgen <mirko@abunchofthings.net>
Added the word 'contain'

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
2012-12-27 10:14:19 -05:00
caec1ddfa4 Bug 8843 - Cannot export bibliographic records by call number
When building the query for exports by call number the script
has the logic reversed. It tries to select records with call
numbers less than the starting call number and greater than
the ending call number. This should be reversed.

To test, test an export in an unpatched system. An export by
call number will return an empty file. After applying the patch
an export by call number should give valid results.

Signed-off-by: Nicole C. Engard <nengard@bywatersolutions.com>
Works as expected.

Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-10-12 16:38:50 +02:00
Jonathan Druart
0f3f61e756 Bug 7986: Export issues for patron
In the circulation page, you can now export (as csv or iso2709) a list
of items which are currently checked out by a borrower.

3 export types:
- iso2709 with items: Export the items list in iso2709 format with item
  informations.
- iso2709 without items: Export the items list in iso2709 format without
  item informations.
- CSV: Export the items list based on a csv profil.

2 new system preferences:
- DontExportFields: a list of fields not to be export
- CsvProfileForExport: The Csv profile name used for the csv export

Test plan:
- Fill the CsvProfileForExport syspref
- go on the borrower circulation page containing checkouts
- Select one or more items and export them to the 3 different formats.
- check if the result file is what you expected

- Test there is no regression with the export authority
- Test there is no regression using tools/export.pl with the command
  line interface

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-09-13 17:58:30 +02:00
Paul Poulain
913aba1a66 Merge remote-tracking branch 'origin/new/bug_8268' 2012-08-28 18:03:34 +02:00
Jared Camins-Esakov
20ad69646f Bug 8268: improve error checking
Valid download attempts were being denied thanks to an incorrect
regular expression. This patch fixes that, and makes it easier to
understand what's going on in the code.

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-08-28 18:02:46 +02:00
Julian Maurice
322a9c7fbb Bug 5600: very tiny follow-up to pass perlcritic
open was called with 2 arguments instead of 3

Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-08-03 17:02:35 +02:00
Julian Maurice
c5e6fb7227 Bug 5600 follow-up: Fix commandline check in order to work under Plack
With plackup, @ARGV can contains elements so we can't check the size of
@ARGV to tell whether we are in command line mode or not.
Instead we check environment variable GATEWAY_INTERFACE.

Also fix the use of a global variable in a subroutine

Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-08-03 16:57:14 +02:00
Julian Maurice
4dea59847a Bug 5600: Command line interface for tools/export.pl
export.pl [--format=format] [--date=date] [--dont_export_items]
  [--deleted_barcodes] [--clean] --filename=outputfile

    * format is either 'xml' or 'marc' (default)
    * date should be entered as the 'dateformat' syspref is set
      (dd/mm/yyyy for metric, yyyy-mm-dd for iso, mm/dd/yyyy for us)
    * records exported are the ones that have been modified since 'date'
    * if --deleted_barcodes is used, a list of barcodes of items deleted
      since 'date' is produced (or from all deleted items if no date is
      specified)
    * --clean removes NSE/NSB

Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-08-03 16:57:08 +02:00
Jared Camins-Esakov
8affddc52d Bug 8268 follow-up: incorporate QA comments
Fixes the following things:
1. Sanitizes log output to prevent an attacker from using a specially
   crafted POST to add extra lines to the log
2. Simplify a regular expression since "..file" cannot be used to
   escape the current directory
3. Makes sure directories are consistent
4. Correct logic issues in misc/cronjobs/backup.sh

Thanks to Frere Sebastien Marie for catching these issues.

Signed-off-by: Robin Sheat <robin@catalyst.net.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-07-12 17:40:22 +02:00
Jared Camins-Esakov
bbcb1d784b Bug 8268: Add database dump to export tool
This patch builds on work by Lars Wirzenius for the Koha packages.

To date, the only way for a Koha librarian to obtain a complete backup
of their system has been to log into the system via SSH (or FTP) to
download the mysqldump file. This patch makes it possible for
superlibrarians in properly configured systems to download night backups
via the staff client's Export tool.

Recognizing that this is functionality with potentially very grave
security implications, system administrators must manually enable these
features in the koha-conf.xml configuration file.

The following configuration settings have been added to the koha-conf.xml
file:
* backupdir => directory where backups should be stored.
* backup_db_via_tools => whether to allow superlibrarians to download
  database backups via the Export tool. The default is disabled, and
  there is no way -- by design -- to enable this option without manually
  editing koha-conf.xml.
* backup_conf_via_tools => whether to allow superlibrarians to download
  configuration backups via the Export tool (this may be applicable to
  packages only). The default is disabled, and there is no way -- by
  design -- to enable this option without manually editing koha-conf.xml.

This commit modifies the following scripts to make use of the new
backupdir configuration option:
* koha-dump and koha-run-backups in the Debian packages
* The sample backup script misc/cronjobs/backup.sh

Note that for security reasons, superlibrarians will not be allowed
to download files that are not owned by the web server's effective user.
This imposes a de facto dependency on ITK (for Apache) or running the
web server as the Koha user (as is done with Plack).

To test:
1. Apply patch.
2. Go to export page as a superlibrarian. Notice that no additional
   export options appear because they have not been enabled.
3. Add <backupdir>$KOHADEV/var/spool</backup> to the <config> section
   of your koha-conf.xml (note that you will need to adjust that so that
   it is pointing at a logical directory).
4. Create the aforementioned directory.
5. Go to export page as a superlibrarian. Notice that no additional
   export options appear because they have not been enabled.
6. Add <backup_db_via_tools>1</backup_db_via_tools> to the <config>
   section of your koha-conf.xml
7. Go to the export page as a superlibrarian. Notice the new tab.
8. Go to the export page as a non-superlibrarian. Notice there is no
   new tab.
9. Run: mysqldump -u koha -p koha | gzip > $BACKUPDIR/backup.sql.gz
   (substituting appropriate user, password, and database name)
10. Go to the export page as a superlibrarian, and look at the "Export
    database" tab. If you are running the web server as your Koha user,
    and ran the above command as your Koha user, you should now see the
    file listed as an option for download.
11. If you *did* see the file listed, change the ownership to something
    else: sudo chown root:root $BACKUPDIR/backup.sql.gz
11a. Confirm that you no longer see the file listed when you look at the
     "Export database" tab.
12. Change the ownership on the file to your web server (or Koha) user:
    sudo chown www-data:www-data backup.sql.gz
13. Go to the export page as a superlibrarian, and look at the "Export
    database" tab. You should now see backup.sql.gz listed.
14. Choose to download backup.sql.gz
15. Confirm that the downloaded file is what you were expecting.

If you are interested, you can repeat the above steps but replace
<backup_db_via_tools> with <backup_conf_via_tools>, and instead of
creating an sql file, create a tar file.

To test packaging: run koha-dump, confirm that it still creates a
usable backup.

------

This signoff contains two changes:

10-1. If no backup/conf files were present, then the message telling you
so doesn't appear and the download button does. Made them behave
correctly.
10-2. The test for a file existing required it to be owned by the
webserver UID. This change makes it so it only has to be readable.

Signed-off-by: Robin Sheat <robin@catalyst.net.nz>
2012-07-12 17:40:21 +02:00
Paul Poulain
74f6e97396 Bug 8202 follow-up UNIMARC authority support
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2012-07-04 16:21:07 +02:00
Jared Camins-Esakov
986b887159 Bug 8202: Add authority export to tools/export.pl
Adds a separate tab on tools/export.pl for exporting authority records.

To test:
1. Try exporting authority records from the "Export data" tool

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Tried different exports and everything worked nicely:
- removing fields from export (example: 942)
- limiting to one authority type
- changing the given file name
- exporting as MARC or XML
2012-07-04 16:21:06 +02:00