On bug 17591 we discovered that there was something weird going on with
the way we export and use subroutines/modules.
This patch tries to standardize our EXPORT to use EXPORT_OK only.
That way we will need to explicitely define the subroutine we want to
use from a module.
This patch is a squashed version of:
Bug 17600: After export.pl
Bug 17600: After perlimport
Bug 17600: Manual changes
Bug 17600: Other manual changes after second perlimports run
Bug 17600: Fix tests
And a lot of other manual changes.
export.pl is a dirty script that can be found on bug 17600.
"perlimport" is:
git clone https://github.com/oalders/App-perlimports.git
cd App-perlimports/
cpanm --installdeps .
export PERL5LIB="$PERL5LIB:/kohadevbox/koha/App-perlimports/lib"
find . \( -name "*.pl" -o -name "*.pm" \) -exec perl App-perlimports/script/perlimports --inplace-edit --no-preserve-unused --filename {} \;
The ideas of this patch are to:
* use EXPORT_OK instead of EXPORT
* perltidy the EXPORT_OK list
* remove '&' before the subroutine names
* remove some uneeded use statements
* explicitely import the subroutines we need within the controllers or
modules
Note that the private subroutines (starting with _) should not be
exported (and not used from outside of the module except from tests).
EXPORT vs EXPORT_OK (from
https://www.thegeekstuff.com/2010/06/perl-exporter-examples/)
"""
Export allows to export the functions and variables of modules to user’s namespace using the standard import method. This way, we don’t need to create the objects for the modules to access it’s members.
@EXPORT and @EXPORT_OK are the two main variables used during export operation.
@EXPORT contains list of symbols (subroutines and variables) of the module to be exported into the caller namespace.
@EXPORT_OK does export of symbols on demand basis.
"""
If this patch caused a conflict with a patch you wrote prior to its
push:
* Make sure you are not reintroducing a "use" statement that has been
removed
* "$subroutine" is not exported by the C4::$MODULE module
means that you need to add the subroutine to the @EXPORT_OK list
* Bareword "$subroutine" not allowed while "strict subs"
means that you didn't imported the subroutine from the module:
- use $MODULE qw( $subroutine list );
You can also use the fully qualified namespace: C4::$MODULE::$subroutine
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
There is a "debug" parameter we are passing from the controller scripts
to C4::Auth::get_template_and_user, but it's not actually used!
Test plan:
Confirm the assumption
Review the changes from this patch
Generated with:
perl -p -i -e 's#\s*debug\s*=\>\s*(0|1),?\s*##gms' **/*.pl
git checkout misc/devel/update_dbix_class_files.pl # Wrong catch
+ Manual fix in acqui/neworderempty.pl
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
It defaults to 0 in get_template_and_user
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch adds a .perlcriticrc (copied from qa-test-tools) and fixes
almost all perlcrictic violations according to this .perlcriticrc
The remaining violations are silenced out by appending a '## no critic'
to the offending lines. They can still be seen by using the --force
option of perlcritic
This patch also modify t/00-testcritic.t to check all Perl files using
the new .perlcriticrc.
I'm not sure if this test script is still useful as it is now equivalent
to `perlcritic --quiet .` and it looks like it is much slower
(approximatively 5 times slower on my machine)
Test plan:
1. Run `perlcritic --quiet .` from the root directory. It should output
nothing
2. Run `perlcritic --quiet --force .`. It should output 7 errors (6
StringyEval, 1 BarewordFileHandles)
3. Run `TEST_QA=1 prove t/00-testcritic.t`
4. Read the patch. Check that all changes make sense and do not
introduce undesired behaviour
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch adds the necessary changes to the script so that the filename
extensions works for exporting authorities.
To test:
1) Attempt to export with all filetypes for biblios and authorities
2) Confirm the extension for the exported file (after clicking the
Export button) is correct for each one
Sponsored-by: Catalyst IT
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
We certainly faced 3 similar bugs due to this syntax: bug 23006, bug
22941 and bug 17526.
To prevent other issues related to this syntax this patch suggests to
replace them all in one go.
Test plan:
Confirm that the 2 syntaxes are similar
Eyeball the patch and confirm that there is no typo!
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
It turns out the modules had the option expected built in, we just didn't call
it. This patch set expands the options for passing to the export.
To test:
1 - Go to Tools->Export data
2 - Perform an export and check 'Remove non-local items'
3 - Note that file contains only 'local' records, but includes all items
on those records
4 - Apply patch
5 - Note Tools->Export data has a new option to remove records not owned
by logged in branch
6 - Export as before checking records option, file should be as before
7 - Now check 'Remove items not owned by logged in branch'
8 - File should now only have local items (may have empty records)
9 - Check both boxes and recieve only 'local' records and items
Signed-off-by: Scott Kehoe <scott@masslibsystem.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
To test:
1 - Save a csv of biblionumber from MS/Excel
2 - Attempt to export records using this file
3 - It fails (mimetype is appliction/vnd.ms-excel)
4 - Apply patch
5 - Try again
6 - It succeeds!
Signed-off-by: Jason Palmer <jpalmer@switchinc.org>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Lari Taskula <lari.taskula@jns.fi>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
The way the export options are displayed at the bottom of the checkouts table
was not consistent.
Prior to this patch set, they are display if ExportRemoveFields or
ExportWithCsvProfile is set.
It does not make any sense, the user could want to export the checkouts in
iso2709 format without having to define a csv profile and fill the pref.
Moreover the behavior of this pref did not match its description: it's used as
a default CSV profile when exporting records from the export tools or the
command line.
This patch set adds a new pref ExportCircHistory and remove
ExportWithCsvProfile. The new pref is set if ExportWithCsvProfile or
ExportRemoveFields were set.
A new dropdown list with the CSV profile list will be displayed in the
export area, at the bottom of the checkouts table.
Note that now --csv_profile_id is mandatory for the export command line
(misc/export_records.pl) if the export format is csv.
Test plan:
0/ Do not execute the DB entry
1/ Clear both ExportWithCsvProfile and ExportRemoveFields prefs
2/ Execute the DB entry
3/ ExportCircHistory should not be set and the export options should not
be displayed at the bottom of the checkouts table.
4/ Remove the pref
DELETE FROM systempreferences WHERE variable='ExportCircHistory';
and reinsert the previous one, with a value:
INSERT INTO systempreferences (variable, value) VALUES
('ExportWithCsvProfile', 'something');
Execute the DB entry again
=> The now pref should be now set
5/ Export some checkouts using the CSV entry
6/ Note that the export tool and commandline script still work using the
csv format. You have to provide a --csv_profile_id option to make it
work.
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Bug 18087 breaks export unless a file is supplied.
Can't use an undefined value as a HASH reference at
/home/vagrant/kohaclone/tools/export.pl line 75.
Test plan:
Export records using a file of id that is not a valid file (not txt or
csv)
Export records using a valid file
Export records without supplying a file
=> The export should work or fail as expected.
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
If an invalid file is used as biblionumber list, we should display a
message.
Test plan:
1/ Use csv, plain text files
=> Should work
2/ Use invalid files (binary files like pdf, doc*, xsl*, etc.)
=> Should not work and see a warning message.
Amended patch after signoff: Remove one warn debug line
Signed-off-by: Joy Nelson <joy@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
In checkouts table, the is an export form (when some exports syspref are enabled).
Export using a CSV profile will create a file with name koha.mrc (same as ISO2709 export).
It would be better with koha.csv.
Bug 14647 manages the export page, this but will only manage for export from checkouts table where file name is hard-coded.
Test plan :
- Enable checkouts exports by setting syspref ExportWithCsvProfile with a profile for record export
- Go to circ page of a patron with checkouts : /cgi-bin/koha/circ/circulation.pl?borrowernumber=xxx
- Show checkouts table
- Select some checkboxes in "Export" column
- Select "CSV" in export format combo-box
- Click on "Export"
=> Without patch, the generated file is koha.mrc
=> With patch, the generated file is koha.csv
- Check ISO2709 export generates a file named koha.mrc
Signed-off-by: Dani Elder <dani@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
In Tools, Export data, you can export with a filter on item type.
When item type is on biblio record, you get the error :
export.pl: DBIx::Class::ResultSet::next(): Unknown column 'biblioitems.itemtype' in 'where clause' at /home/koha/src/Koha/Objects.pm line 150
Looks like its because this code use to be a SQL Select and is now a DBIx call :
C4::Context->preference('item-level_itypes')
? ( 'items.itype' => $itemtype )
: ( 'me.itemtype' => $itemtype )
This patch corrects by using "me.itemtype"
Test plan :
- Use syspref item-level_itypes = biblio record
- Go to Tools > Export data : /cgi-bin/koha/tools/export.pl
- Select an item type
- Click on "Export biblio records"
=> Without patch you get an error 500
=> With patch you get a correct export file
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Fix the issue, no errors.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
The previous calls were wrong, but there is something bad with the DB
structure: export_format.profile should be a unique key.
This patch fixes the previous calls and add a FIXME not to forget to fix
the DB structure.
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Previous test where done with all patches applied,
including this one, and all worked.
No errors
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This patch erase all traces of C4::Csv since it's not used anymore.
All occurrences have been replaced by previous patches to use
Koha::CsvProfiles.
Note that GetMarcFieldsForCsv was not used prior this patch set.
Test plan:
git grep 'C4::Csv'
should not return any result.
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
No more traces of the file.
This produces a koha-qa fail, due to the missing file.
No other errors
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This subroutine returned the export_format_id for a given profile name.
This can be done easily with the Koha::CsvProfiles->search method.
Test plan:
Export records using the misc/export_records.pl script and the
export tool.
If you are exporting using the MARC format, the profile filled in the pref
ExportWithCsvProfile will be used (or the one passed in parameter of
misc/export_records.pl).
If you are exporting using the CSV format, you can choose a profile in
the dropdown list.
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Exported using tool & cmd, marc & csv. Pref is used.
No errors
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This subroutine did the same job as GetCsvProfilesLoop, so this patch
applies the same changes as the previous patch.
Test plan:
1/ Claim some serials, sql profiles should be listed
2/ Export records using the export tool. MARC profiles should be listed.
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Listed sql & marc profiles
No errors
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Field callnumber should be itemcallnumber.
For completeness, prefixing the joined fields with items table prefix.
Note: You should expect Tools/Export to export only the items within a
given itemcallnumber range, but if the biblio has one item in that
range, the biblionumber is selected for export with ALL items. The script
is designed that way (first select biblio numbers based on criteria, and
run export with those biblio numbers).
Test plan:
[1] Select a biblio N with one item with say itemcallnumber X.
[2] Goto Tools/Export. Select biblio range N,N and no itemcallnumber
range. The biblio should be exported.
[3] Select biblio range N,N and itemcallnumber range A,B. No output.
[4] Select biblio range N,N and itemcallnumber range A,<empty>. Output.
[5] Select biblio range N,N and itemcallnumber range <empty>,X. Output.
Signed-off-by: Srdjan <srdjan@catalyst.net.nz>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This patch eliminates all columns retrieved in the biblioitems query
except for biblionumber.
To test:
1 - Go to tools->Export data
2 - Export using various filters and note you get expected records
3 - Apply patch
4 - Ensure exported results match results before patch
Signed-off-by: Srdjan <srdjan@catalyst.net.nz>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This patch replaces the occurrences of
my @foo = $cgi->param('foo');
with
my @foo = $cgi->multi_param('foo');
perl -p -i -e
's/^(\s*my\s*@\w+\s*=\s*)\$(cgi|input|query)\->param\(/$1\$$2\->multi_param\(/xms'
**/*.pl
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
When exporting records (tools/export.pl or misc/export_records.pl), a
file of ids (authid or biblionumber) can be passed to filter the
results.
Bug 14722 has broken this behavior.
Test plan:
Export records and specify a list of records to filter the results.
Prior to this patch, the record with the id 1 was exported.
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Brendan Gallagher brendan@bywatersolutions.com
These 2 subroutines are now not necessary anymore, we can use
Koha::Authority::Types to retrieve the authority types.
This patch won't be easy to test. Indeed, a lot of file are updated.
To test it I would suggest to focus on the merge authority
You should also confirm that the authtypecode and the authtypetext
values are correctly displayed when navigating in the authority module.
Signed-off-by: Frédéric Demians <f.demians@tamil.fr>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Brendan Gallagher brendan@bywatersolutions.com
The existing Koha::Authority does not use Koha::Objects and does not
exclusively deal with DB objects.
It makes much sense to move it to let the space free for a
Koha::Authority and Koha::Authorities modules based on Koha::Object.
Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Brendan Gallagher brendan@bywatersolutions.com
Why a refactoring was need for this script?
The export tool (tools/export.pl) can be called from the command line
and some parts of code were unnecessarity complicated (just look at the
code, you will understand).
Worse still, the script does not provide the same options for both
interface. For instance you cannot export records given a range of
biblionumbers, authids, callnumbers, etc. from the commandline.
What does this patch?
1/ Important: The script tools/export.pl does not work anymore if called from
the command line (should be in the release notes).
2/ The code used to generated a file (csv, iso2709 or xml) has been moved to a new
module (Koha::Exporter::Record) and tests have been provided.
3/ No change is done on the web interface
4/ Some new options have been added to the commandline script
(misc/export.pl):
- starting_authid
- ending_authid
- authtype
- starting_biblionumber
- ending_biblionumber
- itemtype
- starting_callnumber
- ending_callnumber
- start_accession
- end_accession
5/ There is a change in the behavior if an error occurs:
Can't call method "as_usmarc" on an undefined value at Koha/Exporter/Record.pm line 114.
record (number 5530) is invalid and therefore not exported because its reopening generates warnings above at Koha/Exporter/Record.pm line 117.
Before this patch, they were not displayed (using the command line).
What does not do this patch?
It does not provide the 'clean', 'timestamp' and 'deleted_barcodes' options to
the web interface (same as before).
What about the perfs?
With a DB with ~800 biblios (MARC21)
Before: perl tools/export.pl 14.79s user 0.83s system 71% cpu 21.905 total
After: perl misc/export.pl 17.19s user 0.84s system 75% cpu 24.018 total
With a DB with ~6400 biblios (UNIMARC)
Before: perl tools/export.pl 26.55s user 0.76s system 76% cpu 35.498 total
After: perl misc/export.pl 26.78s user 0.84s system 80% cpu 34.494 total
How to test this patch?
Test plan:
A. Web interface:
1/ On the current master, export some records, biblios and authorities (with
the 3 differents exports) playing with the different filters (item type,
libraries, callnumber, accession date, don't export items, remove
non-local items, don't export fields, etc.).
2/ Apply this patch, export again the same records, and compare the
generated files. They must be identical!
3/ Confirm that the export features on the checkout list
(circ/circulation.pl) works as before this patch.
B. The command line
1/ On the current master, export some records, biblios and authorities (with
the 2 differents exports) playing with the different options (date,
deleted_barcodes, clean).
2/ Apply this patch, export again the same records, and compare the
generated files. They must be identical!
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch removes C4::Dates from:
- tools/export.pl
- tools/import_borrowers.pl
Note: For testing, both need preparation without patch, see below.
To test export:
- Without patch applied, go to
Home > Tools > Export data > Export bibliographic records
- Define Start date / End date for Accession dates
- Export bibliographic records as 'without-patch.mrc'
- Do the same with patch, export as 'with-patch.mrc
- Compare the files, they should be the same
To test Import patrons:
- Without patch
- Go to Home > Tools > Import patrons
- Create a patron category like 'TEST' (useful for filtering...)
- Prepare a file with some patrons with category TEST to import.
Fill date of birth, enrolment date, expiry date with values
formatted in syspref format, in iso format and garbage
- Import
- Review the imported patrons (search for category TEST)
- With patch: Change cardnumber and names in import file
- Import
- Review again and compare with results from previous import.
Signed-off-by: Mirko Tietgen <mirko@abunchofthings.net>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
The script dies with
Can't use an undefined value as a HASH reference at tools/export.pl
line 149.
if it is called from the command line.
This is introduced by bug 13040.
The C4::Context->userenv is not defined in this case.
Test plan:
Execute the script using the command line.
With the patch applies you should not get the error.
Signed-off-by: Joonas Kylmälä <j.kylmala@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>
Tools > Export data produces a file badly encoded, when marc/xml file
format is chosen. It works with CSV.
To test:
1. Tools > Export data
2. Choose a limited interval of biblionumber
3. Export in marc/xml/csv.
4. Check files exported at 3: csv file is ok. marc/xml files are
corrupted, ie badly encoded.
5. Apply the patch
6. Repeat steps 3&4, and confirm that all file formats are valid.
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Courret <scourret@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
To reproduce:
1/ Go on tools/export.pl
2/ Export some records and specify you don't want the fields 245 (the
whole field, do not specify a subfield).
3/ Export, BOOM
Software error:
Arguments must be MARC::Field object at /home/koha/src/tools/export.pl line 400.
Test plan:
Apply the patch and confirm the fields 245 are not exported and the
export works as expected.
Signed-off-by: Robin Sheat <robin@catalyst.net.nz>
Export fails to asplode with this patch
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This allows the exporter (Tools -> Export) to have any combination of
branches selected, rather than it being all or only one.
Test Plan:
* Apply the patch
* Go to the exporter, see that instead of a dropdown you now have an
elegently laid out grid of branches you can select from
* Select some branches, run the export
* Note that only records with items in the selected branches are
returned.
* Repeat this with the item related options (as that code was refactored
slightly) and make sure everything is sane.
Sponsored-By: South Taranaki District Libraries
Signed-off-by: Thomas <tomsStudy@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
In the template of the export tools, the value 'marc' is set for
the 'format' variable.
It should be 'iso2709'.
There was a mess in the tools/export.pl script.
2 variables did the same thing: $format and $output_format
This patch fixes that replacing $format with $output_format
Test plan:
Try to export biblio in iso2709 and csv using the export tools and from
the checkout list.
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Before applying the patch, attemping an MARC export from tools/export.pl
resulted in an error. After the patch the export works correctly.
Exports from the checkouts page also work correctly.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes tests and QA script. Also tested authority export.
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Test:
1. Edit record, add 100.000 chars text to 500a
2. xml export produce the record,
3. mrc export do not produce the record, warning on log export.pl:
Record length of 111000 is larger than the MARC spec allows (99999
bytes). at /usr/share/perl5/MARC/File/USMARC.pm line 314. record
(number 139489) is invalid and therefore not exported because its
reopening generates warnings above at...
Signed-off-by: Frederic Demians <f.demians@tamil.fr>
I confirm that exporting biblio records larger than 10000 characters in
ISO2709 produces invalid files. After applying this patch, the culprit
record (too large, but also other inconsistencies preventing record
parsing with MARC::File::USMARC) is not exported anymore. A warning is
produced in Koha Apache log file. Warnings to the user on WUI would be
better, but it isn't the case yet, so it isn't a regression.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
I agree that a visible warning/result message in the staff interface
would be nice, but this works as described.
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Bug 11703 breaks the checkouts export feature.
To reproduce: Fill the ExportWithCsvProfile pref and go on the
circ/circulation.pl page. The export column appears, but not the export
button.
Test plan:
Go on the checkout list (circ/circulation.pl and members/moremember.pl)
and verify the export column and the export button appears.
If you click on the button, a file should be generated.
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Test plan
1/ run ./tools/export.pl
2/ notice an error like cannot open 0: No such file or directory at ./tools/export.pl line 209.
3/ apply patch
4/ run again
5/ no errors
Signed-off-by: Chris <chris@bigballofwax.co.nz>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as described, adding an additional check.
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Since we switched to Template Toolkit we don't need to stick with the
sufix we used for HTML::Template::Pro.
This patch changes the occurences of '.tmpl' in favour of '.tt'.
To test:
- Apply the patch
- Install koha, and verify that every page can be accesed
Regards
To+
P.S. a followup will remove the glue code.
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
The MARC export tool now includes the option to specify a file
containing record IDs to export. When run on the command line,
this is specified using the --id_list_file switch.
Note that the list of IDs acts as a filter on other criteria
for selecting records to import. For example, if you export
all bibs belong to a given library and also specify an ID file,
the bibs must both belong to the library and be in the ID file.
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Work as described. No errors
Tested writing txt file with biblionumbers, loading in tools and exporting
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Test:
1. Edit a bib record, add 100.000 chars text to 500a
2. Go to Tools -> Export data (or run tools/export.pl on
the command line)
3. Verify that exporting as XML includes the record
4. Verify that exporting as MARC does not include the record,
and that you get the following warning:
export.pl: record (number 2025) length 102668 is larger than the MARC spec allows (99999 bytes) at...
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Work as described. No errors
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
The method of checking the logged in user for superlibrarian privileges
is obtuse ( $userenv && $userenv->{flags} % 2 != 1 ) to say the least.
The codebase is littered with these lines, with no explanation given. It
would be much better if we had one subroutine that returned a boolean
value to tell us if the logged in user is a superlibrarian or not.
Test Plan:
1) Apply this patch
2) Verify superlibrarian behavior remains unchanged
Signed-off-by: Joel Sasse <jsasse@plumcreeklibrary.net>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Comments on second patch.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Regex assumed all numeric characters were part of the tag number
This is obviously false as $9 would be a likely candidate
to be removed on export.
The code can be any printing ascii character other than space
according to LoC's website.
Also changed regexp to no longer allow a zero length tag number
which is nonsensical.
The old regex would accept shorter than 3 digit tags but these were
not stripped so I've removed that option, considering it a bug
not a feature
NB assumption that the code makes that a tag is always numeric is
incorrect but works in practice. Handling non-numeric tags is
a 'To be done'
Made code dependent on the regex succeeding. Picking up results
from a previous regex on failure can lead to weird hard to
identify bugs
In the course of testing Paola Rossi <paola.rossi@cineca.it>
spotted that the delete operation was flawed, only removing
the first occurence of the specified tag/tag subfield. Reworked
the delete loop to operate on all occurences
Signed-off-by: Paola Rossi <paola.rossi@cineca.it>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
