If a cell contains an empty string it was not taken into account.
Test plan:
Same as CSV but with ODS
Signed-off-by: Ron Houk <rhouk@ottumwapubliclibrary.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
The CSV import of the biblio frameworks is broken.
A change in LibreOffice resulted in CSV not formatted how the
C4::ImportExportFramework code is expected.
This code is quite broken, it expects all the CSV cells to be quoted.
The "add tests" patch shows the different between what we expected
before this patch, and what is the new version generated by LibreOffice.
Test plan:
Export a biblio framework in CSV
Open it with LibreOffice and save it undef a different name
Create a new biblio framework and use the new file to construct it
Compare the 2 frameworks and make sure they are identical
Signed-off-by: Ron Houk <rhouk@ottumwapubliclibrary.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Ron Houk <rhouk@ottumwapubliclibrary.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Remove a bit of the confusion by removing unless code (not related to
mysql)
Signed-off-by: Ron Houk <rhouk@ottumwapubliclibrary.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Pierre-Marc Thibault <pierre-marc.thibault@inLibro.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
TEST PLAN
---------
1) Log in to staff client
2) Click Koha administration
3) Click MARC bibliographic framework
4) export default as xml
5) Read the error log
-- it's floody and export is default framework
6) export another framework as xml
7) Read the error log
-- it's floody and export is the correct framework
8) apply the patch
9) repeat steps 4-7
-- expected frameworks exported, and no flooding.
10) run koha qa test tools
Signed-off-by: Pierre-Marc Thibault <pierre-marc.thibault@inLibro.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Bug 9701 added 2 new columns to the table marc_tag_structure which broke
the import of biblio frameworks if the CSV file was generated with a
previous version.
Test plan:
- Export a biblio framework from 17.11
- Import it into 18.05+
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
https://bugs.koha-community.org/show_bug.cgi?id=21740
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Logs are full of:
Use of uninitialized value in substitution (s///) at
C4/ImportExportFramework.pm line 282.
Use of uninitialized value in concatenation (.) or string at
C4/ImportExportFramework.pm line 283.
Some field are NULL in DB, we need to init the value to an empty string
to avoid these warnings.
Note: This fix is not related to the original bug
Signed-off-by: Hector Castro <hector.hecaxmmx@gmail.com>
Warnings avoided
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Mainly a
perl -p -i -e 's/^.*3.07.00.049.*\n//' **/*.pm
Then some adjustements
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>
Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
perl -p -i -e 's/^(use vars .*)\$VERSION\s?(.*)/$1$2/' **/*.pm
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>
Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
http://bugs.koha-community.org/show_bug.cgi?id=9987
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
The SQL option for MARC framework imports was subject to a bug whereby
somebody could use it to gain access to arbitrary information in the
database by uploading an SQL file containing unexpected statements.
As it is difficult to securely sanitize SQL, this patch removes the
option to use SQL as an import or export format.
To test:
[1] Verify that SQL no longer appears as an import or export option
for the MARC frameworks.
[2] Verify that exports and imports in CSV, Excel XML, and ODS formats
still work.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Works as advertised. The UI doesn't offer exporting/importing in the SQL format.
Crafting the URL to export SQL fallbacks to a spreadsheet format (ODS).
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as described, passes all tests and QA script.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
RM note: Digest::MD5 is used in C4::ImportExportFramework as part
of an unnecessary reimplementation of functionality supplied by
File::Temp. See bug 10991 for a proposal to remove it.
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Bareword file handle opened at line 558, column 17. See pages 202,204 of PBP. (Severity: 5)
Two-argument "open" used at line 558, column 17. See page 207 of PBP. (Severity: 5)
http://bugs.koha-community.org/show_bug.cgi?id=6679
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Aleksa Vujicic <aleksa@catalyst.net.nz>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Amended to replace some copy-and-paste comments only with consent of MJR.
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Step to reproduce :
- export default framework as SQL
- create new framework
- Import the exported SQL file into this new framework.
Go to default framework :
It disappeared.
If some libraries want to share default frameworks, then it is a major bug.
This patch fixes the test in order to cope with 0 character framework name.
Signed-off-by: François Charbonnier <francois.charbonnier@biblibre.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Function _export_table_csv modified to remove CR/LF in the data
from the database to built a correct csv file.
Function _import_table_csv modified to deal with CR/LF in the data
field. When a line is found with an unproper end (a data field has not
end quote), it's concatenated to a temporary buffer until the next
line ends ok and then it's proccessed.
Signed-off-by: Nicole C. Engard <nengard@bywatersolutions.com>
Signed-off-by: Ian Walls <ian.walls@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Module to Import/Export a Framework structure to CSV/Excel-xml/ODS/SQL in Intranet Administration - MARC Frameworks section.
There are two new links: "Export" to export to a format; and "Import" to import from a file.
The data exported/imported is the one stored in the MySQL tables marc_tag_structure, marc_subfield_structure.
Exported works as follows:
1) CSV: As this format only allows one worksheet, the data from the tables is splitted with a row with #-# cells or with the
names of the fields of the next MySQL table. Each row has as much cells as fields has the MySQL table. The first row contains the
field names, the remaining holds the data.
2) Excel: Excel xml 2003 format. Each MySQL table has its own worksheet in the spreadsheet. Rows and cells data as CSV.
3) ODS: OpenDocument Spreadsheet compressed format, creates a temporary directory to generate the files needed to create the zip file.
Each MySQL table has its own worksheet in the spreadsheet. Rows and cells data as CSV.
4) SQL: Text file, the first row for each table is a delete and the remaining are inserts.
Importing reads the rows from the spreadsheet/text-file as follows:
1) CSV: Each row inserts or updates the associated MySQL table for this framework. At the end of the importing for a MySQL table, deletes the rows in the database that don't possess a correspondence with the spreadsheet.
2) Excel: Imports each worksheet to the associated MySQL table. Works as the CSV for each worksheet.
3) ODS: Creates a temporary directory to decompress and read the content.xml. This file has the data needed to import.
Works as the CSV for each worksheet.
4) Executes the SQL file.
If the file imported has a different frameworkcode that the framework importing, the framecode is changed along the process.
The Csv format will be the default.
It uses perl module Archive::Zip or zip/unzip system command to process ODS files.
To parse the sql files when importing it uses SQL::Statement or homemade parsing.
Signed-off-by: Nicole C. Engard <nengard@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
