Commit graph

11 commits

Author SHA1 Message Date
a80366439e Bug 18632: Remove 'CGI::param called in list context' warnings
Once again, after bug 16154 and bug 16259 we need to remove more
occurrence of CGi->param called in list context.
Refer to bug 15809 for more information.

Test plan:
Make sure you do not see the error on the modified scripts.

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-05-28 22:25:22 -04:00
Mark Tompsett
1ce61ac6a3 Bug 18111: Swap framework_name and frameworkcode
With the frameworkcode changed to an empty string, which really
is closer to the physical reality, some javascript somewhere
breaks. This means the spinner does not appear when you confirm
that you wish to import it.

TEST PLAN
----------
1) apply first patch
2) export default framework, doesn't matter format
3) import default framework using that file.
4) answer yes or whatever it is.
   -- spinner does not appear.
5) apply this patch
6) import default framework use the same file.
7) answer yes or whatever it is.
   -- spinner does appear.
8) decide if you wish to find and fix the javascript conditions.
   if yes -- do so, and then obsolete this once you have.
   if not -- sign this off after running koha qa test tools

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-02-21 19:53:18 +00:00
c12aa9664c Bug 18111: Fix import of default framework
Caused by bug 16035.

Recreate the issue:
Go to Home › Administration › MARC frameworks
For the default framework: Actions > import
It will not work

Test plan:
Confirm test plans from bug 16035 and bug 17389 still pass
Confirm that the import for the default framework now works

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-02-21 19:53:18 +00:00
a1eeffd838 Bug 17389: (bug 16035 follow-up) Fix export non-default frameworks
Trivial patch, no need to explain anything :)

Test plan:
Export default framework
Export other frameworks
=> The frameworkcode column should contain the framework code

Signed-off-by: Hector Castro <hector.hecaxmmx@gmail.com>
Works as advertised

Signed-off-by: Katrin Fischer  <katrin.fischer@bsz-bw.de>

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
2016-10-11 16:18:09 +00:00
Aleisha Amohia
705097f938 Bug 16035: MARC default bibliographic framework Export fix
This patch makes sure that, if there is no framework code,
     $frameworkcode is set to 'default'. This fixes the format issue.
     Then, if $frameworkcode = default, an empty string (rather than
     null) is passed to the ExportFramework method, as the framework
     code in the DB for default codes is "".

To test:
1) Go to Admin -> MARC Bibliographic framework
2) Try to export default framework in all formats, notice the file is
only exported as .ods and is an empty file. Notice warns in intranet
error log.
3) Apply patch and refresh page
4) Confirm exporting default framework works in all formats and
spreadsheet has data
5) Notice warns in error log are gone
6) Confirm export still works for other frameworks

Sponsored-by: Catalyst IT

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-09-08 12:07:55 +00:00
Jonathan Druart
a6c9bd0eb5 Bug 9978: Replace license header with the correct license (GPLv3+)
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>

http://bugs.koha-community.org/show_bug.cgi?id=9987

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2015-04-20 09:59:38 -03:00
Jonathan Druart
e20270fec4 Bug 11944: use CGI( -utf8 ) everywhere
Signed-off-by: Paola Rossi <paola.rossi@cineca.it>
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Dobrica Pavlinusic <dpavlin@rot13.org>

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2015-01-13 13:07:21 -03:00
40d3800cc4 Bug 9114: Make frameworks import/export routines correctly use UTF-8
Currently both the import_export_framework.pl script outputs data with
Perl's default encoding, ISO-8859. This patch properly sets
the binmode to UTF-8 when exporting SQL and CSV files using the PerlIO
layer (":encoding(UTF-8)") for STDOUT.

To test:

Export step test
- Use some ASCII character(s) with DIACRITICS in some field description
  in a chosen framework.
- Export the framework at Administration > MARC frameworks
- Run this to check the file is ISO-8859 encoded:
 $ file export_XXX.csv
 export_XXX.csv: ISO-8859 text, with very long lines
 (Note: try SQL and other output formats too. But not ODS)
- Apply the patch
- Export the framework again (change the name), and test encoding:
 $ file export_XXX_2.csv
 export_XXX_2.csv: UTF-8 Unicode text

Import step test
I assume you have two files, export_XXX.csv (ISO-8859 encoded) and
export_XXX_2.csv (XXX will depend on your framework's code)
- Reset your testing branch to master
- Import export_XXX.csv
- The string with non-ASCII chars is truncated at the first non-ASCII
  char's position (Note: this is the current behaviour).
- Import export_XXX_2.csv
- The non-ASCII chars are broken, the logs show errors on non-UNICODE
  chars.  (Note: even thou UTF-8 is the expected encoding it is
  treated as ISO-8859).
- Apply the patch
- Import the good (UTF-8 as expected) file and check everything worked
  as expected.

No double encoding should occur with either combination of formats.

Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: Magnus Enger <digitalutvikling@gmail.com>
I put some Norwegian and accented letters in a fremawork to test.
Before the patch, the exported CSV came out as ISO-8859, after the
patch it came out as UTF-8. ODS and XML (viewed in LibreOffice)
both looked good, before and after the patch.

Importing the ISO-8859 CSV cut off the strings at the first non-ASCII
char. Importing the UTF-8 CSV worked as epected.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as expected, passes tests and QA script.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-03-14 14:37:31 +00:00
Galen Charlton
94e349ff6c Bug 11666: remove SQL as an option for MARC framework exports and imports
The SQL option for MARC framework imports was subject to a bug whereby
somebody could use it to gain access to arbitrary information in the
database by uploading an SQL file containing unexpected statements.

As it is difficult to securely sanitize SQL, this patch removes the
option to use SQL as an import or export format.

To test:

[1] Verify that SQL no longer appears as an import or export option
    for the MARC frameworks.
[2] Verify that exports and imports in CSV, Excel XML, and ODS formats
    still work.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Works as advertised. The UI doesn't offer exporting/importing in the SQL format.
Crafting the URL to export SQL fallbacks to a spreadsheet format (ODS).

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as described, passes all tests and QA script.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-02-05 19:48:27 +00:00
Galen Charlton
03d4ed2468 Bug 11666: add permission check for MARC framework import/export
This patch makes the MARC framework import/export script require
that the staff user be logged in with appropriate permissions for
managing the MARC frameworks.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>

I can confirm the bug and the solution. After applying the patch
downloading the file without logging in first is no longer possible.
Also passes tests and QA script.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-02-05 19:47:53 +00:00
Salvador Zaragoza Rubio
689bc19154 Bug 5478 - Automate MARC framework import/export
Module to Import/Export a Framework structure to CSV/Excel-xml/ODS/SQL in Intranet Administration - MARC Frameworks section.
    There are two new links: "Export" to export to a format; and "Import" to import from a file.
    The data exported/imported is the one stored in the MySQL tables marc_tag_structure, marc_subfield_structure.

    Exported works as follows:
    1) CSV: As this format only allows one worksheet, the data from the tables is splitted with a row with #-# cells or with the
    names of the fields of the next MySQL table. Each row has as much cells as fields has the MySQL table. The first row contains the
    field names, the remaining holds the data.
    2) Excel: Excel xml 2003 format. Each MySQL table has its own worksheet in the spreadsheet. Rows and cells data as CSV.
    3) ODS: OpenDocument Spreadsheet compressed format, creates a temporary directory to generate the files needed to create the zip file.
    Each MySQL table has its own worksheet in the spreadsheet. Rows and cells data as CSV.
    4) SQL: Text file, the first row for each table is a delete and the remaining are inserts.

    Importing reads the rows from the spreadsheet/text-file as follows:
    1) CSV: Each row inserts or updates the associated MySQL table for this framework. At the end of the importing for a MySQL table, deletes the rows in the database that don't possess a correspondence with the spreadsheet.
    2) Excel: Imports each worksheet to the associated MySQL table. Works as the CSV for each worksheet.
    3) ODS: Creates a temporary directory to decompress and read the content.xml. This file has the data needed to import.
    Works as the CSV for each worksheet.
    4) Executes the SQL file.
    If the file imported has a different frameworkcode that the framework importing, the framecode is changed along the process.

    The Csv format will be the default.
    It uses perl module Archive::Zip or zip/unzip system command to process ODS files.
    To parse the sql files when importing it uses SQL::Statement or homemade parsing.

Signed-off-by: Nicole C. Engard <nengard@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
2011-04-01 21:17:33 +13:00