Main Koha release repository
https://koha-community.org
Tomas Cohen Arazi
40d3800cc4
Currently both the import_export_framework.pl script outputs data with
Perl's default encoding, ISO-8859. This patch properly sets
the binmode to UTF-8 when exporting SQL and CSV files using the PerlIO
layer (":encoding(UTF-8)") for STDOUT.
To test:
Export step test
- Use some ASCII character(s) with DIACRITICS in some field description
in a chosen framework.
- Export the framework at Administration > MARC frameworks
- Run this to check the file is ISO-8859 encoded:
$ file export_XXX.csv
export_XXX.csv: ISO-8859 text, with very long lines
(Note: try SQL and other output formats too. But not ODS)
- Apply the patch
- Export the framework again (change the name), and test encoding:
$ file export_XXX_2.csv
export_XXX_2.csv: UTF-8 Unicode text
Import step test
I assume you have two files, export_XXX.csv (ISO-8859 encoded) and
export_XXX_2.csv (XXX will depend on your framework's code)
- Reset your testing branch to master
- Import export_XXX.csv
- The string with non-ASCII chars is truncated at the first non-ASCII
char's position (Note: this is the current behaviour).
- Import export_XXX_2.csv
- The non-ASCII chars are broken, the logs show errors on non-UNICODE
chars. (Note: even thou UTF-8 is the expected encoding it is
treated as ISO-8859).
- Apply the patch
- Import the good (UTF-8 as expected) file and check everything worked
as expected.
No double encoding should occur with either combination of formats.
Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: Magnus Enger <digitalutvikling@gmail.com>
I put some Norwegian and accented letters in a fremawork to test.
Before the patch, the exported CSV came out as ISO-8859, after the
patch it came out as UTF-8. ODS and XML (viewed in LibreOffice)
both looked good, before and after the patch.
Importing the ISO-8859 CSV cut off the strings at the first non-ASCII
char. Importing the UTF-8 CSV worked as epected.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as expected, passes tests and QA script.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
|
||
|---|---|---|
| acqui | ||
| admin | ||
| authorities | ||
| basket | ||
| C4 | ||
| catalogue | ||
| cataloguing | ||
| circ | ||
| course_reserves | ||
| debian | ||
| docs | ||
| errors | ||
| etc | ||
| install_misc | ||
| installer | ||
| Koha | ||
| koha-tmpl | ||
| labels | ||
| members | ||
| misc | ||
| offline_circ | ||
| opac | ||
| OpenILS | ||
| patron_lists | ||
| patroncards | ||
| plugins | ||
| reports | ||
| reserve | ||
| reviews | ||
| rotating_collections | ||
| selenium | ||
| serials | ||
| services | ||
| skel | ||
| sms | ||
| suggestion | ||
| svc | ||
| t | ||
| tags | ||
| test | ||
| tmp/modified_authorities | ||
| tools | ||
| virtualshelves | ||
| xt | ||
| .htaccess | ||
| .mailmap | ||
| about.pl | ||
| changelanguage.pl | ||
| edithelp.pl | ||
| fix-perl-path.PL | ||
| help.pl | ||
| INSTALL | ||
| install-CPAN.pl | ||
| INSTALL.debian | ||
| INSTALL.fedora7 | ||
| INSTALL.opensuse | ||
| INSTALL.ubuntu | ||
| koha_perl_deps.pl | ||
| kohaversion.pl | ||
| LICENSE | ||
| mainpage.pl | ||
| Makefile.PL | ||
| MANIFEST.SKIP | ||
| README | ||
| README.robots | ||
| rewrite-config.PL | ||
Koha is a free software integrated library system. Koha is distributed under the GNU GPL version 3 or later. Please read the file LICENSE for more details. To install or upgrade Koha, please see the INSTALL file appropriate to your platform. Report bugs at http://bugs.koha-community.org/ Visit the Koha Project website at http://www.koha-community.org/