Bug 24330: When importing patrons from CSV, automatically strip BOM from file if...
authorKyle M Hall <kyle@bywatersolutions.com>
Thu, 2 Jan 2020 15:30:54 +0000 (10:30 -0500)
committerMartin Renvoize <martin.renvoize@ptfs-europe.com>
Thu, 30 Jan 2020 15:52:20 +0000 (15:52 +0000)
commit6d456df81a03096342dd72df99284e23603a7bbc
tree7d81857ab3dc14665f939187f686aae5e60c5414
parentcca2a0827043770877e6db3c7b13a2d6f263d4e6
Bug 24330: When importing patrons from CSV, automatically strip BOM from file if it exists

We have a partner that exports UTF-8 CSV files, and is experiencing the same thing as the author of this article: https://www.freecodecamp.org/news/a-quick-tale-about-feff-the-invisible-character-cd25cd4630e7/

In short, Excel is inserting an invisible UTF-8 character at the start of the file, so that the column name "cardnumber" is actually named "\x{feff}cardnumber", causing "cardnumber" to be blank.

A simple solution is provided here: https://stackoverflow.com/questions/24390034/remove-bom-from-string-with-perl

Test Plan:
1) Download the example.csv file
2) Attempt to import it using the patron import tool
3) Note the invalid column name error
4) Apply this patch, restart all the things!
5) Attempt the import again
6) Assuming you have a branchcode MPL and a cataegory code S, the patron should import!

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Maggie Wong <maggie.wong@yccece.edu.hk>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Koha/Patrons/Import.pm