Bug 24330: When importing patrons from CSV, automatically strip BOM from file if...
authorKyle M Hall <kyle@bywatersolutions.com>
Thu, 2 Jan 2020 15:30:54 +0000 (10:30 -0500)
committerJoy Nelson <joy@bywatersolutions.com>
Mon, 10 Feb 2020 23:28:31 +0000 (23:28 +0000)
commit91980f27a30b2f57c59ea0da955b697ed6a98528
tree27c688c4f10ccc9d9438584db59613924f14153c
parent265f0c4041de9a109b0535d7fab3d95fdfcb34a8
Bug 24330: When importing patrons from CSV, automatically strip BOM from file if it exists

We have a partner that exports UTF-8 CSV files, and is experiencing the same thing as the author of this article: https://www.freecodecamp.org/news/a-quick-tale-about-feff-the-invisible-character-cd25cd4630e7/

In short, Excel is inserting an invisible UTF-8 character at the start of the file, so that the column name "cardnumber" is actually named "\x{feff}cardnumber", causing "cardnumber" to be blank.

A simple solution is provided here: https://stackoverflow.com/questions/24390034/remove-bom-from-string-with-perl

Test Plan:
1) Download the example.csv file
2) Attempt to import it using the patron import tool
3) Note the invalid column name error
4) Apply this patch, restart all the things!
5) Attempt the import again
6) Assuming you have a branchcode MPL and a cataegory code S, the patron should import!

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Maggie Wong <maggie.wong@yccece.edu.hk>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Joy Nelson <joy@bywatersolutions.com>
Koha/Patrons/Import.pm