Here we go, next step then.
As we did not fix the performance issue when autofiltering
the variables (see bug 20975), the only solution we have is to add the
filters explicitely.
This patch has been autogenerated (using add_html_filters.pl, see next
pathces) and add the html filter to all the variables displayed in the
template.
Exceptions are made (using the new 'raw' TT filter) to the variable we
already listed in the previous versions of this patch.
To test:
- Use t/db_dependent/Koha/Patrons.t to populate your DB with autogenerated
data which contain <script> tags
- Remove them from borrower_debarments.comments (there are allowed here)
update borrower_debarments set comment="html tags possible here";
- From the interface hit page and try to catch alert box.
If you find one it means you find a possible XSS.
To know where it comes from:
* note the exact URL where you found it
* note the alert box content
* Dump your DB and search for the string in the dump to identify its
location (for instance table.field)
Next:
* Ideally we would like to use the raw filter when it is not necessary
to HTML escape the variables (in big loop for instance)
* Provide a QA script to catch missing filters (we want html, uri, url
or raw, certainly others that I am forgetting now)
* Replace the html filters with uri when needed (!)
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Bonus fixes:
- Basketgroup: there was an unnecessary space.
" RRP tax excluded" replaced by "RRP tax excluded"
- Late orders:
- leading spaces before every line
- surrounding spaces for [% orders.size %]
Other changes:
- quoted "Total orders in late" to no worry about additional spaces
- Late orders: additional spaces which shouldn't be a problem because
all the fields are enclosed by quotes.
Test plan:
1. Set syspref "delimiter" to "#'s"
2. Change the language e.g. FR-fr
3. Create a vendor with minimal info
4. Create a basket with two items
5. Go to the basket. URL should be
/cgi-bin/koha/acqui/basket.pl?basketno=XXXXX
6. Close this basket
7. "Export as CSV"
8. You should see that the file contains commas "," This is the bug.
Leave the file open for future comparison to ensure that there are
no regressions.
9. Create a basket group with two baskets
(tick the close this basket group check box)
10. Go to the "Closed" tab to see your basket group
11. "Export as CSV"
12. You should see that the file contains commas "," This is the bug.
Leave the file open for future comparison to ensure that there are
no regressions.
13. Go to /cgi-bin/koha/acqui/lateorders.pl
14. Tick two orders. Of the same vendor, otherwise Koha won't let you tick
orders of different vendors.
15. "Export as CSV"
16. You should see that the file contains commas "," This is the bug.
Leave the file open for future comparison to ensure that there are
no regressions.
17. Apply this patch
18. Translation stuff
1. cd misc/translator
2. perl translate update YOUR_LANG
3. manually edit the po file, remove the fuzzy tags.
And translate the string. You can just copy the original and
replace some parts by gibberish.
4. perl translate install YOUR_LANG
19. Re-export the same CSVs and compare with the original version to check that
- delimiter syspref is honored
- the headers are translatable
- there is no whitespace regression (additional newlines or spaces)
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
I am going to try and explain the problem here in order to make it a
have-to-read for next changes.
There are several things to test when and most of the time we break
something when we try to fix something else.
You have to test different CSV export, not only one. There are not all
processed the same way. For instance acqui/csv/basket.tt does not
contain any strings to translate and the translate script will copy the
original file as it.
But reports/csv/orders_by_budget.tt will not! Indeed it contains "TOTAL"
which will be translated. The generated tt file for the translated
language will remove all the carriage returns! That's why we cannot rely
on TT for newlines (contrary to bug 16914 assumption).
There are two possible methods:
1/ Fix the translate script
=> Hum... nope
2/ Remove all carriage returns and make them explicit by including an
file that only contains 1 carriage return
This second method is implemented in this patch.
How it works: Use the PRE_CHOMP "[%-" and POST_CHOMP "-%]"
(http://www.template-toolkit.org/docs/manual/Config.html#section_PRE_CHOMP_POST_CHOMP)
to remove all the newlines that could be added by TT in the original
(en) files. Then include the new_line.inc to add a new line.
That way original and translated files will behave the same way.
The BLOCK in the csv_headers avoid to have the newlines added, in any
cases. For instance: by default we will have an empty line at the end of
the headers, but the translated headers will not have it.
Test plan:
At least 2 signoffs will be needed, please test carefully!
You will need to generate CSV with at least 2 entries!
Test with the non-translated interface (en) and with the language you
want. To test with a translated language you will *have to*:
- cd misc/translator
- perl translate update LANG
- manually edit the po file, remove the fuzzy tags, correctly translate
the string (do not forget the additional %s in the headers)
- perl translate install LANG
1/ Export basket and basketgroup as CSV
2/ Export late orders as CSV
3/ Export items (from the item search) as CSV
4/ Export the 2 reports "Orders by fund" and "Cash register" as CSV
Signed-off-by: Lee Jamison <ldjamison@marywood.edu>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
The translation scripts have an historical tendency to chomp newlines
and we introduced an empty_line.inc file to force newlines when building
CSV output out of our templates (in item search and late orders).
This patch makes the mentioned templates use TT ability to force newlines
plus some misuses of the 'minus' sign.
Test plan:
- Apply the patch
- Do an item search that returns more than one result
- Export as CSV
=> SUCCESS: The CSV file is correctly formed.
- Install any translation:
$ sudo koha-shell kohadev
k$ cd kohaclone/misc/translator
k$ perl translate install <chosen language>
- Enable <chosen language> (e.g. es-ES)
- Repeat the item search
- Export as CSV
=> SUCCESS: The CSV file is correctly formed in your chosen language.
- Have more than one late orders (bummer)
- Go to late orders
- Choose them
- Export as CSV (in english)
=> SUCCESS: The CSV file is correctly formed.
- Switch language
- Go to late orders
- Choose them
- Export as CSV (in english)
=> SUCCESS: The CSV file is correctly formed in your chosen language.
- Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
Bug 7298 tried to fix this issue, but it was not correct.
We have 3 files in acqui/csv:
basketgroup.tt, basket.tt and lateorders.tt
The first 2 don't contain translatable string, and are not modified on
translating the templates (`translate install`)
On the contrary, lateorders.tt has some strings to translate ('Author:',
'Published by:', etc.). After being translated, all carriage returns
between TT tags are removed.
Test plan:
1/ choose a language and update + translate the templates
for instance:
cd misc/translate;
./translate update es-ES; ./translate install es-ES
2/ Go to acqui/lateorders.pl using this language
3/ Generate a csv for 1+ late orders and confirm the first line only
contains the headers.
Signed-off-by: Laurence Lefaucheur <laurence.lefaucheur@biblibre.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Following the same way as bug 10935, the headers are in an include file.
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Translatability tested successfully.
Passes all tests.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Using a template file, the CSV headers become translatable.
Signed-off-by: sonia <koha@univ-lyon3.fr>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
