This patch implements the replacement of TTvariables
'[%...%]' with normal placeholders '%s' for text
inside HTML tags (eg. img, meta, input)
The replacement is done in two files:
* xgettext.pl, used on create/update, so TTvars are not
found on the text to translate (msgid)
* tmpl_proccess3.pl, used on install time, so the correct
translation is found (msgstr) and the right TTvar is put on
the translated text.
To test:
Before applying the patch
1) Update your favorite language
cd misc/traslator
perl translate update xx-YY
2) Build a list (ini.txt) of 'msgid' to do a comparison
msgcat --no-wrap --use-first po/xx-YY-*po -o - | egrep "^msgid" | sort > ini.txt
3) Apply the patch
4) Update your favorite language, again
cd misc/traslator
perl translate update xx-YY
5) Build a final list (end.txt) of 'msgid' to do a comparison
msgcat --no-wrap --use-first po/xx-YY-*po -o - | egrep "^msgid" | sort > end.txt
6) Compare both files, you must find (most) TTvars replaced by '%s'
diff ini.txt end.txt
for example:
< msgid "Translate item type [% itemtype.itemtype | html %]"
> msgid "Translate item type %s"
< msgid "Holds on this item: [% item_loo.item_holds | html %] / Total holds on this record: [% item_loo.holds | html -%]"
> msgid "Holds on this item: %s / Total holds on this record: %s"
< msgid "tag_anchor_[% innerloo.tag | html %]_[% innerloo.index | html %][% innerloo.random | html %]"
> msgid "tag_anchor_%s_%s%s"
< msgid "[% innerloo.tag_lib | html %] - Click to Expand this Tag"
> msgid "%s - Click to Expand this Tag"
7) Translate one of the examples, with more than one variable if
possible, for example this text in xx-YY-staff-prog.po
> msgid "Holds on this item: %s / Total holds on this record: %s"
Check it's not marked as 'fuzzy'
8) Apply the translation
perl translate install xx-YY
9) Check the translated string has all variables in the right order
edit koha-tmpl/intranet-tmpl/prog/es-ES/modules/tools/batchMod-edit.tt
around line '187', first '[% item_loo.item_holds | html %]', then
'[% item_loo.holds | html -%]' on this case,
or use another example
If approved this patch can be backported to current stable
versions.
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Conflicts:
misc/translator/text-extract.pl
misc/translator/text-extract2.pl
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Perl 5.26 (or earlier) introduced a security feature, where implicitly
including the program directory as a Perl library directory no longer
happens (perl -I. ).
This causes translate to fail because it cannot find the *.pm -files in
it's own directory.
This patch adds the familiar mantra
use lib $FindBin::Bin;
to the relevant scripts.
To test:
1. Install Ubuntu18.04 or something else with Perl 5.26
2. Install Koha (we use the dev-install)
3. cd $KOHA_PATH/misc/translator/
4. perl translate create fi-FI
5. Observe problems with missing modules.
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Cannot recreate the issue right now but the changes make sense.
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
This patch gives a partial solution for this problem.
It ignores strings like "[% something %]", but not
"[% IF ( value ) %][% value %][% END %]"
We get 100+ strings less to translate
To test:
1) Update translation files for your preferred language
2) Apply the patch
3) Update again
4) Compare, you must find 100+ strings removed from
translation files
eg.
-msgid "[% SEARCH_RESULT.biblionumber |url %]"
-msgid "[% accepteddate | $KohaDates %]"
-msgid "[% amountoutstanding | format('%.2f') %]"
-msgid "[% authtypetext |html %]"
-msgid "[% barcode_llx |html %]"
-msgid "[% barcode_lly |html %]"
-msgid "[% biblio.quantity.length ? biblio.quantity : 1 %]"
-msgid "[% billingdate | $KohaDates %]"
-msgid "[% borname |html %]"
...
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Verified all strings removed from the po files were
pure TT.
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Since we switched to Template Toolkit we don't need to stick with the
sufix we used for HTML::Template::Pro.
This patch changes the occurences of '.tmpl' in favour of '.tt'.
To test:
- Apply the patch
- Install koha, and verify that every page can be accesed
Regards
To+
P.S. a followup will remove the glue code.
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Using perl v5.18, the following is warned on updating/installing
templates:
Smartmatch is experimental at ./translate line 54.
Moreover, the following should exist with previous perl versions:
perlcritic xgettext
Subroutine prototypes used at line 29, column 1. See page 194 of PBP.
(Severity: 5)
Subroutine prototypes used at line 44, column 1. See page 194 of PBP.
(Severity: 5)
Subroutine prototypes used at line 60, column 1. See page 194 of PBP.
(Severity: 5)
Subroutine prototypes used at line 72, column 1. See page 194 of PBP.
(Severity: 5)
Subroutine prototypes used at line 89, column 1. See page 194 of PBP.
(Severity: 5)
Variable declared in conditional statement at line 105, column 13.
Declare variables outside of the condition. (Severity: 5)
Subroutine prototypes used at line 127, column 1. See page 194 of PBP.
(Severity: 5)
Subroutine prototypes used at line 136, column 1. See page 194 of PBP.
(Severity: 5)
Variable declared in conditional statement at line 190, column 7.
Declare variables outside of the condition. (Severity: 5)
Subroutine prototypes used at line 237, column 1. See page 194 of PBP.
(Severity: 5)
Bareword file handle opened at line 238, column 5. See pages 202,204 of
PBP. (Severity: 5)
Two-argument "open" used at line 238, column 5. See page 207 of PBP.
(Severity: 5)
Subroutine prototypes used at line 277, column 1. See page 194 of PBP.
(Severity: 5)
Subroutine prototypes used at line 311, column 1. See page 194 of PBP.
(Severity: 5)
Bareword file handle opened at line 350, column 9. See pages 202,204 of
PBP. (Severity: 5)
Two-argument "open" used at line 350, column 9. See page 207 of PBP.
(Severity: 5)
Bareword file handle opened at line 353, column 5. See pages 202,204 of
PBP. (Severity: 5)
Bareword file handle opened at line 359, column 5. See pages 202,204 of
PBP. (Severity: 5)
Two-argument "open" used at line 359, column 5. See page 207 of PBP.
(Severity: 5)
perl -wc xgettext.pl
\1 better written as $1 at TmplTokenizer.pm line 474.
main::token_negligible_p() called too early to check prototype at
xgettext.pl line 52.
xgettext.pl syntax OK
Test plan:
Apply this patch and verify the warnings/errors does not appear anymore
and no regression is found on creating/updating/installing templates.
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Work as described, no koha-qa errros.
Tested with perl v5.18.2
No warnings, no problem detected on create/update/install i18n files
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This patch removes several types of strings from the
PO files that cannot be usefully translated, including
ones that consist entirely of punctuation and/or HTML entities.
Test:
1) Update PO files of some lang, xx-YY-*po
cd misc/translator
perl translate update xx-YY
2) Do it again, just in case
3) rm po/xx-YY*po~
4) Extract all msgid's, sorted
cat po/xx-YY*po | egrep "^msgid" | sort | uniq > xx-YY-pre
5) Apply the patch
6) Repeat 1-3
7) Repeat 4 again, other file
cat po/xx-YY*po | egrep "^msgid" | sort | uniq > xx-YY-post
8) Do a diff, inspect results, only strings with %s and \s
diff xx-YY-pre xx-YY-post | less
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Works as described, 380 strings less to 'translate'
No koha-qa errors.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Tested according to test plan, works as described.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
To test:
git checkout 3.6.x
cd misc/translator
./tmpl_process3.pl update -i ../../koha-tmpl/opac-tmpl/prog/en/ \
-s ./po/fr-FR-i-opac-t-prog-v-3006000.po -r
po/fr-FR-i-opac-t-prog-v-3006000.po contains broken diacritics for NORMARC
strings for example
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
fixes regression caused by bug 6752
use encoding(UTF-8) rather than utf-8 for stricter
encoding
Marking output as ':utf8' only flags the data as utf8
using :encoding(UTF-8) also checks it as valid utf-8
see binmode in perlfunc for more details
In accordance with the robustness principle input
filehandles have not been changed as code may make
the undocumented assumption that invalid utf-8 is present
in the imput
Fixes errors reported by t/00-testcritic.t
Where feasable some filehandles have been made lexical rather than
reusing global filehandle vars
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
This is a fairly hacky solution, a counter patch would be more than
welcome
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Updated, translated and installed German po files after applying this patch.
No problems found.
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
This test validate Template Toolkit (TT) Koha files.
For the time being an unique validation is done: Test if TT files
contain TT directive within HTML tag. For example:
<li[% IF
This kind of constuction MUST be avoided because it break Koha
translation process.
This patch transform also translation specific modules into C4 modules
in order to be able to use them in test case:
C4::TTPaser
C4::TmplToken
C4::TmplTokenType
This patch is a Perl adaptation of a Haskell script from Frère Sébastien
Marie.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Notes on testing:
- translate install de-DE - worked ok
- translate update de-DE > translate install de-DE - worked ok
- running the test xt/tt_valid.t - worked ok and pointed out lots of problems.
Found no problems.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
In opac/staff templates .po files, we have comment lines contextualizing
extracted text in templates. Path to template are absolute. For example, we can
have:
#: /home/katrin/kohaclone/koha-tmpl/intranet-tmpl/prog/en/...
modules/cataloguing/addbiblio.tmpl:585
The first part of the pathname is useless. With this patch, we just keep
relative path to tempalte from Koha template main directory. The above example
becomes:
#: intranet-tmpl/prog/en/modules/cataloguing/addbiblio.tmpl:585
To be applied on [3.2]
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
xsl strings are now taken into account.
This is a first draft.
Some parameters of the functions will be taken into account
and <xsl> tags are not translated
Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
- If a string is enclosed by a tag, remove that tag from the extracted string
- Generate automatic comments to provide more information for the translator
- A couple bug fixes
(Actually, xgettext.pl did not know how to handle them in the files listed
in the list of files.)
If the po file is empty (corrupted), $href->{'""'} will be undefined.
We just blindly dereferenced this null value without checking.
when minor whitespace changes occur in the original templates; it also
makes the strings much easier to read (e.g., instead of "foo\n\n\t\t bar",
xgettext.pl will now always generate "foo bar" and tmpl_process3.pl will
understand it to be the same as the original string).
Early termination of analysis if we encounter some strings, such as </h1>
or | or ||, in order to avoid extracting strings that are unnecessarily
long and which doesn't add any meaningful context.
either iso8859-1 or utf8, msgmerge(1) won't crap out. The code is ugly;
the conversion table is hard-coded, and in some place not very appropriate.
However, this does fix the case where a few strings containing French
characters can't be translated. As a side effect, tmpl_process3 can now
also be used for French or other languages using iso8859-1.
word order is too different than the word order of the target language to
yield meaningful translations.
The new scripts use a different translation file format (namely standard
gettext-style PO files).
This seems to reasonably work (e.g., producing an empty en_GB translation
then installing seems to not corrupt the "translated" files), but it likely
will still contain some bugs. There is also little documentation, but try
to run perldoc on the .p[lm] files to see what's there. There are also some
spurious warnings (both from bugs in the new scripts and from buggy third-
party Locale::PO module).
